If worries about robots taking over your job are keeping you up at night, there's good news: people are better than robots at many things, including evaluating how well your content and search engine results are contributing to site search eCommerce.That may sound strange, especially if you’ve been relying on statistical approaches for improving your search results. In fact, this may be something you already employ a robot to do! Those methods are good for some things, but they also have blind spots where serious problems can lurk undetected. If you’re relying on statistical solutions, you may never know the problems are there, while they continue to frustrate your customers and steal your sales, slowly eating away at site search eCommerce.
Analytical Stumbling Blocks
Statistical solutions count things. They count how often people searched on a particular term, how often they clicked through, how often they converted. That is useful information—it’s just not the whole story.One problem is that you can’t count what you can't see. Very often when working with clients, we find that when we search on the name of a product they sell, the engine doesn’t return the right result. For example, the San Francisco Museum of Modern Art store has a product called “Alexander Calder Double Gong Tray.” If you search on that exact term, the product does not show up until the bottom of the second page of search results. If you’re only using analytics tools, you may never learn that.Another issue is that you have to count a lot of something in order to have confidence in your deductions, meaning that you have to have a big enough sample size to ensure you achieve statistical significance. The smaller the sample size, the less value your data has. For example, say you discover that 10% of people who add a certain item to their shopping carts never complete their purchase. You’ll have more confidence in that 10% number if you’re talking about 10,000 out of 100,000 instead of 10 out of 100.If you ignore sample size, then all of your deductions will be unreliable. So, you must achieve statistical significance.
Four Approaches—And Their Blind Spots
With search engine data, you can get a statistically significant sample in four ways—but each approach will leave you blind to something.The first way is to narrow your focus to only those areas where you already have a big enough sample size. For many retailers, this will only include a handful of your most popular search terms, also known as your “head queries.” Obviously, if you only look at your head queries, then you’ll be blind to everything else in the body and the tail—which comprises the vast majority of your search data. That’s an awful lot of information to ignore.The second method is to aggregate, or combine similar things, such as all your tail queries, which are all the long, specific queries, as well as misspellings, alternate spellings and abbreviations. The trouble with this approach is that you can’t break it down, so you lose the specificity that makes the analysis useful. You may learn how well you’re doing on tail queries overall, but you won’t be able to dig deeper to understand individual categories, so you lose significance again.The third way to achieve statistical significance is to increase your time frame, waiting to gather enough data about the thing you want to learn about. The drawbacks here are that you’ll still have areas that won’t produce enough data even after waiting for weeks or months, and you will also have a blind spot when it comes to any changing trends. This isn’t a good approach for eCommerce sites, as retailers tend to change their inventory with the seasons. For example, a sporting goods store doesn’t sell many golf clubs in winter or snowshoes in July.The fourth tactic is to replace statistical inference with human judgment—an approach used by every major search engine in the world. We call this a “curated crowd.” When curated crowds do the work, you’ll get the insight you need to fully understand how well you’re doing across all your search queries—even those in the tail. And you can achieve that understanding with a much smaller data set than the three other approaches require. You can get as granular as you like, breaking your data down to focus on misspellings, alternate spellings, natural language queries or whatever else you’d like to look at. This approach is primarily valuable in gaining insight into root causes, because people can see what statistical methods cannot. This method does not, however, produce a large number of examples for making tactical changes. It is best suited to tell you the “why” behind issues.
The Bottom Line: More Conversions
Analytical tools are useful for many things, but they’ll only take you so far. If you really want to measure your site search eCommerce results or search engine performance — and improve it — you should incorporate both statistical- and human-based methods. Statistical methods will help you identify very common and very specific problems. Human judgment will provide deeper insight into areas that statistical methods cannot penetrate — like your tail queries. Appen is uniquely positioned to provide this holistic approach because of our years of experience helping top eCommerce companies refine their search, our expansive curated crowd, and our leading role in establishing best practices.This unique, combined approach will help you understand how to make your search engine work better, help more customers find what they need quickly and increase your conversion rate. Not to mention delay that robot takeover.Are you interested in dramatically improving your search results? Contact us today.
