Every eCommerce merchant knows the pain.
Search results for hundreds or thousands of queries aren’t producing results that drive business KPIs. Hours are spent manually re-ranking products by popularity (or even worse, “relevance”), creating synonyms for basic misspellings, adding redirects to fix search engine gaps, and finding/fixing zero-result pages — only to mention a few.
The tasks are endless.
And at the end of the day, the product discovery experience — if you’re lucky — is only marginally better than it was the day before.
But why is this?
The problem with traditional search systems
Traditional search systems today aren’t built to make the life of a merchandiser easy.
Many retailers are using search systems built on Lucene (Solr, ElasticSearch, etc.) to power search & discovery. This system is built off a TF-IDF algorithm, which attempts to match products to a user’s search query based only on keyword matching and term frequency.
But these algorithms were built with document search in mind, not eCommerce.
Search systems like ElasticSearch struggle with this problem. Although your users may get more “relevant” search results with Elastic, the system doesn’t truly know what your users want to see or what they want to purchase. It’s the merchandiser’s job to conduct never-ending tests to (hopefully) learn this information.
Trying to manually optimize hundreds or thousands of search queries and browse pages is impossible. The amount of time and resources required just to maintain the site is enough of a task. But that’s just a symptom of the problem.
The root cause is a lack of vital information.
Don’t disagree with your users
Here’s the thing:
A majority of the merchandising work that goes on today is educated guesswork.
It’s rare that merchandisers will know exactly what products to place in specific positions for any given query to guarantee an increase in business KPIs. As a result, they’ll turn to things like “relevance” or “popularity” to guide their work (which isn’t nearly the best indicator of a product’s success).
But intelligent search systems today do know how to rank products for any query to increase business KPIs.
How?
We define clickstream as data on the actions & pathways your users take on your website — whether it’s searching, browsing a category page, clicking, or adding to cart/purchasing.
Rather than guessing what products each and every user wants to see and purchase for any given query, clickstream data shines light into what your users are telling you they want to see. It’s simple: the products that should be ranked the highest are the products that users interact with and purchase most (i.e. the products most likely to lead to an increase in your business KPIs).
Internet giants like Amazon and Google are using clickstream to do incredibly useful things like:
- Find where users are getting stuck. What browse pages, search result pages, etc. do users not convert on? How can these pages improve and what would be the outcome of those improvements?
- Find and optimize which pathways users take to find specific products. How can those pathways be optimized to ensure users find what they’re looking for in the quickest time possible?
- Quickly spot zero-result pages. When users do hit zero-result pages, how do they refine their searches?
- Determine how a user’s geographic location, time of day, etc. affects their purchasing decisions.
…but that’s only the starting point. Many advanced retailers are implementing machine learning algorithms that can collect, read, interpret, and use clickstream data to answer questions like:
- What are the best positions for any given item within any given query to maximize business KPIs that we care about?
- What are the best positions for any given item in a result set for any given user? What is the outcome of introducing personalization?
- What would be the outcome of tailoring each user’s experience across the entire product discovery experience (search, autosuggest, browse, recommendations)?
(Need proof? Check out some of our case studies, or let us show you what this system can look like on your site, with your data.)
In short, these companies are leveraging clickstream data + machine learning to reduce (and in many cases, eliminate) the merchandisers’ tedious & repetitive tasks and allow them to focus on what matters most.
In other words, this allows merchandisers to begin managing by exception.
Merchandising in 2020: Managing by Exception
With the addition of clickstream-backed machine learning, merchandisers no longer have to worry about tasks like manually re-ranking products and creating synonyms and redirects. The AI does that for them.
“Won’t this approach make the merchandiser’s job irrelevant?”
We get this question often at Krestor — and the answer is always a solid no.
Machine learning doesn’t eliminate the job of a merchandiser. In fact, it enables them to be smarter and more effective.
Merchandisers can now focus on being creative and finding ways to drive real business value without relying on educated guesswork.
Maybe you’ve just secured a partnership with a brand and would like to create an awesome collections page to boost their products. Maybe you’d like to curate themed collections around a sale or a holiday and promote them on your home page. Or maybe you’d like to manually merchandise queries with a high lost-sessions rate (see below for example):
When your merchandisers can trust your systems and your data, all of this becomes possible.
Want to see for yourself?
The beacon is a two-line snippet of JavaScript that takes your clickstream data, feeds it into Krestor’s machine learning models, and presents you with a panoramic overview of improvements you’d see by implementing machine learning in your search — as well as the low-hanging value you can capture today.
The Beacon also gives you insights into your user behavior patterns:
- Where are users getting stuck on your site?
- What pathways do users take to find specific products?
- What are patterns of product attribute affinities across search results, browse pages, and in terms of geography and interest graph?
- Across all touchpoints, where is product ranking upside-down? Meaning, where do users interest and your ranking diverge?
- Where are users seeing zero-result or frustrating result pages — and when users hit those pages, how do they refine their searches successfully or unsuccessfully?
The process for installing the Beacon is simple, and can be completed in the span of 2 to 5 minutes as shown in the video below:
There are no front-end or back-end changes that take place after the Beacon is installed — our systems will take care of the rest. Whether you’re making changes to your back-end, re-platforming, or anything else, Beacon will stay up-and-running without interfering with other important tasks.
