Beni x AI: Improving the product ranking algorithm to deliver relevant alternatives
Post 2 of 3 breaking down the role of generative AI in helping Beni deliver on its mission to make resale accessible to everyone.
Before you jump in, have you read the first post in this series? If not, start there!
Okay, now we’re ready.
Making resale accessible is more complicated than it seems. With over 200M listings in our database, aggregated from over 40 different resale platforms, we needed a way to cross check the shopper’s wish with our database to deliver a relevant resale alternative in just a few seconds. Moreso, we needed a way to understand the shoppers intent to determine what the most relevant alternative was for that specific shopper.
Enter: Beni’s Ranking Algorithm.
First, let’s talk about how ranking algorithms work.
A ranking algorithm is a method used to rank items in a list, based on certain criteria or metrics, in a way that is meaningful and useful for the particular application. They can be relatively simple by using basic statistical methods, or highly complex utilizing machine learning and other advanced techniques. They’re critical for search engines like Google to deliver you the most relevant web pages on the internet based on your search, or recommendation systems built into applications like Spotify or Amazon, sorting items (songs, products, etc) based on user preferences, previous interactions, similarities between items, and other user behavior. They’re important for any application with a “feed” (think Facebook, Twitter, LinkedIn) with a goal of optimizing engagement and relevant content for the user, and they’re important for Beni, with a goal of delivering the most relevant secondhand versions of what you are shopping for.
So how did we build our ranking algorithm? When we set off, we generated a set of rules that made sense based on our business and what we are trying to achieve, and began by just hard-coding those rules into our application. We match images of secondhand listings in our database to the image of the item you are shopping for, and by doing so, we generate a “similarity score”. Along the same lines, we generate a set of scores from matching on various text-based attributes like product title and brand. We then add weights to each of those elements, along with some other product filterability, to achieve a rank-ordering of products that we feel is most relevant to the user. The algorithm produces a score on the confidence in our ability to deliver an “exact item”, and we deliver to you those resale listings from highest to lowest confidence.
We initially focused on developing logic to rank items with attributes similar to the product you're shopping for, as this was our primary use case for the extension. However, in the broader context of e-commerce, and other areas of opportunity for Beni, different forms of logic can be utilized to generate recommendations to you in different shopping modes, including:
Showing you items that are often bought in conjunction with the product you're shopping for.
Showing you items purchased by users who share demographic or behavioral similarities with you.
Showing you products that possess attributes akin to the item you are interested in buying.
These multifaceted approaches to product ranking and recommendation help us create a personalized and “smart” shopping experience for you, seamlessly connecting you with items that align with your interests and purchasing habits (and ultimately, giving you a sensation that we’ve read your mind!!).
The first questions that we asked ourselves when building our ranking algorithm from scratch were:
What is the desirable behavior of the system, and when is it achieved?
What are the metrics that we can measure ourselves against to determine whether we are achieving the desired user behavior?
If our goal at Beni is to deliver the most relevant secondhand versions of what you are shopping for, we know we are doing a great job if you are clicking on our listings and ultimately making a purchase. We can now use this information to optimize our ranking algorithm and test changes. As you might imagine, there is more happening at the “top of the funnel” (e.g. the number of times you expand the extension) compared to the “bottom of the funnel” (e.g. that time when you actually bought something that Beni showed you).
Therefore, to be able to iterate quickly, we optimize for earlier signs of engagement instead of optimizing for our end-state goal which is, of course, finding you that really amazing secondhand alternative for half the price. At this point our ranking algorithm is learning from users like you, by iterating on the algorithm and seeing how it improves your engagement with our extension. Ideally we get to a state where we can get multiple changes out every week. Are you seeing the improvements?!
When we feel like we’ve got it just right (we’re not there yet, but I’m sure you can see that we’re working hard!), we can begin to train a model to learn from the product rankings that we’ve been delivering to our users.
In some ways, the rules that we developed early on and over time become features to the ranking model. Eventually, the model will be able to learn and iterate on user behavior faster than our development team. When we get there, I promise to share our learnings with y’all once again. Stay tuned!
 | A guest post by
|