This post still belongs to my Q&A series. However, for this topic, it’s not a somewhat Q&A. Instead, it is the conversation in my mind.
This post is my pov about how the recommendation is used for content-based applications. I’m not a machine learning or AI professional, and this is only my feeling and my experience.
Nowadays, we talk too much about the recommendation system, personalization, and AI. From a rule-based strategy to a magic AI model, we expected to offer our users what they want to see and consume. From another viewpoint, we would like to increase the number to reach the KPIs, which brings us some benefits.
However, is there sometimes you think about is it good or bad?
As a user, sometimes I feel scared about how some products understand my hobby. They keep feeding me that I like but would also like to get rid of it. It’s exhausted. All about you are exposed to the public, and that’s awful.
From the view of a product manager, in some situations, it’s pretty easy to feed users what they want to see to reach the KPIs. But then your product will look like the garbage of all kinds of dirty content. Do you feel good about doing that?
Rule-based model or AI model?
I have a little bit of knowledge about how the modern recommendation system works.
I see that nowadays, people are abused by using the “AI” model, even if they don’t understand what it is and how it works. They keep talking about deep learning, convolutional network, neural network, collaborative filtering, etc. But, sometimes, they would like to use it because it’s trendy, new, or maybe it is good for them to do some marketing activities.
AI comes from something basic such as the rules or statistics. Models itself is a great invention, I think. But the fact is there is no silver bullet to solve all things.
Sometimes I see that some product managers would like to make some noise, then instead of defining a rule-based to solve the user’s problem, they keep “finding the model” first.
As a product manager, you first need to know what you want to do, who you would like to serve, and what you want to offer.
From my experience, first, let’s define the rule based on your need and your problem to solve using data-driven or your knowledge. Then, once the issue is overwhelmed, let’s try with the magic model later. At least, while you define the rules, you will find what you need precisely, the challenge to solve, the cost of operation, and the expected outcome.
Candidates and ranking
I usually ask my colleagues how to pick the candidates — the content that we offer to users— and how to rank them. Of course, I mean the position of choice in the list, especially the three first positions.
Users are lazy. The list’s first three to five items usually get the highest CTR unless you did something stupid.
Let’s imagine you picked a list of 10 items that you think are the best for the user. My question is:
- “Which should be placed at the top?”
- “The top-scoring item.”, I usually get the answer.
- “Why it’s score is highest?” I asked.
- “Because it is the most relevant option for the user based on the AI model scoring.”, he said.
No, it is not a good answer, IMO. When you play a chess game with AI, the best AI knows when to win, when to lose, and when the organizer would like the result to be. It is not the super complicated intelligent algorithm to win all matches.
Back to the topic, you have to manage many situations, not only depending on the result of an AI model. AI model somehow, I think, does not have human feeling and consideration, or at least it is insane to be.
As a product manager, you must balance the organization’s vision, expectations of your product, user needs, user experience, and many things else.
For example, the best option from the result of AI maybe actually have the most significant possibility of catching user clicks based on user history activities. However, it is not matched the organization’s principles, or it does not bring enough revenue.
The characteristics of the content itself can be taken an important part. For example, I treat hard news as fast food. You can not serve yesterday’s disc to users today. This portion also requires the experience and knowledge of the product manager.
Also, similar to the above chess game, you need to know when to chase users, when you need to let them be free, when to remind them of a memory, or when to suggest new chances. It is not the algorithms. It is a human strategy.
Another point is you should consider that try to build a human-like AI model that solves all problems or build a “solution” that uses AI as a tool. That is “management.”
The interests and the noise
Let’s begin with an actual story. First, you build a news website. Then, one day, a meteorite flies over the earth, and people all talk about that. I am also curious about that and keep reading.
It’s ridiculous if your system thinks I love to read about the cosmos because I read about meteorites too much.
It does not usually happen with a robust enough recommendation system, but it usually occurs when a product manager first reaches a recommendation solution with limited resources.
The point here is that user interests are only discovered when you have a long-tail enough history of activities, not based on recent activities.
Once again, the most important thing is yourself. You should be clear about what you would like to offer to users. If you would like to catch the trend in a short time, use recent activities, or would like to serve users based on their interests, use a long-tail one.
Prebuilt or run-time
In most cases, when we think about a recommendation solution, we think about a UI section shown to users when they first open the website or the list of similar items when they scroll down while consuming a specific content.
I see there are three significant contexts of recommendation: prebuilt recommendation, related-content recommendation, and flow-based recommendation.
The prebuilt recommendation is suitable for impressing users the first time using. The related-content recommendation is good for keeping users exploring what may they be interested in at the moment browsing. It’s maybe content-metadata similarity, or in the more complete way, the collaborative filtering. The flow-based recommendation, IMO, is similar to “similar-content recommendation” but somehow more advanced. It cares about where the user has begun, how they have reacted to the system along the path, and how the result affected their mood and makes an adaptive change right after recognizing a signification signal from the user.
You can see the flow-based recommendation in some new media products. The next items will be determined based on the user’s recent feedback on the chain of previously consumed contents.
The major difference here is the moment of feeling and mood.
For example, initially, the system is built and offers you a list of videos with a wide range of categories. You don’t know the complete list the first time, but keep watching and scrolling. Suddenly, you find out that the list of videos is all about sport, that you have stopped scrolling and watching for a while before. The system did not build that list from the beginning because of your recent behavior. Once you give them a signal of burned-out with the content, you quickly see a pivot. Once you quit that loop, the flow will be ended, and it starts a new flow when you get back.
I have experience with a more complex case, the flow of music. Unlike other kinds of content, music is feeling and the soul’s voice.
Which song would be served next is not only about the statistics or algorithm. It should be included the sense of the product manager of understanding music.
Individual or user segment?
It’s so sexy to have a system that individually serves each user’s hobby. But the problem is the scale and the cost.
Technically, there are many ways to optimize the cost, but it is still hard and needs a vast effort. Don’t burden your technical team because of your lack of knowledge.
An alternative good enough solution here is to try to narrow down the problem by grouping the similar uses into some segments and do the same with the content. That can be easier to match them later.
Now you have another problem to solve it is how to group them. It depends on your experience and knowledge.
Individual personalized recommendation somehow can still be used on a small scale to keep UX better but not too costly.
As a product manager, you should sensitively take care of costs. It is not only about the money but also about the workload.
How to communicate with the technical team about recommendation?
Unless you are strong in this field or need a fancy solution to show off, don’t force the technical team to apply your solution. Instead, clearly define your needs and strategy, and discuss them with the technical team to find a suitable solution.
On the other hand, if your technical team is an AI fan. All things they want is to implement a super AI model, then let clear with them your point, keep them calm and find realistic solutions.
The system must be flexible enough to custom, and stable enough to keep the output consistently. At the end of the day, you — the product manager — should have the ability to manage the output of the system.
There is no silver bullet, the AI also. It is only a tool.
“It’s not the plane, it’s the pilot” — Top Gun: Maverick
Final words
The above is just some of my thoughts about recommendations. It’s hard to conduct everything on onetime. I hope to come back to this topic with more deep experience to share.
AI is good. But don’t trust it. Trust your sense. Design your solution, and experience it day by day. In the recommendation field, IMO, one of the most important factors is the sense of the product manager, not the algorithms.
I am impressed with my former boss’ sentence. If something is too hard to solve technically, solve it with a non-technical solution.
Goodluck.
