Blog

Recommendation systems: benefits and development process issues

Recommendation systems: benefits and development process issues

Already for several years, recommendation systems (or recommenders) became essential for every person who uses the Internet on a daily basis. We can face recommenders while using large e-commerce websites like Azamon and eBay, online movie and streaming platforms like Netflix, Hulu, and Spotify.

Today we will share our experience and expertise talking not only about the benefits recommendation engines bring to the business, but also about the most common issues we face during the development.

Interested? Keep on reading!

 

The benefits of the recommendation engines

Today the majority of the recommendation systems are based on machine learning, so its’ main disadvantages partially correlate with the usual issues we face during typical machine learning development, but are still slightly different. Let’s have a closer and a more dedicated look.

 

1. Revenue and Sales Increase

For years the revenue increase is probably the most popular indicator for every business owner. As we already know, recommendation systems can be used in various situations to solve different business goals. Let’s have a closer look at Amazon, where the successful recommendation systems lead to the 29% annual sales increase.

Amazon analyzed the most common purchases made by its users with the purpose to gain the insights their business intelligence systems did not find. The list of purchases is a vast amount of data, so it is impossible to do it manually, as well as it is very complicated and takes a long time to find some correlations between some purchases for traditional data analysis algorithms.

So Azamon used comprehensive machine learning algorithms to processes the data. As a result, after the successful model learning, the developers from Amazon achieved the neural network, that can predict users purchases, according to their shopping history, and recommend them next product to buy. Also, every user was categorized and placed into several groups, according to the data about the users with similar buyer profile (so-called look-a-like audience).

If you know, what most likely the user will buy next, you can build a strategy where you offer the most relevant goods to increase average check.

Amazon stock price increase after the recommendation engine integration 

 

2. User Satisfaction Growth

As you probably already understood, the second advantage comes from the first one – user satisfaction.

When a user sees the personalized feed, generated by the recommendation system: no matter we are talking about music, books, news, movies or e-commerce – he feels less stress and more connected with the service.

So if the service can offer the personalized feed (built according to the correlation with the recent purchases) and understands the user’s needs (from similar look-a-like audience), it tends him to buy more what lead both to the customer satisfaction and revenue increase.

 

3. Turnover Increase

Recommender engines can provide the turnover increase for any business. As we already know, the recommendation engine analyzes the users’ behavior. It can take into consideration the connections between several users too.

Everyone cares about what other people think about us: especially our friends and relatives. The recommender engine can provide quite precise recommendations according to the reviews our friends left about the specific good or service.

Moreover, we can build a model that can recommend the goods or services taking into consideration the reviews from your friends and relatives, what will help the user to stay in touch with its relatives and friends.

This way, we can recommend the original barbershop not only to a similar audience but also to the relatives or friends of a specific user by showing its review. Finally, that will increase on-site offline traffic and generate additional sales. By the way, some huge classifies like Yelp already implemented a similar feature.

 

Unobvious moments we face during recommenders development

It is not customary to talk about shortcomings, but we will. Is should be remarked, that building recommender is not the natural and straightforward process, as it may seem to be.

At Azati we developed several recommended systems so we can share our experience and describe the most common problems we often face.

 

1. Deficiency of Information

Perhaps the most common and significant difficulty is a lack of high-quality data to complete the neural network learning.

It takes a lot of cleared data to create a recommendation system that works efficiently and makes precise suggestions. The neural network that powers recommender is quite sensitive to any data distortion. It means, that if we give the inaccurate and uncleared data – we cannot expect the precise results.

While teaching a neural network for a huge company (like Facebook or Google), you can expect the cleared data without any distortions, just because they have a plenty amount of data to share. But medium and small companies often not to have a lot of data, and the vendor uses it as is.

The great example is from our another project. We developed the system that digitizes documents and is powered by another machine learning model (you may check out Azati.ai to learn more). The purpose of the system was to scan and digitize the reports being generated by another software. Well, that was not the easiest task, just in case at some documents the text was overlaid one to each other in such a way, that even our in-house specialist could not recognize it.

Without cleaning and filtering, even scientific data can unreliable

Finally, it means that if you want to get great suggestions, we recommend you providing efficient and correct data to your vendor to avoid the additional problems in the future. From our expertise, we can say that it worth spent more time on data collection and cleaning than to reteach the neural network several times without clear understating what exactly went wrong.

 

2. Information Variability

Recommendation Engines are always based on the data collected during past or current periods, and it is a cruel fact by the way.

There are several business industries (Media, Online Gaming, Marketing), where the information is changing swiftly. In such sectors, we use the “old data” to complete the learning, while the situation could already be changed. When we finish the teaching, there can already be more precise and fresh data, so the suggestions are less relevant or even irrelevant at all.

There is a simple way, and we can solve this issue: we reteach the neural network after every new period again and again.

But here are some moments, that should be taken into consideration:
1. Machine Learning process takes time: the more data we use to complete the learning – the more time and resources it would consume.

Finally, If there are vast amounts of data you want to use without releasing unnecessary in the process, and plan to add additional datasets every future period – the time for data processing will increase in arithmetic progression.

2. Resource consumption. We can divide it into two categories: human resource consumption and virtual resource consumption.

What concerns human resource consumption, you will probably need a dedicated specialist that will provide the necessary operations to reteach the neural network. Hopefully, you can outsource that process and decrease support costs.

Virtual resource consumption is about renting or buying a dedicated computer powers that are used for machine learning. Unfortunately, you can’t optimize that process: the more data you have, the more time will require to teach the neural network and the more resources it will consume in process.

Hopefully, there is a way to avoid all that complexities – get rid of machine learning and use traditional approaches, methods, and algorithms to process the data. It will be less accurate, more predictive and easy to support. The suggestions will be slightly less accurate, but still useful to users or clients.

 

3. Unpredictable performance

Today we live in an extremely fast-changing world, so businesses all over the globe are in chase of constant revenue increase and turnover growth. The best way to provide year to year growth is to plan everything: develop the strategy and follow it. The every business strategy is primarily based on numbers and data.

As business owners count not only every minute but also the single cent and calculate the Return On Investment (RIO) for every operation they perform, it is rather challenging to estimate the value of the recommendation engine at first.

Although it is easy to understand what benefits recommenders provide, close to impossible to evaluate the вenefits it brings in money terms until you try implementing it to the existing business processes.

These way businesses prefer to invest money in something more predictable. But always there is an enthusiastic company that invents or integrate new technologies making crazy incomes in short terms, and after that, the market follows.

The most excellent example of such technological integration is Netflix. Recommendation platform integration at Netflix led to the increase in the number of screen views more than twice. Now they say, that the recommender generates about 75% of all Netflix views. It sounds like the truth.

What benefits recommendation engine provided at Netflix

But not so many people know, that year to year Netflix improved their recommendation system by holding a public competition with an impressive prize pool. The competition was called “Netflix Prize”. After all, a group of scientists developed a new recommendation algorithm that beat the existed system in 2009.

That example is quite descriptive, showing us the way it works: we integrate recommendation engine into the existing ecosystem, benchmark the performance and tune the algorithm, making it solve the current business goals.

 

Summary

The recommendation system is not the kind of a thing that is easy to imagine in every company and every industry, but if it is suitable for your business and there already are successful integration cases in your industry, so you should probably think about it.

 

The article is provided by Konstantin Lebejko

Continually creates valuable content about AI, Machine Learning, and Conversational Agents for Azati Software. Enjoys visiting different marketing meetings and thinking about the sense of life.

 

By the way, if you are interested in the development of the recommendation system – drop us a line or call +1 (973) 597-1000 and we will provide the consultation for free.

 

Your Name (required)

Your Email (required)