Deep learning-based collaborative filtering recommender systems: a comprehensive and systematic review

Abstract

Nowadays, the volume of online information is growing and it is difficult to find the required information. Effective strategies such as recommender systems are required to overcome information overload. Collaborative filtering is a widely used type of recommender system in e-commerce environments and can simply provide suggestions for users. Recently, deep learning approaches were applied in collaborative filtering to tackle some drawbacks. This systematic review aims to provide a comprehensive review of recent research efforts on deep learning-based collaborative filtering recommender systems. We explain the research methodology and paper selection process and the search query. 102 papers are selected out of 719 papers that were published between 2019 and May 2023. Furthermore, the approaches in the selected papers are classified into two main categories: memory-based and model-based techniques. The main ideas, advantages, disadvantages, used tools, type of neural network, applications, and evaluation parameters of each selected paper are also discussed in detail. It was found that CNN (Convolutional Neural Network), AE (Autoencoder), DNN (Deep neural network), and Hybrid networks are the four mostly used neural networks in recommender systems. Also, Python, MATLAB, and Java are the most frequently used tools in the reviewed papers. Regarding the applications of the recommender systems in the reviewed papers, movies, products, and music recommendation are three most frequent applications. We point out the open issues and future research directions. Some key challenges such as cold start, data sparsity, scalability, and accuracy are still open to be addressed.

Appendix 1: List of evaluation parameters and their description

Evaluation parameter	Description and formula
Confusion matrix	Confusion matrix has four outputs as follows: True Positive (TP): The number of correctly positive predictions True Negative (TN): The number of correctly negative predictions False Positive (FP): The number of predictions that are labelled positive incorrectly False Negative (FN): The number of predictions that are labelled negative incorrectly
Accuracy	Accuracy is calculated as the number of correctly predicted observations over the total number of observations ${\mathbf{Accuracy}} \, = \;\frac{{{\text{TP}} + {\text{TN}}}}{{{\text{TP}} + {\text{TN}} + {\text{FP}} + {\text{FN}}}}$
Precision	Precision is calculated as the number of correctly predicted positive observations over the total predicted positive observations. Indeed, it is the ability of the model not to label a negative sample as a positive ${\mathbf{Precision}} = \frac{{{\text{TP}}}}{{{\text{TP}} + {\text{FP}}}}$
Recall	Recall is calculated as the number of correctly predicted positive observations over all positive observations in the actual class of the dataset. Indeed, it is the ability of the model to predict all positive samples correctly ${\mathbf{Recall}} = \;\frac{{{\text{TP}}}}{{{\text{TP}} + {\text{FN}}}}$
F-measure	F-measure is a weighted average of precision and recall and considers FP and FN $\user2{ }{\mathbf{F}} - {\mathbf{measure}} = \frac{{{\text{Precision}} \times {\text{Recall }}}}{{{\text{Precision}} + {\text{Recall}}}} = \frac{{2{\text{TP}}}}{{2{\text{TP}} + {\text{FP}} + {\text{FN}}}}$
ROC (AUC)	ROC is a curve for illustrating the trade-off between specificity and sensitivity. The AUC is the area under the curve. It is the ability of model for distinguishing positive and negative classes
MSE	Mean squared error (MSE) determines the amount of error in the models. It is calculated as the average squared difference between the predicted and the observed values ${\mathbf{MSE}}\user2{ = }\;\frac{1}{{\text{n}}}\mathop \sum \limits_{{{\text{i}} = 1}}^{{\text{n}}} \left( {{\text{Y}}_{{\text{i}}} - {\hat{\text{Y}}}_{{\text{i}}} } \right)^{2}$
RMSE	The root-mean-square error (RMSE) represents the square root of MSE ${\mathbf{RMSE}} = \sqrt {MSE}$
MAE	Mean absolute error (MAE) is calculated as follow in which Y and X can be predicted value and observed value respectively $\user2{ }{\mathbf{MAE}}\user2{ = }\;\frac{1}{{\varvec{N}}}\mathop \sum \limits_{{{\varvec{i}} = 1}}^{{\varvec{N}}} \left| {{\varvec{Y}}_{{\varvec{i}}} - {\varvec{X}}_{{\varvec{i}}} } \right|$
NDCG	Normalized Discounted Cumulative Gain (NDCG) is a measure for ranking quality. It calculates the Cumulative Gain of a set of results by summing up the total relevance of each item in the result set. Then, the position of each item is discounted for, meaning the lower a relevant item is in the list, the higher the penalty or the discount that the item contributes to the total score
Hit Ratio	The whole hit rate of the system is the count of hits, divided by the test user count. It measures how often we are able to recommend a removed rating, higher is better
Utility	Utility could be measured by evaluating the rating that the user gives to predicted items after consuming them
Quality	The quality of the recommendations is based on how relevant they are to the users and also they need to be interesting
Privacy	Privacy metric focuses on preventing unwanted disclosure and usage of information
Coverage	Coverage is the percentage of the possible recommendations that an implemented recommendation algorithm can produce
MRR	Considering a sample of queries Q, the mean reciprocal rank (MRR) is a statistic measure for evaluating any process that produces a list of possible responses to a sample of queries ${\mathbf{MRR}} = \;\frac{1}{\left| Q \right|}\mathop \sum \limits_{i = 1}^{\left| Q \right|} \frac{1}{{{\text{rank}}_{i} }}$
MAP	Mean average precision (MAP) for a set of queries is the mean of the average precision scores for each query, where Q is the number of queries ${\mathbf{MAP}} = \; \frac{{\mathop \sum \nolimits_{q = 1}^{Q} Ave P\left( q \right)}}{Q}$
Error	An error metric is directly related to RMSE and MAE metrics. It can provide a way for forecasters to quantitatively compare the performance of competing models