Skip to main content
Log in

A gradual rank increasing process for matrix completion

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

The matrix completion problem is easy to state: let A be a given data matrix in which some entries are unknown. Then, it is needed to assign “appropriate values” to these entries. A common way to solve this problem is to compute a rank-k matrix, B k , that approximates A in a least squares sense. Then, the unknown entries in A attain the values of the corresponding entries in B k . This raises the question of how to determine a suitable matrix rank. The method proposed in this paper attempts to answer this question. It builds a finite sequence of matrices \(B_{k}, k = 1, 2, \dots \), where B k is a rank-k matrix that approximates A in a least squares sense. The computational effort is reduced by using B k-1 as starting point in the computation of B k . The ability of B k to serve as substitute for A is measured with two objective functions: a “training” function that measures the distance between the known part of A and the corresponding part of B k , and a “probe” function that assesses the quality of the imputed entries. Watching the changes in these functions as k increases enables us to find an optimal matrix rank. Numerical experiments illustrate the usefulness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Bertsekas, D.P.: Nonlinear Programming, Athena Scientific, Belmont, MA (1995)

  2. Bjorck, A.: Numerical Methods for Least-squares Problems, SIAM, Philadelphia (1996)

  3. Buchanan, A.M., Fitzgibbon, A.W.: Damped Newton Algorithms for Matrix Factorization with Missing Data In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVP’05), 2(2005), 316–322

  4. Chen, P.: Optimization algorithms on subspaces: revisiting missing data problem in low-rank matrix. Int. J. Comput. Vis. 80, 125–142 (2008)

    Article  Google Scholar 

  5. Chen, P., Suter, D.: Recovering the missing components in a large noisy low-rank matrix: application to SFM. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1051–1063 (2004)

    Article  Google Scholar 

  6. Dax, A.: Orthogonalization via delfation: a minimum norm approach for low-rank approximations of a matrix. SIAM J. Matrix Anal. Appl. 30, 236–260 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  7. Dax, A.: Imputing missing entries of a data matrix: a review. J. Adv. Comput. 3, 98–222 (2014)

    Google Scholar 

  8. Elden, L., Savas, B.: A Newton-Grassmann method for computing the best multilinear rank- (r 1,r 2,r 3) approximation of a tensor. SIAM J. Matrix Anal. Appl. 31, 248–271 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  9. Fletcher, R.: Practical Methods of Optimization, Wiley (1980)

  10. Franklin, S.B., Gibson, D.J., Robertson, P.A., Pohlmann, J.T., Fralish, J.S.: Parallel analysis: a method for determining significant principal components. Journal of Vegetation Science 6, 99–106 (1995)

    Article  Google Scholar 

  11. Gabriel, K.R., Zamir, S.: Lower rank approximation of matrices by least squares with any choices of weights. Technometrics 21, 489–498 (1970)

    Article  MATH  Google Scholar 

  12. Golub, G.H., Van Loan, C.F.: Matrix Computations, Johns Hopkins Univ. Press (1983)

  13. Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24, 417–441 and 498–520 (1933)

    Article  MATH  Google Scholar 

  14. Howard, K.I., Gordon, R.A.: Empirical note on the ‘number of factors’ problem in factor analysis. Psychol. Rep. 12, 247–250 (1963)

    Article  Google Scholar 

  15. Jackson, D.A.: Stopping rules in principal component analysis: a comparison of heuristical and statistical approaches. Ecology 74, 2204–2214 (1993)

    Article  Google Scholar 

  16. Jain, P., Netrapalli, P., Sanghavi, S.: Low-rank matrix completion using alternating minimization, Tech. Report. arXiv:1212.0467v1 (2012)

  17. Ledesma, R.D., Valero-Mora, P.: Determining the Number of Factors to Retain in EFA: an easy-to-use computer program for carrying out Parallel Analysis, Practical Assessment Research & Evaluation 12 (2007)

  18. Markovsky, I.: Algorithms and literate programs for weighted low-rank approximation with missing data. Technical Report 18296, ECS, Univ. of Southampton, http://eprints.ecs.soton.ac.uk/18296/ (2009)

  19. Parlett, B.N.: The Symmetric Eigenvalue Problem, Prentice-Hall, Englewood Cliffs, NJ (1980)

  20. Peres-Neto, P.R., Jackson, D.A., Somers, K.M.: How many principal components? Stopping rules for determining the number of non-trivial axes revisited. Comput. Stat. Data Anal. 49, 974–997 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  21. Schechter, S.: Iteration method for nonlinear problems. Trans. Amer. Math. Soc. 104, 179–189 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  22. Schechter, S.: Relaxation methods for convex problems. SIAM J. Numer. Anal. 5, 601–612 (1968)

    Article  MathSciNet  MATH  Google Scholar 

  23. Schechter, S.: Minimization of a convex function by relaxation, Integer and Nonlinear Programming, (J. Abadie, ed.), North-Holland, Amsterdam (1970)

  24. Stewart, G.W.: Matrix Algorithms, vol. I: Basic Decompositions, SIAM, Philadelphia (1998)

  25. Wilkinson, J.H.: The Algebraic Eigenvalue Problem. Clarendon Press, Oxford (1965)

    MATH  Google Scholar 

  26. Zacharih, D., Sundin, M., Jansson, M., Chatterjee, S.: Alternating least-squares for low-rank matrix reconstruction. IEEE Signal Process. Lett. 19, 231–234 (2012)

    Article  Google Scholar 

  27. Zwick, R.W., Velicer, W.F.: Comparison of five rules for determining the number of components to retain. Psych. Bull. 99, 432–442 (1986)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Achiya Dax.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dax, A. A gradual rank increasing process for matrix completion. Numer Algor 76, 959–976 (2017). https://doi.org/10.1007/s11075-017-0292-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-017-0292-2

Keywords

Navigation