Recent Research Reviews Journal
IRO Journals
Volume 2 — Issue 2 — December 2023

Nonnegative Matrix Factorization: A Review

https://doi.org/10.36548/rrrj.2023.2.006
Abdul bin Ismail
School of Engineering and Computing, Manipal International University
PDF
PDF

How to Cite

Ismail, Abdul bin. 2023. “Nonnegative Matrix Factorization: A Review”. Recent Research Reviews Journal 2 (2): 324-42. https://doi.org/10.36548/rrrj.2023.2.006.
ACM ACS APA ABNT Chicago Harvard IEEE MLA Turabian Vancouver AMA

Download Citation

Endnote/Zotero/Mendeley (RIS) BibTeX

Keywords

— Non-Negative Matrix Factorization
— Review
— Survey
— Challenge
Published: 01-11-2023

Abstract

Recent developments in Non-negative Matrix Factorization (NMF) have focused on addressing several challenges and advancing its applicability. New algorithmic variations, such as robust NMF, deep NMF, and graph-regularized NMF, have emerged to improve NMF's performance in various domains. These developments aim to enhance the interpretability, scalability, and robustness of NMF-based solutions. NMF is now widely used in audio source separation, text mining, recommendation systems, and image processing. However, NMF still faces challenges, including sensitivity to initialization, the determination of the appropriate rank, and computational complexity. Overlapping sources in audio and data sparsity in some applications remain challenging issues. Additionally, ensuring the consistency and stability of NMF results in noisy environments is a subject of ongoing research. The quest for more efficient and scalable NMF algorithms continues, especially for handling large datasets. While NMF has made significant strides in recent years, addressing these challenges is crucial for unlocking its full potential in diverse data analysis and source separation tasks.

References

  1. Z.-Y. Zhang, “Nonnegative matrix factorization: Models, algorithms and applications,” in Data Mining: Foundations and Intelligent Paradigms, vol. 2, D. E. Holmes and L. C. Jain, Eds. Berlin, Heidelberg, German: Springer Berlin Heidelberg, 2012, pp. 99–134.
  2. Y.-X. Wang and Y.-J. Zhang, “Nonnegative matrix factorization: A comprehensive review,” IEEE Transactions on Knowledge and Data Engineering, vol. 25, no. 6, pp. 1336–1353, Jun. 2013.
  3. B.-W. Chen, W. Ji, S. Rho, and Y. Gu, “Supervised collaborative filtering based on ridge alternating least squares and iterative projection pursuit,” IEEE Access, vol. 5, pp. 6600–6607, Mar. 2017.
  4. W. Yan, B. Zhang, and Z. Yang, “A novel regularized nonnegative matrix factorization for spectral-spatial dimension reduction of hyperspectral imagery,” IEEE Access, vol. 6, pp. 77953–77964, Dec. 2018.
  5. W. Yan, B. Zhang, Z. Yang, and S. Xie, “Similarity learning-induced symmetric nonnegative matrix factorization for image clustering,” IEEE Access, vol. 7, pp. 166380–166389, Nov. 2019.
  6. J. Huang, F. Nie, H. Huang, and C. Ding, “Robust Manifold Nonnegative Matrix Factorization,” ACM Transactions on Knowledge Discovery from Data, vol. 8, no. 3, pp. 1-21, Jun. 2014.
  7. C. Bo, “Symmetric nonnegative matrix factorization based on box-constrained half-quadratic optimization,” IEEE Access, vol. 8, pp. 170976–170990, Sep. 2020.
  8. L. Du, X. Li, and Y.-D. Shen, “Robust nonnegative matrix factorization via half-quadratic minimization,” in Proc. 12th IEEE International Conference on Data Mining, Brussels, Belgium, 2012, Dec. 10–13, pp. 201–210.
  9. Y. He, F. Wang, Y. Li, J. Qin, and B. Chen, “Robust matrix completion via maximum correntropy criterion and half-quadratic optimization,” IEEE Transactions on Signal Processing, vol. 68, pp. 181–195, 2020.
  10. W. Min, T. Xu, X. Wan, and T.-H. Chang, “Structured sparse non-negative matrix factorization with ℓ2,0-norm,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 8, pp. 8584–8595, Aug. 2023.
  11. D. Kong, C. Ding, and H. Huang, “Robust nonnegative matrix factorization using L21-norm,” in Proc. 20th ACM International Conference on Information and Knowledge Management, Glasgow, Scotland, United Kingdom, 2011, Oct. 24–28, pp. 673–682.
  12. B. Wu, E. Wang, Z. Zhu, W. Chen, and P. Xiao, “Manifold NMF with L21 norm for clustering,” Neurocomputing, vol. 273, pp. 78–88, Jan. 2018.
  13. R. He, W.-S. Zheng, T. Tan, and Z. Sun, “Half-quadratic-based iterative minimization for robust sparse representation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 2, pp. 261–275, May 2014.
  14. T. Jin, R. Ji, Y. Gao, X. Sun, X. Zhao, and D. Tao, “Correntropy-induced robust low-rank hypergraph,” IEEE Transactions on Image Processing, vol. 28, no. 6, pp. 2755–2769, Jun. 2019.
  15. H. J. Yang and C. B. Wei, “Robust data-aware sensing based on half quadratic hashing for the Visual Internet of Things,” IEEE Internet of Things Journal, vol. 10, no. 5, pp. 3890–3900, Mar. 2023.
  16. A. Liutkus, D. Fitzgerald, and R. Badeau, “Cauchy nonnegative matrix factorization,” in Proc. 2015 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, New York, United States, 2015, Oct. 18–21.
  17. A. Zach and G. Bourmaud, “Multiplicative vs. additive half-quadratic minimization for robust cost optimization,” in Proc. 28th British Machine Vision Conference, Newcastle, United Kingdom, 2018, Sep. 03–06.
  18. Y. Zhang, Z. Sun, R. He, and T. Tan, “Robust low-rank representation via correntropy,” in Proc. 2013 2nd IAPR Asian Conference on Pattern Recognition, Naha, Japan, 2013, Nov. 05–08, pp. 461–465.
  19. X. Fu, K. Huang, N. D. Sidiropoulos, and W.-K. Ma, “Nonnegative matrix factorization for signal and data analytics: Identifiability, algorithms, and applications,” IEEE Signal Processing Magazine, vol. 36, no. 2, pp. 59–80, Mar. 2019.
  20. P. O. Hoyer, “Non-negative matrix factorization with sparseness constraints,” Journal of Machine Learning Research, vol. 5, pp. 1457–1469, Dec. 2004.
  21. X. Dai, X. Su, W. Zhang, F. Xue, and H. Li, “Robust Manhattan non-negative matrix factorization for image recovery and representation,” Information Sciences, vol. 527, pp. 70–87, Jul. 2020.
  22. S. Choi, “Algorithms for orthogonal nonnegative matrix factorization,” in Proc. 2008 IEEE International Joint Conference on Neural Networks, Hong Kong, China, 2008, Jun. 01–08, pp. 1828–1832.
  23. Y. Li and A. Ngom, “Versatile sparse matrix factorization: Theory and applications,” Neurocomputing, vol. 145, no. 5, pp. 23–29, Dec. 2014.
  24. S. Yang, C. Hou, C. Zhang, Y. Wu, and S. Weng, “Robust non-negative matrix factorization via joint sparse and graph regularization,” in Proc. 2013 International Joint Conference on Neural Networks, Dallas, Texas, United States, 2013, Aug. 04–09, pp. 1–5
  25. Z. Yang and E. Oja, “Linear and nonlinear projective nonnegative matrix factorization,” IEEE Transactions on Neural Networks, vol. 21, no. 5, pp. 734–749, May 2010.
  26. Z. Yuan and E. Oja, “Projective nonnegative matrix factorization for image compression and feature extraction,” in Proc. Scandinavian Conference on Image Analysis, Joensuu, Finland, 2005, Jun. 19–22, pp. 333–342.
  27. N. Gillis and F. Glineur, “Accelerated multiplicative updates and hierarchical ALS algorithms for nonnegative matrix factorization,” Neural Computation, vol. 24, no. 4, pp. 1085–1105, Apr. 2012.
  28. A. Vandaele, N. Gillis, Q. Lei, K. Zhong, and I. Dhillon, “Coordinate descent methods for symmetric nonnegative matrix factorization,” IEEE Transactions on Signal Processing, vol. 64, no. 21, pp. 5571–5584, May 2016.
  29. H. Kim and H. Park, “Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method,” SIAM Journal on Matrix Analysis and Applications, vol. 30, no. 2, pp. 713–730, 2008.
  30. J. Kim and H. Park, “Toward faster nonnegative matrix factorization: A new algorithm and comparisons,” in Proc. 8th IEEE International Conference on Data Mining, Pisa, Italy, 2008, Dec. 15–19, pp. 353–362.
  31. C.-J. Lin, “Projected gradient methods for nonnegative matrix factorization,” Neural Computation, vol. 19, no. 10, pp. 2756–2779, Oct. 2007.
  32. N. Guan, D. Tao, Z. Luo, and B. Yuan, “Online nonnegative matrix factorization with robust stochastic approximation,” IEEE Transactions on Neural Networks and Learning Systems, vol. 23, no. 7, pp. 1087–1099, Jul. 2012.
  33. P. de Handschutter and N. Gillis, “Deep orthogonal matrix factorization as a hierarchical clustering technique,” in Proc. 29th European Signal Processing Conference, Dublin, Ireland, 2021, Aug. 23–27, pp. 1466–1470.
  34. P. de Handschutter and N. Gillis, “A consistent and flexible framework for deep matrix factorizations,” Pattern Recognition, vol. 134, Feb. 2023.
  35. P. de Handschutter, N. Gillis, and X. Siebert, “A survey on deep matrix factorizations,” Computer Science Review, vol. 42, Nov. 2021.
  36. H. Wang, W. Yang, and N. Guan, “Cauchy sparse NMF with manifold regularization: A robust method for hyperspectral unmixing,” Knowledge-Based Systems, vol. 184, Nov. 2019.
  37. P. de Handschutter, N. Gillis, A. Vandaele, and X. Siebert, “Near-convex archetypal analysis,” IEEE Signal Processing Letters, vol. 27, pp. 81–85, Dec. 2020.
  38. B.-W. Chen and K.-L. Hou, “Half quadratic dual learning for fuzzy multiconcepts of partially-observed images,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 6, no. 4, pp. 994–1007, Feb. 2022.
  39. Y. Zhou, D. Wilkinson, R. Schreiber, and R. Pan, “Large-scale parallel collaborative filtering for the Netflix prize,” in Proc. 4th International Conference on Algorithmic Applications in Management, Shanghai, China, 2008, Jun. 23–25, pp. 337–348.
  40. J. Wang, “Incomplete data analysis,” in Pattern Recognition, C. Travieso-Gonzalez, Ed. London, United Kingdom: IntechOpen, 2020.
  41. W. Ye, “Low-error data recovery based on collaborative filtering with nonlinear inequality constraints for manufacturing processes,” IEEE Transactions on Automation Science and Engineering, 2020.
  42. N. Seichepine, S. Essid, C. Févotte, and O. Cappé, “Soft nonnegative matrix co-factorization,” IEEE Transactions on Signal Processing, vol. 62, no. 22, pp. 5940–5949, Nov. 2014.
  43. N. Gillis and R. Luce, “Robust near-separable nonnegative matrix factorization using linear optimization,” Journal of Machine Learning Research, vol. 15, no. 1, pp. 1249–1280, Jan. 2014.
  44. K. Yoshii, K. Itoyama, and M. Goto, “Student's T nonnegative matrix factorization and positive semidefinite tensor factorization for single-channel audio source separation,” in Proc. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, 2016, Mar. 20–25, pp. 51–55.
  45. N. Nadisic, N. Gillis, and C. Kervazo, “Smoothed separable nonnegative matrix factorization,” Linear Algebra and its Applications, vol. 676, pp. 174–204, Nov. 2023.
  46. N. Guan, L. Lan, D. Tao, Z. Luo, and X. Yang, “Transductive nonnegative matrix factorization for semi-supervised high-performance speech separation,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, Florence, Italy, 2014, May 04–09 pp. 2534–2538.
  47. N. Guan, D. Tao, Z. Luo, and B. Yuan, “Non-negative patch alignment framework,” IEEE Transactions on Neural Networks, vol. 22, no. 8, pp. 1218–1230, Aug. 2011.