Journal of Soft Computing Paradigm
IRO Journals
Volume 2 — Issue 3 — September 2020

Survey on Neural Network Architectures with Deep Learning

https://doi.org/10.36548/jscp.2020.3.006
S. Smys
Professor, Department of CSE, RVS Technical Campus, Coimbatore, India
Joy Iong Zong Chen
Professor, Department of Electrical Engineering, Da-Yeh University, Taiwan
Subarna Shakya
Professor, Department of Electronics and Computer Engineering, Central Campus, Institute of Engineering, Pulchowk, Tribhuvan University, Pulchowk, Lalitpur, Nepal
PDF
PDF

How to Cite

Smys, S., Joy Iong Zong Chen, and Subarna Shakya. 2020. “Survey on Neural Network Architectures With Deep Learning”. Journal of Soft Computing Paradigm 2 (3): 186-94. https://doi.org/10.36548/jscp.2020.3.006.
ACM ACS APA ABNT Chicago Harvard IEEE MLA Turabian Vancouver AMA

Download Citation

Endnote/Zotero/Mendeley (RIS) BibTeX

Keywords

— Deep Learning
— Architectures
— Algorithms
Published: 30-07-2020

Abstract

In the present research era, machine learning is an important and unavoidable zone where it provides better solutions to various domains. In particular deep learning is one of the cost efficient, effective supervised learning model, which can be applied to various complicated issues. Since deep learning has various illustrative features and it doesn't depend on any limited learning methods which helps to obtain better solutions. As deep learning has significant performance and advancements it is widely used in various applications like image classification, face recognition, visual recognition, language processing, speech recognition, object detection and various science, business analysis, etc., This survey work mainly provides an insight about deep learning through an intensive analysis of deep learning architectures and its characteristics along with its limitations. Also, this research work analyses recent trends in deep learning through various literatures to explore the present evolution in deep learning models.

References

  1. Alessio Micheli, Frank-Michael Schleif, Peter Tiňo (2013). Novel approaches in machine learning and computational intelligence. Neurocomputing. 112,1-3.
  2. Iti Chaturvedi, Yew-Soon Ong, Rajesh Vellore Arumugam (2015). Deep transfer learning for classification of time-delayed Gaussian networks. Signal Processing. 110, 250-262.
  3. Daniel Michelsanti, Zheng-Hua Tan, Sigurdur Sigurdsson, Jesper Jensen (2019). Deep-learning-based audio-visual speech enhancement in presence of Lombard effect. Speech Communication. 115,38-50.
  4. Nianyin Zeng, Hong Zhang, Baoye Song, Weibo Liu, Abdullah M. Dobaie (2018). Facial expression recognition via learning deep sparse autoencoders. Neurocomputing, 273, 643-649
  5. Komal Bajaj, Dushyant Kumar Singh, Mohd. Aquib Ansari (2020). Autoencoders Based Deep Learner for Image Denoising. Procedia Computer Science. 171, 1535-1541.
  6. Jianbo Yu, Xiaoyun Zheng, Shijin Wang (2019). A deep autoencoder feature learning method for process pattern recognition. Journal of Process Control. 79, 1-15
  7. Hung-Ping Liu, Yu Tsao, Chiou-Shann Fuh (2018). Bone-conducted speech enhancement using deep denoising autoencoder. Speech Communication. 104, 106-112
  8. Haidong Shao, Hongkai Jiang, Huiwei Zhao, Fuan Wang (2017). A novel deep autoencoder feature learning method for rotating machinery fault diagnosis. Mechanical Systems and Signal Processing. 95, 187-204
  9. Abdul Mujeeb, Wenting Dai, Marius Erdt, Alexei Sourin (2019). One class-based feature learning approach for defect detection using deep autoencoders. Advanced Engineering Informatics. 42, 1-15
  10. Kannadasan, Damodar Reddy Edla, Venkatanareshbabu Kuppili (2019). Type 2 diabetes data classification using stacked autoencoders in deep neural networks. Clinical Epidemiology and Global Health. 7(4),530-535
  11. Ozal Yildirim, Ru San Tan, U. Rajendra Acharya (2018). An efficient compression of ECG signals using deep convolutional autoencoders. Cognitive Systems Research. 52, 198-211
  12. Niklas Renström, Pramod Bangalore, Edmund Highcock (2020), System-wide anomaly detection in wind turbines using deep autoencoders, Renewable Energy, 157, 647-659.
  13. Aboozar Taherkhani, Georgina Cosma, T. M McGinnity (2018). Deep-FS: A feature selection algorithm for Deep Boltzmann Machines. Neurocomputing. 322, 22-37.
  14. Guangzheng Hu, Huifang Li, Yuanqing Xia, Lixuan Luo (2018). A deep Boltzmann machine and multi-grained scanning forest ensemble collaborative method and its application to industrial fault diagnosis. Computers in Industry. 100, 287-296.
  15. Yifeng Li, François Fauteux, Jinfeng Zou, André Nantel, Youlian Pan (2019). Personalized prediction of genes with tumor-causing somatic mutations based on multi-modal deep Boltzmann machine. Neurocomputing. 324, 51-62.
  16. Jian Wu, Thomas R. Mazur, Su Ruan, Chunfeng Lian, Hua Li (2018). A deep Boltzmann machine-driven level set method for heart motion tracking using cine MRI images. Medical Image Analysis. 47, 68-80.
  17. Takashi Kuremoto, Shinsuke Kimura, Kunikazu Kobayashi, Masanao Obayashi (2014). Time series forecasting using a deep belief network with restricted Boltzmann machines. Neurocomputing. 137, 47-56.
  18. Huihui Shen, Hongwei Li (2019). A gradient approximation algorithm based weight momentum for restricted Boltzmann machine. Neurocomputing. 361,40-49.
  19. Andre G. C. Pacheco, Renato A. Krohling, Carlos A. S. da Silva (2018). Restricted Boltzmann machine to determine the input weights for extreme learning machines. Expert Systems with Applications. 961, 77-85.
  20. Zehuan Yuan, Hao Wang, Limin Wang, Tong Lu, Chew Lim Tan (2016). Modeling spatial layout for scene image understanding via a novel multiscale sum-product network. Expert Systems with Applications. 63,231-240.
  21. Frank Rudzicz, Arvid Frydenlund, Sean Robertson, Patricia Thaine (2016). Acoustic-articulatory relationships and inversion in sum-product and deep-belief networks. Speech Communication. 79, 61-73.
  22. Jingqiang Chen, Hai Zhuge (2019). Extractive summarization of documents with images based on multi-modal RNN. Future Generation Computer Systems. 99, 186-196.
  23. Rajib Ghosh, Chirumavila Vamshi, Prabhat Kumar (2019). RNN based online handwritten word recognition in Devanagari and Bengali scripts using horizontal zoning. Pattern Recognition. 92.203-218.
  24. Haiqing Ren, Weiqiang Wang, Chenglin Liu (2019). Recognizing online handwritten Chinese characters using RNNs with new computing architectures. Pattern Recognition. 93, 179-192.
  25. Ruoxuan Cui, Manhua Liu, the Alzheimer's Disease Neuroimaging Initiative (2019). RNN-based longitudinal analysis for diagnosis of Alzheimer’s disease. Computerized Medical Imaging and Graphics. 73, 1-10
  26. Jinyong Wang, Ce Zhang (2018). Software reliability prediction using a deep learning model based on the RNN encoder–decoder. Reliability Engineering & System Safety. 170, 73-82
  27. Routraya, P. Kanungo (2012). Genetic Algorithm Based RNN Structure for Rayleigh Fading MIMO Channel Estimation. Procedia Engineering. 30, 77-84.
  28. Chen Wang, Guohua Peng, Bernard De Baets (2020). Deep feature fusion through adaptive discriminative metric learning for scene recognition, Information Fusion, 63, 1-12.
  29. Yujuan Ding, Wai Keung Wong, Zhihui Lai, Zheng Zhang (2020). Discriminative dual-stream deep hashing for large-scale image retrieval. Information Processing & Management. 57(6), 1-15
  30. Monica M. Y. Zhang, Kun Shang, Huaming Wu (2019). Deep compact discriminative representation for unconstrained face recognition. Signal Processing: Image Communication. 75, 118-127.
  31. Saad Shakeel, Kin-Man Lam (2019). Deep-feature encoding-based discriminative model for age-invariant face recognition. Pattern Recognition, 93, 442-457
  32. Haibo Zhang (2018). Geometric discriminative deep features for traffic image analysis. Journal of Visual Communication and Image Representation. 57, 163-171
  33. Yan Liu, Shusen Zhou, Qingcai Chen (2011). Discriminative deep belief networks for visual data classification. Pattern Recognition. 44(10),2287-2296
  34. Mansoor Iqbal, M. Shujah Islam Sameem, Nuzhat Naqvi, Shamsa Kanwal, Zhongfu Ye (2019). A deep learning approach for face recognition based on angularly discriminative features. Pattern Recognition Letters. 128, 414-419
  35. Haijun Liu, Jian Cheng, Wen Wang, Yanzhou Su, Haiwei Bai (2020). Enhancing the discriminative feature learning for visible-thermal cross-modality person re-identification. Neurocomputing. 398, 11-19
  36. Rui Huang, Wei Feng, Mingyuan Fan, Liang Wan, Jizhou Sun (2018). Multiscale blur detection by learning discriminative deep features. Neurocomputing, 285, 154-166.
  37. Yisen Liu, Songbin Zhou, Wei Han, Weixin Liu, Chang Li (2019). Convolutional neural network for hyperspectral data analysis and effective wavelengths selection. Analytica Chimica Acta. 1086, 46-54.
  38. Abderrahim Mesbah, Aissam Berrahou, Hicham Hammouchi, Hassan Berbia, Mohamed Daoudi (2019). Lip reading with Hahn Convolutional Neural Networks. Image and Vision Computing. 88, 76-83.
  39. Jie Shao, Yongsheng Qian (2019). Three convolutional neural network models for facial expression recognition in the wild. Neurocomputing. 355, 82-92.
  40. Sérgio F. Chevtchenko, Rafaella F. Vale, Valmir Macario, Filipe R. Cordeiro (2018). A convolutional neural network with feature fusion for real-time hand posture recognition. Applied Soft Computing. 73, 748-766.
  41. Måns Larsson, Yuhang Zhang, Fredrik Kahl (2018). Robust abdominal organ segmentation using regional convolutional neural networks. Applied Soft Computing. 70, 465-471.
  42. Ima Essiet, Yanxia Sun, Zenghui Wang (2019). Big data analysis for gas sensor using convolutional neural network and ensemble of evolutionary algorithms. Procedia Manufacturing. 35, 629-634.
  43. Jonathan Gordon, José Miguel Hernández-Lobato (2020). Combining deep generative and discriminative models for Bayesian semi-supervised learning. Pattern Recognition. 100, 1-13
  44. Mourad Gridach (2020). Hybrid deep neural networks for recommender systems. Neurocomputing. 413, 23-30.
  45. Ivano Lauriola, Claudio Gallicchio, Fabio Aiolli (2020). Enhancing deep neural networks via multiple kernel learning. Pattern Recognition. 101, 1-12.
  46. Kejun Wang, Xiaoxia Qi, Hongda Liu, Jiakang Song (2018). Deep belief network based k-means cluster approach for short-term wind power forecasting. Energy, 165, 840-852.
  47. Mingqi Lv, Wei Xu, Tieming Chen (2019). A hybrid deep convolutional and recurrent neural network for complex activity recognition using multimodal sensors. Neurocomputing. 362, 33-40.
  48. Muhammad Alam, Fazeel Abid, Cong Guangpei, L. V. Yunrong (2020). Social media sentiment analysis through parallel dilated convolutional neural network for smart city applications. Computer Communications. 154, 129-137.
  49. Charissa Ann Ronao, Sung-Bae Cho (2016). Human activity recognition with smartphone sensors using deep learning neural networks. Expert Systems with Applications. 59, 235-244.
  50. Konstantinos Charalampous, Antonios Gasteratos (2014). A tensor-based deep learning framework. Image and Vision Computing. 32(1), 916-929
  51. Manoharan, Samuel (2020). Patient Diet Recommendation System Using K Clique and Deep learning Classifiers Journal of Artificial Intelligence. 2(2), 121-130.
  52. Muthukumar, Vignesh, N. Bhalaji (2020).MOOCVERSITY-Deep Learning Based Dropout Prediction in MOOCs over Weeks. Journal of Soft Computing Paradigm (JSCP). 2(3), 140-152.
  53. Vijayakumar, Vinothkanna (2020).Mellowness Detection of Dragon Fruit Using Deep Learning Strategy. Journal of Innovative Image Processing (JIIP). 2(1),35-43.