Applying a Quantum Annealing Based Restricted Boltzmann Machine for MNIST Handwritten Digit Classification
Kurowski Krzysztof 1, Slysz Mateusz 1, Subocz Marek 1, Różycki Rafał 2
1 Poznan Supercomputing and Networking Center
ul. Jana Pawła II 10
61-139 Poznan, Poland
E-mail: krzysztof.kurowski@man.poznan.pl, mslysz@man.poznan.pl, marek.subocz@man.poznan.pl2 Poznan University of Technology
Institute of Computing Science
ul. Piotrowo 2, 60-965 Poznan, Poland
E-mail: rafal.rozycki@cs.put.poznan.plThis paper was guest edited by Dr. Cezary Mazurek
Received:
Received: 30 March 2021; revised: 14 June 2021; accepted: 28 June 2021; published online: 2 July 2021
DOI: 10.12921/cmst.2021.0000011
Abstract:
As indicated in various recent research, there may still be challenges in achieving acceptable performance using quantum computers for solving practical problems. Nevertheless, we demonstrate promising results by using the recent advent of the D-Wave Advantage quantum annealer to train and test a Restricted Boltzmann Machine for the well studied MNIST dataset. We compare our new model with some tests executed on the previous D-Wave 2000Q system and show an improved image classification process with a better overall quality. In this paper we discuss how to enhance often time-consuming RBM training processes based on the commonly used Gibbs sampling using an improved version of quantum sampling. In order to prevent overfitting we propose some solutions which help to acquire less probable samples from the distribution by adjusting D-wave control and embedding parameters. Finally, we present various limitations of the existing quantum computing hardware and expected changes on the quantum hardware and software sides which can be adopted for further improvements in the field of machine learning.
Key words:
D-Wave quantum computer, machine learning, MNIST dataset, quantum annealing, RBM training
References:
[1] K. Kurowski, J. Weglarz, M. Subocz, R. Rozycki, G. Waligóra, Hybrid Quantum Annealing Heuristic Method for Solving Job Shop Scheduling Problem, [In:] Computational Science – ICCS 2020. Lecture Notes in Computer Science 12142, Eds. V.V. Krzhizhanovskaya, G. Závodszky, M.H. Lees, J.J. Dongarra, P.M.A. Sloot, S. Brissos, J. Teixeira, Springer, Cham (2020).
[2] J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe, S. Lloyd, Quantum machine learning, Nature 549, 195–202 (2017).
[3] S.H. Adachi, P. Maxwell, Henderson Application of quantum annealing to training of deep neural networks, arXiv: 1510.06356 (2015).
[4] S. Lloyd, M. Mohseni, P. Rebentrost, Quantum algorithms for supervised and unsupervised machine learning, arXiv: 1307.0411 (2013).
[5] N. Wiebe, A. Kapoor, K.M. Svore, Quantum Deep Learning, arXiv: 1412.3489 (2015).
[6] D. Crawford, A. Levit, N. Ghadermarzy, J.S. Oberoi, P. Ronagh, Reinforcement Learning Using Quantum Boltzmann Machines, arXiv: 1612.05695 (2016).
[7] P. Smolensky, Chapter 6: Information Processing in Dynamical Systems: Foundations of Harmony Theory, [In:] Parallel Distributed Processing: Explorations in the Microstructure of Cognition 1: Foundations, Eds. D.E. Rumelhart, J.L. McLelland, MIT Press, 194–281 (1986).
[8] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-Based Learning Applied to Document Recognition, Proceedings of the IEEE 86(11), 2278–2324 (1998).
[9] M. Benedetti, J. Realpe-Gomez, A. Perdomo-Ortiz, Quantum-assisted helmholtz machines: a quantum-classical deep learning framework for industrial datasets in near-term devices, arXiv: 1708.09784 (2017).
[10] S. Ni, S. Nagayama, Performance comparison on cfrbm between gpu and quantum annealing, Technical report, Mercari (2018).
[11] Quantum annealing based RBM, https://github.com/marek subocz/QRBM.
[12] G.E. Hinton, S. Osindero, Y.W. Teh, A fast learning algorithm for deep belief nets, Neural Comput. 18(7), 1527–1554 (2006).
[13] T. Tieleman, Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient, Proceedings of the 25th international conference on Machine learning, 1064–1071 (2008).
[14] J. Brugger, Ch. Seidel, M. Streif, F. Wudarski, Ch. Dittel, A. Buchleitner, Output statistics of quantum annealers with disorder, arXiv: 1808.06817 (2021).
[15] Scikit-learn documentation, https://scikit-learn.org/stable/modules/generated/sklearn.neuralnetwork.BernoulliRBM.html.
As indicated in various recent research, there may still be challenges in achieving acceptable performance using quantum computers for solving practical problems. Nevertheless, we demonstrate promising results by using the recent advent of the D-Wave Advantage quantum annealer to train and test a Restricted Boltzmann Machine for the well studied MNIST dataset. We compare our new model with some tests executed on the previous D-Wave 2000Q system and show an improved image classification process with a better overall quality. In this paper we discuss how to enhance often time-consuming RBM training processes based on the commonly used Gibbs sampling using an improved version of quantum sampling. In order to prevent overfitting we propose some solutions which help to acquire less probable samples from the distribution by adjusting D-wave control and embedding parameters. Finally, we present various limitations of the existing quantum computing hardware and expected changes on the quantum hardware and software sides which can be adopted for further improvements in the field of machine learning.
Key words:
D-Wave quantum computer, machine learning, MNIST dataset, quantum annealing, RBM training
References:
[1] K. Kurowski, J. Weglarz, M. Subocz, R. Rozycki, G. Waligóra, Hybrid Quantum Annealing Heuristic Method for Solving Job Shop Scheduling Problem, [In:] Computational Science – ICCS 2020. Lecture Notes in Computer Science 12142, Eds. V.V. Krzhizhanovskaya, G. Závodszky, M.H. Lees, J.J. Dongarra, P.M.A. Sloot, S. Brissos, J. Teixeira, Springer, Cham (2020).
[2] J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe, S. Lloyd, Quantum machine learning, Nature 549, 195–202 (2017).
[3] S.H. Adachi, P. Maxwell, Henderson Application of quantum annealing to training of deep neural networks, arXiv: 1510.06356 (2015).
[4] S. Lloyd, M. Mohseni, P. Rebentrost, Quantum algorithms for supervised and unsupervised machine learning, arXiv: 1307.0411 (2013).
[5] N. Wiebe, A. Kapoor, K.M. Svore, Quantum Deep Learning, arXiv: 1412.3489 (2015).
[6] D. Crawford, A. Levit, N. Ghadermarzy, J.S. Oberoi, P. Ronagh, Reinforcement Learning Using Quantum Boltzmann Machines, arXiv: 1612.05695 (2016).
[7] P. Smolensky, Chapter 6: Information Processing in Dynamical Systems: Foundations of Harmony Theory, [In:] Parallel Distributed Processing: Explorations in the Microstructure of Cognition 1: Foundations, Eds. D.E. Rumelhart, J.L. McLelland, MIT Press, 194–281 (1986).
[8] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-Based Learning Applied to Document Recognition, Proceedings of the IEEE 86(11), 2278–2324 (1998).
[9] M. Benedetti, J. Realpe-Gomez, A. Perdomo-Ortiz, Quantum-assisted helmholtz machines: a quantum-classical deep learning framework for industrial datasets in near-term devices, arXiv: 1708.09784 (2017).
[10] S. Ni, S. Nagayama, Performance comparison on cfrbm between gpu and quantum annealing, Technical report, Mercari (2018).
[11] Quantum annealing based RBM, https://github.com/marek subocz/QRBM.
[12] G.E. Hinton, S. Osindero, Y.W. Teh, A fast learning algorithm for deep belief nets, Neural Comput. 18(7), 1527–1554 (2006).
[13] T. Tieleman, Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient, Proceedings of the 25th international conference on Machine learning, 1064–1071 (2008).
[14] J. Brugger, Ch. Seidel, M. Streif, F. Wudarski, Ch. Dittel, A. Buchleitner, Output statistics of quantum annealers with disorder, arXiv: 1808.06817 (2021).
[15] Scikit-learn documentation, https://scikit-learn.org/stable/modules/generated/sklearn.neuralnetwork.BernoulliRBM.html.
