Capsule Network Regression Using Information Measures: An Application in Bitcoin Market
Subject Areas : Financial EconomicsMahsa Tavakoli 1 , Hassan Doosti 2 , Christophe Chesneau 3
1 - Independent Researcher, Mashhad, Iran
2 - macquarie university
3 - Université de Caen-LMNO, Caen, France
Keywords: Prediction, financial market, deep learning,
Abstract :
Predicting financial markets has always been one of the most challengingissues, attracting the attention of many investors and researchers. In this regard, deeplearning methods have been used a lot recently. Due to the desired results, such networks are always in development and progress. One of the networks that is beingimplemented in various fields is capsule network. The first time the classification capsule network was introduced, it was able to attract a lot of attention with its successon MNIST data 1. In such networks, as in the other ones, the parameters are obtainedby minimizing a loss function. In this paper, we first change the classification capsulenetwork to a regression capsule network by modifying the last layer of the network.Then we use different information measures such as Kullnack-Leibler, Lin-Wang andTriangular information measures as a loss function, and compare their results with wellknown models including Artificial Neural Network (ANN), Convolutional Network(CNN) and Long Short-Term Memory (LSTM) as well as common used loss functionssuch as Mean Square Error (MSE) and Mean Absolute Percentage Error (MAPE). Usingappropriate accuracy metrics, it is shown that the capsule network using triangularinformation measure is well able to predict the price of bitcoin for the medium andlong term period including 10, 90 and 180 days.
[1] Kumar, D., Meghwani, S.S., Thakur, M., Proximal support vector machine based hybrid prediction models for trend forecasting in financial markets, Journal of Computational Science, 2016, 17, P.1-13.
Doi: 10.1016/j.jocs.2016.07.006.
[2] Podsiadlo, M., Rybinski, M., Financial time series forecasting using rough sets with time-weighted rule voting, Expert Systems with Applications, 2016, 66, P.219-233.
[3] Chen, Y., and Hao, Y., A feature weighted support vector machine and K-nearest neighbor algorithm for stock market indices prediction, Expert Systems with Applications, 2017, 80, P. 340-355.
Doi: 10.1016/j.eswa.2017.02.044.
[4] Zhong, X., D. Enke, Forecasting daily stock market return using dimensionality reduction. Expert Systems with Applications, 2017,67,P. 126-139, Doi: 10.1016/j.eswa.2016.09.027.
[5] Cavalcante, R.C., Brasileiro, R. C., Souza, V. L.F., Nobrega, J.P., Oliveira, A. L.I., Computational intelligence and financial markets: A survey and future directions, Expert Systems with Applications, 2016, 55, P.194-211, Doi: 10.1016/j.eswa.2016.02.006.
[6] Pan, Y., Xiao, Z., Wang, X., Yang, D., A multiple support vector machine approach to stock index forecasting with mixed frequency sampling, Knowledge-Based Systems, 2017, 122, P.90-102.
Doi: 10.1016/j.knosys.2017.01.033.
[7] Zhang, N., Lin, A., Shang, P., Multidimensional k-nearest neighbor model based on EEMD for financial time series forecasting, Physica A: Statistical Mechanics and its Applications, 2017, 477, P. 161-173.
Doi: 10.1016/j.physa.2017.02.072.
[8] Le, H.H., Viviani, J., Predicting bank failure: An improvement by implementing a machine-learning approach to classical financial ratios, Research in International Business and Finance, 2018, 44, P.16-25.
[9] Lv, D., Huang, Z., Li, M., Xiang, Y., Selection of the optimal trading model for stock investment in different industries, PloS one, 2019, 14(2), P.e0212137. Doi: 10.1371/journal.pone.0212137.
[10] Chen, Z., Li, C., Sun, W., Bitcoin price prediction using machine learning: An approach to sample dimension engineering, Journal of Computational and Applied Mathematics, 2020, 365, P.112395.
[11] Olivas, E.S., Guerrero, J.D.M., Sober , M. M., Benedito , J, R, M., Lopez, A, J, S., Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques. 2009: IGI Global.
[12] McNally, S., Roche, J., Caton. S., Predicting the price of bitcoin using machine learning, 26th Euromicro International Conference on Parallel, Distributed and Network-based Processing, 2018.
[13] Saad, M., M. Saad, J. Choi, D. Nyang, J. Kim and A. Mohaisen,., Toward characterizing blockchain-based cryptocurrencies for highly accurate predictions, IEEE Systems Journal, 2019, 14(1), P.321-332.
Doi: 10.1109/JSYST.2019.2927707.
[14] Jang, H. and J. Lee, An empirical study on modeling and prediction of bitcoin prices with bayesian neural networks based on blockchain information, IEEE Access, 2017,6, P.5427-5437.
Doi: 10.1109/ACCESS.2017.2779181.
[15] Rebane, J., Isak, K., Papapetrou, P., Seq2seq rnns and arima models for cryptocurrency prediction: A comparative study, SIGKDD Fintech, 2018,18,P. 2-6. Doi: 10.475/123_4.
[16] Shintate, T., Pichl, L., Trend prediction classification for high frequency bitcoin time series with deep learning, Journal of Risk and Financial Management, 2019,12(1),P.17. Doi: 10.3390/jrfm12010017.
[17] Davoodi Kasbi, A., Dadashi, I., Stock price prediction using the Chaid rule-based algorithm and particle swarm optimization (pso), Advances in Mathematical Finance and Applications, 2020,5(2),P.197-213.
Doi: 10.22034/amfa.2019.585043.1184.
[18] Ghasemzadeha, M., Mohammad-Karimi, N., Ansari-Samani, H., Machine learning algorithms for time series in financial markets, Advances in Mathematical Finance and Applications, 2020,5(4), P.479-490.
[19] Hsu, M.-W., Lessmann, S., Sung, M.C., Ma, T., Johnson, J. E.V., Bridging the divide in financial market forecasting: machine learners vs. financial economists, Expert Systems with Applications, 2016,61,P.215-234. Doi: 10.1016/j.eswa.2016.05.033.
[20] Patel, J., Shah, S.,Thakkar, P.,Kotecha, K., Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques, Expert systems with applications, 2015,42(1),P.259-268.
Doi: 10.1016/j.eswa.2014.07.040.
[21] Selvin, S., Vinayakumar, R., Gopalakrishnan, E. A., Menon, V. K., Soman, K. P., Stock price prediction using LSTM, RNN and CNN-sliding window model, International conference on advances in computing, communications and informatics, 2017.
[22] Sachdeva, A., Jethwani, G., Manjunath, C., Balamurugan, M., Krishna, A.V.N., An Effective Time Series Analysis for Equity Market Prediction Using Deep Learning Model, International Conference on Data Science and Communication (IconDSC), 2019.
Doi: 10.1109/IconDSC.2019.8817035.
[23] Liu, J., Lin, H., Liu, X., Xu, B., Ren, Y., Diao, Y., Yang, L., Transformer-based capsule network for stock movement prediction. in Proceedings of the First Workshop on Financial Technology and Natural Language Processing. 2019.
[24] Goodfellow, I., et al., Deep learning. Vol. 1. 2016,MIT press Cambridge.
[25] Fischer, T., C. Krauss, and A. Treichel, Machine learning for time series forecasting-a simulation study. 2018, FAU Discussion Papers in Economics.
[26] Korczak, J., Hemes. M., Deep learning for financial time series forecasting in A-Trader system. Federated Conference on Computer Science and Information Systems (FedCSIS). 2017.
[27] Zhou, X., Pan, Z., Hu, G., Tang, S., and Zhao, C., Stock market prediction on high-frequency data using generative adversarial nets, Mathematical Problems in Engineering, 2018,(1),P.1-11, Doi: 10.1155/2018/4907423.
[28] Fischer, T., Krauss, C., Deep learning with long short-term memory networks for financial market predictions, European Journal of Operational Research, 2018,270(2),P. 654-669, Doi:10.1016/j.ejor.2017.11.054.
[29] Gonçalves, R., Ribeiro, V. M., Pereira, F. L.,and Rocha, A. P., Deep learning in exchange markets. Information Economics and Policy, 2019. 47, P.38-51, Doi: 10.1016/j.infoecopol.2019.05.002.
[30] Tavakoli, M., Doosti, H., Forecasting the Tehran Stock market by Machine Learning Methods using a New Loss Function. Advances in Mathematical Finance and Applications, 2021,6(2),P.1-24.
Doi: 10.22034/amfa.2020.1896273.1399.
[31] Long, J., et al., An integrated framework of deep learning and knowledge graph for prediction of stock price trend: An application in Chinese stock exchange market, Applied Soft Computing, 2020,P.106205.
Doi: 10.1016/j.asoc.2020.106205.
[32] Kelotra, A., Pandey, P., Stock market prediction using optimized deep-convlstm model. Big Data, 2020, 8(1),P.5-24. Doi: 10.1089/big.2018.0143.
[33] Bouktif, S., Fiaz, A., Awad, M., Augmented Textual Features-Based Stock Market Prediction, IEEE, 2020,8, P.40269-40282. Doi: 10.1109/ACCESS.2020.2976725.
[34] Chung, H., Shin, K., Genetic algorithm-optimized multi-channel convolutional neural network for stock market prediction, Neural Computing and Applications, 2020, 32(12), P.7897-7914. Doi: 10.1007/s00521-019-04236-3.
[35] Cervelló-Royo, R., Guijarro, F., Forecasting stock market trend: A comparison of machine learning algorithms, Finance, Markets and Valuation, 2020, 6(1), P.37-49.
[36] De Souza, M.J.S., Almudhaf, F.W., Henriqu, B,M., Negredo, A,B,S., Ramos, D,G,F., Sobreiro, V. A., Kimura, H., Can artificial intelligence enhance the Bitcoin bonanza, The Journal of Finance and Data Science, 2019, 5(2), P.83-98. Doi: 10.1016/j.jfds.2019.01.002.
[37] Ji, S., Kim, J., Im, H., A comparative study of Bitcoin price prediction using deep learning, Mathematics, 2019, 7(10), P.898. Doi: 10.3390/math7100898.
[38] Huisu, J., Predicting bitcoin prices by using rolling window LSTM model, Data Science in FinTech (DSF), 2018.
[39] Kim, Y.B., Kim, J.G., Kim, W., Im, J, H., Kim, T.H., Kang, Sh.J., Chang Hun Kim., Predicting fluctuations in cryptocurrency transactions based on user comments and replies, PloS one, 2016, 11(8), P.e0161197.
Doi: 10.1371/journal.pone.0161197.
[40] Li, T.R., T.R., Li, Chamrajnagar, A.S., Fong, X.R., Rizik, N. R., Fu. F.,, Sentiment-based prediction of alternative cryptocurrency price fluctuations using gradient boosting tree model, Frontiers in Physics, 2019, 7, P.98. Doi: 10.3389/fphy.2019.00098.
[41] Banerjee, A., Merugu, S., Dhillon, I. S., Ghosh, J., Clustering with Bregman divergences, Journal of machine learning research, 2005, 6(Oct), P.1705-1749. Doi: 10.1137/1.9781611972740.22.
[42] Villmann, T., Haase, S., Mathematical aspects of divergence based vector quantization using Fréchet-derivatives, University of Applied Sciences Mittweida, 2010.
[43] Villmann, T., Haase, S., Schleif, F.M., Hammer, B., Divergence based online learning in vector quantization, International Conference on Artificial Intelligence and Soft Computing, 2010, Springer.
[44] Cichocki, A., Zdunek, R., Choi, S., Plemmons, R., Amari, Sh,-i.,Novel multi-layer non-negative tensor factorization with sparsity constraints, International Conference on Adaptive and Natural Computing Algorithms, 2007, Springer.
[45] Sabour, S., Frosst, N., Hinton. G.E., Dynamic routing between capsules, Advances in neural information processing systems, 2017.
[46] Gandhmal, D.P., Kumar, K., Systematic analysis and review of stock market prediction techniques, Computer Science Review, 2019, 34, P.100190. Doi: 10.1016/j.cosrev.2019.08.001.
[47] Tilouche, S., Nia, V.P., Bassetto, S., Parallel coordinate order for high-dimensional data, arXiv preprint arXiv,1905.10035, 2019.
[48] Tavakoli, M., Alizadeh Noughabi, H., Mohtashami, G.R., An estimation of Phi divergence and its application in testing normality, Hacettepe Journal of Mathematics and Statistics, 2021, 6(2), P.195-205.
Doi: 10.15672/hujms.629192.
[49] Powers, D.M., Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv preprint arXiv:2010.16061, 2020.