Abstract
The task of recognizing symbols poses a significant challenge owing to the wide variability in human handwriting. Complexity in terms of the structural representation of symbols used in physics expressions is a major challenge in the recognition process The emergence of online handwriting, fueled by the widespread adoption of handheld digital devices, particularly in educational contexts, highlights the critical importance of precise symbol recognition, especially in the teaching and learning process. In contemporary literature, there is a notable emphasis on LaTex sequencing, symbol recognition and parsing. However, deep learning continues to yield promising results in this domain. The convenience of user input provides benefits to e-learning applications. In this study, we propose three approaches for the recognition of physics symbols within physics expressions (1) A proposed Java user interface for taking input from the user, as convenience of user input provides benefits to e-learning applications. (2) Contour-based bounding box segmentation algorithm, which deals with broken symbols within physics expressions. (3) For recognition, we propose a Convolution Neural Network-K-Nearest Neighbor (CNN-KNN) recognition model, as CNN plays an important role in extracting features, which are further provided as input to the K-NN classifier using the dropout method. Combining these three approaches into a symbol recognition model provides state-of-arts results. Handwritten physics symbols were collected from 20 different writers and each writer has written 5 types of physics expressions under different categories like electric flux, Maxwell’s equations, inductance and pointing vector and moment of Interia. There were 25 classes identified from the 780 samples collected from the users. The recognition rate is identified using (1) Using CNN model, which shows an accuracy of 91.48 and (2) Using the proposed hybrid CNN-KNN model the accuracy reported is 98.06.
References
Artieres, T., Marukatat, S., & Gallinari, P. (2007). Online Handwritten Shape Recognition Using Segmental Hidden Markov Models.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
29(2), 205–217.
https://doi.org/10.1109/tpami.2007.38Baró, A., Riba, P., Calvo-Zaragoza, J., & Fornés, A. (2019). From Optical Music Recognition to Handwritten Music Recognition: A baseline.
Pattern Recognition Letters,
123, 1–8.
https://doi.org/10.1016/j.patrec.2019.02.029Bresler, M., Prua, D., & Hlavác, V. (2013). Modeling Flowchart Structure Recognition as a Max-Sum Problem.
2013 12th International Conference on Document Analysis and Recognition, 1215–1219.
https://doi.org/10.1109/icdar.2013.246Coquenet, D., Chatelain, C., & Paquet, T. (2023). End-to-End Handwritten Paragraph Text Recognition Using a Vertical Attention Network.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
45(1), 508–524.
https://doi.org/10.1109/tpami.2022.3144899Huang, B., Du, C., Zhang, Y., & Kechadi, M. T. (2006). A Hybrid HMM-SVM Method for Online Handwriting Symbol Recognition.
Sixth International Conference on Intelligent Systems Design and Applications, 887–891.
https://doi.org/10.1109/isda.2006.61Jeon, H.-K., & Yang, C.-S. (2021). Enhancement of Ship Type Classification from a Combination of CNN and KNN.
Electronics,
10(10), 1169.
https://doi.org/10.3390/electronics10101169Keshari, B., & Watt, S. (2007). Hybrid Mathematical Symbol Recognition Using Support Vector Machines.
Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), 859–863.
https://doi.org/10.1109/icdar.2007.4377037Kherdekar, V. A., Naik, S. A., & Bafna, P. (2023). A Segmentation-based Token Identification for Recognition of Audio Mathematical Expression.
International Journal of Advanced Computer Science and Applications,
14(7), 298–303.
https://doi.org/10.14569/ijacsa.2023.0140733Lai, S.-H., Lepetit, V., Nishino, K., & Sato, Y. (Eds.). (2017).
Computer Vision – ACCV 2016 (Vol. 10112). Springer Cham.
https://doi.org/10.1007/978-3-319-54184-6Le, A. D., Nguyen, H. D., Indurkhya, B., & Nakagawa, M. (2019). Stroke Order Normalization for Improving Recognition of Online Handwritten Mathematical Expressions.
International Journal on Document Analysis and Recognition (IJDAR),
22(1), 29–39.
https://doi.org/10.1007/s10032-019-00315-2Li, Z., Wang, X., Liu, Y., Jin, L., Huang, Y., & Ding, K. (2024). Improving Handwritten Mathematical Expression Recognition via Similar Symbol Distinguishing.
IEEE Transactions on Multimedia,
26, 90–102.
https://doi.org/10.1109/tmm.2023.3260648Long, J., Shelhamer, E., & Darrell, T. (2015). Fully Convolutional Networks for Semantic Segmentation.
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3431–3440.
https://doi.org/10.1109/cvpr.2015.7298965Michael, J., Labahn, R., Grüning, T., & Zöllner, J. (2019). Evaluating Sequence-to-Sequence Models for Handwritten Text Recognition.
2019 International Conference on Document Analysis and Recognition (ICDAR), 1286–1293.
https://doi.org/10.1109/icdar.2019.00208Miyao, H., & Maruyama, M. (2007). An Online Handwritten Music Symbol Recognition System.
International Journal of Document Analysis and Recognition (IJDAR),
9(1), 49–58.
https://doi.org/10.1007/s10032-006-0026-9Nguyen, C. T., Truong, T.-N., Nguyen, H. T., & Nakagawa, M. (2021). Global Context for Improving Recognition of Online Handwritten Mathematical Expressions. In J. Lladós, D. Lopresti, & S. Uchida (Eds.),
Document Analysis and Recognition – ICDAR 2021 (Vol. 12822, pp. 617–631). Springer, Cham.
https://doi.org/10.1007/978-3-030-86331-9_40Ptucha, R., Petroski Such, F., Pillai, S., Brockler, F., Singh, V., & Hutkowski, P. (2019). Intelligent Character Recognition Using Fully Convolutional Neural Networks.
Pattern Recognition,
88, 604–613.
https://doi.org/10.1016/j.patcog.2018.12.017Putri, D. U. K., Pratomo, D. N., & Azhari, A. (2023). Hybrid Convolutional Neural Networks-Support Vector Machine Classifier with Dropout for Javanese Character Recognition.
TELKOMNIKA (Telecommunication Computing Electronics and Control),
21(2), 346–353.
https://doi.org/10.12928/telkomnika.v21i2.24266Qiu, Q., Tan, Y., Ma, K., Tian, M., Xie, Z., & Tao, L. (2023). Geological Symbol Recognition on Geological Map Using Convolutional Recurrent Neural Network with Augmented Data.
Ore Geology Reviews,
153, 105262.
https://doi.org/10.1016/j.oregeorev.2022.105262Sanida, T., Sideris, A., Tsiktsiris, D., & Dasygenis, M. (2022). Lightweight Neural Network for COVID-19 Detection from Chest X-ray Images Implemented on an Embedded System.
Technologies,
10(2), 37.
https://doi.org/10.3390/technologies10020037Sejuti, Z. A., & Islam, M. S. (2023). A Hybrid CNN–KNN Approach for Identification of COVID-19 with 5-Fold Cross Validation.
Sensors International,
4, 100229.
https://doi.org/10.1016/j.sintl.2023.100229Shuvo, S. N., Hasan, F., Ahmed, M. U., Hossain, S. A., & Abujar, S. (2021). MathNET: Using CNN Bangla Handwritten Digit, Mathematical Symbols, and Trigonometric Function Recognition. In S. Borah, R. Pradhan, N. Dey, & P. Gupta (Eds.),
Soft Computing Techniques and Applications (Vol. 1248, pp. 515–523). Springer, Singapore.
https://doi.org/10.1007/978-981-15-7394-1_47Smith, W. A. P., & Pillatt, T. (2023). You Only Look for a Symbol Once: An Object Detector for Symbols and Regions in Documents. In G. A. Fink, R. Jain, K. Kise, & R. Zanibbi (Eds.),
Document Analysis and Recognition - ICDAR 2023 (Vol. 14191, pp. 227–243). Springer, Cham.
https://doi.org/10.1007/978-3-031-41734-4_14Sundaram, S., & Ramakrishnan, A. G. (2014). Performance enhancement of online handwritten Tamil symbol recognition with reevaluation techniques.
Pattern Analysis and Applications,
17(3), 587–609.
https://doi.org/10.1007/s10044-013-0353-7Tang, P., Hui, S. C., & Fu, C.-W. (2013). Online chemical symbol recognition for handwritten chemical expression recognition.
2013 IEEE/ACIS 12th International Conference on Computer and Information Science (ICIS), 535–540.
https://doi.org/10.1109/icis.2013.6607894Urala, K. B., Ramakrishnan, A. G., & Mohamed, S. (2014). Recognition of open vocabulary, online handwritten pages in Tamil script.
2014 International Conference on Signal Processing and Communications (SPCOM), 1–6.
https://doi.org/10.1109/spcom.2014.6984002Xin, G., Cuiyun, L., Jihong, P., & Weixin, X. (2003). HMM based online hand-drawn graphic symbol recognition.
6th International Conference on Signal Processing, 2002, 1067–1070.
https://doi.org/10.1109/icosp.2002.1179973Zhu, R., & Wang, Y. (2012). Application of Improved Median Filter on Image Processing.
Journal of Computers,
7(4), 838–841.
https://doi.org/10.4304/jcp.7.4.838-841
Copyright: © 2024 Ujwala Kolte, Sachin Naik and Vidya Kumbhar. This is an open access article distributed under the terms of the
Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.