MAXIMIZATION OF ENTROPY METHOD FOR PREDICTING THE BEHAVIOR OF COMPLEX SYSTEMS UNDER NOISE CONDITIONS

  • D. I. Symonov Department of mathematical problems of applied informatics, V.M. Glushkov Institute of Cybernetics of the National Academy of Sciences of Ukraine, Kyiv, Ukraine https://orcid.org/0000-0002-6648-4736
Keywords: maximization of entropy, forecasting, machine learning, behavior of complex systems, noise in data

Abstract

The article addresses the problem of predicting the behavior of complex systems in the presence of random noise disturbances. The relevance of this research is driven by the limitations of traditional approaches, which often lose accuracy under conditions of uncertainty and noise. The proposed approach is based on the method of maximum entropy, which allows for the preservation of information content and adaptation to unpredictable changes in the data. The application of this method ensures optimal consistency between the model and empirical observations, even with limited or incomplete data. The study presents an algorithm for iterative parameter optimization using Lagrange multipliers and gradient descent. Particular attention is given to accounting for the mean value of the noise, which enhances the robustness and accuracy of the predictions. The practical section demonstrates the viability of the approach using a system with noisy measurements. The results demonstrate the effectiveness of the maximum entropy method for forecasting in various fields, including financial modeling and engineering process management.

References

Symonov D. I. Entropy method as a tool for optimization of complex systems. Journal of Numerical and Applied Mathematics. 2024. No 1. C. 49–58. https://doi.org/10.17721/2706-9699.2024.1.04

Zakkour A., Perret C., Slaoui Y. Stochastic Expectation Maximization Algorithm for Linear Mixed-Effects Model with Interactions in the Presence of Incomplete Data. Entropy. 2023. 25(3), 473. https://doi.org/10.3390/e25030473

Ciuperca G., Girardin V. Estimation of the Entropy Rate of a Countable Markov Chain. Communications in Statistics – Theory and Methods. 2007. 36(14). P. 2543–2557. https://doi.org/10.1080/03610920701270964

Symonov D. I., Zaika B. Y. Modeling the management of complex information multicomponent systems. Scientific Bulletin of Uzhhorod University: series: Mathematics and Informatics. 2024. 44(1). C. 168–174. https://doi.org/10.24144/2616-7700.2024.44(1).168-174

Bisikalo O. V., Kharchenko V., Kovtun V. V., Krak I. V., Pavlov S. Parameterization of the Stochastic Model for Evaluating Variable Small Data in the Shannon Entropy Basis. Entropy. 2023. 25(2), 184. https://doi.org/10.3390/e25020184

Parkash O., Mukesh. Contribution of maximum entropy principle in the field of queueing theory. Communications in Statistics – Theory and Methods. 2016.45(12). P. 3464–3472. https://doi.org/10.1080/03610926.2013.875574

Bounkhel M., Tadj L., Hedjar R. Entropy maximization for the busy period of asingle server queue. Communications in Statistics – Theory and Methods. 2023. 52. P. 8555–8565. https://doi.org/10.1080/03610926.2022.2065301

Girardin V., Limnios N. Entropy Rate and Maximum Entropy Methods for Countable Semi-Markov Chains. Communications in Statistics – Theory and Methods. 2004. 33(3). P. 609–622. https://doi.org/10.1081/STA-120028687

Symonov D., Symonov Y. Methods for selecting models of functioning of multicomponent information and environmental systems. Scientific Journal "Mathematical Modeling". 2024. Vol. 1, No 50. P. 57–63. https://doi.org/10.31319/2519-8106.1(50)2024.304943

Genshiro K. Information criteria for the predictive evaluation of bayesian models. Communications in Statistics – Theory and Methods. 1997. 26(9). P. 2223–2246. https://doi.org/10.1080/03610929708832043

Gzyl H., Inverardi P. L. N., Tagliani A. Fractional Moments and Maximum Entropy: Geometric Meaning. Communications in Statistics – Theory and Methods. 2014. 43(17). P. 3596–3601.

Mao A., Mohri M., Zhong Y. Cross-Entropy Loss Functions: Theoretical Analysis and Applications. ArXiv. 2023. Abs/2304.07288. https://doi.org/10.48550/arXiv.2304.07288

Published
2025-01-14
How to Cite
Symonov, D. (2025). MAXIMIZATION OF ENTROPY METHOD FOR PREDICTING THE BEHAVIOR OF COMPLEX SYSTEMS UNDER NOISE CONDITIONS. Journal of Numerical and Applied Mathematics, (2), 52-61. https://doi.org/10.17721/2706-9699.2024.2.03