A Review of Curiosity-Driven Learning Systems in Artificial Intelligence
Subject Areas : Information Technology in Engineering Design (ITED) Journal
Saeed Jamali
1
,
Saeed Setayeshi
2
*
,
Sajad Taghvaei
3
,
mohsen jahanshahi
4
1 - Department of Computer Engineering, Central Tehran Branch, Islamic Azad University, Tehran, Iran
2 - Faculty of Nuclear Engineering and Physics, Amirkabir University, Tehran, Iran
3 - Department of Mechanical Engineering, Shiraz University, Shiraz, Iran
4 - Mohsen Jahanshahi, IEEE Senior Member. Associate Prof. of Computer Engineering Department, Central Tehran Branch, Islamic Azad University.
Keywords: Curiosity, Artificial Intelligence, Machine Learning, Reinforcement Learning, Collative Variables, Space Coverage,
Abstract :
One key aspect that can elevate artificial intelligence to a higher level of capability is curiosity. Similar to humans, in artificial intelligence, curiosity can serve as a key mechanism for improving active learning and exploration in complex and unknown environments. This review paper examines efforts to model and simulate curiosity in machines in order to create systems that can automatically and independently exhibit exploratory behaviors. By investigating psychological studies on curiosity and existing computational models in artificial intelligence, this research seeks a deeper understanding of the concept of curiosity and how it can be simulated in machines. Additionally, we have examined the advantages and limitations of existing approaches. The results of this research show that curiosity can serve as an important factor in accelerating learning, increasing the generalizability of models, and improving performance in challenging tasks. Furthermore, by introducing a new metric called "space coverage," we propose new avenues for research in this type of curiosity modeling in artificial intelligence. Finally, along with enumerating some applications, we have attempted to pave the way for the development of more curious and powerful artificial intelligence systems by providing suggestions for future research.
[۱] R. Archana, P. Jeevaraj. "Deep learning models for digital image processing: a review," in Artificial Intelligence Review, vol. 57, no. 1, pp. 11, 2024.
[۲] Mehrish, A., et al. "A review of deep learning techniques for speech processing," in Information Fusion, vol. 99, pp. 101869, 2023.
[۳] M. Soori, B. Arezoo, R. Dastres. "Artificial intelligence, machine learning and deep learning in advanced robotics, a review," in Cognitive Robotics, vol. 3, pp. 54–70, 2023.
[۴] Badue, C., et al. "Self-driving cars: A survey," in Expert systems with applications, vol. 165, pp. 113816, 2021.
[۵] Yin, H., et al. "On-device recommender systems: A comprehensive survey," in arXiv preprint arXiv:2401.11441, 2024.
[۶] Stray, J., et al. "Building human values into recommender systems: An interdisciplinary synthesis," in ACM Transactions on Recommender Systems, vol. 2, no. 3, pp. 1–57, 2024.
[۷] T. Kashdan, M. Steger. "Curiosity and pathways to well-being and meaning in life: Traits, states, and everyday behaviors," in Motivation and Emotion, vol. 31, pp. 159–173, 2007.
[۸] Sun, C., Qian, H., & Miao, C. (2022). From psychological curiosity to artificial curiosity: Curiosity-driven learning in artificial intelligence tasks. arXiv preprint arXiv:2201.08300.
[۹] George Loewenstein. 1994. The psychology of curiosity: A review and reinterpretation. Psychological bulletin 116, 1 (1994), 75.
[۱۰] J. Schmidhuber. "Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts," in Connection Science, vol. 18, no. 2, pp. 173–187, 2006.
[۱۱] Wu, Q., & Miao, C. (2013). Curiosity: From psychology to computation. ACM Computing Surveys (CSUR), 46(2), 1-26.
[۱۲] P. Oudeyer, F. Kaplan. "What is intrinsic motivation? A typology of computational approaches," in Frontiers in neurorobotics, vol. 1, pp. 108, 2007.
[۱۳] D.E. Berlyne. 1960. Conflict, arousal, and curiosity. McGraw-Hill New York.
[۱۴] R. Saunders, J. Gero, "The digital clockwork muse: A computational model of aesthetic evolution," in Proceedings of the AISB, 2001, pp. 12–21.
[۱۵] A. Stein, R. Maier, J. Hähner, "Toward curious learning classifier systems: Combining xcs with active learning concepts," in Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2017, pp. 1349–1356.
[۱۶] F. Abbas, X. Niu. "One size does not fit all: Modeling users’ personal curiosity in recommender systems," in ArXivorg, 2019.
[۱۷] Schaul, T., et al, "Curiosity-driven optimization," in 2011 IEEE Congress of Evolutionary Computation (CEC), 2011, pp. 1343–1349.
[۱۸] R. Zhao, V. Tresp. "Curiosity-driven experience prioritization via density estimation," in arXiv preprint arXiv:1902.08039, 2019.
[۱۹] T. Blau, L. Ott, F. Ramos. "Bayesian curiosity for efficient exploration in reinforcement learning," in arXiv preprint arXiv:1911.08701, 2019.
[۲۰] D. Rezende, S. Mohamed, "Variational inference with normalizing flows," in International conference on machine learning, 2015, pp. 1530–1538.
[۲۱] J. Schmidhuber, "Curious model-building control systems," in Proc. international joint conference on neural networks, 1991, pp. 1458–1463.
[۲۲] P. Oudeyer, F. Kaplan, "How can we define intrinsic motivation?," in the 8th international conference on epigenetic robotics: Modeling cognitive development in robotic systems, 2008.
[۲۳] Dobrynin, D., et al. "Physical and biological mechanisms of direct plasma interaction with living tissue," in New Journal of Physics, vol. 11, no. 11, pp. 115020, 2009.
[۲۴] Jepma, M., et al. "Neural mechanisms underlying the induction and relief of perceptual curiosity," in Frontiers in behavioral neuroscience, vol. 6, pp. 5, 2012.
[۲۵] Todd B Kashdan and Paul J Silvia. 2009. Curiosity and interest: The benefits of thriving on novelty and challenge. Oxford handbook of positive psychology 2 (2009), 367–374.
[۲۶] Celeste Kidd and Benjamin Y Hayden. 2015. The psychology and neuroscience of curiosity. Neuron 88, 3 (2015), 449–460.
[۲۷] G Stanley Hall and Theodate L Smith. 1903. Curiosity and interest. The Pedagogical Seminary 10, 3 (1903), 315–358.
[۲۸] Daniel E Berlyne. 1950. Novelty and curiosity as determinants of exploratory behaviour. British Journal of Psychology 41, 1 (1950), 68.
[۲۹] Abraham Harold Maslow. 1943. A theory of human motivation. Psychological review 50, 4 (1943), 370.
[۳۰] Konrad Z Lorenz. 1981. Exploratory behavior or curiosity. In the Foundations of Ethology. Springer, 325–335.
[۳۱] Donald O Hebb. 1946. On the nature of fear. Psychological review 53, 5 (1946), 259.
[۳۲] Jean Piaget. 2003. The psychology of intelligence. Routledge.
[۳۳] Robert W White. 1959. Motivation reconsidered: The concept of competence. Psychological review 66, 5 (1959), 297.
[۳۴] Edward L Deci and Richard M Ryan. 2010. Intrinsic motivation. The corsini encyclopedia of psychology (2010), 1–2.
[۳۵] William N Dember and Robert W Earl. 1957. Analysis of exploratory, manipulatory, and curiosity behaviors. Psychological review 64, 2 (1957), 91.
[۳۶] C.D. Spielberger and L.M. Starr. 1994. Curiosity and exploratory behavior. NJ: Lawrence Erlbaum Associates, 221–243.
[۳۷] F.F. Schmitt and R. Lahroodi. 2008. The epistemic value of curiosity. Educational Theory 58, 2 (2008), 125–148.
[۳۸] Deci, E. L., & Ryan, R. M. (2000). Self-Determination Theory: Theoretical issues and practical applications. Rochester: University of Rochester Press.
[۳۹] Jirout, J. J., & Klahr, D. (2012). Children's scientific curiosity: In search of an operational definition of an elusive concept. Developmental Review, 32(2), 125-160.
[۴۰] Gottfried, A. E. (1990). Academic intrinsic motivation in young elementary school children. Journal of Educational Psychology, 82(3), 525-538.
[۴۱] Gruber, M. J., Gelman, B. D., & Ranganath, C. (2014). States of curiosity modulate hippocampus-dependent learning via the dopaminergic circuit. Neuron, 84(2), 486-496.
[۴۲] Cantor, G. N., Cantor, J. H., & Ditrichs, R. (1963). Observing behavior in preschool children as a function of stimulus complexity. Child Development, 683-689.
[۴۳] Daniel E Berlyne. 1978. Curiosity and learning. Motivation and emotion 2, 2 (1978), 97–175.
[۴۴] A. Ten, P. Oudeyer, C. Moulin-Frier. "Curiosity-driven exploration," in The Drive for Knowledge: The Science of Human Information Seeking, pp. 53, 2022.
[۴۵] Paul J Silvia. 2005. Cognitive appraisals and interest in visual art: Exploring an appraisal theory of aesthetic emotions. Empirical studies of the arts 23, 2 (2005), 119–133.
[۴۶] Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017). Curiosity-driven exploration by self-supervised prediction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017.
[۴۷] Burda, Y., Edwards, H., Storkey, A., & Klimov, O. (2018). Exploration by random network distillation. arXiv preprint arXiv:1810.12894.
[۴۸] O. Nahirnyi. "Reinforcement Learning Agents in Procedurally-generated Environments with Sparse Rewards,", 2022.
[۴۹] Macedo, L., & Cardoso, A. (1999, January). Towards artificial forms of surprise and curiosity. In Proceedings of the European Conference on Cognitive Science, S. Bagnara, Ed (pp. 139-144).
[۵۰] Saunders, R., & Gero, J. S. (2001). A curious design agent. In CAADRIA (Vol. 1, pp. 345-350).
[۵۱] C. Sun. "Curiosity-driven learning in artificial intelligence and its applications," 2023.
[۵۲] P. Auer, N. Cesa-Bianchi, P. Fischer. "Finite-time analysis of the multiarmed bandit problem," in Machine learning, vol. 47, pp. 235–256, 2002.
[۵۳] A. Strehl, M. Littman. "An analysis of model-based interval estimation for Markov decision processes," in Journal of Computer and System Sciences, vol. 74, no. 8, pp. 1309–1331, 2008.
[۵۴] Bellemare, M., et al. "Unifying count-based exploration and intrinsic motivation," in Advances in neural information processing systems, vol. 29, 2016.
[۵۵] Tang, H., et al. "# exploration: A study of count-based exploration for deep reinforcement learning," in Advances in neural information processing systems, vol. 30, 2017.
[۵۶] J. Fu, J. Co-Reyes, S. Levine. "Ex2: Exploration with exemplar models for deep reinforcement learning," in Advances in neural information processing systems, vol. 30, 2017.
[۵۷] Savinov, N., et al. "Episodic curiosity through reachability," in arXiv preprint arXiv:1810.02274, 2018.
[۵۸] Kim, Y., et al, "Curiosity-bottleneck: Exploration by distilling task-specific novelty," in International conference on machine learning, 2019, pp. 3379–3388.
[۵۹] Alemi, A., et al. "Deep variational information bottleneck," in arXiv preprint arXiv:1612.00410, 2016.
[۶۰] Xu, H., et al. "Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making," in PLOS Computational Biology, vol. 17, no. 6, pp. e1009070, 2021.
[۶۱] Modirshanechi, A., et al. "The curse of optimism: a persistent distraction by novelty," in bioRxiv, pp. 2022–07, 2022.
[۶۲] H. Jiang, Z. Ding, Z. Lu, "Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharing," in Proceedings of the AAAI Conference on Artificial Intelligence, 2024, pp. 17444–17452.
[۶۳] Sun, C., Qian, H., & Miao, C. (2022). From psychological curiosity to artificial curiosity: Curiosity-driven learning in artificial intelligence tasks. arXiv preprint arXiv:2201.08300.
[۶۴] Karaoguz, C., et al, "Curiosity driven exploration of sensory-motor mappings," in Capo Caccia Cognitive Neuromorphic Engineering Workshop, 2011.
[۶۵] R. Raileanu, T. Rocktäschel. "Ride: Rewarding impact-driven exploration for procedurally-generated environments," in arXiv preprint arXiv:2002.12292, 2020.
[۶۶] Parisi, S., et al. "Interesting object, curious agent: Learning task-agnostic exploration," in Advances in Neural Information Processing Systems, vol. 34, pp. 20516–20530, 2021.
[۶۷] Yuan, M., et al. "Rewarding episodic visitation discrepancy for exploration in reinforcement learning," in arXiv preprint arXiv:2209.08842, 2022.
[۶۸] Wang, Y., et al. "Efficient potential-based exploration in reinforcement learning using inverse dynamic bisimulation metric," in Advances in Neural Information Processing Systems, vol. 36, 2024.
[۶۹] Q. Wu, C. Miao, Z. Shen, "A curious learning companion in virtual learning environment," in 2012 IEEE International Conference on Fuzzy Systems, 2012, pp. 1–8.
[۷۰] Q. Wu, S. Liu, C. Miao, "Recommend interesting items: How can social curiosity help?," in Web Intelligence, 2019, pp. 297–311.
[۷۱] P. Oudeyer. "Intelligent adaptive curiosity: a source of self-development," ,2004.
[۷۲] S. Forestier, P. Oudeyer, "Modular active curiosity-driven discovery of tool use," in 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), 2016, pp. 3965–3972.
[۷۳] Colas, C., et al, "Curious: intrinsically motivated modular multi-goal reinforcement learning," in International conference on machine learning, 2019, pp. 1331–1340.
[۷۴] Baker, B., et al. "Emergent tool use from multi-agent autocurricula," in arXiv preprint arXiv:1909.07528, 2019.
[۷۵] Campero, A., et al. "Learning with amigo: Adversarially motivated intrinsic goals," in arXiv preprint arXiv:2006.12122, 2020.
[۷۶] Parker-Holder, J., et al, "Evolving curricula with regret-based environment design," in International Conference on Machine Learning, 2022, pp. 17473–17498.
[۷۷] Zhou, X., et al. "MENTOR: Guiding Hierarchical Reinforcement Learning with Human Feedback and Dynamic Distance Constraint," in arXiv preprint arXiv:2402.14244, 2024.
[۷۸] L. Macedo, A. Cardoso, "The role of surprise, curiosity and hunger on exploration of unknown environments populated with entities," in 2005 portuguese conference on artificial intelligence, 2005, pp. 47–53.
[۷۹] Houthooft, R., et al. "Vime: Variational information maximizing exploration," in Advances in neural information processing systems, vol. 29, 2016.
[۸۰] D. Pathak, D. Gandhi, A. Gupta, "Self-supervised exploration via disagreement," in International conference on machine learning, 2019, pp. 5062–5071.
[۸۱] Berseth, G., et al. "Smirl: Surprise minimizing reinforcement learning in unstable environments," in arXiv preprint arXiv:1912.05510, 2019.
[۸۲] J. Chen. "Reinforcement learning generalization with surprise minimization," in arXiv preprint arXiv:2004.12399, 2020.
[۸۳] Fickinger, A., et al. "Explore and control with adversarial surprise," in arXiv preprint arXiv:2107.07394, 2021.
[۸۴] Haarnoja, T., et al. "Soft actor-critic algorithms and applications," in arXiv preprint arXiv:1812.05905, 2018.
[۸۵] O’Donoghue, B., et al, "The uncertainty bellman equation and exploration," in International conference on machine learning, 2018, pp. 3836–3845.
[۸۶] Nachum, O., et al. "Trust-pcl: An off-policy trust region method for continuous control," in arXiv preprint arXiv:1707.01891, 2017.
[۸۷] Lin, J., et al. "Cat-sac: Soft actor-critic with curiosity-aware entropy temperature," 2020.
[۸۸] Li, K., et al, "Mural: Meta-learning uncertainty-aware rewards for outcome-driven reinforcement learning," in International conference on machine learning, 2021, pp. 6346–6356.
[۸۹] D. Cho, S. Lee, H. Kim. "Outcome-directed reinforcement learning by uncertainty & temporal distance-aware curriculum goal generation," in arXiv preprint arXiv:2301.11741, 2023.
[۹۰] Lee, S., et al. "CQM: curriculum reinforcement learning with a quantized world model," in Advances in Neural Information Processing Systems, vol. 36, 2024.
[۹۱] A. Barto, M. Mirolli, G. Baldassarre. "Novelty or surprise?," in Frontiers in psychology, vol. 4, pp. 907, 2013.
[۹۲] J. Schmidhuber. "Formal theory of creativity, fun, and intrinsic motivation (1990–2010)," in IEEE transactions on autonomous mental development, vol. 2, no. 3, pp. 230–247, 2010.
[۹۳] Storck, J., et al, "Reinforcement driven information acquisition in non-deterministic environments," in Proceedings of the international conference on artificial neural networks, Paris, 1995, pp. 159–164.
[۹۴] J. Schmidhuber, "Adaptive confidence and adaptive curiosity," Citeseer, Tech. Rep., 1991.
[۹۵] Ugur, E., et al, "Curiosity-driven learning of traversability affordance on a mobile robot," in 2007 IEEE 6th international conference on development and learning, 2007, pp. 13–18.
[۹۶] Barto, A., et al, "Intrinsically motivated learning of hierarchical collections of skills," in Proceedings of the 3rd International Conference on Development and Learning, 2004, pp. 19.
[۹۷] N. Chentanez, A. Barto, S. Singh. "Intrinsically motivated reinforcement learning," in Advances in neural information processing systems, vol. 17, 2004.
[۹۸] Sekar, R., et al, "Planning to explore via self-supervised world models," in International conference on machine learning, 2020, pp. 8583–8592.
[۹۹] Nguyen, T., et al, "Sample-efficient reinforcement learning representation learning with curiosity contrastive forward dynamics model," in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021, pp. 3471–3477.
[۱۰۰] Kapturowski, S., et al, "Recurrent experience replay in distributed reinforcement learning," in International conference on learning representations, 2018.
[۱۰۱] Jaques, N., et al, "Social influence as intrinsic motivation for multi-agent deep reinforcement learning," in International conference on machine learning, 2019, pp. 3040–3049.
[۱۰۲] Zheng, L., et al. "Episodic multi-agent reinforcement learning with curiosity-driven exploration," in Advances in Neural Information Processing Systems, vol. 34, pp. 3757–3769, 2021.
[۱۰۳] Mazzaglia, P., et al, "Self-supervised exploration via latent Bayesian surprise," in ICLR2021, the 9th International Conference on Learning Representations, 2021.
[۱۰۴] Mazzaglia, P., et al, "Curiosity-driven exploration via latent bayesian surprise," in Proceedings of the AAAI conference on artificial intelligence, 2022, pp. 7752–7760.
[۱۰۵] Le, H., et al, "Beyond Surprise: Improving Exploration Through Surprise Novelty.," in AAMAS, 2024, pp. 1084–1092.
[۱۰۶] Amin, S., et al. "A survey of exploration methods in reinforcement learning," in arXiv preprint arXiv:2109.00157, 2021.
[۱۰۷] M. Machado, M. Bellemare, M. Bowling, "A laplacian framework for option discovery in reinforcement learning," in International Conference on Machine Learning, 2017, pp. 2295–2304.
[۱۰۸] Machado, M., et al. "Eigenoption discovery through the deep successor representation," in arXiv preprint arXiv:1710.11089, 2017.
[۱۰۹] Hong, Z., et al. "Diversity-driven exploration strategy for deep reinforcement learning," in Advances in neural information processing systems, vol. 31, 2018.
[۱۱۰] Jinnai, Y., et al, "Discovering options for exploration by minimizing cover time," in International Conference on Machine Learning, 2019, pp. 3130–3139.
[۱۱۱] Hazan, E., et al, "Provably efficient maximum entropy exploration," in International Conference on Machine Learning, 2019, pp. 2681–2691.
[۱۱۲] Jinnai, Y., et al, "Exploration in reinforcement learning with deep covering options," in International Conference on Learning Representations, 2020.
[۱۱۳] Amin, S., et al. "Locally persistent exploration in continuous control tasks with sparse rewards," in arXiv preprint arXiv:2012.13658, 2020.
[۱۱۴] Sabbioni, L., et al, "Simultaneously updating all persistence values in reinforcement learning," in Proceedings of the AAAI Conference on Artificial Intelligence, 2023, pp. 9668–9676.
[۱۱۵] Hartikainen, K., et al. "Dynamical distance learning for semi-supervised and unsupervised skill discovery," in arXiv preprint arXiv:1907.08225, 2019.
[۱۱۶] F. Stulp, O. Sigaud. "Robot skill learning: From reinforcement learning to evolution strategies," in Paladyn, Journal of Behavioral Robotics, vol. 4, no. 1, pp. 49–61, 2013.
[۱۱۷] S. Nguyen, P. Oudeyer. "Socially guided intrinsic motivation for robot learning of motor skills," in Autonomous Robots, vol. 36, pp. 273–294, 2014.
[۱۱۸] G. Gordon. "Infant-inspired intrinsically motivated curious robots," in Current Opinion in Behavioral Sciences, vol. 35, pp. 28–34, 2020.
[۱۱۹] Zeng, H., et al. "AHEGC: Adaptive Hindsight Experience Replay With Goal-Amended Curiosity Module for Robot Control," in IEEE Transactions on Neural Networks and Learning Systems, 2023.
[۱۲۰] Wang, T., et al. "Curiosity model policy optimization for robotic manipulator tracking control with input saturation in uncertain environment," in Frontiers in Neurorobotics, vol. 18, pp. 1376215, 2024.
[۱۲۱] Luo, Y., et al, "Curiosity-driven reinforcement learning for diverse visual paragraph generation," in Proceedings of the 27th ACM International Conference on Multimedia, 2019, pp. 2341–2350.
[۱۲۲] Colas, C., et al. "Language as a cognitive tool to imagine goals in curiosity driven exploration," in Advances in Neural Information Processing Systems, vol. 33, pp. 3761–3774, 2020.
[۱۲۳] Hong, Z., et al. "Curiosity-driven red-teaming for large language models," in arXiv preprint arXiv:2402.19464, 2024.
[۱۲۴] Roohi, S., et al, "Review of intrinsic motivation in simulation-based game testing," in Proceedings of the 2018 chi conference on human factors in computing systems, 2018, pp. 1–13.
[۱۲۵] Esteva, A., et al. "Dermatologist-level classification of skin cancer with deep neural networks," in nature, vol. 542, no. 7639, pp. 115–118, 2017.
[۱۲۶] Niu, X., et al, "Surprise me if you can: Serendipity in health information," in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 2018, pp. 1–12.
[۱۲۷] Z. Fu, X. Niu, M. Maher. "Deep learning models for serendipity recommendations: a survey and new perspectives," in ACM Computing Surveys, vol. 56, no. 1, pp. 1–26, 2023.
[۱۲۸] Song, S., et al. "Serious information in hedonic social applications: affordances, self-determination and health information adoption in TikTok," in Journal of Documentation, vol. 78, no. 4, pp. 890–911, 2022.
[۱۲۹] T. Hasan, R. Bunescu, "Topic-Level Bayesian Surprise and Serendipity for Recommender Systems," in Proceedings of the 17th ACM Conference on Recommender Systems, 2023, pp. 933–939.
[۱۳۰] Codevilla, F., et al, "Exploring the limitations of behavior cloning for autonomous driving," in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9329–9338.
[۱۳۱] M. Albilani, A. Bouzeghoub, "Dynamic Adjustment of Reward Function for Proximal Policy Optimization with Imitation Learning: Application to Automated Parking Systems," in 2022 IEEE Intelligent Vehicles Symposium (IV), 2022, pp. 1400–1408.
[۱۳۲] F. Carton, "Exploration of reinforcement learning algorithms for autonomous vehicle visual perception and control," Ph.D. dissertation, Institut Polytechnique de Paris, 2021.
[۱۳۳] M. Hutsebaut-Buysse, "Learning to navigate through abstraction and adaptation," Ph.D. dissertation, University of Antwerp, 2023.
[۱۳۴] Huang, C., et al. "Deductive reinforcement learning for visual autonomous urban driving navigation," in IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 12, pp. 5379–5391, 2021.
[۱۳۵] Wu, Y., et al. "Deep reinforcement learning on autonomous driving policy with auxiliary critic network," in IEEE transactions on neural networks and learning systems, vol. 34, no. 7, pp. 3680–3690, 2021.
[۱۳۶] Yan, Y., et al, "An improved proximal policy optimization algorithm for autonomous driving decision-making," in Fourth International Conference on Sensors and Information Technology (ICSI 2024), 2024, pp. 837–845.