Reimagining robotic surgical precision: Reinforcement learning in swimmer-v1 environments
1 Manager, IT Application, Trinity Information Services, Trinity Health, Livonia, Michigan, USA.
2 Vice President, Information Technology, Deenabandhu Chhotu Ram University of Science & Technology, Sonepat, India.
3 Student, Department of Computer Science and Engineering (IoT), Vasireddy Venkatadri Institute of Technology, Nambur, India.
Review Article
World Journal of Advanced Research and Reviews, 2024, 22(01), 1739–1744
Publication history:
Received on 15 March 2024; revised on 22 April 2024; accepted on 24 April 2024
Abstract:
The current study focuses on how reinforcement learning algorithms tackle complex tasks, specifically analyzing the Swimmer-v1 environment with the reassembly of a serpentine robot in robotic surgeries. Herein, the review pays close attention to two algorithms- Proximal Policy Optimization (PPO) and Deep Deterministic Policy Gradients (DDPG)- focusing on exploration strategies in the Swimmer-v1 environment. Of particular importance here is the mentioning of the fact that the scope of exploration includes the use of parameter noise. Findings show that the DDPG learning algorithm faces outstanding difficulties with local maxima convergence. PPO emerged as the first in terms of algorithm category studied despite continuing issues of high variance. The use of a novel method which consists of tempering the range of variation of standard deviation in action noise gives promising results and can be a road to future improvement and exploration. The study provides a critical understanding of the underlying complexities that may lie hidden within the existing reinforcement learning algorithms. It brings up for repair weak points, particularly in the development of exploration capabilities and convergence stabilities.
Keywords:
Swimmer-v1 Environment; Proximal Policy Optimization (PPO); Surgical Robotics; Robotic Reassembly; Reinforcement Learning; Algorithmic Adaptability; Policy-based algorithm; Stability Analysis
Full text article in PDF:
Copyright information:
Copyright © 2024 Author(s) retain the copyright of this article. This article is published under the terms of the Creative Commons Attribution Liscense 4.0