Novelty search in neuroevolution for end effector positioning

A.Y. Vitiuk, A.Y. Doroshenko


The article considers the use of the neuroevolution algorithm for neural network policies search when creating a controller for a robotic arm, in particular for the subtask of positioning the end effector. Neuro-evolution is a family of machine learning methods that use evolutionary algorithms by imitating the process of natural selection. This approach has been found to be particularly effective for the positioning task, where the final position can be achieved in many optimal ways and therefore requires reinforcement learning. It is noted that the final result of neuroevolution is an optimal network topology, which makes the model more resource-efficient and easier to analyze. The paper considers the process of neural network policy search for controlling a two-dimensional robot with two links. According to the results of the experiments, an increase in the efficiency of the best solution found using novelty search for the NEAT algorithm is noted compared to the NEAT algorithm without novelty search. It was established that the proposed approach allows to obtain an effective neural network policy, which has a minimal configuration, which will allow to increase the speed of the controller, that is critical for the operation of a real system. Thus, the use of novelty search as a method of optimizing the neuroevolutionary process to solve the positioning problem allows to increase the efficiency of the learning process and obtain the optimal network topology.

Prombles in programming 2023; 3: 49-57


neural network; neuroevolution; robotic arm; reinforcement learning; NEAT; novelty search; fitness function


I. Omelianenko, Hands-On Neuroevolution with Python: Build high performing artificial neural network architectures using neuroevolution based algorithms, Packt Publishing, 2019

R. Mahjourian, R. Miikkulainen, Neuroevolutionary Planning for Robotic Control, Department of Computer Science The University of Texas at Austin Austin, 2018

Kenneth O. Stanley and Risto Miikkulainen, Evolving Neural Networks Through Augmenting Topologies, Evolutionary Computation 10 (2): 99 127, 2002. CrossRef

M. W¨urtinger, Neuroevolution for Robot Control, Test Framework and Experimental Evaluation, Institut f¨ur Informatik Lehrstuhl f¨ur Programmierung und Softwaretechnik, 2011. URL:

OpenAI Gym 2D Robot Arm Environment, URL:

Huang, Pei-Chi, Sentis et al., Tradeoffs in Neuroevolutionary Learning-Based Real-Time Robotic Task Design in the Imprecise Computation Framework. ACM Transactions on Cyber-Physical Systems. 3. 1-29. CrossRef



  • There are currently no refbacks.