Evolving Symmetric and Modular Neural Network Controllers for Multilegged Robots Vinod K. Valsalam Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 USA Email: firstname.lastname@example.org Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 USA Email: email@example.com Abstract—Controllers for multilegged robots are characterized by modularity and symmetry. However, the controller symmetries necessary for generating appropriate gaits are often difficult to determine analytically. This paper utilizes a nature-inspired approach called Evolution of Network Symmetry and mOdularity (ENSO) to evolve such controllers automatically. It uses group theory to mutate symmetry systematically, making it more effective than mutating symmetry randomly. This approach was evaluated by evolving modular neural network controllers for a quadruped robot in physically realistic simulations. On flat ground, the resulting controllers are as effective as those having hand-designed symmetries. However, they are significantly faster when evolved on inclined ground, where the appropriate symmetries are difficult to determine manually. The group-theoretic symmetry mutations of ENSO are also significantly more effective at evolving such controllers than random symmetry mutations. Thus, ENSO is a promising approach for evolving modular and symmetric controllers for multilegged robots, as well as solutions to distributed control problems in general. I. I NTRODUCTION Controllers for legged robots are usually designed manually, to ensure that the legs are properly coordinated and the robot is stable. This process typically requires analyzing the sensormotor systems and body-limb dynamics of the robot , which is generally difficult and brittle. Therefore, it is desirable to automate the controller design using learning techniques. One particularly promising such a technique is neuroevolution, which has been shown to perform well in various control domains such as rocket control  and robot control . However, the control outputs in such applications have been relatively simple, and it turns out hard to scale them up to multilegged walking, where there are many outputs. Neural network controllers for multilegged robots can typically be decomposed into functionally identical subnetworks or modules, e.g. each module controlling a leg or a joint. Evolving such modular networks is easier because evolution can search in the smaller parameter space of the modules rather than in the full space of the networks. Based on this observation, researchers have evolved modular neural networks for controlling legged robots , . However, they often specify the connections between the modules manually. The structure of these connections is also important because the symmetries, i.e. permutations of the modules that leave the controller invariant, influence the type of gaits the controller can generate for the robot . In this paper, an approach called Evolution of Network Symmetry and mOdularity (ENSO)  is utilized to evolve both the modules and the symmetries of the controller simultaneously. ENSO uses a compact genotype that stores the parameters of repeated modules only once, while allowing variations between them to evolve. It utilizes group theory to represent symmetry mathematically and encodes symmetry in a form that evolution can manipulate, making it possible to evolve symmetry incrementally. The resulting systematic symmetry mutations avoid large, detrimental changes to controllers (that are otherwise likely with unsystematic mutations), making evolution more effective in finding good solutions. The ENSO approach was evaluated by evolving neural network controllers for a quadruped robot in a physically realistic simulation. The resulting controllers produce effective locomotive behaviors on both flat and inclined grounds. Moreover, on inclined ground these controllers are significantly better than those evolved with hand-designed symmetries, and with random symmetry mutations. Since inclines and other similar complexities are common in the real world, these results suggest that ENSO is a promising approach for designing controllers for legged robots in the real world. II. M ETHOD The robot model used in the simulation resembles a table with a rectangular body supported by legs at the four corners (Figure 1). The legs are cylindrical with capped ends, and attached to the body by a hinge joint having full 360◦ freedom of rotation. The axis of rotation of the joint is tilted to the side, causing the rotating leg to trace a cone. The leg makes contact with the ground when it is at one edge of the cone. Forward and backward locomotion is achieved by coordinating the circular movements of the leg. The robot controller activates the simulated servo motor attached to each joint by specifying the desired joint angular velocity. A neural network controller for this quadruped robot can be constructed using four modules, each controlling a different leg. All modules have the same network architecture shown in Figure 2a. Each module’s input is the joint angle of the leg it controls. The module’s output is the desired angular velocity of that leg. The hidden and output units have sigmoidal activation functions; The input units do not perform any computation. To appear in Exploring New Horizons in Evolutionary Design of Robots: Workshop at the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2009. Fig. 1: The quadruped robot model on inclined ground. The legs are attached to the body by hinge joints with axes of rotation tilted sideways, allowing the legs to make full circular rotation. Locomotion is achieved by coordinating the circular movements of the legs. Although more legs and complex legs can be used, this model is a simple and physically realistic platform that has symmetric and modular controllers, making it possible to evaluate how well ENSO can evolve effective controllers in challenging environments. Visualization videos of the evolved walking behaviors can be seen at the website http://nn.cs.utexas.edu/?enso-robots. . x1 x1 Module 1 x3 x2 1 2 3 4 x4 (a) Network module and initial phenotype graph . x3 . x2 . x1 x1 x2 x3 . x4 x4 (b) Full controller network consisting of four modules Fig. 2: Modular controller network for the quadruped robot model. The input to each module is the leg angle that it controls, and the output is the desired angular velocity of that leg. The full controller network consists of four such modules, each module receiving input from all the other modules. The phenotype graph represents these modules and their connectivity. At the beginning of evolution, this graph has identical vertices (modules) and edges (interconnections), i.e. all vertices and edges have the same combination of network weights. Evolution discovers effective controllers by breaking symmetry to create new types of vertices and edges, and by optimizing the initially random network weights corresponding to those vertices and edges. The modules are connected such that each module receives input from all the other modules (Figure 2b). ENSO represents these modules and their connectivity as a graph, which forms the phenotype of evolution . The modules correspond to the vertices of the graph and the connections between the modules correspond to its edges. The symmetry of this graph represents the symmetry of the controller. ENSO initializes the evolutionary population with maximally symmetric graphs, i.e. all vertices (edges) of the graph are identical with the same combination of network weights. During evolution, ENSO discovers appropriate controller symmetries by utilizing group theory to create new types of vertices and edges systematically and optimizes the network weights corresponding to those vertices and edges. In order to evaluate this approach for evolving controllers, four experimental methods were compared: (1) Evolving controller symmetry systematically using ENSO, (2) evolving controller symmetry randomly without using the group-theory mechanisms of ENSO, (3) evolving controllers with fixed symmetry that is hand-designed to be maximal, and (4) evolving controllers with fixed symmetry that is hand-designed to produce animal-like gaits as was done in . Although the four methods differ in how they evolve symmetry, they all evolve the weights of the controller networks in the same way as ENSO utilizing Gaussian perturbations. The results of these experiments are described next. III. R ESULTS The experiments were run in a realistic physical simulation of robot locomotion on flat ground and on inclined ground. Each controller network was evaluated by making it control the locomotion of a robot for one minute of simulated time. At the end of the simulation, the fitness of the controller network was calculated as the distance the robot traveled. On flat ground, all four methods produced similar fitness through all generations, implying that the hand-designed symmetries are sufficient for controllers to produce fast gaits. However, on the more challenging inclined ground, where the robot must walk across the incline without tipping over or slipping, the appropriate controller symmetries are not known. Therefore, both symmetry evolution methods performed significantly better than those using the hand-designed symmetries. Moreover, the group-theory based systematic symmetry search of ENSO produced controllers with significantly faster gaits than the unsystematic symmetry search of random symmetry evolution. These controllers also generalized better when friction of the ground was reduced. Since such challenging environments are common in the real world and designing controller symmetries for them by hand is difficult, these results suggest that ENSO is a promising approach for evolving controllers for legged robots in the real world. R EFERENCES  R. D. Beer and J. C. Gallagher. Evolving dynamical neural networks for adaptive behavior. Adaptive Behavior, 1(1):91–122, 1992.  J. J. Collins and I. N. Stewart. Coupled nonlinear oscillators and the symmetries of animal gaits. J. of Nonlinear Science, 3(1):349–392, 1993.  D. Floreano and F. Mondada. Evolutionary neurocontrollers for autonomous mobile robots. Neural Networks, 11:1461–1478, 1998.  F. Gomez and R. Miikkulainen. Active guidance for a finless rocket using neuroevolution. In Proceedings of GECCO, pp. 2084–2095, 2003.  P. Holmes, R. J. Full, D. Koditschek, and J. Guckenheimer. The dynamics of legged locomotion: Models, analyses, and challenges. SIAM Review, 48(2):207–304, 2006.  A. J. Ijspeert. A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander. Biological Cybernetics, 84(5):331–348, 2001.  V. K. Valsalam and R. Miikkulainen. Modular neuroevolution for multilegged locomotion. In Proceedings of GECCO, pp. 265–272, 2008.  V. K. Valsalam and R. Miikkulainen. Evolving symmetric and modular neural networks for distributed control. In Proceedings of GECCO, pp. 731–738, 2009.
Link or Click Back
Here will be a configuration form