Assembler Encoding Versus Connectivity Matrix Encoding in the Inverted Pendulum Problem with a Hidden State
The Naval University, ul. Śmidowicza 69, Gdynia, Poland
e-mail: T.Praczyk@amw.gdynia.pl
Received:
Received: 5 January 2009; revised: 25 June 2009; accepted: 25 June 2009; published online: 2 September 2009
DOI: 10.12921/cmst.2009.15.02.177-183
OAI: oai:lib.psnc.pl:673
Abstract:
Assembler Encoding is the Artificial Neural Network encoding method. To date, Assembler Encoding has been tested in the optimization problem and in the so-called predator-prey problem. The paper reports experiments in a next test problem, i.e. in the inverted pendulum problem. To compare Assembler Encoding with other Artificial Neural Network encoding methods in the experiments, two direct encodings were also tested.
Key words:
References:
[1] L. Baird III, Reinforcement Learning Through Gradient Descent. PhD thesis, Carnegie Mellon University, Pittsburgh (1999).
[2] P. Cichosz, Learning systems. WNT, Warsaw (2000).
[3] A. Cangelosi, D. Parisi and S. Nolfi, Cell division and migration in a genotype for neural networks. Network: computation in neural systems 5(4) 497-515 (1994).
[4] D. Curran and C. O’Riordan, Applying Evolutionary Computation to Designing Networks: A Study of the State of the Art. National University of Ireland, technical report NUIGIT- 111002 (2002).
[5] D. Floreano and J. Urzelai, Evolutionary robots with online self-organization and behavioral fitness. Neural Networks 13, 431-443 (2000).
[6] F. Gomez, J. Schmidhuber and R. Miikkulainen, Accelerated Neural Evolution through Cooperatively Coevolved Synapses. Journal of Machine Learning Research 9, 937-965 (2008).
[7] F. Gruau, Neural network Synthesis Using Cellular Encoding And The Genetic Algorithm. PhD Thesis, Ecole Normale Superieure de Lyon (1994).
[8] H. Kitano, Designing neural networks using genetic algorithms with graph generation system. Complex Systems 4, 461-476 (1990).
[9] K. Krawiec and B. Bhanu, Visual Learning by Coevolutionary Feature Synthesis. IEEE Trans. on Systems, Man, and Cybernetics, Part B: Cybernetics 35, 409-425 (2005).
[10] M.L. Littman and C.A. Szepesvari, Generalized Reinforcement-Learning Model: Convergence and Applications. In Proceedings of the Thirteenth International Conference on
Machine Learning 310-318 (1996).
[11] S. Luke and L. Spector, Evolving Graphs and Networks with Edge Encoding: Preliminary Report. In: Koza J.R., ed., Late Breaking Papers at the Genetic Programming
1996 Conference, (Stanford University, CA, USA, Stanford Bookstore, 1996) 117-124.
[12] M. Mandischer, Representation and Evolution of Neural Networks. In: Albrecht R.F., Reeves C.R., Steele U.C., ed., Artificial Neural Nets and Genetic Algorithms (Springer Verlag, New York, 1993) 643-649.
[13] G.F. Miller, P.M. Todd and S.U. Hegde, Designing Neural Networks Using Genetic Algorithms. Proceedings of the Third International Conference on Genetic Algorithms. 379-384 (1989).
[14] D.E. Moriarty, Symbiotic Evolution of Neural Networks in Sequential Decision Tasks. PhD thesis, The University of Texas at Austin, TR UT-AI97-257 (1997).
[15] S. Nolfi and D. Parisi, Growing neural networks. In: Langton C.G., ed., Artificial Life III, (Addison-Wesley, 1992).
[16] P. Nordin, W. Banzhaf and F. Francone, Efficient Evolution of Machine Code for {CISC} Architectures using Blocks and Homologous Crossover. Advances in Genetic Programming III, (Spector L., Langdon W., O’Reilly U. and Angeline P.) 275-299 (1999).
[17] M. Potter, The Design and Analysis of a Computational Model of Cooperative Coevolution. PhD thesis, George Mason University, Fairfax, Virginia (1997).
[18] M. Potter and K.A. De Jong, Evolving neural networks with collaborative species. In: Oren T.I., Birta L.G. (Eds.), Proceedings of the 1995 Summer Computer Simulation Conference, The Society of Computer Simulation 340-345 (1995).
[19] M.A. Potter and K.A. De Jong, A Cooperative Coevolutionary Approach to Function Optimization. The Third Parallel Problem Solving From Nature, Jerusalem, Israel, Springer-Verlag 249-257 (1994).
[20] M.A. Potter and K.A. De Jong, Cooperative coevolution: An architecture for evolving coadapted subcomponents. Evolutionary Computation 8(1), 1-29 (2000).
[21] T. Praczyk, Evolving co-adapted subcomponents in Assembler Encoding. International Journal of Applied Mathematics and Computer Science 17(4) 2007.
[22] T. Praczyk, Procedure application in Assembler Encoding. Archives of Control Science 17(LIII), 1, 71-91 (2007).
[23] T. Praczyk, Modular Neural Networks in Assembler Encoding. Computational Methods in Science and Technology 14(1) (2008).
[24] O. Stanley and R. Miikkulainen, Evolving neural networks through augmenting topologies. Evolutionary Computation 10, 99-127 (2002).
[25] O. Stanley, Efficient Evolution of Neural Networks Through Complexification. PhD thesis, Department of Computer Science, The University of Texas at Austin, August 2004. Technical Report AI-TR-04-314.
Assembler Encoding is the Artificial Neural Network encoding method. To date, Assembler Encoding has been tested in the optimization problem and in the so-called predator-prey problem. The paper reports experiments in a next test problem, i.e. in the inverted pendulum problem. To compare Assembler Encoding with other Artificial Neural Network encoding methods in the experiments, two direct encodings were also tested.
Key words:
References:
[1] L. Baird III, Reinforcement Learning Through Gradient Descent. PhD thesis, Carnegie Mellon University, Pittsburgh (1999).
[2] P. Cichosz, Learning systems. WNT, Warsaw (2000).
[3] A. Cangelosi, D. Parisi and S. Nolfi, Cell division and migration in a genotype for neural networks. Network: computation in neural systems 5(4) 497-515 (1994).
[4] D. Curran and C. O’Riordan, Applying Evolutionary Computation to Designing Networks: A Study of the State of the Art. National University of Ireland, technical report NUIGIT- 111002 (2002).
[5] D. Floreano and J. Urzelai, Evolutionary robots with online self-organization and behavioral fitness. Neural Networks 13, 431-443 (2000).
[6] F. Gomez, J. Schmidhuber and R. Miikkulainen, Accelerated Neural Evolution through Cooperatively Coevolved Synapses. Journal of Machine Learning Research 9, 937-965 (2008).
[7] F. Gruau, Neural network Synthesis Using Cellular Encoding And The Genetic Algorithm. PhD Thesis, Ecole Normale Superieure de Lyon (1994).
[8] H. Kitano, Designing neural networks using genetic algorithms with graph generation system. Complex Systems 4, 461-476 (1990).
[9] K. Krawiec and B. Bhanu, Visual Learning by Coevolutionary Feature Synthesis. IEEE Trans. on Systems, Man, and Cybernetics, Part B: Cybernetics 35, 409-425 (2005).
[10] M.L. Littman and C.A. Szepesvari, Generalized Reinforcement-Learning Model: Convergence and Applications. In Proceedings of the Thirteenth International Conference on
Machine Learning 310-318 (1996).
[11] S. Luke and L. Spector, Evolving Graphs and Networks with Edge Encoding: Preliminary Report. In: Koza J.R., ed., Late Breaking Papers at the Genetic Programming
1996 Conference, (Stanford University, CA, USA, Stanford Bookstore, 1996) 117-124.
[12] M. Mandischer, Representation and Evolution of Neural Networks. In: Albrecht R.F., Reeves C.R., Steele U.C., ed., Artificial Neural Nets and Genetic Algorithms (Springer Verlag, New York, 1993) 643-649.
[13] G.F. Miller, P.M. Todd and S.U. Hegde, Designing Neural Networks Using Genetic Algorithms. Proceedings of the Third International Conference on Genetic Algorithms. 379-384 (1989).
[14] D.E. Moriarty, Symbiotic Evolution of Neural Networks in Sequential Decision Tasks. PhD thesis, The University of Texas at Austin, TR UT-AI97-257 (1997).
[15] S. Nolfi and D. Parisi, Growing neural networks. In: Langton C.G., ed., Artificial Life III, (Addison-Wesley, 1992).
[16] P. Nordin, W. Banzhaf and F. Francone, Efficient Evolution of Machine Code for {CISC} Architectures using Blocks and Homologous Crossover. Advances in Genetic Programming III, (Spector L., Langdon W., O’Reilly U. and Angeline P.) 275-299 (1999).
[17] M. Potter, The Design and Analysis of a Computational Model of Cooperative Coevolution. PhD thesis, George Mason University, Fairfax, Virginia (1997).
[18] M. Potter and K.A. De Jong, Evolving neural networks with collaborative species. In: Oren T.I., Birta L.G. (Eds.), Proceedings of the 1995 Summer Computer Simulation Conference, The Society of Computer Simulation 340-345 (1995).
[19] M.A. Potter and K.A. De Jong, A Cooperative Coevolutionary Approach to Function Optimization. The Third Parallel Problem Solving From Nature, Jerusalem, Israel, Springer-Verlag 249-257 (1994).
[20] M.A. Potter and K.A. De Jong, Cooperative coevolution: An architecture for evolving coadapted subcomponents. Evolutionary Computation 8(1), 1-29 (2000).
[21] T. Praczyk, Evolving co-adapted subcomponents in Assembler Encoding. International Journal of Applied Mathematics and Computer Science 17(4) 2007.
[22] T. Praczyk, Procedure application in Assembler Encoding. Archives of Control Science 17(LIII), 1, 71-91 (2007).
[23] T. Praczyk, Modular Neural Networks in Assembler Encoding. Computational Methods in Science and Technology 14(1) (2008).
[24] O. Stanley and R. Miikkulainen, Evolving neural networks through augmenting topologies. Evolutionary Computation 10, 99-127 (2002).
[25] O. Stanley, Efficient Evolution of Neural Networks Through Complexification. PhD thesis, Department of Computer Science, The University of Texas at Austin, August 2004. Technical Report AI-TR-04-314.
