Learning to balance an NAO robot using reinforcement learning with symbolic inverse kinematic

Basit Öğe Kaydı
dc.authoridErol Barkana, Duygun/0000-0002-8929-0459
dc.authoridTutsoy, Onder/0000-0001-6385-3025
dc.contributor.authorTutsoy, Önder
dc.contributor.authorBarkana, Duygun Erol
dc.contributor.authorColak, Sule
dc.date.accessioned2025-01-06T17:37:03Z
dc.date.available2025-01-06T17:37:03Z
dc.date.issued2017
dc.description.abstractAn autonomous humanoid robot (HR) with learning and control algorithms is able to balance itself during sitting down, standing up, walking and running operations, as humans do. In this study, reinforcement learning (RL) with a complete symbolic inverse kinematic (IK) solution is developed to balance the full lower body of a three-dimensional (3D) NAO HR which has 12 degrees of freedom. The IK solution converts the lower body trajectories, which are learned by RL, into reference positions for the joints of the NAO robot. This reduces the dimensionality of the learning and control problems since the IK integrated with the RL eliminates the need to use whole HR states. The IK solution in 3D space takes into account not only the legs but also the full lower body; hence, it is possible to incorporate the effect of the foot and hip lengths on the IK solution. The accuracy and capability of following real joint states are evaluated in the simulation environment. MapleSim is used to model the full lower body, and the developed RL is combined with this model by utilizing Modelica and Maple software properties. The results of the simulation show that the value function is maximized, temporal difference error is reduced to zero, the lower body is stabilized at the upright, and the convergence speed of the RL is improved with use of the symbolic IK solution.
dc.identifier.doi10.1177/0142331216645176
dc.identifier.endpage1748
dc.identifier.issn0142-3312
dc.identifier.issn1477-0369
dc.identifier.issue11
dc.identifier.scopus2-s2.0-85034611672
dc.identifier.scopusqualityQ2
dc.identifier.startpage1735
dc.identifier.urihttps://doi.org/10.1177/0142331216645176
dc.identifier.urihttps://hdl.handle.net/20.500.14669/2095
dc.identifier.volume39
dc.identifier.wosWOS:000415358800013
dc.identifier.wosqualityQ3
dc.indekslendigikaynakWeb of Science
dc.indekslendigikaynakScopus
dc.language.isoen
dc.publisherSage Publications Ltd
dc.relation.ispartofTransactions of The Institute of Measurement and Control
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanı
dc.rightsinfo:eu-repo/semantics/closedAccess
dc.snmzKA_20241211
dc.subjectComplete symbolic inverse kinematic solution
dc.subjectconvergent value function
dc.subjectMapleSim
dc.subjectModelica
dc.subjectmulti-body modelling software
dc.subjectNAO lower body balancing
dc.subjectreinforcement learning
dc.subjectautonomous humanoid robot
dc.titleLearning to balance an NAO robot using reinforcement learning with symbolic inverse kinematic
dc.typeArticle

Dosyalar

Koleksiyon

WoS İndeksli Yayınlar Koleksiyonu
Scopus İndeksli Yayınlar Koleksiyonu