A Novel Exploration-Exploitation-Based Adaptive Law for Intelligent Model-Free Control Approaches
[ X ]
Tarih
2023
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
IEEE-Inst Electrical Electronics Engineers Inc
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Özet
Model-free control approaches require advanced exploration-exploitation policies to achieve practical tasks such as learning to bipedal robot walk in unstructured environments. In this article, we first construct a comprehensive exploration-exploitation policy that carries quality knowledge about the long-term predictor and the control policy, and the control signal of the model-free algorithms. Therefore, the developed model-free algorithm continues exploration by adjusting its unknown parameters until the desired learning and control are accomplished. Second, we provide an utterly model-free adaptive law enriched with the exploration-exploitation policy and derived step-by-step using the exact analogy of the model-based solution. The obtained adaptive control law considers the control signal saturation and the control signal (input) delay. Performed Lyapunov stability analysis ensures the convergence of the adaptive law that can also be used for intelligent control approaches. Third, we implement the adaptive algorithm in real time on a challenging benchmark system: a fourth-order, coupled dynamics, input saturated, and time-delayed underactuated manipulator. The results show that the proposed adaptive algorithm explores larger state-action spaces and treats the vanishing gradient problem in both learning and control. Also, we notice from the results that the learning and control properties of the adaptive algorithm are optimized as required.
Açıklama
Anahtar Kelimeler
Adaptation models, Process control, Predictive models, Real-time systems, Noise measurement, Analytical models, Manipulator dynamics, Adaptive law, exploitation, exploration, intelligent control, Lyapunov stability, model free, uncertainty, vanishing gradient
Kaynak
Ieee Transactions on Cybernetics
WoS Q Değeri
Q1
Scopus Q Değeri
Q1
Cilt
53
Sayı
1
