A Novel Exploration-Exploitation-Based Adaptive Law for Intelligent Model-Free Control Approaches

Basit Öğe Kaydı
dc.authoridErol Barkana, Duygun/0000-0002-8929-0459
dc.authoridTutsoy, Onder/0000-0001-6385-3025
dc.authoridBALIKCI, KEMAL/0000-0001-6234-5627
dc.contributor.authorTutsoy, Önder
dc.contributor.authorBarkana, Duygun Erol
dc.contributor.authorBalikci, Kemal
dc.date.accessioned2025-01-06T17:36:05Z
dc.date.available2025-01-06T17:36:05Z
dc.date.issued2023
dc.description.abstractModel-free control approaches require advanced exploration-exploitation policies to achieve practical tasks such as learning to bipedal robot walk in unstructured environments. In this article, we first construct a comprehensive exploration-exploitation policy that carries quality knowledge about the long-term predictor and the control policy, and the control signal of the model-free algorithms. Therefore, the developed model-free algorithm continues exploration by adjusting its unknown parameters until the desired learning and control are accomplished. Second, we provide an utterly model-free adaptive law enriched with the exploration-exploitation policy and derived step-by-step using the exact analogy of the model-based solution. The obtained adaptive control law considers the control signal saturation and the control signal (input) delay. Performed Lyapunov stability analysis ensures the convergence of the adaptive law that can also be used for intelligent control approaches. Third, we implement the adaptive algorithm in real time on a challenging benchmark system: a fourth-order, coupled dynamics, input saturated, and time-delayed underactuated manipulator. The results show that the proposed adaptive algorithm explores larger state-action spaces and treats the vanishing gradient problem in both learning and control. Also, we notice from the results that the learning and control properties of the adaptive algorithm are optimized as required.
dc.description.sponsorshipResearch Council (TUB.ITAK) [215E047]; Turkish Academy of Sciences in Scheme of the Outstanding Young Scientist Award
dc.description.sponsorshipThis work was supported in part by the Research Council (TUB.ITAK) under Project 215E047, and in part by the Turkish Academy of Sciences in Scheme of the Outstanding Young Scientist Award (TUBAGEB.IP). This article was recommended by Associate Editor H. Han.
dc.identifier.doi10.1109/TCYB.2021.3091680
dc.identifier.endpage337
dc.identifier.issn2168-2267
dc.identifier.issn2168-2275
dc.identifier.issue1
dc.identifier.pmid34398780
dc.identifier.scopus2-s2.0-85113266677
dc.identifier.scopusqualityQ1
dc.identifier.startpage329
dc.identifier.urihttps://doi.org/10.1109/TCYB.2021.3091680
dc.identifier.urihttps://hdl.handle.net/20.500.14669/1747
dc.identifier.volume53
dc.identifier.wosWOS:000732282100001
dc.identifier.wosqualityQ1
dc.indekslendigikaynakWeb of Science
dc.indekslendigikaynakScopus
dc.indekslendigikaynakPubMed
dc.language.isoen
dc.publisherIEEE-Inst Electrical Electronics Engineers Inc
dc.relation.ispartofIeee Transactions on Cybernetics
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanı
dc.rightsinfo:eu-repo/semantics/closedAccess
dc.snmzKA_20241211
dc.subjectAdaptation models
dc.subjectProcess control
dc.subjectPredictive models
dc.subjectReal-time systems
dc.subjectNoise measurement
dc.subjectAnalytical models
dc.subjectManipulator dynamics
dc.subjectAdaptive law
dc.subjectexploitation
dc.subjectexploration
dc.subjectintelligent control
dc.subjectLyapunov stability
dc.subjectmodel free
dc.subjectuncertainty
dc.subjectvanishing gradient
dc.titleA Novel Exploration-Exploitation-Based Adaptive Law for Intelligent Model-Free Control Approaches
dc.typeArticle

Dosyalar

Koleksiyon

WoS İndeksli Yayınlar Koleksiyonu
PubMed İndeksli Yayınlar Koleksiyonu
Scopus İndeksli Yayınlar Koleksiyonu