Adaptive estimator design for unstable output error systems: A test problem and traditional system identification based analysis

Basit Öğe Kaydı
dc.authoridTutsoy, Onder/0000-0001-6385-3025
dc.contributor.authorTutsoy, Önder
dc.contributor.authorColak, Sule
dc.date.accessioned2025-01-06T17:36:05Z
dc.date.available2025-01-06T17:36:05Z
dc.date.issued2015
dc.description.abstractA key open question in adaptive estimator design is how to assure that the parameters of the proposed algorithms are converging to their almost correct solutions; hence, the learning algorithm is unbiased. Moreover, determining the speed of parameter convergence is important as it provides insight about the performance of the learning algorithms. The main contributions of the article are fourfold: the first one is that the article, initially, introduces an adaptive estimator to learn the discounted Q-function and approximate optimal control policy without requiring linear, discrete time, unstable output error system dynamics, but using only the noisy system measurements. The simulation results show that the adaptive estimator minimizes the stochastic cost function and temporal difference error and also learns the approximate Q-function together with the control policy. The second one is consideration of a different approach by taking a simple test problem to investigate issues associated with the Q-function's representation and parametric convergence. In particular, the terminal convergence problem is analyzed with a known optimal control policy where the aim is to accurately learn only the Q-function. It is parameterized by terms which are functions of the unknown plant's parameters and the Q-function's discount factor, and their convergence properties are analyzed and compared with the adaptive estimator. The third one is to show that even though the adaptive estimator with a large Q-function discount factor yields larger control feedback gains, so that faster state converges upright, the learning problem is badly conditioned; hence, the parameter convergence is sluggish, as the Q-function discount factor approaches the inverse of the dominant pole of the unstable system. Finally, the fourth one is comparison of the state output learned by the adaptive estimator with the ones obtained from traditional system identification algorithms. Simulation result for a higher order unstable output error system shows that the adaptive estimator closely follows the real system output whereas the system identification algorithms do not.
dc.description.sponsorshipTurkish Science and Technology Research Department
dc.description.sponsorshipThe author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was financially supported by the Turkish Science and Technology Research Department.
dc.identifier.doi10.1177/0959651815603910
dc.identifier.endpage916
dc.identifier.issn0959-6518
dc.identifier.issn2041-3041
dc.identifier.issue10
dc.identifier.scopus2-s2.0-84944053526
dc.identifier.scopusqualityQ2
dc.identifier.startpage902
dc.identifier.urihttps://doi.org/10.1177/0959651815603910
dc.identifier.urihttps://hdl.handle.net/20.500.14669/1750
dc.identifier.volume229
dc.identifier.wosWOS:000362674600002
dc.identifier.wosqualityQ3
dc.indekslendigikaynakWeb of Science
dc.indekslendigikaynakScopus
dc.language.isoen
dc.publisherSage Publications Ltd
dc.relation.ispartofProceedings of The Institution of Mechanical Engineers Part I-Journal of Systems and Control Engineering
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanı
dc.rightsinfo:eu-repo/semantics/closedAccess
dc.snmzKA_20241211
dc.subjectAdaptive estimator
dc.subjectbadly conditioned learning
dc.subjectclosed-loop identification
dc.subjectdiscounted Q-function
dc.subjectparameter convergence analysis
dc.subjectunknown and unstable linear system with random output error-type noise
dc.titleAdaptive estimator design for unstable output error systems: A test problem and traditional system identification based analysis
dc.typeArticle

Dosyalar

Koleksiyon

WoS İndeksli Yayınlar Koleksiyonu
Scopus İndeksli Yayınlar Koleksiyonu