Improvement in deep reinforcement learning controller for buck-boost converter with constant power load

DC-DC converter has been used in the commercial and industrial sectors to step up and down the DC voltage. The increasing development of renewable energy technology, battery storage, and DC microgrids has stressed the importance of the usage of the DC-DC converter. However, DC-DC converters with con...

Full description

Saved in:

Bibliographic Details
Main Author:	Koay, Kevin Chen Rong
Format:	Thesis
Language:	English
Published:	2022
Subjects:	TK Electrical engineering Electronics Nuclear engineering
Online Access:	http://eprints.utm.my/id/eprint/99472/1/KoayChenRongMSKE2022.pdf
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my-utm-ep.99472
record_format	uketd_dc
spelling	my-utm-ep.994722023-02-27T07:24:26Z Improvement in deep reinforcement learning controller for buck-boost converter with constant power load 2022 Koay, Kevin Chen Rong TK Electrical engineering. Electronics Nuclear engineering DC-DC converter has been used in the commercial and industrial sectors to step up and down the DC voltage. The increasing development of renewable energy technology, battery storage, and DC microgrids has stressed the importance of the usage of the DC-DC converter. However, DC-DC converters with constant power load (CPL) have experience instability issues such as voltage and frequency fluctuation. This negative effect is prominent to such an extent that it will cause negative input impedance characteristics that cause destabilize effects in the DC system and sensitive electronic components to be damaged. Many techniques are proposed to mitigate the issue, such as using passive and active damping, but they are limited to cost and physical constraints. Therefore, using an intelligent controller to manage the output of the DC-DC converter is a more attractive solution to the issues. The methods have been implemented in the controller using proportional integral derivative (PID), model predictive control (MPC), machine learning, and deep learning. As the system becomes more complex, methods that used PID and MPC controller have become infeasible to be implemented. Therefore, using machine learning and deep learning is an attractive alternative to solve the control issue. Reinforcement learning (RL) and deep reinforcement learning (DRL) have been used to solve complex control problems such as Proximal Policy Optimization (PPO). This project's purpose is to improve the DRL performance via improving reward function and compare both PPO and AC DRL controllers to analyse the performance during induced CPL by using MATLAB for the simulation. The project has shown that by improving the PPO DRL based long short-term memory (LSTM) network for actor and critic agents with an improved reward system will provide overall all improved performance when compared to the benchmark PID controller. By setting an environment in which the DRL controller is able to train properly, the performance of the buck-boost converter controller by AC and PPO DRL algorithm is compared. PPO DRL showing greater performance in converging to the reference point and a faster training period. Moreover, PPO DRL can demonstrate more robustness and improved voltage stability and settling time of the buck-boost converter without the need to further tune its training parameters. 2022 Thesis http://eprints.utm.my/id/eprint/99472/ http://eprints.utm.my/id/eprint/99472/1/KoayChenRongMSKE2022.pdf application/pdf en public http://dms.library.utm.my:8080/vital/access/manager/Repository/vital:149916 masters Universiti Teknologi Malaysia, Faculty of Engineering - School of Electrical Engineering Faculty of Engineering - School of Electrical Engineering
institution	Universiti Teknologi Malaysia
collection	UTM Institutional Repository
language	English
topic	TK Electrical engineering Electronics Nuclear engineering
spellingShingle	TK Electrical engineering Electronics Nuclear engineering Koay, Kevin Chen Rong Improvement in deep reinforcement learning controller for buck-boost converter with constant power load
description	DC-DC converter has been used in the commercial and industrial sectors to step up and down the DC voltage. The increasing development of renewable energy technology, battery storage, and DC microgrids has stressed the importance of the usage of the DC-DC converter. However, DC-DC converters with constant power load (CPL) have experience instability issues such as voltage and frequency fluctuation. This negative effect is prominent to such an extent that it will cause negative input impedance characteristics that cause destabilize effects in the DC system and sensitive electronic components to be damaged. Many techniques are proposed to mitigate the issue, such as using passive and active damping, but they are limited to cost and physical constraints. Therefore, using an intelligent controller to manage the output of the DC-DC converter is a more attractive solution to the issues. The methods have been implemented in the controller using proportional integral derivative (PID), model predictive control (MPC), machine learning, and deep learning. As the system becomes more complex, methods that used PID and MPC controller have become infeasible to be implemented. Therefore, using machine learning and deep learning is an attractive alternative to solve the control issue. Reinforcement learning (RL) and deep reinforcement learning (DRL) have been used to solve complex control problems such as Proximal Policy Optimization (PPO). This project's purpose is to improve the DRL performance via improving reward function and compare both PPO and AC DRL controllers to analyse the performance during induced CPL by using MATLAB for the simulation. The project has shown that by improving the PPO DRL based long short-term memory (LSTM) network for actor and critic agents with an improved reward system will provide overall all improved performance when compared to the benchmark PID controller. By setting an environment in which the DRL controller is able to train properly, the performance of the buck-boost converter controller by AC and PPO DRL algorithm is compared. PPO DRL showing greater performance in converging to the reference point and a faster training period. Moreover, PPO DRL can demonstrate more robustness and improved voltage stability and settling time of the buck-boost converter without the need to further tune its training parameters.
format	Thesis
qualification_level	Master's degree
author	Koay, Kevin Chen Rong
author_facet	Koay, Kevin Chen Rong
author_sort	Koay, Kevin Chen Rong
title	Improvement in deep reinforcement learning controller for buck-boost converter with constant power load
title_short	Improvement in deep reinforcement learning controller for buck-boost converter with constant power load
title_full	Improvement in deep reinforcement learning controller for buck-boost converter with constant power load
title_fullStr	Improvement in deep reinforcement learning controller for buck-boost converter with constant power load
title_full_unstemmed	Improvement in deep reinforcement learning controller for buck-boost converter with constant power load
title_sort	improvement in deep reinforcement learning controller for buck-boost converter with constant power load
granting_institution	Universiti Teknologi Malaysia, Faculty of Engineering - School of Electrical Engineering
granting_department	Faculty of Engineering - School of Electrical Engineering
publishDate	2022
url	http://eprints.utm.my/id/eprint/99472/1/KoayChenRongMSKE2022.pdf
_version_	1776100599831461888

Improvement in deep reinforcement learning controller for buck-boost converter with constant power load

Similar Items