I am a digital electrical engineering expert with a strong focus on Artificial Intelligence, specializing in machine learning, deep learning (DL), and reinforcement learning (RL). My work centers on developing intelligent, AI-driven models for prediction, control, and optimization—particularly in energy management systems for electric vehicles, a critical domain in the advancement of sustainable transportation.

As a Ph.D. graduate from Carleton University and a member of the Intelligent Robotic and Energy Systems (IRES) Research Group, I led several research initiatives applying RL and DL techniques to real-world control challenges. My contributions include designing multi-agent reinforcement learning architectures, building CNN-LSTM models for predictive control, and implementing advanced state estimation techniques for hybrid energy storage systems.

With extensive hands-on experience in Python and its scientific ecosystem—including TensorFlow, Keras, Stable-Baselines3, Ray, Scikit-learn, NumPy, and Pandas—as well as MATLAB/Simulink, I develop end-to-end solutions that bridge cutting-edge research with practical deployment. My goal is to apply AI to solve complex engineering problems and drive scalable, intelligent, and sustainable energy systems.

Armin Lotfy

armin.lotfy@carleton.ca

Postdoctoral Research Fellow

RL and DL Researcher

System and Computer Department
Carleton University
Ottawa
Canada

Google Scholar
Github
Orcide
LinkedIn

Projects

  • Energy Management of Battery Electric Vehicles Using MATD3 Algorithms

    This project focused on developing and implementing a Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3)-based Energy Management System (EMS) for Battery Electric Vehicles (BEVs). The primary objective was to enhance the driving range of BEVs by integrating an advanced MATD3-EMS with battery cell balancing techniques. The project utilized the MATLAB environment to design, simulate, and optimize the proposed system, aiming to achieve efficient energy utilization and extended battery longevity.

  • Centralized Multi-Agent SOC Control for Battery Health Using Proximal Policy Optimization in EVs

    This project proposes a model-free cooperative multi-agent control framework designed to regulate and balance the SOC of lithium-ion battery (LIB) cells in EVs during real-time driving operations. The proposed method utilizes a series architecture comprising three LIB cells, each equipped with a buck-boost converter and a proportional-integral (PI) controller, controlled by a reinforcement learning (RL) agent. The Proximal Policy Optimization (PPO) algorithm is used as the RL agent in this multi-agent framework, where each PPO agent independently manages the SOC of a corresponding battery cell based on observed data. During the training phase, all PPO agents work collaboratively to balance the SOCs of the LIB cells, thereby preventing interruptions in EV performance. The effectiveness of the proposed approach is demonstrated by comparing its performance with single-agent methods such as PPO, Soft Actor-Critic (SAC), and Twin Delayed Deep Deterministic Policy Gradient (TD3), as well as with other multi-agent methods. The results show that the proposed method performs better than the existing approaches, indicating its potential for superior performance.

  • Smart Grid PPO-based Power Controller

    In this project, a Proximal Policy Optimization (PPO)-based power-sharing controller was designed and implemented in MATLAB. The controller is robust to unforeseen fault scenarios and demonstrates adaptability in dynamically adjusting power distribution under varying conditions. By leveraging the PPO algorithm, the controller effectively handles uncertainties and ensures reliable power-sharing performance across different operational scenarios, contributing to improved system resilience and efficiency.

  • Predictive Controller of Battery Electric Vehicles Based on MAPPO (Current Project)

    This project focuses on developing and implementing a battery controller in Python using libraries such as Stable-Baselines3, Gymnasium, Scikit-learn, and TensorFlow. The primary objective is to design a Reinforcement Learning (RL) and Deep Learning (DL)-based Energy Management System (EMS) capable of adapting to various observation signals and predicting optimal control signals in real time. The system under test is a Battery Electric Vehicle (BEV) equipped with a hybrid energy storage system comprising a battery and a supercapacitor, aimed at achieving efficient energy utilization and enhanced performance.