A deep reinforcement learning framework for energy management of extended range electric delivery vehicles

Rule-based (RB) energy management strategies are widely used in hybrid-electric vehicles because they are easy to implement and can be used without prior knowledge about future trips. In the literature, parameters used in RB methods are tuned and designed using known driving cycles. Although promising results have been demonstrated, it is difficult to apply such cycle-specific methods on real trips of last-mile delivery vehicles that have significant trip-to-trip differences in distance and energy intensity. In this paper, a reinforcement learning method and a RB strategy is used to improve the fuel economy of an in-use extended range electric vehicle (EREV) used in a last-mile package delivery application. An intelligent agent is trained on historical trips of a single delivery vehicle to tune a parameter in the engine-generator control logic during the trip using real-time information. The method is demonstrated on actual historical delivery trips in a simulation environment. An average of 19.5% in fuel efficiency improvement in miles per gallon gasoline equivalent is achieved on 44 test trips with a distance range of 31 miles to 54 miles not used for training, demonstrating promise to generalize the method. The presented framework is extendable to other RB methods and EREV applications like transit buses and commuter vehicles where similar trips are frequently repeated day-to-day.