Demonstration of ultralow energy switching mechanisms is imperative for continued improvements in computing devices. Ferroelectric (FE) and multiferroic (MF) order and their manipulation promise an ideal combination of state variables to reach attojoule range for logic and memory (i.e., ~30× lower switching energy than nanoelectronics). In BiFeO3 (BFO), the coupling between the antiferromagnetic (AFM) and FE order is robust at room temperature, scalable in voltage, stabilized by the FE order, and can be integrated into a fabrication process for a beyond-CMOS (complementary metal-oxide semiconductor) era. The presence of the AFM order and a canted magnetic moment in this system causes exchange interaction with a ferromagnet such as Co0.9Fe0.1 or La0.7Sr0.3MnO3. Previous research has shown that exchange coupling (uniaxial anisotropy) can be controlled with an electric field. However, voltage modulation of unidirectional anisotropy, which is preferred for logic and memory technologies, has not yet been demonstrated. Here, we present evidence for electric field control of exchange bias of laterally scaled spin valves that is exchange coupled to BFO at room temperature. We show that the exchange bias in this bilayer is robust, electrically controlled, and reversible. We anticipate that magnetoelectricity at these scaled dimensions provides a powerful pathway for computing beyond modern nanoelectronics by enabling a new class of nonvolatile, ultralow energy computing elements.