A new complete active space configuration interaction (CASCI) method was recently introduced that uses state-averaged natural orbitals from the configuration interaction singles method (configuration interaction singles natural orbital CASCI, CISNO-CASCI). This method has been shown to perform as well or better than state-averaged complete active space self-consistent field for a variety of systems. However, further development and testing of this method have been limited by the lack of available analytic first derivatives of the CISNO-CASCI energy as well as the derivative coupling between electronic states. In the present work, we present a Lagrangian-based formulation of these derivatives as well as a highly efficient implementation of the resulting equations accelerated with graphical processing units. We demonstrate that the CISNO-CASCI method is practical for dynamical simulations of photochemical processes in molecular systems containing hundreds of atoms.