The forthcoming generation of many-core architectures suggests a strong paradigm shift in the way algorithms have been designed to achieve maximum performance in reservoir simulations. In this work, we propose a novel poly-algorithmic solver approach to develop hybrid CPU multicore and GPU computations for solving large sparse linear systems arising in realistic black oil and compositional flow scenarios. The GPU implementation exploits data parallelism through the simultaneous deployment of thousands of threads while reducing memory overhead per floating point operations involved in most BLAS kernels and in a suite of preconditioner options such as BILU(k), BILUT and multicoloring SSOR. On the other hand, multicore CPU computations are used to exploit functional parallelism to perform system partitioning and reordering, algebraic multigrid preconditioning, sparsification and model reduction operations in order to accelerate and reduce the number of GCR iterations. The efficient orchestration of these operations relies on carefully designing the heuristics depending on the timestep evolution, degree of nonlinearity and current system properties. Hence, we also propose several criteria to automatically decide the type of solver configuration to be employed at every time step of the simulation. To illustrate the potentials of the proposed solver approach, we perform numerical computations on state-of-the-art multicore CPU and GPU platforms. Computational experiments on a wide range of highly complex reservoir cases reveal that the solver approach yields significant speedups with respect to conventional CPU multicore solver implementations. The solver performance gain is of the order of 3x which impacts in about 2x the overall compositional simulation turnaround time. These results demonstrate the potential that many core solvers have to offer in improving the performance of near future reservoir simulations.