Orthogonal frequency division multiplexing (OFDM) is prone to frequency offset which gives rise to intercarrier interference (ICI). Recently, a novel OFDM transmission scheme called OFDM with index modulation (IMOFDM) is proposed, which outperforms conventional OFDM in the absence of frequency offset. As in IM-OFDM, partial subcarriers are set to be idle by spatial modulation, the potential of IM-OFDM in ICI reduction is expected. In this paper, we find that this potential vanishes in some real scenarios with ICI, which leads to IM-OFDM exhibiting even worse performance than conventional OFDM. To improve this situation, a novel ICI cancellation scheme is designed, which integrates the ICI self-cancellation technique into the IM-OFDM framework. Via careful designs, the proposed scheme not only provides a solution to ICI problems of IM- OFDM, but also achieves an attractive tradeoff between the spectral efficiency and ICI cancellation performance of the system. Simulations validate that in the presence of carrier frequency offset (CFO), the proposed scheme significantly outperforms existing IM-OFDM and more importantly shows better performance than the conventional OFDM with ICI self-cancellation.