Flowering plants have strikingly distinct genomes, although they contain a similar suite of expressed genes. The diversity of genome structures and organization is largely due to variation in transposable elements (TEs) and whole-genome duplication (WGD) events. We review evidence that chromatin modifications and epigenetic regulation are intimately associated with TEs and likely play a role in mediating the effects of WGDs. We hypothesize that the current structure of a genome is the result of various TE bursts and WGDs and it is likely that the silencing mechanisms and the chromatin structure of a genome have been shaped by these events. This suggests that the specific mechanisms targeting chromatin modifications and epigenomic patterns may vary among different species. Many crop species have likely evolved chromatin-based mechanisms to tolerate silenced TEs near actively expressed genes. These interactions of heterochromatin and euchromatin are likely to have important roles in modulating gene expression and variability within species.