We study the structure of the orbital order Γ=dxz†dxz-dyz†dyz in FeSe in light of recent scanning tunneling microscopy and angle-resolved photoemission spectroscopy (ARPES) data, which detect the shapes of the hole and electron pockets in the nematic phase. The geometry of the pockets indicates that the sign of Γ is different between the hole and electron pockets (Γh and Γe). We argue that this sign change cannot be reproduced if one solves for the orbital order within the mean-field approximation, as the mean-field analysis yields either no orbital order, or order with the same sign of Γe and Γh. We argue that another solution with the opposite signs of Γe and Γh emerges if we include the renormalizations of the vertices in the d-wave orbital channel. We show that the ratio |Γe/Γh| is of order one, independent of the strength of the interaction. We also compute the temperature variation of the energy of dxz and dyz orbitals at the center of electron pockets and compare the results with ARPES data.
Bibliographical noteFunding Information:
We thank A. Coldea, L. Basconses, L. Benfatto, S. Borisenko, D. Chichinadze, R. M. Fernandes, J. Kang, T. K. Kim, W. Ku, L. de' Medici, M. Watson, and Y. M. Wu for useful discussions. R.X. and A.V.C. are supported by the Office of Basic Energy Sciences, U.S. Department of Energy, under Award No. DE-SC0014402. L.C. acknowledges support from the Alexander-von-Humboldt Foundation. Work at BNL is supported by the U.S. Department of Energy (DOE), Division of Condensed Matter Physics and Materials Science, under Contract No. DE-SC0012704. Part of the work was done while A.V.C. was visiting KITP in Santa Barbara. KITP is supported by NSF under Grant No. NSF PHY17-48958.