We present a theoretical investigation of the lateral shift of an infrared light beam reflected from a magnetic film deposited on a non-magnetic dielectric substrate, taking into account the linear magneto-electric interaction in the magnetic film. We use the stationary phase method to evaluate the lateral shift. It is shown that the magneto-electric coupling leads to a six-fold enhancement of the lateral shift amplitude of a p-(s-) polarized incident beam reflected into a s-(p-) polarized beam. A reversal of the magnetization in the film leads to a nonreciprocal sign change of the lateral shift.