We quantify the asymptotic vanishing of the ground-state overlap of two non-interacting Fermi gases in d-dimensional Euclidean space in the thermodynamic limit. Given two one-particle Schrödinger operators in finite-volume which differ by a compactly supported bounded potential, we prove a power-law upper bound on the ground-state overlap of the corresponding non-interacting N-Fermion systems. We interpret the decay exponent γ in terms of scattering theory and find γ=π−2∥arcsin|TE/2|∥2HS, where TE is the transition matrix at the Fermi energy E. This exponent reduces to the one predicted by Anderson [Phys. Rev. 164, 352–359 (1967)] for the exact asymptotics in the special case of a repulsive point-like perturbation.