High power short pulse lasers provide a promising route to study the strong field effects of the quantum vacuum, for example by direct photon–photon scattering in the all-optical regime. Theoretical predictions based on realistic laser parameters achievable today or in the near future predict scattering of a few photons with colliding Petawatt laser pulses, requiring single photon sensitive detection schemes and very good spatio-temporal filtering and background suppression. In this article, we present experimental investigations of this photon background by employing only a single high power laser pulse tightly focused in residual gas of a vacuum chamber. The focal region was imaged onto a single-photon sensitive, time gated camera. As no detectable quantum vacuum signature was expected in our case, the setup allowed for characterization and first mitigation of background contributions. For the setup employed, scattering off surfaces of imperfect optics dominated below residual gas pressures of 1 × 10−4 mbar. Extrapolation of the findings to intensities relevant for photon–photon scattering studies is discussed.