Many consider see-through near to eye displays the successors to the smartphone and envision a multitude of mixed reality and augmented reality applications. The ideal optical imaging system for a see-through near to eye display combines a large field of view (≥ 100°) with a large pupil (≈ 20 mm) and is both lightweight and unobtrusive. In our contribution we first give an overview of challenges related to the design of see-through near to eye displays. Starting from the requirements of the human visual system, we then focus on two main performance parameters: field of view and aperture. These two parameters can be combined in a single parameter, the etendue. We show that the etendue of a see-through near to eye display is comparable to the etendue of lithography lenses and full frame camera lenses. To deliver the same etendue with a much lighter and more compact optical system is one of the main challenges of see-through near to eye displays. We discuss two possible solution paths: to increase the etendue close to the eye and to use foveated imaging concepts.