This paper addresses the need for a new approach to the educational evaluation of software that falls under the rubric "new media" or "multimedia" as distinct from previous generations of Computer-Assisted Language Learning (CALL) software. The authors argue that present approaches to CALL software evaluation are not appropriate for a new genre of CALL software distinguished by its shared assumptions about language learning and teaching as well as by its technical design. The paper sketches a research-based program called "E/Valuation" that aims to assist language educators to answer questions about the educational effectiveness of recent multimedia language learning software. The authors suggest that such program needs to take into account not only the nature of the new media and its potential to promote language learning in novel ways, but also current professional knowledge about language learning and teaching.