gms | German Medical Science

In the last decade, there have been far-reaching changes to medical studies in Germany. The Licensing Regulations of Doctors (ÄAppO) 2002 defined new content requirements for medical education and greater responsibility was given to the universities for course design and assessing student skills. Since then, innovative approaches to education have modernised the traditional curriculum of many medical schools. At other universities, reformed and model curricula are replacing traditional study formats [1]. The medical faculties of the Humboldt University (Charité) and Witten/Herdecke established reformed model study programs even before the adoption of the ÄAppO in June 2002 [2], and in Heidelberg, the reformed curriculum, the Heidelberger Curriculum Medicinale (HeiCuMed), was introduced for all students in the clinical studies’ phase [3]. Overall, at the start of the last decade, 15 of the 34 medical faculties in Germany had embarked on at least partial redesign of their education models [2].

Various reports from Germany and abroad show problems in the long-term implementation of reformed curricula, in some cases considerable and leading to regression of the reforms. The most common causes of this are organisational problems and the need for training teaching staff. Lack of institutional leadership, overloading the teaching staff with clinical work and research, staff shortages, reduced financial resources and lack of infrastructure due to financial constraints in healthcare, as well as inadequate didactic training of medical teachers have been criticised [4], [5], [6], [7]].

So far, only isolated aspects of the implementation and evaluation of curricular reforms in surgical education in Germany have been published. A comprehensive analysis of success factors is still wanting. In Marburg it was observed that organisational deficits, lack of cross-subject integration of course content, poor linking of teaching and examination content, and lack of incentives for the medical teachers had been substantial in causing the low attendance of surgical lectures [8]. Goettingen reported in 2001 that the surgical examination course and the surgical practical course including E-learning, skills training and PBL were better evaluated than the previous regular course although the organisational effort had been great and the integration of course contents was still suboptimal [9]. The Charité observed that less teaching effort was required in PBL-centred surgical practicals than in the regular practical course running in parallel [10]. In Erlangen, the OSCE results improved significantly after the practice aids had been standardised and consistent instructions were provided to the students [11]. Heidelberg reported that surgical teaching in Hei-CuMed was consistently evaluated as positive by the students [12].

The traditional curriculum for surgery in Heidelberg [13] and the reformed HeiCuMed curriculum implemented in 2001 [3], [14] have already been described. HeiCuMed is a theme-centred modular rotational course, which is composed of five thematic blocks lasting 14-20 weeks each. Students take the modules in groups of 25-35 to ensure continuous supervision of students in small groups. The clinical theoretical modules are offered five times a year and the clinical modules are repeated 8-10 times a year.

The surgical section of HeiCuMed lasts 12 weeks with 20-30 teaching units offered per week. It is run with a half-cohort of about 150 students during their sixth or seventh semester. It consists of five modules including the interdisciplinary clinical-pathological conference and emergency medicine. The teaching methodology of the modules includes interactive seminars with an emphasis on case-based, practice-oriented group work. PBL, practical exercises with standardised patients, simulation workstations in skills labs, and bedside teaching constitute the methodological framework. Chalk and talk teaching is limited to a single cardinal symptoms oriented lecture series with one teaching unit per day. The interdisciplinary partner subjects of pathology and radiology are integrated into the modules.

With all the demands of a university clinic, teaching often receives low priority [5]. The reformed HeiCuMed curriculum is based on institutional and structural organisational measures to ensure that teaching is made a priority and that the quality of teaching is increased consistently. The faculty’s own training program for medical teaching staff provides the medical teachers with didactic-methodological tools, helps them to prepare for classes and thereby is expected to have a motivating effect. The course content, teaching materials and training of standardised patients have been standardised, the organisation of modules has been centralised in the thematic blocks and technical support for teaching has been ensured. These measures, along with exemption of the medical teachers from clinical duties for the duration of the module are there to enable medical teachers adequate integration into the teaching and promote their personal contact with the students. A scientific and educational-psychological quality assurance team analyses the evaluation of teaching, sits in on courses, and provides guidance for medical teaching staff.

Currently there is a lack of literature on the long-term success of reform concepts in Germany beyond the implementation phase. For this reason, in the present study longitudinal evaluation of the surgical block of HeiCuMed has been performed.

The work summarises evaluation data from six and a half years or 13 runs of the surgical block of HeiCuMed and carries out a differentiated comparison with evaluation data from the traditional curriculum. It investigated

• if and which teaching forms of the surgical block of HeiCuMed result in sustained effectiveness,
• if the developed organisational structures are sufficient to ensure a sustained improvement of the student evaluation of the teaching program in comparison with the traditional curriculum.

The participants of the study (see Table 1 [Tab. 1]) were students of the Medical Faculty of the Heidelberg School of Medicine at the surgical stage of their studies between October 1999 and February 2008. Student data from the traditional curriculum are taken from the database of Schuerer et al. [13].

The data was retrieved from evaluation questionnaires which had been completed anonymously by the participants, either in writing (1999-2002) or digitally (as of 2003). Personal identification of the data was not possible.

The Heidelberg Inventory for Course Evaluation (HILVE) has already been published [13]. From March 2005 it was replaced by abbreviated questionnaires adapted to the particular courses. The evaluation included structure and content of the courses, performance of the medical teaching staff, and self-assessment of the personal learning progress. Three versions were used which were differentially adapted to the evaluation of the cardinal symptom lecture, the clinical-pathological conference and the modules. The current work is limited to the evaluation of responses to key issues which were in common to all instruments. The relevant questions are given in the results’ section.

The evaluations were based on a seven-point Likert scale with 1=highest and 7=worst rating score or vice versa as noted in the results’ section. For the analysis of the same questions from different questionnaires, the scales were adjusted to each other if necessary by means of reverse transformation.

The evaluation data was tabulated, grouped and evaluated in Microsoft Excel^®. One-way analysis of variance (Anova), two-sided t-test for independent samples and the calculation of Cohen’s d were performed using Excel or SPSS^®16. In the case of a generally significant difference among means (Anova) paired comparisons using t-tests were sometimes also performed. Graphics were generated in Excel^® and processed in Canvas^® (ACD Systems). To assess the reliability of the evaluation data, Cronbach’s α (for raw and standardised data), inter-item co-variation, inter-item correlation, and intra-class correlation (single and average measures) were calculated in SPSS. With one exception (Practicals in 2000, n=76), the analysis was performed using a random sample of 253 students per semester, which represented the different teaching events. The sample size was limited by the width of the Excel and SPSS tables and comprised 16-51% of the data of each semester.

The presentation of the HeiCuMed results, due to space issues, was largely limited to the last 6 semesters of the observation time. By then, the structures of HeiCuMed had achieved stability and most of the medical teaching staff had completed the teacher training.

To assess the inter-rater reliability of the evaluations, several criteria were used (see Table 2 [Tab. 2]). Cronbach’s α in all semesters was 0.99 or higher, whereby these high values in part can be attributed to the large sample size. The inter-item correlations ranged from 0.336 to 0.655 and often exceeded 0.45. The intra-class correlations (0.333 - 0.666, single measures) were relatively high and permitted the use of means in the subsequent analysis [15].

The average overall evaluations of the surgical courses were relatively high throughout the period. The worst overall evaluation rating on a seven-point Likert scale (1=best, 7=worst rating score), 2.46±1.133 (mean±SD), was observed in the traditional curriculum in the summer semester 2000. Following the introduction of HeiCuMed in October 2001, the overall average evaluation score improved for the most part to a level below 2.0 (see Figure 1A [Fig. 1]). The percentage of Grade 1 (best) increased from 12-14% in the traditional curriculum to 30-40% in most cases following the introduction of the new curriculum (see Figure 1B [Fig. 1]). Even clearer was the difference between the assessment of the traditional and the new curriculum if only the practical courses were considered. In the traditional curriculum practical courses only included ward teaching (bedside teaching). The range of practical training in HeiCuMed, on the other hand, includes the teaching of practical skills in skills labs (for example physical examination techniques, sewing, knotting, catheterisation, placement of feeding tubes), bedside teaching, and communication skills using standardised patients. Training with standardised patients also includes taking patient histories and physical examinations. The average rating for these classes under the new curriculum reached the score of 1.8 several times (see Figure 1C [Fig. 1]). The percentage of Grade 1 (best) was up from 12-14% under the traditional curriculum to over 45% as of summer semester 2006 (see Figure 1D [Fig. 1]).

The evaluation of the statement “The event was well organised” under HeiCuMed was significantly better (2.14±0.491, mean±SD, range 3.15 to 1.22) with an average effect size of Cohen’s d 0.43±0.041 compared to the traditional curriculum (2.89±0.918, range 4.78 to 1.74, p=5.6×10^-5).

In the overall evaluation and evaluation of the various teaching methods, the proportion of very satisfied students (Grade 1, dark green in Figure 1B, 1D [Fig. 1] and Figure 2B [Fig. 2]) in HeiCuMed was 30-50%, significantly higher than in the traditional curriculum. Up to 18% of the respondents in the traditional curriculum but less than 10% of the respondents in HeiCuMed expressed dissatisfaction (Grades 4-6, orange, red and purple in Figure 1B, 1D [Fig. 1] and Figure 2B [Fig. 2]). 20-30% of the respondents in the traditional curriculum and 10-20% of those under HeiCuMed gave neutral scores (Grades 2-3, light green and yellow in Figure 1B, 1D [Fig. 1] and Figure 2B [Fig. 2]) to the respective course formats.

The overall average evaluation of lectures (2.43±0.095, mean±SEM) and bedside teaching (2.46±0.043) in the traditional curriculum was almost the same (see Figure 2A [Fig. 2] and Table 3 [Tab. 3]). In comparison with these two components of the traditional curriculum, the ratings for the different elements of the new curriculum, cardinal symptoms lecture (1.99±0.127), seminar (2.13±0.130), practical units (1.87±0.141), bedside teaching (1.86±0.109), working with standardised patients (2.22±0.347) and PBL (1.98±0.205) were significantly and consistently better. The effect size was predominantly moderate (Cohen’s d from 0.25 to 0.55, see Table 3 [Tab. 3]).

The practical teaching units and bedside teaching of HeiCuMed received the best overall scores over the investigation period (see Figure 2A, 2B [Fig. 2]) and reached the highest effect size, Cohen's d of 0.5, compared to bedside teaching under the traditional curriculum (see Table 3 [Tab. 3]). The evaluation of the work with standardised patients was strikingly variable (1.79- 2.28, see Figure 2A [Fig. 2]) and had a low effect size of 0.25 to 0.29 (see Table 3 [Tab. 3]). The reason for this appears to be a variable increase of negative evaluations under HeiCuMed compared with bedside teaching of the traditional curriculum. The increase in negative evaluations in HeiCuMed was accompanied by a reduction of neutral feedback, while the relative share of very positive evaluations was similar to that of other teaching formats under the reformed curriculum (see Figure 2B [Fig. 2]).

Amongst the teaching formats dealing with cognitive content and applied theory, the cardinal symptoms lecture and PBL achieved similar mean total grades (see above), the evaluation of the seminar was slightly lower. Interestingly, PBL achieved a higher proportion of positive grades (see Figure 2B [Fig. 2]), while the evaluation of the cardinal symptoms lecture had a higher effect size (Cohen’s d 0.43) over the traditional curriculum (see Table 3 [Tab. 3]).

The contribution of courses to the subjective gain of knowledge was analysed on the basis of the statement “I have learned something meaningful and important”. The mean score (±SD) for this statement in the traditional curriculum was 2.50±0.613 (range 4.20 to 1.80). In the early years of HeiCuMed before summer semester 2006 the assessment of this statement was only slightly and not significantly better (2.25±0.504, range 3.07 to 1.20, p=0.19). From summer semester 2006 onwards, however, its average rating increased to 2.00±0.488 (range 3.24 to 1.11), significantly higher than under the traditional curriculum (p=0.0042), and had an average effect size of Cohen’s d 0.39±0.086.

The reformed curriculum HeiCuMed represents a comprehensive innovation of content and structure of medical teaching in the Medical School of Heidelberg University. An essential prerequisite for the success of the surgical training curriculum was a large number of experienced professional teaching staff with training in educational methods, central coordination of the surgical curriculum, and support for active student participation through interactive didactics. The organisational and financial effort required presents large university hospitals which run at capacity with a special challenge [4], [5], [6]. In fact, several medical reform curricula around the world experienced regression for this reason [7]. The success achieved by HeiCuMed depends not only on the relevance of its content and the quality of its educational strategy but also on the institutional ability to set up the necessary structures and to implement the various facets of the project effectively in the long term. These structures include first and foremost the package of measures for training the medical teaching staff and reducing their workload during the teaching period. It includes didactic training of the teaching staff, modular design of the courses, releasing teaching staff from clinical duties for the duration of the module, and standardisation of the teaching content and the support materials. Long-term securing and equipping appropriate teaching rooms as well as the recruitment and training of standardised patients in sufficient numbers are also important.

Student evaluation of courses has been established in Germany largely as part of quality assurance. One aspect limiting this type of evaluation is that the quality of teaching is difficult to measure, thus making it difficult to validate the evaluation [16]. Student evaluation is based on a subjective assessment of teaching that varies among groups of students with different learning strategies [17]. Agreement between the assessment of a particular aspect of teaching by different students in different classes may be low [18], [15], but increase considerably if the number of evaluators per event is larger than 20 and if the conditions of the evaluated events are similar [18], [19]. To ensure the reliability of evaluation results in the present study, large samples were analysed with conditions of the evaluated courses being comparable.

In 2004-2005 interviews were conducted with eleven students of the traditional curriculum and 25 of the first HeiCuMed students who had by then reached the Internship Year [12]. Of the eleven graduates of the traditional curriculum, only one (9%) judged the success of learning as “high” and the organisation as “very good”. Three interviewees (27%) judged the relevance of the courses to future working practice as positively. Among the Hei-CuMed graduates, however, 18 (72%) judged their learning success as “high”, 22 (88%) rated the relevance to working practice as positive, and 19 (76%) rated organisation as being very good. The results of these interviews were consistent with the results of written evaluations carried out around that time and suggested that the benefits of HeiCuMed persist into the Internship Year [12].

Previous studies from different universities have shown high satisfaction amongst students with innovative reforms of the medical course. However, usually only one or a few aspects of the curriculum in its launch phase or within a limited period of time were investigated. It remained unclear whether initial or single point student satisfaction extended to the whole curriculum and whether it was shared by subsequent cohorts. The present study therefore aims to answer these questions for the entire reformed surgical curriculum and over a long period of time.

The data shows that the high level of the students’ overall assessment of the surgical block of HeiCuMed was sustained consistently and with little change throughout the observation period of over six years. The proportion of very satisfied students increased several-fold while the percentage of dissatisfied students, in comparison with the traditional curriculum, shrank. This observation is of special importance because even the traditional surgical curriculum in Heidelberg was relatively highly rated by the students [13].

Under each curriculum, the evaluation of each item included positive, neutral and negative ratings. This variability partly was due to differences in the perception of the quality of the different courses and the medical teachers. But it potentially also expressed different approaches to learning: those that prefer teacher-centred learning and those who favour integrative forms of learning.

Divergent approaches to learning and varying perceptions have a limiting impact on the effect size and limit downwards the evaluation variances. In addition, the improvement of the average evaluation results between the traditional and reformed curriculum was limited by the fact that the average evaluation of the traditional curriculum had already been relatively good. In conjunction, this means that in spite of the marked increase in the proportion of very satisfied students under HeiCuMed and the high significance of the differences between the evaluations of both curricula, the effect size remained within a medium range.

The questions, how large is the proportion of learners who cannot cope with a given teaching method, how these students could be integrated into the active learning process, and what effect this proportion of learners has on the teaching evaluation remain open for further investigations.

The practical teaching units and bedside teaching consistently were given the best reviews in the reformed curriculum. Thus, they appear to represent advantageous didactic strategies as compared with the traditional teaching methods. This conclusion has special significance for the teaching under HeiCuMed, which is based on the concept that practical experience encourages case-based application, cross-linking, and consolidation of acquired knowledge. Practical teaching units are an integral part of the daily cycle of lessons under HeiCuMed and the amount of bedside teaching is at least three-fold as large as in the old curriculum.

More than a third of all HeiCuMed respondents rated working with standardised patients and PBL as “very good” and three quarters of all HeiCuMed respondents were satisfied with this course format. On average, the cardinal symptoms lecture and the seminars of HeiCuMed were judged similarly although the cardinal symptoms lecture had a higher effect size due to a smaller variance.

The HeiCuMed seminars were rated significantly better than the lectures of the traditional curriculum, but less well and with higher variability than the practical teaching formats of HeiCuMed. One explanation given by students in free-text answers was that seminar contents that had already been covered in other classes were redundant. Redundancy has also often been perceived with respect to case-based seminars in topics where previously only the theoretical foundation had been laid. A principle of medical education, however, is to approach the same disease from different medical perspectives and to consider it with a view to practical application.

Overall, it may be concluded that the daily cycle of thematic teaching units of HeiCuMed with its alternating teaching formats, interactive seminars, PBL, and different practical approaches has consistently proven itself.

Under the traditional curriculum, the organisational effort to ensure teaching is relatively low. The organisational concept of HeiCuMed is much more complex and includes three main pillars: organisation, teaching, and quality management.

Organization in the field of surgical teaching aims at enabling the medical teachers to fully commit to teaching for defined periods of time and facilitating their response to the individual needs of a student group through teaching continuity. In addition, HeiCuMed is organised so, that several classes with various but co-ordinated methodologies and content may be held simultaneously.

As already discussed above, complex organisational requirements can contribute to the regression of efforts to reform the medical course [7]. The present results confirm that the successful implementation of organisational measures is essential for a successful learning process as evaluated by the students. Consequently, the organisational structure of the surgical curriculum of HeiCuMed can be regarded as a successful model for the implementation of a reformed curriculum at a large university hospital with strong competing demands including patient care, research, and teaching.

The development of the reformed medical course is an ongoing dynamic process that will continue to be observed scientifically. It is important to reach the highest possible proportion of the students through the courses and to support their individual learning process. This makes it necessary to integrate more individual factors of the students into the educational process alongside the course evaluation. We recently therefore launched a study on the factors which determine the motivation of the students including both curricular issues and various aspects of the academic environment. In addition, we have recently established an integrative teaching consultancy service. It includes giving didactic advice to teaching staff by a psychological-educational team that visits their classes (for more information see http://www.medizinische-fakultaet-hd.uni-heidelberg.de/Integrative-Lehrberatung.110222.0.html). The integrative analysis of the data mentioned above together with the student evaluation of the courses and the professionalisation of the teachers by regular didactic-methodological training aims to enable a comprehensive quality management system with the goal of continuous quality improvement and securing the sustainability of the implemented educational structures. At the same time, this constitutes a transferable model for quality management in higher education.

The authors are grateful to the psychologists Gerald Wibbecke and Janine Kahmann for critically proofreading the manuscript and for their fruitful discussions.

The authors declare that they have no competing interests.