A Systematic Review of Publications Using the Dundee Ready Education Environment Measure (DREEM) to Monitor Education in Medical Colleges in Saudi Arabia

Background

In recent years, the Saudi government has attempted to expand the healthcare professional education infrastructure in accordance with the Saudi Vision 2030 plan. Of the 28 government universities and 9 private universities in SA [1], 20 government and 5 private universities offer undergraduate health professional programs [2]. Students in professional education programs in SA are supported through free and subsidized programs [3,4].

The educational environment is the academic and social environment of an institution, which affects how students learn [5]. A safe, valued, participatory, and supportive educational environment leads to more interactions [6,7]. An educational environment that respects the choice and voice of students and focuses on the learning of all students makes students more involved in teaching and learning activities [8]. Over the past 50 years, researchers in teaching, learning, and education have attempted to define and measure the health educational environment [9–11]; the Dundee Ready Educational Environment Measure (DREEM) is almost certainly the most popular inventory used today [7,12]. DREEM has been useful and effective in assessing medical colleges, applied medical science colleges, dental colleges, chiropractic learning, teaching, and educational environments [5,7,12].

In mid-1998, the World Federation for Medical Education made the teaching and learning environment one of the factors used for assessing health education programs [13]. Educators in different health specialties and disciplines often agree that learning, teaching, academic, and clinical educational environment concerns are important factors in health students’ beliefs, knowledge, capabilities, improvement, achievements, and behaviors [7,14]. Evaluation of the educational environment for males and females at various academic levels or teaching phases (preclinical and clinical) is key to assuring high-quality, student-centered programs [5,7,9].

In 1997, Roff et al introduced and developed a suitable instrument with a validation of a universal inventory that measures and assesses the health training settings and medical schools’ environments with a diagnostic tool to evaluate the condition of their school’s teaching and learning environment. This is called the DREEM inventory [7,15]. It is a multidimensional and multicultural tool that can measure the 5 fundamental domains of educational environments, namely, students’ perceptions of learning (SPL), students’ perceptions of teachers (SPT), students’ perceptions of atmosphere (SPA), students’ academic self-perception (SASP), and students’ social self-perception (SSSP) [5,7,12]. DREEM is a 50-item questionnaire that covers a wide range of topics directly relevant to the educational environment (Table 1). It is a closed-ended questionnaire that is scored on a 5-point Likert scale, ranging from strongly agree to strongly disagree. The questionnaire has a maximum score of 200, which indicates the respondent thinks is the ideal educational environments. DREEM was developed by a Delphi panel, and it has been used to highlight the weaknesses and strengths of educational environments in dental colleges, medical colleges, and applied medical science colleges. It has also been translated and copied to many languages, such as Turkish, Romanian, Spanish, Greek, Urdu, and Arabic [16–26]. Table 2 shows how to interpret the overall DREEM score and its 5 domains.

Internationally, not many systematic reviews of DREEM studies have been conducted [15,19,27–30]. Alqahtani (2022) recently published a single systematic review to compare the overall DREEM and domain scores between genders in the educational environment of public and private dental colleges and medical colleges in SA universities [31]. The Dundee Ready Education Environment Measure is a validated tool to evaluate educational environments of all health education setting. This systematic review aimed to identify and evaluate publications using DREEM to monitor the education environment in Saudi Arabian medical, applied medical science, and dental colleges.

Material and Methods

RESEARCH PROTOCOLS AND INCLUSION AND EXCLUSIVE CRITERIA:

The authors created the current search plan by reviewing published papers that used the DREEM inventory to measure educational environment and perceptions in terms of state, context, and population. The research questions were: “What is the overall value of DREEM and its domain scores in medical colleges, applied medical science colleges, and dental colleges in different private and public SA universities, as well as its significance in terms of gender and educational levels or preclinical and clinical phases?” and “Is the overall mean of DREEM and its domains among colleges different in terms of gender and educational level?” Hence, only studies that directly used DREEM on health studies students from different SA cities and were published in English were included. Papers that were published in languages other than English or used indices or inventories other than DREEM to evaluate educational environment were omitted.

SEARCH STRATEGY:

Electronic databases of PubMed, ScienceDirect, Scopus, Wiley Library, and Web of Science were searched for relevant papers published up to October 2022. The search terms used were “DREEM inventory,” “educational environments,” “students’ perception,” “DREEM items,” “DREEM domains,” and “Medical, Applied Medical Science, Dental Colleges”, and “Health specialty students in SA.” Keywords were used independently or in combination by using the Boolean operators “AND,” “OR,” and “NOT” to search for the terms “DREEM,” “educational environment,” and “students’ perception” independently. The gray literature was explored using Google Scholar.

STUDY SELECTION:

Two reviewers (F.A. and M.M.AL.) individually assessed articles for eligibility based on inclusion procedures. After duplicates were removed, the titles and abstracts of the remaining articles were analyzed for relevance. A third reviewer (B.M.M) studied the validity and repetitions of the papers. Systemic reviews and case reports that did not assess educational environment or students’ perceptions among different health specialties were removed. Published articles that did not indicate the number of participants or whose samples had been relatively anticipated in other studies were also removed. Research that addressed the study questions was deemed relevant. The full text of potentially relevant research was reviewed. The references of the selected published research were checked to see if other studies met the inclusion criteria. In case of a disagreement whether to include an article, a third reviewer (B.M.M) was asked.

DATA EXTRACTION AND ANALYSIS:

Two authors (F.A. and M.M.AL.) summarized significant data from each article using customized tables in Microsoft Excel (Microsoft Corp., Redwood, CA, USA). Any disagreements were resolved through discussion with a third author (B.M.M.). Details included authors’ names, publication year, college type (medical colleges, applied medical science colleges, and dental colleges), university and city names, sample size, response rate (RR), male-to-female ratio, educational levels, and preclinical or clinical phases. Moreover, overall DREEM and domain scores, as well as their interpretations, are presented. Finally, the significance of gender and educational levels was reported.

QUALITY OF INVOLVED PAPERS:

Two authors (F.A. and M.M.AL.) judged the value of included published articles based on the Joanna Briggs Institute’s critical appraisal tool for cross-sectional studies [35,36]. Any differences were resolved through discussion until consensus was reached with a third author (B.M.M.). The exterior validity of papers was evaluated through criteria, such as the overall DREEM and domain values and their interpretation, as well as the significance of gender, educational levels, and statistical methods used.

Results

STUDY SELECTION:

The search strategy generated 546 results. A total of 450 articles were excluded because they were duplicates or unrelated to this review. The remaining papers were assessed based on their titles and abstracts. In addition, 56 published articles were removed because they were not conducted in SA (36) or used other indices (20). This review assessed the full text of 40 articles that met all of the inclusion criteria based on predefined eligibility criteria. The included articles used DREEM in 25 medical colleges, 5 applied medical science colleges, and 10 dental colleges. Figure 1 shows how these articles were selected for this review.

QUALITY OF SELECTED PAPERS:

The evaluation of the risk of bias revealed that more than half of the 40 articles had methodological limitations, resulting in an overall rating of medium to high risk of bias. Most of the papers used objective measures to determine DREEM and its 5 domains. In addition, some medical college studies did not mention the means and values of the 5 domains. The risk of bias assessment showed that all 40 included papers suffered from methodological weaknesses [35,36], leading to an overall high risk of bias. Most of the studies showed attrition bias with a lack of responses. There was limited information on how the response rate was managed. Table 3 contains the summary of the risk of bias.

STUDY CHARACTERISTICS:

A total of 25 medical colleges studies remained, including 7 in Riyadh, 5 in Jeddah, and 3 each in Dammam, Jazan, and Al Maddinah Al Manwareh [37–61]. Applied medical science colleges studies (5) were conducted in Abha, Jeddah, Riyadh, Al Maddinah Al Manwareh, and Makkah [62–66]. Lastly, in the 10 dental studies, 3 were performed in Al Maddinah Al Manwareh and 2 were conducted in Jazan [20–21,67–74]. Table 3 summarizes the characteristics of the selected papers’ DREEM and its 5 domains conducted in different health specialties in SA.

PUBLICATION YEARS, RR, AND GENDER RATIO: Most of the selected papers were published between 2011 and 2019 in all colleges (medical colleges, dental colleges, and applied medical science colleges). The percentages of the published articles in 2020 and later were 40%, 20%,16% for dental colleges, applied medical science colleges, and medical colleges, respectively. In relation to RR, the highest percentage was registered in applied medical science studies (98%), followed by dental (77%) and medical studies (72%). The male-to-female ratio in dental colleges participants was equal, whereas the ratio of females was higher in medical colleges and applied medical science colleges (80% and 62%, respectively) compared with males (20% and 38%, respectively) (Figure 2).

OVERALL DREEM AND SPL, SPT, SASP, SPA, AND SSSP DOMAIN VALUES: Table 4 and Figure 3 show the interpretations of the overall value of DREEM and its mean. The overall DREEM scores were 80%, 75%, and 70% for medical colleges, applied medical science colleges, and dental colleges, respectively, which indicates that most of them were “more positive than negative”, with values between 101 and 150. Meanwhile, a single study recorded as “Excellent,” with values between 151 and 200, and it was conducted in medical colleges in UQU in Makkah City.

Regarding the domains, for the educational environment of SPL, the values for dental colleges, medical colleges, and applied medical science colleges were 90%,76%, and 50%, respectively. These percentages imply that SPL is “a more positive approach, with values of 25–36 out of 48”. For SPT, the percentages were 88%, 75%, and 70% for medical colleges, applied medical science colleges, and dental colleges, which indicates that it is “moving in the right direction, with values of 23–33 out of 44”. In addition, a single study for SPT recorded 4%, which implied “model course organizer” and with values of 34–44. Regarding the SASP domain, the maximum values were almost equal for all colleges (75–80%), indicating that SASP is “feeling more on the positive side” in the educational environment. For SPA, the proportions were slightly higher in medical colleges (76%) compared with the 60% for applied medical science colleges and dental colleges, which indicated that the educational environment is “a more positive atmosphere”; for applied medical science colleges and dental colleges (40%), “there are many issues that need to be changed”, and their values were between 13 and 24 out of 48. Finally, SSSP values were almost equal and could be interpreted as “not very bad” with mean scores between 15 and 21 out of 28”. A single study for SSPP indicated “very good socially”, with mean values from 22 to 28. These interpretations were according to the values presented in Table 2.

SIGNIFICANCE OF GENDER AND EDUCATIONAL LEVEL: Significant difference between genders was highest in applied medical science colleges (40%), followed by in medical colleges (32%), whereas most of dental colleges studies (70%) did not have any gender significance. In medical colleges and dental colleges, significant differences in educational levels were almost equal, accounting for 32% and 30% for “yes” and “no”, respectively. Significant differences were also observed in applied medical science colleges studies (60%; Figure 4).

Discussion

PUBLICATION YEARS, RR, AND GENDER RATIO:

After 2020, 40%, 20%, and 16% of publications were for dental colleges, applied medical science colleges, and medical colleges, respectively; these values were considerably lower compared with the percentage of articles published before 2020. Other systematic reviews by Behkam et al [24] and Farooqi et al [27] arrived at the same conclusions. Miles et al [15] used studies before 2010, and their review is in parallel with the percentage of studies in the current paper, as presented in Table 3 and Figure 2. RR was high in all colleges, particularly in applied medical science colleges (98%), which could be due to the limited number of students in each specialty. Females had higher percentages than males, which agrees with the results of Park et al [18] and Hernández-Crespo [17] in assessment of Korean and Spanish health college students, respectively. In a recent study conducted in Germany, Serrano et al [75] found that the percentage of women was higher than that of men. In other published articles [28,30] and in a single systematic review in SA [31], the same ratios were given.

OVERALL VALUE OF DREEM:

Males and females in SA go to separate buildings, with their own classrooms, offices, exam duties, and staff, as well as different patterns of teaching and educational environment. Thus, a significant difference exists between genders in most medical colleges, applied medical science colleges, and dental colleges. Regarding the overall value of DREEM for educational environment among gender, female students have a slightly higher mean DREEM score than male students. The overall DREEM score value in the medical, dental, and applied medical science colleges studies in this systematic review indicate that the students’ perceptions of educational environment in this colleges are “more positive than negative”. This result agrees with most studies conducted in different health specialties in different countries [17–18,27–28,30]. Recently, Serrano et al [75] assessed the educational environments in dental colleges in the Netherlands, and Behkam et al [24] investigated medical colleges at the Dundee University Medical School; both studies recorded slightly higher DREEM scores than the scores in the current review. Another study, performed by Hernández-Crespo et al [17] in Spain, indicated that the overall DREEM score of the educational environment among dental colleges students was “more positive than negative” in different academic years of 2010–2011, 2013–2014, 2014–2015, and 2015–2016. Slightly higher values, which were interpreted as “excellent,” were recorded among dental colleges students in Europe and China [27–28,30]. However, in this review, a single study conducted recently in medical colleges in UQU in Makkah City recognized a similar overall DREEM score. Another study [66] conducted among applied medical science colleges students recorded an overall DREEM score that was near excellent. However, the overall DREEM results in this review revealed that the mean score of the studies after 2019 was higher than that before 2019 in most of the included colleges. One reason for this could be that most of the recent studies were conducted in an intermediate educational environment; moreover, teaching and learning methods have changed, and technology has advanced. As a result of different reviews worldwide, the same range of overall DREEM was found [18,27,29]. A recent review by Alqahtani [31] found similar values in medical colleges and dental colleges in SA with different educational levels, as shown in Figure 3 and Table 4.

Zamzuri et al [76] were the first to analyze the EE for dental colleges in Malaysia, particularly for dental assistants and dental prosthesis students from a dental training institute, registering 125 and 118 out of 200, respectively. Furthermore, in a study involving 126 students from the Dentistry School of Manipal (India), Thomas et al [77] found an educational environment mean score of 115/200. A similar score was reported by Stratulat et al [78]. These values are similar to or lower than the values in the current review in dental colleges.

SPL, SPT, SASP, SPA, AND SSSP DOMAIN VALUES:

Figure 3 and Table 3 present the DREEM domains (SPL, SPT, SASP, SPA, and SSSP). SPL values were about the same as those in previous reviews in different countries [27,74,79], whereas SPT and SPA values were higher in the current review than in previous reviews conducted in Spain, SA, and Germany [17,26,31] and compared with other articles worldwide [27,76,78]. In terms of students’ perceptions of educational environments, medical colleges obtained significantly higher mean scores on domain items related to teaching strategies, teachers, and their social lives. Similar findings were obtained from Spain, the Netherlands, Australia, and Germany [17,27,75–79]. According to recently published literature, students’ perceptions for educational environment for SASP were equal in medical colleges, applied medical science colleges, and dental colleges [26,31,38,74,77]. SSSP values were almost equal among the 3 types of specialties and were interpreted as “not very bad”, with mean scores between 15 and 21 out of 28 [27,31,76,79]. A single study for SSPP indicated “very good socially”, with mean values from 22 to 28 (Table 2). These values have not been reported in any previous reviews, which could be because this is a newly established private college.

SIGNIFICANT DIFFERENCES IN GENDER AND EDUCATIONAL LEVELS:

As shown in Figure 4 and Table 3, significant differences were recorded between genders among different health specialties, with the highest value in dental colleges studies. In studies conducted among different health specialties worldwide, comparable findings were obtained [17–18,26,30–31,75]. Some papers indicated that females recorded a higher level of DREEM compared with males [52,54,68] in medical colleges, whereas others claimed the opposite [55,65]. Few articles mentioned significant differences among colleges [46,54,64], universities [47,60], and specialties [62,64], and other studies indicated significant differences among parameters such as marital status and cumulative grades [69,73]. Among different educational levels or study phases (preclinical and clinical), significant differences were found among students from medical colleges [39,42,46,49,52,55,57], applied medical science colleges [63,65], and dental colleges [20,74]. Similar conclusions were achieved worldwide [17,24,31,39,75].

A limitation of the current review is that it did not include studies from colleges of pharmacy, and the number of applied medical science colleges was small. In addition, it did not include many studies from private colleges. Another limitation could be the lower percentage of articles in recent years.

Conclusions

The following conclusions can be drawn from this systematic review. Overall, the DREEM scores were “more positive than negative” among all involved colleges, with scores between 101 and 150. In relation to the 5 domains of DREEM, medical colleges studies ranged from 75% to 88%, dental colleges studies had higher scores of 85% to 90%, and applied medical science colleges had lower scores of 50% to 75%. The gender percentage with significant value was considerably higher in dental colleges than in medical colleges and applied medical science colleges, whereas educational levels were higher in applied medical science colleges studies.