Assessing the Methodological Quality of Systematic Reviews in Inclusive Education: A Methodological Review

Jiaoli LI

Disertační práce

Abstract:

Abstract Systematic reviews (SRs) are increasingly used to support evidence-based practice and decision-making in inclusive education (IE). However, many standards for conducting, reporting, and appraising SRs were originally developed in health-related fields, and their applicability to the heterogeneous, context-sensitive, and practice-oriented field of IE remains insufficiently examined. This dissertation therefore investigates how SR evidence in IE is produced, evaluated, reported, and used in decision-making contexts. The dissertation first examines the methodological standards used to judge SRs of effectiveness in special and inclusive education. A meta-review of 11 umbrella reviews identified five critical appraisal tools: the JBI Checklist, AMSTAR 2, ROBIS, OQAQ, and CASP. These tools shared several domains, including search strategy, eligibility criteria, study selection, critical appraisal, synthesis methods, and interpretation. However, some items, particularly those related to meta-analysis, were frequently not applicable, suggesting a partial mismatch between conventional appraisal tools and the evidence structures commonly found in education reviews. Building on this appraisal perspective, the dissertation then evaluates the methodological and reporting quality of 115 SRs in IE published between 2019 and February 2024. The findings show that many reviews clearly reported their objectives, eligibility criteria, search strategies, and synthesis approaches. Nevertheless, recurring weaknesses were identified in protocol registration, duplicate appraisal, data extraction procedures, publication bias assessment, and certainty-of-evidence evaluation. Narrative synthesis was widely used, whereas meta-analysis was less frequent. This pattern indicates that the quality challenges of IE SRs are not simply technical shortcomings, but are closely related to the diverse review purposes, evidence types, and synthesis needs within the field. Finally, the dissertation moves from evidence production to evidence use by exploring how SRs are understood and applied by evidence users in IE decision-making. Qualitative interviews with six participants showed that SRs were valued as credible resources for orientation, justification, and reflection, but were rarely used as direct prescriptions for action. Users' trust in SRs depended not only on methodological rigor, but also on contextual relevance, feasibility, professional experience, stakeholder acceptability, and institutional conditions. This dissertation argues that SRs in IE should be understood as part of a broader methodological and decision-making ecology. It highlights the need for an integrated guidance ecology that connects methodological rigor with applicability, interpretability, and practical usability in IE.

Abstract:

Abstract Systematic reviews (SRs) are increasingly used to support evidence-based practice and decision-making in inclusive education (IE). However, many standards for conducting, reporting, and appraising SRs were originally developed in health-related fields, and their applicability to the heterogeneous, context-sensitive, and practice-oriented field of IE remains insufficiently examined. This dissertation therefore investigates how SR evidence in IE is produced, evaluated, reported, and used in decision-making contexts. The dissertation first examines the methodological standards used to judge SRs of effectiveness in special and inclusive education. A meta-review of 11 umbrella reviews identified five critical appraisal tools: the JBI Checklist, AMSTAR 2, ROBIS, OQAQ, and CASP. These tools shared several domains, including search strategy, eligibility criteria, study selection, critical appraisal, synthesis methods, and interpretation. However, some items, particularly those related to meta-analysis, were frequently not applicable, suggesting a partial mismatch between conventional appraisal tools and the evidence structures commonly found in education reviews. Building on this appraisal perspective, the dissertation then evaluates the methodological and reporting quality of 115 SRs in IE published between 2019 and February 2024. The findings show that many reviews clearly reported their objectives, eligibility criteria, search strategies, and synthesis approaches. Nevertheless, recurring weaknesses were identified in protocol registration, duplicate appraisal, data extraction procedures, publication bias assessment, and certainty-of-evidence evaluation. Narrative synthesis was widely used, whereas meta-analysis was less frequent. This pattern indicates that the quality challenges of IE SRs are not simply technical shortcomings, but are closely related to the diverse review purposes, evidence types, and synthesis needs within the field. Finally, the dissertation moves from evidence production to evidence use by exploring how SRs are understood and applied by evidence users in IE decision-making. Qualitative interviews with six participants showed that SRs were valued as credible resources for orientation, justification, and reflection, but were rarely used as direct prescriptions for action. Users' trust in SRs depended not only on methodological rigor, but also on contextual relevance, feasibility, professional experience, stakeholder acceptability, and institutional conditions. This dissertation argues that SRs in IE should be understood as part of a broader methodological and decision-making ecology. It highlights the need for an integrated guidance ecology that connects methodological rigor with applicability, interpretability, and practical usability in IE.

Keywords

inclusive education, systematic review, quality assessment, evidence-based decision-making, critical appraisal tools, evidence usability

Jazyk práce: angličtina

Datum vytvoření / odevzdání či podání práce: 3. 6. 2026

Obhajoba závěrečné práce

• Vedoucí: doc. Mgr. Jiří Kantor, Ph.D.

UNIVERZITA PALACKÉHO V OLOMOUCI

Pedagogická fakulta

Doktorský studijní program / obor:
Special Education / Special Education