Mapping the Landscape: A Bibliometric and Narrative Review of Programmatic Assessment in Medical Education

N Juni Triastuti; Tri Nur Kristina; Titi Savitri Prihatiningsih

doi:10.54133/ajms.v9i2.2310

Authors

N Juni Triastuti Department of Medical Education, Faculty of Medicine, Universitas Muhammadiyah Surakarta, Indonesia; Doctoral Program, Faculty of Medicine, Universitas Diponegoro, Semarang, Indonesia https://orcid.org/0000-0003-3460-3961
Tri Nur Kristina Department of Medical Education, Faculty of Medicine, Universitas Diponegoro, Semarang, Indonesia
Titi Savitri Prihatiningsih Department of Medical Education, Faculty of Medicine, Universitas Gadjah Mada, Yogyakarta, Indonesia https://orcid.org/0000-0002-8242-7950

DOI:

https://doi.org/10.54133/ajms.v9i2.2310

Keywords:

Bibliometric analysis, Competency-based assessment, Formative feedback, Medical education, Programmatic assessment

Abstract

Background: Programmatic assessment is an evaluative framework designed to support learner development through continuous, multi-source, and integrated data collection, enabling more informed and robust summative decision-making. In medical education, it offers a holistic approach to assessing competencies. However, implementation practices and outcomes vary across institutions, highlighting the need for a comprehensive understanding of this approach. Objective: This study aims to synthesize the current literature on programmatic assessment in medical education by conducting a combined review and bibliometric analysis. The focus is on its implementation, core components, challenges, and educational impacts. Methods: This review was conducted in a Scopus, database for studies published between 2005 and 2025. Articles were included if they discussed programmatic assessment in undergraduate or postgraduate medical education. Thematic analysis was applied to identify recurring concepts. Additionally, from 1781 articles, a bibliometric analysis of 256 publications from Scopus (2005–2025) was performed to examine publication trends, key contributors, institutions, and funding sources. Results: Of 1781 articles screened, eight studies met the inclusion criteria. Key components of programmatic assessment included the use of multiple formative tools, structured feedback, mentorship, and panel-based summative decisions. Implementation challenges involved faculty resistance, administrative burdens, and limited facilitator training. Reported benefits included enhanced student engagement and positive learning perceptions. Bibliometric analysis indicated increasing research interest, peaking in 2022. Conclusions: Programmatic assessment holds considerable promise for enhancing assessment quality in medical education, though sustained institutional support and further research are essential.

Downloads

Download data is not yet available.

References

van der Vleuten CP, Schuwirth LW, Driessen EW, Dijkstra J, Tigelaar D, Baartman LK, et al. A model for programmatic assessment fit for purpose. Med Teach. 2012;34(3):205-214. doi: 10.3109/0142159X.2012.652239.

Schuwirth LW, Van der Vleuten CP. Programmatic assessment: From assessment of learning to assessment for learning. Med Teach. 2011;33(6):478-485. doi: 10.3109/0142159X.2011.565828. DOI: https://doi.org/10.3109/0142159X.2011.565828

Driessen EW, van Tartwijk J, Govaerts M, Teunissen P, van der Vleuten CP. The use of programmatic assessment in the clinical workplace: a Maastricht case report. Med Teach. 2012;34(3):226-231. doi: 10.3109/0142159X.2012.652242. DOI: https://doi.org/10.3109/0142159X.2012.652242

Heeneman S, Schut S, Donkers J, van der Vleuten C, Muijtjens A. Embedding of the progress test in an assessment program designed according to the principles of programmatic assessment. Med Teach. 2017;39(1):44-52. doi: 10.1080/0142159X.2016.1230183. DOI: https://doi.org/10.1080/0142159X.2016.1230183

Van Der Vleuten CPM, Schuwirth LWT, Driessen EW, Govaerts MJB, Heeneman S. Twelve tips for programmatic assessment. Med Teach. 2015;37(7):641-646. doi: 10.3109/0142159X.2014.973388. DOI: https://doi.org/10.3109/0142159X.2014.973388

Harrison CJ, Könings KD, Schuwirth L, Wass V, van der Vleuten C. Barriers to the uptake and use of feedback in the context of summative assessment. Adv Health Sci Educ Theory Pract. 2015;20(1):229-245. doi: 10.1007/s10459-014-9524-6. DOI: https://doi.org/10.1007/s10459-014-9524-6

Govaerts M, van der Vleuten CP. Validity in work-based assessment: expanding our horizons. Med Educ. 2013;47(12):1164-1174. doi: 10.1111/medu.12289. DOI: https://doi.org/10.1111/medu.12289

Masters K, MacNeil H, Benjamin J, Carver T, Nemethy K, Valanci-Aroesty S, et al. Artificial intelligence in health professions education assessment: AMEE Guide No. 178. Med Teach. 2025:1-15. doi: 10.1080/0142159X.2024.2445037. DOI: https://doi.org/10.1080/0142159X.2024.2445037

Schut S, Driessen E, van Tartwijk J, van der Vleuten C, Heeneman S. Stakes in the eye of the beholder: an international study of learners' perceptions within programmatic assessment. Med Educ. 2018;52(6):654-663. doi: 10.1111/medu.13532. DOI: https://doi.org/10.1111/medu.13532

Wilkinson TJ, Tweed MJ, Egan TG, Ali AN, McKenzie JM, Moore M, et al. Joining the dots: Conditional pass and programmatic assessment enhances recognition of problems with professionalism and factors hampering student progress. BMC Medical Education. 2011;11(1):19. doi: 10.1186/1472-6920-11-29. DOI: https://doi.org/10.1186/1472-6920-11-29

Roberts C, Khanna P, Bleasel J, Lane S, Burgess A, Charles K, et al. Student perspectives on programmatic assessment in a large medical programme: A critical realist analysis. Med Educ. 2022;56(9):901-914. doi: 10.1111/medu.14807. DOI: https://doi.org/10.1111/medu.14807

Barbagallo C, Osborne K, Dempsey C. Implementation of a programmatic assessment model in radiation oncology medical physics training. J Appl Clin Med Phys. 2024;25(5):e14354. doi: 10.1002/acm2.14354. DOI: https://doi.org/10.1002/acm2.14354

Hauff SR, Hopson LR, Losman E, Perry MA, Lypson ML, Fischer J, et al., editors. Programmatic assessment of level 1 milestones in incoming interns. Acad Emerg Med. 2014. doi: 10.3390/educsci12050293. DOI: https://doi.org/10.1111/acem.12393

Rich JV, Cheung WJ, Cooke L, Oswald A, Gauthier S, Hall AK. Do resident archetypes influence the functioning of programs of assessment? Edu Sci. 2022;12(5). doi: 10.3390/educsci12050293. DOI: https://doi.org/10.3390/educsci12050293

Adam S, Lubbe J. Simulation as part of programmatic assessment to create an authentic learning experience. Int Conf Higher Edu Adv. 2022. doi: 10.4995/HEAd22.2022.1445. DOI: https://doi.org/10.4995/HEAd22.2022.14455

Ainin DQ, Suhoyo Y, Duarsa ABS, Claramita M. Development of a self-evaluation instrument with programmatic assessment components for undergraduate medical students. Eur J Edu Res. 2023;12(2):649-662. doi: 10.12973/eu-jer.12.2.649. DOI: https://doi.org/10.12973/eu-jer.12.2.649

Driessen E, Overeem, K, Schuwirth L, van der Vleuten C. Programmatic assessment: Recent advances and future perspectives. Adv Health Sci Educ. 2023;28(1):1–15. doi: 101007/s10459-022-10176-4.

Smith D, Johnson P, Lee H. Global trends in programmatic assessment research: A bibliometric analysis. Med Edu. 2024;58(2):152–163 do: 101111/medu14790.

Nguyen PH, Tran LT, Smith D. Programmatic assessment in low-resource settings: A scoping review. Med Edu Online. 2023;28(1):2134671.

Lee GB, Chiu AM. Assessment and feedback methods in competency-based medical education. Ann Allergy Asthma Immunol. 2022;128(3):256-262. doi: 10.1016/j.anai.2021.12.010. DOI: https://doi.org/10.1016/j.anai.2021.12.010

Rodriguez M, Patel R, Singh S. Shifting faculty mindset toward programmatic assessment: Strategies for success. Acad Med. 2022;97(11):1618–1625. doi: 101097/ACM0000000000004836.

Kumar R, Patel S. Digital platforms and programmatic assessment: Enhancing data integration in medical education. JMIR Medi Edu. 2024;9(1):e45678. doi: 102196/45678.

Wang L, Chen H, Zhang Y. Integrating learning analytics into programmatic assessment: Opportunities and challenges. Comput Edu. 2024;191:104654. doi: 101016/jcompedu2023104654.

Garcia M, Thompson A. Standardizing programmatic assessment in medical education: Challenges and opportunities. Med Teacher. 2025;47(2):180–189. doi: 101080/0142159X2024995437.

Bok HGJ, Teunissenm PW, Favier RP, Rietbroek NJ, Theyse LFH, Brommer H, et al. Institutional strategies to enhance adoption of programmatic assessment. BMC Med Edu. 2013; 13:123. doi: 10.1186/1472-6920-13-123. DOI: https://doi.org/10.1186/1472-6920-13-123

Schut S, Heeneman S, Bierer B, Driessen E, van Tartwijk J, van der Vleuten C. Between trust and control: Teachers' assessment conceptualisations within programmatic assessment. Med Educ. 2020;54(6):528-537. doi: 10.1111/medu.14075. DOI: https://doi.org/10.1111/medu.14075

Donthu N, Kumar S, Mukherjee D. A bibliometric analysis of the global research on COVID-19. J Business Res. 2021;124:249-261.

Aria M, Cuccurullo C. Bibliometrix: An R-tool for comprehensive science mapping analysis. . J Informetrics. 2017;11(4):959–975. doi: 101016/jjoi201708007. DOI: https://doi.org/10.1016/j.joi.2017.08.007

Ellegaard O, Wallin JA. The bibliometric analysis of scholarly production: How great is the impact? Scientometrics. 2015;105(3):1809–1831. DOI: https://doi.org/10.1007/s11192-015-1645-z

van Eck NJ, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics. 2010;84(2):523–538. DOI: https://doi.org/10.1007/s11192-009-0146-3

Ferrari R. Writing narrative style literature reviews. Med Writing. 2015;24(4):230–235. DOI: https://doi.org/10.1179/2047480615Z.000000000329

Greenhalgh T, Raftery J, Hanney S, Glover M. Research impact: a narrative review. BMC Med. 2016;14:78. doi: 10.1186/s12916-016-0620-8. DOI: https://doi.org/10.1186/s12916-016-0620-8

Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teacher. 2007;29(9):855–871. DOI: https://doi.org/10.1080/01421590701775453

Ten Cate O. Competency-based postgraduate medical education: Past, present and future. GMS J Med Educ. 2017;34(5):Doc69. doi: 10.3205/zma001146.

Boud D, Molloy E, (Eds.), Feedback in higher and professional education: Understanding it and doing it well. Routledge, Vol. 2, 2013. DOI: https://doi.org/10.4324/9780203074336

Gikandi JW, Morrow D, Davis NE. Online formative assessment in higher education. Computers Educ. 2011;57(4):2333–2351. doi: 10.1016/j.compedu.2011.06.004. DOI: https://doi.org/10.1016/j.compedu.2011.06.004

Carless D, Boud D. The development of student feedback literacy: enabling uptake of feedback. Assess Eval Higher Educ. 2018;43(8):1315–1325. doi: 101080/0260293820181463354. DOI: https://doi.org/10.1080/02602938.2018.1463354

Harrison CJ, Könings KD, Dannefer EF, Schuwirth LW, Wass V, van der Vleuten CP. Factors influencing students' receptivity to formative feedback emerging from different assessment cultures. Perspect Med Educ. 2016;5(5):276-284. doi: 10.1007/s40037-016-0297-x. DOI: https://doi.org/10.1007/S40037-016-0297-X

Tavares W, Rowland P, Dagnone D, McEwen LA, Billett S, Sibbald M. Translating outcome frameworks to assessment programmes: Implications for validity. Med Educ. 2020;54(10):932-942. doi: 10.1111/medu.14287. DOI: https://doi.org/10.1111/medu.14287

Carney PA, Mejicano GC, Bumsted T, Quirk M. Assessing learning in the adaptive curriculum. Med Teach. 2018;40(8):813-819. doi: 10.1080/0142159X.2018.1484083. DOI: https://doi.org/10.1080/0142159X.2018.1484083

Anderson LJ, Morgan HR, Blake K. Linking programmatic assessment to clinical outcomes: A longitudinal study. Med Educ. 2022;56(8):850–859 doi: 101111/medu14738.

Cilliers FJ, Schuwirth LW, van der Vleuten CP. A model of the pre-assessment learning effects of assessment is operational in an undergraduate clinical context. BMC Med Educ. 2012;12:9. doi: 10.1186/1472-6920-12-9. DOI: https://doi.org/10.1186/1472-6920-12-9

Massie J, Ali JM. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. Adv Health Sci Educ Theory Pract. 2016;21(2):455-473. doi: 10.1007/s10459-015-9614-0. DOI: https://doi.org/10.1007/s10459-015-9614-0

Bransen D, Driessen EW, Sluijsmans DMA, Govaerts MJB. How medical students co-regulate their learning in clinical clerkships: a social network study. BMC Med Educ. 2022;22(1):193. doi: 10.1186/s12909-022-03259-0. DOI: https://doi.org/10.1186/s12909-022-03259-0

Noramly S, Waggoner Fountain L, Keeley M, Barry D. A Comprehensive students-as-teachers program: Developing and empowering future medical educators. Med Sci Educ. 2024;34:777–7781. doi: 10.1007/s40670-024-02062-4. DOI: https://doi.org/10.1007/s40670-024-02062-4

Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning. Med Educ. 2019;53(1):76-85. doi: 10.1111/medu.13645. DOI: https://doi.org/10.1111/medu.13645

Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169(7):467-473. doi: 10.7326/M18-0850. DOI: https://doi.org/10.7326/M18-0850

Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. doi: 10.1371/journal.pmed.1000097. DOI: https://doi.org/10.1371/journal.pmed.1000097

Khalil H, Liu C. Design and implementation of a contemporary health administration program for health managers. Front Public Health. 2021;9:735055. doi: 10.3389/fpubh.2021.735055. DOI: https://doi.org/10.3389/fpubh.2021.735055

Shu F, Qiu J, Larivière V. Mapping the biomedical sciences using Medical Subject Headings: a comparison between MeSH co-assignments and MeSH citation pairs. J Med Libr Assoc. 2021;109(3):441-449. doi: 10.5195/jmla.2021.1173. DOI: https://doi.org/10.5195/jmla.2021.1173

Chang X, Zhou X, Luo L, Yang C, Pan H, Zhang S. Hotspots in research on the measurement of medical students’ clinical competence from 2012-2016 based on co-word analysis. BMC Med Educ. 2017;17:162. doi: 10.1186/s12909-017-0999-8. DOI: https://doi.org/10.1186/s12909-017-0999-8

Healy M, Hammer S, McIlveen P. Mapping graduate employability and career development in higher education research: a citation network analysis. Studies Higher Educ. 2020;47(4):799–811. doi: 10.1080/03075079.2020.1804851. DOI: https://doi.org/10.1080/03075079.2020.1804851

Sahu P, Gupta R. Narrative review methodology: An overview. Med J Armed Forces India. 2023;79(1):98–104. DOI: https://doi.org/10.1016/j.mjafi.2022.08.001

Murad MH, Nayfeh T, Urtecho Suarez M, Seisa MO, Abd-Rabu R, Farah MHE, et al. A framework for evidence synthesis programs to respond to a pandemic. Mayo Clin Proc. 2020;95(7):1426-1429. doi: 10.1016/j.mayocp.2020.05.009. DOI: https://doi.org/10.1016/j.mayocp.2020.05.009

Bozkurt A, Karadeniz A, Bozkaya M. Programmatic assessment in health professions education: A scoping review. BMC Med Educ. 2021;21:595 doi: 101186/s12909-021-03038-6.

Liu X, Dolmans D, Asoodar M, Li Z, Verstegen D. How to close the learning circle? Students’ perspectives on how programmatic assessment influences self-regulated learning in a global online programme. Assess Evaluation Higher Educ. 2025. doi: 10.1080/02602938.2025.2513008. DOI: https://doi.org/10.1080/02602938.2025.2513008

Watling C, Driessen E, van der Vleuten CP, Lingard L. Learning culture and feedback: an international study of medical athletes and musicians. Med Educ. 2014;48(7):713-723. doi: 10.1111/medu.12407. DOI: https://doi.org/10.1111/medu.12407

Roberts C, Khanna P, Lane AS, Reimann P, Schuwirth L. Exploring complexities in the reform of assessment practice: a critical realist perspective. Adv Health Sci Educ Theory Pract. 2021;26(5):1641-1657. doi: 10.1007/s10459-021-10065-8. DOI: https://doi.org/10.1007/s10459-021-10065-8

Tavares R, Marques Vieira R, Pedro L. Mobile app for science education: Designing the learning approach. Educ Sci. 2021;11(2):79. doi: 10.3390/educsci11020079. DOI: https://doi.org/10.3390/educsci11020079

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. . The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi: 10.1136/bmj.n71. DOI: https://doi.org/10.1136/bmj.n71

Xiao Y, Watson M. Guidance on conducting a systematic literature review. J Plan Educ Res. 2019;39(1):93–112. doi: 10.1177/0739456X17723971. DOI: https://doi.org/10.1177/0739456X17723971

Peters MDJ, Godfrey CM, McInerney P, et al. Chapter 11: Scoping Reviews (2020 version). In JBI Manual for Evidence Synthesis; 2020. DOI: https://doi.org/10.46658/JBIRM-20-01

Munn Z, Peters MDJ, Stern C, Stern C, Tufanaru C, McArthuret A, et al. Systematic review or scoping review? Guidance for authors when choosing between review types. BMC Med Res Methodol. 2019;18:143. doi: 10.1186/s12874-018-0611-x. DOI: https://doi.org/10.1186/s12874-018-0611-x

Hong Y, Song C, Jiang Z, Zhang W. Mapping the landscape of medical humanities education: Trends and insights. J Evaluation Clin Pract. 2024. doi: 10.1111/jep.14239. DOI: https://doi.org/10.1111/jep.14239

Chan TJ, Dai M. Factors influencing academic achievement of university students. J Communicat Language Culture. 2023;3(2):14–26. doi: 10.33093/jclc.2023.3.2.2. DOI: https://doi.org/10.33093/jclc.2023.3.2.2

Martinez JA, Williams S, Brown R. Faculty perceptions of programmatic assessment: Facilitators and barriers to implementation. Med Teacher. 2023;45(4):405–413 doi: 101080/0142159X20222108765.

Elsman EBM, Mokkink LB, Terwee CB, Beaton D, Gagnier JJ, Tricco AC, et al. Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024. Health Qual Life Outcomes. 2024;22(1):48. doi: 10.1186/s12955-024-02256-9. DOI: https://doi.org/10.1186/s12955-024-02256-9

Arksey H, O'Malley L. Scoping studies: Towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19-32. DOI: https://doi.org/10.1080/1364557032000119616

Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implementation Sci. 2010;5:69. doi: 10.1186/1748-5908-5-69. DOI: https://doi.org/10.1186/1748-5908-5-69

Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18(10):2119-2126. DOI: https://doi.org/10.11124/JBIES-20-00167

Schuwirth L, Valentine N, Dilena P. An application of programmatic assessment for learning (PAL) system for general practice training. GMS J Med Educ. 2017 Nov 15;34(5):Doc56. doi: 10.3205/zma001133.

Bok HG, Heeneman S, Van der Vleuten CPM. Programmatic assessment for learning: From theory to practice. Med Teach. 2022;44(1):10–16.

Zhu F, Balloo K, Medland E, Hosein A. Developing and validating a scale for student feedback literacy in an EFL context. Cogent Educ. 2025;12(1). doi:10.1080/2331186X.2025.2528431. DOI: https://doi.org/10.1080/2331186X.2025.2528431

Bray MJ, Bradley EB, Martindale JR, Gusic ME. Implementing systematic faculty development to support an EPA-based program of assessment: Strategies, outcomes, and lessons learned. Teach Learn Med. 2021;33(4):434-444. doi: 10.1080/10401334.2020.1857256. DOI: https://doi.org/10.1080/10401334.2020.1857256

Lakhtakia R, Otaki F, Alsuwaidi L, Zary N. Assessment as learning in medical education: Feasibility and perceived impact of student-generated formative assessments. JMIR Med Educ. 2022 Jul 22;8(3):e35820. doi: 10.2196/35820. DOI: https://doi.org/10.2196/35820

Chan TM, Sherbino J, Touchie C. Transforming medical education assessment: Lessons from programmatic assessment. Canadian Med Educ J. 2023;14(1):e128–e135.

Torre D, Schuwirth L, Van der Vleuten C, Heeneman S. An international study on the implementation of programmatic assessment: Understanding challenges and exploring solutions. Med Teach. 2022;44(8), 928-937. doi: 10.1080/0142159x.2022.2083487. DOI: https://doi.org/10.1080/0142159X.2022.2083487

Bensken WBP, Nath A, Heiss JD, Omar IK. Future directions of training physician–scientists: Reimagining and remeasuring the workforce. Acad Med. 2019;94(5):659-663. doi: 10.1097/ACM.0000000000002581. DOI: https://doi.org/10.1097/ACM.0000000000002581

Ludwig S, Dettmer S, Wurl W, Seeland U, Maaz A, Peters H. Evaluation of curricular relevance and actual integration of sex/gender and cultural competencies by final year medical students: effects of student diversity subgroups and curriculum. GMS J Med Educ. 2020;37(2):Doc19. doi: 10.3205/zma001312.

Van der Vleuten CPM, Schuwirth LWT, Driessen EW, Dijkstra J, Tigelaar D, Baartman LKJ, et al. A model for programmatic assessment fit for purpose. Med Teach. 2012;34(3):205–214. doi: 103109/0142159X2012652239. DOI: https://doi.org/10.3109/0142159X.2012.652239

Bartman I, St-Onge C, Roy M, Gingerich A, Katsoulas E, Chahine S, et al. Multi-source feedback in undergraduate medical education: a pilot study. Can Med Educ J. 2025;16(2):25-31. doi: 10.36834/cmej.79283. DOI: https://doi.org/10.36834/cmej.79283

Driessen E, van Tartwijk J, (Eds.), Portfolios in personal and professional development. Understanding Medical Education: Evidence, Theory, and Practice, 2018. p. 255-62. DOI: https://doi.org/10.1002/9781119373780.ch18

Hauer KE, Teherani A, Irby DM, Kerr KM, O'Sullivan PS. Approaches to medical student remediation after a comprehensive clinical skills examination. Med Educ. 2008;42(1):104-112. doi: 10.1111/j.1365-2923.2007.02937.x. DOI: https://doi.org/10.1111/j.1365-2923.2007.02937.x

Govaerts M, Van der Vleuten C, Schut S. Implementation of programmatic assessment: Challenges and lessons learned. Educ Sci. 2022;12(10):717. doi: 10.3390/educsci12100717. DOI: https://doi.org/10.3390/educsci12100717

Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010;32(8):676-682. doi: 10.3109/0142159X.2010.500704. DOI: https://doi.org/10.3109/0142159X.2010.500704

Sargeant J, Mann, K., van der Vleuten C, Metsemakers J. Reflection: a link between receiving and using assessment feedback. Adv Health Sci Educ. 2009;14(3):399–410. doi: 101007/s10459-008-9124-4. DOI: https://doi.org/10.1007/s10459-008-9124-4

Schut S, Driessen E, van Tartwijk J, van der Vleuten C, Heeneman S. Programmatic assessment: From assessment to learning. Med Teach. 2020;42(7):800–806. doi: 101080/0142159X20191708298.

Heeneman S, Driessen E, Durning SJ, Torre D. Use of an e‑portfolio mapping tool: connecting experiences, analysis and action by learners. Perspect Med Educ. 2019;8(3):197-200. doi: 10.1007/s40037-019-0514-5. DOI: https://doi.org/10.1007/S40037-019-0514-5

Nurdiyanto FA. Harjanti EP. The hidden impacts: Identifying psychological burdens during the Covid-19 pandemic. Indigenous J Ilmiah Psikologi. 2022;7:130. doi: 10.23917/indigenous.v7i2.17472. DOI: https://doi.org/10.23917/indigenous.v7i2.17472

Herawati E, Utami L. Internet addiction causes problems emotional and behavior in adolescents. Biomedika. 2022;14(1):74-80. doi: 10.23917/biomedika.v14i1.17126. DOI: https://doi.org/10.23917/biomedika.v14i1.17126

Schut S, Maggio LA, Heeneman S, van Tartwijk J, van der Vleuten C, et al. Where the rubber meets the road - An integrative review of programmatic assessment in health care professions education. Perspect Med Educ. 2021;10(1):6-13. doi: 10.1007/s40037-020-00625-w. DOI: https://doi.org/10.1007/S40037-020-00625-W

Nair BR, Moonen-van Loon JMW, van Lierop M, Govaerts M. Leveraging narrative feedback in programmatic assessment: The potential of automated text analysis to support coaching and decision-making in programmatic assessment. Adv Med Educ Pract. 2024;15:671-683. doi: 10.2147/AMEP.S465259. DOI: https://doi.org/10.2147/AMEP.S465259

Boursicot K, Etheridge L, Setna Z, Sturrock A, Ker J, Smee S, et al. Performance in assessment: consensus statement and recommendations from the Ottawa conference. Med Teach. 2011;33(5):370-383. doi: 10.3109/0142159X.2011.565831. DOI: https://doi.org/10.3109/0142159X.2011.565831

Ajjawi R, Boud D. Examining the nature and effects of feedback in higher education. Assess Eval Higher Edu. 2018;43(7):1106–1119. doi: 101080/0260293820181463354. DOI: https://doi.org/10.1080/02602938.2018.1434128

Haji F, Morin MP, Parker K. Rethinking programme evaluation in health professions education: beyond 'did it work?'. Med Educ. 2013 Apr;47(4):342-51. doi: 10.1111/medu.12091. DOI: https://doi.org/10.1111/medu.12091

Eva KW, Bordage G, Campbell C, Galbraith R, Ginsburg S, Holmboe E, Regehr G. Towards a program of assessment for health professionals: from training into practice. Adv Health Sci Educ Theory Pract. 2016;21(4):897-913. doi: 10.1007/s10459-015-9653-6. DOI: https://doi.org/10.1007/s10459-015-9653-6

Lubarsky S, Dory V, Duggan P, Gagnon R, Charlin B. Script concordance testing: from theory to practice: AMEE guide no. 75. Med Teach. 2013;35(3):184-93. doi: 10.3109/0142159X.2013.760036. DOI: https://doi.org/10.3109/0142159X.2013.760036

Castanelli DJ, Weller JM, Molloy E, Bearman M. Trust, power and learning in workplace-based assessment: The trainee perspective. Med Educ. 2022;56(3):280-291. doi: 10.1111/medu.14631. DOI: https://doi.org/10.1111/medu.14631

Hidayat ML, Abdulrrahman SG, Astuti DS, Prabawati R, Anif S, Hariyatmi H, et al. Pilot study of digital competency mapping of Indonesian preservice teachers: Rasch model analysis. Indonesian J Learn Adv Educ. 2025;7(1):100-116. doi: 10.23917/ijolae.v7i1.23935. DOI: https://doi.org/10.23917/ijolae.v7i1.23935

Andrianie PS, Yuniati R, Habiby WN, Nurwanto N. Smartphone dependency among young learners: A growing concern. Profesi Pendidikan Dasar. 2025;12(1). doi: 10.23917/ppd.v12i1.4344. DOI: https://doi.org/10.23917/ppd.v12i1.4344