Advanced Assessment to Accelerate Diagnostic Skill Acquisition, Phase II

Project Summary
Today s clinical learning environments do not provide the level of deliberate practicedirect supervisionand
rigorous assessment and feedback needed to develop diagnostic reasoning expertiseClinical performance
assessment emphasizes learner evaluation over learner developmentlacks rigor and utility for developmental
purposesand clinical teachers have expressed particular difficulty with diagnosing reasoning deficits for
remediation purposesFurthermedical studentsdiagnostic reasoning does not improve over the course of
clinical training and senior medical students have highly variable diagnostic performance that is often rated
below expectations according to theory based and validated scoring criteriaIndependent practice does not
necessarily enhance the context for clinical reasoningthe majority of physiciansmedical errors are thought to
be diagnostic in natureWe propose to improve undergraduate medical education to minimize the time to
clinical competency for first year residents through targeted diagnostic reasoning skill development thatintegrates basic science and clinical instructionprovides deliberate practice with structuredcase based
learning opportunitiesandenables anytime anywhere learning that fits with the demanding schedules of
most medical studentsSouthern Illinois University School of MedicineSIUSOMis a recognized leader in
using performance based clinical competency exams to enhance reasoning skill acquisition among medical
studentsThese exams feature clinical scenarios with standardized patients followed by diagnostic justification
essays which require students to explicitly describe the thought process used to reach a final diagnosisThese
essays are the most reliable method of assessing diagnostic strategies but are not in use in the majority of
medical schoolsthough interest in improving diagnostic reasoning instruction and assessment during
undergraduate medical education is widespreadBarriers to the widespread adoption of this approach arethe time consuming need to hand score each essayandthe difficulty in accurately and consistently
identifying the causes of strategy failuresThis project will develop an application to provide automated scoring
of diagnostic justification essaysidentification of the underlying causes of failure when students perform
poorlyand feedback with instructional strategies for remediation specific to each deficitWe propose these
specific aimsImprove reliability of human scoring of DXJ essaysExtend the automated scoring
algorithmsAutomated reasoning failure categorization and remediationComplete the software
development required for delivering the commercial productEvaluate predictive validity of automatically
scored DXJ essaysThe proposed product represents a significant shift in undergraduate medical training and
through Phase III dissemination will address a critical gap between education and practice in academic
medicine