FAME: Fundamentals of Assessment in Medical Education

About the Course

Course Design

FAME is intended to be a basic-level course on educational assessment. We are targeting persons who are members of medical faculties and who are likely to be relatively new to teaching and assessment. FAME is designed around two complementary dimensions, namely, four assessment themes and four assessment frames.

Definition of Frames

Assessment of Knowledge and Reasoning usually involves classroom testing of student learning. Often, these assessments are associated with assigning marks or course grades. Paper and computer-based testing are the method of convenience, with heavy reliance on multiple choice questions (MCQ) formats.

Assessment of Skills concerns selected aspects of clinical performance, some of which may be related to physical examination maneuvers, therapeutic procedures, or interpersonal skills. Usually these assessments require higher-fidelity simulations associated with Objective Structured Clinical Exam (OSCE), standardized patients (SP), computer-based simulation, or mechanical devices and mannequins. Oral examinations with short or long cases may also be considered to fall within this frame as well. Checklists and ratings scales are almost always required, and administration of these assessments often occurs in special settings.

Assessment of Workplace Performance is more often invoked for issues involving graduate doctors and practitioners where naturalistic observation may be needed. Assessment instruments will more likely relate to collecting observational data, but this may be augmented with systematic data-gathering from indirect measures, such as, chart audits. It is likely that multiple assessments may be more frequent in this frame.

Assessment of Programs occurs most commonly when curriculum changes of any dimension are contemplated, or have been implemented, and there is a need to document the outcomes. The scale of the assessment is conceivably very large, involving multiple measures of different facets of the institutional environment, possibly undertaken over an extended period of time.

Participants of the 2010 course will be asked to select either Assessment of Skills or Workplace Performance as an area for more in-depth discussion.

Definition of Themes

Test Material Development is addressed in two segments. The first concentrates on establishing the purpose(s) of the assessment and defining the content that should be included in the evaluation. The second segment addresses assessment formats including multiple choice items, constructed responses and simulation based methods. Matching methods to assessment needs is emphasized in these segments.

Scoring, Analysis and Reporting addresses the processes used to summarize and disseminate assessment data. Methods for generating individual test scores, aggregating performance data, equating scores, and providing meaningful feedback to examinees will be highlighted.

Standard Setting describes the processes used to establish cut-scores for an assessment. Both test-centered and examinee-centered approaches will be discussed. For norm referenced evaluations, techniques for identifying poorly performing individuals will be identified.

Test Score Validation highlights the activities that can be undertaken to provide evidence that the test scores and associated interpretations are justifiable and fair. Techniques that can be used to discover potential threats to the validity of assessment scores will be identified.