Managed care and the corporatization of health care present challenges to medical professionalism. The defining, promoting, and teaching of medical professionalism have therefore become major concerns for medical educators, and the need for a systemic assessment of professionalism has become apparent. In 2002, the Medical Student Performance Evaluation Committee of the Association of American Medical Colleges developed and published a Draft Professionalism Assessment Form (PAF). The Keck School of Medicine of the University of Southern California pilot tested the instrument during the fall and spring semesters of 2005-2006. The purpose of this study was to begin the process of validating this instrument by determining inter-rater reliability.

Data were collected on 163 of 168 (97.02%) of the students of the class of 2008 Keck School of Medicine enrolled in the course Professionalism and the Practice of Medicine (PPM). One hundred and ten PPM students (65.48%) were evaluated by two faculty. Data were also collected on the second-year students enrolled in ICM. Each instructor completed one evaluation on 83 of 168 (49.40%) ICM students.

Almost all students received a rating at the midpoint of the scale indicating appropriate professional behavior, and the data showed almost negligible variation. The conclusion, based on the results of this study, is that the PAF, as presently constituted, lacks inter-rater reliability and therefore validity. As such, the PAF should not be used for academic decisions such as grading or promotions. However, the instrument could be used for formative evaluation of students in discussions of professional behaviors.