Assessing Students
Assessment of students is offered in two forms: formative feedback that should occur frequently throughout the rotation or course; and summative assessment at the end of the rotation.
Assessment of learners comes in two basic forms:
- Formative, often called "feedback," should occur frequently throughout the rotation or course; and
- Summative, which happens at the end of the rotation. The distinctions are summarized in the table below:
Formative Assessment "Feedback" | Summative Assessment "Final Evaluation" | |
---|---|---|
Timing | During a course of study, ideally frequent, immediate or nearly so | At the end of a course of study |
Purpose | Check learner's understanding/skill in order to plan subsequent instruction | Inform learner and instructor whether knowledge/skill has been attained |
Grading | Ungraded or low stakes | Graded, high stakes |
Feedback to Learner | Learner-centered; responsive to learner needs | Standard/benchmark-centered; not responsive to specific learner needs |
Giving Successful Feedback
Feedback must be useful information, delivered appropriately to a receptive learner in a supportive environment that encourages the learner to change.
Feedback Pearls
Establish a "coaching" environment from the start so the learner is expecting and looking forward to your insights as to how to improve on nearly every patient.
- Give feedback as soon as possible after observing the event.
- Always ask the learner's assessment first. This allows you to build on strengths and immediately address the learner's greatest concerns.
- Use concepts, language, and tone that make sense to the learner so they can incorporate the feedback into future performance.
- Base feedback on first-hand information (things you have personally observed).
- Focus on behaviors, not personality traits, and provide information, not judgments.
- Provide specific information about things that can be changed, not general "good" or "bad" comments.
- Don't forget to give feedback on things done well to reinforce the behavior or improve it further.
- Feedback on something done poorly must enable the learner to improve their performance. Follow up to reinforce positive changes and continue the "quality enhancement" cycle.
- Practice "praise in public, critique in private" to encourage patients and staff to be part of the learner's development.
Start with either an open-ended question e.g. "How did things go with the last patient?" or a more focused question if you are working with the learner on a particular aspect e.g. "How well do you think you did on the history (or physical exam) on this patient?"
Most learners are pretty harsh critics of their own performance. They tend to immediately discuss their areas of weakness and frequently start strategizing how to improve. This allows the teacher to emphasize positives and give advice on improvements - not only a pleasant situation but also powerful and enduring adult learning as the learner is taking responsibility and "working it out" for themselves. Conversely, if the student has limited insights on their suboptimal performance, the teacher needs to provide a reality check on the expected level of performance. This is much better done as soon as you become aware of the issue than coming as a shock in the form of a poor evaluation at the end of the course.
A genuine comment on the positive behaviors encourages the learner to repeat them. This is especially true if you can link the behavior to a positive outcome e.g. "You took care to encourage the patient to explain why she had not been taking her medicine. Now we can work with her on finding cheaper alternatives". Positive feedback also has more impact if it is specific (not just "good job") and is reinforced, acknowledges changes made by the learner, or is part of an overall plan for improvement rather than just an isolated event. (e.g. "I was very pleased to see you using the techniques we discussed last week to put an anxious patient at ease.").
The message must be perceived by the learner that you can see specific ways to make them an even more effective clinician. The learner may have substantial insight. If the area you want to address was not volunteered by the learner, try a question such as, "If you could go back and do the patient visit again, what would you do differently and why?"
Feedback that is perceived as a demand to do things "only my way" will not succeed in the long term. Explain what needs to be changed and why to provide incentive to change. Often a demonstration is appropriate. Avoid blame or personal comments as these create barriers to change. Ensure that student is capable of making the change -- they may need more information (suggest readings or resources to help) or opportunities to practice a skill (e.g. suturing) - and express your confidence in them.
Finally, keep the level of training in perspective and don't overwhelm the learner by forcing too many changes at once or asking them to do impossible things. e.g. "Your histories are very complete but are taking a long time. I would like you to focus more on the presenting complaint and being ready to give me a concise presentation after 15 minutes....it takes practice!"
Consistency is essential for positive changes to become integrated into normal functioning. Acknowledging positive changes reinforces them and stimulates the learner's motivation to continue to become even more proficient in patient care. To help remember the basics and structure for your feedback, some experts suggest "serving the feedback sandwich":
- What was done well?
- What needs to be improved?
- How to make changes?
Remembering to ask for the learner assessment first ensures they are ready to digest the "sandwich" and follow-up verifies it provided benefit.
For those who like mnemonics, we found TOLD AS and IMPROVE as ways to remember the key aspects of feedback.
Feedback should be:
Timely (as soon as possible after the behavior)
Observed (based on your personal observations)
Limited (to a few items, not overwhelming)
Descriptive (expressed objectively as facts, not opinions)
Actions (concerns professional behaviors)
Specific (specific, changeable behaviors)
Giving effective feedback involves:
Identifying objectives clearly before the learning sessions
Making an environment that includes regular, constructive feedback (feedback-friendly)
Performance assessment (directly observed)
Responding to learner self-assessment
Objective, outcome-oriented, information on specific behaviors
Validating what was done well
Establishing a practical plan for follow-up and additional feedback
The bottom line for feedback:
- We work together to care for patients.
- You already do these things well.
- Here's how to make your patient care even better.
- I will keep working with you to become the best possible clinician.
Performing Successful Summative Assessment
Summative assessment (formerly called "evaluation") is a formal process that takes place at a predetermined time, usually at the end of the course, and aims to determine:
- What did the student achieve during the experience?
- Where did they end up according to a predetermined standard?
Summative assessment by preceptors involves completion of forms that record:
- your assessment of the learner's performance in specific areas (e.g. history-taking, physical examination skills, communication with patients) according to some scale plus
- your subjective comments about the learner
Call the course director if you have ANY uncertainties about scoring, especially what is the "normal" range or you have a student who you think may not meet the director's expectations.
The subjective comments section allows you to elaborate on the numerical score by explaining specific strengths or weaknesses that influenced the score. It also allows you to describe how the student functioned in the practice environment.
Although by the end of the course, you may have developed a strong personal relationship with the student, your comments should focus on specific professional behaviors and attributes. "Great student!" is less helpful than "takes a thorough history but needs to work on being more efficient."
Assessment is hard so preceptors can be tempted to procrastinate but this only makes the process harder. Just like completing patient records quickly keeps your recall fresh, "just do it."
Start by doing your homework. Read through the evaluation documents at the beginning of the course and check them periodically during the course to avoid the awkward situation of having to provide an opinion on something the learner did not do or about which you have very limited information.
If in doubt about what is expected, ask! Then orient your student at the beginning of the course, setting clear expectations so the student isn't surprised by their final assessment.
Short notes on specific items that occurred during the course and/or regular notes on the learner's progress can be invaluable memory prompts when completing the end of course assessment. A useful tool is the "Plus/Delta Card".
It occurs when a student is so nice and enjoyable to work with that evaluations are unreasonably high. This can also occur when a learner has a reputation for being "great" or has had superior grades on previous courses. To avoid the overall impression obscuring problems and falsely inflating grades, assess each item strictly on its merits.
Grades can also be influenced by negative reactions to a student, especially personal attributes or attitudes that don't fit with your practice. Call the course director early if you anticipate any difficulty in working with a learner or providing a valid, objective evaluation.
It is very important as it concerns how the learner performed in your practice environment but the course director typically includes input from other supervisors, written exams, etc., to determine the student's grade.
This can feel awkward but it is vital to your student's development as a physician. If you oriented your student clearly at the start and observed and gave effective feedback over the course of your interactions, then there should be no surprises. If you think your evaluation is harsh or has a lot of negatives, the reason may not be that "I am just a harsh evaluator" or even "I am not doing this correctly" consider:
You may be in the best position to detect significant negatives that are masked in other experiences: e.g. students with limited knowledge base can frequently "get by" on wards or faculty interactions but not in the office. The things you are perceiving may resonate with reports from other faculty members or evaluators: the course director needs objective input from multiple sources to validate concerns.
Conversely, if all your evaluations are superior and you seem to get only the very best students, consider:
Are you grading too highly? Check the criteria for scores and discuss your interpretations with the course director
Can you verify that the student consistently performs at the level of your assessment? Did you only see them on good days or do you only remember the positives?
Bottom Line
In summary, here is a mnemonic to remember the key features of summative assessment:
Get ready. Review objectives and forms. Ask for help if unsure of the process.
Review expectations and timing of evaluations with the learner ("no surprises").
Assess the learner regularly and keep notes. Give regular feedback.
Discuss progress and be sure to collect data during the rotation.
Evaluation paperwork completed promptly and accurately.
The bottom line on summative assessment: A good evaluation summarizes the learner's status at the end of the course. It enables the learner and their advisors to continue their development as a physician.
Be honest and objective in your evaluations. Deficiencies will show up sooner or later. The earlier they are detected and addressed in medical education the better. Your third-year student will be the admitting intern in less than 24 months!
Finally, do not hesitate to discuss concerns about students or how you do evaluations with the course director.