As medical schools shift from time-based to competency-based curricula, portfolios have emerged as a progressive tool to assess student learning. Here’s how Vanderbilt University School of Medicine uses e-portfolios to transform students’ assessments, strengthen partnerships with faculty and track students’ progress as they advance in training.
Using open-source software, Vanderbilt has created a complex e-portfolio system that charts students’ performance across a core set of competencies based on the Accreditation Council for Graduate Medical Education’s graduate medical education (GME) milestones. The project was first announced last year as part of Vanderbilt’s novel work as a founding member of the AMA’s Accelerating Change in Medical Education Consortium.
Since the announcement, Anderson Spickard, III, MD, assistant dean of educational informatics and technology at Vanderbilt, has worked among a group of dedicated faculty who are refining the e-portfolio system—and their efforts are paying off.
Faculty have created new ways to collect data on students across multiple written, audio and electronic files. This data is linked to each student’s individual e-portfolio profile and used to create unique performance measures for student training.
One e-portfolio can contain data from faculty, coaches and peers as well as self- or clinical assessments, Dr. Spickard told educators during the AMA’s CHANGEMEDED2015 conference. “We even collect their notes from electronic health records and use natural language techniques to automatically display all of the major concepts of each note,” he said. Faculty use this information to monitor clinical cases students encounter and provide students feedback on their notes in each case. “Accumulating and sorting this information for personal feedback is a part of creating individualized learning, especially for students on clinical rotations.”
Effective data meets effective coaching
“The assessments and information that come into these portfolios, as you can imagine, can be overwhelming,” Dr. Spickard said, which is why students are paired with e-portfolio coaches, who also help them read their portfolios and properly assess their dashboard metrics.
Each student is assigned a portfolio coach for the entirety of their education. Teachers who elect to become portfolio coaches work with 10 students each. Students meet with their coaches three times a year to review their educational data and determine what areas they need to work on.
“This leads to two very important outcomes,” Dr. Spickard said. “One, the student makes reflective summary goals and designs a personal learning plan [that] captures data from his or her portfolio dashboard and timeline. Secondly, the summation from the portfolio coach … offers an important piece of information for Vanderbilt’s promotion committee, [which] evaluates when students are ready to progress to the next stage of training.”
How educators use e-portfolios to chart student progress
During a demonstration of the e-portfolio system, Dr. Spickard logged into an anonymous student’s e-portfolio profile and selected “systems-based practice” from a list of domains on the portfolio dashboard.
A Web page loads with a graph featuring a series of dots—each representing individual forms and notes related to the student’s performance in systems-based practice—mapped over a series of months.
“I can search by date, or I can search by the type of form to see the story of this student as he or she moves along in our new curriculum,” he said, pointing his cursor to one of the dots, which featured information on the student’s grades in systems-based practice.
In one click, he sorted the student’s grades going back to December, noting that “scores were initially low from student and faculty assessments.” But after expanding the dashboard screen, he could see that the student’s systems-based learning grades actually improved by the summer.
“We can also click on one of these student dash points to see who entered the student’s data or form,” he said. This helps faculty determine if the data they’re using for each student represents a high-stakes assessment or one-time encounter with faculty or clinicians.
Tracking assessments is simply one way to use e-portfolio data, Dr. Spickard said. The portfolio system’s fluid functionality allows faculty and students to ask multiple questions about their performance, explores progress trends and compares student assessments across all courses and training years.
Tapping technology for millennials
Because more students are encountering data about themselves, Vanderbilt faculty also are capturing data on students’ receptivity to assessments.
“Currently, we’ve learned that second-year students are rated more highly in receptivity to feedback than they were when they were first-years, and that may just be a development that’s natural or a result of the portfolio review process,” Dr. Spickard said. “And finding such group results only takes about 30 seconds to determine …. This starts to really paint the picture of how we are doing.”
As more data accumulates in the e-portfolio system, Vanderbilt will continue refining how e-portfolios capture student data, Dr. Spickard said.
The school has already launched a mobile app for clinicians to provide quick narrative reports or record audio notes in real time and send them directly to students’ e-portfolios. The app aims to give tech-savvy residents a new way to capture their observations about students in the busy clinical environment.
“We want to connect with millennial learners and meet the expectations they have” for innovative models, technology, learning and progress reports, Dr. Spickard said.
Interested in more med ed innovations?
- Review 9 med ed challenges educators and consortium members want to solve right now.
- Learn how educators are creating the impossible for future physician training.
- Discover the current projects the 11 founding members of the AMA’s Accelerating Change in Medical Education Consortium have underway.
- Find out about the 21 schools that are new members of the consortium.
