Great post. Are you familiar with the Anki tool for spaced repetition? I've found it to be really useful...wouldn't know how to go about spaced repetition without it frankly (for myself, not for teaching) https://ankiweb.net
Somewhat niche, but Anki is the massively preferred study tool by a majority of American medical students right now! I am finishing up my 4th year and pretty much all of my friends and I have been using it throughout med school. Medicine is one of the fields that still requires some of the largest rote memorization (ie what antibiotic works on this bacteria, what is the gene mutation in this super rare lysosomal storage disease), and so spaced repetition learning has risen to the top as the strategy med students leverage to retain all the random facts. It’s super useful and this article was a great representation of how I feel about all the bits and bobs I’ve learned about the body!!
Interesting, I didn't know that. Makes sense that in a field where you have to learn efficiently, spaced repetition has risen to the top. Ironic that so many k-12 teachers don't know what it is.
You have some US MD and DO candidates doing 1000+ anki flash cards per day in their first two years of med school, it’s really a crazy platform that’s shouldering a majority of our foundational learning. Whether that’s good or bad is still up for debate in the med ed community! I tend to argue that anki generally prioritizes memorization (which is needed for the big 8+hr exams) vs understanding (which is needed for dealing with an actual patient who you’re responsible for)
A comparison I would make is that I use flashcards to practice Spanish vocabulary. One part of learning a language is learning, words, there are a lot of words, and flashcards are an efficient way to learn them. But I notice that if I **only** learn a word via flashcards, my understanding of that word is brittle and I struggle to use it in context. If I have a few chances to hear that word in context, use it a few different ways, conjugate it, etc, then I'm much better able to use it. If I only use a word a few times in context and don't use flashcards, then I'm at risk of not retaining it. So I would say that memorization and understanding are both important, but that understanding piece is much harder to do so it's easy to end up on the anki train and ignoring the understanding piece.
There's a book by Gabriel Wyner called Fluent Forever that really embraces cognitive psychology and spaced repetition and the author suggests making several flashcards for each vocabulary item. You have the word in pictures, in writing, and also in sentences specifically related to focused topic areas. The system seems pretty good!
It doesn't eliminate the need for practice, but it does help reduce the decontextualization problem where you only remember in the moment when you're pulling something up on Anki.
I'm curious, to what extent is Anki or spaced repetition more generally incorporated into your formal medical school training, vs. something that you all are doing yourselves outside of class?
Asking b/c I'm a university professor looking for ways to effectively integrate it into my own teaching...I've found it so valuable for myself since adopting ~1 year ago that I want to share it with my students in an optimal way.
With that context, I would recommend either making a deck specific to material you want covered in your courses, or looking up anki decks that are premade and currently available online that cover similar content! The cards are quite easy to make and many tutorials exist on YouTube, all the software is free (expect the phone app, which for $25 I’m pretty sure med students across the world are single-handedly propping up Anki’s costs lol), and then once a good deck is made, it can be handed down generationally for students to leverage
There are “Ohio State specific decks” that med students from my institution have also made that prepare us well for in-house exams too, but again none of that is approved/supported by the school, just heresay of which decks are “good” and would prep you for exams
It is not at all included into our formal education (at my institution or any others that I’m aware of that matter). There are actually “third party resources” of premade anki decks of flash cards that have been effectively selected for that are made for the medical school board exams. For instance, “AnKing” is a well known anki deck of 21,000+ flash cards that medical students will do to prepare for the USMLE STEP1 board exam, which is the major standardized test that all med students have to pass
Thanks Dennis, this is all helpful. That does sound like a pretty remarkable divergence between the formal and informal components of contemporary U.S. medical education w/r/t Anki
I'm a little reluctant to create pre-made decks for the students for two reasons. (1) I don't want it to include material that we haven't covered yet in the semester, and (2) I'm hoping to instill the broader habit in my students of using Anki, meaning creating decks and cards themselves.
So right now I'm leaning toward introducing Anki and spaced repetition to them in an early class session and having them create a deck for the course, then as the term unfolds explicitly identifying key concepts that they'd do well to memorize, and encouraging/assigning them to create corresponding Anki cards.
FYI Michael Nielsen created this online interactive textbook on quantum computing that directly integrates spaced repetition: https://quantum.country/qcvc
First of all, what a great visualization tool!! I primarily work with students who take IB math courses, and I am going to share this blog post with them to help them to think about how to approach their own personal studying/reviewing. Because the IB is a two-year course, when we get into the review season, many Year 2 students struggle with recalling topics that they learned in Year 1... or conversely, they'll spend so much time reviewing those topics instead of the most recent topic (which they think they'll remember easily because they "just learned it") and do very poorly on the practice exam. I'm working with some Year 1's right now and I'm going to send it to them to encourage them to think intentionally about their review, starting now.
Second, this reminds me of Henri Picciotto's idea on "Lagging Homework" (https://blog.mathed.page/2013/06/29/lagging-homework/). He describes this practice as "extending exposure", but really by extending the time period of the exposure, he's also building in more retrieval opportunities (to use your language from the post). To be perfectly honest, I come back to Henri's idea about lagging homework once every few years because I always aim to employ it in my classroom and then the logistics just kind of get a bit overwhelming and I end up re-syncing my homework and my lessons... but I do think that all of the points he makes are very valid and accurate.
Thanks! I look at lagging homework as a convenient half-measure for spacing. It's convenient because you just take your current resources and change the dates. But I see two risks. First, not enough practice right away means students won't learn it the first time. Spacing is great, but that initial boost from some early retrieval is also valuable and I wouldn't want to trade that away. Second, it's still really only one round of retrieval. I'd rather use a shorter round of practice early on, and then multiple rounds of retrieval spaced into the future. Which is harder to execute, thus the appeal of a simple lagged homework approach.
How much of this is just that retrieval is harder than processing an exposure? But that exposure is necessary for retrieval? I think it might be all of it.
The thing is that active production is also more difficult than passive comprehension in lots and lots of areas. (I'm thinking by analogy of P vs NP problems -- identifying solutions is fundamentally easier than generating them.)
So what is the surprise...maybe it's that the feeling associated with exposure and comprehension is so enticing? It FEELS like understanding?
Yea I think at a basic level the phenomenon is "thinking deeply is the best way to learn." But there are some corollaries, including that having something fresh in our minds feels like we have learned it. I often arrive home and say to myself, "ok I need to remember to do x thing" then walk in the house and immediately forget it. I'm convinced in the moment I'll remember, then it's gone. So exposure often fools us because we can't look and see what's in our long-term memory vs what's in our working memory at the moment.
Yeah, seems like maybe the core issue is that the subjective experience of retrieval doesn't *feel* all that different depending on the pathway. Like our subjective experience doesn't notice the pathways. Or something.
Something I've been thinking about is that this feels like what's happening "under the hood" in the forgetting curve/spaced repetition phenomenon. Nothing against the forgetting curve, it's a useful way to help people understand memory. But I wonder what a "first principles" perspective on cognitive science would look like. I'd argue storage strength/retrieval strength would be one element. There are plenty others, and I'm not sure how useful this would be, but it's fun to think about.
Ooh, I really appreciate this — are you saying that this is a different phenomenon than spaced repetition, or that it's the "intrinsic perspective" (to cite the wonderful Erik Hoel) on the same thing?
I'm not sure I understand the meaning of "intrinsic perspective" but that might be what I mean. Storage/retrieval is the underlying architecture of the mind, and spaced repetition emerges out of storage/retrieval. The reason I find storage/retrieval more fundamental is that you can explain other phenomena as well, including the shape of the forgetting curve, the difference between retrieval and relearning, why more effortful learning feels harder but is more effective, and maybe more.
I think there aren't many concepts that lend themselves to a simulation but I'm finding attention and how it links to WM and forming schemas fascinating.
Yea working memory is something I want to think about more. It's not clear to me how working memory interacts with the storage/retrieval framework. My current interpretation is that there's an inverse relationship between storage strength and working memory load. More storage strength = more room to think about other stuff in working memory. But I'm not sure how to connect that to everything else working memory does.
Another good visual would be a sort of timeline or graphic on how education related cognitive science has changed in the last 1-3 decades. I have co-workers who still teach things that happen to prove a decades ago such as multiple learning styles. For me, it would be nice to have an update on what has changed disproven or been discovered since I was in university. Unfortunately, my professional development has never included updates to pedagogical theory. Instead I have to suffer through hour-long workshops on word banks and building relationships
I struggle with cognitive load theory because it feels to me like a constellation of different ideas
I wonder how to put together a timeline like what you describe. I started teaching in 2012. What records are there of popular research from 20 or 30 years ago?
I haven't had a chance to look too close at cognitive load beyond the general idea of finite working memory and how quickly that gets used up. I suppose that is an important caveat - I would not be surprised if psychological models are taken out of context when presented to educators. I witnessed an abysmal and inaccurate presentation on effect size (teachers effect size is .4! Make your effect bigger!) to the staff in my district.
As far as a timeline, I suppose it would be prudent to investigate what was being taught in teacher education programs in the 90s.
Great post. Are you familiar with the Anki tool for spaced repetition? I've found it to be really useful...wouldn't know how to go about spaced repetition without it frankly (for myself, not for teaching) https://ankiweb.net
H/T this essay by Michael Nielsen: https://augmentingcognition.com/ltm.html
Yes! I'm a fan. I put random notes from ed research I wan to remember into flashcards.
Somewhat niche, but Anki is the massively preferred study tool by a majority of American medical students right now! I am finishing up my 4th year and pretty much all of my friends and I have been using it throughout med school. Medicine is one of the fields that still requires some of the largest rote memorization (ie what antibiotic works on this bacteria, what is the gene mutation in this super rare lysosomal storage disease), and so spaced repetition learning has risen to the top as the strategy med students leverage to retain all the random facts. It’s super useful and this article was a great representation of how I feel about all the bits and bobs I’ve learned about the body!!
Interesting, I didn't know that. Makes sense that in a field where you have to learn efficiently, spaced repetition has risen to the top. Ironic that so many k-12 teachers don't know what it is.
You have some US MD and DO candidates doing 1000+ anki flash cards per day in their first two years of med school, it’s really a crazy platform that’s shouldering a majority of our foundational learning. Whether that’s good or bad is still up for debate in the med ed community! I tend to argue that anki generally prioritizes memorization (which is needed for the big 8+hr exams) vs understanding (which is needed for dealing with an actual patient who you’re responsible for)
A comparison I would make is that I use flashcards to practice Spanish vocabulary. One part of learning a language is learning, words, there are a lot of words, and flashcards are an efficient way to learn them. But I notice that if I **only** learn a word via flashcards, my understanding of that word is brittle and I struggle to use it in context. If I have a few chances to hear that word in context, use it a few different ways, conjugate it, etc, then I'm much better able to use it. If I only use a word a few times in context and don't use flashcards, then I'm at risk of not retaining it. So I would say that memorization and understanding are both important, but that understanding piece is much harder to do so it's easy to end up on the anki train and ignoring the understanding piece.
There's a book by Gabriel Wyner called Fluent Forever that really embraces cognitive psychology and spaced repetition and the author suggests making several flashcards for each vocabulary item. You have the word in pictures, in writing, and also in sentences specifically related to focused topic areas. The system seems pretty good!
It doesn't eliminate the need for practice, but it does help reduce the decontextualization problem where you only remember in the moment when you're pulling something up on Anki.
Cool! That makes sense.
I'm curious, to what extent is Anki or spaced repetition more generally incorporated into your formal medical school training, vs. something that you all are doing yourselves outside of class?
Asking b/c I'm a university professor looking for ways to effectively integrate it into my own teaching...I've found it so valuable for myself since adopting ~1 year ago that I want to share it with my students in an optimal way.
With that context, I would recommend either making a deck specific to material you want covered in your courses, or looking up anki decks that are premade and currently available online that cover similar content! The cards are quite easy to make and many tutorials exist on YouTube, all the software is free (expect the phone app, which for $25 I’m pretty sure med students across the world are single-handedly propping up Anki’s costs lol), and then once a good deck is made, it can be handed down generationally for students to leverage
There are “Ohio State specific decks” that med students from my institution have also made that prepare us well for in-house exams too, but again none of that is approved/supported by the school, just heresay of which decks are “good” and would prep you for exams
It is not at all included into our formal education (at my institution or any others that I’m aware of that matter). There are actually “third party resources” of premade anki decks of flash cards that have been effectively selected for that are made for the medical school board exams. For instance, “AnKing” is a well known anki deck of 21,000+ flash cards that medical students will do to prepare for the USMLE STEP1 board exam, which is the major standardized test that all med students have to pass
Thanks Dennis, this is all helpful. That does sound like a pretty remarkable divergence between the formal and informal components of contemporary U.S. medical education w/r/t Anki
I'm a little reluctant to create pre-made decks for the students for two reasons. (1) I don't want it to include material that we haven't covered yet in the semester, and (2) I'm hoping to instill the broader habit in my students of using Anki, meaning creating decks and cards themselves.
So right now I'm leaning toward introducing Anki and spaced repetition to them in an early class session and having them create a deck for the course, then as the term unfolds explicitly identifying key concepts that they'd do well to memorize, and encouraging/assigning them to create corresponding Anki cards.
FYI Michael Nielsen created this online interactive textbook on quantum computing that directly integrates spaced repetition: https://quantum.country/qcvc
Anyways thanks again!
First of all, what a great visualization tool!! I primarily work with students who take IB math courses, and I am going to share this blog post with them to help them to think about how to approach their own personal studying/reviewing. Because the IB is a two-year course, when we get into the review season, many Year 2 students struggle with recalling topics that they learned in Year 1... or conversely, they'll spend so much time reviewing those topics instead of the most recent topic (which they think they'll remember easily because they "just learned it") and do very poorly on the practice exam. I'm working with some Year 1's right now and I'm going to send it to them to encourage them to think intentionally about their review, starting now.
Second, this reminds me of Henri Picciotto's idea on "Lagging Homework" (https://blog.mathed.page/2013/06/29/lagging-homework/). He describes this practice as "extending exposure", but really by extending the time period of the exposure, he's also building in more retrieval opportunities (to use your language from the post). To be perfectly honest, I come back to Henri's idea about lagging homework once every few years because I always aim to employ it in my classroom and then the logistics just kind of get a bit overwhelming and I end up re-syncing my homework and my lessons... but I do think that all of the points he makes are very valid and accurate.
Thanks! I look at lagging homework as a convenient half-measure for spacing. It's convenient because you just take your current resources and change the dates. But I see two risks. First, not enough practice right away means students won't learn it the first time. Spacing is great, but that initial boost from some early retrieval is also valuable and I wouldn't want to trade that away. Second, it's still really only one round of retrieval. I'd rather use a shorter round of practice early on, and then multiple rounds of retrieval spaced into the future. Which is harder to execute, thus the appeal of a simple lagged homework approach.
This is great, Dylan. Super useful visualisation. Thanks so much for sharing. I’ll feature it in 3 Read Friday this week.
Thanks! If you have ideas for other cognitive science topics that would benefit from a visualization, let me know!
How much of this is just that retrieval is harder than processing an exposure? But that exposure is necessary for retrieval? I think it might be all of it.
The thing is that active production is also more difficult than passive comprehension in lots and lots of areas. (I'm thinking by analogy of P vs NP problems -- identifying solutions is fundamentally easier than generating them.)
So what is the surprise...maybe it's that the feeling associated with exposure and comprehension is so enticing? It FEELS like understanding?
Anyway, great post.
Yea I think at a basic level the phenomenon is "thinking deeply is the best way to learn." But there are some corollaries, including that having something fresh in our minds feels like we have learned it. I often arrive home and say to myself, "ok I need to remember to do x thing" then walk in the house and immediately forget it. I'm convinced in the moment I'll remember, then it's gone. So exposure often fools us because we can't look and see what's in our long-term memory vs what's in our working memory at the moment.
Yeah, seems like maybe the core issue is that the subjective experience of retrieval doesn't *feel* all that different depending on the pathway. Like our subjective experience doesn't notice the pathways. Or something.
Gonna use that app! Dovetails very nicely with the usual "forgetting curves / spaced-repetition" chart that I show people when I talk about this.
Something I've been thinking about is that this feels like what's happening "under the hood" in the forgetting curve/spaced repetition phenomenon. Nothing against the forgetting curve, it's a useful way to help people understand memory. But I wonder what a "first principles" perspective on cognitive science would look like. I'd argue storage strength/retrieval strength would be one element. There are plenty others, and I'm not sure how useful this would be, but it's fun to think about.
Ooh, I really appreciate this — are you saying that this is a different phenomenon than spaced repetition, or that it's the "intrinsic perspective" (to cite the wonderful Erik Hoel) on the same thing?
I'm not sure I understand the meaning of "intrinsic perspective" but that might be what I mean. Storage/retrieval is the underlying architecture of the mind, and spaced repetition emerges out of storage/retrieval. The reason I find storage/retrieval more fundamental is that you can explain other phenomena as well, including the shape of the forgetting curve, the difference between retrieval and relearning, why more effortful learning feels harder but is more effective, and maybe more.
Excellent
Brilliant blog and model! It is far easier to visualise than anything else I've seen before, so thank you for taking the time to make it.
Thanks! I appreciate it. Are there other cognitive science topics you can think of that would benefit from a similar visualization?
I think there aren't many concepts that lend themselves to a simulation but I'm finding attention and how it links to WM and forming schemas fascinating.
Yea working memory is something I want to think about more. It's not clear to me how working memory interacts with the storage/retrieval framework. My current interpretation is that there's an inverse relationship between storage strength and working memory load. More storage strength = more room to think about other stuff in working memory. But I'm not sure how to connect that to everything else working memory does.
Cognitive load theory.
Another good visual would be a sort of timeline or graphic on how education related cognitive science has changed in the last 1-3 decades. I have co-workers who still teach things that happen to prove a decades ago such as multiple learning styles. For me, it would be nice to have an update on what has changed disproven or been discovered since I was in university. Unfortunately, my professional development has never included updates to pedagogical theory. Instead I have to suffer through hour-long workshops on word banks and building relationships
I struggle with cognitive load theory because it feels to me like a constellation of different ideas
I wonder how to put together a timeline like what you describe. I started teaching in 2012. What records are there of popular research from 20 or 30 years ago?
I haven't had a chance to look too close at cognitive load beyond the general idea of finite working memory and how quickly that gets used up. I suppose that is an important caveat - I would not be surprised if psychological models are taken out of context when presented to educators. I witnessed an abysmal and inaccurate presentation on effect size (teachers effect size is .4! Make your effect bigger!) to the staff in my district.
As far as a timeline, I suppose it would be prudent to investigate what was being taught in teacher education programs in the 90s.