Cognitive Science

"Learning is highly individual."

RememberMore was a direct response to Graham Nuthall's research that:

"In our research has found that students already know, on average, about 50 percent of what a teacher intends his or her students to learn" and that "different students will know different things, and all of them will know only about 15 percent of what the teacher want them to know,” (Nuthall, 2007 p35).*

With the benefit of a growing interest in research-informed pedagogy, together with Alex Warren, we set about a series of iterative classroom trials that would lead to the development of RememberMore, making learning - highly individual and Successive.

ResearchEd Surrey: Surrey 2020

ResearchEd Surrey - Google Slides.mp4
Record in Sept 2020 - it is on my today list to re-record this presentation as retrieval continues to be a vibrant topic of research. (15.04.21)

Research Informed. Experience led.

RememberMore draws heavily on Cognitive Science, specifically on the direct and indirect learning gains of "The Testing Effect" or Retrieval Practice and Successive Relearning (retrieval practice and spacing), Memory and Forgetting, Confidence-based assessment, Metacognition, Attention and Flow (and more) and educator expertise, educator and learning feedback.

"Learning is rarely a one-shot affair. Single, isolated experiences seldom give birth to learning. What creates or shapes learning, is a sequence of events or experiences, each building on the effects of the previous one," Nuthall (2007, p155).

First we will review the direct and indirect benefits of "The Testing Effect" or Retrieval Practice as we move to underline the benefits and importance of Successive Relearning.

Retrieval practice

"Does retrieval practice improve Learner learning in school and classroom settings? Based on our literature review, our response for researchers and educators is an unequivocal 'yes.'" (Agarwal, et al, 2020)

“Retrieval practice consistently benefits Learner learning.” Agarwal, et al, (2020).

As Prof Rob Coe, summarises, the majority of studies come from laboratory settings (223 vs 30 in classrooms according to the meta-analysis from Adesope et al, 2017), the effect sizes are similar in both (0.62 for lab studies vs 0.67 for classroom). The small number of studies conducted in primary schools (10 effects, mean 0.64) and secondary schools (19 effects, mean 0.83) are also comparable in their results to those in post-secondary settings (228 effects, mean 0.60).

"If the eventual goal is to be able to retrieve that knowledge from memory, perhaps practicing retrieval of that information would be a better way to learn. Indeed, retrieval practice is one of the most effective ways of solidifying new knowledge, although this fact is underappreciated by most learners (and teachers)," McDermott (2020).

Retrieval practice not only enhances students’ memory performance in a future test (Roediger & Karpicke, 2006) but also helps them make more accurate predictions about future performances (Ariel & Dunlosky, 2011; King, Zechmeister, & Shaughnessy, 1980). Furthermore, that "to be able to retrieve, use, and apply knowledge in the long term, it is highly effective to practice retrieving, using, and applying knowledge during learning," (Karpicke & Aue, 2015, p.318).

Adesope, Trevisan, and Sundarajan (2017) concluded in their meta-analysis that "practice tests are more beneficial for learning than restudying and all other comparison conditions," such as restudying (e.g., reviewing notes and homework) and filler activities such as resting, solving puzzles, and so forth. The average effect size was 0.61 (which is considered high to very high). These benefits persisting across a wide array of educational levels, settings, and testing formats and procedures.

These meta reviews consolidate years of research. "Retrieval practice is strongly supported by over 100 years of research and is one of only two learning techniques rated by Dunlosky et al. (2013) as having ‘high utility’ for classroom practice."

"The benefit of retrieval practice is one of the most robust findings in cognitive psychology," Roediger & Karpicke, 2006; Storm, Bjork & Storm (2010).

"Retrieval Practice alone can provide improved recall performance by as much as 10-20%. When combined with Spaced Retrieval, the effect is multiplied," Dobson, (2013).

"Practicing retrieval one time doubled long-term retention, and repeated retrieval produced a 400% improvement in retention relative to studying once." Karpicke and Roediger (2010).

The act of retrieving information from memory actually alters the retrieved memory by elaborating on the existing memory trace and/or creating additional retrieval routes. One consequence of these changes is that the probability of successful retrieval in the future is increased, making testing a potent mechanism for enhancing long term retention.” Roediger, H. L., & Butler, A.C. (2013).

"The process of retrieval itself alters knowledge in anticipation of demands we may encounter in the future. Retrieval is therefore not only a tool for assessing learning but also a tool for enhancing learning," (Roediger & Karpicke, 2006a).

Retrieval practice strengthens our memory and long-term learning with research showing that the active process of retrieving information, improves learners complex thinking and application skills, their organisation of knowledge and their ability to transfer of knowledge to new concepts. Retrieval practice is not merely memorisation – it increases understanding.

Based on Rowland’s (2014) meta-analysis, retrieval practice effects become more robust as initial retrieval success increases, especially when initial retrieval is greater than 75%. Second, across 159 studies, the overall effect size of retrieval practice relative to repeated study was g - 0.50, and 81% of comparisons favoured retrieval practice over repeated study.

Not only does retrieval practice helps us retain learning, it help us figure out what we do know. There is a degree of dialogic thinking required when you assess your own level of confidence. This crucial benefit of retrieval practice is called metacognition, or awareness of what we know and don’t know. Add to that the know benefits of prediction, Crouch, et al., (2004).

Studies using an fMRI showed that retrieval practice, compared with re-study, involved more monitoring and working memory-related brain activity (Liu, Liang, Li, & Reder, 2014). Is this where the learning reside? This would suggest that students may benefit from practice tests prior to an exam, which can not only improve their exam performance, but also allow for better metacognitive monitoring based on their subjective experience during the practice test.

Practicing retrieval one time doubled long-term retention relative to reading the text once (34% vs 15%), and engaging in repeated retrieval increased retention to 80%.

Long-term retention after studying once, practicing retrieval once (followed by rereading), or practicing repeated retrieval. Practicing retrieval one time doubled long-term retention, and repeated retrieval produced a 400% improvement in retention relative to studying once" Karpicke and Roediger (2010).

This is particular important as learners will report re-reading as more effective - when in fact learners who use retrieval practice remember over 50% more than those learners who adopt re-reading strategies. See Karpicke and Blunt (2011).

Glover’s (1989) study, final recall was greater when an initial test had occurred. Both college students remembering an essay and seventh-grade students learning parts of a flower benefited from having taken a test (without feedback) 4 days before the final test. Data from Glover (1989).

All students were given a final test 4 days after the study session. In the intervening time, however, about half the students were asked to retrieve the studied material 2 days after the initial study session, with no instructor feedback given on this initial test.

Performance on the final test differed markedly as a function of whether a prior retrieval attempt had occurred. For essays, the students who had taken the initial test outperformed those who had not (0.38 and 0.16 recall probabilities, respectively). Similarly, for diagrams, the students taking an initial test outperformed those who had not (0.37 and 0.20 recall probabilities, respectively). This difference on a final test as a function of whether or not an initial test occurred is referred to as the testing effect, Glover (1989).

Students’ metacognitive judgements of learning (predicted recall) opposite to the pattern of students’ actual long-term retention.

The more times students repeatedly read the material, the better they believed they had learned it. However, students’ actual learning exhibited the opposite pattern. The more times students practiced actively retrieving the material, the better they retained it in the long term.

What is deeply alarming is that 93% of students students incorrectly endorsed massed study as being more effective for learning than spaced study (McCabe, 2015).

Final recall (a) after repeatedly studying a text in four study periods (SSSS condition), reading a text in three study periods and then recalling it in one retrieval period (SSSR condition), or reading a text in one study period and then repeatedly recalling it in three retrieval periods (SRRR condition).

Practicing retrieval produces greater gains in meaningful learning than "Study," "Repeated Study" and elaborative studying with "Concept Mapping" across a range of testing conditions (verbatim and inference short-answer questions). What is more, this was in stark contrast to what the students predicted, Karpicke and Blunt (2011).

The average percentage of correct test questions for each group was 67% for retrieval practice, 27% for study once, 49% for repeated study, and 45% for concept mapping.

The two experiments reported here establish the potency of a successive relearning intervention for enhancing student learning by demonstrating meaningful improvements in course exam performance and on long-term retention tests. (Rawson et al., 2013)

Learners who used a successive relearning strategy over a 56-day period could remember more than 75% of the information after a year and 60% of the information after five years, Janes, Dunlosky, Rawson and Jasnow, (2020).

What is more, student predict that four periods of reading and re-reading (red) would positively predict performance and that retrieval or testing (green) will be far less effective. When quite simply - the reverse is true. Simply, students may not know what is best for their learning.

Even when they do know the benefits, even when they have experienced the benefits, 58% of students, who knew better, indicated that they would not practice active retrieval.

"The effects of retrieval practice were largely the same for children with varying levels of reading comprehension and processing speed scores," (Karpicke, Blunt and Smith, 2016).

Sotola and Crede (2021) found that, across 52 independent classroom studies (N = 7864) students were 2.5 times more likely to pass a course if their instructor incorporated frequent low-stakes quizzing than if the instructor did not. 52 independent real classes (N = 7864) suggests a moderate association (d = .42) between the use of quizzes & academic performance. Effects are even stronger when quiz performance contributed to class grades (d = .51). Food for thought.

More than 20% of school age student Argawal et al., (2014) and more than 50% of college students Miyatsu et al., (2018) report using flashcards.

The was already a significant body of research focused on the direct and indirect benefits of testing before this area of research saw a resurgence in around the early 2000s. Subsequently, the research boundaries were expanded, investigating the spacing or distribution (blocked or interleaved), the role of feedback and mode retrieval (free or recognition recall for example) including computer assisted learning shortly afterwards.

I first became aware of retrieval practice in 2011 following the Purdue University press release and started exploring it's use in my own teaching a little while after,

On revisiting retrieval practice in 2019, aware of the broadening research base, I became more interested in Dr Kathleen Rawson research, in Successive Relearning and swiftly become focused on leverging these benefits for my own students.

The press release is still available.

Why Successive Relearning

Like most educators, I first encountered retrieval practice before my investigations led to the work Dr Kathleen Rawson and Successive Relearning - specifically, Rawson, Dunlosky, & Sciartelli, (2013).

Conducted in an authentic Introductory Psychology course, college students learned 64 course concepts using a computerised flashcard program under the following conditions.

  • Successive Relearning: Students engaged in retrieval practice spaced throughout the semester, and they had to retrieve information correctly three times.

  • Spaced Restudy: Students engaged in restudying, spaced throughout the semester (without retrieval) - arguably the most common approach

  • Baseline: Students engaged in "business as usual" without using the flashcard program

In two experiments, the successive relearning condition increased students' course exam performance by a letter grade (from a C to a B) compared to the spaced restudy and baseline conditions, consistent for both higher and lower performing students and more robust for three spaced retrieval sessions compared to only one retrieval session. I simply wanted that same benefit for my students.

However at this point, I had not connected Rawson's findings with Nutthal's. Second, I would later recognise that this paper introduced three important concepts: initial learning to criterion, repeated, spaced, relearning and the pitfalls of self-regulation (when students were able to "drop cards" there was no benefit of retrieval on exam performance) also consistent with prior research. And so, my focused turned to Successive Relearning which much more closely represented classroom interactions, that is repeated retrieval, spaced and with feedback, rather than merely retrieval practice.

Successive Relearning (retrieval to criterion + spacing + feedback) Spacing and Scheduling

It is well known that spaced practice, in which time or other events occur between repetitions, promotes better learning and long-term retention than does massed practice, in which repetitions occur in immediate succession without intervening events. To start that review, you when need to start with the research of (Bahrick, 1979). Of particular interest here, spacing has larger and longer-lasting effects on retention when practice trials for a given item are distributed across days versus within a session (Cepeda et al., 2006; Kornell 2009).

"When you want to learn something that that's going remain with you, that you can retain for the long term, you know one exposure, you know one encounter with the information is probably going to be insufficient... one shot, one encounter with a set of materials isn't likely to promote long term retention so we know we in order to retain something well it's probably the case that we need to re-encounter, revisit, review the information again and again." Dr Sean Kang (Spacing Study Sessions Enhances Learning).

Rawson and Dunlosky (2011, 2012a) demonstrate undergraduates who successively relearned key concept definitions showed relatively high levels of retention on cued recall tests administered in the lab 1 and 4 months after relearning (up to 68% and 49% respectively, depending on the particular schedule of relearning, as compared with around 11% in a baseline control condition).

Relearning potency - is more that just a dosage effect. Three relearning sessions boosted 1-week retention to 80% even after a 3-week delay (76%). By comparison, the most effective combination of the two initial learning conditions only yielded 38% retention after 1 week.

Items correctly recalled three times during initial learning, which yielded 26% retention 1 week later. By comparison, items correctly recalled a total of three times across initial learning plus two relearning sessions; this schedule yielded 68% retention 1 week later. Thus, correctly recalling items one time in three sessions versus three times in one session yielded a 262% increase in retention test performance (d 2.17) Rawson et al., (2018).

Which, reconnected with Nuthall's research. Nuthall's emphasis on "the accumulation of at least three different sets of complete information about a concept makes the difference between a concept that is never quite learned and one that firmly connects to and integrates with previous knowledge, and hence is learned and remembered."

The need to "connect and integrate" knowledge (tags) and lastly the his use of the term "successive," now jumped off the page, having read so much of Dr Katherine Rawson's research. Nuthall's remarkable prediction rate for concepts learned (88%) and those not learned (85%) concepts, (Nuthall 2007, p127) - now garnered my attention.

More recently, in two applied research experiments, Successive Relearning boosted students’ learning of course content by at least 10%, Janes et al., (2020).

The advantage of successive relearning over single-session learning (i.e., relearning potency) was substantial. 50, 70, 80 markers for relearning, once, twice and thrice times, are useful if approximate. Notably, longer lasting retention or durability, 1 week, and 3 weeks later, was reported, Rawson et al., (2018).

Of greatest interest, did successive relearning enhance performance on actual course measures? "The answer is decisively yes: Successive relearning improved performance by more than a letter grade based on the grading metric used by the instructor (84% versus 72%) for course exam questions tapping practiced concepts versus baseline control concepts, Rawson et al., (2018). Importantly, a similar effect was obtained when successive relearning was unsupervised [t(22)=2.01, p=0.029, d=0.55], establishing the potency of successive relearning even when implemented under conditions with potentially lower fidelity than when administered in the lab.

Durability and efficiency: With both short-term and long-term goals in mind, Rawson and Dunlosky (2011) offer the following recommendation. Practicing to three correct recalls during initial learning, followed by three subsequent relearning sessions.

With three recalls during initial learning, learners correctly recalled 80% of the content 2 days after a second relearning session and and 77% 1 week after a third relearning session, and 57% recall after 4 – 6 weeks. With respect to efficiency, the 3+3 schedule is preferable to schedules involving more relearning sessions as additional relearning session increased recall by only 2%. Learning gains increased across items when quizzed once (g ! 0.444), twice (g ! 0.601) and three times or more (g ! 0.642) Unlimited attempts (g ! 0.762).

Most recently, Higham et al., (2021) reported that recall of course material at the end of the semester was better for relearning compared to restudying. Perhaps more importantly, that increased recall during relearning sessions was associated with further learning benefits including improved metacognition, increased self-reported sense of mastery, increased attentional control, and reduced anxiety. Furthemore, that students found successive relearning to be enjoyable and valuable.

Spacing and Scheduling

Cepeda et al., (2006) research summary of 254 studies involving more than 14,000 participants found that students scored about 10% higher on tests after spaced study than after massed study.

Several studies have also reported positive outcomes from administering summative assessments that are shorter and more frequent rather than longer and less frequent (e.g., one exam per week rather than only two or three exams per semester), not only for learning outcomes but also on students’ ratings of factors such as course satisfaction and preference for more frequent testing (Dunlosky, 2013).

Kim et al., (2019) the probability of retaining information in memory for a longer period of time (e.g., a month or longer) is higher if the spacing interval is also long (e.g., 11 days or longer). In contrast, shorter spacing intervals (e.g., one day) have been found to be more beneficial for shorter retention intervals (e.g., one week). Thus, the optimal amount of spacing between an initial learning event and a relearning event increases as the retention interval increases.

In fact, an influential review aimed at informing best practices in teaching and learning in educational settings concluded that the interval between two study occasions should be approximately 10% to 20% of the retention interval (Pashler et al., 2007).

In the experiment (Karpicke & Bauernschmidt, 2011), students learned a list of foreign language words (e.g., Swahili vocabulary words like "mashua — boat") across cycles of study and recall trials. Merely studying the words once without ever recalling them produced extremely poor performance (average recall was 1%). Practicing until each translation was recalled once was much better.

Massed retrieval — repeating the translations three times immediately — produced no additional gain in learning. Repeated retrieval enhanced learning only when the repetitions were spaced, and indeed, the effects of repeated spaced retrieval were very large. In a single experiment, simple changes that incorporated spaced retrieval practice took performance from nearly total forgetting to nearly 75% one week after an initial learning experience. Another way of looking at it is that repeated retrieval with long intervals between each test produced a 200% improvement in final recall relative to repeated retrieval with three massed tests, Karpicke & Bauernschmidt (2011).

There is a wealth of research highlighting the benefits of spacing over massed cramming. Using spacing instead of cramming, can result in a 10% to 30% difference in final test results - in the long term. This finding has been found throughout a range of tasks, including remembering key words, random facts or solving maths problems.

Learning gains increased across items when quizzed once (g = 0.444), twice (g = 0.601) and three times or more (g = 0.642) Unlimited attempts (g = 0.762) Yang et al (2021). Overall, hence an effect size of g = 0.499 can be roughly translated into a change in GPA of about 0.30 – 0.35 points. This is equivalent to moving from the 50th to the 69th percentile of a normal distribution. Although g = 0.499 is only a medium-sized effect according to conventional descriptors, it is a notably large effect by the standards of educational interventions.

Where g = Hedges Effect Size. Small 0.2, Medium 0.5 and Large 0.8.

"Personalized spaced" practice improved retention by 12.4% over massed practice, 8.3% over generic spaced practice. 1 month later, 16.5% over massed practice, 10.0% over generic spaced practice. Lindsey et al., (2014).

Research has shown that the longer you need to remember information, the more powerful the "Testing Effect" is and the less effective simply reading the content is.

Within-subjects, students retained spaced content better than massed content in the precalculus course. Between-subjects, students for whom some retrieval practice was spaced, compared to those for whom all practice was massed, performed better on the final exam in the precalculus class and on exams in the calculus class. These findings suggest that spaced retrieval practice can have a meaningful, long-lasting impact on educational outcomes, Hopkins, et al, (2016).

Research has shown that there may be an optimum gap to leave depending on when you’ll need to retrieve the information. RememberMore ensures that you inform that gap.

I know you are going to ask next... What is the recommended spacing gap between study or retrieval practice sessions? "If you want to know the optimal distribution of your study time, you need to decide how long you wish to remember something." Cepeda et al., (2008).

A gap of between 10% and 20% of the “total time you want to retain the information” can only be given as a guide. For example: If the test is in a month, you should review the information around once a week. If the test is in a week, review the information daily.

The idea behind expanding retrieval is intuitive, and for many years it was recommended as the optimal spaced repeated retrievals schedule. Whilst expanding schedules (0-1-5-9) are far superior to massed schedules (0-0-0-0), the surprising finding in several recent studies has been that equally spaced schedules produce retention that is the same as or sometimes better than retention produced by expanding schedules , Karpicke and Roediger, (2007a).

On spacing of flashcards specifically, Miyatsu et al., (2018) makes two recommendation. First, students should keep studying and testing themselves even after they get an item correct, contrary to "the conventional wisdom that once you get an item correct, you should stop studying it," hereby strengthening learners memory for the target information, and avoiding the pitfall of inaccurate metacognition (i.e., not discriminating between learned and unlearned information). Check. RememberMore ensures that this happens. Second, students should space out their studying of a given flash card. Check.

Most recently, working in Year 7 maths classrooms, Emeny, Hartwig & Rohrer (2021) reported that spaced practice produced higher test scores than did massed practice, and test score predictions were relatively accurate after spaced practice yet grossly overconfident after massed practice.

"Teacher should shift their mindset so that the practice of a skill or concept is seen not as material that should be squeezed into 1 or 2 consecutive class meetings but rather as material that can be distributed across many lessons.” Williams Emeny.

Ten Benefits of Testing

Testing in schools and in training, are usually employed diagnostically. Undertaken for purposes of assessment, to assign learners grades, or rank or both. Yet tests can serve other purposes in educational settings that greatly improve performance:

Benefit 1: The Testing Effect: Retrieval Aids Later Retention

Benefit 2: Testing Identifies Gaps in Knowledge

Benefit 3: Testing Causes Students to Learn More from the Next Study Episode

Benefit 4: Testing Produces Better Organisation of Knowledge

Benefit 5: Testing Improves Transfer of Knowledge to New Contexts

Benefit 6: Testing can Facilitate Retrieval of Material That was not Tested

Benefit 7: Testing Improves Metacognitive Monitoring

Benefit 8: Testing Prevents Interference from Prior Material when Learning New Material

Benefit 9: Testing Provides Feedback to Instructors

Benefit 10: Frequent Testing Encourages Students to Study

Roediger H, Putnam A, Smith M, et al. (2011) Ten benefits of testing and their applications to educational practice. In: Mestre J (ed.)

Why Forgetting is important

“It is natural to think that learning consists of building up knowledge or skills in our memories and that forgetting is losing some of what was built up. The relationship between learning and forgetting, however, is not so simple and is, in some respects, quite the opposite. One of the “important peculiarities” of human learning is that certain conditions that produce forgetting — that is, decrease our ability to access what we have stored in our memories — actually create opportunities to enhance our level of learning,” Bjork and Bjork (2019).

In fact, “the act of retrieving information from memory actually alters the retrieved memory by elaborating on the existing memory trace and/or creating additional retrieval routes. One consequence of these changes is that the probability of successful retrieval in the future is increased, making testing a potent mechanism for enhancing long term retention,” (Howard, 2014).

“People usually believe that forgetting happens over time; if you don’t use a memory, you lose it. This may be hard to believe, but sometimes the memory isn’t gone—it’s just hard to get to. So, more important than the passage of time or disuse is the quality of the cues you have to get to the memory.” Dan Willingham

“Just understanding the power of retrieval practice is crucial… we carry around a flawed mental model of how we learn and remember.” Robert Bjork

A knowledge of forgetting is almost as important to effective retrieval practice as a knowledge of learning. “In very short order we lose something like 70 percent of what we’ve just heard or read. After that, forgetting begins to slow… but the lesson is clear: a central challenge to improving the way we learn is finding a way to interrupt the process of forgetting,” (Brown et al., 2014, p. 28).

Motivation and Improved Attendance

Numerous studies have provided supporting evidence for the motivation explanation. Schrank (2016) found that class quizzes increase attendance; Heiner, Banet, and Wieman (2014) reported that a preannounced quiz encourages students to read the assigned textbook material and prepare better before class; Yang et al. (2017) showed that frequent tests drive learners to allocate more time to learning; Szpunar et al. (2013) observed that learners make more notes when they are frequently tested; Jing et al. (2016) found that frequent tests reduce task-unrelated thoughts (i.e., mind wandering) while watching lecture videos; and Weinstein et al. (2014) found that frequent tests induce high test expectancy which in turn boosts test performance.

"The impact of achievement on self-concept is greater than the impact of self-concept on achievement."

"Lack of motivation is a logical response to repeated failure." Caroline Spalding

Pekrun et al., (2017) reported a "reciprocal effects model of emotion and achievement," where positive emotions (enjoyment and pride) positively predicted their subsequent end-of-the- year math grades, and grades, in turn, positively predicted the development of positive emotions. Where Math-related negative emotions (anger, anxiety, shame, hopelessness, and boredom) were negative predictors of subsequent math grades, and grades, in turn, were a negative predictor for the development of negative emotions.

These findings were consistent for seven discrete emotions, four time intervals, two different measures of achievement (grades, test scores), while controlling for students’ gender, intelligence, and critical demographic background variables.

Remember, "emotions indeed have an influence on adolescents’ achievement, over and above the effects of general cognitive ability and prior accomplishments," Pekrun et al., (2017).

These findings go beyond correlational evidence for achievement and emotion, disentangling the directional effects underlying the emotion-achievement link. It is why 'Reorder' is such an important feature in RememberMore. With "success is expected to generally increase perceived control, thus enhancing positive emotions, and failure is expected to decrease control, leading to negative emotions." Pekrun, et al., (2017).

Emotions have effects on "adolescent" students’ academic achievement and that these effects are not merely a by-product of prior performance. More likely, they represent a true causal influence of students’ emotion experiences. Therefore: who do we strengthen adolescents’ positive emotions (and minimize their negative emotions).

Pekrun, et al., (2017) outline that the results imply reciprocity. Providing students with opportunities to experience success and mastery over competition goals may help to promote positive emotions (and prevent negative emotions) Pekrun, et al., (2014) .

Kriegbaum, et al., (2018) review the relative importance of intelligence and motivation as predictors of school achievement: A meta-analysis of 74 studies (N = 80,145). They reported average correlations between intelligence (r = 0.44) and motivation (r = 0.27) with school achievement and between intelligence and motivation (r = 0.17). A path model showed that 24% of variance in school achievement was explained overall, of this overall explained variance in school achievement, 66.6% was uniquely explained by intelligence and 16.6% uniquely by motivation. Thus, the results show that both intelligence and motivation contribute substantially to the prediction of school achievement.

On success-rates with RememberMore look to keep success rates. When assessing students’ individual work and oral responses to class discussions, Rosenshine found that the most effective fourth-grade math teachers had a student success rate of 82%. On the other hand, the least effective math teachers only had success rates of 73%. So aim for the upper end and above of this band and make use of the 'Reorder' feature in Classroom.

Attention and Flow: Simple and clean by design

RememberMore is designed and built on the fundamental principle of memory - that "memory is as thinking does." Based on the tenets of ‘flow’ and the two forms of effort:

  • Concentration on the task - (simple and clean)

  • Deliberate control of attention - (minimising distraction)

The most important design criterion for learning is that the most relevant pieces of information are attentionally focused upon and thus processed deeply. RememberMore harnesses and directs attention, first externally, then internally and finally metacognitively, whilst at the same time minimising all other distractions or competition for it.

The construction of RememberMore also utilises question cueing, permitted shorter, simpler questions, reducing error rates, improving productivity, concentrates and directs attention. Shorter questions also more accurate self-assessment.

Simply: RememberMore is optimised for learning.

(Furthermore ‘notes’ extend the learning opportunities with ‘Key Questions’ icon signposting key content or Threshold Concepts.)

"Memory is the residue of thought," Daniel Willingham (2003).

Why ask learners to successfully recall a card more than once?

"Restudying an item that one can already retrieve correctly can have enormous memory benefits." Bjork and Kornell (2008). It also address the fundamental learning flaw - "stability bias," that learners "overestimating remembering and underestimating learning."

Why ask learners to successfully recall a card after it has been learnt?

When making study decisions, it is natural to focus on items that one has struggled with, but it is important to realize that memory is anything but stable (Kornell, 2012). Items that were answered correctly can be forgotten, and items that were not answered correctly at one point may have been learned during subsequent studying.

Cognitive Load Theory

Cognitive load theory (CLT) is an instructional theory based on human cognitive architecture and evolutionary psychology (Sweller et al., 2011; Sweller et al., 2019). It can be described by five fundamental principles (Sweller & Sweller, 2006).

Cognitive Load Cognitive Load Theory was recently described by British educationalist Dylan Wiliam as the single most important thing for teachers to know,’ (Wiliam 2017).

Cognitive load theory is built upon two commonly accepted ideas. The first, that there is a limit to how much new information the human brain can process at one time. The second, there are no known limits to how much stored information can be processed at one time.

"The implications of working memory limitations on instructional design can hardly be overestimated... Anything beyond the simplest cognitive activities appear to overwhelm working memory. Prima facie, any instructional design that flouts or merely ignores working memory limitations inevitably is deficient," Sweller, van Merrienboer & Paas (1998).

Most cognitive load effects are caused by an instructional procedure that overwhelms working memory during learning. Learning is enhanced when compared with an alternative instructional procedure that reduces working memory load. For example, presenting information in a split-attention format requires the use of more working memory resources than presenting the same material in a physically integrated format, leading to the split-attention effect. If intrinsic cognitive load is low because element interactivity is low, the effect will not be obtained because even under split-attention conditions, total cognitive load does not exceed working memory resources.

The aim of cognitive load research is therefore to develop instructional techniques and recommendations that fit within the characteristics of working memory, in order to maximise learning. Cognitive Load Theory supports explicit models of instruction, because such models tend to accord with how human brains learn most effectively (Kirschner, Sweller & Clark 2006).

Explicit instruction involves teachers clearly showing students what to do and how to do it (rather than having learners discovering or constructing information for themselves). It will come as no surprise then, that RememberMore is built, or at least designed with Cognitive Load Theory very much in mind, minimising Split Attention and Redundancy Effects.

Confidence-based assessment and metacognition

Metacognition and self-regulation approaches that encourage learners to think about their own learning (often by teaching them specific strategies for planning, monitoring and evaluating their learning) have consistently high levels of impact with learners. However, as Rivers (2020) summarises,

"Without support, learners lack metacognitive awareness of testing as a tool to enhance memory but do recognise that testing can be used as a monitoring tool."

Learners can accurately monitor their learning while using practice testing when judgements are made in contexts that are representative of those encountered during a criterion test. That this accuracy improves as learners get older.

In educational contexts, learners report using less effective strategies equally or more often than practice testing. Learners tend to test themselves only under conditions that encourage retrieval success, and rarely use a strategy involving repeated successful retrieval even when it would lead to improved retention.

Testing can improve both the absolute and relative accuracy of metacognitive judgements. Absolute accuracy refers to the degree to which judgement corresponds with performance, whereas relative accuracy refers to the degree to which learners can discriminate between material that is better or less well learned. This was explored using RememberMore and reported under my Our Story.

Couchman et al., (2014) Real-time metacognitive monitoring – measured by confidence ratings for each individual question – accurately predicted performance and were a much better decisional guide than retrospective judgements.

It is clear that metacognition and Successive Relearning are intertwined.

Learners are frequently, overconfident regarding their learning status, and test performance provides diagnostic feedback to inform them about the gap between their anticipated and actual learning level (Szpunar, Jing, & Schacter, 2014), which then motivates them to expend more effort to narrow the perceived gap. In addition, frequent tests may induce high test expectancy (i.e., expecting to be tested subsequently), and test expectancy is an important motivator driving students to commit more effort to prepare for subsequent tests (Agarwal & Roediger, 2011; Szpunar, McDermott, & Roediger, 2007; Yang, Chew, Sun, & Shanks, 2019).

Confidence is the hall-mark of metacognitive judgements and the most commonly used for investigating metacognition. These predictions are usually made on a numeric confidence scale expressed as percentages; e.g. 0–20–40–60–80–100% (with the highest classification accuracies at the terminal ends of the scale) or binary (yes/no). Subjects’ predictions are good (much better than in the standard JOL procedure, in which the cue and target are presented together at study), Putnam & Roediger, (2013) as they are in RememberMore.

Confidence-based assessment is not a new idea. Underwood (1966) demonstrated that individuals predicted their own learning with considerable success, more recently Siedlecka et al., (2016) reported confidence accuracy percentages above 70%, "response accuracy always correlated with confidence." With Dougherty and colleagues (2005) reporting that both retrospective judgement about how confident they were that their answer was correct, and a Judgement of Learning about how confident they would be able to recall the target word in a later test, are both highly correlated. Finally (Chen et al., 2019) reported improved coefficients r=.655 between judgements post testing, significantly higher than the correlation for re-study trials, r=.338, t=8.773, p<.001. With cured recall, superior to multiple choice or recognition recall.

Also that JOLs decreased with increasing study time, Hoffmann-Biencourt et al., (2010).

Furthermore, self-testing, as promoted by RememberMore, produces higher performance and that learners’ metacognitive control becomes more effective with experience, Kornell and Son (2009). On self assessment, the two meta-analyses (Graham et al., 2015; Sanchez et al., 2017) demonstrated a positive association between self-assessment and learning, on average, “students who engaged in self-grading performed better (g = 0.34) on subsequent tests than did students who did not.”

Four reasons to encourage self-assessment with RememberMore

  • Increases learning (fills gaps in knowledge) and academic performance (improved accuracy of performance) and transfer to new questions

  • Improve self-regulation (of which learning strategy are effective and which to employ)

  • Enhance self-efficacy and agency (motivation-success-motivation)

  • Gains can be achieved without supervision (RememberMore is personalised and adaptive)

We prompt self-assessment with RememberMore to encourage learners to think about what and how they are learning (meta-cognition) and to take advantage of the Hypercorrection Effect (errors endorsed with higher confidence are more likely to be corrected on a the final test than are errors endorsed with lower confidence) in that RememberMore always provide the retrieval prompt answer. It is also important to note that "...confidence-weighted practice tests led to greater benefits in the ability of test-takers to answer new but related questions than did standard multiple-choice practice tests," Sparck et al., (2016).

There are essentially two types of metacognitive judgements. Both prospective and retrospective judgements correlate with actual performance accuracy (Chua et al., 2009) with confidence offering a "significant effect of the confidence judgement on final test performance." RememberMore employs prospective thinking (during pre-reveal) and Retrospective Confidence Judgements (RCJs) to adapt the Successive Relearning spacing. Learners who make RCJs prior to their restudy decisions are more accurate at identifying items in need of being restudied, nudging learners "toward better utilizing of valid information when deciding which items are in need of further study." (Dougherty, et al., 2005; Hines, et al., 2009; Wattier & Collins, 2011; Robey, et al., 2017).

Retrospective confidence judgements (RCJs) are better predictors of memory recall than Judgements of Learning.

We know that retrieval practice is a powerful memory modifier. Thinking about something changes it. Together with Sadler's (2006) observation that "The act of assessing one’s own judgement of learning is one of the most effective ways to deepen a memory trace," we adopted Confidence-based assessment knowing that the connection of confidence and knowledge provides an acceleration of learning and improves student performance, (Adams and Ewen, 2009).

Also be aware that learners (younger, male and low attaining) tend to be overconfident in predicting their own learning - what is referred to as the “stability bias.” Learners tend to terminate their encoding before materials are sufficiently committed to memory and therefore 20% over learning is recommended. Note that with RememberMore, cards will always be retained and reviewed, even once "coded" as memorised. There is no advantage to "memorising" cards faster.

As Dylan William states “the best person to mark a test is the person who just took it.” (

RememberMore aligns with the evidence that strongly suggests that self-assessment is most beneficial, in terms of both achievement (performance) and self-regulated learning (progress), when it is used formatively.

We concluded that confidence may be a good marker for accuracy with cued recall. (Luna and Martín-Luengo, 2011). The Confidence-Accuracy relationship tends to be much stronger in studies of recall memory. Leading to the conclusion “that confidence may be a good marker for accuracy with cued recall.”

Research suggests we are more likely to be slightly under-confident in the knowledge we are unconfident of, and a little over-confident of the knowledge we are confident of. One of the metacognition benefits of retrieval practice is that it improves the calibration of students’ judgments and reduces overconfidence (Hacker and Bol, 2019) and engaging in practice testing (as compared to restudying) can also improve relative monitoring accuracy.

Interestingly, in the workplace, Confidence-based assessment may have an even greater impact.

"The fusion of knowledge and confidence that leads to behavioral outcomes and empowers people to act. People who are confidently correct will take actions that are productive," Bruno, (1995).

One key benefit of Confidence-based assessment is that a "critical factor in retrieval-based learning is initial retrieval success." Confidence-based assessment always presents the retrieval target.

Metacognitive monitoring appears to be an independent skill that can be cultivated separately from knowledge-based learning (Swanson, 1990) and may have profound implications for education.

Barenberg and Dutke (2019) study examined the potential of retrieval practise during learning to improve the accuracy of confidence judgements in future retrieval. In the final test, the proportion of correct answers and the proportion of confident answers were higher with retrieval practice than compared to the control condition. Moreover, the confidence judgments were more accurate and less biased.

Barenberg and Dutke (2019) conclude that the confidence judgments can "stimulate the learners to reflect their understanding of learning topics and the quality of knowledge they acquired." What is more, that this reflexion can help them to identify learning topics that need further clarification and help them to develop the accuracy of their confidence judgments.

Judgments of Knowing, Learning, and Memory - Do I Know It. As most other metacognitive judgments, feelings of knowing have behavioral consequences and predict how much time and effort people will - or will not invest in a memory search (Dunlosky & Metcalfe, 2009; Koriat, 2007). Judgments of Learning, Will I Remember It? Are moderately predictive of people’s actual performance Dunlosky & Metcalfe, (2009). RememberMore takes the positive elements of each, trusting the learning with both the cue and the response.

An interesting area of study is combines response latency and confidence. Ackerman and Koriat (2011) report "When the test format was easy (Experiment 1), 2nd graders were as accurate as 5th graders in monitoring the accuracy of their answers, and the latency of their responses was no less predictive of accuracy. When the task was more difficult, age differences emerged. Nevertheless, in all experiments and for both age groups, response latency was found to have added value for predicting accuracy over and above that of confidence.

How reliable is teacher assessment?

"Over 40 years of empirical research shows that teachers' ability to estimate student learning is rather poor (Urhahne & Wijnia, 2021), Nevertheless, it can be enriched by the learners' own ability to estimate their learning gains."

Self assessment (and self regulation and performance forecasting)

As previously highlighted self-regulation approaches that encourage learners to think about their own learning have consistently high levels of impact with learners. The logic is that, like self-regulation, self-evaluation of the quality attributes of one’s own work draws on metacognitive competencies. It is also seen as a potential way for teachers to reduce their own assessment workload, making students more responsible for tracking their progress and feedback provision (Sadler & Good, 2006).

Self-assessment practices can be grouped into three major types: self-ratings, self-estimates of performance, and criteria or rubric based assessments. RememberMore uses confidence based assessments, a form of self-ratings. A number of studies have shown that students who engage in self-assessment experience positive gains in their learning. Most studies report positive effects of having students self-assess,the median effect lies between .40 and .45, a moderate effect consistent with values reported in Black and Wiliam (1998).

Promoting metacogitive practice (knowledge / skills) is an area of interest, particularly when connect with meso cycles (cycles within a scheme of learning, 8-12 lessons in English), assessment and feedback. More on this area soon once the terms data has been reviewed.


Interleaving simple mixes topics instead of focusing or blocking the same topic. This is a long-term strategy. It is more effective when those topics are related.

Rohrer and Taylor (2007) reported greater performance for those students who learned through interleaving (63% accuracy) compared to blocking (20% accuracy).

Both spacing and interleaving strategies lead to slower and more error-prone learning in the short-term however, they boost memory retention when implemented over a longer time span. But this is also one of the reasons students, in spite on being told their test scores, still report they prefer that massed, re-reading type strategies.

Using these two strategies RememberMore enhances inductive learning (making patterns and connections) and later memory retrieval. Spacing benefits memory recall whilst interleaving enhances inductive learning and later memory retrieval.

Butler et al., (2017) found that interleaving or viability applied to retrieval practice with "superior transfer of knowledge to new examples," suggesting that repeatedly retrieving and applying knowledge to different examples is a powerful method for acquiring knowledge that will transfer to a variety of new contexts.

Recent research shows that the mnemonic benefits of taking a test are not limited to the specific questions or facts that were tested; retrieval practice also improves transfer of knowledge to new contexts.


The concept of elaboration among cognitive psychologists is broad and can mean a lot of different things, however elaborative interrogation is essentially asking ‘how’ and ‘why’ questions, elaborating on the opening question. It is constructing meaning of the subject matter by explaining, elaborating, making inferences, connecting new facts and ideas to prior knowledge.

The purpose of elaboration is primarily to improve our memory for the new information. An additional benefit is that it begins to organize our knowledge in a more coherent way.

In one of the largest and most comprehensive meta analysis of undergraduate STEM education published, Freeman et al., (2014) reported an average examination scores improved by about 6% in "active learning sections," (elaboration) and failure rates reduced from 34% to 22% where elaboration was established practice.

Across several instructional conditions and settings, Nesbit and Adesope (2006) found the use of concept maps was associated with increased knowledge retention. With (Butler, Godbole, & Marsh, 2013) show further elaborating on the correct answer in the feedback can be important for later application of that knowledge. In fact, activation of elaborative information which would occur to a greater extent during retrieval practice over restudying — i is one mechanism that is used to explain the broader concept of and effectiveness of Successive Learning.

Before we go on, not all elaboration techniques are equal.

Elaborative information tends to increases performance only when it helped specify the relevance of "target information." In this situation, RememberMore tags / cues provide excellent opportunities for elaboration as does making connections between the cues, between prompts or any presented information within RememberMore, particularly with Classroom. RememberMore "Notes" add yet a further layer.

Equally important is giving learners the opportunity to explain the prompts themselves. Doing so encourages the learners to infer the missing information, to synthesize the presented information. Those learners encouraged to self-explain outperformed those who read and reread material (Griffin, Wiley, & Thiede, 2008).

The Potentiating Effect (pre-questions)

Familial with post-testing gains, most educators would probably assume that giving students a test on material before they had learned it, would have little impact on student learning beyond providing teachers with insight into their students knowledge base. You may be surprised to learn of potentiating impact of pretesting. Simply, that pretesting (and pre-questions) have the potential to increase the efficiency and utility of the future learning that follows.

Research suggests that tests can be valuable learning events, even if learners cannot answer test questions correctly. That prior tests enhance the encoding of not-yet-taught and not-yet retrieved items. Even when the rate of success obtained in the pre-test was very low (in Richland et al.’s study, participants got as many as 95% of the pre-test answers wrong), learning with a pre-test was better than just studying twice.

Directly comparing both opportunities, post-testing and pre-testing relative to an extended reading condition, on a retention test 7 days later, Latimier et al., (2019) reported both posttesting (d = 0.74) and pre-testing effects (d = 0.35), with significantly better retention in the former condition.

Wissman, Rawson, and Pyc (2011) reported that retrieval practice of one set of material may facilitate learning of later material, which may be related or unrelated.

Similarly Karpicke (2009) reported the benefits of active retrieval improves students’ encoding when they restudy material.

"Pretrieval" practice - asking questions before teaching has a beneficial effect for learning. Most often attributed to learners being "primed" or "selectively attending" to pre-tested material - during the taught phase.

"Studies have shown that prequestions - asking students questions before they learn something - benefit memory retention," (Carpenter, Rahman and Perkins, 2018).

Pre-questions are things you can ask your students about material that they have not yet learned. Research shows that students who had been asked pre-questions were later able to recall almost 50% more information than their peers who had not. They were also able to remember other key information from the lesson too. Carpenter, et al., (2016).

Cues or tags

The diagnostic value of retrieval cues — the degree to which cues help people recover particular target knowledge to the exclusion of competing candidates — is the critical factor for all learning.

Retrieval may enhance learning because it improves the match between a cue and particular desired knowledge, or it may enhance learning by constraining the size of the search set— the set of potentially recoverable candidates that comes to mind in the context of a cue (Karpicke & Blunt, 2011b; Karpicke & Zaromb, 2010).

Carpenter (2009) items recalled from weak cues were retained better over time, such that this advantage was eliminated or reversed at the time of the final test. Of note, is that diminishing cues conditions consistently outperformed the study-only condition and the accumulating cue condition, Finley et al. (2011). RememberMore is currently exploring how best to apply this empirical research.

Simply, the design of RememberMore promotes efficient prompt design with retrievals tags enabling shortened retrieval prompts, reducing error rates. Furthermore, tags then act as a knowledge "filter," learners now free-recall from a redacted knowledge base, increasing retrieval rates.

Pre-sights, feedback and rewards

Learning pre-sights provide the learner feed-forward about what they are about to review. Insights provide the learner with immediate feedback about the information they have just reviewed and deeper insights provide feedback and progress over time.

"Retrieval practice is often effective even without feedback (i.e. giving the correct answer), but feedback enhances the benefits of testing," Roediger and Butler, (2011).

Kornell and Son (2009) found that across experience with multiple study-test cycles, learners learned to self-test themselves more and did so at a faster rate when there was feedback on the tests at the end of each cycle.

Finally, rewards. Learners’ mnemonic performance is enhanced when incentives are present. RememberMore employs rewards, carefully scheduled and carefully constructed to complement each other and to deter ‘learner retirement.’ Also note that learners are more responsive to increased costs than increased benefits.

Social comparisons can also incentivize learners at recall with Fraundorf and Benjamin (2015) suggesting that learners are sensitive to how their experience with materials compares with others’.

Reducing test anxiety

Test anxiety is quite common. This nervousness can negatively affect student performance on those tests and can even impact their feelings about school and education. Retrieval practice had quite the positive effect on test anxiety experienced by middle and high school students. Data collected from 1408 students between the ages of 11-18 showed 72% of students reported retrieval practice made them less nervous for tests and exams, 81% of students reported they felt the same amount of test anxiety or less in the class that used retrieval practice relative to their other classes.

“This finding suggests that experiencing retrieval practice makes students less anxious regarding upcoming tests and exams for classes in which retrieval practice was implemented." Agarwal et al. (2014). Similarly Sullivan, (2017) found that percentage elevated to 90% for College students.

Furthermore, in another study, it was shown the retrieval practice was “had the most potential to create memories that were resilient to stress.,” Whereby retrieval practice appeared to create more distinct routes to the information to be retrieved, even if cortisol impairs one or more pathway, another may still be viable for retrieval.

Back to those re-reading strategies. Increased stress made re-reading strategies even more vulnerable. Studies showed that these learners experienced even great performance impairment. Up to 32% worse.

Students enjoy Retrieval Practice

In perhaps the most extensive study of this effect, Agarwal et al. (2014) surveyed 1408 middle and high school students who had experienced classroom-based retrieval practice programs (e.g., like the one implemented by McDaniel et al., 2011). The key findings from their survey were that 92% of students viewed the classroom retrieval practice activities positively, believing that retrieval practice helped them learn, and 72% said that frequent retrieval practice helped them feel less nervous about classroom exams.

"Students perceived retrieval practice as helpful for retaining and retrieving information over time, for monitoring their comprehension, and for cueing important information for subsequent restudy," Cogliano, et al., (2018).

This concurs with what learners tell us about learning with Classroom and to a greater degree RememberMore.

Text Expectancy

If "Successive Relearning" aids retention, expectation of any kind of test, enhances the processing of studied material. is an important motivator driving students to commit more effort to prepare for subsequent tests (Agarwal & Roediger, 2011; Szpunar, McDermott, & Roediger, 2007; Yang, Chew, Sun, & Shanks, 2019). Simple, test performance is higher when students expect a final cumulative test compared to when they do not, even when the students are not provided with an opportunity to restudy the material, Szpunar, McDermott, and Roediger (2007). Why? Two theories are presented: By reducing learners’ mind-wandering during studying and second, by reducing interference from previously studied information Weinstein et al., (2014).

Other researchers have similarly found increased performance for learners expecting either a multiple-choice or essay test (compared to students with no test expectation). The research also suggests that telling students the type of test they should expect (e.g., multiple-choice or essay) and the target learning goal (e.g., memory, comprehension, or application) can benefit their learning.

Expecting an upcoming test can be helpful to learners, actually gaining experience with the test leads, to the greatest adoption of optimal learning strategies, Colak, (2015).

Low stakes - 'the poker chip'

The more with teach with both and RememberMore, the more I am convinced that success drives motivation. That in turn, this motivation then drives success. The success-motivation-success cycle. What is more, I am convinced that the RememberMore approach protect learners (sometimes) fragile self-confidence. Here is how.

"If class was a poke game and you give each learner only one poker chip. Then each time you ask a question of the class you are asking learners to go all-in. Unsurprisingly, many learners choose to hold on to their chip, even those who are relatively confident, and look down. A few will risk, though only a few." -- My thanks to Chris Smith for this!

With low-stakes RememberMore, learners get both the prompt and the desired response, and they get to hold onto their chip. The knowledge become "sticky," they learn, and eventually remember, more. With a growing security, a deepening knowledge, their willingness to risk that chip become more and more likely.

The importance of prior knowledge for learning

It is not a question of whether or not prior knowledge is important for learning, it is a question of how important it is, and how important it is in facilitating the acquisition and consolidation of new learning. It is "all the knowledge one has before learning about a particular topic."

Here in the UK, setting aside data that describe the socio-economic circumstances of children, prior attainment at the end of primary school is found to be the most powerful available predictor of Secondary School attainment.

What about in classrooms? Prior knowledge has long been considered the most important factor influencing learning and learner achievement, with both "the amount and quality of prior knowledge positively influencing both knowledge acquisition and the capacity to apply higher-order cognitive problem-solving skills," Hailikari et al., (2008); Dochy et al., (1999); Chen, et al., (2018). With Dochy et al. (1999) go as estimating that between 30-60% of the variance in learning outcomes is explained by prior knowledge. Finally, given our focus on Successive Relearning, Van Kesteren, et al., (2018) demonstrated how prior knowledge structures are suggested to facilitate many stages of memory processing such as encoding, consolidation, and retrieval and reduce the interference of competing memories.

Cognitive research continues to uphold the importance of content knowledge in understanding academic material and the world around us. It is the foundation that enables us to make sense of what we learn. A seminal study by Recht and Leslie (1988) has shown that content knowledge is a better predictor of a learner's understanding of a text than reading ability; learners who are familiar with the relevant content of an article, understand it better than do their peers who are presumed better readers.

In the broader conversation of a "modern" education, we are often reminded that our learners future success, within an increasingly automated, artificially intelligent world, will demand more non-routine, creative thinking and refined reasoning skills than said declarative or substantive knowledge. However, this proscribed dichotomy is arguably misleading: with what do we make connections or challenge assumptions, if not with the thoughts, ideas and knowledge stored in our memory.

Value knowledge. Define what needs to be learnt, design the learning, and signpost exits routes from it. Without question, an essential factor in developing an integrated knowledge framework is aligning the educators expectations of learner prior knowledge, the learners actual knowledge base and what is to be learnt. Defining the parameters for substantive or declarative knowledge is important for both the educator and learner. Challenge your learners in your classes to acquire it, to use it and share it. And when they do, notice it and celebrate it. This is no easy task, though ignoring or undermining the importance of content knowledge in the hope that it will be assimilated is not a fair strategy either.

Vocabulary and simple definition acquisition is possibly one of the most obvious use cases for RememberMore (all the memorable when mixed with other content and knowledge). Why is vocabulary acquisition so important? Word knowledge is crucial to reading comprehension and determines how well students will be able to comprehend the texts they read. Now I accept that comprehension is far more than recognising words and remembering their meanings, however, if a student does not know the meanings of a sufficient proportion of the words in the text, comprehension is impossible.

Vocabulary experts agree that adequate reading comprehension depends on a person already knowing between 90 and 95 percent of the words in a text (Hirsch, 2003). Knowing at least 90 percent of the words enables the reader to get the main idea from the reading and guess correctly what many of the unfamiliar words mean, which will help them learn new words. Readers who do not recognise at least 90 percent of the words will not only have difficulty comprehending the text, but they will miss out on the opportunity to learn new words.

The importance of knowledge.

Knowledge (background knowledge) makes one a better reader in two ways. (There is a greater probability that you will have the knowledge to successfully make the necessary inferences to understand a text and second, a rich background knowledge means that you will rarely need to reread a text in an effort to consciously search for connections in the text.) Knowledge help you remember new information. A rich network of associations makes memory strong, with new material more likely to be remembered, as it is more likely to be related to what is already in memory. Converserly, remembering new information, on a brand new topic, is more difficult, it has nothing to connect with, get stuck to. (Remembering new information, on a familiar topic, is relatively easy if you practice developing associations between your existing network and the new material).

Knowledge for thinking

Knowledge enhances thinking in two ways in that . First, it frees up more working memory. Second, serves more known knowledge to work with. The area of education most fervently explores is problem solving, most often in mathematics and science, that is not to say knowledge for thinking is applicable to other subject areas - chess normally gets a mention here too. Burns (2004) positioned that the recognition process accounts for most of the differences among top blitz chess players, drawing from hundreds of thousands of stored sequences or moves.

RememberMore supports revision

We foresee RememberMore being adopted as a revision tool. How might RememberMore support effective revision practice in addition to adaptive, spaced retrieval? RememberMore:

is mobile, accessible, encourages learners to build retrieval / revision into their practice.

tags break down learning into manageable (and potentially, interleaved) chunks

enable revision as a moments notice, offer the opportunity for shorter, more frequent, revisions periods


Calibration has been defined as the degree of fit between a person’s judgement of performance and his or her actual performance (Keren, 1991). As such, calibration reflects a metacognitive monitoring process that provides information about the status of one’s knowledge or strategies at a cognitive level. Based on this information, control at a metacognitive level can be exerted to regulate one’s knowledge or strategies (though often this is under utlised by learners). Therefore, greater accuracy in a person’s judgments of performance, being well calibrated, creates greater potential for self-regulation.

Incentives to be accurate, significantly improved calibration accuracy only among lower-achieving students, Schraw et al. (1993).


Routines are at the heart of the integration of RememberMore and (CRM). We know first hand, the power of routines, securing overt written retrieval routines, moving to to covert routines that can present extensive Successive Relearning opportunities.

Inspired by Peps McCrea thread on routines, here is how RememberMore promotes a productive, learning environment.

RememberMore is exceeding fast to set up, less than 10s. There is minimal on screen distraction. Instruction can be independent, coral or draw on educator-learner interaction. High learner agency, self-assessment, presenting both retrieval prompt and response, high success rates, as well as the routines themselves, promotes confidence and safety. What Ben Windsor coined "The Deadpool Effect" of RememberMore, now on it's forth iteration:

The Deadpool Effect: Two parts psychological security (the correct answer is always provided) + one part student agency (self-marking and correction) + one part low stakes (retrieval not assessment).

Importantly, class learning is integrated with RememberMore, which also measures performance and tracks progress.

"One of my students is up to 3 hours. Now in class, where before I got shrugs, I know have a hand shooting up frequently and vigorously." @MrClassics

Having RememberMore routines almost instantly available, means educators have greater resource to allocate to teaching.

Importantly, learner 'attention' is shift from "how to," to retrieving or thinking about the activity presented. Learners working on the RememberMore app are focused the classroom is remarkable quiet. Successive Relearning via the app is personalised.

See our classroom routines page for a range of routines and we hope you will share you ideas with us.

Generating questions

One further area of professional interest, is instructing learners to generate questions based on the learning material which has been shown to yields medium to large effects on comprehension, recall, and problem solving (Song, 2016). Similar to retrieval, generating questions may stimulate a deeper processing and reflection of the learning material, as well as retrieval practice in comparison with restudying. However, in most of the reviewed studies, learners were trained on how to generate questions effectively and practiced this strategy in advance and under supervision. In addition, the learning material involved only short text passages, and only short-term effects were examined.

Of course, generating questions also give further opportunity for retrieval!

Vocabulary and disciplinary literacy

"Though cognitive-science research indicates that 95% of children can learn to read, rates of reading failure are typically around 30%," (Hempenstall, 2013) cited in Snow (2020). PISA reports 20% of 15 year olds read below "acceptable standards," (PISA 2015; Jerrim & Shure, 2016).

Oral language development is critically important for reading and writing which in turn underpins academic achievement. Oral language and reading as “inseparable friends,” who take turns to piggy-back on each other during the school years and beyond. We also know that reading and literacy are inextricable intertwined. That "Language is literacy is language," Snow (2016) and that influence is bidirectional.

Why should you be interested in vocabulary acquisition, accuracy, fluency and retention? Adding to the research of Lipowski et al., (2013). and Goosens et al. (2014a, 2014b, 2016) reading is how older learners acquire new knowledge, almost exclusively, new vocabulary.

Exposing learners to increasingly complex sentences and discourse, to background knowledge, is not enough in of itself to move a child, a learner, from being a ‘good talker’ to a ‘good reader.’ Moving a child, a learner, from being able to decode words, to understanding their meaning, requires teaching.

Learners rely on knowledge (that includes a rich and broad vocabulary) to attend to, and make sense of what they are reading. Vocabulary are the clues of "meaning making." We have to teach / instruct learners, to broaden their vocabulary needed for schooling and independent reading / learning. Vocabulary not present in oral communication.

Reading then, is required to access knowledge and it's importance increases as students get older.

Precisely how much vocabulary we need is unclear. A minimum estimate for minimally acceptable reading and listening comprehension is 95% recognition of words, which would need around 3,000 word families. Most researcher put the figure for unassisted reading and listening at 98% word recognition, which research has shown to require 6,000 word families for listening and 8,000 for reading, Hu and Nation (2000).

"We know that the explicit teaching of vocabulary matters. Disciplinary literacy matters. And it matters to every teacher and pupil. Indeed, success in each and every classroom depends upon it." Alex Quigley

Beck suggests we need to teach 7,000 Tier Two word families. Or 700 words per year. Beck et al., (1982) research calls for 400 words minimum, 400 words per year conforms to the rate at which improvements in word knowledge and in comprehension of text would be required "to be proficient." Introducing 400 is a clear maker. 700 is the aim. Nation (2006) shows that relationship quite clearly (see Table 14 and Figure 1). 95% coverage can be achieved by 5,000 word families. 98% coverage is achieved by an additional 2% provided by 6-9,000 words and an additional 1% of proper nouns.

We know from designing RememberMore class reader decks, most texts offer easily 80-120 words or word families to teach. That is our starting point, along with Dr Averil Coxhead's "Academic Word List" - 🚧 here 🚧.

16.07.21 Roots, Prefixes and Suffixes added. 99 cards.

20.07.21 We added "The Dirty Thirty" 30 Most Common Misspellings list to Classroom. We are working on a disciplinary list for Secondary Education.

How to present that vocabulary is currently being investigated:

  • First iteration: vocab: definition - (notes - use or textual reference) and tagged.

  • Second iteration: vocab or phonetic vocab: definition (notes - use or textual reference) and tagged.

  • Current iteration: vocab (word class) or phonetic vocab: definition (notes - use or textual reference) and tagged.

Knowledge transfer

Transfer of learning, or transfer, is the application of learned concepts or information to new or novel situations. Now, to remind you, Retrieval does more than measure memory, it modifies memory in a way that makes information retrieved on a test more likely to be remembered in the future than it would have been otherwise.

Butler et al., (2017) found that repeatedly retrieving and applying knowledge to different examples is a powerful method for acquiring knowledge that will transfer to a variety of new contexts.

There is evidence to suggest that retrieval practice can foster transfer across a variety of contexts and that it is most effective when coupled with feedback or elaboration. Of course, learners need to have been successful initially, need to recognise that their knowledge applies to this new or novel context.


The reporting of boundary effects for the testing effect have been highlighted in light a EEF f

Bertilsson et al., (2020) recently focused on had reported on how individual differences in personality traits and working memory capacity moderate the size of the retrieval-practice benefits. Learning was assessed at three time points: five minutes, one week, and four weeks after practice. The results revealed a significant testing effect at all three time points. That is to be expected. Further, the results showed no association between the testing effect and the personality traits, or between the testing effect and working memory, at any time point.

  • The type of recall, free recall, cued recall, recognition recall (MQCs), confidence levels.

  • The type of information, most regularly paired associates, by that is not the only knowledge investigated. Also declarative vs procedural.

  • How easy the knowledge is to learn or perceived to be (ELER - easy to learn, easy to remember) vs desirable difficulties.

  • Age of learner, curriculum area.

  • Covert or Overt. Direct vs self-regulated.

Cognitive Neuroscience

For the most part, my interest in Retrieval Practice and Successive Relearning has been from a Cognitive Science and behavioural perspective that has long emphasised the role of retrieval and in memory updating, however more recently the neural mechanisms and basis behind these process or processes are currently being explored by Neuroscience researchers.