Dual Coding and Cognitive Load
Cover image by Chen from Pixabay
By Megan Sumeracki
In today’s blog, I’m covering an older paper by Mousavi, Low, and Sweller from 1995 (1) titled Reducing Cognitive Load by Mixing Auditory and Visual Presentation Modes. The goal of their paper was to experimentally test whether heavy cognitive overload could be reduced for students by presenting geometry worked examples using dual coding (in this case, visual and auditory formats). Our cognitive resources are limited, but we have the ability to process information in multiple formats simultaneously. The idea is that using multiple modalities, the amount that can be processed at once can functionally increase, thereby increasing learning.
For example, take the diagram below (from the original published paper). According to the researchers, there is a visual diagram and text that needs to be integrated before the learner can understand the worked problem, and engaging in this integration requires cognitive resources. If the presentation of the problem relieved the learner of that integration process, then those additional resources would be available for other types of processing that could lead to more learning.
Image from paper by Mousavi, Low, & Sweller, 1995 (1). The image shows a worked problem with problem statements (a given statement and a prove statement), a diagram of the geometry involved, and four proof statements.
The Experiments
The researchers conducted six experiments. I will focus on the details of the first experiment here, and I will note the overall findings from the paper in the conclusion.
Experiment 1
Students from Year 8 in Wollongong, Australia (8th grade equivalent in the US) participated in the experiment. The students were randomly assigned to one of three conditions in which the modalities of the geometry worked examples (like the figure shown above) was manipulated:
Simultaneous presentation: In this condition, students saw all components of the worked problem visually (like the figure shown above), and at the same time, they listened to a recording that read the statements in the worked problem out loud. For example, students in this condition could see the first proof statement “AO = CO (radii of the circle)” and they could hear a person read “AO = CO (radii of the circle)” played in the recording.
Visual-visual presentation: In this condition, students saw the worked problem visually only (i.e., there was no recording).
Visual-auditory presentation: In this condition, students saw the diagram portion of the worked problem, but not the statements. Instead, the statements were only presented in the recording, and students listened to the statements while viewing the diagram.
The experimental procedure had three phases:
Phase 1: Introduction
The students first studied a written explanation to make sure the students were familiar with basic geometry principles of congruent triangles.
Phase 2: Learning
In this learning phase, students saw four problems. Two were presented as worked examples (like the figure shown above), and two were very similar and required that the students solve them. Students learned one worked example based on the condition they were assigned, either seeing and hearing the statements (simultaneous group), seeing the statements only (visual-visual presentation), or hearing the statements only (visual-auditory presentation). Those that experienced the recording of the statements listened to the recording twice. Those in the visual-visual presentation group were allowed to study the problem for up to 5 minutes.
Then, a similar problem was presented and the students were asked to solve it themselves. If they made a mistake, then they were told an error was detected and they were instructed to correct it before proceeding. They were allowed to keep attempting to solve the problem until they got the correct solution, or until 5 minutes passed (at which time the correct solution was shown).
This was then repeated with a second worked problem and similar solution problem.
Phase 3: Testing
In this testing phase, the students completed problems that were similar to the initial problems they learned, and problems that were different but required the use of the same geometry concepts (i.e., transfer problems, see this blog and this blog for more on transfer). The students were able to keep attempting to solve the problems to get a solution, and the researchers measured how much time the students took to get to the correct solutions (with maximum time being 5 minutes before moving on to the next problem).
Experiment 1 Results
On the test, the students in the visual-auditory group spent less time solving the problems than the other two groups (visual-visual and simultaneous). These results supported the idea that presenting material in multiple modalities reduced demand on cognitive load, and increased learning and performance later. The results were statistically significant for the similar test problems in Experiment 1 (not transfer problems, but significant effects were found in other experiments in the paper for the transfer problems).
Conclusion
In the first experiment described above, and the other experiments presented in the paper, the researchers found that mixing the modalities of presentation led to better test performance later. This was consistently true for the similar types of problems. For the transfer problems, across experiments sometimes the results were statistically significant, while other times the difference was in the correct direction but not statistically significant. Across experiments, the overall pattern of results suggests that mixed modalities did lead to greater learning and better performance, even on the transfer problems.
Overall, the researchers note that if students have to split their attention across multiple sources of information, then integrating that information is demanding and requires resources. Reliving the students of needing to do this integration work frees up those resources to be used in other ways. Mixed modalities can improve student learning.
Interestingly, the simultaneous presentation and visual-visual presentation conditions do seem to leverage dual coding, but with a visual and verbal (i.e., written word) mix of modalities. If we think of dual coding this way, a single code purely verbal/written condition was not included. Performance might have been even worse if there were single-code conditions, where students learned by reading written statements without any auditory presentation and without a diagram. Still, these experiments highlight that just adding images, even a meaningful diagram, may not be optimal. Our cognitive resources can still be overloaded, and easing these cognitive demands in some way should help. If you’re interested in learning more about ways to reduce cognitive load when using multimedia learning and/or dual coding, you can read this blog (also read in this podcast episode).
References
(1) Mousavi, S. Y., Low, R., & Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology, 87(2), 319-334. https://doi.org/10.1037/0022-0663.87.2.319
