PhD Dissertation Research
"Exploring the Effects of AI-Generated Pedagogical Agents in Instructional Videos on Learning"
Research Background
Recent advancements in generative artificial intelligence (GenAI) have made it easier to create hyper-realistic, human-like multimedia content, including audio, text, images, and video. While this technology has sparked significant concerns regarding its misuse, it also holds tremendous potential to transform traditional teaching and learning methods. However, the cognitive and educational impacts of such AI-generated content remain largely unexplored. Thus, it is crucial to urgently investigate the untapped potential of GenAI technology, especially in educational contexts.
*A sample footage of an AI-generated pedagogical agent in an instructional video.
Research Motivation
Drawing inspiration from the researcher’s previous work with augmented reality (AR) in educational settings—where human-like virtual characters were used to assist autistic individuals in practicing social interactions—this study aims to investigate the potential of AI-generated pedagogical agents in educational contexts.
Purpose
This dissertation study investigated the effects of AI-generated pedagogical agents (PAs) within instructional videos on learning outcomes, cognitive load, and attention.
Research Design
This dissertation study investigated the effects of AI-generated pedagogical agents (PAs) within instructional videos on learning outcomes, cognitive load, and attention. A 2x2 mixed factorial design was employed, examining the type of PA (AI-generated vs. human) as a between-subjects factor and instructional video order (learning content) as a within-subjects factor. A total of 58 adult participants were assigned to one of four conditions, viewing two 3-minute instructional videos on distinct topics. Post-video assessments evaluated retention, transfer, and cognitive load, while eye-tracking data measured attention allocation.
Theoretical Framwork
-
Social Agency Theory (Mayer et al., 2003)
-
The interaction between humans and digitally presented learning materials is primarily social.
-
Social cues (e.g., human-like gestures, eye gaze, and voice) can prime a social stance in the learner, which leads to deeper cognitive processing, and results in a more meaningful learning outcome.
-
-
Principles of Multimedia Learning (Mayer, 2001)
-
​Voice Principle: individuals learn more effectively from human voices than from machine voices in multimedia learning.
-
Embodiment Principle: human-like body gestures, movement, eye contact, and facial expressions of pedagogical agents promote deeper learning.​
-
Pilot Study
A 2x2 between-subject design pilot study was conducted to investigate the effects of the type of PA (AI-generated, real-life human), and voice (AI-generated, human voice) on an individual's learning outcomes, cognitive load, motivation, and attention. Results from this study provided some preliminary evidence that PA appearance influences learners’ retention and cognitive load, but not attention. The type of PA influenced learners' perception of the agent's ability to facilitate learning, its human-like qualities, and its engagement level. However, it did not affect its credibility.
Findings
Contrary to pilot study findings, the dissertation results revealed no significant main effects of PA type on learning outcomes or cognitive load, challenging Social Agency Theory’s premise that human agents inherently enhance learning through social cues. Notable order effects were identified for retention, underscoring the influence of video sequence on learning outcomes. These findings suggest that AI-generated PAs neither significantly impede learning nor surpass the effectiveness of human PAs. Additionally, they reflect advancements in technology between the pilot and full dissertation studies, conducted one year apart.
Implications
This research contributes to the growing discourse on AI’s role in education, offering valuable insights into the design of AI-driven instructional tools. By uncovering nuanced interactions between instructional agents, content type, and learning processes, the study highlights the potential and limitations of generative AI in educational applications. The findings emphasize the importance of optimizing content delivery sequences and leveraging technological advancements for effective educational design.
​
This ongoing work will contribute to the growing understanding of the impact of AI in education, provide evidence of the efficacy of AI-generated PAs in instructional videos for learning, and narrow the gap between human-computer interaction research and education.
Future Directions
The Potential to Bridge Findings with ASD Research
There is emerging evidence that supports the use of an avatar with live animation to intervene with students with autism spectrum disorder (ASD; Charlton et al., 2020). Similarly, research on AI-generated pedagogical agents could extend this research, and inform pedagogical design practices in educational settings. Previous research has found that students with ASD are engaged by animated avatars and benefit from seeing visual models of target skills when learning social skills (Bellini & Akullian, 2007). In another study, the use of a computer-animated avatar elicited higher motivation for students with ASD to learn, with fewer disruptive behaviors, compared to more traditional personal instruction (Bosseler & Massaro, 2003). Yoshida et al. (2022) have also found an intervention with a humanoid avatar may change and improve the motivation to communicate via speech. Perhaps, this suggests we could expand the research of AI-generated PAs and connect it to assisting individuals with special needs in educational settings in a similar way. Therefore, the research is worth doing. Findings will serve as a stepping stone in understanding the potential of using AI-generated pedagogical agents to assist diverse learner needs, such as learners with special needs like ASD, and contribute to the ongoing further development of generative AI technology and its applications in educational settings.
Contact Me
Interested to learn more about my work? Contact me here!