UCSB Research Shows IMMERSE AI Avatar Coaches Outperform Live Traditional Tutors

‍

Ye Yuan, PhD. Researcher at University of California, Santa Barbara

https://www.linkedin.com/in/ye-yuan999/

Partner Background

The Spanish department at the University of California, Santa Barbara (UCSB) sought to increase students’ opportunities to speak Spanish outside the classroom. In collaboration with Lingua Meeting, the program offered students online conversation sessions with native Spanish-speaking coaches. These sessions were scheduled for three students with one coach. The goal of integrating IMMERSE AI practice was to promote speaking through providing one-on-one, low-pressure, and self-paced supplementary practices.

The Challenge

Students reported having limited opportunities to actively speak during the small-group conversation sessions with Lingua Meeting. Many also expressed feeling anxious when interacting with the Spanish-speaking coaches and noted a persistent fear of making mistakes. Additionally, some students found it inconvenient to schedule sessions with coaches located in different time zones, which sometimes made it difficult to find suitable meeting times.

The Solution 

IMMERSE AI practice provided UCSB students with a low-stress, judgment-free environment for speaking practice. The one-on-one format maximized students' opportunity to speak. Additionally, the platform offered on-demand access, allowing students to log in and practice at their convenience. Powered by GPT4, IMMERSE AI supported open-ended conversation, enabling students to revisit and practice the same topic multiple times for deeper engagement and reinforcement.

Results and Impact

A comparative study included 46 students in Spanish 3, divided into an experimental group that practiced with IMMERSE AI chatbots and a control group that practiced with Lingua Meeting coaches. The findings highlighted lower anxiety and comparable engagement of the experimental group.

‍

• Lower during-practice anxiety: Used just 30 minutes per week for 6 weeks, IMMERSE AI practice reduced learners’ second-language speaking anxiety by 31%, compared with 22% among the group working with Lingua Meeting. Students also experienced 54% lower anxiety when talking with IMMERSE AI chatbots than with the human coaches.

• ‍Increased willingness to communicate: Students' willingness to communicate increased by 17% after practicing with IMMERSE AI chatbots for 6 weeks, the same as with Lingua Meeting. This demonstrates IMMERSE AI chatbots were just as effective as the live tutors at increasing willingness to communicate, while providing far greater flexibility in accessing them.

• ‍Positive in future adoption: 56% of students reported that IMMERSE AI practice was as useful as, or even more useful than, human-led practice and would consider continuing to use IMMERSE for Spanish speaking practice.

Lessons Learned and Takeaways

Detailed interaction analysis showed that students produced more task-related words when talking with AI chatbots than with human coaches. The finding indicates that IMMERSE AI practice has the potential to facilitate concise and focused communication, leading to more efficient conversational practice. The study advocates for supplementing human-led practice with AI-driven practice to further promote language production and engagement.

Student Perspectives

Comments from students show they appreciated the flexibility and effectiveness of working with the IMMERSE AI practice avatars and found them easier to talk to than traditional live tutors:

‍

• “I liked that it was easier to schedule. You didn't have to set aside 30 minutes of your day in advance. It could be more like whenever you have the time.”

• “I would like to have weekly AI practice to solidify my vocabulary and grammar knowledge.”

• “I often felt that in IMMERSE I was able to, I guess, control the conversation more compared to in Lingua Meeting. And I felt like that was one of the reasons that I felt less anxious.”

• “I liked it. I thought it was pretty stress-free and easy to use.”

• “I felt like there was a lot less pressure talking to an AI in comparison to a human instructor.”

• “(What) I really liked about it was that it was on our own time. So it took away the stress of having another person being able to actively judge you and still getting the same feedback, but you're also able to do it on your own time, which was really helpful to me because I did it like mainly in the middle of the night.”

IMMERSE

Interested in combining on-demand AI avatar coaching with immersive, small-group practice sessions with highly qualified language trainers? See how IMMERSE can elevate your language learning experience.