Japanese Language Learning in VR
Japanese Language Learning in VR
Japanese Language Learning in VR
Process
Competitive Analysis
Interviews
Concept Testing
Body-storming
Final Design Documentation
Final Presentation to Stakeholders
Competitive Analysis
Interviews
Concept Testing
Body-storming
Final Design Documentation
Final Presentation to Stakeholders
My Role
VR "specialist"
Interviewer
Researcher
UI Designer
VR "specialist"
Interviewer
Researcher
UI Designer
Timeline
4 Months (Jan - April 2023)
4 Months (Jan - April 2023)
Tools Used
Miro
Meta Quest 2 VR Headsets
Google Docs
Paper Prototyping
Miro
Meta Quest 2 VR Headsets
Google Docs
Paper Prototyping
Project Goal
Project Goal
Picture this: you are exploring a new language and culture with people of that culture in a playful, supportive environment. With VR, this can be achieved from anywhere. This was our goal when a Japanese Professor approached our UX department with the idea of creating a VR experience to help teach his Japanese 101 students. This project was very open-ended with the only requirements being that it needed to be a VR experience and educational.
Picture this: you are exploring a new language and culture with people of that culture in a playful, supportive environment. With VR, this can be achieved from anywhere. This was our goal when a Japanese Professor approached our UX department with the idea of creating a VR experience to help teach his Japanese 101 students. This project was very open-ended with the only requirements being that it needed to be a VR experience and educational.
Picture this: you are exploring a new language and culture with people of that culture in a playful, supportive environment. With VR, this can be achieved from anywhere. This was our goal when a Japanese Professor approached our UX department with the idea of creating a VR experience to help teach his Japanese 101 students. This project was very open-ended with the only requirements being that it needed to be a VR experience and educational.
Understanding the Problem Space
Understanding the Problem Space
Understanding the Problem Space
Before diving into ideation, we first needed to learn about the different aspects of our problem space- our users, the platform, and the language learning pedagogy.
Before diving into ideation, we first needed to learn about the different aspects of our problem space- our users, the platform, and the language learning pedagogy.
Before diving into ideation, we first needed to learn about the different aspects of our problem space- our users, the platform, and the language learning pedagogy.
Users
Users
Our primary users are first-year Japanese students (a Japanese 101 course) at Purdue University. To learn how to best design for these users, we had these research goals:
Identify pain points for students learning Japanese characters as a second language.
Understand how students are retaining information related to learning Japanese as a second language.
Understand our users motivations for learning and practicing Japanese.
Our primary users are first-year Japanese students (a Japanese 101 course) at Purdue University. To learn how to best design for these users, we had these research goals:
Identify pain points for students learning Japanese characters as a second language.
Understand how students are retaining skills and information related to learning Japanese as a second language.
Understand our users motivations for learning and practicing Japanese.
Our primary users are first-year Japanese students (a Japanese 101 course) at Purdue University. To learn how to best design for these users, we had these research goals:
Identify pain points for students learning Japanese characters as a second language.
Understand how students are retaining skills and information related to learning Japanese as a second language.
Understand our users motivations for learning and practicing Japanese.
Virtual Reality
Virtual Reality
Among my team, I was the one with the most experience with VR, so I helped my team experience VR for the first time. We played some popular games, such as Beat Saber and Superhot. This activity helped the team understand the embodiment that VR affords. We also discussed different VR aspects to consider such as grabbing/placement of objects, movements that can cause motion sickness, etc.
Among my team, I was the one with the most experience with VR, so I helped my team experience VR for the first time. We played some popular games, such as Beat Saber and Superhot. This activity helped the team understand the embodiment that VR affords. We also discussed different VR aspects to consider such as grabbing/placement of objects, movements that can cause motion sickness, etc.
Among my team, I was the one with the most experience with VR, so I helped my team experience VR for the first time. We played some popular games, such as Beat Saber and Superhot. This activity helped the team understand the embodiment that VR affords. We also discussed different VR aspects to consider such as grabbing/placement of objects, movements that can cause motion sickness, etc.
Japanese Pedagogy
Japanese Pedagogy
Lastly, we wanted to incorporate current methods of teaching Japanese to new students, including writing, sound/shape association with Hirigana and Katakana, and vocabulary. Only 1 of our team members had past experience with the Japanese language, so our sponsor provided resources that he uses for his students to us so we could learn these vital aspects of the language.
Lastly, we wanted to incorporate current methods of teaching Japanese to new students, including writing, sound/shape association with Hirigana and Katakana, and vocabulary. Only 1 of our team members had past experience with the Japanese language, so our sponsor provided resources that he uses for his students to us so we could learn these vital aspects of the language.
Lastly, we wanted to incorporate current methods of teaching Japanese to new students, including writing, sound/shape association with Hirigana and Katakana, and vocabulary. Only 1 of our team members had past experience with the Japanese language, so our sponsor provided resources that he uses for his students to us so we could learn these vital aspects of the language.
Finding Opportunities for a Better Experience
Finding Opportunities for a Better Experience
Finding Opportunities for a Better Experience
To learn more about our users painpoints, we interviewed 5 Japanese 101 students, asking them about their past experience with Japanese (if they had any), what they struggle most with, and what methods do they currently use to help their learning.
To learn more about our users painpoints, we interviewed 5 Japanese 101 students, asking them about their past experience with Japanese (if they had any), what they struggle most with, and what methods do they currently use to help their learning.
To learn more about our users painpoints, we interviewed 5 Japanese 101 students, asking them about their past experience with Japanese (if they had any), what they struggle most with, and what methods do they currently use to help their learning.
User Painpoints
User Painpoints
User Painpoints
The students often struggle learning the different characters used in Japanese, as it is a new "alphabet."
They also struggle to recall different characters when reading/writing.
The students often struggle learning the characters used in Japanese, as it is a new "alphabet."
They also struggle to recall different characters when reading/writing.
The students often struggle learning the characters used in Kanji, as it is a new "alphabet."
They also struggle to recall different characters when reading/writing.
Their Solutions
Their Solutions
Many interviewees stated they used apps like Duolingo to help them practice.
Another method they use is to listen to various forms of Japanese entertainment.
Many interviewees stated they used apps like Duolingo to help them practice.
Another method they use is to listen to various forms of Japanese entertainment.
Many interviewees stated they used apps like Duolingo to help them practice.
Another method they use is to listen to various forms of Japanese entertainment.
Sketching and Ideation
Sketching and Ideation
Sketching and Ideation
After exporing existing games, we had several sketching/idea discussion sessions. We explored some ideas such as an "escape room" style game, a scavenger hunt, and a painting game.
After exporing existing games, we had several sketching/idea discussion sessions. We explored some ideas such as an "escape room" style game, a scavenger hunt, and a painting game.
After exporing existing games, we had several sketching/idea discussion sessions. We explored some ideas such as an "escape room" style game, a scavenger hunt, and a painting game.
But after seeing the different interactions available in VR, we thought to make a "Sushi Tycoon" game where the user is an owner of a sushi restaurant, and they learn Japanese through assembling different sushi.
But after seeing the different interactions available in VR, we thought to make a "Sushi Tycoon" game where the user is an owner of a sushi restaurant, and they learn Japanese through assembling different sushi.
But after seeing the different interactions available in VR, we thought to make a "Sushi Tycoon" game where the user is an owner of a sushi restaurant, and they learn Japanese through assembling different sushi.
User interacts with item for word recognition
User interacts with item for word recognition
User interacts with item for word recognition
We explored various aspects of the sushi tycoon game, such as the different interactions, the onboarding process, and how to retain users.
One aspect we wished to incorporate was writing, so we thought of adding "sauce" to the sushi as a certain character that corresponds to the order, shown on the left. This would help with the user's pain-point of recalling characters.
We explored various aspects of the sushi tycoon game, such as the different interactions, the onboarding process, and how to retain users.
One aspect we wished to incorporate was writing, so we thought of adding "sauce" to the sushi as a certain character that corresponds to the order, shown on the left. This would help with the user's pain-point of recalling characters.
We explored various aspects of the sushi tycoon game, such as the different interactions, the onboarding process, and how to retain users.
One aspect we wished to incorporate was writing, so we thought of adding "sauce" to the sushi as a certain character that corresponds to the order, shown on the left. This would help with the user's pain-point of recalling characters.
Following the Sushi Resurant theme, I sketched out in Procreate what the users's navigation menu would look like. I thought to make the menu like a Recipe book, with the different pages having the player's progress and game options.
Following the Sushi Resurant theme, I sketched out in Procreate what the users's navigation menu would look like. I thought to make the menu like a Recipe book, with the different pages having the player's progress and game options.
Following the Sushi Resurant theme, I sketched out in Procreate what the users's navigation menu would look like. I thought to make the menu like a Recipe book, with the different pages having the player's progress and game options.
I also made a simple menu mockup where the user can upgrade parts of the restaurant, including the wallpaper, decor, and dishes using the in-game currency the user wins from playing. Having rewards like this helps to keep the player motivated and adds fun customization options.
I also made a simple menu mockup where the user can upgrade parts of the restaurant, including the wallpaper, decor, and dishes using the in-game currency the user wins from playing. Having rewards like this helps to keep the player motivated and adds fun customization options.
I also made a simple menu mockup where the user can upgrade parts of the restaurant, including the wallpaper, decor, and dishes using the in-game currency the user wins from playing. Having rewards like this helps to keep the player motivated and adds fun customization options.
Putting it All Together
During this project, there was one hurdle that my team and I had: we could not actually prototype this VR game digitally. At the time, none of us had any experience with a VR program such as Unity, so we had to find another way to demonstrate our ideas. And then we had a lightbulb moment:
During this project, there was one hurdle that my team and I had: we could not actually prototype this VR game digitally. At the time, none of us had any experience with a VR program such as Unity, so we had to find another way to demonstrate our ideas. And we had a big lightbulb moment:
During this project, there was one hurdle that my team and I had: we could not actually prototype this VR game digitally. At the time, none of us had any experience with a VR program such as Unity, so we had to find another way to demonstrate our ideas. And we had a big lightbulb moment:
If VR is meant to replicate real-life interactions, then why don't we use real-life to replicate VR?
If VR is meant to replicate real-life interactions, then why don't we use real-life to replicate VR?
If VR is meant to replicate real-life interactions, then why don't we use real-life to replicate VR?
This is what led us to our primary method to bring our ideas to life: body-storming. We created paper prototypes of the different sushi restaurant materials and had several teammates "act out" a gameplay session. By doing this, it showed us what sort of interactions we could include and how the flow of the game would go.
This is what led us to our primary method to bring our ideas to life: body-storming. We created paper prototypes of the different sushi restaurant materials and had several teammates "act out" a gameplay session. By doing this, it showed us what sort of interactions we could include and how the flow of the game would go.
This is what led us to our primary method to bring our ideas to life: body-storming. We created paper prototypes of the different sushi restaurant materials and had several teammates "act out" a gameplay session. By doing this, it showed us what sort of interactions we could include and how the flow of the game would go.
Final Deliverable: Demo Video
Final Deliverable: Demo Video
After doing the body-storming, we created and presented to the sponsors a video demonstrating what the VR game's onboarding would look like. We wrote a script and filmed the video in a first-person POV to simulate the VR experience. We even got our sponsors in parts the video!
After doing the body-storming, we created and presented to the sponsors a video demonstrating what the VR game's onboarding would look like. We wrote a script and filmed the video in a first-person POV to simulate the VR experience. We even got our sponsors in parts the video!
After doing the body-storming, we created and presented to the sponsors a video demonstrating what the VR game's onboarding would look like. We wrote a script and filmed the video in a first-person POV to simulate the VR experience. We even got our sponsors in parts the video!
How the Game is Played
How the Game is Played
The player is given a sushi order by a non-player character (NPC) by speaking a certain phrase, and then the user has to find the ingredients that have corresponding characters to assemble sushi and recreate the phrase.
The player is given a sushi order by a non-player character (NPC) by speaking a certain phrase, and then the user has to find the ingredients that have corresponding characters to assemble sushi and recreate the phrase.
The player is given a sushi order by a non-player character (NPC) by speaking a certain phrase, and then the user has to find the ingredients that have corresponding characters to assemble sushi and recreate the phrase.
To help further explain the game's onboarding process, as well as the overall gameplay, a couple of teammates and I created the following User Flow Diagrams using Miro. By doing this, we discovered a few aspects of the gameplay that we had not considered initially, such as the onboarding for beginner Japanese students vs more experienced students, how to keep players motivated, and the use of pacing for learners.
To help further explain the game's onboarding process, as well as the overall gameplay, a couple of teammates and I created the following User Flow Diagrams using Miro. By doing this, we discovered a few aspects of the gameplay that we had not considered initially, such as the onboarding for beginner Japanese students vs more experienced students, how to keep players motivated, and the use of pacing for learners.
To help further explain the game's onboarding process, as well as the overall gameplay, a couple of teammates and I created the following User Flow Diagrams using Miro. By doing this, we discovered a few aspects of the gameplay that we had not considered initially, such as the onboarding for beginner Japanese students vs more experienced students, how to keep players motivated, and the use of pacing for learners.
Evaluation: Usablility Testing
Evaluation: Usablility Testing
To test our VR experience prototype, we recruited 5 fellow UX students to "play" our game, like we did with our body-storming. Ideally we would have recruited some Japanese 101 students, but due to time constrains, we were unable. With this testing, we wanted to make sure our game flow and instructions were understandable and observe any pain points and areas with room for improvement in our current iteration.
From our usability testing, we gained the following insights and implemented them into our final design:
To test our VR experience prototype, we recruited 5 fellow UX students to "play" our game, like we did with our body-storming. Ideally we would have recruited some Japanese 101 students, but due to time constrains, we were unable. With this testing, we wanted to make sure our game flow and instructions were understandable and observe any pain points and areas with room for improvement in our current iteration.
From our usability testing, we gained the following insights and implemented them into our final design:
To test our VR experience prototype, we recruited 5 fellow UX students to "play" our game, like we did with our body-storming. Ideally we would have recruited some Japanese 101 students, but due to time constrains, we were unable. With this testing, we wanted to make sure our game flow and instructions were understandable and observe any pain points and areas with room for improvement in our current iteration.
From our usability testing, we gained the following insights and implemented them into our final design:
Users wanted immediate feedback rather than waiting until the end of the "order."
Users wanted immediate feedback rather than waiting until the end of the "order."
Users wanted immediate feedback rather than waiting until the end of the "order."
A hint system should be integrated for those struggling during the game.
A hint system should be integrated for those struggling during the game.
A hint system should be integrated for those struggling during the game.
Including context-based vocabulary or phrases could help with real-life application of learning material.
Including context-based vocabulary or phrases could help with real-life application of learning material.
Including context-based vocabulary or phrases could help with real-life application of learning material.
Lessons Learned
Lessons Learned
One of the biggest challenges we faced was trying to make a game without the use of any game development software/tools. We used body-storming to demonstrate our final experience and created a transition document for developers who could use our designs and insights to bring our ideas to life. It was very fulfilling to be able to use my past VR experience to help create this new experience for the Purdue Japanese professor and his students.
This was also my first time working in a larger team of UX designers. As a Master's student and my team largely being undergraduates, I approached this team with a supportive and receptive mentality; I was confident that they had the skills to successfully complete this project, so I did not attempt micromanage them and instead used the opportunity to learn their methods. Occasionally, during sponsor meetings, I would give our more "quiet" members encouragement by starting the conversation, then looping them in with updates about our progress. By the end of the project, I noticed how much more confident they were, especially when presenting the final deliverables. Overall, being able to collaborate and learn from them was a wonderful opportunity.
One of the biggest challenges we faced was trying to make a game without the use of any game development software/tools. We used body-storming to demonstrate our final experience and created a transition document for developers who could use our designs and insights to bring our ideas to life. It was very fulfilling to be able to use my past VR experience to help create this new experience for the Purdue Japanese professor and his students.
This was also my first time working in a larger team of UX designers. As a Master's student and my team largely being undergraduates, I approached this team with a supportive and receptive mentality; I was confident that they had the skills to successfully complete this project, so I did not attempt micromanage them and instead used the opportunity to learn their methods. Occasionally, during sponsor meetings, I would give our more "quiet" members encouragement by starting the conversation, then looping them in with updates about our progress. By the end of the project, I noticed how much more confident they were, especially when presenting the final deliverables. Overall, being able to collaborate and learn from them was a wonderful opportunity.
One of the biggest challenges we faced was trying to make a game without the use of any game development software/tools. We used body-storming to demonstrate our final experience and created a transition document for developers who could use our designs and insights to bring our ideas to life. It was very fulfilling to be able to use my past VR experience to help create this new experience for the Purdue Japanese professor and his students.
This was also my first time working in a larger team of UX designers. As a Master's student and my team largely being undergraduates, I approached this team with a supportive and receptive mentality; I was confident that they had the skills to successfully complete this project, so I did not attempt micromanage them and instead used the opportunity to learn their methods. Occasionally, during sponsor meetings, I would give our more "quiet" members encouragement by starting the conversation, then looping them in with updates about our progress. By the end of the project, I noticed how much more confident they were, especially when presenting the final deliverables. Overall, being able to collaborate and learn from them was a wonderful opportunity.
Want more details of our process?
Want more details of our process?
You can view the full documentation here:
You can view the full documentation here:
You can view the full documentation here:
