PROJECT

Pocket Pathologist

Designing a mobile application that guides users in pronouncing words in a new language.

More Information
Outcome

a mobile application that can perform analysis on the user’s recording input to identify the errors in their vowel’s pronunciation and provide guidance on how to improve their vowel’s pronunciation.

Timeline

September 2017 - April 2018

Tools

Figma, Adobe Illustrator, Adobe Photoshop, Android Studio

role

Lead Product Designer

Team

Daniel Charpentier, Roozbeh Moayyedian, Omid Ettehadi, Dr. Hamid Timorabadi

the Problem

According to the 2016 Census, only 58% of Canadian residents have English as their mother tongue. As globalization and immigration increase yearly, the ethnocultural diversity of the global population increases. Particularly, Canada’s population diversity is projected to increase from 20% of the population being foreign-born in 2006 to around 28% by 2031. With this, the importance of learning new languages also increases. One approach to learning new languages that have been popular in the past is the “Communicative Approach”. This refers to methods primarily focusing on the function of language as a means of communication and not structure. In this approach, you mainly focus on the pronunciation to increase the person’s ability to communicate, rather than grammar to make them speak correctly.

Currently, there are several existing applications that offer a combination of text, pictures and audio to help the user learn a new language, especially English. They associate visuals of the text and its audio pronunciation to help reinforce the new word. In these applications, pronunciation support is limited to an audio recording of the word being played for the user, giving them the chance to repeat the word and improve their pronunciation based on repetition. This leaves the user with the task of self-evaluation. It is difficult to develop a general system to provide personalized feedback on pronunciation. One of the main challenges is defining what pronunciation is incorrect. The origin of the speaker and the accent of the speaker heavily affects the pronunciation of a person. In addition, due to the number of dialects available for each language, it is hard to determine the correctness of the pronunciation of a word in that specific language. Therefore, evaluating the pronunciation of a word is given to the users in most learning language applications.

A design challenge emerged...

How can we improve people's ability to communicate in a new language?
the Solution

We designed Pocket Pathologist, a mobile application that acts as a language pathologist that you can always carry with you. Pocket Pathologist gives users the ability to practice their pronunciation without the need to visit a speech-language pathologist and without the need to understand how languages work. 

The application asks users to pronounce their desired word, performs analysis on the users' recording to identify the errors in their vowel's pronunciation, and guides users on how to improve their vowel's pronunciation. Our solution focuses on the International Phonetic Alphabets (IPA), an alphabetic system of phonetic notation devised by the International Phonetic Association as a standardized representation of the sounds in all spoken languages. It translates each word to IPA and compares users' input frequency with the value for each IPA character.

The application focused on vowel pronunciation. The three main defining characteristics that we focus on are:

  1. Height: How high your tongue reaches into your mouth while pronouncing.
  2. Roundedness: How round your lips are while you are pronouncing.
  3. Backness: How far your tongue reaches into your mouth while pronouncing.

All three characteristics affect the frequencies and the harmonies that are generated when we pronounce a word. By performing frequency analysis, we can identify what vowels the user is pronouncing. Using the characteristics of the users' pronounced vowels, the application comments on the users' issues and provides guidance on how to improve their pronunciation. The design also displays a visual graph that shows how far apart the user's pronunciation is from the correct pronunciation.

Using IPA, the project can be expanded to all languages, dialects and accents by simply adding a database entry for that language.

No items found.

But how did it all happen?

As the lead designer and developer of the project, I conducted interviews with experts of the field to identify a unique opportunity to address. I led the team in developing a prototype, and creating a new algorithm to detect inaccurate pronunciation. I then ran user testing sessions to evaluate the success of the algorithm in detecting correct pronunciation.
the Process

I employed a Lean UX process. This methodology helped speed up the process and allowed quick conversion of data collected from experts to prototypes. The iterative nature of the process, allowed the prototype functionalities to be built over time, allowing enough time for testing of each feature of the product. Furthermore, I was able to take advantage of the maker community’s wide range of backgrounds, skills, and knowledge to identify design opportunities and evaluate the built prototype.

Lean UX Process

What did I learn?

the Challenges

The biggest challenge was the time constraint set on the project. Our team only had four weeks to implement the algorithm and thoroughly test the system before running any study to measure the system's usability. In addition to the time issue, we also had to deal with limited technology, not having access to the right tools to test our implementation before creating our high fidelity prototype. We had to use a custom-made circuit with a high noise level to test the system, which took an additional two weeks in our early stages.

Reflection

Our designed system is highly sensitive to noise and requires a quite room in order to work accurately. This is one thing that was only found out after running the final study in a crowded space with the final product. This raised the issue of what other kinds of environmental factors can have an effect on our system's accuracy. One thing I took away from this experience was to consider the impact of different environmental factors on the built prototype during early testings to make sure I won't run into similar issues in other projects.

Final notes

This project was awarded the Capstone Distinction Award by the Department of Electrical and Computer Engineering at the University of Toronto.

Scroll back up