Amirah Mitchell: Intersecting AI with human emotion

Amirah Mitchell, sophomore student majoring in biomedical engineering student with a minor in data science.

In the summer of 2022, Amirah Mitchell started working on developing an artificial intelligence program that would understand human emotion, using deep learning approach. This was after she was given the opportunity to be a part of WVU’s Summer Undergraduate Research Experience. The biomedical engineering undergraduate researcher used the opportunity to start her project on how the technology could predict the emotional state of humans, whether happy, sad, angry, etc.

Research Community Spotlight Series: Written and photographed by Nathaniel Godwin, WVU Research Office

"I knew from the start that this emotion detection program would not be able to stand alone in the idea of detecting emotion. However, for me at least, it started with machine learning, it goes deeper than this to become neural network or deep learning."

Q: Give us some 42general background about yourself.

A: I’m from a suburb near Canton, Ohio. However, I graduated from Massillon Washington High-School. I have five siblings and I grew up with my mom and dad, with my siblings in the house for a while. I’m the second oldest in the family. I am majoring in biomedical engineering, with a minor in data science here at WVU.

Q: During WVU’s 2022 Summer Undergraduate Research Experience Symposium, you submitted a poster and made a presentation on deep learning approach to understanding human emotion. Tell us about it.

A: The SURE program allows undergraduate students to perform research tasks with a mentor over the summer for eight weeks with a stipend included. There is a class associated with it to help you improve professionally with research. This is to make sure you’re keeping up with things aligned with improving yourself and your academic identity. Improving one professionally, of course was with your resume, workshops that helped for networking, writing a professional letter, talking about research with your peers, elevator pitches and everything. So, through the summer I was able to work with  Jeremy Dawson (professor over the lab) and Aeddon Berti (graduate student). They were my two mentors during that period. 

After discussions with Dr. Dawson, I started my approach to decipher how I will have this “emotion detection program,” as his biometrics lab is equipped to do a lot of things with face recognition. I knew from the start that this emotion detection program would not be able to stand alone in the idea of detecting emotion. However, for me at least, it started with machine learning, it goes deeper than this to become neural network or deep learning. This is a computational model that can predict the classifications you gave it with your data input. I used the Jetson-Nano device by Nvidia to make this program; along with other supplemental material from courses provided by Nvidia.

Q: How would you apply this program for problem-solving in society?

A: There are useful ways to implement this into society and I found it amazing to have a start to this conversation. For example, people with autism called Alexithymia; it prevents one from understanding their emotion. If there is an application or device they can have on them to help understand how they are feeling, it might make their lives easier. I had a picture that was on my poster denoting the ‘accuracy’ of the program at the time which was about 40 percent at that time. At the symposium when a doctor (one of my judges) came up to me and told me that if there was a 40 percent chance she could know how one of her patients was feeling, she’d feel much more confident in her workspace. I never knew this perspective and how this is something doctors have to go through because they may only be able to guess how their patient is feeling. In another scenario, if someone is in their car, and they are experiencing fatigue after a long night shift, the program would help to protect and alert drivers around them (about the tired state of the driver by their side to make them more cautious).

Q: What was your motivation to embark on this program?

A: I wanted to do something different with the knowledge and resources I had available. Also, I wanted to make that contribution very impactful on our society. There are a lot of things one can do as an engineer to enhance life, but I wanted to do something that could touch almost every person anyone could know with this idea. Although this idea may not be original, but its potential gets greater as time goes on. Everyone has experienced something that has to do with their emotions, or at least know someone who has.

Q: What important areas of this research are you most interested in exploring further?

A: I will be looking at getting more data. The results and conclusions I want to have will have to include an accuracy that’s based on multiple people (a diverse group of people), but you cannot other people’s data without them being told about their privacy and rights.

How can I say this is an emotion from someone with the cultural practices factored in? In some cultures, facial expressions might not directly be linked with socially accepted correlations between facial expression and emotion in America. This idea can bring up many other discrepancies. If someone is depressed, they might smile but they are not actually happy. Also, implementing this in society and having cameras around people might be seen as creepy (for public areas). Also, there would need to be an inclusive way to reach the high-priority cases for detecting emotion because some people can’t afford cars, a phone, or any device with the appropriate features for this program.

Q: If you can travel back in time, what scientific or engineering discovery would you like to be a part of?

A: I would have to say the invention of computers, the creation of the internet, the creation of Python. Computer engineers have really contributed to my success today.

Q: If you can travel into the future, what engineering discovery/solutions would you want to be a part of?

A: Robots are getting more advanced nowadays, and I think it’s amazing to experience these advances as a teenager. However, I am still going to stick with artificial intelligence and machine learning – this is basically what makes the brain of the robot. Also, with artificial intelligence, since I am a biomedical engineer, I would be more interested in regenerative medicine in the future.

---