WVU research team steers students through murky waters of ChatGPT coding

West Virginia University researchers are studying how high school seniors and college students can utilize ChatGPT to learn how to code, while recognizing the potential shortcomings of the AI-powered chatbot. (WVU Photo/Alyssa Reeves)

Researchers at West Virginia University are preparing high school seniors and college students to harness the power of ChatGPT, the popular artificial intelligence chatbot, through coding while addressing the platform’s potential shortcomings.

Benjamin M. Statler College of Engineering and Mineral Resources

In a paper published in Quantitative Biology, a team led by Gangqing “Michael” Hu, assistant professor in the WVU School of Medicine Department of Microbiology, Immunology and Cell Biology, explored how they could help students overcome the fear of learning code and enhance their critical thinking skills.

“This is like kids wading in the muddy shoreline seeking beautiful seashells,” Hu said. “The kids are the beginning students and the muddy shoreline is ChatGPT. The beautiful seashells represent all the attractive opportunities which beginners cannot resist. But the ChatGPT shoreline is muddy with challenges such as the uncertainty from the chatbot’s response — including misleading artifacts — and students’ overreliance on AI for coding.”

ChatGPT produces human-like responses to text-based conversations and is being used by multiple companies to respond to customer inquiries and provide general information. Anyone can use it to seek information on a plethora of subjects.

One of the responses from ChatGPT can be code, and in this case the platform becomes a coding tool through prompting.

“This is a new tool of learning coding and there are some misconceptions,” Hu said. “Students may think coding is not important because of prompting or they may have a fear in trying to learn it. We need to educate them on the purpose of this chatbot-assisted learning. Because some of the code from the chatbot can be wrong, students need to use critical thinking to be able to tell if the answer is correct and, if it is not, how to find a solution.”

One of the biggest drawbacks to ChatGPT is that generated responses to questions can be either correct, incorrect or incomplete. In fact, it takes a human to provide carefully crafted prompts to fully harness the tool in providing valid and robust results.

Inspired by adaptive learning in educational literature, the team used the OPTIMAL model to facilitate chatbot-aided scientific data analysis. OPTIMAL, which stands for Optimization of Prompts Through Iterative Mentoring and Assessment, involves a series of steps to improve communication with a chatbot. In this case, it was geared toward bioinformatics, the science of collecting and analyzing large amounts of biological, medical and health information. Researchers say the model can be used for other purposes as well, such as finance and economics.

“The OPTIMAL model is like rubber boots for the children to wear at the muddy shoreline,” Hu said. “The boots protect the kids from getting dirty much like the model is a protective mechanism to prevent the students from being misled by inaccurate information from the chatbot. The model aims to improve both coding skills and prompting skills through an iterative communication with a chatbot guided by critical thinking and assessment.”

Following the OPTIMAL model, students review all the information needed for input and receive guidance on how to create a set of draft prompts. Once they input the prompt, the chatbot produces code and students are ready to give it a test.

If error messages result after running the code, students must evaluate the error and determine the best way to proceed, such as instructing the chatbot to revise the code or debugging the code manually.

The process continues until the code no longer issues errors and outputs a result for critical assessment. At the end of the session, students reflect on the entire communication process and review the code to identify any missing details to finalize the prompts.

The research team found that merely using the chatbot as a code-generating tool may limit creative thinking and that reviewing the code at the end of each session is just as important as optimizing the prompts.

The work brought together researchers from a spectrum of disciplines: Evelyn Shue, student volunteer in the Department of Microbiology, Immunology and Cell Biology; Bingxin Li, WVU John Chambers College of Business and Economics; Xin Li, WVU Benjamin M. Statler College of Engineering and Mineral Resources; Zifeng Feng, University of Texas at El Paso; and Li Liu, Arizona State University.

The team plans to evaluate the OPTIMAL model’s effectiveness in enhancing traditional bioinformatics education for beginners in the classroom.

“This assessment should encompass improvements in coding skills, prompting skills, problem-solving abilities, and critical thinking during interactions with a chatbot,” Hu said. 

Future research will delve into more profound insights and strategies on applying ChatGPT to precision education.

“There are certainly other sorts of rubber boots,” Hu said. 

---