Companionship Chatbot
Listen for the tone, not just the words.
In 30 seconds
- Problem
- Make an LLM companion listen for the tone of a message, not just its words.
- My part
- I led the four-person capstone team, handling the planning and direction, and designed the backend LLM: the DistilGPT-2 server and the sentiment module. One teammate designed the React Native app, and two others helped with it briefly.
- Result
- First Place, Undergraduate Capstone, at the KSU C-Day Computing Showcase, Spring 2024.
The project in numbers
- Code
- 1,360 lines of code in the final project.
- Components
- 28 components.
- Emotions
- 8 emotions the sentiment module can name.
- Test prompts
- 10 labelled prompts in the report’s test.
Check the source Final report · PDF
How it works
The app is React Native, with login, chat and past conversations. Each message goes to a Python server, where a sentiment module names the emotion in it and a DistilGPT-2 model, using the QuAC dataset, writes the reply. Firebase Realtime Database handles logins and user information. The report also notes that storing chat history as context for the model was withheld from the final product.
The module names one of eight emotions: joy, sadness, anger, fear, surprise, disgust, trust or anticipation. Knowing the emotion lets the reply console or celebrate.
The module sits in the LLM pipeline on the Python server: it reads textual cues in each message to name the emotion, and the model adjusts its reply to fit; the public report does not name the classifier behind it.
Check the source Final report · PDF Code · GitHub
- Accounts Firebase Realtime Database
- React Native Mobile app
- First Sentiment module
- Then DistilGPT-2
- Python server
- The app signs people in and stores their information with Firebase Realtime Database.
- The app sends each message to the Python server.
- The sentiment module names the emotion before DistilGPT-2 writes the reply.
- The reply goes back to the app, ready to console or celebrate.
Eight emotions
| Emotion | Prompts |
|---|---|
| Joy (two prompts) | 2 |
| Sadness (two prompts) | 2 |
| Anger | 1 |
| Fear | 1 |
| Surprise | 1 |
| Disgust | 1 |
| Trust | 1 |
| Anticipation | 1 |
Two prompts from that set:
“I just got promoted at work and I'm ecstatic about the new opportunities!”
Joy
“I lost my favorite necklace today, and it's left me feeling really down.”
Sadness
The deployment plan
The C-Day abstract describes the Azure deployment as a plan. The report sized it two ways. For a fast-growing user base, two Azure NC6s_v3 GPU servers, one on the east coast and one on the west, each with a replicated Firebase database, so a regional outage would not take the app down. For an early, small user base, one server and one database in the east.
| Setup | Requests per minute | Users | Monthly cost |
|---|---|---|---|
| One server, east (early stage) | 1,200 | 10,000 | about half |
| Two servers, east and west | about 2,400 | about 22,000 | $5,070.56 |
Check the source Server architecture · PDF
Watch and read
One hard decision
Every message goes through the sentiment module before the reply. Without it, as the report puts it, the model was a basic conversationalist; with it, the reply can console a sad message or celebrate good news.
One limit
The report’s test fed the 10 labelled prompts to the backend and noted only the emotion it detected. It ignored the replies and reports no accuracy figure; the report’s reason is that testing an LLM widely is costly.
The evaluation lab shows how I evaluate a classifier instead, on open data: a held-out test set, bootstrap intervals, a threshold you can move, and failures picked by rule, not by hand.