Projects

A voix haute

Facilitate the semantic analysis & vocabulary of students for the French baccalaureate

NLP MediaPipe FastAPI Docker

Improve our learners' speech...

"A voix haute" was my second project in my work-study at MySeriousGame, in collaboration with CNED.

the goal was to develop a POC (Proof of Concept) AI solution that could study the grammatical structure of sentences and identify the relationships between words in speech to produce a prediction on the level of oral expression.

There was a bonus : I had to spot the candidates' parasitic gestures with the help of a camera (e.g. combing their hair, scratching their nose, etc... not good for oral presentation :)

I therefore used NLP and computer vision techniques for this project.

At the start of the project, I didn't have much data. So I went looking for open source datasets to complete my data. I took the ted talks. Although these speeches are of a higher quality and not representative of what a candidate would produce on the first try, I decide to embark on the analysis and creation of a 1st machine learning algorithm on these data, which will serve as studies.

Cleaning

I used SpaCy to clean the data, in order :

• I've turned every speech into lower case
• I've normalized the data into a standard Unicode format is necessary to ensure compatibility and consistency, especially when handling texts containing special or non-ASCII characters.
• I've removed punctuation marks to simplify the text and focus on key words, eliminating characters that could be sources of noise or confusion.
• I haven't removed the stopwords and apostrophes because I want to keep the meaning of the words and the logical connectors (to be able to count them) before going through the speeches with SpaCy.
• Finally, I ran each speech through functions either to generate familiar words, quotations (data augementation) or to remove vocabulary, logical connectors (data reduction).

TF-IDF

I have trained my model with TF-IDF method to prepare my text data. Using TF-IDF transforms the raw text into a meaningful set of features, highlighting the relative importance of words in each document. This approach should help my model to understand and efficiently extract patterns in text data, enabling improved prediction performance.

About MediaPipe

To track down parasitic gestures, I used Google's MediaPipe library. This library enables to track hand movements in a video in real time, with precise and detailed analysis.

Avoiding distracting gestures during a speech is crucial to maintaining the audience's attention on the main message. Mastering gestures allows the candidate to maintain better eye contact and create stronger engagement with the audience.

The End and feedback

I leveraged FastAPI to develop the backend for this proof-of-concept (POC). To ensure portability and ease of deployment, I then containerized the entire application using Docker.

This intense year-long AI apprenticeship, culminating in two projects, significantly deepened my theoretical and practical AI knowledge. I gained hands-on experience developing AI applications with Python libraries (SpaCy, PyTorch) and Docker. This experience highlighted the importance of data management, algorithm implementation, teamwork, and project delivery, reinforcing that AI is also about problem-solving and understanding business needs. I'm excited to apply and further develop these skills in the future of AI and e-learning.

Offcial screenshots :