Voice and Linguistic Analysis to Determine Emotions in Friends' Interactions

Djerroud-project

Voice and Linguistic Analysis to Determine Emotions in Friends’ Interactions Welcome to my project, where I explore the intersection of voice analysis, linguistic analysis, and natural language processing (NLP) to uncover the emotions generated during interactions among friends. This project analyze speech patterns, text exchanges, and vocal nuances to detect and interpret emotional states and determine the closeness betweeen the characters.

Features:

Voice Analysis: Utilizes Praat for detailed spectrograms, pitch, formant analysis, and other acoustic features. Emotion Detection: Employs OpenSmile for extracting vocal features and identifying emotional cues from speech. Linguistic Analysis: Implements NLTK for tokenization, parsing, and sentiment analysis of textual interactions. NLP Models: Integrates state-of-the-art models from SpeechBrain and Hugging Face for comprehensive speech and text processing. Interactive Visualizations: Provides insights through visual representations of emotional states and linguistic patterns. Goals:

To understand how friends’ vocal expressions and word choices convey emotions. To develop a robust system for real-time emotion detection and linguistic analysis. To create a dataset and models that can be utilized for further research in social dynamics and emotional intelligence. Join me in this journey of decoding the emotional landscapes of social interactions through the power of voice and linguistic analysis!

Project Definition Objective Analyzing emotional dynamics in social interactions using voice, linguistic, and text analysis techniques applied to the “Friends” TV show dataset.

Background Dataset Source:

Obtained from MarieSTL, Pierre Bellec’s Lab, ensuring credibility and relevance to the research question.

Modality:

Includes audio (MP3) and textual transcripts (JSON) of dialogues.

Suitability:

Ideal for capturing real-life interactions among friends and exploring emotional dynamics.

Tools Voice Analysis

Librosa, Praat, OpenSMILE Linguistic Analysis NLTK, SpaCy, Gensim Emotion Detection Hugging Face Transformers

Data Content

Audio files (MP3) extracted from video episodes JSON transcript files containing dialogue text

Deliverables Expected Outputs

GitHub repository with code scripts and Jupyter notebooks Documentation (README.md) Dataset files (audio, JSON transcripts) Analysis workflow/pipeline Training materials and models

Results Achievements

Integrated voice and linguistic analysis for emotional dynamics Developed emotion detection models for audio and text inputs Established a robust analysis pipeline from data preprocessing to visualization

Progress Overview Summary

Successfully implemented tools for voice and linguistic analysis Challenges included data integration and model optimization Tools I Learned During This Project Praat, OpenSMILE for voice analysis Hugging Face Transformers for emotion detection Docker, Git/GitHub for version control and project management

Conclusion and Acknowledgement Next Steps

Enhance model performance and expand dataset coverage Prepare findings for publication and seek collaboration opportunities Acknowledgement Special thanks to MarieSTL, venkatesh and Pierre Bellec’s Lab for providing the “Friends” dataset Gratitude to the open-source community for tools and libraries used

More details

Additional information is available in the notebooks and presentation folders.