Sillah Babar

Academic

Career

Verbatim text

Lipsol, an Urdu Lip Reading Application
Lipsol is a research based project that lays the foundation of predicting sequences in Urdu using cropped out lip frames. The Urdu language consists of 36 characters and 62 sounds in comparison to English’s 26 characters and 44 sounds. This makes sequence prediction in Urdu an arduous task to model. Our project aims to predict sequences through lip movement which will in future tackle problems such as aiding people who are hard of hearing, picking sensitive words from cctv camera footage and having conversations that have privacy concerns. We have collected a dataset of 20 people each having spoken 108 sentences and applied various deep learning models , feature extraction and data augmentation techniques to conclude our project with a mobile lip reading application that will use the front camera to demonstrate the prediction of sequences. In our project, we also explored an approach to predicting words in real time which will also be a part of the mobile app.

LIPSOL A lip reading model for the Urdu language
Architecture
Lip Detection
Deep Learning Model (Recognising Words)
NLP Model
Video Feed
Mobile Application
Timeline
Literature Review Dataset Creation Face Recognition Image Segmentation & Processing
ITERATION 2
Context Analysis Transfer Learning of Algorithm Application of Language Model
ITERATION 4
Sep - Oct
Nov - Dec
Feb - Apr
May - June
Lip Detection Feature Extraction Sentence Slicing
Testing of Model Creation of Mobile Application Writing a Research Paper
ITERATION 1
ITERATION 3
Tools and Technologies
Supervised by Dr. Muhammad Asif Naeem
Ibrahim Aamer (190607) | Sillah Babar (192029) | Noveen Fatima (192047)
Technology Used:
Tensorflow,Flutter,Python,Flask,Keras,Open-cv
Supervisor Name:
Dr.Asif Naeem
Group Members:
Sillah Babar (19i-2029)
Ibrahim Aamer (19i-0607)
Noveen Fatima (19i-2047)

AI enrichment