Audio to Sign and Sign Language to Text Conversion using Python
A Desktop based app designed and developed using python programming language. There are two main features of the project, firstly to convert audio to sign language and other to convert sign language to text. Its aim is to bridge the communication gap with deaf and dumb people.
Modules / APIs / Algorithm
1. Tkitner : Graphical module of python to design and develop the Graphical User Interface of the tool.
2. Tenserflow : Framework used for training the CNN model
3. Keras API : for providing a high level of abstraction.
4. CNN model (2D): Convolutional Neural Network. For Recognizing Gestures of characters
5. easyocr : API used for doing OCR of hindi characters from image
6. Pandas : Library used in python for accessing csv files.
7. numpy : Library in python used for accessing images in array format
8. SpeechRecognition : Library in python used for converting speech to text
Audio to Sign Language Conversion using Python
Steps of the system : Audio to Sign Language
– Converting speech to text (using speech recognition)
– splitting text into characters
– displaying images of alphabets’ respective sign language in sentence format
>>> Sign To Text: It includes two phases; 1st phase is training & testing the model & 2nd phase is using the model…
i) ALGORITHM FOR TRAINING AND TESTING THE MODEL :-
1) First of all we need to download the dataset; we downloaded from UCI Machine Learning Repository ; https://archive.ics.uci.edu/ml/datasets/Devanagari+Handwritten+Character+Dataset#
2) Dataset includes, png format images of resolution 32*32, so we need to convert the dataset to csv file.
3) we fetched all the images and stored the binary formatted value of image in csv.
4) dataset is ready to use.
5) After getting the dataset, we will train the model.
6) For training CNN2D sequential model is used.
7) first of all we need to prepare two parts of dataset for training and testing purpose.
8) for dividing, firstly the dataset is shuffled and then divided to 80-20 ratio.
9) Since the dataset is all set, prepare the model’s architecture.
10) Layers of sequential model are: CONV2D > AVERAGEPOOLING2D > DROPOUT > CONV2D > AVERAGEPOOLING2D > DROPOUT > FLATTEN > DENSE > DROPOUT > DENSE
11) Use the activation function as ReLu.
12) After passing from all these layers, we will fit our training data to Model. And set epochs as 35. with batch size of 64
13) After finishing it, we will send testing data to evaluate the testing.
14) visualizing the results using matplotlib module.
15) saving the model. ii) ALGORITHM FOR USING THE MODEL :-
1) Load the model
2) load the module OpenCV for getting live frames from webcam
3) setting the upper and lower range of blue color, for detecting the blue color object.
4) Applying flip, cvtColor, inRange, medianBlur, GaussianBlur & threshold layers of OpenCV into frame for removing noise and detecing the blue color.
5) track the movement of the blue object and draw line in the route of movement
6) Once the blue object is not found, sending the gesture motion to predict the character.
7) before prediction we need to preprocess the image by , resizing it, converting to numpy array,and reshaping it.
8) this array is used as parameter for keras function “predict”.
9) predict function gives some value between 0 to 37.
10) this value is searched in dictionary of characters (we already made to store characters)
11) if found, value is printed.
Sign Language to Text Conversion using Python | Audio to Sign Language Project
Software Requirements :-
- Coding Language : Python
- Implementation: Software Framework.
- Operating system : Windows 10 / 11.
- Graphical User Interface : Tkinter
Hardware Requirement:-
- Input Devices : Keyboard, Mouse.
- System : Pentium i3 Processor.
- Hard Disk : 500 GB.
- RAM : 4 GB.
☎️ Contact Us For More Queries:-
📲 Call/WhatsApp: +91-9460060699
🌎 Website: www.techieprojects.com
📺 Instagram: @pythonprojects_
💡 Checkout Related Projects:-
1. Android App:- Click Here
2. Java Projects:- Click Here
3. OpenCV Projects:- Click Here
4. Data Science Projects:- Click Here
5. Data Analytics Projects:- Click Here
5. Deep Learning Projects:- Click Here
6. Cyber Security Projects:- Click Here
7. Machine Learning Projects:- Click Here
8. Image Processing Projects:- Click Here
9. Web Development Projects:- Click Here
10. Game Development Projects:- Click Here
11. Artificial Intelligence Projects:- Click Here
12. Database Management System:- Click Here