Deaf and Dumb people are a big part of growing community where they use sign language i.e., gestures for communicating with others. But interaction between normal and hearing impaired people becomes difficult as most of the normal people can't understand sign language gestures meaning and on the other hand deaf and dumb people can't understand naturally spoken language. There are about 70 million deaf hearing-impaired as well as hearing people in the world who use sign language as their first language or mother tongue. In Bangladesh alone, there are around 2.4 million people use sign language. Therefore, automatic sign language translation system is vital to improve communication for hearing impaired people. This paper represents a review on existing Bangla Sign Language translation systems to communicate with hearing impaired people and also proposes a real time bidirectional Bangla Sign Language translator. Analyzing the existing systems provides us necessary information about their working process, success rate, system drawbacks and limitations indicating the fields where those systems need to upgrade for a better performance. Based on the analysis to overcome the limitations, we propose a bidirectional communication system that includes conversion of naturally spoken Bangla Language to Bangla Sign Language and vice versa. Our proposal includes real time dynamic bidirectional Bangle Sign Language translation system which is preciously needed for a fruitful communication with hearing impaired people.