Design of a Road- side Threat Alert System for Deaf Pedestrians

Listening is an important human skill which plays a vital role in our day to day life. Moreover, the reactions of people are very much correlated to the sounds they hear. Thus, deaf people have to undergo many difficulties in identifying threats when walking along roads. Many deaf pedestrians have lost their lives when walking along a road or crossing a road. Although many devices that support deaf pedestrians are being employed around the world, most of those devices are very expensive and also do not provide a detailed notification to the user. Hence, an improved deaf pedestrian assisting device is highly preferred. This paper proposes an electronic road-side threat alert system which would be a handy device to assist deaf pedestrians to identify objects moving behind them such as vehicles & other threats. With the help of this wearable device, they would be able to sense the environment around them. The proposed system is based on a microcontroller hosting a Bayesian classifier unit and an array of ultrasonic sensors for distance measuring. The notification to the deaf pedestrian is provided via a set of vibrating motors. A prototype implemented has demonstrated very good accuracy in the identification of threats at a low cost.


Introduction
As the four skills -listening, speaking, reading and writing -are important for our day to day activities, hearing plays a vital role in everyone"s life. People react according to the sounds they hear. Thus, when walking along a road or crossing a road, deaf people face many difficulties due to their inherited disability. Therefore, this paper introduces a device which will help deaf pedestrians to understand threats that come from sides or behind and separately identify objects such as vehicles. With the help of this device, they may also be able to understand what is close to them.
Furthermore, by understanding the levels of the threats, the deaf pedestrian will be able to avoid risk situations. Thus, by using this device, deaf people would be able to walk along or cross roads with a reduced risk.
There are several devices currently available such as the Vibering Jewellery Sensing System, Deaf Alarm Table, Hearing Aids, etc., that assist deaf pedestrians [1 -3].
The Vibering Jewellery Sensing System [1] consists of a pair of sound sensing rings and a wristwatch. The rings have to be worn on both hands and the wristwatch on one hand. The two rings function much like ears. They pick up the sound and transmit it to the wristwatch which then vibrates and alerts the pedestrian not only on the type of the sound but also on the distance to and the position of the source of the sound. The jewellery senses system identifies only vehicles behind the deaf person and it does not classify the vehicles.
The Chinese system, "Deaf Alarm Table" can identify threats during emergencies by sensing the sounds and then alerting the hearing impaired pedestrians [2]. This system employs the mechanism of vibration to help the hearing impaired as it generates warnings during an emergency. However, the Deaf Alarm Table  can identify only the loud sounds closer to the pedestrian.
The Hearing Aid is another device currently available [3], typically worn inside or behind the ear which amplifies sounds to help the deaf people. However, this device helps only to amplify the sound and the system cannot be used by persons completely deaf. Moreover, it does not support threat classification. The proposed roadside threat alert system is expected to be superior over the aforementioned systems and is expected to provide a complete solution to deaf pedestrians.
The rest of the paper is organized as indicated below. Section 2 presents the proposed system architecture, Section 3 explains the Bayesian classification algorithm and Section 4 presents the process of classifier training.
Finally, Section 5 produces the test results to validate the design followed by the conclusion in Section 6.

Figure 1 -System block diagram
As already mentioned in Section 1, this system can identify vehicles types and other threats close to a deaf pedestrian and can alert the latter in an easily understandable form. The system consists of four ultrasonic distance measuring sensors, a sound detecting sensor, a main control unit which hosts the Bayesian classification algorithm, three vibrating motors (for notification), an object detection notifying unit and a power supply ( Figure 1). All these sub-units are to be placed on a wearable belt.

Ultrasonic Distance Measuring Sensor
The ultrasonic sensor is an electrical device which is employed to convert electrical signals into ultrasonic waves and vice versa. An ultrasonic distance measuring sensor module will consist of a transmitter and a receiver which are available either as separate ultrasonic transmitter / receiver units or embedded together as a single transceiver unit. In any of the two configurations, the sensor functions as shown in Figure 2.  d1 -Distance between the transmitter and the object  d2 -Distance between the receiver and the object  d3 -Distance between the transmitter and the receiver  d -Distance between the ultra-sonic sensor module and the object  θ -Angle between the transmitted wave and the received wave Note that in a practical situation, d3 will be of a very small value compared to d1 and d2; and therefore, Given the speed of the ultrasonic wave , where t is the travel time of the coded ultrasonic wave.

Sound Detecting Sensor
The sound detecting sensor mainly consists of a condenser microphone which converts sound energy (mechanical) into electrical energy. This detected sound has different average analogue amplitude levels depending on the type and the Sub-process route 2 (for heavy vehicles)

Start
Closest obstacle is closer than 15m? Wait Identify whether the close obstacle is a vehicle or other threat and follow the classification (Sub-process route 1)

A heavy vehicle?
A large vehicle?

Main Control Unit
The control unit receives sensor readings, identifies the class of the threat/vehicle based on a Bayesian classification and then sends a signal the notification unit as depicted in Figures 3 -8.

Bayesian Classification
Note that the proposed system has different notification functions such as:  Time varying vibrating functions  Level varying vibrating functions These functions are based on different types of vehicles as per the system"s classification results. Therefore, this system requires a well performing classification technique which is both simple and accurate. Therefore, a Bayesian Classification based classifier is highly desired [4 -7]. Furthermore, this system has two cascaded stages of classifications for increasing the accuracy of the system through the following two stages:  First stage of classification via distance sensors  Second stage of classification via sound sensors.
Let us label the four ultrasonic sensors as S1, S2, S3, S4 and the sound sensor as S5. These sensors are placed at the back side of the belt as shown in Figure 9. Furthermore, recall that this classification is solely based on the distance between the obstacle and the sensors.

Probability Functions for the First Stage Classification of Vehicles
Let the a-priori probability of different classes of vehicles and other threats be given by: P(HR) = P( ) -A-priori probability of a heavy vehicle on the right hand side P(HL) = P( ) -A-priori probability of a heavy vehicle on the left hand side P(LR) = P( ) -A-priori probability of a large vehicle on the right hand side P(LL) = P( ) -A-priori probability of a large vehicle on the left hand side P(SR) -A-priori probability of a small vehicle on the right hand side P(SL) -A-priori probability of a small vehicle on the left hand side P(SB) -A-priori probability of a small vehicle at the back side P(OR) -A-priori probability of other threats on the right hand side P(OL) -A-priori probability of other threats on the left hand side P(OB) -A-priori probability of other threats at the back side. where ---- (6) With (3), (4), (5) and (6), values can be calculated. Then, the class of the vehicle / threat under consideration will be given by: The above classifier can be implemented on a microcontroller board but the computational complexity will be a concern. Therefore, in order to evaluate (4), we will consider McLaurin series expansion"s first three components [8] given by:

Second Stage of Classification
After completing the first stage of classification, the system will re-evaluate the classification results with the use of the sound detecting sensor (S5) and take a final decision on the input sample data. This second stage of classification is most important for A5, A6 and A7 groups where small vehicles and other threats are separated based on the input from S5. Finally, it will increase the accuracy of the overall classification. Note that sound is expected to be a good demarcation property for small vehicles and other threats. Figure 10 demonstrates the class regions of vehicles and the other threats in a plane of sensor readings.

Notification Unit
The notification unit is also an important subunit and consists of an array of vibrating motors. Moreover, the vibrating patterns implemented can be easily identified by the M1 M3 M2

Left side
Right side deaf pedestrians. The proposed vibrating motor placing arrangement of the belt is as shown in Figure 11.

Prototype Implementation
A prototype system was implemented using SRF02 ultrasonic distance measuring units, an Arduino Mega controller board and a condenser microphone based sound detection module.
Since this system requires a direct current (DC) voltage of +5V and +9V, the powering of the system is achieved by two 9V batteries complete with proper regulator circuits. Furthermore, the implemented battery level indication sub system can identify the state of the two batteries separately and indicate them using Light Emitting Diodes (LEDs) ( Table 1). The physical construction of the proposed system comprises of a wearable belt to which all the electronic devices including the batteries and sensors will be fixed ( Figure 12). The deaf pedestrian can wear this belt around his waist when walking along the road.

Training for Classification
The prototype implemented was used to collect 10 sets of sample data under pre-known vehicle/threat types of each category on the roads. Furthermore, sound sensor readings were obtained for 28 known vehicles/other threats from small vehicles and other groups. A sample of results are listed in Table 2 and  Table 3. Note -Sensor digital reading will be between 0 and 1024

A-priori Probability of Vehicle Classes and Other Threats
Furthermore, an observation was carried out along many roads to calculate the a-priori probabilities of different vehicles and threats and the results are listed in Tables 4-7.

Test Results and Discussion
Using the same prototype, the validity was tested for a known set of test case samples and the accuracies obtained are as follows:  Probability of detecting closer objects All closer objects were identified 100% within 5m to 10m range.
 Probability of classifying vehicle types It is apparent that the proposed system and the prototype implemented are capable of achieving high accuracies and these facts therefore validate the design.

Conclusion and Future Work
Due to the absence of a sense of sound, deaf pedestrians face many difficulties while on the road and some have met with serious accidents as a result. The proposed system is of immense benefit to deaf pedestrians and has shown promising accuracies in prediction / alerting. However, this particular design and its implementation suffer from a few drawbacks. One such drawback is that it was made of low cost, low range ultrasonic sensors which can detect vehicles only within a few tens of meters. This range may not be adequate to warn when it comes to fast vehicles. Hence, implementation, testing and validation under long range ultrasonic sensors would be an interesting research area for the future and by using these high quality sensors, the system will be able to meet a very highly accurate vehicle classification even for long ranges. At the same time, by adding more sensors it would be possible to predict the types of vehicles with much higher accuracies. Nevertheless, even with the low quality sensors, the prototype provided 100% accuracy in the detection of a short range threat thus it can immensely contribute to eliminate the risks faced on the roads.