Flying Robin- IBM call for code (Event)

About: 

IBM call for code :

The Call for Code 2019 Global Challenge is a worldwide developer competition that seeks technology solutions for natural disaster preparedness, response, and recovery.

It is supported by the IBM Code and Response™ initiative, a multi-year program dedicated to creating and deploying open source technologies to tackle the world's biggest challenges.

For more information visit, >>> IBM call for code website

Flying Robin:

In regards to the events in Rio de Janeiro in Brazil, the flood claimed the life around 900 people. The main reason for this was poor post-disaster management, because of which a lot of citizens were not detected or found this led to leaving them in the mercy of the flood. However, if there existed a technology which can pinpoint exact location and as well as the scenario in which the survivors are trapped, could minimalize considerably low. This is the same in case of any post-disaster scenario, anywhere in the world the number of causalities will remain high and as the help and rescue team waste time in searching rather than on saving them keeps the odds of them surviving significantly bleak. This is where my team took up the challenge, to optimize the current scenario by making an application implementing drones and AI to pinpoint and understand the situation of the detected survivor is in. These situations for now can include rooftop survivors, boat rescue, stranded neighborhood, and submerged village.

The concept behind our project:

• Flying Robin is an application which is used to pinpoint and analyze the situation of the stranded survivors during a calamity.
• Flying Robin is used with the assistance of a drone with a video camera.
• This application takes in inputs of drones GPS location height of the drone from the ground and the vertical and horizontal angles of the video camera attached to the drone.
• These inputs are sent to camera_perspective.py file to calculate the region covered by the video camera and give us the latitudes and longitudes of the rectangle so formed.
• This particular frames of the video are unique with the help of the above-obtained latitudes and longitude values is then saved in a different folder(Here it is test_data). In a nutshell, those pictures are saved where the rectangle so formed from the camera is not overlapping with the previous rectangle thereby ensuring each time a new and distinct image is taken and does not create an error of detecting a single survivor multiple times.
• Now we run a pedestrian detection using OpenCV-python.If pedestrians are detected a rectangle is used to surround their body at that location
• Then we take the center point of the rectangle and convert it into percentage where the point lies on x-axis and y-axis
• This value is passed through the camera_perspective.py file to get that points latitude and longitude value
• This value and the file path of the distinct image file saved in step no 5 is stored in a pickle list, (Here they are Latitude_test.pickled, Longitude_test.pickled and Image_paths_test.pickled)
• Now, these pickled are called in a flask application to plot the latitudes and longitudes on the map.
• The file paths from Image_paths_test.pickled is passed through an image classification which is made in IBM cloud using Watson visual recognition API
• Now we make a list of labels according to the situation that is depicted in the image.
• We pass this list of labels with the coordinate and plot it on the map using flask web application.

• This method can significantly reduce the time taken by the search and rescue team to move into action instead os wasting time on finding the survivors

Github Repository:

will be updated soon