How Automated Image Analysis Can Help Us in the Fight Against COVID-19

For nearly a year now, the whole world has been in a state of emergency. SARS-CoV-2 has had a massive impact on our lives. Mandatory wearing of face masks, curfews and lockdowns have become part of our daily lives. But despite all of that, the infection rate remains high, and people are starting to have doubts as to the effectiveness of the measures that have been introduced. Where does the problem lie though? Are people wearing masks, and are they even observing the social distancing rules? 
 

Up until now, finding a comprehensive answer to that question seemed hardly possible. There are simply too many places where large crowds of people can gather. Evaluating CCTV footage also seems an impossibility, given the sheer amount of CCTV cameras these days. How are you supposed to properly analyze thousands of hours of footage? 

Fortunately, this is exactly what has now become feasible thanks to the latest AI technology. Today, we can check the content of video footage or images within just a fraction of a second. In this blog post, we will show you how we can use this automatic image recognition technology, taking our social distancing app as an example. 

Music: Time Out by Atch

SoundCloud: soundcloud.com/atch-music
Creative Commons — Attribution 3.0 Unported — CC BY 3.0
Free Download / Stream: bit.ly/l-time-out
Music promoted by Audio Library youtu.be/NPnLxESfv50

In order to identify the persons in a video or in an image, our app accesses  Azure Cognitive Services from Microsoft. Apart from automated language and text recognition, the latter also offers a range of services related to automated image recognition. Azure Cognitive Services covers a broad range of features from facial recognition, customizable object detection, which can recognize new objects with only a small number of images, up to specially trained AI for individual use cases. 

With the “Computer Vision” service, for example, large varieties of pre-defined objects can be detected. (The word “object” is used in the broad sense here. It also includes living things such as plants, animals or human beings.) While this service yields very reliable results when the number of objects is small, things are quite different once there is a larger number of objects. That’s why in crowded places, only a limited number of persons are recognized. 

However, since adhering to social distancing rules is especially difficult in busy places, we have used Azure’s customization option for our app and have made use of special neural networks which work well even when there’s a lot going on in the streets (and therefore also in the image/video). With each object that is detected, there is a probability of between 0% and 100% that it is actually a person. Our image recognition app gives you the opportunity to configure a threshold value for exactly this probability. What’s particularly essential in this regard is to figure out the optimum threshold value and achieve a balance between false positives (e.g. statues that are identified as persons) and false negatives (persons that the AI “overlooks”). A threshold value between 30% and 40% yields the best results. Once the persons have been identified, the distance between them can be measured in order to check how many people are observing the social distancing rules.

There are various fields of application for automated image or video analysis. For example, it is also conceivable to check whether people are wearing masks in addition to measuring the distance between them. However, possible uses go far beyond the current pandemic. They include:

- Checking whether occupational health and safety rules are adhered to (safety distances and danger zones) on constructions sites or when track works are carried out
- Locating particular persons in buildings (e.g. doctors in hospitals in case they are urgently needed for an emergency)
- Analysis of movement patterns in supermarkets and malls for optimizing product placement
- Analysis of strategies in case of sports such as soccer or handball
- Detection of abandoned luggage or suspicious behavior in airports and train stations
- Counting the number of persons at large-scale events
- Measurements based on image data for maintaining quality in production.

In principle, AI is useful whenever there are large amounts of video footage or images from which valuable information can be obtained by means of analysis.

Would you like to know more about image recognition systems and the use of Microsoft Azure Cognitive Services ? We are always available for personal consultation. Send us an email or give us a call. Our experts will be happy to discuss your ideas with you.

Would you like to know how exactly image recognition works and what the current state of research is? Then go on and read the second part of this blog post.