OpenCV - detection of human body

Thread Starter

FlyingDutch

Joined Mar 16, 2021
55
Hello,

I am trying to write application (using OpenCV library) in C++ for detecting in room human bodies (in many various positions). I am using "openCV" version 4.5.1 on Windows 10 OS with USB camera attached to my laptop. As a compiler (IDE) I am using "Visual Studio 2019" (which is working fine). As a target hardware (for detection) I am going to use "Raspberry Pi Compute Module 4" - see links:

https://www.raspberrypi.org/products/compute-module-4/?variant=raspberry-pi-cm4001000

https://datasheets.raspberrypi.org/cm4/cm4-datasheet.pdf

with a MIPI Camera attached to it.

First I tried to write a face detection program using "Haar cascade classifier". I used pre-trained classifiers from "OpenCV" folder: "\opencv\build\etc\haarcascades". Face detection is working efficiently - I test it on few video files downloaded from "YouTube". Then I tried to detect a bodies with this method ("Haar cascade classifier") and pre-trained classifiers from "OpenCV" folder: "\opencv\build\etc\haarcascades". I used for detection such classifiers:

1) haarcascade_fullbody.xml
2) haarcascade_upperbody.xml
3) haarcascade_lowerbody.xml

Results in detection human bodies are disappointing. So I started to wonder if "Cascade Harr Classifier" is proper method for detecting human bodies. Maybe someone could to point me better method for this task - I mean different algorithm/method. I would care the algorithm would be implemented using OpenCV library (I prefer C++, but Python is also OK).

I have also second question related to comparison "OpenCV" and "Keras" (together with Tensorflow). Has somebody an exprierience and can compare a results achieved in full human body detection using OpenCV or Keras framework.

Thanks in advance and Regards
 

Papabravo

Joined Feb 24, 2006
16,792
I would like to detect people who lost their consciousness in elevator.

Regards
It's big news that this is even a problem. Who is driving this? Is it benevolent building owners? How about elevator companies, concerned for the well being of elevator riders? Since everybody is now working from home, elevators and tall buildings may slip into disuse.
 

Thread Starter

FlyingDutch

Joined Mar 16, 2021
55
Hi @Papabravo,

that's funny what you wrote ;)but, it is real issue. If someone try automatically sterilize elevator (by ultra violet lamp ) after use by people, then he has to have certainty that nobody is in it. The dose of thirty minutes of ultra violet radiation (from UVC lamp) can seriously burn somebody. So infra-red motion sensors are not enough, and that doesn't matter if person lying on the floor of elevator is drunk or lost their consciousness (a difficult world for me).
So camera with microcontroller is additional protection for elevator sterilizer (using UVC lamp).

When you know the purpose of program maybe someone write something contributing to my problem.

Regards
 

djsfantasi

Joined Apr 11, 2010
7,827
Blue skying, based on a project I am working on.

Mount an ultrasonic distance sensor (like a Ppping or equivalent) on two axis driven by stepper or RC servo motors. Then, you can scan the interior of the elevator in two dimensions to see if there is any object inside.

The movements between positions of the distance sensor can be relatively coarse, since the object is relatively large. This is important, as the sensor needs to be calibrated at each position to capture the distances of the floor without any enclosed objects.

Then, repeat the scan as desired and test that the contemporaneous distance measured is consistent with an empty elevator.
 

Papabravo

Joined Feb 24, 2006
16,792
It seems to me like a variation of the overload sensor could detect the absence of measurable human weight. Take the elevator to a maintenance location, treat the elevator and return it to service.
 

Thread Starter

FlyingDutch

Joined Mar 16, 2021
55
It seems to me like a variation of the overload sensor could detect the absence of measurable human weight. Take the elevator to a maintenance location, treat the elevator and return it to service.
Hello,
it is not possible due to regulations related to elevators (which are very restrictive). Nobody can change in any way operation of elevator (from reason of security). Only one type of devices which can be installed in elevator are light fitings or just sterilizing lamp (and installed device must be compact, and vandal proof).

Regards
 

Thread Starter

FlyingDutch

Joined Mar 16, 2021
55
Blue skying, based on a project I am working on.

Mount an ultrasonic distance sensor (like a Ppping or equivalent) on two axis driven by stepper or RC servo motors. Then, you can scan the interior of the elevator in two dimensions to see if there is any object inside.

The movements between positions of the distance sensor can be relatively coarse, since the object is relatively large. This is important, as the sensor needs to be calibrated at each position to capture the distances of the floor without any enclosed objects.

Then, repeat the scan as desired and test that the contemporaneous distance measured is consistent with an empty elevator.
Hello,

also not possible due regulations related to elevators (it needs installation equipment in more than one location). And it must be vandal proof solution - in case of camera I found such version. Two motion sensors and camera (and of course UVC lamp) are built in one compact vandal proof device.

Regards
 

zazas321

Joined Nov 29, 2015
892
Hello. Can you clarify for me a little bit please. You have mentioned :
Only one type of devices which can be installed in elevator are light fitings or just sterilizing lamp

How are you going to fit a microcomputer and a camera in there then? A microcomputer and a camera is neither light fitting or a sterilizng lamp
 

Thread Starter

FlyingDutch

Joined Mar 16, 2021
55
Hello. Can you clarify for me a little bit please. You have mentioned :
Only one type of devices which can be installed in elevator are light fitings or just sterilizing lamp

How are you going to fit a microcomputer and a camera in there then? A microcomputer and a camera is neither light fitting or a sterilizng lamp
Hello,

a microcompuer (Raspberry Pi Compute Module) and small (MIPI) camera (and also two motion sensors) will be built up in casing of sterilizing lamp. Yes it is acceptable by regulations concerning elevators.

Regards
 

zazas321

Joined Nov 29, 2015
892
Understood. So are you allowed to embed any other electronics in the casing ? If you are, I would recommend you look into thermal sensors such as Omron D6T or MLX90640. If you combine them with with an ultrasonic sensor you should get a pretty reliable system which should recogise whether there is any living object inside an elevator even when not moving
https://answers.opencv.org/question/210645/detection-of-people-from-above-with-thermal-camera/
https://makersportal.com/blog/2018/...h-a-64-pixel-infrared-sensor-and-raspberry-pi

You should be able to retrieve information from every pixel and if the distance is set correctly, you should be able to cover the whole elevator without any blind spots

Or you are not able to add any additional sensors?
 
Last edited:

Thread Starter

FlyingDutch

Joined Mar 16, 2021
55
Hello,

No I didn't read this article in past, basically it concerns few types of ANN (Artificial Neural Networks). I wrote in my first post in this thread about "Keras" framework for deep learning (and Tensorflow as target runtime). I have expierience how to train ANN in Keras but it needs at least few thousands of images with proper examples. This is a reason that I started to look for a pre-trained classifiers in OpenCV library. It is likely that I will write target code in Keras and not in OpenCV because I have greater exprierience using Keras than OpenCV.

Thanks for link and regards
 

Thread Starter

FlyingDutch

Joined Mar 16, 2021
55
Understood. So are you allowed to embed any other electronics in the casing ? If you are, I would recommend you look into thermal sensors such as Omron D6T or MLX90640. If you combine them with with an ultrasonic sensor you should get a pretty reliable system which should recogise whether there is any living object inside an elevator even when not moving
https://answers.opencv.org/question/210645/detection-of-people-from-above-with-thermal-camera/
https://makersportal.com/blog/2018/...h-a-64-pixel-infrared-sensor-and-raspberry-pi

You should be able to retrieve information from every pixel and if the distance is set correctly, you should be able to cover the whole elevator without any blind spots

Or you are not able to add any additional sensors?
Hello,

I attempted using sensor similiar to this from second link (not exactly the same) and angle of sight was to small and it not encompassing all elevator space. It is reason for what I decided for camera with fish eye lens (angle of sight > 160 degrees).

Regards
 

zazas321

Joined Nov 29, 2015
892
From the D6t datasheet, I have found out :
1616505138949.png
Considering the average height of an elevator ~2 meters, it could work?
https://omronfs.omron.com/en_US/ecb/products/pdf/en_D6T_catalog.pdf


I do not have any experience with the computer vision, but the idea looks cool. I would assume that you are going to want to detect any object inside an elevator? The easiest method would probably be to compare 2 images ( you take image of an empty elevator) and you compare it to the image you take before you want to sterilize. If there are significant differences between 2 images, then you know there is an object or a person. There will of course be difference if a person is standing, laying down, sleeping or whatever compared to an empty elevator

Il stick around to see how you progress. Make sure to keep us updated :)
 
Last edited:

Thread Starter

FlyingDutch

Joined Mar 16, 2021
55
From the D6t datasheet, I have found out :
View attachment 233437
Considering the average height of an elevator ~2 meters, it could work?
https://omronfs.omron.com/en_US/ecb/products/pdf/en_D6T_catalog.pdf


I do not have any experience with the computer vision, but the idea looks cool. I would assume that you are going to want to detect any object inside an elevator? The easiest method would probably be to compare 2 images ( you take image of an empty elevator) and you compare it to the image you take before you want to sterilize. If there are significant differences between 2 images, then you know there is an object or a person. There will of course be difference if a person is standing, laying down, sleeping or whatever compared to an empty elevator

Il stick around to see how you progress. Make sure to keep us updated :)
Hello @zazas321,

I tried sensor of practically the same method of operating, wich has similiar parameters of this you introduced. There is a problem which significantly decline parameters of infra-red sensors. On oposite sides of sterilizer lamp (distance is less than 50 cm) are motion sensors which are emmiting infra-red radiation. When these sensor are switched off additional sensor is operating fine, but then someone switched them on it's operating is disturbed. This is reason I started thinking about camera working with visible light. Thanks for your post it's valuable. If I would has any results I write about it in this thread.
Thanks and Regards
 

Thread Starter

FlyingDutch

Joined Mar 16, 2021
55
Hello,

yesterday I installed OS (Linux) on "Raspberry PI Compute Module 4 (on photo this module is mounted in Raspbbery PI Compute Module 4 IO Board). Today I am installing (compiling from sources) OpenCV library on this module according to this tutorial:

https://learnopencv.com/install-opencv-4-on-raspberry-pi/

See photos:

IMG_20210327_171314.jpg
and

IMG_20210327_171403.jpg
I have also first results in human body detection (write about later).

Regards
 

Thread Starter

FlyingDutch

Joined Mar 16, 2021
55
Hello again,

I have few days to investigate and try few approaches to solve my problem - detection of human bodies in a room. I focused on solutions implemented in OpenCV library, and this time I didn't take into consideration solutions based on ANNs (Artificial Neural Networks), especially implemented in Keras framework.

First I decided to take into account two different methods - one for face detection and second for human body detection. In target solution both methods will be operating on image frames from camera and if one of them would give positive result it will be adopted that human was detected.

For face detection I decided to use "Face detection using Haar cascade Classifier". See this link:

https://www.bogotobogo.com/python/O...n_Face_Detection_Haar_Cascade_Classifiers.php


This pretrained Haar classifier supplied with OpenCV - 'haarcascade_frontalface_default.xml' properly detect on more or less than sixty percent of images (on which it hasn't be trained). I am going to train it additionally on more samples made by me (photos of my colleagues) - like descripted in this page:

https://docs.opencv.org/2.4/doc/user_guide/ug_traincascade.html

The second method (algorithm) for detection human bodies is based on HOG descriptor (Histogram of Oriented Gradients) connected with "Support Vector machines" (SVM). See these links:

https://learnopencv.com/histogram-of-oriented-gradients/

https://www.pyimagesearch.com/2015/11/09/pedestrian-detection-opencv/

https://www.thepythoncode.com/article/hog-feature-extraction-in-python

https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_ml/py_svm/py_svm_index.html

It also has proper detection of humaqn bodies ratio on level about sixty percent, and I also would like to train SVM with HOG Descriptor (pretrained form OpenCV). I think this page is a good start point for doing it:

https://stackoverflow.com/questions/57598104/training-of-svm-and-prediction-using-opencv-with-python

After additional training of this two methods and checking it inference (prediction) on new images I would like implement CNN(Convolutional Neural Networks) with Keras framework. After implementation I would like to compare OpenCV methods (first two described above) with its Keras analogue.

See this links to familiarize yourself with CNN:

https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53

https://cs231n.github.io/convolutional-networks/

All code will be implemented in Python programming language (with needed libraries).

Best Regards
 
Top