Is this really "on" the RPi though? They mention inference on ARM Mali GPU (which is cool! But I don't think the RPis use Mali right?) but it wasn't clear to me if this was just using the RPi to take a picture and send it to an AWS server to do the actual inference, then return the results?
The last time I fiddled with inference on a RPi was several years ago when it took a couple seconds even for inference of a basic ImageNet CNN reading low-res images from the RPi camera (can't remember specifics sorry - I think it was on a RPi 2 or 3). FPS was something like 0.5-0.2FPS which made "real time" usage kinda difficult. I don't know what performance would be like now on a RPi4, but GPU accelerated inference on the Broadcom GPU would be nice.
Hi, Mattlondon, it's really nice to have your reply.
You are asking a good question, the 'Mali GPU' supporting is not for RPi, it's on the board which has 'Mali GPU' as RK3288/3399 or others (MTK/....).
For now, the reference code is running on RPi cpu, a light MXNET model do the extracting of face feature, the detection code is also running on RPi, the classifier is on RPi too.
The API server is the APP logic server, AWS/MINIO is storage layer.
The light model is https://github.com/deepinsight/insightface/wiki/Model-Zoo#34...
which running MobileNet as backbone. IF running on Mali GPU, ResNet50 can be used. Inference duration depends on the GPU power. if running on Mali 720 MP2, should be something close to 0.3s.
Who is SharpAI? This post is very much giving off the vibe of being a personal project but judging from their website[0] it's a Chinese company that's behind it.
Besides, the claim on their help center page that
> SharpAI DevKit is the world's first video camera that can run machine learning models and perform inference on devices
is not true at all. There are countless other companies offering similar products. (Which I'm not going to spell out here because I don't think technology as dangerous as surveillance AI deserves advertizing.)
I can recommend Nextcloud app Face Recognition [1] for the purpose of a FOSS face recognition. You can run that on a Raspberry Pi or whatever. You can off-load the face recognition to a different computer, if you desire.
Not relating to this project, but there's no reason you can't do offline facial recognition. I've blogged about doing offline object detection on raspberry pi and facial recognition is much simpler [0]. I've done it using the face_recognition [1] python package although I never released anything and it works fine. Only catch is you can only process a few frames a second, but unless you're trying to capture faces in fast moving environment, this should be fine.
I will note that aws facial recognition is far superior to the python package. If you need accuracy I would suggest an ensemble method where you use an inferior model (python package) to some level of confidence then offload it to the superior model that's off device for the final call and saving down the vectors.
Offline facial recognition is the way Apple does their face ID unlock mechanism. If it required an always on network connection and cloud inference the system would fail miserably.
Privacy is the most important issue for the AI surveillance camera. Use open source and BYOD server is the only way to solve this problem. SharpAI DeepCamera provides private cloud architecture to save all your information on your own storage.
> Use open source and BYOD server is the only way to solve this problem.
I don't think technology is the solution here. In fact, it is much more the problem here – an abundance of surveillance technology projects like this one will only result in more surveillance, not less. The average Joe doesn't care about the technology stack the surveillance camera[0] on the subway is running on. Instead, the mere presence of the camera will cause him to have a feeling of being watched and, thus, to behave differently.
The solution can only be a political one: Restrict the usage of AI in surveillance technology.
[0]: aka "security cameras", as they euphemistically called by their manufacturers.
650REDHAIR | 5 years ago
Does anyone have a good report on facial recognition during covid with the prevalence of masks?
coleca | 5 years ago
Yes NIST recently released a study: https://pages.nist.gov/frvt/html/frvt_facemask.html
mattlondon | 5 years ago
Is this really "on" the RPi though? They mention inference on ARM Mali GPU (which is cool! But I don't think the RPis use Mali right?) but it wasn't clear to me if this was just using the RPi to take a picture and send it to an AWS server to do the actual inference, then return the results?
The last time I fiddled with inference on a RPi was several years ago when it took a couple seconds even for inference of a basic ImageNet CNN reading low-res images from the RPi camera (can't remember specifics sorry - I think it was on a RPi 2 or 3). FPS was something like 0.5-0.2FPS which made "real time" usage kinda difficult. I don't know what performance would be like now on a RPi4, but GPU accelerated inference on the Broadcom GPU would be nice.
[OP] simbaz | 5 years ago
Hi, Mattlondon, it's really nice to have your reply. You are asking a good question, the 'Mali GPU' supporting is not for RPi, it's on the board which has 'Mali GPU' as RK3288/3399 or others (MTK/....). For now, the reference code is running on RPi cpu, a light MXNET model do the extracting of face feature, the detection code is also running on RPi, the classifier is on RPi too. The API server is the APP logic server, AWS/MINIO is storage layer. The light model is https://github.com/deepinsight/insightface/wiki/Model-Zoo#34...
which running MobileNet as backbone. IF running on Mali GPU, ResNet50 can be used. Inference duration depends on the GPU power. if running on Mali 720 MP2, should be something close to 0.3s.
mendeza | 5 years ago
This is really cool! I wonder how the performance would be on a Jetson Nano!
[OP] simbaz | 5 years ago
yes, it would be really nice if run on Jetson Nano with GPU accelerator...
codethief | 5 years ago
Who is SharpAI? This post is very much giving off the vibe of being a personal project but judging from their website[0] it's a Chinese company that's behind it.
Besides, the claim on their help center page that
> SharpAI DevKit is the world's first video camera that can run machine learning models and perform inference on devices
is not true at all. There are countless other companies offering similar products. (Which I'm not going to spell out here because I don't think technology as dangerous as surveillance AI deserves advertizing.)
[0] https://translate.google.com/translate?hl=en&sl=auto&tl=en&u...
rexreed | 5 years ago
Of course the OP is shilling SharpAI. Look at all of the OP contributions on HN.
Fnoord | 5 years ago
Nice default credentials.
I can recommend Nextcloud app Face Recognition [1] for the purpose of a FOSS face recognition. You can run that on a Raspberry Pi or whatever. You can off-load the face recognition to a different computer, if you desire.
[1] https://github.com/matiasdelellis/facerecognition
bko | 5 years ago
Not relating to this project, but there's no reason you can't do offline facial recognition. I've blogged about doing offline object detection on raspberry pi and facial recognition is much simpler [0]. I've done it using the face_recognition [1] python package although I never released anything and it works fine. Only catch is you can only process a few frames a second, but unless you're trying to capture faces in fast moving environment, this should be fine.
I will note that aws facial recognition is far superior to the python package. If you need accuracy I would suggest an ensemble method where you use an inferior model (python package) to some level of confidence then offload it to the superior model that's off device for the final call and saving down the vectors.
[0] https://medium.com/ml-everything/offline-object-detection-an...
[1] https://github.com/ageitgey/face_recognition
rexreed | 5 years ago
Offline facial recognition is the way Apple does their face ID unlock mechanism. If it required an always on network connection and cloud inference the system would fail miserably.
[OP] simbaz | 5 years ago
Privacy is the most important issue for the AI surveillance camera. Use open source and BYOD server is the only way to solve this problem. SharpAI DeepCamera provides private cloud architecture to save all your information on your own storage.
codethief | 5 years ago
> Use open source and BYOD server is the only way to solve this problem.
I don't think technology is the solution here. In fact, it is much more the problem here – an abundance of surveillance technology projects like this one will only result in more surveillance, not less. The average Joe doesn't care about the technology stack the surveillance camera[0] on the subway is running on. Instead, the mere presence of the camera will cause him to have a feeling of being watched and, thus, to behave differently.
The solution can only be a political one: Restrict the usage of AI in surveillance technology.
[0]: aka "security cameras", as they euphemistically called by their manufacturers.