| Skill Level | Area of Focus | Operating System | Platform/Hardware |
|---|---|---|---|
| Intermediate | Computer Vision, Artificial Intelligence | Android | Artificial Intelligence |
FaceBlock is a tool that is designed to allow individuals to record themselves without exposing the identities of those around them. Using both deep neural networks and classical machine learning methods, FaceBlock uses facial detection and tracking to block unwanted faces with custom emojis and icons in videos and streams.
Objective
With the ubiquity of smartphones, live video streaming has exploded in popularity on platforms like Facebook, Instagram, and YouTube. As more and more people are caught on video every day, significant concerns have been raised over online privacy. We wanted to utilize leading edge solutions from Qualcomm Technologies to help protect the privacy of others in an increasingly streamed and shared world.
Materials Required / Parts List / Tools
- Tensorflow
- Tensorflow Object Detection API
- Android SDK
- Android NDK
- OpenCV4Android
- Qualcomm Neural Processing SDK
Source Code / Source Examples / Application Executable
Additional Resources
- Object Detection API Installation Documentation
- WIDER Dataset Download Instructions
- Preparing TFRecords For Training Model Guide
- Converting from Frozen Tensorflow Model to Qualcomm Neural Processing SDK DLC format
Build / Assembly Instructions
Compatibility
In order to have the detection model run in real-time, GPU acceleration must be used. This is only supported on the Galaxy S8/S8+ and Galaxy S9.Project will likely work on any Android M+ phone. DSP acceleration is not available in this version of the project.
Dependencies
- Android SDK v27
- Android NDK v16b
- Tensorflow v1.8 (optional)
- Tensorflow Object Detection API (November 17, 2017 Release) (optional)
- OpenCV4Android v3.2.4 (optional)
- Qualcomm Neural Processing SDK v1.15 (optional)
The optional items are listed for users who wish to train and deploy their own Tensorflow based models onto the app. They are not required if a user simply want to use the defaults provided.
Build Instructions
Begin by setting up your Android development environment and installing the above dependencies. Ensure you have the correct version of the Android SDK, Android NDK and Android Studio installed. The converted DLC models used are supplied in this project, so the portion of this guide will be about the build process.
After setting up your environment, import your project through Android Studio as an existing project.
Building OpenCV4Android with Tracking
If you are not using the prebuilt APK, you must then build OpenCV for Android in order to get access to the “tracking” module. The high level overview of this process is as follows:Clone the OpenCV project and the OpenCV extra modules repository
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git
From here, you build OpenCV4Android using the supplied build script. This can be found in opencv/platforms/android. For this project, Android SDK Build-tools 27.0.3 and Android API Level 27 were used. For this step, make sure you have the Ninja build system and the Apache Ant command line tool installed. An example of running the build script is as follows:
export ANDROID_NATIVE_API_LEVEL=27
export ANDROID_PROJECTS_BUILD_TYPE=ANT
./build_sdk.py \
--sdk_path YOUR_SDK_LOCATION \
--ndk_path YOUR_NDK_LOCATION \
--extra_modules_path YOUR_OPENCV_CONTRIB_LOCATION/modules \
--config ndk-16.config.py YOUR_OPENCV_LOCATION/build_android \
YOUR_OPENCV_LOCATION
Adding OpenCV4Android to the project
After the OpenCV Android build is complete, we can now import OpenCV as a module in Android Studio.
To import OpenCV4Android as a module, go to
Menu -> "File" -> "New" -> "Module" -> "Import Gradle project"Then, add the dependency into the application module by going to
"Open Module Settings" (F4) -> "Dependencies"Building complete project
After syncing has completed, build the project and deploy to one of the devices listed above.
Advanced Instructions
This repository contains the required Tensorflow Checkpoints and SNPE DLC files required to run the app. However, if you wish to train your own model, please refer to the instructions located in the Tensorflow Models Object Detection API repository.
We trained a MobileNetV1 SSD model on the WIDER dataset. Your dataset of choice must be converted to a TF record format for training. To convert the model using the SNPE converter, you must first freeze your model with the Object Detection export script.
Usage Instructions
When you open FaceBlock, the app will automatically block all faces. Turn off Block All and then tap the screen to make the largest face in the center unblocked. Tap the emoji icon in the UI to select a different emoji. Use the bar underneath the emoji to change the size of all emojis blocking faces. Turn on rotate to block faces correctly when the phone is rotated counter-clockwise. The bar at the bottom sets the threshold on what confidence the app needs before recognizing something as a face.
Sign up for the Developer Newsletter.
Get software and hardware tool resources to help optimize your development delivered to your inbox weekly.
