Detect Jenga blocks in real-time: a hands-on guide with Edge Impulse and TensorFlow Lite

Step 1: Set Up Your Project in Edge Impulse

Head to edgeimpulse.com and create a new project. Name it something like:

Jenga_Ondevice

Choose Object Detection as the project type. This will configure your project to use bounding boxes and image-based training workflows.

Step 2: Collect Diverse, Labelled Data

What You'll Need:

• A camera (phone, webcam, or embedded device)
• A physical Jenga tower (or similar blocks)
• Edge Impulse mobile app or web uploader

Collecting Data:

Use Edge Impulse’s Data Acquisition tab or mobile app:

• Capture images in varied lighting conditions (natural light, indoor lighting, shadows)

• Vary distance: close-ups and far shots
• Vary angles: side views, top-down, diagonals
  
  
• Include 3 key states:
  • Stable: Block in place
  • Removed: Block visibly pulled out
  • Empty: No block in the space

Tip: Capture at least 100 images per class to start.

Step 3: Label with Bounding Boxes

Use the Labeling tool in Edge Impulse:

• Draw tight bounding boxes around visible Jenga blocks
• Label them according to their state: e.g. "stable", "removed", "empty"

Label consistency is crucial. Bounding box quality directly affects detection accuracy. If possible, zoom in and label small blocks precisely.

Step 4: Design the Impulse with FOMO

Here’s where the Edge Impulse pipeline kicks in.

Add Processing Blocks:

• Image resize: e.g. 96x96 or 160x160
• Color depth: RGB (for visual patterns)

Add Learning Block:

• Select FOMO (Object Detection)

FOMO uses a single-shot detection (SSD)-style head with stride convolution, meaning it outputs a grid of class probabilities and locations in one pass. It's efficient and perfect for low-powered devices.

Train the Model:

• Set training parameters:
  • Epochs: 50–100
  • Learning rate: 0.001 (start here)
  • Validation split: 20–30%
• Use data augmentation: flip, rotate, brightness changes

Monitor:

• Loss curve
• mAP (mean Average Precision)
• Confusion matrix

You’re aiming for balanced precision across classes.

Step 5: Export as TensorFlow Lite C++ Library

Once satisfied with performance:

1. Go to the Deployment tab
2. Choose TensorFlow Lite (C++)
3. Click Build and download the archive

Inside the zip:

• .tflite model file
• C++ header + source wrappers
• Build scripts to integrate with embedded or Android systems

Step 6: Integrate the Model with the Project

• Extract the downloaded C++ library.
  
  
• Copy only the below extracted files into the example-android-inferencing/example_static_buffer/app/src/main/cpp directory, dont copy the CMake.txt file.
  
  edge-impulse-sdk/
  model-parameters/
  tflite-model/

Cloning the Base Repository

We created an example repository that contains an Android Studio project with C++ support. Clone or download this repository:

Run the Windows / Linux / OSX script to fetch resources

Import the Project to Android Studio

Choose the project to import

• Android

1. Open Android Studio.
2. Select Open an existing Android Studio project.
3. Navigate to the cloned repository and select it.

Download CPP Project from Edge Impulse

1. Go to Edge Impulse Studio.
2. Export your trained model as a C++ library.
3. Download the exported model.

Integrate the Model with the Project

1. Extract the downloaded C++ library.

2. Copy the extracted files into the example-android-inferencing/example_static_buffer/app/src/main/cpp directory, dont copy the CMake.txt file.

Paste in the Test Feature Set for the CPP Test

1. Obtain the test feature set from Edge Impulse Studio test impulse tab.
2. Paste the test feature set into the raw_features array the native_lib.cpp licated in the cpp directory.

Build and Run the Project

1. In Android Studio, click on Build > Make Project.
2. Once the build is successful, run the project on an Android device or emulator.

Step 7: Test, Iterate, Improve

You’ll quickly spot challenges:

• False positives under poor lighting
• Occlusion when hands block view
• Confusing “Removed” vs. “Empty”

Fixes:

• Add more training data for edge cases
• Retrain with weighted class loss
• Fine-tune model hyperparameters

Final Thoughts — Building Something Resilient

What you’re doing here is building a full pipeline:

Image capture → Labeling → Training → Export → On‑device inference

Edge Impulse + FOMO + TFLite makes this pipeline fast and manageable. And once you’re past the initial training & setup, you can focus your energy on refining, optimizing, and making the detection system robust to real‑world variation.

Now: go build something awesome. Maybe a smart camera that warns when the tower is unstable. Or a game scoreboard that auto‑reports which block was pulled. The possibilities are block‑by‑block endless.

You’ve created a real-time, on-device system capable of understanding the state of a Jenga tower — block by block.

This workflow can be adapted for:

• Board game automation
• Warehouse item tracking
• Embedded visual inspection
  
  

Have you tested the guide and want to share how it’s working? Join our Developer Discord and share your project with us for the chance to be highlighted!

Additional Resources

• Edge Impulse Documentation
• FOMO Model Architecture
• TensorFlow Lite Android Guide