Neuroveil: Twin Mind Interface

Neuroveil synchronizes brainwaves between participants, allowing them to converge to a shared brain state, and control that brain state to elicit certain effects. Brainwaves are measured via the OpenBCI electrode cap and Cyton receivers and sent to an ESP32 module. Frequencies are selected from the input data and outputted to photo biomodulation glasses and a cymatics generator that visualizes the synchronization state. The data from the ESP32 is also sent to a Qualcomm® RB3 Gen 2 development kit to perform inference on a user’s brain data.

Objective

Our inspiration originates with the Dream Machine from the 60s, a simple flickering light causing profound visual effects, and the plethora of research after it showing health improvements, neuron regeneration, and treatments of mental disease.

Tools

Source Code 

Additional Resources

Build / Assembly Instructions

Windows Host PC

1. Install Docker onto Windows host PC
2. Clone the GitHub repo to host PC
3. Install Python packages that are in the project directory - Synchroflow/python/ai-model-dev

Qualcomm RB3 Gen 2

1. Connect host PC to Qualcomm RB3 Gen2 following the instructions at this link
   1. Setup SSH either over Ethernet or WiFi with PuTTY
2. SCP the following files into the same directory on the Qualcomm RB3 Gen2:
   1. Synchroflow/qualcomm/Docker
   2. Sychroflow/python/demo/.

OpenBCI Cyton

1. Follow OpenBCI documentation to setup connection to PC and stream real-time data: https://github.com/openbci-archive/Docs/blob/master/Tutorials/01-Cyton_Getting%20Started_Guide.md
2. Run the command - python3 python/brainflow/brain_serve_delta.py from PC terminal while on 2.4GHz network

ESP-32 (Photobiomodulation glasses)

1. Install Arduino IDE
2. Upload Arduino/delta_power_wifi.ino to ESP32 on LED glasses
3. Repeat this setup on a separate host IP and port for each LED glasses
4. Trade glasses so each user gets each other's brain data

ESP-32 (Speaker)

1. Upload Arduino/speaker_wifi.py to ESP32 connected to speaker

Project Walkthrough

OpenBCI cap uses Cyton microcontroller to stream raw EEG readings to local web server using WiFi and web sockets

1. ESP32s connected to LED glasses and cymatic speaker receive data via the websocket using the delta_power_wifi.ino file. which does:
   1. converts power values to frequency
   2. informs glasses to flash at the determined frequency
   3. outputs the determined frequency to the speaker
2. New user data is saved to the host PC, and then sent to the Qualcomm RB3 Gen 2 for inference
   1. Data was collected by having each team member wear the headset for 5 minutes, and labelling each streaming session by the team member ID (integer 1-5)
   2. Since there are 5 team members, there is a label for each of the five groups respectively
   3. Transformer model was then trained on this data
   4. Test data, generated from a different user wearing the headset, will be analyzed by the transformer model and will infer which team member’s brain data the user’s brain data is most similar to.

Usage Instructions

Host PC

1. Streamed data from OpenBCI hardware will be saved locally to Host PC
2. Prepare raw data for model by inputting it into Sychroflow/python/ai-model-dev/preprocessdata2.py
3. SCP processed user data to Qualcomm RB3 Gen 2 using PuTTY

Qualcomm RB3 Gen 2

1. Navigate to directory with all of the project files
2. Build Docker container:
   1. docker build -t docker1 .
   2. Successful output:

3. Run Docker container:
   1. docker run -it docker1
4. The following will then happen:
   1. Docker will install necessary python packages
   2. testmodel3.py will load in transformer_model.pth and feed it the data from labeled_data.csv
   3. The loaded model from transformer_model will run inference on the inputted data and output a string result to the command line.
   4. Successful output: