Personal project / Prototype

Hand-Controlled Virtual Lab

Exploring hand-controlled interaction in a virtual laboratory using an OAK-D camera, TensorFlow, and Unity.

OAK-DTensorFlowUnityPythonTCP

Prototype recordings

See the interaction

Hand-position controlThe camera feed and Unity scene shown together during the hand-control experiment. Silent recording.

Virtual lab environmentA walkthrough of the prototype laboratory interface and simulated equipment. Silent recording.

Original recordings from the project repository.

The idea

Make simulated lab equipment accessible through visual interaction. The prototype explored basic hand-position control as a first step toward a gesture-controlled laboratory.

How it works

The documented workflow trains a hand detector, converts it for OAK-D inference, and sends detections to Unity over TCP. The repository includes recordings of the prototype in action.

01Capture camera frames
02Detect hand position
03Send detections over TCP
04Update the Unity scene

Project scope

This is an exploratory prototype with basic hand control. Gesture reliability and richer interaction remain areas for improvement.

← Back to projects