Ibrahim Salman
Send a brief
Ibrahim Salman
Send a brief
All projects

Quantum Singularity

A WebGL2 particle simulation you control with your hands and your voice.

Live demo ↗Code on GitHub ↗
Status
Live
Built with
Three.js, WebGL2, MediaPipe, Web Audio API, Playwright
Quantum Singularity's black-hole topology: a disc of glowing particles with a jet through its centre
The Singularity topology, rendered live in the browser.

What this demonstrates for your project

Send a brief →Discuss interactive engineering →
Problem
Dense browser particle simulations with multi-modal inputs struggle with frame drops, high CPU-GPU memory transfers, and mobile stability.
What was built
A real-time WebGL2 GPU particle simulation in a single static HTML file (~1,600 lines) driven by MediaPipe hand gestures and Web Audio API.
What it proves
Up to 110,000 GPU particles derived procedurally in vertex shaders without per-frame CPU position uploads; automated Playwright test suites.
Status & context
Live demo running in browser; public repository on GitHub. Client-side execution with zero camera/audio data leaving the device.

Overview

What it is

Up to 110,000 GPU particles arranged into six mathematical topologies, controlled by three fused, real-time input channels: pointer, hand gestures, and live sound.

Point to attract the field, open your palm to control camera distance, or just talk near the mic and watch bass and treble reshape the simulation, all running on-device through MediaPipe hand tracking and the Web Audio API. Camera and microphone are strictly opt-in and neither stream ever leaves the browser. Try the live demo, press B for engineering notes or P for a live GPU-time readout.

What I built

Key decision: No particle position is ever computed on the CPU. Every particle's location is derived procedurally inside the vertex shader from its index and a topology function each frame, avoiding per-frame CPU uploads of a 1.3 MB position buffer.

Gesture recognition fuses a trained neural classifier with a hand-written geometric classifier through a quality-weighted rule, then runs the result through a temporal state machine so a gesture can't flicker on or fail to release from a single bad frame. Every easing and smoothing path (camera distance, gesture hysteresis, audio decay) runs through a frame-rate-independent exponential approach to keep response timing consistent across refresh rates. Quality (particle count, pixel ratio, ML inference rate) is tiered at boot from device capability, then adjusted live from measured frame time.

Nine seconds of the field running in the browser, recorded from the live site.
The full Quantum Singularity interface: topology buttons and dynamics sliders on the left, the opt-in hand-tracking panel on the right, and the particle field between them
The whole interface: topology and dynamics controls on the left, the opt-in hand-tracking panel on the right.

Impact / Learnings

Built and tested across desktop and mobile, with automated Playwright suites validating particle simulation stability, gesture input fusion, audio analysis, and shader compilation.

This was a deep dive into WebGL2 shaders, real-time performance optimization (target: 60fps at 110k particles), and multi-modal input processing in the browser.

Architecture

Input Fusion

  1. Inputs: MediaPipe Hands from the camera, Web Audio from the mic, keyboard and mouse
  2. Input fusion engine
  3. GPU particle shaders (WebGL2)
  4. Canvas at 60fps (target)

Deployment

Architecture
One static index.html, about 1,600 lines
Libraries
Three.js, MediaPipe hand tracking, Web Audio API
Fallback
WebGL context loss and restore, no GPU timer, no camera or mic, offline
Testing
Playwright suites for gestures, audio, accessibility and shader compilation

Need an interactive, real-time browser experience?

Send a brief →

Related projects