R02 / Multimodal perception

Working CUDA pipeline

Reidentification

Identity across views, voices and sessions.

A live camera-and-audio system combines person, body, face and voice information. Learned profiles reconnect observations to a stored identity rather than relying only on temporary tracker or speaker labels.

Computer visionVoice fingerprintsIdentity fusionCUDA

01 / The interaction

How can recognition survive a new session?

The system keeps learned face and voice profiles, names and their associations across sessions. A new speaker label can change after a restart; matching stored voice features reconnects it with the existing identity.

Cross-session recognition
01Observe
02Match profiles
03Fuse identity

Temporary tracking labels and persistent identities are separate concepts.

02 / Research & engineering

What went into it.

Visual recognition

Person detection and body/face recognition supply visual identity cues from the camera stream.

Voice profiles

Diarization and voice-fingerprint extraction support speaker recognition and stored profile matching.

Identity fusion

Fusion logic combines cues and maintains associations between voice profiles and people.

GPU execution

Six neural-model components run on CUDA. Capture, mouth-motion processing, fusion and the UI remain on CPU.

Compatibility

CPU/CUDA feature comparisons and saved-name recovery checks helped preserve existing profiles during the migration.

03 / Progress

Built. Tested. Next.

Implemented

Six GPU model components

The live application reported all six neural-model components using CUDA.

Checked

Stored-profile compatibility

Existing names and associations were preserved and recovered after the inference migration.

Measured

81 → 56 ms per frame

A recorded local camera benchmark used about 31% less processing time. This describes that test setup, not general throughput.

Verified

75 software tests

The migration included software checks plus CPU/CUDA feature-compatibility tests.

The recorded benchmark evaluates processing time on the tested setup. It does not establish recognition accuracy across populations or operating conditions.

Continue exploring

Portable brain interfaces