Crate media 

media — GStreamer-backed audio + video capture, playback orchestration, and the data models that wisp (renderer) and app (Tauri/Leptos shell) consume.

Three-way split

Boundaries are load-bearing. Crossing them once would bloat every wisp consumer (storybook, headless export, future plugins) with GStreamer’s build + license footprint.

• media (this crate) owns: GStreamer capture (audio + video), GStreamer playback, audio/video data models (audio::AudioChunk, decode::VideoFrame re-exported under video), the clock::MediaClock + clock::MediaTime timing model, histogram::AudioHistogram quantization, the manifest::RecordingManifest session descriptor (planned — see manifest for the scaffolded module), and the device enumeration model.
• wisp owns visual composition only. It must not depend on this crate’s GStreamer integration. It receives VideoFrame / Texture handles, waveform-bar geometry, cursor state, and timeline timestamps via typed structs — and renders them through its sprite + graphics pipelines.
• app (Tauri + Leptos) owns UI orchestration. It calls this crate’s Leptos/Tauri integration seam and the wisp renderer; it does not poke GStreamer or wgpu directly.

Layering

 ┌──────────────────────────────────────────────┐
 │ app  (Tauri shell + Leptos UI)              │
 │   - calls `media::commands::*`              │
 │   - feeds `wisp` typed data                 │
 └────────────┬──────────────────┬──────────────┘
              │                  │
    ┌─────────▼────────┐   ┌─────▼────────────┐
    │   media (this)   │   │     wisp         │
    │  - GStreamer     │   │  - render        │
    │  - timing model  │   │  - sprite/graphics│
    │  - histogram     │   │  - text + mask   │
    │  - manifest      │   │  - filter + blend│
    └────────┬─────────┘   └──────────────────┘
             │      typed data (VideoFrame,
             │      WaveformBarRect, MediaTime, …)
             ▼
        (no direct dep on wisp)

Build-on-decode

decode::VideoFrame + decode::VideoStream already carry the BGRA-frame contract used throughout the project. This crate re-exports them under video rather than reinventing the wheel. M-MEDIA.6 (gstreamer::video_test_source) builds on decode’s GstreamerPipeStream.

GStreamer integration choice

CLI-pipe pattern — spawn gst-launch-1.0 as a child process and pipe raw bytes through fdsrc / fdsink. Documented in gstreamer. No compile-time libgstreamer dependency.

Leptos/Tauri seam

M-MEDIA.21 (AUT-117) lands the command/service surface at media::commands::* — list_devices, start_capture, stop_capture, current_clock, latest_histogram_window, latest_video_handle. Until then, this module is a placeholder.

Track status

Each M-MEDIA chunk lands as its own commit on m-media branch. Sub-modules fill in chunk-by-chunk:

Module	Chunk	Status
gstreamer	M-MEDIA.1 (AUT-97)	scaffolded
clock	M-MEDIA.2 (AUT-98)	scaffolded
audio	M-MEDIA.3 (AUT-99)	scaffolded
video	(re-export)	scaffolded
histogram	M-MEDIA.8 (AUT-104)	scaffolded
manifest	M-MEDIA.20 (AUT-116)	scaffolded

Re-exports

pub use audio::AudioChunk;

pub use audio::AudioChunkError;

pub use audio::AudioFormat;

pub use audio::SampleFormat;

pub use camera::CameraDevice;

pub use camera::list_cameras;

pub use clock::MediaClock;

pub use clock::MediaDuration;

pub use clock::MediaTime;

pub use clock::Timestamped;

pub use histogram::AudioBar;

pub use histogram::AudioHistogram;

pub use microphone::MicrophoneDevice;

pub use microphone::list_microphones;

pub use mock_audio::SilenceSource;

pub use mock_audio::SineWaveSource;

pub use mock_audio::StepPulseSource;

pub use waveform::BarMetric;

pub use waveform::WaveformBarRect;

pub use waveform::WaveformDisplayMode;

pub use waveform::WaveformLayout;

pub use waveform::mono_bars;

pub use waveform::stereo_bars;

Modules

audio

Audio sample + chunk data model (M-MEDIA.3 / AUT-99).

audio_mix

AudioMixer — sum two mono/stereo F32 input streams into a single interleaved F32LE output (M-EXPORT.2 of M-RECORD-EXPORT).

camera

Camera device enumeration (M-CAM.1 / AUT-255) — CLI-pipe pattern.

clock

Shared timeline — MediaTime, MediaDuration, MediaClock, Timestamped<T> (M-MEDIA.2 / AUT-98).

encode

VideoEncoder trait + OutputFormat enum + per-OS GStreamer pipeline builders (M-EXPORT.1 of M-RECORD-EXPORT; live variant M-QUAL.1).

gstreamer

Shared GStreamer probe — structured diagnostic for capture/playback preconditions (M-MEDIA.1 / AUT-97).

gstreamer_audio

GStreamer audio capture (M-MEDIA.5 / AUT-101) — CLI-pipe pattern.

gstreamer_video

GStreamer video capture (M-MEDIA.6 / AUT-102) — CLI-pipe pattern.

histogram

Audio histogram quantization (M-MEDIA.8 / AUT-104).

manifest

Recording session manifest.

microphone

Microphone device enumeration (M-MIC.0 / AUT-277) — CLI-pipe pattern.

mock_audio

Deterministic mock audio sources (M-MEDIA.4 / AUT-100).

sync

Audio/video sync harness (M-MEDIA.7 / AUT-103).

video

Video frame surface — re-exports decode’s VideoFrame + VideoStream so M-MEDIA consumers don’t pull decode directly.

waveform

Waveform bar geometry (M-MEDIA.9 / AUT-105).

Structs

VideoFrame

One decoded frame in BGRA8 layout, the canonical wgpu input format for per-frame texture uploads.

Traits

VideoStream

Pull-based source of decoded video frames.