GStreamer audio capture

Linear: AUT-101

Spawns gst-launch-1.0 with a pipeline that emits normalized F32LE raw audio on stdout, then chunks the byte stream into AudioChunks with contiguous PTS.

Round-tripped through the bundled MP3 fixture

The bundled 34.9-s sample-audio.mp3 (deterministic 440 Hz sine, generated locally via gstreamer's audiotestsrc + lamemp3enc — license-clean by construction) feeds GstreamerAudioCapture::from_file. After decode you get a stream of AudioChunks carrying normalized f32 samples.

First 100 ms of the decoded MP3. The flat lead-in is the MP3 decoder's priming samples (mpeg-layer-3 has a brief encoder delay); the visible sine cycles afterward are the 440 Hz tone the fixture encodes. Rendered as a min/max envelope per pixel column so the high-frequency content reads as a band rather than aliased noise.

Same audio, quantized at 200 ms buckets across the full 34.9 s. Uniform bars — a pure tone has constant amplitude so every bucket has the same peak and rms. RMS sits ≈ 0.707 × peak (the canonical sinusoid identity A/√2).

api

Two source modes

GstreamerAudioCapture::test_source(format, freq_hz)
  audiotestsrc wave=sine freq=F
    ! audioconvert
    ! audioresample
    ! audio/x-raw,format=F32LE,rate=R,channels=C,layout=interleaved
    ! fdsink fd=1

GstreamerAudioCapture::from_file(path, format)
  filesrc location=PATH
    ! decodebin
    ! audioconvert
    ! audioresample
    ! audio/x-raw,format=F32LE,rate=R,channels=C,layout=interleaved
    ! fdsink fd=1

test_source is the AUT-101 deliverable — deterministic, no external file required. from_file is the companion fixture path that decodes a real audio file (the bundled crates/media/tests/fixtures/sample-audio.mp3, a deterministic 35-s 440 Hz sine).

Real fixtures matter for downstream tests: M-MEDIA.8 (audio histogram) needs to assert numeric correctness on real-world signal, not just mock data. The fixture's pure 440 Hz sine has a known RMS (amplitude / √2) so the assertions stay tight.

Lifecycle

GstreamerAudioCapture owns the child process. Drop kills the child and waits — without this, gst-launch-1.0 keeps decoding into a dropped pipe and burns CPU. Matches the decode::GstreamerPipeStream pattern.

#![allow(unused)]
fn main() {
use media::audio::AudioFormat;
use media::gstreamer_audio::GstreamerAudioCapture;

let fmt = AudioFormat::mono_f32(48_000);
let mut cap = GstreamerAudioCapture::test_source(fmt, 440.0)?;

// Read 100 ms chunks for 1 second.
for _ in 0..10 {
    let chunk = cap.next_chunk(4_800)?;
    println!(
        "pts={:.3}s rms={:.3} peak={:.3}",
        chunk.pts().as_seconds(),
        chunk.rms(),
        chunk.peak(),
    );
}
Ok::<(), media::gstreamer_audio::Error>(())
}

Format support

Only SampleFormat::F32 is supported at construction — the pipeline caps it to F32LE explicitly and converting integer formats at this seam would push sample-format complexity into the public API for no payoff. [crate::audio::SampleFormat::I16] / U8 exist for future capture backends that need to declare an upstream layout, but the public from_file / test_source constructors require F32. A non-F32 format returns Error::UnsupportedFormat.

Integration tests

4 tests in crates/media/tests/gstreamer_audio_integration.rs, each skip-guarded via media::gstreamer::is_available:

Manual regression

just gate                           # runs the integration tests via nextest
                                    # (skips silently if GStreamer absent)
cargo run -p media --example gst_audio_dump  # planned M-MEDIA.11 follow-up