ANDRADE-MUÑOZSAN FRANCISCO --:--:-- · overcast · 13°

/projects/dj-agent — n°005AUDIO · MLLIVE

asleep23 ingests · 4 judges · 199 edits building · 28 checkson the ops board →report filed 07 jul · 06:52

DJ Agent

Q:Can we leverage agentic intelligence to curate an authentic vibe?

A personal DJ that turns my real library into a vibe database, learns my taste from how I annotate tracks, and plans beatmatched sets a deterministic critic must pass before any audio renders.

On the bench since

JUN 2026

Last update

JUN 2026

Status

LIVE

Stage

Bench — raw, still moving

representations
512-d clap (track + section) · 384-d taste · bpm / camelot / lufs columns
agents
2 llm (architect, selector) — everything else deterministic
critic gate
≥70% harmonic · ≤6 bpm jump — set per genre · ≤4 lufs arc-rmse · 0 artist repeats — always
state
allin1 segments for real · genre-aware sets · 256 fast tests

Spotify DJ is a recommender in a costume — it legally can't touch audio, and it optimizes for everyone. This bench goes the other way: personal use, my actual files, my taste as the center of gravity, and real beatmatched renders that need the raw decodable audio streaming services can never give an algorithm.

Here's the whole idea in one run — watch the booth think:

The booth, thinking

brief “two hours, rooftop, sunset into night — start mellow, peak late, land soft” → detect(): first keyword hit, “sunset” → downtempo profile → architect
── target arc · - - proposed set
on air

7A

104

8A

105

8B

106

8B

107

9B

108

3A

109

9A

110

10A

112

10B

111

10B

110

11B

108

12B

105

12 slots — each is a section of a track, not the whole file · ◆ splice: legal key move · clash

critic — downtempo profile

harmonic82% ≥ 70%
max bpm jump3.0 ≤ 6.0
arc rmse4.2 ≤ 4.0 lufs
artist repeats0 = 0

approval — human

Approve this set for rendering? [y/N] y

xml, m3u8 and set sheet are written on the yes — only the render costs compute: stretch dst÷src, equal-power crossfade, bass swapped at 180 Hz, overlap = half the outgoing section, quantized to a power-of-two bar count → one .wav

violet = the model imagining · outline = the model has no say — code or a human decides · pulse = the set, playing

The rules here are real and running: the Camelot legality test, the four thresholds, the arc interpolation and RMSE math execute on this page exactly as coded in the repo. Profile selection is a keyword table, not a model — detect() takes the brief's first keyword hit (“sunset” sits in downtempo's list) and --genre overrides it. Only the jump and harmonic gates are genre-variable: downtempo's equal the open defaults shown, hip-hop loosens them to 12.0 / 40%, house tightens to 4.0 / 80%; arc-RMSE and artist repeats are critic-wide constants. The set itself is representative, built to exercise those rules.

2026.06.11

allin1 runs for real; the critic learns genre

Two ADRs landed in one day, and both close gaps between what the architecture claimed and what actually executed:

  • allin1, finally (ADR 0012). The "allin1 → librosa fallback" chain had a secret: allin1 had never run. Dependency pins blocked the install, and once bridged, its attention layer materialized full T×T score matrices on CPU — 11.3 GB peak for a 30-second clip; a real track OOMed the machine. The fix is two parts: the allin1 CLI now runs as a subprocess from whatever Python can host it, JSON-cached per track, and a vendored memory-linear attention patch does the same math in O(T·k) — verified to float32 epsilon, and it restores the trained positional bias the upstream API migration silently dropped. Same clip now peaks at 2.07 GB, a full track at 5.2 GB. First real output: BPM 133 and 12 sensible sections on a bachata test track. Librosa is finally just the fallback.
  • Genre profiles as data (ADR 0013). The critic's transition gates — BPM jump and harmonic compatibility — stopped being constants. A frozen profile table — nine genres plus an open default that is exactly the old behavior — feeds the architect, selector, critic, mixer and set sheet from one place. The same set with a 7-BPM jump and one key clash now passes as hip-hop and fails as house; that exact case is a unit test. Slots grew into entry → core → exit play spans sized to the genre's airtime window, crossfades quantize down to powers of two capped per genre, and approval writes a third artifact: a printable set sheet with per-transition bar counts and blend-style notes.

The fast suite went from 227 to 256 tests, still green offline.

2026.06.10

Rescope: the pipeline runs end-to-end

The clap-first-rescope branch landed — the live pipeline now runs brief-to-render without a gap, and the test suite grew from 103 to 227 fast tests, all green without a database, a model, or audio on disk. Three ADRs went code-complete with it:

  • Link ingestion (ADR 0009). Paste a Spotify or YouTube URL and the curator resolves metadata, pulls the audio via yt-dlp to FLAC, and feeds it the same idempotent ingest path as a local file. A bulk judge (dj.taste.judge <url>) scores a whole playlist against my taste vectors without downloading anything.
  • Two-form export (ADR 0010). Every approved plan exports a rekordbox.xml for manual decks and an .m3u8 always; --render still produces the automatic .wav. The plan is the product — the render is one of its outputs.
  • Grid truth (ADR 0011). Real downbeat phase estimation, section bounds snapped to the beat grid, and first_downbeat_s exported as the TEMPO anchor — so what rekordbox shows matches what the mixer assumed.

2026.06.05

Code-complete, audio-pending

A full review pass landed across phases 0–5: 103 fast tests green, every deterministic module covered. The best catch was real: librosa's beat tracker folds double-tempo genres — DnB, footwork — down to half-tempo, and a naive octave correction elsewhere was quietly un-folding them back. Fixed before any real audio ever got graded by it.

Next gate is contact with reality: the first full ingest of my actual library, where the allin1 → librosa fallback chain finds out which segmentation backend actually installs and which genres break the beat grid.

2026.06.04

The selector earns its loop

The Critic is the interesting half: harmonic compatibility, BPM continuity, energy-arc RMSE, artist spacing — pure math, reused unchanged as the eval scorecard. The Selector revises against its notes until pass or budget (three revisions, then best-so-far wins). The console at the top of this bench plays that exact loop. With no API key the whole pipeline degrades to a deterministic arc and greedy nearest-fit ordering, which is also the A/B baseline the agents have to beat.

2026.06.03

Capture taste before you own the file

The strongest taste signal happens while listening — usually on Spotify, where there's no file to tag. So capture decouples from ingest: a review left in the moment ("Bicep — Glue: dreamy 3am closer, 5") parks in a pending_taste table and auto-applies when the Curator ingests a confident match — ISRC, or name plus duration. Anything ambiguous surfaces for a one-line manual resolve instead of guessing.

That last part is the design: a wrong auto-match would poison the taste layer silently, and the taste layer is the only thing here that can't be recomputed from the audio.

2026.06.01

Bench assembled — two vectors, one row

"Sounds similar" and "is my kind of track" are different questions, so every track carries both: a 512-d CLAP acoustic vector and a 384-d taste vector embedded from my own notes — separate columns on the same Postgres row, blendable per query. "Surprise me" leans acoustic; "my favorites" leans taste. Sections are first-class rows with their own vectors, because a set doesn't mix tracks, it mixes the right parts of tracks on musical boundaries.

Open questions

  1. Q-01Does taste propagation hold at ~150 hand tags, or does it quietly need 2,000?
  2. Q-02Genre detection is first-keyword matching — good enough for one-genre briefs, but what should ‘melodic techno into trance’ do?
  3. Q-03Phase-locked downbeats at the splice: seams currently sit within ~1.5 BPM of each other — is that audible on a real floor or only in a DAW?
  4. Q-04Plan approval is the acceptance metric — does approving a set as data actually predict liking it as audio?

The thinking that runs underneath this bench

Four Agents, One Nervous System