dejvino/music-video-gen
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: dejvino:8dfd8392f3c8572f4967fd73187494cde0bda32b
Branches Tags
dejvino:main
..
compare: dejvino:ef1d24e692742b6044fed03c8b8ff6e6c2ee9a9d
Branches Tags
dejvino:main

57 Commits
8dfd8392f3 ... ef1d24e692

Author SHA1 Message Date
Dejvino
ef1d24e692 Director cuts between palettes; the look panel shows which one is live 
One palette for a whole song reads as one colour once five minutes
have passed. The track now keeps 2-4 audio-tilted palettes (forced
scheme/hue diversity so switches are legible) and the director owns
how they are traversed per cue — sequential/pong/kindLed/storyLed/
contrast — with cut or OKLCH blend timed to the cue crossfade. The
fine OKLCH paletteArc drift still rides on top of whichever base is
active.

The look tab renders the full set as a list; the row matching
ArcDriver.paletteIndexAt(frame) gets an active border/glow and
follows playback/scrub live, with a blend badge while two palettes
interpolate.

Co-Authored-By: internal-model
 2026-08-20 12:45:42 +02:00
Dejvino
df609424d5 Synthwave verges stay off the road; camera stays level unless the tail says otherwise 
Synthwave Run: roadside passers stream on the verges, not on the road.
Each passer places at roadHalf + psize*0.92 + gap where roadHalf mirrors
the grid's abs(p.x*perspective) < 0.8, so even a large chorus form's
edge stays clear of the centre lane reserved for the protagonist.
Lateral wobble reduced to worst ~0.025 inward vs. minimum 0.07 gap so
it can never carry back onto the road. Four random chorus objects
rushing horizon->camera make each song's traffic its own; on-road
traffic the hero weaves around is left as a separate kind later.
Also carries the previous chorus passer loop, hero castSolid, and
texture 0.4 / form+staging declarations still pending in the working
tree from the last step.

Personality camera: uniform boxes put as many tracks at the wild edge
as near level, so the grid swung too often. Replace with a bell curve
(Box-Muller over the seeded Rng) — mode at level, rarity in the tails.
driftRate/sway/swayRate/spin/breathe now via gaussAround(centre,sigma)
centred near level (spin at 0, sign from the Gaussian itself) with
audio tilting the centre (fast→more drift/sway/spin, dynamic→breathe)
rather than displacing the whole box.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-20 11:25:42 +02:00
Dejvino
c216e31413 Synthwave Run drives the song's protagonist instead of the same car 
The grid, mountains and sun keep their palette-driven forms and are still
individually switchable so the arc driver can strip the scene back to
just the grid for a breakdown, but the car was the same silhouette in
every video — a hexagonal track and a round track drove the same stepped
rect with red discs. The thing on the road is now the song's own
protagonist: the same solid every other stage in the video draws, seen
here as one hero above the grid, turning slowly so its outline changes
across the song rather than being one silhouette from every angle.

One castSolid instance sized by stageScale() so a song of few huge forms
drives a huge hero and a song of many small ones drives a small one,
lit by castLit with an inkStroke rim and a soft ground ellipse for
contact. Gated by the existing u_car param so a breakdown still drops
to just the grid the same way it used to. Declares form, ink, staging
and shape/space/camera/style (texture 0.4); the lint's backtick and
consumes gates stay green.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-20 10:21:26 +02:00
Dejvino
3a0cf26e6c The filmstrip fills in as it renders 
A full build is a couple of minutes of GPU work, and the page spent all of it
showing an empty screen and then everything at once. The interesting failure —
a strip whose frames could be shuffled without anyone noticing — is visible in
the first row, so waiting for the seventeenth to look at the first is a
needlessly slow way to find it out.

Rows are appended as they land, in bank order, and re-sorted once at the end
when there is finally something to sort by. One row renderer handles both raw
pixels, which is what exists mid-build, and the compressed blobs the cache
holds afterwards, so what you watch appear is what you keep.

The song is announced BEFORE its render rather than only after it, too:
synthesising and rendering one is tens of seconds, and a page that says nothing
until the first one lands looks broken for exactly as long as that takes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-20 09:02:10 +02:00
Dejvino
89e05459c0 Every shot stands on something, and the frame has two ends 
A section used to be one scene, and two thirds of the library is composable —
sparse by design, elements ON something. Cast as backgrounds anyway, they left
9 of 40 sampled frames under 20% painted, the darkest at 0.3%: a minute and a
half of a few bright things on black, invisible to every gate because every
gate on the stack was a limit rather than a floor.

Every section now stands on a GROUND: a canvas that fills the frame, cast per
section kind so a shot cut changes the shot and not the world. When the shot
fills the frame itself it IS the ground — two canvases stacked is two pictures
fighting. Above that, a coverage BUDGET: director appetite times the section's
energy times where the story is, capped at two frames' worth of material.

The measured facts move into the repo. scenes/metadata.json is generated from
the gallery — coverage as a shot, coverage as a bed, variety, the structural
profile — tracked in git, stamped with a fingerprint of the scenes and the
metric definitions, and refreshed from gallery.html. `surface` is derived from
it rather than declared; nine scenes claimed `canvas` while painting under a
third of the frame, and declaring it is now a lint error. The generator weights
every layering choice by measured structural distance, because family labels
and the render disagree: two `geometric` scenes can be 0.31 apart and a `flow`
and an `organic` scene 0.04.

The gallery's 0.1 red line is gone. It was right when a section was one scene
and wrong now — nineteen scenes were failing a bar for being consistent, which
is a virtue in an ingredient.

Chasing the numbers turned up four real faults:

  * A scene that reads prev() cannot be a ground. It returns the whole
    composited frame including the layers above it, so a datamosh under a shot
    is eating it: the render stopped reproducing from a seek and two WebGL
    contexts diverged by 91/255 against a tolerance of 4.
  * Screen was the wrong operator for a shot over a bed. It lightens, so a
    median quarter of every frame clipped to paper and whole sections rendered
    100% white. Replaced by a lumakey — the shot's brightness is its alpha.
  * Feedback was an accumulator: a still image settled at 2.3x its own
    brightness. Fine over black, fatal over a filled ground. Normalised at 0.6,
    plus a highlight shoulder so the top rolls off instead of clipping.
  * useTrack never prewarmed, so a fresh Show's first frame differed from every
    later render of it — the export-breaking hazard Compositor.prime documents.

Blazing is a decision now, not a side effect: directors declare an appetite for
it, a section must be loud and late in the story to earn one, and quiet kinds
never do. The ceiling gate matches that — a hard cap per section, and no more
than a fifth of them hot at all.

Rendered across twelve videos, middle of every section:

    painted        51% mean, darkest 0.3%  ->  87% mean, darkest 43%
    clipped white  24% median, worst 100%  ->   1% mean, worst 30%
    separation     0.10                    ->  0.44

Seven scenes can ground a section — five geometric, two organic — so every
quiet section of every video stands on one of two beds. That is the library's
largest hole and it is scene work: there is no minimal or flow canvas that
fills half the frame without reading prev().

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-20 09:02:01 +02:00
Dejvino
7d151be6e1 Pylon Grid stands on the chorus instead of on a strut 
The legs were the last thing in this scene it had invented for itself: a
fixed-width vertical capsule, the same object in every video, with only the
palette separating one track's structure from another's. Against migrated crowns
they read as scaffolding left up around a finished building — and structure is
half this frame, so half of it was still saying what the SCENE is rather than
what the song is.

A leg is now a stack of the song's chorus. How many members make up a column is
the song's element size deciding it — a track of few huge forms stands on three,
a track of many small ones on a ladder of nine — and how wide the column is comes
from the chorus's own proportions. Each member turns at its own angle, so a
column is one form seen many ways rather than one form printed nine times. That
makes `staging` a third declared artifact, which the lint caught before the gate
had to.

Measured on the crop that contains the change, since the frame-level score
cannot see it: over six identities the bottom third of the frame moves 13.4/255
with the old strut and 22.2/255 with the chorus stacked. The whole-frame variety
score goes 0.1322 to 0.1348, and the header now explains why the two disagree
and why both are right — the crowns dominate a descriptor averaged over the
whole image.

Cost is unchanged in practice: 7.7ms/frame at 4K, 12.4ms with every param at the
top of its range, against a 60ms ceiling. The column's bounding box is two
comparisons for a pixel outside it, which with a hundred and sixty pylons is
most of the budget.

The crown is resolved before the leg now. Resolved the other way round, the
first-wins rule let the column paint over the object standing on it and every
subject was cut off at the waist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 17:49:29 +02:00
Dejvino
61e0c66f61 Pylon Grid stands the song's object on its pylons 
The scene already had the depth axis the solid wants — near rows large, far rows
small, converging on the track's horizon — which made the flat crown the one
element in it drawn without perspective: a sticker facing the viewer on top of a
receding structure. Each crown is now marched, and each pylon turns its own at
its own angle, so a single field shows the form from a dozen sides at once.

12/12 on the gate, 7ms/frame at 4K against a 60ms ceiling. A hundred and sixty
instances stay affordable on the two guards MIGRATION.md now documents, and here
first-wins is the correct occlusion as well as the cheap one, because the rows
are already walked near to far.

IT MEASURES SLIGHTLY WORSE and the scene's header says so: 0.141 to 0.134 over
8 songs x 8 seeds, nearly all of it layout (0.156 to 0.126). Two candidate
causes were tested and both came back flat — keeping the far rows stamped scored
0.132, and running the lit surface through inkPattern scored 0.132 here and
moved Effigy by 0.0009. It is the solid itself: a body always fills its own
silhouette, where a stamped cast form paints anywhere between an outline and a
disc depending on whether the identity is hollow, and that swing was worth three
hundredths of layout variety.

Kept because the score does not measure what was wrong. A crown that ignores the
perspective of the field it stands in is a defect no coverage variance reports,
and the header records the number and the two-line revert for whoever disagrees.

The inkPattern experiment is reverted rather than left in as a plausible-looking
no-op, with its measurements written where the next person will look for them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 17:24:28 +02:00
Dejvino
0a89676cc8 One scene, wide: songs against seeds 
Six thumbnails is enough to rank sixty-eight scenes and far too few to study
one. Worse, six songs at one seed each confounds the two inputs: a scene whose
frames are interchangeable might be ignoring the identity or ignoring the music,
and the library gallery cannot say which.

So a scene's name in the gallery now opens it on a grid — songs down, seeds
across, up to every song in the bank by ten draws. Buttons pick the size and
walk onto a fresh block of seeds, which is a much better answer to "is it flat
or was that ten unlucky rolls" than staring at the same ten. The state lives in
the URL, so every button is also a back button.

Three scores rather than one, and the two new ones are the diagnosis: ACROSS
SEEDS is the same song with a different draw, so low means the scene ignores the
identity; ACROSS SONGS is the same draw against different music, so low means it
ignores the song. They point at different fixes. The closest pair is outlined,
because at 170 cells no eye is finding it.

The first version hashed the song name into the seed so a column would not be
one roll repeated down the grid. It reads better and measures nothing: with the
seed varying on both axes the two scores are the same comparison, and they came
back within 0.001 of each other for every scene tried. A column holds its seed
fixed now, which is what makes it attributable to the music — and the scores
separate, with the seed moving every scene tried more than the song does.

170 cells in 9s. The songs are the slow part and are analysed one at a time so
the count moves, then cached for the session.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 17:16:15 +02:00
Dejvino
89277705c4 Nothing is held for two minutes, not even one visual 
planShots skipped cut planning entirely when a section had a single visual to
show — "nothing to cut to" — and the exemption swallowed the ceiling with it.
Adding a scene shifted the casting enough to land a 150 BPM track in that case,
and it held one image for 151.98 seconds against a 22-second limit. Any new
scene could have exposed it; the shot-length gate had simply never met a
single-visual section before.

The premise was wrong rather than the arithmetic. A shot boundary changes the
FRAMING as well as the image, so the same visual filmed again at another size is
a shot, and the ceiling is about how long one image is held rather than about how
many images a section has. Single-visual sections now plan shots like any other.

What the single case still suppresses is the hard cut: cutting straight between
two framings of one image is a jump cut, so those boundaries always dissolve.

Two intermediate versions were wrong in ways worth not repeating: splitting the
section into equal spans put a 23.5s shot past the ceiling once the cuts snapped
to downbeats, and read as a metronome (cv 0.048) to the rhythm check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 17:16:02 +02:00
Dejvino
144c8c918e The song brings a body, not only an outline 
A silhouette is the same picture from every angle, so a scene that turns one is
showing you the same shape rotated. That is the ceiling the cast has been under:
it can be notched and hollowed and it still cannot be walked around.

So the identity generates an ASSEMBLY — two to six parts, each a primitive with
an offset, a scale, a rotation and a boolean op, all under a symmetry. The
symmetry is the load-bearing half: parts unioned at random positions read as
debris, the same parts folded read as designed, and only a designed object is
worth calling a protagonist. Prism parts are the existing 2D profile extruded,
so the solid and the silhouette stay one character rather than two generators
running side by side.

It travels as data like every other artifact: three vec4 rows per part in
u_formPart, plus the scalars. Scenes declare `consumes: ['form']` and get
castSDF3, castMarch, castSolid, castChorusSolid and castLit; with no identity
they fall back to the flat profile extruded, so the helpers are safe to call
unconditionally. The chorus is the same rows with fewer parts and its own
proportions — a relative, not a second generator, and no extra uniforms.

Three scenes carry it. Effigy is new and holds the object still while it turns.
Floating Geometry and Swarm were already loops of stamps and are now loops of
bodies; Swarm is what the chorus solid exists for. That is 34.8% of videos
containing a 3D cast, against 11.9% when only Effigy had it.

Measured, the outline does change rather than merely spin: over one turn the lit
area of Effigy's subject varies 14-113% against Soloist's 3-51% for the same
rotation. Whether that reaches the variety blocks is not yet measured, and
HOWTO-variety says so rather than claiming the win.

Four costs, each found by measuring rather than by reading:

  * every pixel evaluated every instance's field — Swarm at 59ms/frame against a
    60ms ceiling. Bounding-sphere reject first, now 12.3ms.
  * instances overlap several deep at the top of the size range, and marching
    all of them made Floating Geometry's own gate run for minutes. First-wins
    instead of last-wins, which was arbitrary either way.
  * a normal inside the march loop multiplies four copies of the SDF by the step
    count, because GLSL unrolls a fixed bound. Hoisted out.
  * the helpers in the shared preamble made all 68 scenes compile what 3 of them
    call. FORM_PREAMBLE is appended per scene instead.

The lint's backtick check was green through two of my own breakages: quoting a
name in a doc comment adds backticks in PAIRS, so parity survives and the
pair-scanner just re-partitions the file. It now finds where a shader literal
opens and requires the next backtick to be a real terminator — which
immediately found a second stray pair.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 17:15:49 +02:00
Dejvino
518e9ea731 Epic 4: record why direction was negative, and what is left 
The first measurement offered three readings for a direction of -0.14. The
third one — the arc is not reaching the image — was correct, and it is now
+0.24 against an arcless reference of -0.02, with the floor rising exactly as
§8 predicted.

Records what the two dead channels were, since neither was visible from the
source and both passed every gate that existed: the overlay path had an empty
candidate set (0/144 stacks), and the camera moved the frame by a median of
0.029 of a half-frame at a random angle so successive shots cancelled. Both
cleared their checks because every bound on them was a ceiling. A gate on a
device needs a floor.

Also ranks what remains, with the broken seed-variety instrument first: three
of the four open items are measurements and they all report through it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 06:58:27 +02:00
Dejvino
1ca9fa2f1f Let scenes layer on each other, and give the camera somewhere to look 
Two devices that were built, wired up, and doing nothing.

OVERLAYS. buildStack gated layering on `surfaceOf(m) === 'composable'`, and
no scene in the library declared `surface` — so the only thing that could ever
sit on top was the one scene declaring `role: 'accent'`, and the overlay roster
excluded accents by construction. Empty intersection, every time: 0/144 stacks
carried an overlay. Every layered frame in every song was the same particle
field.

Labelled the library from the phase 12 coverage gate rather than by eye: the
37 scenes painting under 30% of the frame are composable. Particle Field loses
its privileged slot and becomes one of them, keeping only `background: false`,
which is the honest part — points in empty space cannot carry a section alone.
The reserved accent slot is gone; one roster, two passes at it. 1 distinct
overlay scene becomes 27, and the rate lands at 34% of stacks after trimming a
base chance that had been tuned while the branch was dead.

THE CAMERA. framing.shift moved the frame by a median of 0.029 of a half-frame
at a fresh random angle every shot, so successive offsets cancelled and the
median jump at a cut was 0.014. Present in every frame, visible in none.

look/Camera.js is the director's camera department: the story says tension,
order and which act a section is in, and this turns that into where the frame
looks and how it travels there. Each director names a camera. Jump distance
follows tension and act, speed follows energy, and the curve is one of four.
Cuts choose between reframing and matching, so two scenes can still read as one
place. Median offset 0.190, median reframe 0.135, and 94% of shots now move
during the shot rather than only at the cut.

Four new gates, each with a FLOOR — the device this replaces passed every
existing check while doing nothing, because the only bound was a ceiling.

Also lands the in-progress Epic 4 story layer it builds on: Story.js, phase 13,
and the direction statistic.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 06:48:13 +02:00
Dejvino
eda8ac3810 Composable vs canvas: say which scenes are meant to sit on top of others 
The top of the gallery is full of black frames with a bright element on them.
That is genuinely varied and thin to watch, and the reason it ranks is that the
gallery previews every scene ALONE — which is not how anyone watches one.

Measured, the split is not marginal: 37 of 66 scenes paint under 30% of the
frame, and the sparsest are exactly the ones topping the ranking. Storm Rift and
Pendulum Trace paint 0%, Droste Feedback and Spectrum Sculpture 2%. At the other
end Block Mosh covers 98% and Scan Tear 96%.

So a scene now declares a `surface`. A canvas fills the frame and belongs
underneath; two of them stacked is two pictures fighting rather than one picture
with depth. A composable scene is mostly empty by design and reads as elements
ON something. The distinction was implicit in `role: 'accent'`, which marked the
depth passes and said nothing about the sixty scenes treated as interchangeable
backgrounds while half of them were sparse.

Verified rather than trusted, in the pattern the rest of the project uses: a
gate renders every scene and measures what it actually paints, failing a
composable that covers more than 55% — it would hide whatever it sits on — and a
canvas that covers less than 8%, which is a canvas in name only. The gate also
prints the whole coverage table, since that list is how the library gets
labelled in the first place.

Casting uses it: only composable scenes are drawn as overlays, and because what
goes on top is now guaranteed to leave the shot underneath visible, layering
happens far more often — the overlay chance went from 0.12 to 0.3 at its base.

The gallery shows coverage and surface beside each score, sorts by sparsest, and
says in the header why the two numbers explain each other.

Nothing is labelled `composable` yet. Every scene keeps the behaviour it had
through `surfaceOf`, which defaults to canvas unless the scene is an accent, so
this commit changes the mechanism and not the output. The coverage table is what
the labelling should be read off.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 18:20:24 +02:00
Dejvino
c099501ead Raise the variety bar to 0.1 
Deliberately a target rather than a description of where the library sits. Of
the scenes measured since the region block went in, only Droste Feedback at
0.324 is comfortably over; Plasma Bloom is 0.082, Voronoi Shatter and Apollonian
Gasket 0.062, Moiré Grid 0.047. Most of the library fails this, which is the
point — a bar set where the work already is measures nothing.

Third value in two days, and both previous moves were instrument corrections
rather than changes of mind: the motion block was reading zero for the entire
library, and adding the region block made the total a mean over six blocks
instead of five. Both raised every score, so 0.04 had quietly drifted down
relative to the thing it was judging. Documented as something to re-read off a
fresh gallery whenever the descriptor changes, rather than a number to carry
forward.

The marker now sits at the middle of each meter and the grades land where they
should: 0.047, 0.062 and 0.082 all red, 0.140 amber, 0.324 green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 18:11:33 +02:00
Dejvino
b879abe230 A region block: how differently the parts of a frame behave from each other 
The descriptor could say where a frame's energy sat and not whether one part of
the picture was doing something different from another — and that second thing
is what "nothing to watch" means. A pattern spread evenly over the screen has no
subject precisely because every region is the same region, and `layout`
normalises that away: a uniform field and a field with a form standing in it can
come out at nearly the same distribution.

So each ninth of the frame is now characterised in its own right — how much
detail, which way it runs, how many elements — and expressed as its deviation
from the frame's average. Heterogeneity rather than level, so a brighter or
busier frame does not register as a more varied one.

It is immediately the strongest block in the descriptor, and it credits scenes
the old one was underrating:

    Apollonian Gasket  0.039 -> 0.062   region 0.180
    Plasma Bloom       0.053 -> 0.082   region 0.182
    Moiré Grid         0.037 -> 0.047   region 0.097
    Droste Feedback    0.219 -> 0.324   region 0.446

Apollonian Gasket is the interesting one. It was called out as looking good and
scoring badly, and the answer turns out to be partly that the instrument was
missing the axis it is good on rather than that beauty and variety are simply
different things. Both were true; only one of them was the instrument's fault.

The metric's own gates still hold, which is the condition for believing any of
this: recolour moves structure 0.019 while moving colour 0.723, a quarter turn
moves it 0.0000, and the same scene against itself is 0.0000 against 0.0992 for
two different scenes.

EVERY SCORE HAS MOVED. The total is a mean over six blocks now rather than five,
and the 0.04 bar was derived under the old one. It needs re-deriving from a fresh
gallery before it is used to judge anything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 18:07:11 +02:00
Dejvino
8defef3ed0 Tell a dead shader apart from a boring scene in the gallery 
A failed shader compile renders black, and black frames are identical to each
other, so every block scores 0.000 and the row reads as the least varied scene
in the library rather than as a broken one. That happened yesterday — the
subject helpers were declared above the ink they call, GLSL has no forward
declarations, the whole preamble failed to compile, and the gallery reported a
variety of exactly 0.000 with no indication anything was wrong. It is a
particularly bad failure mode for a tool whose entire job is ranking scenes by
that number.

A scene whose first frame has no luminance and no variance is now reported as
broken, with both figures and a pointer to the console, and the row is coloured
apart from the merely flat ones so it cannot be mistaken for a bad score. The
summary counts them separately.

Verified against both answers rather than only the happy one: a healthy Moiré
Grid comes back 0.0365 with no error, and a deliberately broken copy of the same
scene comes back with the diagnosis instead of a zero.

Recorded in HOWTO-variety.md alongside the other instrument traps, since the
general lesson outlives this instance — a broken render produces the most boring
possible numbers rather than an error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 16:55:16 +02:00
Dejvino
c950f8f3b1 Give a full-frame field a subject the eye can track, and let the song choose 
what it does

"One pattern across the screen" scenes have nothing to watch. Warping the field
around a focal point was the previous attempt and it did not fix that — measured
on Voronoi it raised orientation variety while layout stayed at 0.004, which is
another way of saying the cells changed and the frame still had no subject.

So a subject is now a form that does something TO the field, and which thing is
the song's decision rather than the scene's: shift, warp, punch, morph or
overlay. The same form punching a hole, bending the pattern or running it at
another rate are three different videos, and across six songs the bank picks
overlay, morph, shift, morph, warp and punch — so it is a real axis rather than
a constant with five names.

subjectSDF, subjectMask, subjectEdge and subjectWarp are in the contract, so any
field scene can take a subject without knowing where the focal points are.

Moiré Grid is the pilot and implements all five impacts. 0.031 to 0.037 — up a
fifth, and the layout block moved from essentially nothing to 0.018, which is
the first time anything has moved layout on a full-frame field. That is the
number that speaks to the complaint: the frame's energy is no longer spread
evenly, so there is somewhere to look.

It is still under the 0.04 bar, and I have not seen it. The score says there is
now a subject; whether it is worth watching is the question the gallery cannot
answer.

One bug worth recording: the subject helpers were placed above the ink in the
preamble, and subjectEdge draws with inkStroke. GLSL has no forward
declarations, so the whole preamble failed to compile and every scene using it
rendered pure black — which the gallery reported as a variety of exactly 0.000
across all six blocks rather than as an error. An all-zero row means a dead
shader, not a boring scene.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 16:52:57 +02:00
Dejvino
10eeddcf6d A guidance document for building visualizers that vary 
HOWTO-visualizers.md gets a scene working. Nothing said how to make one look
different from song to song, which is the thing the library is worst at and the
thing every measurement this epic has been about.

Written as evidence rather than advice: each claim carries the number behind it,
so a later change can contradict it. Several already contradict things believed
earlier in the same week.

It records the failures at least as carefully as the wins, because they were more
informative and each looked obviously right beforehand — effects as parameters
inflating a score without changing a picture, concentrating disturbance instead
of adding it, the roster-size theory that a direct sweep found to be nothing, and
reading a form as a metric rather than drawing it. It also documents what the
descriptor cannot see, since half of "why is my score low" is there: brightness,
colour, rotation, quality, and layout for anything that fills the frame.

And it is explicit about which numbers to trust. Direct render comparisons have a
noise floor of zero; aggregate ratios swing enough to have produced four
withdrawn conclusions in one epic.

Ongoing by design, with the open questions listed: coupling has never moved off
zero, fixed-geometry scenes have no known route to variety, and the 0.04 bar was
set against scores that were depressed by a measurement bug.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 16:45:38 +02:00
Dejvino
5a9dfa6b2b A scatter point in Isometric Blocks — which measures worse, twice 
Asked for, built, and honest about the result: the focal point costs this scene
a quarter of its variety and I could not make it pay for itself.

    uniform scatter, no focus     0.0490
    focus, trading scatter for it 0.0374
    focus, added on top of it     0.0365

The first version spent the scatter budget near the focus and calmed the rest,
which put most of the field back on the rigid lattice and took the orientation
variety with it, 0.164 down to 0.105. That diagnosis looked right, so the second
version made the focus additive — every cell keeps its full scatter and the ones
near the point get more. It changed nothing: 0.0365, orientation still 0.104.

So the diagnosis was wrong and I do not have the real one. The remaining suspect
is the height modulation, which lifts blocks toward the focus and may be
flattening the skyline's variety between songs by dominating it — but that is a
guess, and two guesses have already been wrong here.

The feature is committed rather than reverted because the metric does not
measure what was asked for. A scatter point is a compositional idea, and the
gallery scores how much a scene changes BETWEEN songs, which is a different
question — Apollonian Gasket makes the same point from the other direction. This
may well look better and score worse. It needs eyes on it before the number
decides.

`gather` is a parameter and seed-driven like any other, so the amount varies per
song as asked; the focal points themselves come from the identity.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 16:42:55 +02:00
Dejvino
b1e58a6603 A focus: somewhere for a full-frame field to be about, tried on Voronoi 
Edge-to-edge textures have no composition to vary. Two songs of Voronoi Shatter
measured 0.004 apart in the layout block, and that was honest rather than a
metric failure — the cells changed and the arrangement did not, because there
was no arrangement.

So the identity now decides a FOCUS: one to three points, a reach, and a pull
that either draws the field in and densifies it or opens a void. The points come
off the same lattice everything else is placed on, so a scene using them is
composing in the song's terms rather than inventing a centre of its own.
focusField and focusWarp are in the contract, available to any field scene that
wants somewhere to be about.

Voronoi Shatter is the pilot. It tiles in the warped coordinate, so cells crowd
toward the focus or pull away from it, and its seams tighten where the field
gathers so the effect reads as a change in the shatter rather than a brightness
blob laid over one. 0.0463 to 0.0588, comfortably clear of the bar.

It did NOT work the way the hypothesis said. The gain is in orientation, 0.140
to 0.194, and in texture; the layout block moved from 0.004 to 0.007, which is
still nothing. Warping where the cells sit changes what they look like without
changing where the energy is, because the field still covers the frame corner to
corner. Layout stays blind to this family until a field is allowed to actually
fall away from its focus and stop being full-frame — which is a bigger change to
what these scenes are, and worth deciding rather than sliding into.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 16:39:35 +02:00
Dejvino
8d9b8a593a Fix a permanently-zero motion block, and let the song arrange Isometric Blocks 
Investigating three pieces of feedback turned up a measurement bug underneath
all of them. The gallery built four of the descriptor's five structural blocks
and never rendered a second frame, so `motion` came back 0.000 for every scene
in the library and dragged every score down by a fifth. It read as a property of
the scenes; it was a missing render. Every number in the gallery so far,
including the ones the 0.04 bar was calibrated against, was depressed by it.

    Plasma Bloom      0.0532 -> 0.0624
    Voronoi Shatter   0.0431 -> 0.0463
    Apollonian Gasket 0.0340 -> 0.0388

The Plasma Bloom versus Voronoi Shatter oddity is real and is the metric's
limit rather than a mistake. Plasma Bloom is a centred subject whose position
moves between songs, so its layout distance is 0.096; Voronoi Shatter is
edge-to-edge cells, and a full-frame texture has the same layout however its
cells fall, so its layout distance is 0.004. The descriptor cannot see "the
cells are different" as composition, and for full-frame work the layout block
contributes almost nothing. Worth knowing before the bar is used to judge that
family.

Apollonian Gasket scoring low with good-looking frames is the gallery working:
it measures how much a scene changes between songs, not how good it looks. A
scene can be beautiful six times and identical six times.

Isometric Blocks gets the displacement it was asked for. Every block sat exactly
on its lattice slot, so the plan of the field was the same plan in every song
and only the heights moved — a layout distance of 0.005. `scatter` steps each
cell off its slot by a fixed amount of its own, bounded under half a cell so
blocks do not cross and break the isometric read, and it drives the block's
height and side wall as well as its ground position. 0.034 to 0.049, clearing
the bar, with orientation variety nearly doubling as the rigid lattice softens.

Its layout distance stayed at 0.004, for the same reason Voronoi's did: the
field fills the frame either way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 16:32:30 +02:00
Dejvino
71ddd37711 A minimum variety bar in the gallery, drawn as a red line 
A visualizer that looks the same wherever it appears does not just fail to vary
— it leaks between songs, because it is cast into many of them and the viewer
who watches two videos ends up recognising the shot rather than the song. That
makes the floor a property of the scene, so the gallery now states it and the
scene has to clear it.

0.04 to start with, and honestly a starting bar rather than a derived one: set
where the flat cluster measurably sits, to be raised as scenes are lifted to it.

Drawn twice, because the two readings are different questions. Every meter
carries a red marker at the bar, so a single row can be judged on its own. And
in the default sort the list is cut by one red line where the scores cross it,
which answers the browsing question — everything above that line repeats itself
between songs.

Also restored progressive drawing, which the caching rewrite had quietly lost:
every draw had moved to the end of the build, leaving three minutes of spinner
and nothing to look at. The first rows are the interesting ones, since the sort
puts the repetitive scenes on top.

Verified by seeding the cache with rows spanning the bar and reloading: four
below, four above, the line between them, the marker at 20% of each meter, and
the grade colours switching at the right place.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 16:29:09 +02:00
Dejvino
e2e5104646 Take the effects off the visualizers and leave them to the grade 
Forty-nine scenes carried their own `glow`, every one of the sixty-five added
its own grain, and a handful had bloom, chroma, haze, smear or trail besides —
all of which the post chain already does, with an envelope and a threshold no
scene can see. Two uncoordinated effect layers is the smaller half of the
problem. The larger half is that an effect knob sampled per song makes a scene
look varied across parameter draws while its structure never moves, so the
gallery and the variety harness were both being told a scene was original when
only its halo had changed.

Removed in three shapes. Scene grain goes entirely — the post chain owns it.
Statements that ADD a falloff term scaled by the effect are bloom by another
name and are deleted. What remains is an intensity on something already drawn,
so the uniform is pinned to its default and the knob deleted: the picture
survives, the fake variety does not. Reactive entries driving those params went
too, since an effect modulated by the audio was the most convincing fake of the
lot.

That broke the style trait, which is the interesting part. Surface treatment had
been leaving the scenes for two commits — sigGrain to post, sigEdge to
inkStroke — and removing the last of it left `style` with almost nothing to
express: a style+shape signature had two eligible scenes in the whole library.
The trait is not obsolete, it has moved, so inkStroke and inkMask now read
u_sigLine and u_sigSoft alongside the ink's own weight and edge. A scene drawing
in the song's hand honours the track's line weight by construction, and the
runtime probe confirms it rather than taking it on trust. This reverses a call
made two commits ago for a reason that only became true now.

One check needed re-aiming rather than fixing. Two independently built shows are
two WebGL contexts, and engine/hash.js says at the top that bit-exactness is a
same-context guarantee; the check had been demanding it anyway and getting away
with it because the scenes it happened to cast were bit-stable. A casting change
put smooth-gradient scenes in frame and the last bit moved. Measured before
touching the check: three levels out of 255 across 2.6% of pixels, invisible.
The cross-context comparison is now a distance with a ceiling of four, and
bit-exactness within one show is still demanded by the check below it.

89/89 checks, 11/11 tests, all static gates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 16:21:34 +02:00
Dejvino
e90951c277 Cache the gallery against a fingerprint of the source that built it 
Three minutes of GPU work is fine once and intolerable on every page load, so
the gallery now builds once and comes back from IndexedDB after that.

The key is the part worth getting right. A cache you clear by hand is worse than
no cache: it will eventually show last week's pixels while you are judging this
morning's change, and it will look like a render, so you will believe it. Here
the key is a fingerprint of every file under src/, collected through
import.meta.glob so it covers the shaders, the identity, the look generator, the
engine and the descriptors without naming any of them — the file that
invalidates a render is exactly the one nobody remembers to list. Edit anything
and the entry stops matching and the page rebuilds without being asked.

Thumbnails are WebP blobs rather than raw pixels: sixty-five scenes at six
frames of 256x144 is 57MB raw, and measured, the compressed form is 4.1MB for
the whole library. Saving a build evicts every other one, so the store cannot
grow without bound.

The spinner is a CSS animation on purpose. Building blocks the main thread in
bursts and a JS-driven spinner would freeze mid-turn, which reads as a hang
exactly when the page most needs to look alive.

Verified rather than assumed, six behaviours: the fingerprint is stable across
calls; a build round-trips through storage and decodes to a real image (51%
non-black); a different fingerprint misses; saving a new key evicts the old; and
appending one comment to one scene file moved the fingerprint from 125-f842b05e
to 125-c056d7b9, which is the invalidation working end to end.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 07:19:22 +02:00
Dejvino
c6d96c9d40 A gallery, and a debug index to reach it from 
Sixty-five visualizers, six renders each, one per song — each carrying that
song's whole identity: its cast, ink, lattice, palette and a fresh parameter
draw. Side by side, a scene that cannot be changed by the song is obvious in a
way no aggregate could show, which is the complaint this answers.

It scores as well as shows. Each row carries the mean structural distance
between its own six frames, on the same descriptor the variety harness uses and
with colour excluded, so six palettes cannot disguise one image. Sorted
least-varied first, because browsing sixty-five scenes hunting for the
repetitive ones is precisely what a sort order should do for you.

The result names names. Thirteen scenes barely change across six songs, and they
fall into two groups that were already known separately. The ink-only
migrations — Analog Wow, Halftone Misprint, Pitch Shatter, Block Mosh, Scan
Tear — are the shallow tier flagged in MIGRATION.md, where the whole migration
was one posterisation. And Moiré Grid, Isometric Blocks, Quasicrystal, Truchet
Fold, Voronoi Shatter and Apollonian Gasket are the structural twin cliques the
library sweep found weeks of measurement ago, arriving here by a completely
different route: the sweep compared scenes to each other, the gallery compares a
scene to itself, and they agree on the same offenders.

debug.html collects the tools, since there are now enough of them that knowing
which to open is its own problem. It also carries the two things a newcomer
would otherwise learn the hard way: which measurements are safe to steer by
(direct render comparisons) and which are not (aggregate ratios), and what to do
when a page renders perfectly and does nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 07:13:54 +02:00
Dejvino
4217cca3d3 Rename clicktrack.js — a content blocker was eating it and killing the app 
A file called `clicktrack.js` matches the click-tracking telemetry patterns that
EasyPrivacy and similar lists block by substring. In a browser with a content
blocker the request never completes, and because it is a module import that
takes the entire graph with it: main.js never runs, no handler is ever bound,
and every control in the UI sits there looking correct and doing nothing.

The failure is unusually expensive to diagnose because everything else looks
healthy. The dev server returns 200 with the right MIME type, curl fetches it
fine, node imports it fine, and the app loads perfectly in any browser without a
blocker — which is how it passed every check here. Only the console names it,
and only as one line about a module that failed to load.

Renamed to metronome.js, which is also the better name for what it does. The
button id went with it, since cosmetic filter rules can hit ids too.

The general rule, recorded at the top of the file: anything shipped to a browser
and named like tracking will be treated as tracking. Avoid click, track,
analytics, pixel, beacon and ad in filenames and URL paths, however honest the
code behind them is.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 07:01:23 +02:00
Dejvino
4267c588aa Fix two checks the migration broke, and re-aim the signature gate 
Phase 2 kept its own list of contract uniforms and never learned about the
identity artifacts, so Prism Bloom reading u_castSides looked like an undeclared
uniform. It now takes the list from the contract itself.

Phase 9 demanded that every cast scene honour the track's signature, which was
true when the signature was a hard filter and stopped being true when it became
a weight. The check was asserting the old contract against the new generator.

Rather than delete it, it now checks the claim that actually matters. A track may
reach outside its signature — that change was made deliberately, because the
filter was disqualifying a third of the library and funnelling eleven scenes into
half of all videos — but it must still LEAN on it. Honouring scenes have to
carry the majority of the cast against a chance baseline near 25%, and section
anchors, which open a section and return most often, have to honour it almost
always.

89/89 checks pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 06:55:28 +02:00
Dejvino
df1ce9a957 Withdraw the separation figure: it is a ratio of two small differences 
Three runs of the post-migration song variety measurement give separation 0.314,
0.555 and 0.065 — mean 0.311, half-range 0.245. The 0.38 quoted from a single
run was meaningless.

The cause is structural rather than a sample-size accident. Separation is
(observed - floor) / (ceiling - floor), and here the numerator is 0.0089 while
floor and observed each carry an error bar of 0.015 to 0.019. Dividing one small
difference by another amplifies the noise in both. The metric cannot resolve the
thing it was built to report.

What survives: the ceiling now sits above the floor on every run, with zero
variance, where before the epic it landed underneath and the ratio was not
computable at all. That is a change in kind and it is solid. The magnitude is
not.

The decomposition is the measurement to steer by — noise floor of exactly zero,
effects an order of magnitude above any drift — because it compares renders
directly instead of dividing differences of aggregates. Documented as such.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 06:48:38 +02:00
Dejvino
c3b67f45bd Record the migration's end-to-end numbers 
Song separation went from not computable — the reference sat below the floor —
to 0.38, with the floor falling from 0.1449 to 0.1185 and the gap widening.
Every structural block moved: scale 40 to 59%, orient 65 to 89%, layout 60 to
111%, texture 89 to 81%, motion 53 to 65% of the reference.

Container variety rose throughout, 0.0557 to 0.0815 to 0.0901, which is the stop
condition holding: the library did not homogenise as it converged on shared
content.

Coupling did not move and is recorded as such. Whether a song looks different in
proportion to how it sounds different is still unsolved, and the migration was
never aimed at it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 06:45:59 +02:00
Dejvino
572b090b83 Fix two scenes that were failing their gate before the migration touched them 
The library-wide gate run left two failures after the migration was complete,
both of the shape "renders nothing at the bottom of a param range". Running the
same gate against the pre-migration source shows them failing identically, so
the migration did not cause either — it exposed them. The per-scene battery is
opt-in and nobody had ever run it across all sixty-five at once.

The defect is the RANGE, not the shader. Spectrum Sculpture at radius 0.15 is
too small to register and Circuit Bloom at grown 0.25 has no pads yet, and both
values are ones the generator is free to sample. Floors raised to where the
scene actually draws something.

All sixty-five now pass the full battery: renders, animates, deterministic,
distinct, param sweep, flash rate, every declared trait and every declared
artifact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 06:44:04 +02:00
Dejvino
2d3ed8eb02 Finish the migration: all 65 scenes on the identity artifacts 
Two helpers made the bulk of it mechanical. `inkStroke` is a drop-in for
sigEdge — the same line at the identity's weight rather than the track's — and
`castForm` is a drop-in for sigForm, same signature so call sites do not change
shape. With those in place the substitution table is one-to-one:

    sigForm(  -> castForm(     sigShape( -> castMain(     sigEdge( -> inkStroke(

Forty-seven scenes went through that pass in one run: twenty-six take the cast
and the ink, twenty-one take the ink alone. Then the gate ran over all sixty-five
and found three the pass had broken, which is the entire reason it exists.

Spectrum Sculpture rendered pure black. It had been using sigShape as a RADIAL
METRIC rather than drawing it, and the cast carries notches and a hollow — an
annulus used as a radius turns a sculpture inside out. Reverted to sigShape and
dropped to ink only. The lesson generalises: a scene that reads a form as
geometry is not a scene that draws it, and the classifier cannot tell those
apart from the source.

Eclipse Field stopped honouring its `style` trait. It opts out of surface grain,
so sigEdge was its only style evidence, and the ink replaced it. The trait claim
is now dropped — and so is the lint change that had let inkMask count as style
evidence, which was wrong and was hiding exactly this. A trait is a property of
the track a scene may honour; an artifact is content it draws. Taking the ink
says nothing about whether a scene responds to u_sigLine.

Circuit Bloom went empty at the bottom of its `grown` range, where the pads were
carried by a hairline and the ink's stroke is thinner than the edge it replaced.
Now filled as well as stroked.

Also: the backtick check now covers every shader literal rather than only the
preamble, because a mechanical pass over sixty files reintroduced one
immediately. Three rounds lost to that typo is enough.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-18 06:39:19 +02:00
Dejvino
e5bb7d77e0 A reproducible migration, and the first eighteen scenes through it 
The library is sixty-one scenes, which is too many to convert from memory or
from taste, so the migration is a queue with a gate rather than a judgement call
per file.

Three pieces. A classifier reads each shader and assigns a tier from evidence in
the source — drawn, figure, field, treatment — so two passes over the library
reach the same answer and the work has an order. MIGRATION.md is the recipe per
tier, written to be followed mechanically. And a gate makes the result
verifiable: `consumes` is now a schema field, the lint enforces it in both
directions, and the per-scene battery renders each scene under two deliberately
distant identities and requires the picture to change.

That gate is the part that matters. Without it `consumes` is a comment, and the
whole inversion becomes unverifiable at exactly the point where it stops being
checkable by eye. With it, a scene that declares the cast and ignores it fails.

Eighteen scenes migrated. Four by hand at the drawn tier — Firefly Drift,
Metaballs, Floating Geometry, Prism Bloom — and ten at the field tier by script,
which is one declaration and one wrapped return. All eighteen pass.

The field tier is honestly marginal and the gate says so: every one of the ten
moves by 37 to 39 of 255, against 173 to 255 for the drawn tier, and the
uniformity across ten unrelated scenes is the tell. That is one global
posterisation applying, not ten scenes expressing anything. Cheap, real, shallow.

The decomposition moved from identity being worth 54% of the container to 158%,
but the stage set changed underneath the measurement and part of that is
Metaballs expressing a cast better than Constellation did. What survives the
caveat is the useful finding: a migrated library scene carries the identity
better than a stage written from scratch to carry it. The four bespoke stages
were the wrong shape of effort.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 21:59:24 +02:00
Dejvino
00ad1d8c2b Measure where visual difference comes from, and correct the epic's framing 
Two cheap measurements settled a question three expensive ones had not, and both
of them contradicted the diagnosis offered for it.

The stated diagnosis was that the identity's expressive range had become the
bottleneck. It has not. A census of the identities themselves — no GPU, seconds
to run — puts mean distance between twelve songs at 0.43 with no near-identical
pairs and full coverage of every decision space: six of six fills, five of five
lattices, six of six protagonist forms, three of three element scales. The songs
are handed genuinely different designs.

The decomposition then asked whether those designs reach the picture, by holding
the container fixed and varying only the identity, then the reverse:

    identity only    0.0299
    container only   0.0557
    both             0.1101
    neither          0.0000

Identity is worth 54% of what the container is worth, against an instrument
noise floor of exactly zero, and the two compose to more than their sum. The
inversion works at the frame level. What it does not do is replace the container.

That corrects EPIC-3 §5, which proposed a song picking two to five stages on the
theory that shared content would substitute for container variety. Container
variety is the larger of the two effects and identity adds to it. Four stages
with a rich identity throws away the 0.056 the library was already providing —
which is exactly the shape of every measurement in this epic: stages have the
lowest floor of any arm and no advantage in spread.

The direction is therefore not a small set of stages. It is the whole library
consuming the cast: keep the sixty-one containers and make them draw the song's
content rather than their own. The migration was filed as a cost; it is the
payoff.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 21:42:04 +02:00
Dejvino
6e4106048b Staging: a shared lattice, and the element size the songs were all sharing 
The cast and ink slice lowered the floor as predicted but left `scale` — feature
size — consistently WORSE than the legacy arm, 0.033 against 0.053 and well
outside the noise. Every stage still chose its own element size from its own
param range, so every song landed in the same place: the videos shared a cast
and also, accidentally, shared how big everything was.

Staging is the third artifact. A lattice — grid, radial, spiral, scatter,
strata — with jitter, spread, a size hierarchy and a bias toward the middle or
the edges, transported the same way the cast is: uniforms plus a `stageNode`
function in the preamble, no new engine plumbing. Constellation, Swarm and
Procession place on it; the stage keeps its motion and gives up its composition.
Soloist takes only the scale, since a close-up has no composition to share.

`elementScale` is the part that mattered. It is the song's answer to "how big is
this made of", spanning about a factor of six, and it is the decision that was
missing rather than mis-set. Measured, over twelve songs and three runs:

    stages spread  +0.0111 ±0.0043  →  +0.0143 ±0.0008
    scale block      0.033 ±0.004   →    0.041 ±0.007

Against the legacy arm at +0.0108 ±0.0071 the harness still says
indistinguishable, and it is right to: the gap is +0.0035 and the legacy arm's
own run-to-run range is twice that. What can be said is narrower and holds up.
Stages have the lowest floor of the three arms by a clear margin — 0.082 against
0.095 and 0.104 — so sharing content does make a video look like itself, which
was the central prediction. And the targeted fix moved the block it was aimed at
in the direction it was aimed.

Also added: a lint check that the shader preamble contains no backticks. Twice
now one has closed the template literal and produced a check page that hangs on
"starting…" with an empty console, which is an expensive way to find a typo.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 21:36:55 +02:00
Dejvino
577ec107f6 Sweep pool size directly, and withdraw the claim that it mattered 
The Epic 3 arms appeared to show that a small casting roster was the largest
available win. Swept directly across 4, 8, 16 and 32 over twelve songs with
three pool draws each, the differences are 0.003 to 0.007 against a run-to-run
noise of +/-0.003 to +/-0.005. Pool size does nothing measurable.

The arms varied two things at once — smaller pool AND the same pool for every
song — and only the stages-versus-legacy comparison inside them held everything
else constant. That one still stands at +14%.

POOL_SIZE stays at 8, now on grounds the metric cannot see: about nine distinct
scenes in a video rather than seventeen.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 21:27:58 +02:00
Dejvino
ec38def1a9 Epic 3, first slice: the song brings its own cast 
Four stages, an identity layer, and an A/B that says the idea is right and the
reason it works is not the reason I expected.

A stage has no image of its own. It owns arrangement — a procession, a
constellation, a soloist, a swarm — and what it arranges comes from the track:
`castMain` and `castChorus` for the forms, `inkMask` and `inkValue` for the hand
they are drawn in. The identity generates a protagonist and a chorus with sides,
notches and hollows, plus an ink treatment of weight, edge, fill, outline and
posterisation. All of it travels as uniforms, so it is data rather than code and
a stage consumes it without knowing any other stage exists.

The protagonist IS the signature form rather than a second opinion about it.
They were separate draws in the first version, which let a track built on
hexagons put a round protagonist on screen — the signature said one thing and
the picture said another, and the shape trait stopped meaning anything for
stages. The cast now reads its geometry from the personality live and adds the
notches and hollows that turn a shape into a character.

Measured across seven songs, three arms, same instrument:

    arm                              floor   observed   spread
    stages, four of them            0.0801     0.1133   +0.0332
    legacy scenes, four of them     0.0830     0.1122   +0.0292
    the unrestricted generator      0.1058     0.1146   +0.0088

The prediction in EPIC-3 §7 was that sharing a cast would drop the floor
sharply. The floor did drop sharply — by a quarter — but nearly all of that came
from casting FEWER scenes per video, not from sharing content: the legacy arm,
four ordinary scenes with no cast at all, gets most of the same benefit. Stages
add about 14% on top of that.

That is worth knowing before building the other four registers. The single
largest available win was hiding in the roster size all along, and it costs one
number to take.

One result runs the wrong way and is recorded rather than explained: coupling is
+0.29 on the legacy arm and -0.18 on the stages. At twenty-one pairs neither is
distinguishable from zero, so it is a flag for a larger sample rather than a
finding.

Also fixed: a backtick inside the shader preamble's template literal, which
closed it and made every check page hang on "starting…" with no error in the
console.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 21:17:56 +02:00
Dejvino
75e0f0ef01 Make the whole library reachable, and measure what that did and did not buy 
Three changes, from the list the measurements pointed at.

The personality's values now centre on the audio and the seed picks within that
centre, rather than every value being drawn from the seed alone. A bright track
sits high in its frame, a dynamic one has depth, noise earns corners where tonal
material stays round. The signature's own choice is tilted by the music too — it
decides which scenes a track can cast, so leaving it to the seed meant the most
consequential decision in the generator had no relationship to the song.

The signature stopped being a hard filter. As a filter it was the single largest
cause of sameness: a scene declaring all four traits was eligible for every track
and opened half of all videos, one declaring two was eligible for one track in
fourteen, and eleven scenes out of sixty-one carried nearly everything. It is a
weight now, worth six times at full honour.

That fix alone made things worse, which is worth recording. The filter was doing
two jobs — collapsing the library, and giving each track a DIFFERENT pool to cast
from — and removing it kept the second loss. Every track drew from the same
weighted library and measured song separation fell. So each track now draws its
own pool of about a third of the library, weighted by the signature but sampled
without replacement, keeping the differentiation and dropping the bias.

Motion became a character rather than a rate. Tempo was the only lever and tempo
compresses; stillness is a separate question from speed, and it can now halve the
animation rate or raise it by a third.

What it bought, measured against the same instrument: every scene in the library
is now cast, where sixteen were never reached; identical casts across seeds went
from four pairs to none; and the raw structural distances all rose — motion by
57%, layout by 22%, scale by 23%.

What it did not buy: videos also became more varied INTERNALLY, by more than they
became different from each other. Two songs are no more distinguishable relative
to how much one video already changes over its own length than before, and the
coupling between musical distance and visual distance is still indistinguishable
from noise at this sample size. The ceiling reference now sits below real
outputs, so the separation ratio is reported as not computable rather than as a
large number.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 20:12:30 +02:00
Dejvino
a946f0e105 wip: instrument guards 2026-08-17 20:10:15 +02:00
Dejvino
2bbf5bcf42 Run both variety tests against the song bank, and fix the ceiling twice more 
The tests now run on real bank entries. The seed gates used to run on a
two-section synthetic whose only kinds were intro and outro, so half the scene
library was unreachable and the number was measuring that rather than the seed.

Probes are labelled by section kind and occurrence rather than by index, which
is what makes two different songs comparable at all — they have different
section counts, so probe 3 of one is not probe 3 of the other, and matching by
position would compare a drop against an outro and score the mismatch as
variety. For two seeds of one song the labels are identical and this changes
nothing, which is the point.

The song test measures one thing the seed test does not: coupling, the rank
correlation between how different two tracks sound and how different their
videos look. Separation alone can be had by a generator that ignores the audio
and hashes the file, and that would be a perfect score for a completely wrong
video. Separation without coupling is not variety, it is a different seed per
file.

The ceiling took two more attempts. Recasting every layer at random averages a
dozen scenes together and a dozen random scenes converge on the same generic
busy image, so two references came out closer to each other than two real
videos and blocks scored over 100% of achievable. Forcing one scene per
reference collapsed the other way: a video that never changes scene has almost
no internal variation, so the ceiling landed BELOW the floor, which is a
within-video quantity. A reference has to match the structure of what it bounds.
They now keep the real pipeline — rosters, shots, per-section sampling, so a
reference rotates between three or four scenes exactly as a real video does —
while drawing from disjoint slices of the library. Same complexity, nothing in
common.

The fingerprint measurement had the same shape of error: pooling every probe
mixed in how much each video varies over its own length, which is large for
everything, and washed the answer to a flat 100% while the separation score said
almost everything was collapsed. One vector per video now.

Both tests fail as committed. Seed separation 0.04, song separation 0.03,
coupling -0.03 — two different songs differ from each other by about as much as
one video differs from itself, and that difference has no relationship to the
music. Colour scores 112%.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 20:00:14 +02:00
Dejvino
e0c0dd664a Give the sections shapes, and the songs an opinion about their own edges 
Two complaints, one cause: stages were flat blocks and the noise bed was glued
to the pad level, so the bed was a constant hiss under a track that never went
anywhere.

Sections have contours now — rise, fall, swell, dip, surge — sampled per note
rather than per stage, so a build builds through its chords instead of stepping
between two flat halves. The noise gets its own per-stage level instead of
following the pad, which makes it an arrangement element: a riser through a
build with its filter sweeping up, a wash under a drop, nearly absent in a
breakdown. That is also the better test signal, since the segmenter classifies a
section partly on its energy slope and a build that does not build is one it has
to guess at.

Transition hardness is a per-song axis. Genres differ on this more than they
differ on tempo — an ambient record dissolves between its sections and a club
record cuts — and until now every song in the bank cut. Soft songs crossfade
across a couple of bars; the hardest get the pre-drop trick, where everything
stops for most of a beat before the loud stage lands. Five soft, three mid, nine
hard, and the builder fails if the bank ever loses either end.

The contours immediately broke the axis they were layered onto: section-to-
section contrast is dynamic range, so adding it put a floor of 0.50 under a
statistic that had reached 0.18. That is what compression IS, so contour depth
now scales with the track's dynamics — a limitered master has shallow section
contrast as well as a shallow crest. Back to 0.29, which is as low as enveloped
notes and hard cuts will go.

One regression accepted rather than fixed: the soft fades cost the beatless
entries their tempo detection, since the attack that gave a drone a pulse is
exactly what a crossfade removes. `drone` reads 170bpm for a 62bpm source. A
beatless track has no tempo and the detector is guessing either way; `ember`
exists to anchor the low end with a beat that is actually there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 07:43:59 +02:00
Dejvino
d59dd9d5dd Give the song bank notes instead of a test tone 
The tonal content was a sustained sine stack at a fixed root, and the stage
multiplier scaled that root directly — so a drop arrived at roughly 1.2kHz with
ten harmonics stacked on top of it and held there for thirty seconds. Every
statistic it was built to control came out correct. It was also unlistenable,
and a bank nobody can stand to play is a bank nobody audits.

There is a scale, a chord progression, a bass on the chord roots and a seeded
motif over the loud stages. Key, mode and progression come off the song's seed
so two entries are not the same four chords at different tempos. This is the
plainest thing that qualifies as music — not trying to be good, trying to be
playable enough that a person will listen to the bank and notice what is wrong
with it.

Register is the real fix. `stage.root` used to multiply the fundamental and now
adds voices upward instead: bass and chord stay where they belong, a lead octave
arrives when the arrangement opens up. Brightness still moves the harmonic
content and the key's register, but across an octave and a half rather than a
piercing sweep. Harmonics above Nyquist are dropped rather than left to alias,
since folded energy would corrupt the centroid and flatness the bank exists to
control.

Two measured consequences. Tempo detection on the beatless entries got better,
not worse — the bass gives a pulse where the drone gave nothing, and `drone` now
reads 62bpm instead of 88. And enveloped notes put a floor of 0.27 under the
measured dynamic range, because the gaps between notes are gaps a limitered
master does not have; note decay flattens toward a hold as dynamics falls, which
takes the compressed end back to 0.18.

Coverage still passes on all six axes and all six section kinds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 07:34:51 +02:00
Dejvino
cfe77fd524 Finish the song bank: axis ranges that are real, and a collinearity check 
Completes the bank whose first half went in with the scene commits. The
remaining work was all in what "covered" means.

Two axis ranges were wrong, and being wrong made the coverage report lie in
both directions. Loudness is a raw mean spectral magnitude, not a normalised
0..1, so every song read as 0.00 and the axis looked dead. Centroid is mapped
to a log frequency axis, so a track of nothing but sub-bass and a 55Hz pad
still measures 0.28 and pure hiss measures 0.93 — against a nominal [0,1] the
bank would have reported a permanent 50% gap that no synth change could close.
Both now carry the range the statistic can really take on, with the reason.

Spread on every axis is not coverage of the space, so the tool also measures
correlation between axes. It immediately found brightness and noisiness moving
together at r=0.95: noise colour had been tied to brightness, one axis wearing
two names, leaving the dark-and-noisy quadrant unreachable. Noise colour is now
its own parameter — hiss over a sub-bass pad is an ordinary record.

That got r to 0.94, and no further. Flatness is the geometric mean of the
spectrum over its arithmetic mean across the whole band, so a signal only
measures flat if it has energy everywhere, which is the same thing as measuring
bright; band-limiting the noise to hold the centroid down empties the top and
drops the flatness with it. It is a property of the analyser and real material
does the same, so it is allowlisted with its reason. Anything not on that list
still fails — the check is there to catch a bank that has collapsed, not to
relitigate physics every run.

The output is gitignored. It is 172MB of deterministic audio derived from a
table that will keep changing: rebuild it, do not carry it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 07:03:02 +02:00
Dejvino
0c5b0a8d4d Close out the twenty: what the gates taught 
Three lessons worth more than the scenes: a slow axis has to move a large
low-frequency area or the camera's own drift beats it, whole-frame luminance on
the kick is the strobe the flash gate exists for and it arrives by accident, and
a style trait expressed only as grain measures as no style at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 07:00:45 +02:00
Dejvino
905c217193 Three glitch scenes: a press, a loop and a tape 
Halftone Misprint is a printing fault rather than an electronic one — the image
is never damaged, only separated and reassembled out of register, and nothing
else in the library is made of dots. Droste Feedback scales the previous frame
where Time Smear translates it, so the image never clears. Analog Wow is the
continuous, wet counterpart to Scan Tear and Block Mosh: the error varies
smoothly down the frame because every line was written at a different moment.

Three things that had to be got right. The halftone ruling is a count of dots
across the frame, not a pixel pitch, or a 4K export is the same dot on a bigger
sheet. The Droste loop has to CONTRACT — expanding pushes every copy off the
edge and leaves a plume instead of a corridor. And all three expressed style
only through grain at first, which is the Side Quest 1 complaint: they now put
the track's line weight and edge softness into the dot, the ring and the band
boundary, taking the measured style response from 24/11/18 to 86/255/88.

Neither feedback scene declares a slow axis, with the numbers recorded in the
files: their own history moves the ten-second average further than any parameter
does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 06:55:43 +02:00
Dejvino
91e74ff167 A cast census, and the arrangement it needed to be true 
"Twenty-three of sixty-one scenes are never cast" was an artefact of the song it
was measured on. synthesizeSectioned has one change point, so it segments into
exactly two sections and both are quiet kinds — and intro, breakdown and outro
are restricted to the restful families for every director. Half the library was
unreachable before a seed was drawn, and the measurement reported that as a
casting failure.

synthesizeArrangement builds a real one: intro, build, drop, breakdown, drop,
outro, shaped to hit the segmenter's own classifier rather than to sound like
anything. Measured against a bank of those, two scenes out of sixty-one are
never a background, not twenty-three.

The census exists because "never cast" without a reason is unactionable. A scene
can die at the signature gate, at the director's family table, or in the roster
draw, and those are three different repairs — a trait declaration, a table, and
arithmetic that neither fixes. It reports which one, and it separates being cast
as a background from appearing as a translucent overlay, because a library can
look fully used while nine scenes carry every frame.

What it finds is that eligibility is almost entirely a function of how many
traits a scene declares. Four-trait scenes are eligible for every track and open
half of all videos; two-trait scenes are eligible for one track in fourteen; the
single scene declaring one trait is eligible for none, ever.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 06:45:42 +02:00
Dejvino
656e062069 A seed variety test, and a metric that had to be gated before it was believed 
Every other phase asks whether one video is correct. This asks whether two are
different — the failure the suite could not see, since a generator that ignores
its seed passes determinism, flash safety and liveness perfectly.

Frames reduce to a structural descriptor built to be blind to the cheap axes and
sensitive to the expensive ones: standardized luma kills exposure and palette, a
Laplacian pyramid gives the radial spectrum, and the gradient-angle histogram is
carried through a DFT magnitude so a rotation shifts it without moving it.
Colour is measured and never counted; its only job is to expose the case where
two seeds differ by a palette swap and nothing else.

The score means nothing on its own, so it sits between two references the same
instrument produced: a floor of how far one video travels from itself across its
own sections, and a ceiling of the same pipeline with every layer recast at
random. Two checks gate the instrument before any number from it is trusted —
recolour must move structure ~0 while moving colour a lot, and a quarter turn
must not move it at all.

Three things this got wrong first and now does not. Averaging each video's
probes into one descriptor washed out the structure being measured and put the
floor above the ceiling; probes are matched instead, which is fair because the
track is held fixed. Cosine distance on all-positive histograms scored unrelated
scenes at 0.05, too compressed to be read; chi-square replaces it. Single-link
clustering in the library sweep chained overlapping pairs into a fourteen-scene
group that did not exist; complete-link means every pair inside a group is
really a twin.

The sweep runs against all 61 visualizations, and it finds what the per-scene
distinct gate cannot, because that one compares raw pixels and structural twins
are merely differently coloured.

Both rendered gates fail as committed. That is the point of them: separation is
0.07 against a 0.35 target, two 4-cliques of scenes are one look each, and a
third of the library is never cast. The instrument passes; the generator does
not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-17 06:40:04 +02:00
Dejvino
875c3868dd Three geometric scenes, and two ways to strobe a whole frame 
Voronoi Shatter (cells with no fixed shape, re-cut on the phrase), Apollonian
Gasket (a packing built by inversion, so zooming finds more circles rather than
finer noise) and Isometric Blocks (a lit, gridded, solid surface where Floating
Geometry is bodies adrift).

Two flash failures, both the same mistake in different clothes. Apollonian's hard
depth cutoff popped a whole generation of discs in and out as the fold crossed
it — now faded rather than cut. Isometric Blocks drove block height off the low
end, so every tower in the frame grew and shrank together: 4/s. Moving the
reaction to the edges and to a hash-chosen quarter of the tops keeps the pulse
and drops the frame-wide swing.

Voronoi Shatter declares no slow axis: its re-cut puts the ten-second average
0.127 from itself between any two windows, the highest noise floor in the
library, and no parameter competes with that.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-16 23:44:50 +02:00
Dejvino
50ceb7afa1 Four structural scenes: a span, a well, an arcade and an aisle 
Suspension Span is two curves and a rhythm of hangers where Girder Lattice is a
texture of members. Stairwell Descent puts the vanishing point dead centre and
turns each flight, so the spiral is in the structure rather than in the motion.
Aqueduct March is masonry — the light comes through holes cut in a solid wall —
and Data Aisle is the interior, close counterpart to Neon City's exterior.

Two bugs worth naming: the arch openings compared a cell-local centre against a
global coordinate, which put every arch out in the wings and left a blank wall;
and the stairwell's depth fade was keyed on floor index, so it only darkened the
few pixels at the centre and measured as doing nothing.

Aqueduct March declares no slow axis and records the three candidates measured.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-16 23:25:34 +02:00
Dejvino
c8658a6d15 Four minimal scenes, and a transition control that was reading its denominator 
Pendulum Trace (a harmonograph over its own fading ghost), Contour Map (a survey
of a landscape with a tide in it), Shoji Grid (backlit panels that slide) and
Balance Stack (a leaning pile under a crossing sun).

Two of the four leave the slow axis undeclared and say why in the file: in a
frame this empty the camera's own drift moves the ten-second average more than
anything in the scene can, so a declaration would be a claim Phase 11 contradicts.

Phase 4's transition control changed from max(before, after) to their sum. These
scenes are calm, so casting them into the mid-shot control windows halved the
denominator while the boundary peak stayed put — 0.124 with them, 0.130 without.
For most of the window both stacks are live, four layers with accents in overlay
and screen, and a nonlinear blend of two moving stacks moves more than either
alone; "no worse than the busier half" was never a property a dissolve had. A
real pop still clears the threshold by an order of magnitude.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-16 10:40:23 +02:00
Dejvino
b01e3aff6f Three organic scenes: a reaction, a mat and a surface 
Turing Bloom runs activator-inhibitor in the feedback buffer, so the pattern is
formed rather than drawn — the one organic here that will not go uniform. It
declares no slow axis on purpose, and says why: its own convergence path moves
the ten-second average by 0.10, and every candidate axis measured under that.

Mycelium Web puts an organic on the ground instead of standing in front of the
camera; the colony front is legible as area, which is also what makes its axis
measurable. Scale Mosaic is a log-polar lattice with a shear, so the rows are
spirals and every scale is the track's signature form.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-16 10:10:21 +02:00
Dejvino
10a4cff0f6 Three flow scenes with a still point in them 
Rain Column, Magnet Lines and Kármán Street. The flow family was seven ways of
advecting a noise field; none of them had an upstream, an obstacle or a rule.
These do: rain falls in three parallax layers under one shear, the field lines
are exact contours of the poles' stream function so they close on themselves,
and the vortex street is shed by a body that never moves.

The slow axes took four rounds to land. A fine-grained axis — line count, drop
density — loses to the camera's own drift in the time-averaged comparison Phase
11 makes, so all three ended up on a param that moves a large low-frequency
area: streak length, pole spread, and a wake haze that dissolves the streamlines
it widens over.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-15 22:42:21 +02:00
Dejvino
47e6fa33f9 Twenty more scenes: the list, and why these twenty 
Even spread across families is not useful spread — casting disqualifies any
scene that misses a signature trait, so space and shape are worth more than
another entry in a full family. Each scene is specced against the neighbour it
must not resemble, because that is the gate with the numeric floor.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-15 22:19:13 +02:00
Dejvino
e6f5a2f0d5 Epic 2.5.2: make the framing actually reach the image 
Revisiting the framing layer turned up that it was not reaching four of the
forty-two scenes, and that none of its gates could have told us.

Those gates check the PLAN — cue sizes, the distribution of shot sizes across
a population, headroom, seek determinism — and a plan that never reaches the
image passes every one of them. Rendered at wide, normal and close, Scan
Tear, Pylon Grid, Pitch Shatter and the 3D Particle Field came back
byte-identical at every size.

The cause was a category error in the first implementation. Framing was
applied inside sigCamera, which is gated on the `camera` personality trait —
so a scene that declined the track's drift and sway silently declined the
shot size as well. That gating is right for a TRAIT and wrong for framing,
which is not one: framing is where the camera is standing for this shot, and
no scene should be exempt from it because of an unrelated art-direction
decision.

- Framing now lives in the shader epilogue, applied to the coordinate every
  fragment scene is handed, so honouring it is not optional. uv is left
  unframed on purpose: it is screen space, and prev() and sigGrain belong to
  the output image rather than to the scene being filmed.
- Scan Tear and Pitch Shatter build their image from uv deliberately — a
  signal artefact happens to the signal, not to the world behind it. They now
  slice on raw uv and build the field they displace from a new framedUv(p),
  so the tear stays locked to the frame while the imagery behind it is filmed
  wide or close.
- Particle Field receives framing in update() and honours it as a camera
  dolly, which is what framing literally is when a layer has a real camera.
  Distance divided by scale, matching the fragment path where the coordinate
  is divided by it.

New gate renders instead of inspecting: 42 of 42 scenes now respond to
framing, weakest Ridge Terrain at 0.22 of its own brightness, against a 0.05
floor. Also drops a stale comment on Layer.setFraming that still claimed
sigCamera applied it.

105/105 checks pass including the slow set.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-06 20:18:08 +02:00
Dejvino
c92c398d18 Drop the scratch probes, and stop them coming back 
_audioprobe, _captureprobe, _decodetest, _encprobe, _exporttest, _muxprobe
and _playtest were one-off harnesses for debugging the decode/encode/mux
path. They were untracked working-tree files until `git add -A` in 17a583a
swept them into that commit — my mistake, not a deliberate decision.

Removed, and _*.html added to .gitignore so a broad `git add` cannot pick up
the next batch either.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-06 20:09:56 +02:00
Dejvino
8f5ff3a239 Side quest 2: give three churning scenes something to develop 
Curl Flow, Signal Decay and Circuit Bloom had no parameter that changed their
structure. Each was one process at one scale, statistically identical
everywhere and at every moment — which is the technical statement of "it
looks the same for five minutes". Every pixel moving, the image never
changing.

Each got the large-scale structure it was missing, and the slow axis now has
something real to walk:

- Curl Flow gains a CURRENT: one broad band carries the filaments and the
  rest runs bare. The axis is the band's strength rather than its position,
  and that choice was measured — moving the band preserves the frame's total
  energy, so a ten-second average of it is nearly the same image wherever it
  sits (0.0028 across a full sweep). Opening and closing it changes how much
  of the frame is lit at all. The current gates the final image rather than
  only the vein term, because the feedback trail and the flow tint each fill
  the frame on their own.
- Signal Decay gains a DAMAGE FRONT: lock falls away on one side of a moving
  boundary, so the stack has a shape instead of every lane being an
  independent coin flip. Hiss had to be gated by it too — hiss rises as lock
  falls, so without that a damaged lane simply traded signal for noise and
  carried the same energy, and the front cancelled itself out.
- Circuit Bloom now actually BLOOMS. `reach` was a fixed vignette, so the
  packets ran, the pads blinked and the board never changed. Its range stops
  at 1.6 because the frame's far corner is ~1.4 units out and travel spent
  past that changes nothing — the first attempt wasted most of the axis up
  there.

The gate itself was wrong, and this is the more important half of the commit.
It compared the scene at thirty seconds against the scene at two and a half
minutes and divided by what it did between those points with frozen
parameters. Those are two different places in the song, so the denominator
was full of audio reactivity: it was really asking "does the axis move this
scene MORE than the music does", which no scene should have to pass. Circuit
Bloom failed at 0.92x while its mean luminance moved 5.6x across the axis — a
large structural change scored as nothing.

The experiment now holds time constant and varies only the axis, against a
control that is the same parameters over the next window — the residual churn
the ten-second average failed to cancel, which is the noise floor it has to
beat. With the axis disabled the two parameter sets are identical and the
ratio is 0.00 by construction, so the companion check is a real discriminator
rather than a formality. It reads 0.00x on all six scenes.

Moiré Grid 28.7x · Gate Corridor 14.3x · Truchet Fold 3.3x · Signal Decay
1.9x · Curl Flow 1.7x · Circuit Bloom 1.4x, against a 1.25x bar.

104/104 checks pass including the slow set. Closes SIDE-QUESTS.md §2 and the
open half of EPIC-2.md §3.4.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-06 19:51:51 +02:00
Dejvino
c44b527fe8 Side quest 3: fix the Horizon Lines determinism flake at the cause 
Phase 7's determinism check reported Horizon Lines at 2/255 against a 1/255
tolerance, but only when phase 7 ran in isolation — a full run warmed the GPU
first and it passed. A gate whose verdict depends on preceding load will
eventually stay green through a real regression, and determinism is the one
property this whole project is built on.

The cause turned out to be geometry, not the driver. The lines were drawn
with `smoothstep(u_thickness * (0.5 + u_sigLine), 0.0, d)` where the width is
0.008 scene units against a 0.028-unit pixel at 720p — a line under a THIRD
of a pixel wide, ramping from full brightness to nothing across that third.
The parameter range goes down to 0.002, which is a fourteenth of a pixel.
That is a near-vertical cliff, and a cliff turns a float wobble of 1e-7 in a
cancelling subtraction into a whole byte of colour. Nothing else in the
library sums 40 such terms.

Replaced with analytic coverage: `sat((w - d) / px + 0.5)`, where px is the
exact height of a pixel in scene units. The transition now always spans one
pixel, so no pixel sits on a discontinuity — and a line thinner than a pixel
comes out DIM instead of being drawn at full brightness wherever a pixel
centre happens to land on it. That second part is a real image fix as well as
a determinism one: it is the end of the shimmer this scene has always had.

Width stays in scene units, so resolution independence is untouched — the
320x180 vs 1280x720 diff still reads 0.00183 against a 0.06 limit.

Measured: worst delta over two identical 20-frame captures drops from 2 to at
most 1, usually 0. Phase 7 now passes in isolation, repeatedly, which is the
condition that was failing. Horizon Lines is no longer even the worst scene
on that metric.

Also removed the dead `total` accumulator while in there.

104/104 checks pass including the slow set. See SIDE-QUESTS.md §3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-06 19:41:58 +02:00
Dejvino
17a583a87f Side quest 1: give four scenes art direction other than grain 
Classic Wave, Silk Ribbon, Kaleido Tunnel and Slow Orb all declared the
`style` trait and honoured it with `col += sigGrain(uv)` and nothing else. A
declared trait is a contract — the disqualification rule in Personality.js is
the only thing keeping off-design scenes out of a track — so honouring it
with dirt meant these four could not take the `texture: 0` opt-out the grain
work introduced. Phase 9 measured their style response at exactly 0. They sat
at texture: 0.35 as a stopgap, which kept speckle on the library's cleanest
scenes purely to keep a gate green.

Each already had the knob; it just was not wired to the track:

- Classic Wave contrasts its wave through a bare smoothstep(0.2, 0.8). The
  transition width now comes from u_sigSoft and u_sigLine, centred on 0.5 so
  changing the hand does not change the exposure. Crest concentration is
  driven separately by u_sigLine, because a track is free to sample the
  scene's own u_softness at zero and the art direction must still show.
- Silk Ribbon's strand width is u_sigLine and its falloff exponent u_sigSoft:
  a sharp track gets a filament with a defined edge, a soft one a haze.
- Kaleido Tunnel drew its grid against a bare 0.42 — a line weight with no
  name. It is now a weight and a feather, both from the track.
- Slow Orb's body edge multiplies the scene's softness by the video's, and
  gains a sigEdge rim so a sharp-handed track gets a defined limb.

All four are now texture: 0. Style response measured against a maximally
soft-handed versus maximally sharp-handed personality: Classic Wave 111,
Silk Ribbon 216, Kaleido Tunnel 206, Slow Orb 102, out of 255 — previously 0.

104/104 checks pass including the slow set. See SIDE-QUESTS.md §1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-08-06 19:35:50 +02:00
141 changed files with 24315 additions and 601 deletions
Show Stats Expand all files Collapse all files

8
flow-state/.gitignore vendored
View File

@ -2,3 +2,11 @@ node_modules
dist dist
.vite .vite
out out
# Scratch probes: one-off harnesses for debugging decode/encode/mux issues.
# They are throwaway by nature and were committed once by accident.
_*.html
# The test song bank. Generated, deterministic, and ~170MB as audio — rebuild it
# rather than carrying it: node tools/build-song-bank.js
test/songs/

38
flow-state/EPIC-2.md
View File

@ -158,6 +158,15 @@ The middle row is the remaining work, and it is **not** mechanical: those scenes
parameter that changes their structure, so they need shader changes that introduce one. parameter that changes their structure, so they need shader changes that introduce one.
Parameter automation cannot substitute for structure a scene does not have. Parameter automation cannot substitute for structure a scene does not have.
**Closed by side quest 2** (SIDE-QUESTS.md §2). All three got the structure they were missing,
and the measurement was corrected in the process — the ratios above were produced by an
experiment that compared two different points in the SONG, so its denominator was full of audio
reactivity and it was really asking "does the axis move this scene more than the music does".
Circuit Bloom failed at 0.92× while its mean luminance moved 5.6× across the axis. The gate now
holds time constant and varies only the axis, against the metric's own noise floor. Current
figures: Moiré Grid 28.7× · Gate Corridor 14.3× · Truchet Fold 3.3× · Signal Decay 1.9× ·
Curl Flow 1.7× · Circuit Bloom 1.4×, against a 1.25× bar, with the axis disabled reading 0.00×.
### 3.5 A framing layer ### 3.5 A framing layer
The one that raises the ceiling rather than the floor. A shared zoom / crop / scale envelope The one that raises the ceiling rather than the floor. A shared zoom / crop / scale envelope
@ -181,9 +190,32 @@ actually change size, sizes stay in headroom, and two drivers over one look agre
What this pass does not do, and the reason it is "simple": the framing is constant within a shot. What this pass does not do, and the reason it is "simple": the framing is constant within a shot.
A zoom that moves during a shot is a separate device and would fight the drift LFO and the slow A zoom that moves during a shot is a separate device and would fight the drift LFO and the slow
axis, both of which already own continuous motion. Scenes built outside the shader contract (the axis, both of which already own continuous motion.
single 3D layer) are not framed yet. Both are the obvious next steps and neither is required for
a working project. **Revisited, and it was not reaching four scenes.**
The gates above check the PLAN — cue sizes, distribution, headroom, determinism — and a plan
that never reaches the image passes every one of them. It did. Framing was applied inside
`sigCamera`, which is gated on the `camera` personality trait, so a scene that declined the
track's drift silently declined the shot size too. That is correct for a trait and wrong for
framing, which is not one: it is where the camera is standing, and no scene should be exempt
from it because of an unrelated art-direction decision. Rendered, Scan Tear, Pylon Grid, Pitch
Shatter and the 3D layer were identical at every size, and nothing noticed.
Three fixes:
- Framing moved out of `sigCamera` and into the shader epilogue, applied to the coordinate every
fragment scene is handed. `uv` is deliberately left unframed — it is screen space, and `prev()`
and `sigGrain` belong to the output image rather than to the scene being filmed.
- Screen-space glitch scenes (Scan Tear, Pitch Shatter) build their image from `uv` on purpose:
a signal artefact happens to the signal, not to the world behind it. They now slice on raw
`uv` and build the field they displace from `framedUv(p)`, so the tear stays locked to the
frame while the imagery behind it is filmed wide or close.
- The 3D layer receives `framing` in `update` and honours it as an actual camera dolly, which is
what framing literally is when a scene has a real camera.
And a gate that renders rather than inspecting the plan: **42 of 42 scenes now respond to
framing**, weakest Ridge Terrain at 0.22 of its own brightness.
--- ---

392
flow-state/EPIC-3.md Normal file
View File

@ -0,0 +1,392 @@
# Epic 3 — the song brings its own cast
Epic 2 asked whether the output was worth watching. This one asks a narrower question that
the variety harness has now answered numerically, twice, with the same result:
> Two different songs are about as different from each other as one video is from itself
> five minutes later.
Everything below follows from taking that seriously.
---
## 1. What the measurements actually say
From `checks.html?variety=1` and `?songs=1`, both run against the song bank:
| | seed variety | song variety |
|---|---|---|
| floor — one video against itself | 0.143 | 0.145 |
| observed — two seeds / two songs | 0.145 | 0.151 |
| colour block | 111% of reference | 92% |
| coupling — musical distance → visual distance | — | no signal at n=21 |
Two numbers matter more than the rest.
**The floor is enormous.** A video differs from itself, across its own sections, by nearly as
much as it differs from a video of another song. That is not a subtle failure — it says the
thing we are shipping has no identity. A viewer cannot recognise a video as *this song's
video*, because its own opening and its own drop have less in common than its drop has with
some other track's drop.
**Colour is the only register doing work.** It scores at or above the reference while every
structural register sits below. The generator varies the palette and shuffles which shader
runs; it does not vary *what is on screen*.
The last round of fixes (per-track casting pools, soft signature weighting, motion character)
raised every raw structural distance — motion by 57% — and made all 61 scenes reachable. It
did not close the gap, because it raised the floor by as much as it raised the ceiling. More
scenes reachable means more rotation *within* a video too. Pulling harder on the same lever
will keep doing that.
**The lever is wrong, not weak.** Scene choice is a choice of *container*. Two containers
showing the same nothing look alike, and the same container showing two different things
looks different. We have been varying containers.
---
## 2. The inversion
Today a scene is self-contained. `metaballs.js` knows how to make metaballs and needs nothing
from the track except a palette and eighteen `u_sig*` uniforms it is free to ignore — and
most do, because they are modifiers on an image the shader already had.
The proposal turns that around.
> A song generates an **identity**: a small set of design decisions that produce concrete,
> reusable **artifacts**. Visualizers are **stages** that know how to arrange, draw and
> animate artifacts they are given, and a song picks two to five of them. A stage with no
> artifacts has no image.
The comic-book framing is the useful one. A comic is not held together by its panel layouts.
It is held together by the fact that the same characters, drawn in the same hand, keep walking
through it. Change the panel layouts and it is still recognisably the same comic; change the
characters and it is a different book even if every panel is laid out identically.
Panel layouts are what we have been varying.
The hard rule that makes this work, and the one thing that cannot be compromised:
> **An artifact is content a stage could not have invented for itself.** If a stage renders
> acceptably when the artifact is replaced by a default, the artifact is a modifier and it
> will be ignored exactly the way `u_sigSides` is ignored today.
---
## 3. The five registers
An identity is decided before any stage is cast, in five registers plus a timeline. Each
register is a set of *decisions*; each decision produces *artifacts* that stages consume.
### CAST — who is on screen
Two to four members, each with a role. Roles matter more than counts: a cast where everyone
is equal reads as wallpaper.
- **protagonist** — the form that carries most screen time, usually large and few.
- **chorus** — many small copies of a second form; the texture of the piece.
- **antagonist** — optional, and only present in some identities. The thing that interrupts:
a form that does not belong to the same family and shows up on transients.
Each member is one of a few *kinds*, and the kind determines what artifact it bakes:
| kind | what it is | artifact produced |
|---|---|---|
| `glyph` | a closed silhouette | SDF baked to a texture: sides, corner radius, elongation, notch depth/count, hollow ratio, aspect |
| `filament` | a line or ribbon | thickness profile along its length, taper, waviness, dash pattern, end caps |
| `field` | an unbounded surface | cell structure (voronoi / reaction / flow / weave), scale, anisotropy, contrast |
| `aperture` | a hole or window — a negative character | the same SDF machinery, used as a mask |
`shape.sides` / `roundness` / `elongation` already exist in `Personality.js` and are the seed
of this. The difference is that today they are numbers a shader may consult, and here they are
baked into an actual distance field that a stage *draws*. A stage that draws elements draws
**this** form, or it does not get to be in this video.
### INK — how they are drawn
The hand. Same cast, different ink, is a different book.
- **weight** — hairline, medium, heavy, variable-width.
- **edge** — hard vector, soft glow, dry brush, chalk, aliased/pixelated.
- **fill** — flat, gradient ramp, hatch, stipple, halftone dots, empty (outline only).
- **outline** — none, thin, heavy, double-struck, offset (misregistered).
- **value structure** — high-key, low-key, two-tone posterised, full range.
Artifacts: a **stroke profile** (a small 1D LUT of width and alpha across an edge), a **fill
texture** (hatch/stipple/halftone tile, generated), and a **value curve** (1D LUT). All three
are cheap textures every stage samples the same way. This is the register that most cheaply
makes two videos unmistakably different, because it changes every pixel of every stage at
once — and unlike a palette swap it changes *structure*, so the harness will see it.
### STAGING — where they are placed
- **lattice** — grid, radial, spiral, scattered (poisson), stacked/strata, horizon-anchored,
packed (apollonian-ish).
- **scale distribution** — uniform, few-large-many-small (power law), bimodal.
- **depth** — flat, layered parallax, true perspective.
- **occupancy** — how much of the frame is used, and whether the centre or the edges carry it.
Artifact: a **point set** with per-point scale, rotation and depth — generated once per song,
possibly a few hundred entries in a texture. Stages that place things place them *here*.
This is the register that fixes the `layout` block, which has been the weakest structural
number in every run. It is weak because every scene composes itself and they all converge on
"interesting thing, middle of frame".
### CHOREOGRAPHY — how they move
- **locomotion** — drift, orbit, pulse-in-place, march, tumble, swarm, fall, breathe.
- **timing** — continuous, beat-quantised, swung, stuttered, triggered-and-decay.
- **coherence** — do cast members move together or independently?
- **response map** — which audio feature drives which property. This is currently the
`reactive` block on each scene, decided by the scene author. It should be decided by the
identity, so a song has *one* idea about what a transient does to the picture.
Artifacts: a **motion path** (parametric curve or short keyframe list) and a **response
table**. The existing `motion.stillness` / `churn` characters are the first two knobs of this.
### EFFECTS — what happens to the whole frame
Mostly exists (`post`, `feedback`, `grain`). Worth adding the comic vocabulary, because it is
event-driven rather than constant and events are what the current output lacks:
- speed lines and impact bursts on transients
- registration offset / misprint on a drop
- screen tone and halftone as a *treatment* rather than as two separate scenes
- panel splits — the frame divided, two stages visible at once
- ghosting and echo tied to the beat grid rather than to a decay constant
### BEATS — what happens when
The narrative layer, and the one that turns a set of decisions into an authored piece:
- **entrance** — the cast does not all arrive at once. The chorus enters on the first build.
- **escalation** — cast count, density and ink weight climb across the track.
- **reduction** — a breakdown drops to the protagonist alone, held, on an empty stage.
- **payoff** — the antagonist appears exactly once, at the biggest moment.
This is the register the current system has nothing at all for, and it is why five minutes
feels long.
---
## 4. What an artifact is, technically
Everything above has to survive contact with a fragment-shader pipeline. Four transport
mechanisms cover all of it:
1. **SDF atlas texture** — glyphs and apertures baked once at load into an R8 or RG16F
texture. New engine capability: generating and uploading a texture. Modest work; the
renderer already manages framebuffers.
2. **1D LUT textures** — stroke profiles, value curves, palette ramps. Trivial.
3. **Tile textures** — hatch, stipple, halftone, field structure. Generated procedurally into
a texture once, then sampled — which also makes them cheaper than computing them per pixel
per frame, as scenes do today.
4. **Uniform blocks** — point sets, response tables, motion coefficients. The point set may
want a data texture if it exceeds a few dozen entries.
All four are *data*. None of them is code. That is what makes an artifact reusable across
stages that have never heard of each other, and it is what makes the identity serialisable —
which the editor, the preset system and the check harness all need.
---
## 5. Stages replace scenes
A stage declares what it consumes and what it can express:
```js
export const swarmStage = {
name: 'Swarm',
consumes: ['glyph', 'pointSet', 'strokeProfile'], // hard requirements
optional: ['fillTile', 'aperture'],
expresses: {
locomotion: ['swarm', 'drift', 'orbit'],
depth: ['flat', 'parallax'],
scaleDistribution: ['power', 'uniform'],
},
params: { /* as today */ },
};
```
Casting becomes a **matching** problem instead of a trait-filter problem: which stages can
express *this* identity's choreography and staging, given the artifacts it produced? That is a
much better question than the one `sceneHonours` asks, because it is about capability rather
than about which traits a scene author happened to list — the accident that put eleven
over-declared scenes in half of all videos.
Two to five stages per song, as proposed, is the right number and worth defending: it is
enough to cut between so the video is not static, and few enough that the cast is recognisable
in all of them. It should also be *derived*, not fixed — a long dynamic track earns five, a
four-minute ambient piece wants two.
---
## 6. Mapping functions
"Song + seed picks one function" is the generalisation of the current `director`, and it is
the right place for it. A mapping function is a *style of interpretation*: given an identity,
how do artifacts bind to stages?
- **literal** — protagonist is drawn plainly and large; one stage per section kind.
- **abstract** — the cast is never shown directly, only as apertures, shadows and negative
space. Same artifacts, unrecognisable result.
- **escalating** — cast count and ink weight climb monotonically; the video is one long build.
- **antagonistic** — two members are in visual conflict all the way through; stages are
chosen in pairs that disagree.
- **serial** — each section is a variation on the previous one rather than a cut to something
new; stages are chosen for adjacency on the library's structural map.
Directors already prove the mechanism works and that it is worth having several. Five
interpretations over a rich identity is a far larger space than five family orderings over a
fixed library.
---
## 7. Why this should move the numbers
Falsifiable predictions, so this can be checked rather than believed:
- **Floor drops sharply.** Two to five stages sharing one cast, one ink and one point set will
produce probes that agree on `scale`, `texture` and `orient` across a video's own length.
This is the single largest available win — the floor is currently 0.143 out of an observed
0.145.
- **Observed rises.** Artifacts are content. Two songs differ in what is on screen, not in
which of 61 shaders is running.
- **Coupling gets somewhere to attach.** Artifact generation is a natural place to read the
audio, and unlike trait-weight tilting it produces a *continuous* mapping: a slightly
brighter track gets a slightly sharper ink, not a different scene.
- **Colour stops dominating** — not by dialling the palette back, but because the structural
registers finally vary as much as it does.
- **The structural twins stop mattering.** `Tide Rings ≈ Quasicrystal ≈ Classic Wave` are
twins because they all draw their own generic content. Give them different casts and they
are different images.
- **The ceiling problem may dissolve.** Three constructions have failed because any reference
that restricts casting also flattens the reference's own richness. With an identity layer
there is a much better reference available: *the same song with a different identity*.
---
## 8. What happens to the 61 scenes
The library is real work and most of it survives — but not untouched, and pretending otherwise
would be the way this epic quietly fails.
- **Natural stages** (~20): anything that already places discrete elements — `metaballs`,
`voronoi-shatter`, `isometric-blocks`, `scale-mosaic`, `firefly-drift`, `floating-geometry`.
These want a glyph and a point set and mostly have parameters for both already.
- **Natural fields** (~15): `curl-flow`, `turing-bloom`, `mycelium-web`, `plasma-bloom`. These
become `field` artifact *producers* as much as consumers — a field character can be baked
from them and then sampled by other stages.
- **Natural treatments** (~10): `halftone-misprint`, `analog-wow`, `scan-tear`, `signal-decay`,
`time-smear`. These are ink and effects, not stages. Promoting them out of the scene library
and into the identity is a strict improvement — they are currently competing for screen time
with actual images.
- **Hard cases** (~15): scenes whose whole identity is one fixed image — `apollonian-gasket`,
`truchet-fold`, `quasicrystal`. Either they take a glyph as their repeating unit, which is
usually a small change and a large payoff, or they stay as legacy self-contained scenes with
a lower casting weight.
Incremental path: artifacts are *optional* at first, with neutral defaults, so every existing
scene keeps working. A scene opts in by declaring `consumes`. The variety harness then answers
the only question that matters — does a video built from opted-in stages score better than one
built from legacy scenes? Measure that on five stages before touching the other fifty-six.
---
## 9. Risks
- **Homogenisation within a song.** Sharing a cast across stages is the point, but overdone it
makes every shot the same shot. The floor is currently far too high; it is possible to
overshoot into far too low, and a video with no internal variation is boring in a new way.
The harness measures both directions, so this is checkable — but nobody will check it unless
the target is stated as a *range* rather than "lower".
- **The artifact contract is a real API.** `shader-contract.js` and `lint-scenes` need to
understand `consumes`/`expresses`, and getting that wrong makes every scene harder to write.
`HOWTO-visualizers.md` is currently a good document because the contract is simple.
- **SDF baking is new machinery** in a renderer that has so far only ever managed
framebuffers.
- **Migration is the real cost.** Sixty-one scenes is a lot of surface, and a half-migrated
library where some stages honour the cast and some do not is *worse* than either end state,
because the ones that ignore it read as the shot filmed somewhere else.
---
## 9a. What the first slice actually measured
Four stages shipped, and the A/B ran: same songs, same instrument, same pool
size, the only difference being whether the scenes draw the song's cast or their
own content.
arm floor observed spread
stages, four of them 0.0801 0.1133 +0.0332
legacy scenes, four of them 0.0830 0.1122 +0.0292
the unrestricted generator 0.1058 0.1146 +0.0088
The stages-versus-legacy comparison is the internally valid one — everything
except the content sharing is held constant — and it says the inversion helps by
about 14%. Real, in the predicted direction, and much smaller than hoped.
The third row looked at first like the headline: that a small roster, not the
content sharing, was carrying the improvement. It is not. That arm varies two
things at once — the pool is smaller AND it is the same pool for every song —
and a direct sweep of pool size alone (§ `checks.html?sweep=1`, twelve songs,
three draws each) finds differences of 0.003 to 0.007 against a run-to-run noise
of ±0.003 to ±0.005. Pool size does nothing measurable between 4 and 32.
Two lessons worth keeping. Arms that differ in more than one way cannot be read
as if they differed in one. And this metric's noise floor at seven songs is
large enough to invent findings — anything under about 0.01 of spread needs
repeats before it is believed.
## 9b. Where the difference comes from — and the correction to §5
Two cheap measurements settled what three expensive ones could not.
**The identity is not the bottleneck.** `tools/identity-census.js` over twelve
songs: mean identity distance 0.43, no near-identical pairs, and full coverage
of every decision space — six of six fills, five of five lattices, six of six
protagonist forms, three of three element scales. The songs are handed genuinely
different designs.
**The identity does reach the frame.** `checks.html?decompose=1` holds the stage
fixed and varies only the identity, then holds the identity fixed and varies
only the stage:
identity only 0.0299 one stage, each song's cast, ink and lattice
container only 0.0557 one identity, four different stages
both 0.1101 what the generator actually does
neither 0.0000 the same thing twice — the noise floor
Identity is worth 54% of what the container is worth, on a noise floor of
exactly zero, and the two compose to more than their sum.
That is the mechanism working, and it corrects the framing in §5. The proposal
there was that a song picks two to five stages — that the identity would
*replace* container variety. It does not. Container variety is still the larger
of the two effects, and identity adds to it rather than substituting for it. A
generator with four stages and a rich identity throws away the 0.056 it could
have had from the library.
So the direction is not "a small set of stages". It is **the whole library
consuming the cast**: keep the sixty-one containers, and make them draw the
song's content instead of their own. The migration in §8 stops being a nice-to-
have and becomes the entire point — and the payoff is additive with everything
the library already provides.
## 10. The smallest experiment worth running first
One artifact, three stages, one measurement. Do not build the whole identity layer on a
prediction.
1. Implement `glyph` only: an SDF baked from `shape.sides/roundness/elongation/notches` into a
texture, plus the plumbing to bind it.
2. Convert three existing element-placing scenes to consume it — `metaballs`,
`floating-geometry`, `scale-mosaic` are the least invasive.
3. Generate a look forced to those three stages, and run the seed and song variety tests
against it.
The prediction is specific: **the floor should drop and the `texture` and `orient` blocks
should agree across probes within a video**, while the between-song distance holds or rises.
If the floor does not move, sharing a cast is not sufficient and the ink register is where the
work actually is — which is worth knowing after two days rather than after two months.

584
flow-state/EPIC-4.md Normal file
View File

@ -0,0 +1,584 @@
# Epic 4 — the song tells a story
Epic 3 asked what is on screen. This one asks a question none of the existing layers can
answer:
> Why is this the *last* drop rather than the first one?
Nothing in the generator knows. And a viewer who cannot tell the difference is watching a
loop with good production design.
---
## 1. What is actually missing
The generator has four timescales, and `ArcDriver`'s header lists them honestly: per frame,
per shot, per section, whole song. Three of the four are cyclic or local. The fourth — the
"whole song" one — turns out to be much thinner than it reads:
* **`_slowAxisFor`** is the only thing in the system that travels one way across the track.
It moves one or two params per scene, `journey = smoothstep(t/duration)`, in a direction
picked by `rng.bool()`. It is real progression, and it is blind: it does not know where the
drop is, it does not know a section from a boundary, and half the time it travels the wrong
way for what the song is doing.
* **`paletteArc.underDrift`** — a bounded hue crawl, plus per-*kind* offsets.
* **`buildSlope`** — one bar of lookahead. Local by design.
Everything else that decides what a video looks like is keyed on section **kind**, and kinds
recur:
```
assignRostersByKind() roster per KIND — every drop cuts between the same visuals
KIND_ENERGY / biasFor() same energy, density, motion for every drop
rhythmFor(energy) same cutting pattern for every section of that energy
derivePaletteArc() kindHue.drop — the same hue offset at every drop
frameShot(style, ...) framing from previous shot + energy; no notion of when
```
So the fourth drop is cast from the same roster as the first, biased to the same energy,
cut at the same rate, tinted the same hue and framed by the same rule. The *content* of the
video is a function of `kind`, and `kind` has no arrow of time in it. That is precisely a
song structure without a story: recurrence without consequence.
This is not a bug in any of those modules. Kind-keying is what gave the video its identity
(§`assignRostersByKind`), and it should stay. What is missing is the second coordinate.
> Today a section is identified by **what kind of thing it is**.
> It should be identified by **what kind of thing it is, and where in the story it sits**.
---
## 2. The proposal, in one paragraph
A new pure module, `look/Story.js`, runs once per track between segmentation and the look
generator. It reads the section list and the summary, picks a **plot** the way
`directors.js` picks a director, locates the song's **key moments**, and emits a small
**narrative state** per section and per frame. Everything that currently keys on `kind`
keys on `(kind, story)` instead. No scene changes, and — in the first slice — no new
uniforms: the story acts by rewriting inputs the whole library already consumes, so it
applies to all 61 scenes on day one.
---
## 3. What a story is, here
Three layers, smallest to largest.
### Position — where a section sits
Derived, not invented. From the sections `segment.js` already produces:
```js
{
index, kind,
ordinal, ordinalOf, // "the 3rd of 4 drops" — the single most useful missing fact
act, // setup | development | turn | climax | resolution
isMoment, // one of the key moments below
}
```
`ordinal` alone unlocks most of the devices in §5, and costs a `Map` and a loop.
### Moments — the frames the story turns on
Measured from the track, never imposed. A short list, and each one has a definition that
falls out of data already on the FeatureTrack:
| moment | definition |
|---|---|
| **arrival** | first section whose energy clears 0.6 × max — the video's first "here it is" |
| **turn** | largest energy *fall* between adjacent sections after the arrival |
| **climax** | max-energy section; ties broken toward the later one |
| **resolution** | first section after the climax whose energy stays below it |
A two-section track collapses these onto each other, and that is fine — a track with no
structure gets almost no story, which is the correct behaviour and not a degenerate case
to defend against.
### Plot — what the track does with them
One coherent narrative shape per track, chosen weighted-random with the audio tilting the
odds, exactly the way `pickDirector` works, and for the same reason: a fixed mapping from
measured features to narrative is how a library ends up with one story per genre.
| plot | shape |
|---|---|
| **emergence** | almost nothing, then something. Reveal rises monotonically and stays. |
| **escalation** | each recurrence of a kind is further than the last. A ratchet, not a curve. |
| **collapse** | order → entropy. The climax is a breakdown of the thing, not a peak of it. |
| **return** | ABA. The outro rhymes with the intro, transformed by what happened between. |
| **unveiling** | the protagonist is withheld until the climax and then it is all there is. |
Each plot is a set of curves over five **story variables**, evaluated per frame:
```
tension 0..1 how hard everything is pushed
reveal 0..1 how much of the song's identity has been shown
closeness 0..1 wide and distant → close and involved
population 0..1 sparse → crowded → alone
order 0..1 regular → broken (or the reverse; the plot decides the sign)
```
These are *staged* curves, not ramps: they hold flat inside a section and move at
boundaries, with a step at the climax. That is what makes them read as a story rather than
as a slow zoom — a story advances in scenes.
---
## 4. Where it plugs in
Every one of these is an existing call site gaining an argument.
| site | change |
|---|---|
| `biasFor()` | tension modulates energy/density **within the kind's envelope**, bounded to ±0.2, so a breakdown at high tension is still a breakdown |
| `sampleValues(…, temperament)` | `extremity` scaled by tension — the ratchet for **escalation** |
| `assignRostersByKind()` | roster stays per kind; *which member plays* becomes a story decision (§5.1) |
| `buildStack()` | overlay chance follows `population`, not just energy |
| `rhythmFor()` | later acts pick from faster patterns; the resolution gets a held shot |
| `frameShot()` | `closeness` biases the size draw |
| `derivePaletteArc()` | new `'narrative'` mode keyed on act rather than on kind |
| `ArcDriver._paramsAt()` | `journey` comes from `story.journeyAt(frame)`; the slow axis gets its **sign from the plot**, not from `rng.bool()` |
| `ArcDriver.update()` | passes a story-shifted personality clone, memoised on rounded `reveal` exactly as `_paletteAt` memoises on rounded shift |
That last one is how the story reaches Epic 3's content registers without a new uniform:
`setPersonality` is already called every frame, and the identity uniforms are derived from
the personality object. Scaling `notchDepth`, `hollow`, `inkOutline`, `posterize` and
`latScaleSpread` toward their full values as `reveal` rises makes the song's cast literally
arrive over the course of the video.
---
## 5. The devices, ranked by legibility per unit of work
**5.1 The anchor is earned.** Today `roster[0]` opens every section of its kind. Instead,
reserve it: earlier drops play companions, and the anchor arrives at the climax. Same
roster, same identity, and now the biggest visual in the video lands on the biggest moment.
Roughly twenty lines.
**5.2 Recapitulation.** Under the **return** plot, the outro re-casts the intro's scene,
with the story's parameters rather than the intro's. The oldest device in music video and
the cheapest one here — the scene is already built and cached.
**5.3 The ratchet.** Under **escalation**, `ordinal/ordinalOf` scales temperament extremity
and slow-axis travel per recurrence. The fourth drop is measurably further out than the
first, on every param the scenes declare.
**5.4 Reveal schedule.** §4's identity scaling. Under **unveiling** the protagonist's
`u_cast*` form is a near-circle until the climax, then snaps to full at a downbeat.
**5.5 Punctuation.** A budget of **two or three single-use events** for the entire video,
spent at the moments in §3 — the only cut to black, the only symmetry-fold flip, the only
feedback reset. Single-use is the whole point: a device used twice is a style, used once it
is a moment. These are the only additions that need a flash-gate review.
---
## 6. How to know if it worked
The existing instrument can measure this almost unmodified. Build the section×section
descriptor distance matrix that `checks/variety` already knows how to produce, and ask
three questions of it:
* **Direction.** Does distance correlate with |i−j| beyond what kind explains? Today this is
~0 by construction: the matrix is kind-blocked, all drops mutually near, and time is
invisible. A story makes it a gradient. This is the headline number.
* **Recurrence.** For a repeated kind, is `d(first, last) > d(first, middle)`? That is the
ratchet, and it is the one a viewer names as "it kept going somewhere".
* **Coherence bound.** Adjacent-section distance must stay under the existing ceiling. A
story that maximises Direction by shuffling scenes is the failure mode, and this is the
gate that catches it.
Plus: each punctuation fires exactly once, on a downbeat, under the flash limit; and every
curve is a pure function of frame, so seek-exactness and the determinism grep hold.
**A prediction, stated in advance:** the seed-variety *floor* — how far a video travels from
itself — will **rise**, because that is what progression is. Epic 3 spent its effort pushing
that number down. Both are correct, and the instrument is what is wrong: it measures
distance and calls it drift, with no way to tell wandering from travelling. The fix is one
extra statistic, not a retreat from the feature — split the self-distance into an *ordered*
component (monotone with time; a story) and an *unordered* residual (a shuffle). Ship that
statistic **before** the feature, or the first honest measurement of Epic 4 will read as a
regression and be argued about instead of read.
---
## 7. Risks
**The story overrides the song.** A plot that declares a climax where the track is quiet is
worse than no plot. Mitigation is structural: moments are *found* in the audio (§3), never
placed by the seed, and tension is bounded inside the kind envelope so the quiet-kind
coupling in `directors.js` — the one that keeps an intro off a strobing scene — still holds
absolutely.
**Every video tells the same story.** The exact failure `directors.js` was written to fix.
Same mitigation: five plots, weighted, with seeded curve parameters inside each.
**It becomes a slow zoom.** If the curves are smooth ramps, this is an effect, not a
narrative. Staged curves with plateaus and a step at the climax are load-bearing, not a
refinement.
**Short tracks.** Under three sections, most of this has nothing to work with. Degrade to
the current behaviour explicitly rather than letting the curves do something arbitrary.
---
## 8. The smallest experiment worth running first
Do not build five plots on a prediction. Three changes, no new uniforms, no scene edits:
1. `Story.js` with position and moments only — no plot templates, one hardcoded
**escalation** curve.
2. Wire it to exactly two sites: the slow-axis sign/magnitude in `_paramsAt`, and the anchor
reservation in §5.1.
3. Add the **Direction** statistic to the variety report and run the song bank.
The prediction is specific: **Direction moves off zero and Coherence holds**, while the
between-song distance is unchanged — the story should differentiate a video *from itself in
time*, and have no opinion about other songs. If Direction does not move, the story is not
reaching the image and the rest of the epic is worth nothing until it does.
---
## 11. What was built, and what it measured
Built, in `look/Story.js` plus one argument at each call site listed in §4:
* position (`ordinal`/`ordinalOf`/act), the four moments, five plots with seeded curves;
* the anchor is earned (§5.1), the ratchet on temperament (§5.3), recapitulation (§5.2),
the reveal schedule on the identity uniforms (§5.4), story-driven cut rate, framing
closeness, and a `narrative` palette-arc mode;
* the slow axis now takes its **direction from the track** rather than a per-scene coin
flip, and its journey from the staged story curve;
* Phase 13 (`checks/phase13.js`, 8 checks, no GPU) and the **direction** statistic in the
variety report (`checks/variety/signature.js`).
Punctuation (§5.5) was **not** built — it is the only part that needs a flash-gate review,
and it is worth doing after the numbers below are understood rather than before.
### The first direction measurement
`checks.html?variety=1&library=0&seeds=6`, one song, against the arcless single-scene
reference:
```
floor 0.1697 direction -0.14
observed 0.1594 arcless ref 0.01
ceiling 0.1996
```
The prediction in §8 was that direction moves off zero **upward**. It did not. Three
readings, in the order they should be checked:
1. **The recapitulation is fighting the statistic, by construction.** Roughly two videos in
five recap, and `return` — the plot most likely to — is an arch that comes back. Its
first and last probes are *deliberately* similar, which is exactly what a negative rank
correlation between time separation and distance means. The statistic as written cannot
tell ABA from no story at all; it may need to be measured against the journey curve
rather than against clock time.
2. **Six seeds of one song is a small sample**, and the probe count (5) makes each video's
correlation rest on ten pairs.
3. **The arc may not be reaching the image**, which is the reading that matters and the one
§8 was written to expose. If 1 and 2 are controlled for and direction stays at zero, the
story is moving parameters that do not change the picture — the same failure the slow
axis had before scenes declared `slowAxis`, and the fix would be the same: name the
levers rather than guessing at them.
The no-story control arm (the same report with the story layer bypassed) **did not
complete** — the run hung in the browser after the first arm, so the floor and separation
figures above are not yet attributable to this work either way. That comparison is the next
thing to run, and it should be run before any conclusion is drawn from the numbers.
---
## 12. The second direction measurement — reading 3 was right
Same invocation, `checks.html?variety=1&library=0&seeds=6`, after the two fixes in §12.1:
```
§11 now
direction -0.14 +0.24 arcless reference -0.02
floor 0.1697 0.2130
```
§8 predicted direction moves off zero **upward** and that the floor **rises**, because that
is what progression is. Both happened, and the epic's headline hypothesis is confirmed.
Of the three readings offered in §11, **reading 3 was the correct one**: the arc was not
reaching the image. The story layer was never the problem — it had nothing to speak
through. Two of the channels a narrative would have to travel down were inert, and neither
failure was visible from the source:
**The overlay path was structurally dead.** `buildStack` gated layering on
`surfaceOf(m) === 'composable'`, and no scene in the library declared `surface` — so the
only scene that could ever sit on top was the one declaring `role: 'accent'`, which the
overlay roster excluded by construction. Empty intersection, every time: **0/144 stacks
carried an overlay**. `population` had been wired to the overlay chance per §4 and moved a
number nothing read. Fixed by labelling the library from the phase 12 coverage gate — the
37 scenes painting under 30% of the frame are composable — and collapsing the reserved
accent slot into one path. 1 distinct overlay scene became 27, at 34% of stacks.
**The camera was inert.** `framing.shift` moved the frame by a median of **0.029** of a
half-frame at a fresh uniform angle every shot, so successive offsets cancelled and the
median jump at a cut was **0.014**. `closeness` reached shot SIZE per §4; the recentre got a
per-track constant and a die roll. Fixed by `look/Camera.js` — see §12.1. Median offset is
now 0.190 and the median reframe 0.135.
This is the same failure mode the slow axis had before scenes declared `slowAxis`, exactly
as reading 3 predicted, and it had the same shape: a lever that existed, was wired, was
covered by gates, and moved nothing. **Both passed every check that existed**, because every
bound on them was a ceiling. A device doing nothing clears a ceiling comfortably.
> The lesson worth carrying out of this epic: a gate on a *device* needs a floor, not only a
> limit. Three of the four new camera checks are floors for this reason.
Readings 1 and 2 are still uncontrolled — the recap arch and the small sample both still
apply, and the honest statement is that direction was measured under them both times, so the
*change* is attributable even though neither absolute figure is clean.
### 12.1 Built since §11
* `look/Camera.js` — the director's camera department, and the answer to "who translates
story into imagery". Story says tension, order and act; this turns that into where the
frame looks and how it travels there. Each director in `directors.js` names a camera.
Jump distance follows tension and act, speed follows energy, curve is one of four, and a
cut chooses between reframing and matching so two scenes can still read as one place.
94% of shots now move *during* the shot — the recentre is no longer per-shot constant,
which is a deliberate departure from the rule in `framing.js` (that rule was right about
size and wrong about where the camera is looking).
* `surface` declared across the library, `role: 'accent'` retired in favour of
`surface: 'composable'` plus `background: false`, and one layering path instead of two.
* Four camera gates in phase 11, each with a floor.
---
## 13. What is left
Ranked. Three of the four are measurements, and they all report through an instrument that
is currently broken — hence the ordering.
**13.1 The seed-variety instrument does not report.** Not in the original plan, and now the
first thing to fix:
```
separation — not computable: the reference landed below the floor
floor 0.2130 · ceiling 0.2179 · all six seeds read "thin"
```
The ceiling — videos built from casts sharing *no scenes* — is 0.2179 against a floor of
0.2130. Real seeds already differ by about as much as maximally-unrelated ones, so
separation divides by a near-zero interval and comes back `NaN`. This is the `separation
NaN` in the standing phase 12 failure, and it has been there across every run including the
pre-work baseline.
§11 anticipated this precisely — *"the instrument is what is wrong"* — and proposed the
direction split as the fix. But direction was added **alongside** the broken separation
rather than replacing it, so the gate now leads with an unreadable number while a working
one sits underneath. The decision to make: repair the ceiling, or retire separation and read
**(floor, direction)** as the pair.
**13.2 The coherence bound (§6).** Never built. Optional while direction sat at zero;
load-bearing now that it is +0.24, because nothing currently stops future work from buying
direction by making adjacent sections incoherent — the named failure mode in §6.
**13.3 Recurrence (§6).** Never built. `phase13` has a spec-level cousin — was the kind
recast, did tension move more than 0.08 — but not the descriptor-distance statistic
`d(first, last) > d(first, middle)`. That is the ratchet measured in the image rather than
in the spec, and §5.3 calls it the one a viewer names as "it kept going somewhere".
**13.4 Punctuation (§5.5).** Still the only part of the plan wholly unbuilt. Nothing in
`src/` mentions it. Needs the flash-gate review, which is why it was deferred; §11 said to
do it once the numbers were understood, and they now are.
**13.5 The no-story control arm.** Still never completed. Less urgent than when §11 was
written — direction clears the arcless reference by 0.26, which is hard to explain without
a story — but it remains the only thing that would attribute the floor rise (0.1697 →
0.2130) to this work rather than to everything else that changed alongside it.
---
## 14. Composition — the frame has a floor and a ceiling
Not part of the original plan. It comes out of the same place §12 did: the layer stack was
being asked to carry the video, and half of what it was stacking was nothing.
**The floor.** Two thirds of the library is `composable` — sparse by design, elements ON
something. `canBackground` let almost all of it anchor a section anyway, so a section's
bottom layer was regularly a scene painting 2% of the frame with black behind it. Measured
across six videos, sampling the middle of every section: **9 of 40 frames were under 20%
painted, the darkest at 0.3%**. That is a minute and a half of a few bright things on black,
and no gate could see it, because every gate on the stack was a limit rather than a floor.
Every section now stands on a GROUND: a canvas painting at least half the frame, cast per
section kind so a section's shot cuts change the shot and not the world. When the shot is
itself a full canvas it *is* the ground — two canvases stacked is two pictures fighting. The
same measurement now reads **mean 92% painted, darkest 63%, nothing under 30%**, and phase 12
gates it in the image, not in the spec.
**The ceiling.** Layering had a rate but no budget, so how full a frame got was a die roll
that knew nothing about the song. A stack now has a coverage budget — director appetite
(`crowd`, 1.05 brutalist to 1.7 corrupt), times the section's energy and density, times where
the story is — capped at **200%**, two frames' worth of material. The ground and the shot are
paid for first; the budget governs what may be stacked on top, and the odds of an overlay
fall off as the headroom does rather than only at the wall.
**What it cost, and what it bought.** Same instrument, same six seeds:
```
baseline with grounds
separation 0.10 0.44
direction 0.06 0.21 (arcless reference 0.02 → 0.08)
library 80% 97% of what the library can express
scale 42% 56%
motion 68% 96%
texture 74% 90%
```
The ceiling *fell* (0.2334 → 0.2031) while observed rose, so part of that separation gain is
the reference arm coming down: videos that share a ground vocabulary are less unalike even
when they share no shots. The honest reading is that the floor rose 0.009, direction tripled,
and the ceiling moved toward the floor — the numbers are up, and not all of the rise is
signal.
**A scene that reads the previous frame cannot be a ground.** `prev()` returns the whole
composited frame, *including whatever is layered on top of this scene*, so a datamosh under a
shot is not grounding it, it is eating it. Measured the moment one became a bed: the render
stopped reproducing from a seek, and two WebGL contexts diverged by 91/255 against a ceiling
of 4 — phase 4 caught both. `readsHistory` is derived from the shader source rather than
declared, and costs the ground pool six of twenty scenes, five of them glitch.
`scenes/coverage.js` is the measured coverage of every scene, pasted back out of the phase 12
gate, which re-measures and fails on drift over a tenth of a frame. A new scene has no entry
and therefore cannot ground anything until someone renders it.
### 14.1 The measurements moved into the repo
Everything in §14 depends on a number — how much of the frame a scene paints — and the
first version of it was a table pasted into a source file by hand. That is the same mistake
`surface` already was: a measured fact written down by a person, correct on the day and
wrong after the next shader edit.
**`src/scenes/metadata.json`** is now generated, tracked in git, and holds every measured
fact about every visualizer: coverage as a shot, coverage as a bed, the variety score, the
per-block scores, and the mean structural profile. Nothing in it is typed by a human.
`surface` is derived from it (canvas at 50%, composable below) and declaring `surface:` in a
scene file is now a lint error.
It is refreshed from **gallery.html → refresh metadata**, which re-renders the library and
writes the file through a dev-only endpoint in `vite.config.js`. The file carries a
fingerprint of everything that can move a number in it — the scenes, the shader contract,
the identities and palettes they are handed, the descriptor definitions — and phase 12 fails
when that stops matching. A stale metadata file does not produce a stale report, it produces
wrong videos, so it has to be detectable.
**Measuring at the wrong bias measures nothing.** Coverage is mostly a function of a scene's
parameters, and the first measurement took one number per scene at the busiest section of
six songs. Metaballs measures 69% there and painted **6%** as an intro's ground. Three
things came out of chasing that, in order of how much each was worth:
* A bed is drawn with a **moderate hand**. `extremity` pushes parameters toward the ends of
their ranges, and half of what decides coverage is a parameter with no `bias` key at all —
Metaballs' `threshold` — so extremity is the only thing moving it and one end is an empty
frame. Grounds sample at `extremity * 0.2`. The extremes belong to the shot.
* Energy and density are **floors** for a ground, not reductions. Calming the bed by
lowering energy is how to empty it: scenes bias their fill against `energy` as often as
against `density`.
* Coverage is measured **twice**, once as a shot and once as a bed, at the same bias and
temperament the generator will use. `GROUND_BIAS` and `groundTemperamentFrom` live in
`scenes/surface.js` and are imported by both the generator and the measuring pass, so the
two cannot drift apart.
Rendered result across six videos, sampling the middle of every section: **mean 87% painted,
darkest 30%**, against 51%/0.3% before any of this.
**Eleven scenes can ground a section** — five organic, five geometric, one structural. There
is no `minimal`, `flow` or `glitch` canvas in the library that paints half the frame without
reading `prev()`, so every quiet section in every video stands on one of five organic beds.
That is the largest remaining hole and it is library work, not generator work: the fix is
writing dense, self-contained canvases in the three thin families.
### 14.2 The red line is gone
`MIN_VARIETY` — 0.1, drawn across the gallery, "below this a scene is the same picture
wherever it appears" — was right when a section was one scene and is wrong now. A section is
a ground, a shot on it and sometimes a pass over that, so what a viewer sees is a
combination, and a scene that is reliably itself is a good ingredient in one. Nineteen
scenes were failing a bar for being consistent.
What replaced it is a level up, and it is the reason the profiles are in the metadata: the
generator weights every layering choice by **structural distance** — how unalike two scenes
measured, on the same descriptor the variety harness compares videos with. Family labels and
the render disagree often enough to matter; two `geometric` scenes can be 0.31 apart and a
`flow` and an `organic` scene 0.04, and stacking the second pair is one picture at double
density. Measured over 436 stacked pairs: **mean distance 0.268, zero near-twins**.
### 14.3 The other end of the frame
The floor had a ceiling missing. Measured across twelve videos, sampling the middle of every
section: **a median 24% of every frame was clipped to pure white, and whole sections
rendered at 100%.** The videos were washed out, and no gate could see it because every guard
on the frame asked whether there was enough in it.
Three causes, all introduced or exposed by §14, in the order they were found:
**The shot was screened over its ground.** Screen is a lightening operator — correct for a
few bright elements over a bed, and over a filled canvas it drives everything toward white.
Replaced with a **lumakey** blend (`passes.js`): the shot's own brightness is its alpha, so
it replaces the ground where it paints and leaves it where it does not, keeping its own
colour instead of adding it to the bed's. Median clipping 24% → 11%.
**The feedback loop was an accumulator.** `cur + hist * decay * amount` settles a still
image at `1/(1 - decay*amount)` times its drawn brightness — **2.3x** at the settings the
generator hands out. Survivable when a frame was a few bright things on black; fatal the
moment every section stood on a filled ground. Turning feedback off took a blown frame from
100% to 34% mean luminance, which is the whole diagnosis in one number. It now divides by
the same gain, at 0.6 rather than 1.0: full normalisation took the lift out along with the
blowout — the median frame fell from 95% painted to 71% and fourteen sections dropped
through the black-frame floor.
**Nothing rolled off.** The grade clipped. A **highlight shoulder** now compresses everything
above 0.75 toward but never to 1. Clipped white is not brightness, it is missing information:
every difference inside it has been deleted.
Rendered, after all three — 74 sections, twelve videos:
```
before after
painted 51% mean 88% mean, darkest 43% (was darkest 0.3%)
clipped white 24% median 0% median, worst 21% (was worst 100%)
luminance 77% median 42% median, worst 80%
```
The gate is now two-ended: `composition · a rendered section is neither black nor blown out`.
**A ground has to survive the song's identity.** Chasing the dark end turned up the same
class of error as §14.1: a ground measured at 61% painted **2%** in one particular video,
because that song's ink treatment is `hollow` — outlines, no fill. Two changes came out of
it. The world is now drawn solid and the song's hand is kept for the subject
(`groundPersonalityFrom`), and eligibility takes the **worst** identity into account as well
as the mean: a ground fills the frame on average and never vanishes. Holding the worst case
to the full 50% would leave *four* castable beds in the library, which is a worse video than
an occasionally dim intro.
**Seven scenes can ground a section** — five geometric, two organic. Every quiet section of
every video stands on one of two beds. Restated from §14.1 because it got worse, not better:
this is the library's largest hole.
### 14.4 Blazing on purpose
Screening the shot over its ground and letting the two brightnesses sum is a real effect — a
drop that goes to paper for eight bars reads as the song peaking. The mistake in §14.3 was
not the effect, it was that the effect was the **default**: every section did it, so a median
quarter of every frame in every video was clipped, and nothing about that said "peak".
It is a decision now. Each director declares a `blaze` appetite — brutalist 0.05 (mass does
not glow), corrupt 0.5 (overload is the subject) — and a section still has to earn one: loud,
and late in the story. Quiet kinds never blaze, because a breakdown that goes white is not a
choice, it is a bug with a rationale. The default composite is the lumakey, which keeps the
shot's own colour.
The ceiling gate matches the intent rather than banning brightness. Per section, only a hard
cap — past about half the frame at full white there is no picture left to read. Across the
population, **no more than a fifth of sections may be hot at all**, which is the number that
actually distinguishes a director choosing to peak from a pipeline with no headroom.
Measured now: **1 of 74 sections blazing**, worst 30% clipped.

320
flow-state/HOWTO-variety.md Normal file
View File

@ -0,0 +1,320 @@
# How to build a visualizer that varies
`HOWTO-visualizers.md` covers making a scene that *works*. This one covers making
a scene that looks different from one song to the next — a separate skill, and
the one the library is currently worst at.
Everything here is a measurement rather than an opinion, and the numbers are
quoted so a future change can contradict them. Several already contradict things
believed earlier in the same week. Living document: add what you learn, and
delete what stops being true.
---
## The one-line version
> A scene varies when the SONG can change what is on screen. It does not vary
> because its parameters moved.
Parameters mostly change the same picture. Content changes the picture.
---
## How to know if you succeeded
```
gallery.html your scene, six times, on six songs — with a score
checks.html?scene=X the per-scene gate, including `consumes:` lines
```
The gallery score is the mean structural distance between a scene's own six
frames, on the same descriptor the variety harness uses. High means the song
changes this scene a lot; low means it looks like itself wherever it is cast.
**There is no bar.** There used to be one — 0.1, drawn across the gallery as a
red line — and it was removed when composition arrived. A section is no longer
one scene: it is a ground, a shot standing on it, and sometimes a pass over
that, so what a viewer sees is a COMBINATION. A scene that is reliably itself is
a perfectly good ingredient in one, and the old line failed it for being
consistent.
What replaced it is a level up. The interesting quantity is how unalike the
scenes in one stack are, and the generator now weights its choices by exactly
that — see `structuralDistance` in `scenes/surface.js`. Your scene does not have
to be varied on its own. It has to be unlike the things it will be stacked with,
and that is measured for you.
Some reference points, measured across six songs:
```
0.324 Droste Feedback varies a lot
0.082 Plasma Bloom
0.062 Voronoi Shatter after a focus
0.062 Apollonian Gasket
0.047 Moiré Grid after a subject; 0.031 before
```
### The measured metadata
`src/scenes/metadata.json` is generated, tracked in git, and holds every
measured fact about every scene: coverage, variety, per-block scores and the
structural profile the distance above is computed from. Nothing in it is
hand-written, and two decisions read it:
* **`surface`** — canvas at 50% coverage or more, composable below. It used to
be declared per scene and it drifted; nine scenes claimed `canvas` while
painting under a third of the frame. Declaring `surface:` in a scene file is
now an error.
* **which scenes may be a GROUND** — a canvas that does not call `prev()`, fills the frame
when sampled as a bed, and does not vanish under any of the six identities. Coverage is
measured twice, once as a shot and once as a bed, because a scene paints wildly different
amounts at different parameters: Metaballs is 69% as a drop's shot and 6% as an intro's
ground.
Refresh it from **gallery.html → refresh metadata** after changing a shader or a
metric. The file carries a fingerprint of the scenes, the shader contract, the
identities and palettes, and the descriptor definitions; when that stops
matching, phase 12 fails and tells you to re-measure. The write goes through a
dev-only endpoint in `vite.config.js`, so it only works under `npm run dev`.
---
## What the descriptor can and cannot see
Design against this, because half of "why is my score low" is here.
**It is blind to:**
- **Brightness and contrast.** Frames are standardised to zero mean and unit
variance before anything is measured. A scene that only gets brighter has not
changed.
- **Colour.** Measured, reported, and excluded from the score. Six palettes
cannot disguise one image, which was the entire point.
- **Rotation**, on the scale, orientation and texture blocks.
- **Quality.** It measures change between songs, nothing else. Apollonian Gasket
scores 0.039 and looks great. Both facts are true and neither implies the
other.
**It sees:**
| block | what it measures | how to move it |
|---|---|---|
| `scale` | feature size — fine grain vs large forms | let the song set element size (`stageScale()`) |
| `orient` | grid vs radial vs stripes, rotation-blind | break or vary a regular lattice |
| `layout` | where in the frame structure sits | move the subject, change what is empty |
| `texture` | element count, sparsity, mirror and radial symmetry | vary how many things there are |
| `region` | how differently the parts of the frame behave from each other | give the frame a subject, so one region is unlike its neighbours |
| `motion` | what moves and where, not how much | vary the KIND of movement, not the rate |
**`region` is the block that rewards having something to look at.** Layout says
where the energy is, normalised, so a uniform field and a field with a subject in
it can normalise to nearly the same answer. Region characterises each part of the
frame in its own right — detail, direction, element count — and reports how far
each deviates from the frame's average. A pattern spread evenly deviates by
nothing everywhere, which is exactly why there is nothing to watch.
Measured when it was added: Apollonian Gasket 0.039 to 0.062 and Plasma Bloom to
0.082, both on region alone, because both have a subject the old blocks were not
crediting. It is the strongest single block in the descriptor.
**A known blind spot.** `layout` is nearly useless for full-frame fields. Voronoi
Shatter measures 0.004 there no matter what changes, because edge-to-edge cells
occupy the same frame however they fall. That is honest — there genuinely is no
arrangement — but it means a whole family is judged on four blocks instead of
five. If your scene fills the frame corner to corner, expect to earn your score
on `orient` and `texture`.
---
## What has actually worked
Ordered by measured effect.
### 1. Draw the song's content instead of your own
The largest single lever. Take `consumes: ['cast', 'ink', 'staging']` and draw
`castMain`/`castChorus` where you would have drawn your own primitive. Recipe in
`MIGRATION.md`.
Measured: identity is worth 117% of what the container is worth
(`checks.html?decompose=1`) — swapping the song's cast under a fixed scene moves
the picture more than swapping the scene under a fixed cast. A migrated library
scene also carries the identity better than a stage written from scratch to
carry it, which was a surprise and is why the whole library was migrated rather
than replaced.
### 1b. Give the subject a third dimension — payoff still unmeasured
`castSolid` / `castChorusSolid` / `castMarch`, and `consumes: ['form']`. The
song's protagonist as an assembly of solids instead of an outline, so its
silhouette CHANGES as the shot moves rather than merely rotating. Recipe in
`MIGRATION.md`, including the two guards that keep many instances affordable.
Three scenes carry it — Effigy, Floating Geometry, Swarm — which is 34.8% of
videos containing at least one, against 11.9% when only Effigy had it. That is
reach, not payoff.
The payoff is still unmeasured: the variety report has not been re-run against
a library with these in it, so nothing here says the videos are more varied.
What is measured is narrower — across four seeds, the lit area of Effigy's
subject varies 14-113% over one turn against Soloist's 3-51% for the same
rotation, which says the outline genuinely changes rather than merely spinning.
Whether that reaches the variety blocks is the open question, and it is the next
thing to run. Do not migrate a field scene onto it hoping for a win.
### 2. Let the song decide element SIZE
`stageScale()`. Not the size of your features relative to each other — the size
of the whole vocabulary. A song of six huge forms and a song of four hundred
tiny ones are different videos before anything else is decided.
Measured: adding `elementScale` to the identity moved stages from
+0.0111 ±0.0043 to +0.0143 ±0.0008, and the `scale` block from 0.033 to 0.041.
### 3. Displace a regular lattice
If your scene is a grid, let the song push things off it. Bounded — past about
half a cell the structure the grid was providing stops reading.
Measured: Isometric Blocks 0.034 → 0.049, with `orient` nearly doubling from
0.093 to 0.164 as the rigid lattice softened.
### 4. Give a full-frame field somewhere to be about
`focusWarp(p, amount)` and `focusField(p)`. The song picks one to three focal
points; tile in the warped coordinate so cells crowd toward them or pull away.
Measured: Voronoi Shatter 0.0463 → 0.0588. **Note what moved:** `orient` 0.140 →
0.194, while `layout` went 0.004 → 0.007, i.e. nowhere. Warping where cells sit
changes what they look like without changing where the frame's energy is. The
gain is real; the stated reason for it was wrong.
---
## What has measurably NOT worked
Recorded because the failures were more informative than the wins, and because
each of these looked obviously right beforehand.
### Effects as parameters
Forty-nine scenes had their own `glow`; every one had its own grain. An effect
knob sampled per song makes a scene look varied across parameter draws while its
structure never moves — it inflates the score without changing the picture, and
it fights the grade, which already does bloom, grain, chroma and vignette with
an envelope the scene cannot see.
**Do not add glow, bloom, grain, haze, chroma or trails to a scene.** The post
chain owns them. `npm run lint:scenes` does not catch this yet; reviewers should.
### Concentrating disturbance instead of adding it
Isometric Blocks, twice. Spending the scatter budget near the focal points and
calming the rest measured 0.049 → 0.0374 — most of the field went back onto the
rigid lattice and took the orientation variety with it. Making the focus
additive instead recovered nothing: 0.0365.
Two plausible diagnoses, both wrong. The lesson is narrow and worth keeping:
**a focal point has to be something the field gains, not something the rest of it
pays for** — and even that framing did not rescue it, so the real cause is still
unknown.
### Making the roster smaller
A whole afternoon went into the theory that a track drawing on fewer scenes
would look more like itself. Swept directly across pool sizes 4, 8, 16 and 32
over twelve songs with three draws each: differences of 0.003 to 0.007 against a
run-to-run noise of ±0.003 to ±0.005. **Nothing.** The theory came from an
experiment whose arms differed in two ways at once.
### Reading a form as a metric rather than drawing it
Spectrum Sculpture used `sigShape` as a radial distance rather than as a subject.
Migrating it to `castMain` rendered pure black, because the cast carries notches
and a hollow and an annulus used as a radius turns a sculpture inside out. If
your scene consults a form's geometry rather than drawing it, it wants
`sigShape`, and the cast is not for you.
---
## Scene shapes, and how hard each is to vary
- **Element-placing** (`metaballs`, `floating-geometry`, `firefly-drift`) —
easiest. Take the cast, place on `stageNode`, keep your motion. Nearly all the
wins above are this shape.
- **Single figure** (`slow-orb`, `eclipse-field`) — take the cast and
`stageScale()`. Composition is yours to vary; most such scenes centre their
subject and never move it, which is free `layout` left on the table.
- **Full-frame field** (`voronoi-shatter`, `plasma-bloom`, `curl-flow`) — hard.
`layout` is unavailable to you. Earn it on `orient` and `texture`, and consider
a focus.
- **Fixed geometry** (`truchet-fold`, `quasicrystal`, `apollonian-gasket`) —
hardest, and currently unsolved. The image *is* the maths, so the cast cannot
be pasted on top of it. The open idea is to make the cast the repeating UNIT
the tiling is built from, which is a real rewrite per scene rather than a
recipe step. Nobody has tried it yet.
- **Treatment** (`analog-wow`, `scan-tear`, `halftone-misprint`) — these are
effects wearing a scene's clothes. They score 0.031–0.040 and their whole
migration was one posterisation. They probably belong in the identity's
effects register rather than competing for screen time as subjects.
---
## Trusting the numbers
This harness has two classes of measurement and only one is safe to steer by.
**Direct render comparisons** — the gallery score, `checks.html?decompose=1`, the
per-scene gate — compare rendered frames to each other. Noise floor of exactly
zero. Trust these.
**Aggregate ratios** — chiefly `separation`, which divides one small difference
by another — swing wildly. Three runs of the same post-migration measurement gave
0.31, 0.56 and 0.07. **Four conclusions were drawn and withdrawn during this epic
for exactly that reason.** Anything under about 0.01 of spread needs
`&repeats=3` before it is believed, and a difference that changes sign with the
sample size is not a difference.
Two more traps, both of which have bitten:
- **A score of exactly 0.000 is a dead shader, not a boring scene.** The gallery
now says so outright — the row goes purple and carries the luminance and
variance — but the underlying trap is general: a broken render produces the
most boring possible numbers rather than an error.
- **A permanently-zero block looks like a property of your scene.** The gallery
built four of the descriptor's five blocks for weeks; `motion` came back 0.000
for every scene and depressed every score by a fifth. If a block is identical
across many unrelated scenes, suspect the instrument.
- **Arms that differ in more than one way cannot be read as if they differed in
one.** This produced the roster-size result above, and it was believed for a
day.
---
## Checklist for a new or reworked scene
1. `consumes` declared, and the `consumes:` lines pass in
`checks.html?scene=<name>`. A declared artifact that is ignored is worse than
one not declared, because the casting code believes it.
2. No glow, bloom, grain, haze, chroma or trail parameters.
3. Element size goes through `stageScale()`.
4. Placement goes through `stageNode()` unless the composition IS the scene.
5. Gallery score above 0.1, checked before and after.
6. The rest of the per-scene battery still passes — especially `distinct`. The
more scenes share a cast, the easier it is to become one of your neighbours.
---
## Open questions
- **Coupling is zero and has never moved.** Whether a song *looks* different in
proportion to how it *sounds* different measures -0.02 ±0.03. Every change in
this epic left it there. Nobody has attacked it directly.
- **Fixed-geometry scenes** have no known route to variety.
- **The `layout` blindness** for full-frame fields would need a new descriptor
block measuring cell statistics rather than where structure sits.
- **The bar is a target, not a description.** Most of the library does not clear
0.1 — that is the point of it. It has moved from 0.04 to 0.05 to 0.1
as the descriptor gained a block and lost a bug, both of which raised every
score. Expect to move it again whenever the descriptor changes, and re-read it
off a fresh gallery rather than carrying the old number forward.

10
flow-state/HOWTO-visualizers.md
View File

@ -192,3 +192,13 @@ http://localhost:5180/checks.html?slow=1
Everything. Phases 2, 5 and 7 iterate the registry so a new scene is covered Everything. Phases 2, 5 and 7 iterate the registry so a new scene is covered
automatically; Phases 8-10 cover how the look generator uses it. Run this once automatically; Phases 8-10 cover how the look generator uses it. Run this once
before committing. before committing.
---
## Making it VARY
This document gets a scene working. Making it look different from one song to
the next is a separate skill with its own measurements, its own failures worth
not repeating, and its own gate — the gallery, with a minimum score of 0.04.
See `HOWTO-variety.md`.

108
flow-state/LIBRARY-20.md Normal file
View File

@ -0,0 +1,108 @@
# Twenty more scenes
The library sat at 42 — seven per family, evenly. Even spread is not the same as
useful spread: casting in `look/Personality.js` disqualifies any scene that does
not honour *all* of the track's signature traits, so the pool a given track draws
from is much smaller than the shelf count suggests. `space` and `shape` are the
thin traits, and they are where a new scene is worth most.
So: twenty scenes, roughly three or four per family, weighted towards `space`
(8 of 20) and `shape` (7 of 20), and each one written against a named neighbour
it must not look like. That second column is the real spec — "no two scenes
render the same image" is a gate with a numeric floor, and a scene that cannot
say what it is *instead of* does not exist yet.
Procedure per scene is `.claude/skills/build-visualizer/`: scaffold, write the
`scene()` body, `npm run lint:scenes`, `checks.html?scene=…`, then the full suite
before the batch commit.
## flow
| # | scene | traits | what it is | not like |
|---|---|---|---|---|
| 1 | Rain Column | camera, space, style | falling sheets of rain in depth layers, sheared by wind | Smoke Column rises and diffuses; this falls and stays discrete |
| 2 | Magnet Lines | shape, camera, style | iron-filing field lines around drifting poles | Curl Flow advects noise; here the field is solved, not sampled |
| 3 | Kármán Street | shape, camera, space, style | vortices shedding off a fixed body, drifting downstream | Vortex Drift spins the whole frame; this has one still obstacle and a wake |
## organic
| # | scene | traits | what it is | not like |
|---|---|---|---|---|
| 4 | Turing Bloom | camera, style | reaction–diffusion spots and stripes, grown in feedback | Ink Bleed spreads and stops; this reaches a living equilibrium |
| 5 | Mycelium Web | camera, space, style | threads creeping over a dark substrate, nodes lighting where they meet | Flora grows upright and symmetric; this is flat, branching and lateral |
| 6 | Scale Mosaic | shape, camera, style | phyllotaxis of overlapping scales, breathing | Cell Divide splits; this tiles a surface that never divides |
## minimal
| # | scene | traits | what it is | not like |
|---|---|---|---|---|
| 7 | Pendulum Trace | camera, style | one harmonograph curve drawn into a slowly decaying trail | Silk Ribbon is a wide ribbon; this is a single hairline and its ghost |
| 8 | Contour Map | camera, space, style | sparse topographic contours of a slow height field | Horizon Lines is a parallel bundle; contours are closed and nested |
| 9 | Shoji Grid | shape, camera, style | backlit paper panels with the signature form cut out of them | Moiré Grid interferes; this has four or five panels and holds still |
| 10 | Balance Stack | shape, camera, space, style | a few forms stacked on a horizon, long shadows | Salt Flat has one distant object; this one is near and about to fall |
## structural
| # | scene | traits | what it is | not like |
|---|---|---|---|---|
| 11 | Suspension Span | shape, camera, space, style | cable curves and towers passing overhead | Girder Lattice is a truss field; this is two curves and a rhythm of hangers |
| 12 | Stairwell Descent | camera, space, style | a recursive stairwell seen straight down | Gate Corridor moves through gates; this falls through floors |
| 13 | Aqueduct March | shape, camera, space, style | receding arches, tiered | Pylon Grid is open frames; arches are solid and carry a skyline |
| 14 | Data Aisle | camera, space, style | two racks of indicators receding to a vanishing point | Neon City is exterior and vast; this is interior and close |
## geometric
| # | scene | traits | what it is | not like |
|---|---|---|---|---|
| 15 | Voronoi Shatter | camera, style | drifting Voronoi cells, edges lit, re-seeding on phrase lines | Truchet Fold is a regular grid; these cells are irregular and move |
| 16 | Apollonian Gasket | camera, style | circle-inversion packing, infinitely nested | Quasicrystal is additive interference; this is exact recursive geometry |
| 17 | Isometric Blocks | shape, camera, style | an isometric field of extruded blocks rising on band energy | Floating Geometry drifts free; this is a solid, gridded, lit surface |
## glitch
| # | scene | traits | what it is | not like |
|---|---|---|---|---|
| 18 | Halftone Misprint | camera, style | CMY halftone screens, misregistered | nothing else in the library draws dots |
| 19 | Droste Feedback | camera, style | recursive zoom-and-rotate video feedback around a small motif | Time Smear translates; this scales, so the image eats itself |
| 20 | Analog Wow | camera, style | tape warp: horizontal wobble, chroma bleed, dropout streaks | Scan Tear and Block Mosh are quantised and blocky; this is continuous and wet |
---
## What landed
All twenty, in six commits, each batch gated before it went in. The library is
at **62 scenes**: flow, organic, geometric and glitch at ten each, structural and
minimal at eleven. `space` went from 20 scenes to 28 and `shape` from 21 to 28,
which was the point — those are the traits that decide whether a track has
anything to cast.
Three things came out of the build that are worth keeping:
**A slow axis needs a large, low-frequency area, not a detailed one.** Phase 11
compares ten-second averages, and the camera's own drift moves that average by
0.01-0.02 on any high-contrast image. An axis that changes line count, drop
density or tier count loses to it; one that changes a wake's width, a colony's
extent, a tide level or a deck's height beats it. Seven of the twenty declare an
axis. Six do not, and each of those files records the candidates measured and
what they scored — Turing Bloom, Voronoi Shatter, Droste Feedback and Analog Wow
because a feedback or re-cutting scene's own history moves the average further
than any parameter can, Contour Map and Balance Stack because the frame is too
empty for anything in it to out-move a slow pan.
**Two ways to strobe a whole frame, both found by the flash gate.** Driving block
heights off the low end made every tower in Isometric Blocks grow together — a
frame-wide luminance cycle on the kick, 4/s. A hard recursion cutoff in
Apollonian Gasket popped a whole generation of discs in and out as the fold
crossed it. Local terms and a faded cutoff fixed both.
**Style expressed only as grain is not style.** Six of the twenty first shipped
with a style response the render gate measured at 11-24 out of 255 — the Side
Quest 1 complaint, reproduced from scratch. Putting the track's line weight and
edge softness into the actual subject (the dot, the ring, the seam, the course,
the band boundary) took them to 86-255.
One check changed: Phase 4's transition control is the sum of the two adjacent
shots' motion rather than the larger of them. See the commit; the short version
is that a dissolve has both stacks live and a nonlinear blend of two moving
stacks moves more than either alone, so the old control was reading its own
denominator once the library had calm scenes in it.

324
flow-state/MIGRATION.md Normal file
View File

@ -0,0 +1,324 @@
# Migrating a scene onto the identity artifacts
The recipe `tools/migration-status.js` reports progress against. Sixty-one
scenes is too many to convert from taste, so this is written to be followed
mechanically and to produce the same result twice.
Background is EPIC-3.md. The one-line version: a scene draws its own content, a
migrated scene draws the *song's* content — its cast, in its ink, on its lattice
— and the measured payoff is additive with everything the scene already did.
```bash
node tools/migration-status.js # the queue, and what each scene needs
node tools/migration-status.js --all # every scene and its tier
```
---
## The rule that decides everything
> **An artifact is content a scene could not have invented for itself.**
If the scene still looks right with the identity switched off, it has used the
artifact as a modifier and the migration has not happened. That is not a style
preference — it is the exact failure mode of `u_sigSides`, which thirty scenes
declare and most quietly ignore, and it is why the gate below exists.
---
## The four tiers
The classifier assigns these from the shader source. Check its answer, don't
trust it: it reads patterns, and a scene that places elements in an unusual way
will be misfiled.
| tier | what it looks like | consumes |
|---|---|---|
| **drawn** | loops over discrete elements at computed positions | `cast, ink, staging` |
| **figure** | one or a few forms, its own composition | `cast, ink` |
| **field** | a continuous surface — noise, flow, terrain | `ink` |
| **treatment** | an effect over an image rather than an image | `ink` |
| **solid** | a camera in a space, marching a distance field | `form, ink` |
Accents are excluded. They are mostly-empty depth passes, not subjects.
---
## Recipe: `drawn`
The highest payoff and the least invention. Four edits.
**1. Declare it.** Add to the module, above `params`:
```js
consumes: ['cast', 'ink', 'staging'],
```
**2. Replace the primitive with the cast.** Whatever the loop was drawing —
a circle, a box, `sigShape`, a bespoke SDF — becomes one of:
```glsl
float d = castMain(q) * size; // the protagonist: large, few
float d = castChorus(q) * size; // the chorus: small, many
```
`q` is the element-local coordinate, `(p - pos) / size`. Multiplying the result
back by `size` restores it to scene units, which is what the ink expects.
Use `castMain` when the scene draws a handful of things and `castChorus` when it
draws a crowd. A scene that draws both should use both — that is what the two
members are for.
**3. Replace the placement with the lattice.**
```glsl
vec3 node = stageNode(fi, float(u_count)); // xy position, z scale multiplier
vec2 pos = node.xy;
float size = u_size * node.z;
```
Keep whatever the scene did that was *motion* — a drift, an orbit, a wander, a
march. Give up what was *composition*. The split is the point: identity owns
where things are, the scene owns what they do.
A scene whose composition IS its identity — a spiral that must be a spiral — can
keep it and take `stageScale()` alone, which is the song's element size. Declare
`staging` either way.
**4. Replace the edge with the ink.**
```glsl
col = mix(col, pal(i + 1), inkMask(d, uv));
```
`inkMask` does fill, fill treatment (hatch, stipple, halftone), outline and edge
hardness in one call. Delete the scene's own `smoothstep(soft, -soft, d)` and
its `sigEdge` — the ink supersedes both.
Then wrap the return:
```glsl
return vec4(inkValue(col), 1.0);
```
**Do not** delete `sigCamera`, `sigGrain`, `sigHorizonY` or `sigAir`. Traits and
artifacts are different layers and both still apply.
---
## Recipe: `figure`
Steps 1, 2 and 4, skipping the lattice. Declare `consumes: ['cast', 'ink']`.
Take `stageScale()` if the figure has a size worth scaling and add `staging` if
you do.
---
## Recipe: `solid` — the cast in three dimensions
For a scene that has a camera in a space rather than a plane: a raymarcher, a
corridor, anything where the subject can be walked around. Declare
`consumes: ['form', ...]`.
```glsl
vec3 ro = vec3(0.0, lift, -dist); // eye, in object radii
vec3 rd = normalize(fw * lens + rt * p.x + up * p.y);
vec3 n;
float hit = castMarch(ro, rd, dist + 3.0, n);
if (hit > 0.0) col = castLit(n, rd); // lit in the track's palette
```
`castSDF3(vec3)` is the distance field if you want to place, repeat or carve
with it yourself; `castNormal3` is its normal. All of them fall back to the flat
profile extruded when a track brought no assembly, so they are safe to call
unconditionally.
### Many instances
For a scene that was stamping `castMain` in a loop — a field, a swarm, a belt —
swap the stamp for `castSolid`, which marches one instance orthographically in
its own frame:
```glsl
vec2 local = (p - centre) / size; // exactly what castMain was given
if (dot(local, local) > 1.6 || painted > 0.5) continue;
vec3 n;
float hit = castSolid(local, castTurn(yaw, pitch), n);
if (hit > 0.0) { painted = 1.0; col = castLit(n, vec3(0.0, 0.0, 1.0)); }
```
`castChorusSolid` is the same for a chorus member — the protagonist's body plan
with fewer parts and its own proportions, which is what a field of many should
be drawing.
**Both guards in that snippet are load-bearing**, and each was found by a
measurement rather than by review:
* the bounding-sphere reject, because without it every pixel evaluates every
instance's distance field — Swarm measured 59ms/frame at 4K against a 60ms
ceiling;
* `painted`, because instances overlap several deep at the top of the size
range, and marching all of them made Floating Geometry's own gate run for
minutes. Which instance wins where they overlap was always arbitrary, so
first-wins costs nothing.
A third rule lives in the contract rather than in your scene: take the surface
normal AFTER the march loop, never inside it. GLSL unrolls a fixed-bound loop,
so a normal in the loop body multiplies four more copies of the assembly SDF by
the step count. For the same reason these helpers are compiled only into scenes
that declare `form` — see FORM_PREAMBLE.
Worth the cost only if the shot MOVES relative to the object. A solid held at
one angle is a silhouette with shading, and `cast` draws that for a fraction of
the price — the assembly earns its keep through the outline changing, which
needs either the object turning or the camera travelling.
---
## Recipe: `field` and `treatment`
There are no elements to replace, so this is one edit plus a judgement.
```js
consumes: ['ink'],
```
```glsl
return vec4(inkValue(col), 1.0);
```
If the field already dithers, hatches or posterises internally, replace that
with `inkPattern(uv)` so the treatment is the song's rather than the scene's.
If it does not, `inkValue` alone is the whole migration — a value structure
shared across every scene in a video is worth having and costs one line.
Be honest about `treatment` scenes. Most of them are effects wearing a scene's
clothes, and EPIC-3 §8 argues they should move into the identity's EFFECTS
register rather than compete for screen time as subjects. Migrating one is a
holding action, not the answer.
---
## Verifying — the part that makes this reproducible
A migration is not done when the code looks right. It is done when the gate
passes:
```
checks.html?scene=<Scene%20Name>
```
Every artifact in `consumes` must show a passing line:
```
PASS consumes: cast delta 255/255 (floor 24)
PASS consumes: ink delta 255/255 (floor 24)
PASS consumes: staging delta 255/255 (floor 24)
```
That check renders the scene twice under two deliberately distant identities and
requires the picture to change. A scene that declares `cast` and ignores it
fails here, which is the only reason `consumes` can be trusted at library scale.
`npm run lint:scenes` enforces the other half in both directions: declaring an
artifact without calling it, and calling one without declaring it. The second
matters more than it looks — an undeclared artifact hides the scene from this
report and from anything that later selects scenes on capability.
Then the usual battery still applies. `renders something`, `animates`,
`deterministic`, `distinct`, `param sweep`, `flash rate` and every declared
trait must all still pass. A migration that breaks `distinct` has made the scene
into one of its neighbours, which is a real risk here: the more scenes share a
cast, the more two weakly-composed ones converge.
---
## Measuring the payoff
Per scene, the gate. Across the library, two numbers:
```
checks.html?decompose=1 identity against container, measured apart
checks.html?experiment=1 stages against legacy, with error bars
```
`decompose` is the one to watch. Before the migration, over the four purpose-
built stages:
```
identity only 0.0299 container only 0.0557 identity = 54% of container
```
After the first eighteen scenes, over the eight that consumed the cast:
```
identity only 0.1287 container only 0.0815 identity = 158% of container
```
Complete, over all thirty-two:
```
identity only 0.1054 container only 0.0901 identity = 117% of container
```
The number that matters in that row is `container only`, which went 0.0557 →
0.0815 → 0.0901 as the migration progressed. That is the stop condition below
holding: the library did not homogenise. `identity only` is noisier than it
looks because the scene it probes is picked from the migrated set and changes
between runs — Constellation, then Metaballs, then Floating Geometry — so read
it as "comparable to the container" rather than as a precise ratio.
End to end, song variety across twelve songs — and a warning about how to read
it. A single run after the migration gave separation 0.38. Three runs give:
```
separation 0.314, 0.555, 0.065 mean 0.311, half-range +/-0.245
floor 0.1073 +/-0.0145
observed 0.1162 +/-0.0185
ceiling 0.1434 +/-0.0000
```
**Do not quote the separation figure.** It is a ratio of two small differences —
`(observed - floor) / (ceiling - floor)` — and the numerator here is 0.0089
while each of its terms carries an error bar twice that size. The ratio is not
measuring the generator at this sample size, it is amplifying the noise in both.
The 0.38 that appeared in a single run was meaningless, as was reporting it.
What can be said: the ceiling now sits reliably ABOVE the floor, with zero
variance across runs, where before the epic it landed underneath and the ratio
was not computable at all. That is a change in kind and it is solid. The
magnitude is not.
The trustworthy measurement in this harness is the DECOMPOSITION, not the
top-line ratio. `decompose` has a noise floor of exactly 0.0000 and effects an
order of magnitude larger than any drift, because it compares renders directly
rather than dividing differences of aggregates. Steer by that.
Coupling did not move: -0.02 +/-0.03. Whether a song LOOKS different in
proportion to how it SOUNDS different remains unsolved, and nothing in the
migration addressed it.
Read that with its caveat: the stage set changed underneath the measurement, so
part of the jump is that Metaballs expresses a cast more strongly than
Constellation did rather than that the migration itself moved anything. What it
does establish is the thing worth knowing — a migrated LIBRARY scene carries the
identity better than a stage written from scratch to carry it. The bespoke
stages were the wrong shape of effort. As scenes migrate, the
`identity only` number should climb while `container only` holds — because the
whole point of the correction in EPIC-3 §9b is that these add rather than trade.
If `container only` falls as scenes migrate, the migration is homogenising the
library and should stop.
---
## A caution learned the expensive way
This harness's noise floor is large enough to invent findings. Three claims in
this epic were made from single runs and withdrawn after repeats: that roster
size was the dominant lever, that stages beat legacy by 14%, and that the
identity's range was the bottleneck. Anything under about 0.01 of spread needs
`&repeats=3` before it is believed, and a difference that changes sign with the
sample size is not a difference.
Migrate in batches, measure after each batch, and expect the per-batch effect to
be inside the noise. The trend across batches is the signal.

61
flow-state/README.md
View File

@ -26,6 +26,25 @@ Analysing the whole track up front also buys the thing a causal analyser cannot
a build can *anticipate* its drop and arrive at the transition already at full a build can *anticipate* its drop and arrive at the transition already at full
tension, instead of reacting once the drop has landed. tension, instead of reacting once the drop has landed.
## The story
Everything above happens at a moment. On top of it the track gets a **story**: a
plot chosen per song, four **moments** found in the audio (arrival, turn, climax,
resolution), and a position for every section — *which* of the four drops this
is, not just that it is a drop.
That position moves what the video does. The kind's anchor scene is reserved for
the climax rather than opening every section of its kind; the dials ratchet with
each recurrence; the cutting rate follows tension and the resolution holds; the
song's cast arrives over the video instead of being fully stated in the first
shot; and under a *return* plot the outro re-casts what the intro opened on.
The story never overrides the song. Its moments are read off the section
energies, and its effect on parameter bias is bounded so it can decide which
drop this is and never whether a breakdown is one. `D` shows the current plot,
act and journey; the timeline marks the moments. `EPIC-4.md` has the design.
## Working with it ## Working with it
| | | | | |
@ -45,7 +64,7 @@ under the playhead; **lock** protects a section from further rerolls. Every
parameter the generator chose is exposed under the *scene* tab and can be edited parameter the generator chose is exposed under the *scene* tab and can be edited
live. live.
The **click track** button (look tab) mixes an audible click onto the detected beat The **metronome** button (look tab) mixes an audible click onto the detected beat
grid. If the clicks don't sit on the beat, tempo detection is wrong and everything grid. If the clicks don't sit on the beat, tempo detection is wrong and everything
downstream inherits it — check this first when a track looks off. downstream inherits it — check this first when a track looks off.
@ -56,9 +75,43 @@ npm test # audio pipeline against synthetic ground truth
npm run lint:scenes # determinism grep + scene schema/shader agreement npm run lint:scenes # determinism grep + scene schema/shader agreement
``` ```
`http://localhost:5180/checks.html` runs the GPU gates for every phase. Add `http://localhost:5180/filmstrip.html` renders every song in the bank as one frame
every 30 seconds, so a video that is busy and going nowhere is visible as a row of
interchangeable stills. `http://localhost:5180/checks.html` runs the GPU gates for
every phase. Add
`?slow=1` for the full suite, `?phase=5` for one phase. `?slow=1` for the full suite, `?phase=5` for one phase.
### Seed variety test
Every other phase asks whether one video is correct. Phase 12 asks whether two
videos are *different* — the one failure the rest of the suite cannot see, since
a generator that ignores its seed passes determinism and liveness perfectly.
```
http://localhost:5180/checks.html?variety=1
```
Frames are reduced to a structural descriptor that is deliberately blind to
brightness, contrast, hue and rotation, and sensitive to feature scale,
orientation structure, composition, texture statistics and motion. Colour is
measured but never counted — its job is to expose the case where two seeds
differ only in palette. The score sits between two references the same
instrument produced: the **floor** is how far one video travels from itself
across its own sections, the **ceiling** is the same pipeline with every layer
recast at random. Two checks gate the instrument itself before any number from
it is trusted.
The report also prints **direction**: the rank correlation between how far apart two
probes are in time and how far apart they look. `drift` cannot tell a video that
travels from one that merely keeps changing, and a video with a story raises it on
purpose — so read the two together. A floor that rises *with* direction is an arc; a
floor that rises without one is a shuffle.
The report also sweeps **all** visualizations in the library, every pair, and
names the structural twins — different scenes that are the same look in
different colours, which the per-scene `distinct` gate cannot catch because it
compares raw pixels.
## Adding a scene ## Adding a scene
Quick walkthrough: `HOWTO-visualizers.md`. Quick walkthrough: `HOWTO-visualizers.md`.
@ -108,9 +161,9 @@ Scenes that composite over a background rather than being one declare
``` ```
src/ src/
audio/ decode, STFT analysis, tempo, segmentation, FeatureTrack, click track audio/ decode, STFT analysis, tempo, segmentation, FeatureTrack, metronome
engine/ Timeline, Renderer, Layer, Compositor, passes, seeded rng, flash safety engine/ Timeline, Renderer, Layer, Compositor, passes, seeded rng, flash safety
look/ palette (OKLCH), LookGenerator, ArcDriver look/ palette (OKLCH), LookGenerator, ArcDriver, Story
params/ declarative schema, validation, serialisation params/ declarative schema, validation, serialisation
scenes/ the library — shader/ and layers3d/ scenes/ the library — shader/ and layers3d/
export/ WebCodecs exporter export/ WebCodecs exporter

209
flow-state/debug.html Normal file
View File

@ -0,0 +1,209 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>flow-state · debug</title>
<style>
:root { color-scheme: dark; }
body {
margin: 0; padding: 24px 24px 80px;
background: #0b0d12; color: #d6dae3;
font: 13px/1.6 ui-monospace, SFMono-Regular, Menlo, monospace;
}
h1 { font-size: 15px; letter-spacing: .12em; text-transform: uppercase; color: #7d8698; margin: 0 0 6px; }
h2 { font-size: 12px; letter-spacing: .1em; text-transform: uppercase; color: #6b7280; margin: 26px 0 10px; font-weight: 600; }
p.lede { color: #8b94a7; max-width: 80ch; margin: 0 0 8px; }
a { color: #7dd3fc; text-decoration: none; }
a:hover { text-decoration: underline; }
.grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(330px, 1fr)); gap: 10px; }
.card {
background: #11141b; border: 1px solid #1c2030; border-radius: 4px;
padding: 12px 14px; border-left: 3px solid #2a3040;
}
.card.slow { border-left-color: #eab308; }
.card.fast { border-left-color: #22c55e; }
.card.cli { border-left-color: #6366f1; }
.card h3 { margin: 0 0 4px; font-size: 13px; }
.card p { margin: 0 0 8px; color: #8b94a7; font-size: 12px; }
.card .meta { color: #565f70; font-size: 11px; }
code, pre {
background: #080a0f; border: 1px solid #1a1e28; border-radius: 3px;
padding: 1px 5px; color: #b8c0d0; font-size: 12px;
}
pre { padding: 8px 10px; overflow-x: auto; margin: 6px 0 0; }
.note { color: #6b7280; font-size: 12px; max-width: 80ch; }
.warn { color: #eab308; }
</style>
</head>
<body>
<h1>flow-state · debug</h1>
<p class="lede">
Everything here answers a question the app itself cannot. The app shows you
one video; these show you whether the generator is doing its job across
many.
</p>
<h2>Looking at output</h2>
<div class="grid">
<div class="card fast">
<h3><a href="/gallery.html">gallery</a></h3>
<p>Every visualizer, six times, on six different songs' content — cast,
ink, lattice, palette and parameters all varying. Sorted least-varied
first, so the scenes that always look the same come to the top.</p>
<div class="meta">~3 min · the one to open when a scene feels familiar ·
<code>HOWTO-variety.md</code> for how to fix what it finds</div>
</div>
<div class="card fast">
<h3><a href="/gallery.html?scene=Effigy&amp;songs=6&amp;seeds=6">one scene, wide</a></h3>
<p>The gallery for a single visualizer, on a grid of songs (rows) against
seeds (columns) — up to every song in the bank by ten draws. Six
thumbnails ranks a library; it is far too few to study one scene.
Scores the two axes separately, which is the thing the library
gallery cannot tell you: <strong>across seeds</strong> low means the
scene ignores the identity, <strong>across songs</strong> low means it
ignores the music. The closest pair in the grid is outlined.</p>
<div class="meta">seconds, after the songs are analysed once ·
click any scene name in the gallery to open it here ·
the buttons change grid size and walk onto fresh seeds</div>
</div>
<div class="card">
<h3><a href="/filmstrip.html">filmstrip</a></h3>
<p>Every song in the bank, one frame every 30 seconds, left to right.
The gallery asks whether a scene looks the same in every song; this
asks whether a song looks the same as itself four minutes later.
Scored on <code>direction</code> — whether the distance between two
frames grows with the time between them — so the videos that are busy
and going nowhere sort to the top.</p>
<div class="meta">~6 min, then cached · <a href="/filmstrip.html?every=15">?every=15</a> tighter ·
<a href="/filmstrip.html?song=centre,ember">?song=</a> just these ·
<code>EPIC-4.md</code> for the arc it is checking</div>
</div>
<div class="card fast">
<h3><a href="/">the app</a></h3>
<p>Drop a track and watch it. <code>test/songs/*.wav</code> holds the
synthetic bank if you want something with known properties.</p>
<div class="meta">build the bank with <code>npm run build:songs</code></div>
</div>
</div>
<h2>Is it correct?</h2>
<div class="grid">
<div class="card fast">
<h3><a href="/checks.html">phase gates</a></h3>
<p>Every acceptance gate in <code>PLAN.md</code>, run for real. This is
the one to check before committing.</p>
<div class="meta">
<a href="/checks.html?slow=1">?slow=1</a> full suite ·
<a href="/checks.html?phase=12">?phase=12</a> one phase
</div>
</div>
<div class="card fast">
<h3>single scene</h3>
<p>The whole per-scene battery for one visualizer: renders, animates,
deterministic, distinct, param sweep, flash rate, and every declared
trait and artifact.</p>
<div class="meta"><code>checks.html?scene=Metaballs</code> — the loop you
are in while writing or migrating one</div>
</div>
</div>
<h2>Is it varied?</h2>
<p class="note">
These measure whether two videos differ, which no other gate can see — a
generator that ignores its input passes determinism and liveness perfectly.
<span class="warn">Read the caution below before trusting a single run.</span>
</p>
<div class="grid">
<div class="card slow">
<h3><a href="/checks.html?decompose=1">decompose</a></h3>
<p>Identity against container, measured apart: hold the scene fixed and
vary the song's content, then the reverse. <strong>The most
trustworthy number here</strong> — it compares renders directly, and
its noise floor is exactly zero.</p>
<div class="meta">~1 min</div>
</div>
<div class="card slow">
<h3><a href="/checks.html?songs=1&amp;count=12">song variety</a></h3>
<p>Twelve songs from the bank, each with its own audio-derived seed. Also
reports coupling — whether songs that <em>sound</em> different come
out <em>looking</em> different, which is still unsolved.</p>
<div class="meta">~2 min · <a href="/checks.html?variety=1">?variety=1</a>
for the seed version, one song and many seeds</div>
</div>
<div class="card slow">
<h3><a href="/checks.html?experiment=1&amp;count=12&amp;repeats=3">arms, with error bars</a></h3>
<p>Three pools compared over repeated runs. The template for any A/B
here: it prints the difference against its own noise band and says
outright when the two are indistinguishable.</p>
<div class="meta">~5 min</div>
</div>
<div class="card slow">
<h3><a href="/checks.html?sweep=1&amp;count=12&amp;sizes=4,8,16,32&amp;repeats=3">pool sweep</a></h3>
<p>How many scenes a track should draw on. Swept rather than guessed —
and the answer turned out to be that it does not matter between 4 and
32.</p>
<div class="meta">~5 min</div>
</div>
</div>
<h2>Command line</h2>
<div class="grid">
<div class="card cli">
<h3>migration status</h3>
<p>Which scenes draw the song's content, which still draw their own, and
what each remaining one needs. Recipe in <code>MIGRATION.md</code>.</p>
<pre>node tools/migration-status.js</pre>
</div>
<div class="card cli">
<h3>cast census</h3>
<p>Which visualizers actually get cast, across songs and seeds — and for
the ones that do not, which of the four gates killed them.</p>
<pre>node tools/cast-census.js 12 40</pre>
</div>
<div class="card cli">
<h3>identity census</h3>
<p>How far apart the songs' identities are before anything is rendered.
No GPU. Run this before blaming the visualizers.</p>
<pre>node tools/identity-census.js 12</pre>
</div>
<div class="card cli">
<h3>song bank</h3>
<p>Rebuild the test songs and verify they still span every feature the
generator reads. Fails loudly if an axis has collapsed.</p>
<pre>npm run build:songs
npm run check:songs</pre>
</div>
<div class="card cli">
<h3>static gates</h3>
<p>Determinism grep, schema/shader agreement both ways, and the backtick
check that guards the shader template literals.</p>
<pre>npm run lint:scenes
npm test</pre>
</div>
</div>
<h2>Reading the variety numbers</h2>
<p class="note">
This harness has two classes of measurement and only one of them is safe to
steer by. <strong>Direct render comparisons</strong> — the gallery score,
<code>decompose</code>, the per-scene gate — have a noise floor of zero and
mean what they say. <strong>Aggregate ratios</strong> — chiefly
<code>separation</code>, which divides one small difference by another —
swing wildly between runs: three runs of the same measurement gave 0.31,
0.56 and 0.07. Four conclusions were drawn and withdrawn during this epic
for exactly that reason. Anything under about 0.01 of spread needs
<code>&amp;repeats=3</code> before it is believed, and a difference that
changes sign with the sample is not a difference.
</p>
<h2>If a page is dead</h2>
<p class="note">
A module that fails to load takes the whole graph with it: the page renders,
every control is present, and nothing is wired to anything. The app carries a
boot guard that says so. The usual causes are a content blocker — a file
named like tracking will be treated as tracking, which cost this project a
session over a file called <code>clicktrack.js</code> — or a dev server
caught mid-restart.
</p>
</body>
</html>

279
flow-state/filmstrip.html Normal file
View File

@ -0,0 +1,279 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>flow-state · filmstrip</title>
<style>
:root { color-scheme: dark; }
body {
margin: 0; padding: 20px 24px 60px;
background: #0b0d12; color: #d6dae3;
font: 13px/1.5 ui-monospace, SFMono-Regular, Menlo, monospace;
}
h1 { font-size: 15px; letter-spacing: .12em; text-transform: uppercase; color: #7d8698; margin: 0 0 4px; }
a { color: #7dd3fc; }
.lede { color: #8b94a7; max-width: 84ch; margin: 0 0 14px; }
.bar { position: sticky; top: 0; z-index: 5; background: #0b0d12; padding: 10px 0 12px; border-bottom: 1px solid #1c2030; margin-bottom: 16px; }
button, select {
background: #171b24; color: #d6dae3; border: 1px solid #2a3040;
padding: 5px 10px; font: inherit; border-radius: 3px; cursor: pointer;
}
button:hover:not(:disabled) { border-color: #4a5468; }
button:disabled { opacity: .5; cursor: default; }
#status { color: #8b94a7; margin-left: 10px; }
#status.cached { color: #4ade80; }
.spinner {
display: none; width: 13px; height: 13px; vertical-align: -2px;
border: 2px solid #2a3040; border-top-color: #7dd3fc; border-radius: 50%;
animation: spin .8s linear infinite; margin-right: 6px;
}
.building .spinner { display: inline-block; }
@keyframes spin { to { transform: rotate(360deg); } }
#progress { height: 2px; background: #1a1e28; margin-top: 8px; border-radius: 2px; overflow: hidden; display: none; }
.building #progress { display: block; }
#progress i { display: block; height: 100%; width: 0; background: #7dd3fc; transition: width .2s; }
.row { margin-bottom: 22px; border-left: 3px solid #232838; padding-left: 12px; }
.row.static { border-color: #ef4444; }
.row.churn { border-color: #eab308; }
.row.arc { border-color: #22c55e; }
.row.broken { border-color: #a855f7; background: #150e1c; }
.head { display: flex; align-items: baseline; gap: 12px; margin-bottom: 6px; flex-wrap: wrap; }
.name { font-size: 14px; color: #e6eaf2; }
.meta { color: #6b7280; font-size: 11px; }
.num { color: #9aa3b5; }
.num b { color: #d6dae3; font-weight: normal; }
.story { color: #a5b4fc; font-size: 11px; }
.watch { font-size: 11px; text-decoration: none; }
.watch:hover { text-decoration: underline; }
.err { color: #f87171; }
/* The strip scrolls sideways rather than wrapping: a filmstrip that wraps
stops being a filmstrip, and reading it left to right IS the measurement. */
.strip { display: flex; gap: 4px; overflow-x: auto; padding-bottom: 6px; }
.strip figure { margin: 0; flex: 0 0 auto; width: 168px; }
.strip canvas { width: 100%; display: block; background: #05070a; border-radius: 2px; aspect-ratio: 16 / 9; }
.strip figcaption { font-size: 10px; color: #565f70; margin-top: 3px; line-height: 1.35; }
.strip .t { color: #8b94a7; }
.strip .sc { display: block; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.moment { color: #fbbf24; }
/* The climax is the one frame worth finding at a glance. */
figure.climax canvas { outline: 2px solid #fbbf24; outline-offset: -2px; }
</style>
</head>
<body>
<h1>flow-state · filmstrip</h1>
<p class="lede">
Every song in the bank, one frame every <span id="everyLabel">30</span> seconds,
left to right. The gallery asks whether a scene looks the same in every song;
this asks whether a song looks the same as <em>itself</em> four minutes later.
A strip whose frames could be shuffled without anyone noticing is a video with
no arc, however busy it is.
<strong>drift</strong> is how far the video gets from itself and
<strong>direction</strong> is whether that distance grows with time — churn
scores the first and not the second. The climax frame is outlined.
<a href="/debug.html">← all debug tools</a>
</p>
<div class="bar" id="bar">
<span class="spinner"></span>
<button id="rebuild">rebuild</button>
<select id="sort">
<option value="direction">sort: least direction first</option>
<option value="drift">sort: least drift first</option>
<option value="name">sort: bank order</option>
<option value="plot">sort: by plot</option>
</select>
<span id="status">checking cache…</span>
<div id="progress"><i></i></div>
</div>
<div id="out"></div>
<script type="module">
import { buildFilmstrip, songNames, STRIP, DEFAULT_EVERY, DEFAULT_DURATION } from '/src/checks/filmstrip.js';
import { blit } from '/src/checks/blit.js';
import { hashString } from '/src/engine/rng.js';
import {
sourceFingerprint, loadGallery, saveGallery,
pixelsToBlob, blobToCanvas,
} from '/src/checks/gallery-cache.js';
const params = new URLSearchParams(location.search);
const every = Number(params.get('every')) || DEFAULT_EVERY;
const duration = Number(params.get('duration')) || DEFAULT_DURATION;
const only = params.get('song') ? params.get('song').split(',') : null;
document.getElementById('everyLabel').textContent = every;
const bar = document.getElementById('bar');
const out = document.getElementById('out');
const status = document.getElementById('status');
const sortSel = document.getElementById('sort');
const progress = document.querySelector('#progress i');
const rebuildBtn = document.getElementById('rebuild');
let rows = [];
// Read against the arcless reference the variety report prints: around zero is
// a video that is as unlike itself after ten seconds as after four minutes.
const grade = (row) => (row.error ? 'broken'
: row.direction < 0.15 ? 'static'
: row.direction < 0.45 ? 'churn' : 'arc');
// Nine stills are an argument about a video, and the only reply to them is the
// video. The app takes the bank's wav straight from `?song=`, and the seed is
// the one renderSong() uses — hashed off the name — so what plays is the take
// the strip measured rather than a fresh draw that happens to share a title.
const watchLink = (name) =>
`/?song=${encodeURIComponent(name)}&seed=${hashString(name)}`;
/**
* One song's row.
*
* Shots arrive as raw pixels while the build is running and as compressed blobs
* once it has finished (and on every later sort, out of the cache). Both are
* drawn here rather than in two places, because the whole point of painting
* during the build is that what you see then is what you keep.
*/
function renderRow(row) {
const el = document.createElement('div');
el.className = `row ${grade(row)}`;
el.innerHTML = `
<div class="head">
<span class="name">${row.name}</span>
<a class="watch" href="${watchLink(row.name)}" target="_blank"
title="play this song in the app, on the seed this strip was rendered with">watch ▸</a>
<span class="meta">${row.bpm ?? '?'} bpm · ${row.sections ?? '?'} sections · ${row.director || ''}</span>
<span class="num">direction <b>${(row.direction ?? 0).toFixed(2)}</b> · drift <b>${(row.drift ?? 0).toFixed(3)}</b></span>
<span class="story">${row.error ? `<span class="err">${row.error}</span>` : (row.storyLine || '')}</span>
</div>
<div class="strip"></div>`;
const strip = el.querySelector('.strip');
for (const shot of row.shots || row.frames || []) {
const fig = document.createElement('figure');
if (shot.moment === 'climax') fig.className = 'climax';
const canvas = document.createElement('canvas');
fig.appendChild(canvas);
const cap = document.createElement('figcaption');
const mm = Math.floor(shot.time / 60);
const ss = String(Math.round(shot.time % 60)).padStart(2, '0');
cap.innerHTML =
`<span class="t">${mm}:${ss}</span> ${shot.kind}` +
(shot.moment ? ` <span class="moment">${shot.moment}</span>` : '') +
`<span class="sc">${shot.scene}</span>` +
`<span class="sc">j ${shot.journey.toFixed(2)} · t ${shot.tension.toFixed(2)}</span>`;
fig.appendChild(cap);
strip.appendChild(fig);
if (shot.pixels) {
// Straight from the render, mid-build. Synchronous and cheap — it is
// one song's nine frames, not the whole bank's.
blit(canvas, shot.pixels, STRIP.width, STRIP.height);
} else {
// Decoded lazily: seventeen songs of nine frames is 150 images and
// decoding them synchronously stalls the page.
blobToCanvas(canvas, shot.blob);
}
}
return el;
}
function draw() {
const mode = sortSel.value;
const order = songNames();
const sorted = rows.slice().sort((a, b) => (
mode === 'name' ? order.indexOf(a.name) - order.indexOf(b.name)
: mode === 'drift' ? a.drift - b.drift
: mode === 'plot' ? ((a.story?.plot || '').localeCompare(b.story?.plot || '') || a.direction - b.direction)
: a.direction - b.direction));
out.innerHTML = '';
for (const row of sorted) out.appendChild(renderRow(row));
}
sortSel.addEventListener('change', draw);
function summarise(built) {
const flat = rows.filter((r) => !r.error && r.direction < 0.15).length;
const meanDir = rows.length
? rows.reduce((a, r) => a + (r.direction || 0), 0) / rows.length : 0;
return `${rows.length} songs · every ${every}s of ${duration}s · ` +
`mean direction ${meanDir.toFixed(2)} · ${flat} with no direction` +
(built ? ` · built ${new Date(built).toLocaleString()}` : '');
}
// The cache key carries the sampling parameters as well as the source, so
// changing `?every=` does not silently show you the previous spacing's pixels.
const cacheKey = () => `filmstrip:${every}:${duration}:${only ? only.join(',') : 'all'}:${sourceFingerprint()}`;
async function build() {
bar.classList.add('building');
rebuildBtn.disabled = true;
rows = [];
out.innerHTML = '';
const total = (only || songNames()).length;
const built = await buildFilmstrip({
duration, every, names: only,
// Painted as they land, in bank order, rather than after the whole run.
// A full build is a couple of minutes of GPU work and the interesting
// failure — a strip whose frames could be shuffled — is visible in the
// first row. Waiting for the seventeenth to look at the first is a
// needlessly slow way to find that out. Re-sorted once at the end, when
// there is finally something to sort by.
onSongStart: (done, of, name) => {
status.textContent = `rendering ${name}… (${done}/${of})`;
progress.style.width = `${((done - 1) / of) * 100}%`;
},
onSong: (done, of, row) => {
progress.style.width = `${(done / of) * 100}%`;
out.appendChild(renderRow(row));
},
});
// Compress once, here, rather than holding 150 raw frames: the strips are
// stored and redrawn from these blobs on every sort.
for (const row of built) {
rows.push({
...row,
frames: undefined,
shots: await Promise.all((row.frames || []).map(async (f) => ({
time: f.time, kind: f.kind, scene: f.scene, act: f.act,
tension: f.tension, journey: f.journey, moment: f.moment,
blob: await pixelsToBlob(f.pixels, STRIP.width, STRIP.height),
}))),
});
}
bar.classList.remove('building');
rebuildBtn.disabled = false;
const stamp = Date.now();
status.textContent = summarise(stamp);
status.className = '';
draw();
await saveGallery(cacheKey(), { built: stamp, rows });
return total;
}
async function load() {
const cached = await loadGallery(cacheKey());
if (cached && cached.rows && cached.rows.length) {
rows = cached.rows;
status.textContent = summarise(cached.built);
status.className = 'cached';
draw();
return;
}
await build();
}
rebuildBtn.addEventListener('click', () => build());
// Dev handle, so a headless run can wait for the build and read the numbers
// without scraping the DOM.
window.__FILMSTRIP__ = { get rows() { return rows; } };
load().then(() => { window.__FILMSTRIP_DONE__ = true; });
</script>
</body>
</html>

562
flow-state/gallery.html Normal file
View File

@ -0,0 +1,562 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>flow-state · gallery</title>
<style>
:root { color-scheme: dark; }
body {
margin: 0; padding: 20px 24px 60px;
background: #0b0d12; color: #d6dae3;
font: 13px/1.5 ui-monospace, SFMono-Regular, Menlo, monospace;
}
h1 { font-size: 15px; letter-spacing: .12em; text-transform: uppercase; color: #7d8698; margin: 0 0 4px; }
a { color: #7dd3fc; }
.lede { color: #8b94a7; max-width: 78ch; margin: 0 0 14px; }
.bar { position: sticky; top: 0; z-index: 5; background: #0b0d12; padding: 10px 0 12px; border-bottom: 1px solid #1c2030; margin-bottom: 16px; }
button, select {
background: #171b24; color: #d6dae3; border: 1px solid #2a3040;
padding: 5px 10px; font: inherit; border-radius: 3px; cursor: pointer;
}
button:hover:not(:disabled) { border-color: #4a5468; }
button:disabled { opacity: .5; cursor: default; }
#status { color: #8b94a7; margin-left: 10px; }
#status.cached { color: #4ade80; }
.ctx { display: flex; gap: 8px; flex-wrap: wrap; margin: 10px 0 0; font-size: 11px; color: #6b7280; }
.ctx span { background: #11141b; padding: 3px 7px; border-radius: 3px; }
/* Building blocks the main thread in bursts, so the spinner is a CSS
animation — it keeps turning through the GPU work where a JS-driven one
would freeze and look like a hang. */
.spinner {
display: none; width: 13px; height: 13px; vertical-align: -2px;
border: 2px solid #2a3040; border-top-color: #7dd3fc; border-radius: 50%;
animation: spin .8s linear infinite; margin-right: 6px;
}
.building .spinner { display: inline-block; }
@keyframes spin { to { transform: rotate(360deg); } }
#progress { height: 2px; background: #1a1e28; margin-top: 8px; border-radius: 2px; overflow: hidden; display: none; }
.building #progress { display: block; }
#progress i { display: block; height: 100%; width: 0; background: #7dd3fc; transition: width .2s; }
.row { margin-bottom: 22px; border-left: 3px solid #232838; padding-left: 12px; }
.row.flat { border-color: #ef4444; }
.row.thin { border-color: #eab308; }
.row.good { border-color: #22c55e; }
.row.broken { border-color: #a855f7; background: #150e1c; }
.head { display: flex; align-items: baseline; gap: 12px; margin-bottom: 6px; flex-wrap: wrap; }
.name { font-size: 14px; color: #e6eaf2; }
.fam { color: #6b7280; }
.score { color: #9aa3b5; }
.meter { position: relative; display: inline-block; width: 130px; height: 7px; background: #1a1e28; border-radius: 4px; vertical-align: middle; }
.meter i { display: block; height: 100%; background: #4ade80; border-radius: 4px 0 0 4px; }
/* The bar, on every meter, so a row can be read on its own. */
.meter b { position: absolute; top: -2px; bottom: -2px; width: 2px; background: #ef4444; }
.row.flat .meter i { background: #ef4444; }
.row.thin .meter i { background: #eab308; }
.blocks { color: #5c6577; font-size: 11px; }
/* Coverage sits next to the score because the two explain each other: a
scene painting 2% of the frame scores well for variety and is thin to
watch alone. */
.surf { font-size: 11px; padding: 1px 6px; border-radius: 3px; }
.surf.canvas { background: #16232b; color: #7dd3fc; }
.surf.composable { background: #2a1f30; color: #d8b4fe; }
.thumbs { display: grid; grid-template-columns: repeat(6, 1fr); gap: 6px; }
.thumbs figure { margin: 0; }
.thumbs canvas { width: 100%; display: block; background: #05070a; border-radius: 2px; aspect-ratio: 16 / 9; }
.thumbs figcaption { font-size: 10px; color: #565f70; margin-top: 3px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.err { color: #f87171; }
.cutline { position: relative; border-top: 2px solid #ef4444; margin: 26px 0 24px; }
.cutline span {
position: absolute; top: -9px; left: 12px; background: #0b0d12;
padding: 0 8px; color: #ef4444; font-size: 11px; letter-spacing: .04em;
}
@media (max-width: 1100px) { .thumbs { grid-template-columns: repeat(3, 1fr); } }
/* ---- single-scene sweep --------------------------------------------- */
.name a { color: inherit; text-decoration: none; border-bottom: 1px dotted #3a4256; }
.name a:hover { color: #7dd3fc; }
.btns { display: inline-flex; gap: 4px; margin-left: 4px; }
.btns button.on { border-color: #7dd3fc; color: #7dd3fc; }
.sweep { display: grid; gap: 5px; align-items: center; margin-top: 8px; }
.sweep .colhead { font-size: 10px; color: #565f70; text-align: center; }
.sweep .rowlabel {
font-size: 11px; color: #8b94a7; text-align: right;
padding-right: 8px; white-space: nowrap;
}
.sweep .cell { position: relative; line-height: 0; }
.sweep canvas {
width: 100%; display: block; background: #05070a;
border-radius: 2px; aspect-ratio: 16 / 9;
}
/* The two cells that measured closest — on a grid this wide they are the
only pair an eye has no chance of finding on its own. */
.sweep .cell.twin canvas { outline: 2px solid #ef4444; outline-offset: 1px; }
.sweep .cell.twin::after {
content: 'twin'; position: absolute; top: 2px; left: 3px;
font-size: 9px; color: #ef4444; line-height: 1;
}
.axes { display: flex; gap: 18px; flex-wrap: wrap; margin: 10px 0 2px; }
.axes div { font-size: 12px; }
.axes b { color: #e6eaf2; font-weight: normal; }
.axes .weak b { color: #ef4444; }
.axes small { color: #6b7280; display: block; font-size: 11px; }
</style>
</head>
<body>
<h1>flow-state · gallery</h1>
<p class="lede">
Every visualizer rendered six times, once per song, each with that song's
complete identity — its cast, ink, lattice, palette and sampled parameters.
A scene whose six frames are interchangeable is one the song cannot change.
Sorted least-varied first, scored on the same structural descriptor the
variety harness uses, with colour excluded so six palettes cannot disguise
one image.
<strong>Coverage</strong> is shown beside each score because the two explain
each other: the highest-scoring scenes are often a few bright elements on
black, which is varied and thin to watch on its own. Those are
<em>composable</em> — they are meant to sit on top of a <em>canvas</em>.
<strong>Click a scene's name</strong> to open it on its own, across a much
wider grid of songs and seeds — six frames ranks a library and is far too
few to study one scene.
<a href="/debug.html">← all debug tools</a>
</p>
<div class="bar" id="bar">
<span class="spinner"></span>
<span id="libctl">
<button id="rebuild">rebuild</button>
<button id="refresh" title="re-measure the library and write src/scenes/metadata.json — the git-tracked numbers the generator composes with">refresh metadata</button>
<select id="sort">
<option value="variety">sort: least varied first</option>
<option value="name">sort: by name</option>
<option value="family">sort: by family</option>
<option value="coverage">sort: sparsest first</option>
</select>
</span>
<span id="sweepctl" hidden>
<button id="back">← all scenes</button>
<select id="scenesel"></select>
<span class="btns" id="sizes"></span>
<span class="btns">
<button id="prevseeds" title="the same grid on the previous block of seeds">← seeds</button>
<button id="nextseeds" title="the same grid on the next block of seeds — the answer to &quot;is it flat, or was that ten unlucky draws&quot;">seeds →</button>
<button id="rerender" title="render this grid again">re-render</button>
</span>
</span>
<span id="status">checking cache…</span>
<div class="ctx" id="ctx"></div>
<div id="progress"><i></i></div>
</div>
<div id="sweepinfo"></div>
<div id="out"></div>
<script type="module">
import { buildGallery, galleryContexts, THUMB, blit } from '/src/checks/gallery.js';
import { measureLibrary, metadataDrift, writeMetadata, metadataIsFresh } from '/src/checks/metadata.js';
// A gallery showing numbers the generator is not using is a trap, so say so.
const STALE_NOTE = metadataIsFresh() ? '' : ' · metadata is STALE — press refresh';
import {
sourceFingerprint, loadGallery, saveGallery, clearGallery,
pixelsToBlob, blobToCanvas,
} from '/src/checks/gallery-cache.js';
import { sweepGrid, sweepScene, prepareSongs, SWEEP_SIZES } from '/src/checks/scene-sweep.js';
import { Engine } from '/src/engine/Engine.js';
import { scenes } from '/src/scenes/registry.js';
const bar = document.getElementById('bar');
const out = document.getElementById('out');
const status = document.getElementById('status');
const sortSel = document.getElementById('sort');
const progress = document.querySelector('#progress i');
const rebuildBtn = document.getElementById('rebuild');
const refreshBtn = document.getElementById('refresh');
let rows = []; // { name, family, role, variety, byBlock, blobs, error }
let contexts = [];
// There is no bar any more — see checks/gallery.js for why the 0.1 red line
// went. What is left is a broken/working distinction and a relative shading, so
// the eye can still find the flattest scenes in a sorted list without anyone
// pretending a threshold decides whether a scene is allowed to exist.
const grade = (v, err) => (err ? 'broken' : v < 0.08 ? 'thin' : 'good');
function draw() {
const mode = sortSel.value;
const sorted = rows.slice().sort((a, b) => (
mode === 'name' ? a.name.localeCompare(b.name)
: mode === 'coverage' ? (a.coverage || 0) - (b.coverage || 0)
: mode === 'family' ? (a.family.localeCompare(b.family) || a.variety - b.variety)
: a.variety - b.variety));
out.innerHTML = '';
for (const row of sorted) {
const el = document.createElement('div');
el.className = `row ${grade(row.variety, row.error)}`;
const blocks = Object.entries(row.byBlock || {})
.map(([k, v]) => `${k} ${v.toFixed(3)}`).join(' · ');
el.innerHTML = `
<div class="head">
<span class="name"><a href="?scene=${encodeURIComponent(row.name)}" title="open this one on a wide grid of songs and seeds">${row.name}</a></span>
<span class="fam">${row.family}${row.role === 'accent' ? ' · accent' : ''}</span>
<span class="surf ${surfaceFor(row)}" title="paints ${((row.coverage || 0) * 100).toFixed(0)}% of the frame across the six songs — this is what decides the label">${surfaceFor(row)} ${((row.coverage || 0) * 100).toFixed(0)}%</span>
<span class="score">
<span class="meter"><i style="width:${Math.min(100, row.variety * 500)}%"></i></span>
${row.variety.toFixed(3)}
</span>
<span class="blocks">${row.error ? `<span class="err">${row.error}</span>` : blocks}</span>
</div>
<div class="thumbs"></div>`;
const grid = el.querySelector('.thumbs');
(row.blobs || []).forEach((blob, i) => {
const fig = document.createElement('figure');
const canvas = document.createElement('canvas');
fig.appendChild(canvas);
const cap = document.createElement('figcaption');
cap.textContent = contexts[i] ? contexts[i].name : '';
cap.title = contexts[i] ? contexts[i].identity : '';
fig.appendChild(cap);
grid.appendChild(fig);
// Decoded lazily and in the background; sixty-five rows of six is
// 390 images and decoding them all synchronously stalls the page.
blobToCanvas(canvas, blob);
});
out.appendChild(el);
}
}
sortSel.addEventListener('change', draw);
// The label a scene WOULD carry from this build's numbers, which is not
// necessarily the one the checked-in metadata gives it — that is the whole
// point of showing it here.
const surfaceFor = (row) => ((row.coverage || 0) >= 0.5 ? 'canvas' : 'composable');
function summarise(built) {
const broken = rows.filter((r) => r.error).length;
const canvases = rows.filter((r) => surfaceFor(r) === 'canvas').length;
return `${rows.length} scenes · ${canvases} canvas / ${rows.length - canvases} composable` +
(broken ? ` · ${broken} BROKEN` : '') +
` · ${rows.filter((r) => (r.coverage || 0) < 0.3).length} paint under 30% of the frame` +
(built ? ` · built ${new Date(built).toLocaleString()}` : '') + STALE_NOTE;
}
async function build(fingerprint) {
bar.classList.add('building');
rebuildBtn.disabled = true;
rows = [];
out.innerHTML = '';
status.className = '';
status.textContent = 'analysing six songs…';
await new Promise((r) => setTimeout(r, 0));
contexts = galleryContexts(6).map((c) => ({ ...c }));
document.getElementById('ctx').innerHTML = contexts
.map((c) => `<span title="${c.identity}">${c.name}</span>`).join('');
const started = Date.now();
await buildGallery({
contexts,
onScene: async (done, total, name, row) => {
status.textContent = `rendering ${done}/${total} · ${name}`;
progress.style.width = `${(done / total) * 100}%`;
// Show rows as they land. The caching rewrite moved every draw to
// the end, which left three minutes of spinner and nothing to look
// at — and the first scenes are the interesting ones, since the
// sort puts the repetitive ones on top.
rows.push({
name: row.module.name, family: row.module.family,
role: row.module.role || 'stage', variety: row.variety,
coverage: row.coverage, surface: row.surface,
byBlock: row.byBlock, error: row.error,
blobs: await Promise.all(row.thumbs.map(
(px) => pixelsToBlob(px, THUMB.width, THUMB.height))),
});
if (done % 8 === 0 || done === total) draw();
},
});
const payload = {
fingerprint,
built: Date.now(),
contexts: contexts.map((c) => ({ name: c.name, identity: c.identity })),
rows,
};
const stored = await saveGallery(fingerprint, payload);
bar.classList.remove('building');
rebuildBtn.disabled = false;
draw();
status.textContent = `${summarise(payload.built)} in ${((Date.now() - started) / 1000).toFixed(0)}s` +
(stored ? '' : ' · not cached (storage unavailable)');
window.__GALLERY__ = rows.map((r) => ({ name: r.name, variety: r.variety, error: r.error }));
}
// ---- single-scene sweep -------------------------------------------------
// One scene over a grid of songs (rows) and seeds (columns). See
// src/checks/scene-sweep.js for what the two axes are for.
const sweepInfo = document.getElementById('sweepinfo');
const sceneSel = document.getElementById('scenesel');
const sizesEl = document.getElementById('sizes');
/** The sweep's whole state lives in the URL, so every button is a back button. */
function sweepState() {
const p = new URLSearchParams(location.search);
return {
scene: p.get('scene'),
songs: Number(p.get('songs')) || 6,
seeds: Number(p.get('seeds')) || 6,
offset: Number(p.get('offset')) || 0,
};
}
function goSweep(patch) {
const next = { ...sweepState(), ...patch };
const p = new URLSearchParams({
scene: next.scene, songs: next.songs, seeds: next.seeds, offset: next.offset,
});
history.pushState(null, '', `?${p}`);
renderSweep();
}
let sweepRun = 0; // cancels an in-flight grid when the buttons move on
async function renderSweep() {
const state = sweepState();
const module = scenes.find((m) => m.name === state.scene);
const run = ++sweepRun;
document.getElementById('libctl').hidden = true;
document.getElementById('sweepctl').hidden = false;
document.getElementById('ctx').innerHTML = '';
out.innerHTML = '';
// The page's own lede describes the library view, which is the wrong thing
// to be reading over a grid of one scene.
document.querySelector('.lede').innerHTML =
'One visualizer over a grid of songs and seeds. <strong>Rows are songs</strong> — ' +
'different audio, so different features, sections and biases. ' +
'<strong>Columns are seeds</strong> — the same audio with a different roll of ' +
'the identity, palette and parameters. A flat row means the seed does nothing ' +
'here; a flat column means the song does nothing; the two scores below say ' +
'which. Frames are the busiest section of each look, where scenes are most ' +
'likely to converge. <a href="?">← all scenes</a> · ' +
'<a href="/debug.html">all debug tools</a>';
if (!module) {
sweepInfo.innerHTML = `<p class="err">no scene named "${state.scene}"</p>`;
return;
}
sceneSel.value = module.name;
for (const b of sizesEl.children) {
b.classList.toggle('on', Number(b.dataset.songs) === state.songs
&& Number(b.dataset.seeds) === state.seeds);
}
bar.classList.add('building');
status.className = '';
sweepInfo.innerHTML = '';
// Analysed one at a time so the count moves; cached, so only the first grid
// of a session waits for it.
await prepareSongs(state.songs, (done, total, name) => {
status.textContent = `${module.name}: analysing songs ${done}/${total} · ${name}`;
progress.style.width = `${(done / total) * 100}%`;
});
if (run !== sweepRun) return;
const { cells, songNames, seedLabels } = sweepGrid({
songs: state.songs, seeds: state.seeds, seedOffset: state.offset,
});
if (run !== sweepRun) return;
// The grid is laid out before anything is rendered, so the shape of what is
// coming is visible immediately and each cell fills in where it belongs.
const grid = document.createElement('div');
grid.className = 'sweep';
grid.style.gridTemplateColumns = `auto repeat(${seedLabels.length}, 1fr)`;
grid.appendChild(document.createElement('span'));
for (const label of seedLabels) {
const h = document.createElement('span');
h.className = 'colhead';
h.textContent = label;
grid.appendChild(h);
}
const canvases = [];
cells.forEach((cell, i) => {
if (i % seedLabels.length === 0) {
const label = document.createElement('span');
label.className = 'rowlabel';
label.textContent = cell.song;
grid.appendChild(label);
}
const holder = document.createElement('span');
holder.className = 'cell';
const canvas = document.createElement('canvas');
canvas.width = THUMB.width; canvas.height = THUMB.height;
holder.title = `${cell.name} · seed ${cell.seed}\n${cell.identity}`;
holder.appendChild(canvas);
grid.appendChild(holder);
canvases.push({ canvas, holder });
});
out.appendChild(grid);
const started = Date.now();
const engine = new Engine({ ...THUMB });
let result;
try {
result = await sweepScene({
engine, module, cells,
onCell: async (i, total, pixels) => {
if (run !== sweepRun) return;
blit(canvases[i].canvas, pixels, THUMB.width, THUMB.height);
status.textContent = `${module.name}: ${i + 1}/${total} cells`;
progress.style.width = `${((i + 1) / total) * 100}%`;
// Yield every row, so the grid fills in visibly instead of
// freezing the page for a hundred and seventy renders.
if (i % seedLabels.length === seedLabels.length - 1) {
await new Promise((r) => setTimeout(r, 0));
}
},
});
} finally {
engine.dispose();
}
if (run !== sweepRun) return;
if (result.closest) {
canvases[result.closest.a].holder.classList.add('twin');
canvases[result.closest.b].holder.classList.add('twin');
}
// Three numbers rather than one: which AXIS is flat is the thing the
// library gallery cannot tell you, and it decides what to go and fix.
const weak = (v) => (v < 0.08 ? ' weak' : '');
const twin = result.closest
? `${cells[result.closest.a].name} ≈ ${cells[result.closest.b].name} at ${result.closest.distance.toFixed(3)}`
: '—';
sweepInfo.innerHTML = `
<div class="axes">
<div class="${weak(result.variety)}">overall <b>${result.variety.toFixed(3)}</b>
<small>every cell against every other</small></div>
<div class="${weak(result.bySeed)}">across seeds <b>${result.bySeed.toFixed(3)}</b>
<small>same song, different draw — low means the scene ignores the identity</small></div>
<div class="${weak(result.bySong)}">across songs <b>${result.bySong.toFixed(3)}</b>
<small>same draw, different music — low means the scene ignores the song</small></div>
<div>closest pair <b>${twin}</b>
<small>outlined in the grid</small></div>
</div>
<div class="blocks">${Object.entries(result.byBlock)
.map(([k, v]) => `${k} ${v.toFixed(3)}`).join(' · ')}</div>`;
bar.classList.remove('building');
status.textContent = `${module.name} · ${cells.length} cells ` +
`(${songNames.length} songs × ${seedLabels.length} seeds, from ${seedLabels[0]}) ` +
`in ${((Date.now() - started) / 1000).toFixed(0)}s`;
window.__SWEEP__ = {
scene: module.name, variety: result.variety,
bySeed: result.bySeed, bySong: result.bySong,
};
}
function initSweepControls() {
// Alphabetical, not registry order: the registry is chronological — the
// order scenes were added — which is meaningless to anyone hunting for one
// by name in a list of sixty-seven.
sceneSel.innerHTML = scenes.filter((m) => m.kind === 'fragment')
.map((m) => m.name).sort((a, b) => a.localeCompare(b))
.map((name) => `<option value="${name}">${name}</option>`).join('');
sceneSel.addEventListener('change', () => goSweep({ scene: sceneSel.value, offset: 0 }));
sizesEl.innerHTML = SWEEP_SIZES.map((s) =>
`<button data-songs="${s.songs}" data-seeds="${s.seeds}" ` +
`title="${s.songs} songs × ${s.seeds} seeds = ${s.songs * s.seeds} renders">${s.label}</button>`).join('');
sizesEl.addEventListener('click', (e) => {
const b = e.target.closest('button');
if (b) goSweep({ songs: Number(b.dataset.songs), seeds: Number(b.dataset.seeds) });
});
document.getElementById('nextseeds').addEventListener('click',
() => goSweep({ offset: sweepState().offset + sweepState().seeds }));
document.getElementById('prevseeds').addEventListener('click',
() => goSweep({ offset: Math.max(0, sweepState().offset - sweepState().seeds) }));
document.getElementById('rerender').addEventListener('click', renderSweep);
document.getElementById('back').addEventListener('click', () => { location.search = ''; });
window.addEventListener('popstate', renderSweep);
}
async function main() {
if (sweepState().scene) {
initSweepControls();
await renderSweep();
return;
}
const fingerprint = sourceFingerprint();
const cached = await loadGallery(fingerprint);
if (cached) {
rows = cached.rows;
contexts = cached.contexts;
document.getElementById('ctx').innerHTML = contexts
.map((c) => `<span title="${c.identity}">${c.name}</span>`).join('');
draw();
status.className = 'cached';
status.textContent = `cached · ${summarise(cached.built)}`;
window.__GALLERY__ = rows.map((r) => ({ name: r.name, variety: r.variety, error: r.error }));
return;
}
// No entry for this source. Something under src/ changed since the last
// build — or there has never been one — so rebuild without being asked.
await build(fingerprint);
}
// ---- metadata refresh ---------------------------------------------------
// The gallery is where the library gets measured, so it is where the measured
// facts are written back. The numbers go to src/scenes/metadata.json — tracked
// in git, read by the look generator, and stamped with a fingerprint of the
// scenes and the metric definitions so a stale file is detectable rather than
// merely old. See src/checks/metadata.js.
refreshBtn.addEventListener('click', async () => {
refreshBtn.disabled = true;
rebuildBtn.disabled = true;
bar.classList.add('building');
const started = Date.now();
try {
const fresh = measureLibrary({
onScene: (done, total, name) => {
status.textContent = `measuring ${done}/${total} · ${name}`;
progress.style.width = `${(done / total) * 100}%`;
},
});
const { moved, gone } = metadataDrift(fresh);
await writeMetadata(fresh);
const changes = moved.slice(0, 6).map((m) => `${m.name} ${m.note}`).join(' · ');
status.textContent =
`metadata written in ${((Date.now() - started) / 1000).toFixed(0)}s · ` +
`${Object.keys(fresh.scenes).length} scenes · ` +
(moved.length ? `${moved.length} moved — ${changes}` : 'nothing moved') +
(gone.length ? ` · dropped ${gone.join(', ')}` : '');
} catch (err) {
status.textContent = `metadata refresh failed: ${err.message} ` +
'(the write endpoint only exists under npm run dev)';
} finally {
bar.classList.remove('building');
refreshBtn.disabled = false;
rebuildBtn.disabled = false;
progress.style.width = '0%';
}
});
rebuildBtn.addEventListener('click', async () => {
await clearGallery();
await build(sourceFingerprint());
});
main();
</script>
</body>
</html>

67
flow-state/index.html
View File

@ -80,6 +80,73 @@
<input type="file" id="file-input" accept="audio/*" hidden> <input type="file" id="file-input" accept="audio/*" hidden>
<audio id="audio" hidden></audio> <audio id="audio" hidden></audio>
<div id="boot-error" hidden>
<div class="be-inner">
<div class="be-title">the app did not start</div>
<div class="be-body">
<p>Every control is inert because <code>main.js</code> never ran — the
markup you are looking at is the static page. Nothing is wrong with
your audio file.</p>
<pre id="be-detail"></pre>
<p class="be-hint">A module that fails to load takes the whole graph
with it, and the usual cause is a fetch that hit the dev server while
it was restarting. A hard reload normally fixes it; if it does not,
restart <code>npm run dev</code>.</p>
</div>
</div>
</div>
<script>
// BOOT GUARD. Loaded before the module and deliberately not a module
// itself, so it survives whatever kills the module graph.
//
// A module that fails to load — a 404, a MIME error, a syntax error, a
// dev server caught mid-restart — aborts main.js silently. The page
// still renders, every button is still there, and not one of them is
// wired to anything. That failure cost a real debugging session: the UI
// looked correct and simply did nothing, which is the most expensive
// way for a front end to break.
(function () {
var shown = false;
function show(detail) {
if (shown) return;
shown = true;
var box = document.getElementById('boot-error');
var pre = document.getElementById('be-detail');
if (pre) pre.textContent = detail || 'no detail available';
if (box) box.hidden = false;
}
// A module that never executes never sets this.
window.addEventListener('error', function (e) {
// Module load failures arrive as an error event on the script
// element rather than as a window error with a message.
if (e && e.target && e.target.tagName === 'SCRIPT') {
show('failed to load: ' + (e.target.src || 'unknown module') +
'\n\nOpen the console — the first "Loading failed for the ' +
'module" line names the file that actually broke.');
} else if (e && e.message) {
show(e.message + (e.filename ? '\n at ' + e.filename + ':' + e.lineno : ''));
}
}, true);
window.addEventListener('unhandledrejection', function (e) {
var r = e && e.reason;
show('unhandled rejection during startup:\n' + ((r && r.stack) || r));
});
// The backstop for the case no error event fires at all: if the app
// has not announced itself in a few seconds, it is not going to.
setTimeout(function () {
if (!window.__FLOW_STATE_READY__) {
show('main.js did not finish starting within 8 seconds and no ' +
'error was reported.\n\nCheck the console for the first red ' +
'line — a failed module import is the usual cause.');
}
}, 8000);
}());
</script>
<script type="module" src="/src/main.js"></script> <script type="module" src="/src/main.js"></script>
</body> </body>
</html> </html>

5
flow-state/package.json
View File

@ -9,7 +9,10 @@
"preview": "vite preview", "preview": "vite preview",
"lint:scenes": "node tools/lint-scenes.js", "lint:scenes": "node tools/lint-scenes.js",
"test": "node --test test/*.test.js", "test": "node --test test/*.test.js",
"new:scene": "node tools/new-scene.js" "new:scene": "node tools/new-scene.js",
"build:songs": "node tools/build-song-bank.js",
"check:songs": "node tools/build-song-bank.js --check",
"cast:census": "node tools/cast-census.js"
}, },
"license": "ISC", "license": "ISC",
"devDependencies": { "devDependencies": {

35
flow-state/src/Show.js
View File

@ -6,8 +6,14 @@ import { generateLook, rerollLook, rerollSection } from './look/LookGenerator.js
import { ArcDriver } from './look/ArcDriver.js'; import { ArcDriver } from './look/ArcDriver.js';
import { grainEnvelope } from './look/grain.js'; import { grainEnvelope } from './look/grain.js';
import { hashSamples } from './engine/rng.js'; import { hashSamples } from './engine/rng.js';
import { subjectOf } from './look/stack.js';
const FADE_SECONDS = 1.5; /**
* The opening and closing fade to black. Exported because anything SAMPLING a
* video has to know about it: a probe at t=0 reads a black frame, which is
* correct output and a useless sample. See checks/filmstrip.js.
*/
export const FADE_SECONDS = 1.5;
/** /**
* A loaded track plus its look, rendered. * A loaded track plus its look, rendered.
@ -40,8 +46,12 @@ export class Show {
* Decode, analyse, and generate a look. `onProgress(stage, fraction)` is * Decode, analyse, and generate a look. `onProgress(stage, fraction)` is
* called throughout; analysis is CPU-bound and will block the main thread * called throughout; analysis is CPU-bound and will block the main thread
* for a second or two on a long track. * for a second or two on a long track.
*
* `seed` replaces the audio-derived one. Only the debug pages pass it, and
* only so that watching a song reproduces the look they measured — the
* filmstrip seeds off the song's name, not off its samples.
*/ */
async load(file, onProgress = null) { async load(file, onProgress = null, { seed = null } = {}) {
const report = (stage, p) => onProgress && onProgress(stage, p); const report = (stage, p) => onProgress && onProgress(stage, p);
report('decoding', 0); report('decoding', 0);
@ -59,7 +69,9 @@ export class Show {
report('look', 0.97); report('look', 0.97);
const samples = monoSamples(audioBuffer); const samples = monoSamples(audioBuffer);
this.setLook(generateLook(this.track, { samples })); this.setLook(seed === null
? generateLook(this.track, { samples })
: generateLook(this.track, { seed }));
this.engine.timeline.setDuration(this.track.duration); this.engine.timeline.setDuration(this.track.duration);
this.engine.setFeatureProvider(featureProviderFor(this.track)); this.engine.setFeatureProvider(featureProviderFor(this.track));
@ -71,12 +83,22 @@ export class Show {
return this; return this;
} }
/** Attach an already-analysed track. Used by the check harness and by tests. */ /**
* Attach an already-analysed track. Used by the check harness and by tests.
*
* Prewarms, like every other path that installs a look. It was the one that
* did not, and that was invisible while most sections were a single layer:
* with a ground under every shot there are always at least two programs to
* link, and the first frame a fresh Show rendered came out different from
* every later render of it — the exact hazard Compositor.prime documents,
* caught by the phase 4 first-render check.
*/
useTrack(track, look) { useTrack(track, look) {
this.track = track; this.track = track;
this.engine.timeline.setDuration(track.duration); this.engine.timeline.setDuration(track.duration);
this.engine.setFeatureProvider(featureProviderFor(track)); this.engine.setFeatureProvider(featureProviderFor(track));
this.setLook(look || generateLook(track, { seed: 1 })); this.setLook(look || generateLook(track, { seed: 1 }));
this.prewarm();
return this; return this;
} }
@ -103,6 +125,9 @@ export class Show {
setPalette(palette) { setPalette(palette) {
this.look.palette = palette; this.look.palette = palette;
if (this.look.palettes && this.look.palettes.length) {
this.look.palettes[0] = palette;
}
this.arc.setPalette(palette); this.arc.setPalette(palette);
this.osd.setPalette(palette); this.osd.setPalette(palette);
} }
@ -111,7 +136,7 @@ export class Show {
setSectionParam(sectionIndex, name, value) { setSectionParam(sectionIndex, name, value) {
const section = this.look.sections[sectionIndex]; const section = this.look.sections[sectionIndex];
if (!section) return; if (!section) return;
section.layers[0].params[name] = value; subjectOf(section.layers).params[name] = value;
this._lastLayers = null; this._lastLayers = null;
} }

13
flow-state/src/audio/clicktrack.js → flow-state/src/audio/metronome.js
View File

@ -1,6 +1,17 @@
// Click track — the Phase 1 gate, and the most useful validation tool in the // The metronome — the Phase 1 gate, and the most useful validation tool in the
// project. // project.
// //
// NAMED CAREFULLY. This file was `clicktrack.js` until a content blocker ate it:
// `clicktrack` is a click-tracking telemetry pattern that EasyPrivacy and
// friends block by substring, so the request died in the browser, the module
// graph died with it, main.js never ran, and every control in the app went
// inert while looking perfectly fine. The server was serving it with a 200 the
// whole time.
//
// Anything shipped to a browser and named like tracking will be treated as
// tracking. Avoid `click`, `track`, `analytics`, `pixel`, `beacon` and `ad` in
// filenames and URL paths, however honest the code behind them is.
//
// Beat detection cannot be judged by watching visuals: a grid that is 20ms late // Beat detection cannot be judged by watching visuals: a grid that is 20ms late
// or at half tempo still "looks kind of right". Mixing an audible click onto the // or at half tempo still "looks kind of right". Mixing an audible click onto the
// detected grid makes the answer immediate and unambiguous. Downbeats get a // detected grid makes the answer immediate and unambiguous. Downbeats get a

31
flow-state/src/audio/segment.js
View File

@ -14,7 +14,8 @@ export const SECTION_KINDS = ['intro', 'build', 'drop', 'sustain', 'breakdown',
const ANALYSIS_HZ = 4; // coarse grid for the similarity matrix const ANALYSIS_HZ = 4; // coarse grid for the similarity matrix
const KERNEL_SECONDS = 6; // half-width of the checkerboard kernel const KERNEL_SECONDS = 6; // half-width of the checkerboard kernel
const MIN_SECTION_SECONDS = 12; const MIN_SECTION_SECONDS = 4;
const DROP_HEAD_BARS = 16; // how long a drop stays a drop before it is just the track
function median(values) { function median(values) {
if (!values.length) return 0; if (!values.length) return 0;
@ -170,6 +171,31 @@ function classify(section, context) {
return 'sustain'; return 'sustain';
} }
/**
* A drop is an EVENT — the arrival — and what follows is the track's main body,
* however loud that stays. classify() has no memory of what came before, so a
* long loud stretch labels every one of its sections 'drop' and the video holds
* peak intensity for minutes on end.
*
* Collapse each run of drops down to its head: the opening sections keep the
* label until a phrase's worth of time is spent, and the rest become 'sustain'.
* Runs are read off the ORIGINAL labels so the demotion never cascades, and any
* other kind in between — a breakdown, a build — re-arms the next drop.
*/
function collapseDropRuns(sections, secondsPerBar) {
const budget = DROP_HEAD_BARS * secondsPerBar;
const original = sections.map((s) => s.kind);
let spent = 0;
for (let i = 0; i < sections.length; i++) {
if (original[i] !== 'drop') { spent = 0; continue; }
const continues = i > 0 && original[i - 1] === 'drop';
if (!continues) spent = 0;
else if (spent >= budget) sections[i].kind = 'sustain';
spent += sections[i].duration;
}
}
/** /**
* @returns {Array<{index,start,end,startFrame,endFrame,kind,energy,slope,flux,centroid}>} * @returns {Array<{index,start,end,startFrame,endFrame,kind,energy,slope,flux,centroid}>}
*/ */
@ -262,5 +288,8 @@ export function segment(raw, frameCount, fps, tempo) {
); );
}); });
const bpm = Number.isFinite(tempo.bpm) && tempo.bpm > 1 ? tempo.bpm : 120;
collapseDropRuns(sections, (4 * 60) / bpm);
return sections; return sections;
} }

143
flow-state/src/audio/songbank.js Normal file
View File

@ -0,0 +1,143 @@
// The song bank: a set of tracks that between them exercise every input the
// look generator reads.
//
// Why this has to exist. Everything downstream of the audio is a function of
// five summary statistics and a section list, and until now every measurement
// in this project was taken on ONE synthetic track — which turned out to have
// two sections, both quiet, making half the scene library unreachable and the
// resulting numbers wrong. A single song cannot tell you whether the generator
// is varied; it can only tell you what it does with that song.
//
// The bank is built against what the generator actually consumes, not against
// what sounds like a reasonable spread of music:
//
// summary.bpm motion bias, animation rate, personality pace
// summary.meanCentroid director odds, line weight, softness, bloom, vignette
// summary.meanFlatness director odds, chroma, saturation, surface texture
// summary.dynamicRange feedback amount and decay, contrast
// summary.meanLoudness per-section energy, temperament intensity
// section kinds which families each director opens — the casting gate
//
// Coverage is a claim, so `tools/song-coverage.js` measures it rather than
// trusting this table. Each entry names the corner it is here to occupy; if a
// song stops occupying it, that is a bug in the synth, and the coverage tool is
// what catches it.
//
// These are test signals, not music. They exist to be measured.
import { FeatureTrack } from './FeatureTrack.js';
import { synthesizeSong } from './synth.js';
/**
* @typedef {object} SongSpec
* @property {string} name
* @property {string} covers the corner of the feature space this one holds
*/
export const SONGS = [
// --- slow, quiet, spacious -------------------------------------------
{ name: 'drone', bpm: 62, arrangement: 'ambient', brightness: 0.05, noise: 0.02, dynamics: 0.95, transition: 0.05,
covers: 'beatless · darkest · most dynamic' },
{ name: 'air', bpm: 76, arrangement: 'ambient', brightness: 0.90, noise: 0.25, noiseColour: 0.15, dynamics: 0.85, transition: 0.1,
covers: 'beatless but bright — brightness without a beat' },
{ name: 'ember', bpm: 68, arrangement: 'club', brightness: 0.20, noise: 0.05, dynamics: 0.55, transition: 0.8,
covers: 'slowest track with a detectable beat — anchors the low tempo end' },
{ name: 'elegy', bpm: 84, arrangement: 'ballad', brightness: 0.10, noise: 0.03, dynamics: 0.90, transition: 0.12,
covers: 'slow, dark, one build and one payoff' },
{ name: 'hymn', bpm: 92, arrangement: 'ballad', brightness: 0.80, noise: 0.20, dynamics: 0.70, transition: 0.25,
covers: 'slow and bright — separates tempo from brightness' },
{ name: 'still', bpm: 100, arrangement: 'sparse', brightness: 0.35, noise: 0.06, dynamics: 0.75, transition: 0.2,
covers: 'the degenerate two-section case, which must not break' },
// --- mid tempo, the bulk of the space ---------------------------------
{ name: 'dusk', bpm: 118, arrangement: 'classic', brightness: 0.02, noise: 0.01, dynamics: 0.60, transition: 0.45,
covers: 'darkest and most tonal with a full arrangement' },
{ name: 'centre', bpm: 124, arrangement: 'classic', brightness: 0.50, noise: 0.20, dynamics: 0.55, transition: 0.6,
covers: 'the middle of every axis — the null hypothesis' },
{ name: 'glare', bpm: 128, arrangement: 'classic', brightness: 0.95, noise: 0.55, dynamics: 0.50, transition: 0.9,
covers: 'bright and noisy together' },
{ name: 'slab', bpm: 126, arrangement: 'club', brightness: 0.45, noise: 0.25, dynamics: 0.05, transition: 0.95,
covers: 'limitered — lowest dynamic range, longest sections' },
{ name: 'chrome', bpm: 132, arrangement: 'club', brightness: 0.85, noise: 0.06, noiseColour: 0.1, dynamics: 0.35, transition: 0.85,
covers: 'bright and tonal at club tempo' },
{ name: 'murk', bpm: 88, arrangement: 'classic', brightness: 0.12, noise: 0.85, noiseColour: 0.95, dynamics: 0.70, transition: 0.35,
covers: 'dark tonal content under bright hiss — the quadrant that breaks the brightness/noise correlation' },
// --- fast, dense, broken ----------------------------------------------
{ name: 'lattice', bpm: 140, arrangement: 'breaks', brightness: 0.60, noise: 0.35, dynamics: 0.45, transition: 0.92,
covers: 'most sections — makes rosters actually rotate' },
{ name: 'grit', bpm: 150, arrangement: 'breaks', brightness: 0.15, noise: 0.90, noiseColour: 0.85, dynamics: 0.40, transition: 0.88,
covers: "noisiest and dark — the corrupt director's home ground" },
{ name: 'plate', bpm: 138, arrangement: 'club', brightness: 0.25, noise: 0.70, noiseColour: 0.9, dynamics: 0.10, transition: 0.95,
covers: 'noisy AND compressed — two extremes at once' },
{ name: 'runner', bpm: 174, arrangement: 'breaks', brightness: 0.70, noise: 0.40, dynamics: 0.50, transition: 0.9,
covers: 'fastest with a broken arrangement' },
{ name: 'flare', bpm: 168, arrangement: 'classic', brightness: 0.95, noise: 0.85, dynamics: 0.25, transition: 0.7,
covers: 'brightest and noisiest — the far corner' },
];
const DURATION = 120;
// Analysis is CPU-bound seconds per track and every caller wants the same
// tracks, so the bank is built once and held.
const cache = new Map();
/** One song from the bank, analysed. */
export function song(name, { fps = 60, duration = DURATION } = {}) {
const cacheKey = `${name}:${fps}:${duration}`;
if (cache.has(cacheKey)) return cache.get(cacheKey);
const spec = SONGS.find((s) => s.name === name);
if (!spec) throw new Error(`no song named "${name}" — have: ${SONGS.map((s) => s.name).join(', ')}`);
const entry = {
...spec,
track: FeatureTrack.fromAudioBuffer(synthesizeSong({
bpm: spec.bpm,
duration,
arrangement: spec.arrangement,
brightness: spec.brightness,
noise: spec.noise,
noiseColour: spec.noiseColour ?? null,
dynamics: spec.dynamics,
transition: spec.transition ?? 0.5,
seed: 1 + SONGS.indexOf(spec),
}), { fps }),
};
cache.set(cacheKey, entry);
return entry;
}
/**
* The whole bank, analysed.
*
* `count` takes an evenly spaced subset rather than the first n, so a cheap run
* still spans the space instead of testing five slow ambient tracks.
*/
export function songBank({ count = null, fps = 60, duration = DURATION } = {}) {
const specs = count && count < SONGS.length
? Array.from({ length: count }, (_, i) => SONGS[Math.round(i * (SONGS.length - 1) / (count - 1))])
: SONGS;
return specs.map((s) => song(s.name, { fps, duration }));
}
/** Feature axes the bank claims to span, and where each is read. */
export const AXES = [
{ key: 'bpm', label: 'tempo', of: (t) => t.summary.bpm, range: [60, 180],
reads: 'motion bias · animation rate · personality pace' },
// Not [0,1]: centroid is mapped to a LOG frequency axis, so a track made of
// nothing but sub-bass and a 55Hz pad still measures ~0.28 and pure hiss
// measures ~0.9. The range here is what the statistic can really take on for
// music, and calling it [0,1] would report a permanent 50% gap that no
// synth change could close.
{ key: 'centroid', label: 'brightness', of: (t) => t.summary.meanCentroid, range: [0.25, 0.95],
reads: 'director odds · line weight · softness · bloom · vignette' },
{ key: 'flatness', label: 'noisiness', of: (t) => t.summary.meanFlatness, range: [0, 1],
reads: 'director odds · chroma · saturation · surface texture' },
{ key: 'dynamics', label: 'dynamic range', of: (t) => t.summary.dynamicRange, range: [0, 1],
reads: 'feedback amount and decay · contrast' },
// Raw mean spectral magnitude, not a normalised 0..1 — biasFor divides
// section energy by it, so its scale is whatever the analyser produces.
{ key: 'loudness', label: 'loudness', of: (t) => t.summary.meanLoudness, range: [0, 0.012],
reads: 'section energy · temperament intensity' },
{ key: 'sections', label: 'section count', of: (t) => t.sections.length, range: [2, 8],
reads: 'roster rotation · how often the image is allowed to change' },
];

513
flow-state/src/audio/synth.js
View File

@ -119,6 +119,519 @@ export function synthesizeSectioned({
return buffer; return buffer;
} }
/**
* A broadband noise bed. This is the only way to move spectral FLATNESS, which
* the generator reads to decide how noisy a track is — it picks the director
* partly on it, and the whole grade follows. Kicks, hats and pads are all
* tonal or transient, so without this the bank could only ever produce tonal
* tracks and half the director table would be unreachable.
*
* `tilt` shapes it with a one-pole filter: 0 is dark rumble, 1 is bright hiss.
* Brightness and noisiness have to be independently controllable or the bank
* cannot tell the two apart when it reports coverage.
*/
function addNoise(data, sampleRate, from, to, gainAt, tiltAt, seed = 12345) {
const start = Math.max(0, Math.round(from * sampleRate));
const end = Math.min(data.length, Math.round(to * sampleRate));
if (end <= start) return;
let s0 = seed >>> 0;
const rnd = () => {
s0 = (Math.imul(s0 ^ (s0 >>> 15), s0 | 1) + 0x6d2b79f5) >>> 0;
return ((s0 >>> 14) & 0xffff) / 0xffff - 0.5;
};
// Gain and tilt are functions of position through the stage, not constants.
// Held constant, the bed was a continuous hiss over the entire track — the
// one thing every listener noticed first. As a sweep it becomes a riser into
// a drop and a wash that recedes, which is both easier to hear past and a
// better test signal: a rising filter is real spectral movement for the
// analyser to find.
let lp1 = 0, lp2 = 0;
const span = end - start;
for (let s = start; s < end; s++) {
const u = (s - start) / span;
const tilt = tiltAt(u);
// Two cascaded one-poles rather than one. A single pole is a 6dB/octave
// slope, which leaves so much top on "dark" noise that a noisy track
// always measured bright — the dark-and-noisy corner, which is the
// corrupt director's entire home ground, was not reachable at all.
const a = 0.004 + tilt * 0.9;
const white = rnd();
lp1 += a * (white - lp1);
lp2 += a * (lp1 - lp2);
// Below half tilt the lowpass IS the signal; above it, what the lowpass
// removed is.
data[s] += (tilt > 0.5 ? white - lp2 : lp2 * 3) * gainAt(u) * 2;
}
}
/**
* How a stage's energy moves across its own span.
*
* Stages were flat blocks: a drop was thirty seconds at one level, and the only
* shape available was a `ramp` flag on the build. That is both dull to listen to
* and a worse test signal than it looks, because the segmenter classifies a
* section partly on its energy SLOPE — a build that does not build is a build
* the classifier has to guess at.
*
* `u` runs 0..1 through the stage.
*/
const CONTOURS = {
flat: () => 1,
rise: (u) => 0.22 + 0.78 * u ** 1.5,
fall: (u) => 1 - 0.72 * u ** 1.2,
swell: (u) => 0.35 + 0.65 * Math.sin(Math.PI * u),
dip: (u) => 1 - 0.55 * Math.sin(Math.PI * u),
// Loud on the downbeat, easing off, then lifting back at the very end into
// whatever comes next.
surge: (u) => 0.72 + 0.28 * Math.cos(Math.PI * 2 * u),
};
const contourFor = (shape) => CONTOURS[shape] || CONTOURS.flat;
// --- notes ----------------------------------------------------------------
// The bank started out as sustained sine stacks at a fixed root, and a stage
// multiplier of 4 put that root above 1kHz with ten harmonics on top of it.
// Measurably it was fine. To listen to it was a test tone, and nobody can
// review a bank they cannot bear to play — a song you skip is a song you never
// notice is wrong.
//
// So the tonal content is notes now: a scale, a chord progression, a bass
// following the roots, and a seeded motif over the loud stages. This is
// deliberately the plainest music that qualifies as music. It is not trying to
// be good; it is trying to be listenable enough that a human will actually
// audit the bank, while keeping every measured statistic under the same control
// as before.
/** Semitone offsets. Minor and its neighbours — nothing here should sound jolly. */
const SCALES = {
minor: [0, 2, 3, 5, 7, 8, 10],
dorian: [0, 2, 3, 5, 7, 9, 10],
pentatonic: [0, 3, 5, 7, 10],
};
const SCALE_NAMES = Object.keys(SCALES);
/** Degree sequences, one chord per two bars. All resolve back to the tonic. */
const PROGRESSIONS = [
[0, 5, 3, 4],
[0, 3, 4, 3],
[0, 6, 5, 4],
[0, 2, 5, 4],
];
/** Equal temperament from a root frequency. */
const semitone = (root, n) => root * 2 ** (n / 12);
/**
* One note: a harmonic stack under an envelope.
*
* Harmonics above Nyquist are skipped rather than allowed to alias — aliasing
* would fold energy back down into the spectrum and quietly corrupt both the
* centroid and the flatness the bank exists to control.
*/
function addNote(data, sampleRate, at, duration, freq, gain, harmonics = 3, attack = 0.012, sustain = 0) {
const start = Math.max(0, Math.round(at * sampleRate));
const end = Math.min(data.length, Math.round((at + duration) * sampleRate));
if (end <= start || freq <= 0) return;
const usable = [];
for (let h = 1; h <= harmonics; h++) {
if (freq * h < sampleRate * 0.4) usable.push(h);
}
if (!usable.length) return;
const norm = gain / Math.log2(usable.length + 1);
const attackSamples = Math.max(1, attack * sampleRate);
// Long enough to sustain through the note, short enough that consecutive
// notes articulate instead of smearing into the drone this replaced.
//
// `sustain` flattens that decay toward a hold. A limitered master has no
// gaps in it, and once the tonal bed became enveloped notes the gaps between
// them put a floor of 0.27 under the measured dynamic range — the compressed
// end of the bank simply stopped being reachable.
const decay = (1.6 / Math.max(0.08, duration)) * (1 - sustain * 0.94);
for (let s = start; s < end; s++) {
const i = s - start;
const t = i / sampleRate;
const env = Math.min(1, i / attackSamples) * Math.exp(-t * decay);
let v = 0;
for (const h of usable) v += Math.sin(2 * Math.PI * freq * h * t) / h;
data[s] += v * env * norm;
}
}
/**
* The tonal bed for one stage: bass, chords, and a motif over the loud stages.
*
* `register` is the old stage `root` multiplier, reinterpreted. It used to
* multiply the fundamental — which is how a drop ended up at 1.2kHz — and now
* it adds VOICES upward instead: the bass and chord stay where they belong and
* a lead octave arrives when the arrangement gets loud. Same shape in the
* measurements, an octave of headroom instead of a shriek.
*/
function addTonalBed(data, sampleRate, from, to, {
gainAt, root, scale, progression, harmonics, beat, register, rng, sustain = 0,
}) {
const bar = beat * 4;
// The contour is sampled per note rather than per stage, so a rise actually
// rises through the chords instead of stepping between two flat halves.
const gainOf = (t) => gainAt((t - from) / Math.max(1e-6, to - from));
const degrees = SCALES[scale];
const chordAt = (index) => {
const degree = progression[index % progression.length];
return [0, 2, 4].map((step) => degrees[(degree + step) % degrees.length]
+ 12 * Math.floor((degree + step) / degrees.length));
};
let chordIndex = 0;
for (let t = from; t < to; t += bar * 2, chordIndex++) {
const chord = chordAt(chordIndex);
const span = Math.min(bar * 2, to - t);
// Bass: the chord root, one per bar, low and simple.
for (let b = 0; b < 2 && t + b * bar < to; b++) {
addNote(data, sampleRate, t + b * bar, Math.min(bar, to - t - b * bar),
semitone(root, chord[0]) / 2, gainOf(t + b * bar) * 1.1,
Math.max(2, harmonics - 2), 0.02, sustain);
}
// Chord: held across the two bars, mid register.
for (const note of chord) {
addNote(data, sampleRate, t, span, semitone(root, note),
gainOf(t) * 0.5, harmonics, 0.25, sustain);
}
// Motif: only once the arrangement has opened up, so quiet stages stay
// quiet and the loud ones have something on top that moves.
if (register >= 1.5) {
const step = beat / (register >= 3 ? 2 : 1);
const octave = register >= 3 ? 4 : 2;
for (let n = 0; t + n * step < to && n * step < span; n++) {
if (rng() < 0.3) continue; // rests, so it phrases
const pick = degrees[Math.floor(rng() * degrees.length)];
addNote(data, sampleRate, t + n * step, step * 0.9,
semitone(root, pick) * octave, gainOf(t + n * step) * 0.30,
Math.max(2, harmonics - 1), 0.008);
}
}
}
}
/**
* Arrangement shapes, as stage lists.
*
* A song's SHAPE is a feature the generator reads as directly as its tempo —
* section kinds decide which families each director opens, so a bank that only
* contains one arrangement cannot exercise the casting logic no matter how
* widely it spreads tempo and brightness. Spans are fractions of the track.
*
* Stage fields: span, kick, hat, pad, noise, root (register), shape (contour).
*
* `noise` is per stage rather than following the pad level, which is what made
* the bed a constant hiss across the whole track. It is now an arrangement
* element: a riser through a build, a wash under a drop, nearly absent in a
* breakdown.
*/
export const ARRANGEMENTS = {
// Beatless. Segments into quiet kinds only — the case that must stay
// representable, since it is what an actual ambient track looks like.
ambient: [
{ span: 0.30, kick: 0, hat: 0, pad: 0.10, noise: 0.25, root: 1.0, shape: 'swell' },
{ span: 0.25, kick: 0, hat: 0, pad: 0.20, noise: 0.60, root: 1.5, shape: 'rise' },
{ span: 0.25, kick: 0, hat: 0, pad: 0.12, noise: 0.35, root: 1.0, shape: 'fall' },
{ span: 0.20, kick: 0, hat: 0, pad: 0.07, noise: 0.15, root: 0.75, shape: 'fall' },
],
// The standard shape: everything present, once.
classic: [
{ span: 0.12, kick: 0.00, hat: 0.00, pad: 0.09, noise: 0.15, root: 1.0, shape: 'swell' },
{ span: 0.16, kick: 0.55, hat: 0.20, pad: 0.14, noise: 1.00, root: 1.5, shape: 'rise' },
{ span: 0.22, kick: 1.00, hat: 0.50, pad: 0.24, noise: 0.35, root: 4.0, shape: 'surge' },
{ span: 0.14, kick: 0.05, hat: 0.02, pad: 0.07, noise: 0.10, root: 1.0, shape: 'dip' },
{ span: 0.24, kick: 1.00, hat: 0.55, pad: 0.26, noise: 0.40, root: 4.0, shape: 'surge' },
{ span: 0.12, kick: 0.25, hat: 0.08, pad: 0.08, noise: 0.20, root: 1.0, shape: 'fall' },
],
// Long and level: a club tool that states its groove and stays there. The
// case that produces `sustain`, which the classic shape barely reaches.
club: [
{ span: 0.10, kick: 0.45, hat: 0.15, pad: 0.10, noise: 0.30, root: 1.0, shape: 'rise' },
{ span: 0.30, kick: 0.90, hat: 0.45, pad: 0.20, noise: 0.30, root: 2.0, shape: 'surge' },
{ span: 0.12, kick: 0.35, hat: 0.15, pad: 0.12, noise: 0.20, root: 1.0, shape: 'dip' },
{ span: 0.34, kick: 1.00, hat: 0.55, pad: 0.22, noise: 0.40, root: 3.0, shape: 'surge' },
{ span: 0.14, kick: 0.50, hat: 0.20, pad: 0.10, noise: 0.20, root: 1.0, shape: 'fall' },
],
// Many short sections. Stresses the segmenter and produces the highest
// section counts, which is what makes rosters rotate.
breaks: [
{ span: 0.10, kick: 0.30, hat: 0.10, pad: 0.08, noise: 0.20, root: 1.0, shape: 'rise' },
{ span: 0.12, kick: 0.95, hat: 0.55, pad: 0.20, noise: 0.35, root: 3.0, shape: 'surge' },
{ span: 0.10, kick: 0.05, hat: 0.02, pad: 0.06, noise: 0.10, root: 1.0, shape: 'dip' },
{ span: 0.14, kick: 1.00, hat: 0.60, pad: 0.24, noise: 0.40, root: 4.0, shape: 'surge' },
{ span: 0.10, kick: 0.10, hat: 0.03, pad: 0.07, noise: 0.15, root: 0.75, shape: 'fall' },
{ span: 0.16, kick: 0.60, hat: 0.30, pad: 0.16, noise: 1.00, root: 2.0, shape: 'rise' },
{ span: 0.16, kick: 1.00, hat: 0.65, pad: 0.26, noise: 0.40, root: 4.0, shape: 'surge' },
{ span: 0.12, kick: 0.20, hat: 0.05, pad: 0.08, noise: 0.15, root: 1.0, shape: 'fall' },
],
// One long rise into one payoff, then gone. Slow material's shape.
ballad: [
{ span: 0.28, kick: 0.00, hat: 0.00, pad: 0.10, noise: 0.20, root: 1.0, shape: 'swell' },
{ span: 0.30, kick: 0.40, hat: 0.10, pad: 0.18, noise: 0.85, root: 1.5, shape: 'rise' },
{ span: 0.24, kick: 0.85, hat: 0.35, pad: 0.26, noise: 0.35, root: 3.0, shape: 'surge' },
{ span: 0.18, kick: 0.15, hat: 0.03, pad: 0.09, noise: 0.15, root: 1.0, shape: 'fall' },
],
// Two sections, both quiet. Kept because it is the degenerate case the old
// bank produced by accident, and the generator must not break on it.
sparse: [
{ span: 0.5, kick: 0.10, hat: 0.02, pad: 0.08, noise: 0.20, root: 1.0, shape: 'swell' },
{ span: 0.5, kick: 0.35, hat: 0.15, pad: 0.16, noise: 0.30, root: 2.0, shape: 'rise' },
],
};
/**
* One song, from a spec.
*
* The four continuous knobs map onto exactly the four summary statistics the
* look generator reads, so a bank built on them can be checked for coverage
* against what the generator actually consumes rather than against what seemed
* like a reasonable spread of audio.
*
* bpm → summary.bpm
* brightness → summary.meanCentroid (pad register, harmonics, noise tilt)
* noise → summary.meanFlatness (broadband bed)
* dynamics → summary.dynamicRange (how far quiet stages fall below loud)
*/
export function synthesizeSong({
bpm = 128,
duration = 120,
sampleRate = 44100,
arrangement = 'classic',
brightness = 0.5,
noise = 0.15,
noiseColour = null,
dynamics = 0.6,
transition = 0.5,
seed = 1,
} = {}) {
const stages = ARRANGEMENTS[arrangement] || ARRANGEMENTS.classic;
const length = Math.round(duration * sampleRate);
const buffer = new MockAudioBuffer(2, length, sampleRate);
const left = buffer.getChannelData(0);
const right = buffer.getChannelData(1);
const beat = 60 / bpm;
// Dynamic range is measured as a crest factor — the 40th percentile of
// frame RMS against the 95th — so moving it means changing the RATIO
// between the quiet stages and the loud ones, not the overall level.
//
// A gamma on each stage's level relative to the loudest stage does that.
// The loudest stage is pinned at its original value and everything below it
// is pushed down (high dynamics) or pulled up (limitered), so a dynamic
// track does not simply come out quieter — which is what a straight floor
// offset did, and it moved the measured range by 0.2 across the whole bank.
const gamma = 0.25 + dynamics * 3.2;
// Percussion is transient, so a kick-driven track has a low RMS floor
// between hits no matter how hard it is limitered — the gamma alone could
// not get the measured range below 0.67. What actually fills the gaps is
// SUSTAINED content, so the pad and the noise bed swell as dynamics falls.
// That is also what a limitered master really sounds like.
const sustainBoost = 1 + (1 - dynamics) ** 2 * 6;
const peakOf = (field) => Math.max(...stages.map((s) => s[field] || 0), 1e-6);
const peaks = { kick: peakOf('kick'), hat: peakOf('hat'), pad: peakOf('pad') };
const level = (x, field) => (x <= 0 ? 0 : peaks[field] * (x / peaks[field]) ** gamma);
// Brightness has to reach genuinely dark, and a hat is broadband: leaving
// any hat in at brightness 0 held the measured centroid above 0.65 no matter
// what the pad did. Below a quarter brightness the hats go entirely.
//
// Brightness moves the HARMONIC content and the key's register, but only
// within an octave and a half. It used to set the fundamental outright,
// across 55Hz to 315Hz and then multiplied by the stage — which is how a
// drop arrived at 1.2kHz with ten harmonics stacked on it.
const baseRoot = 55 * 2 ** (brightness * 1.5);
const harmonics = Math.max(1, Math.round(1 + brightness ** 1.5 * 9));
const hatPresence = Math.max(0, (brightness - 0.22) / 0.78);
// Key and progression come off the seed, so two songs in the bank are not
// the same four chords at different tempos.
let rngState = (seed * 2654435761 + 12345) >>> 0;
const rng = () => {
rngState = (Math.imul(rngState ^ (rngState >>> 15), rngState | 1) + 0x6d2b79f5) >>> 0;
return ((rngState ^ (rngState >>> 14)) >>> 0) / 4294967296;
};
const scale = SCALE_NAMES[Math.floor(rng() * SCALE_NAMES.length)];
const progression = PROGRESSIONS[Math.floor(rng() * PROGRESSIONS.length)];
const root = semitone(baseRoot, Math.floor(rng() * 12));
// The noise bed's COLOUR is its own axis, defaulting to the track's
// brightness but separable from it. Tying the two together made spectral
// flatness a function of centroid — measured across the bank they came out
// at r = 0.95, one axis wearing two names, and the whole dark-but-noisy
// quadrant was unreachable. Hiss over a sub-bass pad is an ordinary record.
const tilt = 0.05 + (noiseColour === null ? brightness : noiseColour) * 0.9;
// How abruptly one stage becomes the next. Genres differ on this more than
// they differ on tempo: an ambient record dissolves between its sections and
// a club record cuts, and until now every song in the bank cut.
//
// It is not only a listening difference. The segmenter finds boundaries by
// looking for change, so a soft track genuinely has fewer of them — section
// count is the axis the bank covered worst, and this is the knob that moves
// it for a musical reason rather than by padding the stage list.
const bar = beat * 4;
const fade = (1 - transition) ** 1.5 * bar * 2.5;
// The hardest transitions get the pre-drop trick: everything stops for a
// moment before the loud stage lands. Nothing states "this is a cut" more
// plainly, and it is what an EDM master actually does.
const gapBeats = transition > 0.7 ? (transition - 0.7) / 0.3 * 0.9 : 0;
let at = 0;
let stageIndex = 0;
const bounds = [];
{
let cursor = 0;
for (const stage of stages) {
const from = cursor;
cursor = Math.min(duration, cursor + stage.span * duration);
bounds.push([from, cursor]);
}
}
for (const stage of stages) {
const [from, to] = bounds[stageIndex];
const next = stages[stageIndex + 1];
at = to;
stageIndex++;
const span = Math.max(1e-6, to - from);
const contour = contourFor(stage.shape);
// Compression flattens the contrast BETWEEN sections, not just within
// them, so a limitered track gets shallower contours. Without this the
// contours alone put a floor of 0.50 under the measured dynamic range
// and the compressed end of the bank stopped existing — the section
// shapes were quietly undoing the axis they were layered onto.
const depth = 0.12 + dynamics * 0.88;
const fadeU = Math.min(0.45, fade / span);
// A hard cut into a LOUDER stage gets the gap; a fall into a quiet one
// does not, because silence before a breakdown is just silence.
const gapU = next && (next.kick + next.pad) > (stage.kick + stage.pad) * 1.3
? Math.min(0.2, (gapBeats * beat) / span) : 0;
/** The stage's own shape, with its edges softened or cut. */
const shapeAt = (u) => {
let v = 1 + (contour(Math.max(0, Math.min(1, u))) - 1) * depth;
if (fadeU > 1e-4) {
if (u < fadeU) v *= 0.5 - 0.5 * Math.cos(Math.PI * (u / fadeU));
if (u > 1 - fadeU) v *= 0.5 - 0.5 * Math.cos(Math.PI * ((1 - u) / fadeU));
}
if (gapU > 1e-4 && u > 1 - gapU) v *= 0.04;
return v;
};
let index = Math.round(from / beat);
for (let t = from; t < to; t += beat, index++) {
const u = (t - from) / span;
const shaped = shapeAt(u);
const gain = level(stage.kick, 'kick') * shaped;
if (stage.kick > 0.02 && gain > 0.005) {
addKick(left, sampleRate, t, gain * (index % 4 === 0 ? 1 : 0.8));
}
if (stage.hat > 0.02 && hatPresence > 0) {
const h = level(stage.hat, 'hat') * shaped * hatPresence * 1.4;
addHat(left, sampleRate, t + beat / 2, h, index + seed);
if (h > 0.3) addHat(left, sampleRate, t + beat / 4, h * 0.6, index + seed + 7);
}
}
const padLevel = level(stage.pad, 'pad') * sustainBoost;
addTonalBed(left, sampleRate, from, to, {
gainAt: (u) => padLevel * shapeAt(u),
root, scale, progression, harmonics, beat,
register: stage.root,
sustain: 1 - dynamics,
rng,
});
if (noise > 0.01) {
const bed = noise * (stage.noise ?? 0.3) * padLevel * 6;
// A riser sweeps its filter up as it goes; everything else holds its
// colour. This is the difference between a bed that builds tension
// and one that is just always there.
const rising = stage.shape === 'rise';
addNoise(left, sampleRate, from, to,
(u) => bed * shapeAt(u),
rising ? (u) => Math.min(0.98, tilt + u * 0.5) : () => tilt,
(seed * 7919 + stageIndex) >>> 0);
}
}
for (let i = 0; i < length; i++) right[i] = left[i] * 0.98;
return buffer;
}
/**
* A track with a full ARRANGEMENT — intro, build, drop, breakdown, drop, outro.
*
* `synthesizeSectioned` has one change point, so it segments into exactly two
* sections and both of them are quiet kinds. That is fine for testing that the
* segmenter finds a boundary, and it was quietly useless for anything that
* measures what the generator DOES with a song: intro, breakdown and outro are
* restricted to the restful families for every director, so a two-section track
* cannot reach geometric, glitch or structural scenes at all. Half the library
* is unreachable before the seed is even drawn, and a test built on it will
* report that as a casting failure.
*
* The stages here are shaped to hit the segmenter's own classifier: a build
* needs a rising energy slope, a drop needs energy and flux together, and a
* breakdown needs to fall well below the median.
*/
export function synthesizeArrangement({
bpm = 128,
duration = 120,
sampleRate = 44100,
brightness = 1,
density = 1,
} = {}) {
const length = Math.round(duration * sampleRate);
const buffer = new MockAudioBuffer(2, length, sampleRate);
const left = buffer.getChannelData(0);
const right = buffer.getChannelData(1);
// Proportions of the track, in order. Kick gain, hat gain, pad root, pad gain.
const stages = [
{ span: 0.12, kick: 0.00, hat: 0.00, root: 110, pad: 0.09 }, // intro
{ span: 0.16, kick: 0.55, hat: 0.20, root: 165, pad: 0.14, ramp: true }, // build
{ span: 0.22, kick: 1.00, hat: 0.50, root: 440, pad: 0.24 }, // drop
{ span: 0.14, kick: 0.05, hat: 0.02, root: 110, pad: 0.07 }, // breakdown
{ span: 0.24, kick: 1.00, hat: 0.55, root: 440, pad: 0.26 }, // drop
{ span: 0.12, kick: 0.25, hat: 0.08, root: 110, pad: 0.08 }, // outro
];
const beat = 60 / bpm;
let at = 0;
for (const stage of stages) {
const from = at;
const to = Math.min(duration, at + stage.span * duration);
at = to;
let index = Math.round(from / beat);
for (let t = from; t < to; t += beat, index++) {
// A build ramps across its own span so the energy slope is positive
// enough for the classifier to call it one.
const ramp = stage.ramp ? (t - from) / Math.max(1e-6, to - from) : 1;
const gain = stage.kick * density * (0.25 + ramp * 0.75);
if (gain > 0.02) addKick(left, sampleRate, t, gain * (index % 4 === 0 ? 1 : 0.8));
if (stage.hat > 0.02) {
const h = stage.hat * density * ramp;
addHat(left, sampleRate, t + beat / 2, h, index + 1);
if (h > 0.3) addHat(left, sampleRate, t + beat / 4, h * 0.6, index + 7);
}
}
addPad(left, sampleRate, from, to, stage.pad, stage.root * brightness);
}
for (let i = 0; i < length; i++) right[i] = left[i] * 0.98;
return buffer;
}
/** Silence, for degenerate-input checks. */ /** Silence, for degenerate-input checks. */
export function synthesizeSilence({ duration = 10, sampleRate = 44100 } = {}) { export function synthesizeSilence({ duration = 10, sampleRate = 44100 } = {}) {
return new MockAudioBuffer(2, Math.round(duration * sampleRate), sampleRate); return new MockAudioBuffer(2, Math.round(duration * sampleRate), sampleRate);

2
flow-state/src/audio/tempo.js
View File

@ -4,7 +4,7 @@
// a causal detector has to converge, and lags for the first several bars of every // a causal detector has to converge, and lags for the first several bars of every
// section. Here the grid is exact from frame zero, and phase is fitted globally. // section. Here the grid is exact from frame zero, and phase is fitted globally.
// //
// The click track (audio/clicktrack.js) exists to validate this by ear. If the // The metronome (audio/metronome.js) exists to validate this by ear. If the
// clicks don't sit on the beat, nothing downstream can be trusted — every timing // clicks don't sit on the beat, nothing downstream can be trusted — every timing
// artefact in the finished video traces back to this file. // artefact in the finished video traces back to this file.

21
flow-state/src/checks/blit.js Normal file
View File

@ -0,0 +1,21 @@
/**
* WebGL readback into a canvas, the right way up.
*
* Its own module because two debug pages need it and neither should pull in the
* other's dependencies to get it. It lived in checks/gallery.js, and importing
* it from the filmstrip dragged the entire gallery — the engine, the song bank,
* the look generator, the whole scene library — into a page that wanted ten
* lines of pixel copying.
*/
export function blit(canvas, pixels, width, height) {
canvas.width = width;
canvas.height = height;
const ctx = canvas.getContext('2d');
const image = ctx.createImageData(width, height);
const row = width * 4;
for (let y = 0; y < height; y++) {
const src = (height - 1 - y) * row;
image.data.set(pixels.subarray(src, src + row), y * row);
}
ctx.putImageData(image, 0, 0);
}

196
flow-state/src/checks/filmstrip.js Normal file
View File

@ -0,0 +1,196 @@
// The filmstrip: every song in the bank, sampled across its own length.
//
// The gallery answers "does this scene look the same in every song". This
// answers the question one level up and one axis over: does a SONG look the
// same as itself, half an hour of playback later.
//
// Nothing else shows that. The variety harness reduces a video to a number, the
// app shows one video at the speed of the song, and the phase gates never look
// at two moments of the same track side by side. So the failure this exists to
// catch — a video that is doing plenty and going nowhere — has until now been
// something you could only notice by watching four minutes and remembering what
// the first minute looked like.
//
// One frame every thirty seconds, straight across. A strip whose frames could
// be shuffled without anyone noticing is a video with no arc, whatever its
// drift score says. See look/Story.js for the layer meant to fix that, and
// checks/variety/signature.js directionOf for the same question as a number.
//
// Everything renders through the whole normal pipeline — Show, ArcDriver, post,
// the lot — rather than through a bare Engine as the gallery does, because the
// arc IS the subject here.
import { Show, FADE_SECONDS } from '../Show.js';
import { generateLook } from '../look/LookGenerator.js';
import { song, SONGS } from '../audio/songbank.js';
import { hashString } from '../engine/rng.js';
import { frameDescriptor, motionDescriptor } from './variety/descriptors.js';
import { descriptorDistance, directionOf, STRUCTURAL } from './variety/signature.js';
import { describeStory } from '../look/Story.js';
export const STRIP = { width: 256, height: 144 };
/** Seconds between frames. Thirty is a section or two at most tempos. */
export const DEFAULT_EVERY = 30;
/**
* How long a song is synthesised for.
*
* Long enough to have an arc to sample, short enough that the whole bank
* finishes in one sitting: every extra thirty seconds is another frame for each
* of seventeen songs, and each frame costs its warm-up. `?duration=240` for the
* fuller version when a specific song is in question.
*/
export const DEFAULT_DURATION = 180;
/**
* Frames of warm-up before each probe.
*
* Feedback and any stateful layer need to be converged or the probe measures
* the trail of a black frame — a structure every probe shares, which would make
* the whole strip look more alike than it is. Half of what the variety harness
* uses, because this is 17 songs deep and the cost is linear.
*/
const WARMUP = 24;
export const songNames = () => SONGS.map((s) => s.name);
const mean = (xs) => (xs.length ? xs.reduce((a, b) => a + b, 0) / xs.length : 0);
function blockMean(byBlock) {
const values = STRUCTURAL.map((b) => byBlock[b]).filter((v) => v !== undefined);
return mean(values);
}
/**
* Render one song's strip.
*
* @returns {{name, frames: Array, drift: number, direction: number, look}}
*/
export function renderSong(show, name, { duration, every }) {
const { track } = song(name, { duration });
// Seeded off the song's name, so a strip is stable between rebuilds and two
// songs are not accidentally handed the same look.
const look = generateLook(track, { seed: hashString(name) });
show.useTrack(track, look);
const frames = [];
const descriptors = [];
// Pulled clear of the opening and closing fades. A probe at t=0 reads a
// black frame — correct output, useless sample — and the same at the very
// end, so the first and last cells sit just inside the video proper. The
// caption shows the time actually sampled rather than the nominal one.
const inside = (t) => Math.min(
Math.max(t, FADE_SECONDS + 0.5),
Math.max(0, track.duration - FADE_SECONDS - 0.5));
for (let nominal = 0; nominal < track.duration - 0.5; nominal += every) {
const t = inside(nominal);
const frame = Math.min(track.frameCount - 2, Math.round(t * track.fps));
show.warmUp(frame, WARMUP);
const pixels = Uint8Array.from(show.readPixels(show.renderFrame(frame)));
// A second frame five later, so the motion block is measured rather
// than silently scored zero — the same omission the gallery had.
const moved = Uint8Array.from(show.readPixels(show.renderFrame(frame + 5)));
const state = show.arc.state;
const sectionIndex = track.sectionIndexAt(frame);
const section = look.sections[sectionIndex] || {};
frames.push({
time: t,
frame,
pixels,
kind: state.kind || section.kind || '',
scene: state.sceneName || '',
act: state.act || '',
tension: state.tension ?? 0.5,
journey: state.journey ?? 0,
moment: momentAt(look.story, sectionIndex),
});
const still = frameDescriptor(pixels, STRIP.width, STRIP.height);
const motion = motionDescriptor(pixels, moved, STRIP.width, STRIP.height);
descriptors.push({ ...still, motion: motion.scale.concat(motion.layout) });
}
// The two numbers that say what the strip shows, on the same descriptor the
// variety harness uses: how far this video gets from itself, and whether
// that distance grows with time or is just churn.
const gaps = [];
const dists = [];
for (let i = 0; i < descriptors.length; i++) {
for (let j = i + 1; j < descriptors.length; j++) {
gaps.push(frames[j].time - frames[i].time);
dists.push(blockMean(descriptorDistance(descriptors[i], descriptors[j])));
}
}
return {
name,
frames,
drift: mean(dists),
direction: directionOf(gaps, dists),
// How much the FIRST half of the video resembles the last — the reading
// that separates a video which ends somewhere new from one that has
// come back on purpose. A recap should show up here and nowhere else.
endsApart: dists.length ? dists[dists.length - 1] : 0,
story: look.story,
storyLine: describeStory(look.story),
director: look.director,
bpm: Math.round(track.summary.bpm),
sections: look.sections.length,
};
}
function momentAt(story, index) {
if (!story || !story.moments) return '';
const m = story.moments;
if (index === m.climax) return 'climax';
if (index === m.turn) return 'turn';
if (index === m.arrival) return 'arrival';
if (index === m.resolution) return 'resolution';
return '';
}
/**
* Build every strip, reporting each song as it lands.
*
* One Show for the whole run: `useTrack` replaces the track and rebuilds the
* arc, and constructing a Show per song would rebuild the GL context seventeen
* times for nothing.
*/
export async function buildFilmstrip({
duration = DEFAULT_DURATION, every = DEFAULT_EVERY, names = null,
onSong = null, onSongStart = null,
} = {}) {
const list = names && names.length ? names : songNames();
const show = new Show({ ...STRIP });
const rows = [];
try {
for (let i = 0; i < list.length; i++) {
// Announced BEFORE the work, not only after it. Synthesising and
// rendering one song is tens of seconds, and a page that says
// nothing until the first one lands looks broken for exactly as
// long as the first one takes.
if (onSongStart) {
onSongStart(i + 1, list.length, list[i]);
await new Promise((r) => setTimeout(r, 0));
}
let row;
try {
row = renderSong(show, list[i], { duration, every });
} catch (err) {
row = { name: list[i], frames: [], drift: 0, direction: 0, error: err.message };
}
rows.push(row);
if (onSong) onSong(i + 1, list.length, row);
// Yield, so a row paints as it lands instead of the page freezing
// for the whole run and then showing everything at once.
await new Promise((r) => setTimeout(r, 0));
}
} finally {
show.dispose();
}
return rows;
}

153
flow-state/src/checks/gallery-cache.js Normal file
View File

@ -0,0 +1,153 @@
// Caching for the gallery, keyed on the source that produced it.
//
// Building the gallery takes three minutes of GPU work, which is fine once and
// intolerable every time the page is opened. But a cache that has to be cleared
// by hand is worse than no cache: it will eventually show you last week's
// pixels while you are trying to judge this morning's change, and you will trust
// it because it looks like a render.
//
// So the key is a fingerprint of the code itself. Edit any file under src/ and
// the fingerprint changes, the old entry stops matching, and the gallery
// rebuilds without being asked. Nothing to remember and nothing to invalidate.
//
// Thumbnails are stored as WebP blobs in IndexedDB rather than raw pixels in
// localStorage: sixty-five scenes at six frames of 256x144 is about 57MB raw,
// which localStorage would refuse and IndexedDB would rather not hold either.
// Compressed it is a few megabytes.
const DB_NAME = 'flow-state-gallery';
const STORE = 'builds';
const DB_VERSION = 1;
/**
* A fingerprint of every source file that can change what the gallery renders.
*
* `import.meta.glob` is resolved by Vite at build time, so this covers the
* shaders, the identity, the look generator, the engine and the descriptors
* without naming any of them — which matters, because the file that invalidates
* a render is exactly the one nobody remembers to list.
*/
const SOURCES = import.meta.glob('/src/**/*.js', { query: '?raw', import: 'default', eager: true });
export function sourceFingerprint() {
// Sorted, so the hash does not depend on glob iteration order.
const paths = Object.keys(SOURCES).sort();
let h = 0x811c9dc5 >>> 0;
const mix = (str) => {
for (let i = 0; i < str.length; i++) {
h ^= str.charCodeAt(i);
h = Math.imul(h, 0x01000193) >>> 0;
}
};
for (const path of paths) {
mix(path);
mix(String(SOURCES[path]));
}
return `${paths.length}-${h.toString(16).padStart(8, '0')}`;
}
function openDb() {
return new Promise((resolve, reject) => {
const request = indexedDB.open(DB_NAME, DB_VERSION);
request.onupgradeneeded = () => {
const db = request.result;
if (!db.objectStoreNames.contains(STORE)) db.createObjectStore(STORE);
};
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
});
}
function tx(db, mode, fn) {
return new Promise((resolve, reject) => {
const t = db.transaction(STORE, mode);
const request = fn(t.objectStore(STORE));
request.onsuccess = () => resolve(request.result);
request.onerror = () => reject(request.error);
});
}
/** The cached build for this exact source, or null. */
export async function loadGallery(fingerprint) {
try {
const db = await openDb();
const record = await tx(db, 'readonly', (store) => store.get(fingerprint));
db.close();
return record || null;
} catch {
// A debug page is not worth failing over a storage quota or a private
// window that refuses IndexedDB. Fall through to rebuilding.
return null;
}
}
/**
* Which page a key belongs to.
*
* The store holds more than one page's builds now — the gallery keys on the
* bare source fingerprint, the filmstrip prefixes its sampling parameters — and
* "drop every other build" has to mean every other build OF THIS PAGE. Without
* the namespace the two evict each other on every save, and each page rebuilds
* for three minutes every time you visit the other one.
*/
const namespaceOf = (key) => {
const k = String(key);
return k.includes(':') ? k.slice(0, k.indexOf(':')) : 'gallery';
};
/**
* Store a build, and drop every other one from the same page.
*
* Only the current source is ever wanted, and keeping stale builds around is how
* a cache quietly grows to hundreds of megabytes of images nobody will look at.
*/
export async function saveGallery(fingerprint, payload) {
try {
const db = await openDb();
const keys = await tx(db, 'readonly', (store) => store.getAllKeys());
const mine = namespaceOf(fingerprint);
await tx(db, 'readwrite', (store) => {
for (const key of keys) {
if (key !== fingerprint && namespaceOf(key) === mine) store.delete(key);
}
return store.put(payload, fingerprint);
});
db.close();
return true;
} catch {
return false;
}
}
export async function clearGallery() {
try {
const db = await openDb();
await tx(db, 'readwrite', (store) => store.clear());
db.close();
} catch { /* nothing to clear */ }
}
/** Raw RGBA (bottom-up, as WebGL hands it over) to a compressed blob. */
export function pixelsToBlob(pixels, width, height) {
const canvas = document.createElement('canvas');
canvas.width = width;
canvas.height = height;
const ctx = canvas.getContext('2d');
const image = ctx.createImageData(width, height);
const row = width * 4;
for (let y = 0; y < height; y++) {
const src = (height - 1 - y) * row;
image.data.set(pixels.subarray(src, src + row), y * row);
}
ctx.putImageData(image, 0, 0);
return new Promise((resolve) => canvas.toBlob(resolve, 'image/webp', 0.85));
}
/** Draw a stored blob into a canvas, sized to it. */
export async function blobToCanvas(canvas, blob) {
const bitmap = await createImageBitmap(blob);
canvas.width = bitmap.width;
canvas.height = bitmap.height;
canvas.getContext('2d').drawImage(bitmap, 0, 0);
bitmap.close();
}

209
flow-state/src/checks/gallery.js Normal file
View File

@ -0,0 +1,209 @@
// The gallery: every visualizer, six times, on six different songs' content.
//
// The complaint this answers is one a metric could not raise — watching a few
// videos, certain scenes announce themselves. You have seen that one before.
// You cannot tell which ones from a number, because the harness measures whole
// videos and a scene that always looks like itself is averaged in with
// everything around it.
//
// So this renders each scene under six complete identities — six casts, six ink
// treatments, six lattices, six palettes, six parameter draws — and puts them
// side by side. A scene whose six thumbnails are interchangeable is a scene the
// song cannot change, and that is the definition of the problem.
//
// Then it scores them, using the same structural descriptor the variety harness
// runs on, so the gallery can be sorted worst-first. Browsing sixty-six scenes
// looking for the repetitive ones is exactly the job a sort order should do.
import { Engine } from '../engine/Engine.js';
import { blit } from './blit.js';
import { scenes } from '../scenes/registry.js';
import { sampleValues } from '../params/schema.js';
import { Rng, hashString } from '../engine/rng.js';
import { featureProviderFor } from '../audio/FeatureTrack.js';
import { songBank } from '../audio/songbank.js';
import { generateLook } from '../look/LookGenerator.js';
import { describeIdentity } from '../look/Identity.js';
import { frameDescriptor, motionDescriptor } from './variety/descriptors.js';
import { frameLuminance, frameVariance } from '../engine/hash.js';
import { descriptorDistance, STRUCTURAL } from './variety/signature.js';
const THUMB = { width: 256, height: 144 };
/*
* There used to be a MIN_VARIETY floor here — 0.1, drawn as a red line across
* the gallery — on the theory that a scene which looks the same in every song
* leaks that sameness between videos.
*
* That was true when a section was ONE scene. It is not any more: a section is
* a ground, a shot over it and sometimes a pass over that, so what a viewer
* sees is a combination, and a scene that is reliably itself is a perfectly
* good ingredient in one. Held against a floor, such scenes were failing for
* being consistent.
*
* The score is still measured, still reported, and still what the gallery sorts
* by — it is genuinely the right question to ask about a scene you are working
* on. It is no longer a bar anything has to clear, and the interesting quantity
* moved up a level: how unalike the scenes in one stack are. That lives in
* scenes/metadata.json and is read by the look generator.
*/
/**
* Six contexts, one per song: everything a scene is handed when it is cast.
*
* Built from real bank entries rather than invented, so what the gallery shows
* is what the generator would actually produce — the same identities, palettes
* and section biases, only with the scene held fixed instead of chosen.
*/
export function galleryContexts(count = 6) {
return songBank({ count }).map((entry) => {
const look = generateLook(entry.track, { seed: hashString(entry.name) });
// The busiest section, because that is where a scene is asked for the
// most and where two scenes are most likely to converge.
const section = look.sections.reduce(
(best, s) => (s.bias.energy > best.bias.energy ? s : best), look.sections[0]);
return {
name: entry.name,
track: entry.track,
palette: look.palette,
personality: look.personality,
bias: section.bias,
frame: section.startFrame + Math.floor((section.endFrame - section.startFrame) * 0.5),
identity: describeIdentity(look.personality.identity),
};
});
}
/**
* Render one scene across every context.
*
* @returns {{thumbs: Uint8Array[], variety: number, byBlock: object}}
*/
export function renderScene(engine, module, contexts) {
const thumbs = [];
const descriptors = [];
for (const ctx of contexts) {
engine.timeline.setDuration(ctx.track.duration);
engine.setFeatureProvider(featureProviderFor(ctx.track));
const rng = new Rng(hashString(`${module.name}:${ctx.name}`));
const params = sampleValues(module, rng, ctx.bias, ctx.personality.temperament);
engine.setLayerSpecs([{
module,
params,
seed: rng.int(0, 0x7fffffff),
opacity: 1,
blend: 'normal',
palette: ctx.palette,
personality: ctx.personality,
}]);
engine.compositor.reset();
// A few frames of warm-up so anything with state is past its first frame.
for (let f = ctx.frame - 6; f < ctx.frame; f++) engine.renderFrame(f);
const pixels = Uint8Array.from(engine.readPixels(engine.renderFrame(ctx.frame)));
// A second frame, so MOTION is measured rather than silently scored zero.
//
// The descriptor has five structural blocks and this only ever built
// four of them, so `motion` came back 0.000 for every scene in the
// library and dragged the mean down by a fifth across the board. It
// looked like a property of the scenes; it was a missing render.
const moved = Uint8Array.from(engine.readPixels(engine.renderFrame(ctx.frame + 5)));
thumbs.push(pixels);
const still = frameDescriptor(pixels, THUMB.width, THUMB.height);
const motion = motionDescriptor(pixels, moved, THUMB.width, THUMB.height);
descriptors.push({ ...still, motion: motion.scale.concat(motion.layout) });
}
// How different this scene's six outputs are from each other, on the same
// structural descriptor the variety harness uses — colour excluded, because
// six palettes would otherwise make every scene look varied.
const byBlock = {};
let total = 0, pairs = 0;
for (let i = 0; i < descriptors.length; i++) {
for (let j = i + 1; j < descriptors.length; j++) {
const d = descriptorDistance(descriptors[i], descriptors[j]);
for (const b of [...STRUCTURAL, 'colour']) byBlock[b] = (byBlock[b] || 0) + (d[b] || 0);
total += STRUCTURAL.reduce((a, b) => a + (d[b] || 0), 0) / STRUCTURAL.length;
pairs++;
}
}
for (const b of Object.keys(byBlock)) byBlock[b] /= pairs || 1;
// How much of the frame this scene paints, averaged over the six. Free —
// the pixels are already here — and it is the number that explains the
// ranking: a scene covering 2% of the frame is a few bright things on
// black, which scores well for variety and is thin to watch on its own.
// Whether that is a problem depends on whether it is ever layered.
let covered = 0;
for (const px of thumbs) {
let lit = 0;
for (let i = 0; i < px.length; i += 4) {
if (px[i] + px[i + 1] + px[i + 2] > 90) lit++;
}
covered += lit / (px.length / 4) / thumbs.length;
}
// A dead shader scores zero on every block, which is indistinguishable from
// a very boring scene if you only read the number — and it happened: the
// subject helpers were declared above the ink they call, the whole preamble
// failed to compile, and every scene rendered black while the gallery
// reported a variety of exactly 0.000. Say which it is.
const lum = frameLuminance(thumbs[0]);
const variance = frameVariance(thumbs[0]);
const dead = lum < 0.002 || variance < 0.001;
return {
thumbs,
// The six per-context descriptors, so a caller can average them into a
// profile rather than re-render the library to get one. See
// checks/metadata.js — this function is the only place the scenes are
// rendered under real identities, and everything measured about a scene
// comes out of here.
descriptors,
variety: pairs ? total / pairs : 0,
coverage: covered,
byBlock,
error: dead
? `renders nothing — luminance ${lum.toFixed(4)}, variance ${variance.toFixed(4)}. ` +
'A failed shader compile scores 0.000 on every block; check the console for GLSL errors.'
: undefined,
};
}
/**
* Build the whole gallery, reporting progress as it goes.
*
* @param {(done:number, of:number, name:string, row:object) => void} onScene
*/
export async function buildGallery({ contexts, onScene, only = null } = {}) {
const list = only
? scenes.filter((m) => only.includes(m.name))
: scenes.filter((m) => m.kind === 'fragment');
const engine = new Engine({ ...THUMB });
const rows = [];
try {
for (let i = 0; i < list.length; i++) {
const module = list[i];
let row;
try {
row = { module, ...renderScene(engine, module, contexts) };
} catch (err) {
row = { module, thumbs: [], variety: 0, byBlock: {}, error: err.message };
}
rows.push(row);
if (onScene) onScene(i + 1, list.length, module.name, row);
// Yield so the page can paint each row as it lands rather than
// freezing for a minute and then showing everything at once.
await new Promise((r) => setTimeout(r, 0));
}
} finally {
engine.dispose();
}
return rows;
}
export { blit, THUMB };

104
flow-state/src/checks/main.js
View File

@ -1,5 +1,9 @@
import { runAll, summarize, allChecks } from './framework.js'; import { runAll, summarize, allChecks } from './framework.js';
import { runSceneGate } from './scene-gate.js'; import { runSceneGate } from './scene-gate.js';
import {
varietyReportLines, songVarietyReportLines, experimentReportLines, poolSweepLines,
decomposeReportLines,
} from './variety/print.js';
// Registering a phase's checks is a side effect of importing it. // Registering a phase's checks is a side effect of importing it.
import './phase0.js'; import './phase0.js';
@ -14,6 +18,8 @@ import './phase8.js';
import './phase9.js'; import './phase9.js';
import './phase10.js'; import './phase10.js';
import './phase11.js'; import './phase11.js';
import './phase12.js';
import './phase13.js';
const out = document.getElementById('results'); const out = document.getElementById('results');
const summaryEl = document.getElementById('summary'); const summaryEl = document.getElementById('summary');
@ -51,6 +57,104 @@ async function main() {
return; return;
} }
// Seed variety mode: the full diagnostic table rather than a pass/fail.
// Phase 12 gates on this measurement, but a gate answers "is it bad" and
// this answers "which axis is flat", which is the question you have while
// fixing it.
//
// checks.html?variety=1 8 seeds, plus the whole library
// checks.html?variety=1&seeds=16 slower, tighter
// checks.html?variety=1&library=0 skip the library sweep (much faster)
if (params.get('variety')) {
summaryEl.textContent = 'seed variety: rendering seeds and sweeping the library…';
const started = Date.now();
const { lines, ok, headline } = await varietyReportLines({
seeds: Number(params.get('seeds')) || 8,
probes: Number(params.get('probes')) || 5,
library: params.get('library') !== '0',
});
out.innerHTML = `<pre class="scene-gate">${lines.join('\n')}</pre>`;
summaryEl.textContent = `${headline} · ${((Date.now() - started) / 1000).toFixed(1)}s`;
summaryEl.className = ok ? 'ok' : 'bad';
window.__CHECKS__ = { variety: true, ok, lines };
window.__CHECKS_DONE__ = true;
console.log('[variety]\n' + lines.join('\n'));
return;
}
// Song variety mode: the same instrument, with the SONG as the variable.
//
// checks.html?songs=1 6 songs from the bank
// checks.html?songs=1&count=10 more of the bank, slower
if (params.get('songs')) {
summaryEl.textContent = 'song variety: synthesising the bank and rendering each song…';
const started = Date.now();
const { lines, ok, headline } = await songVarietyReportLines({
songs: Number(params.get('count')) || 6,
probes: Number(params.get('probes')) || 5,
});
out.innerHTML = `<pre class="scene-gate">${lines.join('\n')}</pre>`;
summaryEl.textContent = `${headline} · ${((Date.now() - started) / 1000).toFixed(1)}s`;
summaryEl.className = ok ? 'ok' : 'bad';
window.__CHECKS__ = { songs: true, ok, lines };
window.__CHECKS_DONE__ = true;
console.log('[song-variety]\n' + lines.join('\n'));
return;
}
// The Epic 3 A/B: stages against comparable legacy scenes.
// checks.html?experiment=1
if (params.get('experiment')) {
summaryEl.textContent = 'epic 3 experiment: rendering three arms across the song bank…';
const started = Date.now();
const { lines, ok, headline } = await experimentReportLines({
songs: Number(params.get('count')) || 6,
probes: Number(params.get('probes')) || 4,
repeats: Number(params.get('repeats')) || 3,
});
out.innerHTML = `<pre class="scene-gate">${lines.join('\n')}</pre>`;
summaryEl.textContent = `${headline} · ${((Date.now() - started) / 1000).toFixed(1)}s`;
summaryEl.className = ok ? 'ok' : 'bad';
window.__CHECKS__ = { experiment: true, ok, lines };
window.__CHECKS_DONE__ = true;
console.log('[experiment]\n' + lines.join('\n'));
return;
}
// checks.html?sweep=1 — how big a track's casting pool should be.
if (params.get('sweep')) {
summaryEl.textContent = 'sweeping casting pool size across the song bank…';
const started = Date.now();
const { lines, ok, headline } = await poolSweepLines({
songs: Number(params.get('count')) || 6,
sizes: params.get('sizes') ? params.get('sizes').split(',').map(Number) : null,
repeats: Number(params.get('repeats')) || 1,
});
out.innerHTML = `<pre class="scene-gate">${lines.join('\n')}</pre>`;
summaryEl.textContent = `${headline} · ${((Date.now() - started) / 1000).toFixed(1)}s`;
summaryEl.className = ok ? 'ok' : 'bad';
window.__CHECKS__ = { sweep: true, ok, lines };
window.__CHECKS_DONE__ = true;
console.log('[sweep]\n' + lines.join('\n'));
return;
}
// checks.html?decompose=1 — identity against container, measured apart.
if (params.get('decompose')) {
summaryEl.textContent = 'decomposing identity vs container…';
const started = Date.now();
const { lines, ok, headline } = await decomposeReportLines({
songs: Number(params.get('count')) || 6,
});
out.innerHTML = `<pre class="scene-gate">${lines.join('\n')}</pre>`;
summaryEl.textContent = `${headline} · ${((Date.now() - started) / 1000).toFixed(1)}s`;
summaryEl.className = ok ? 'ok' : 'bad';
window.__CHECKS__ = { decompose: true, ok, lines };
window.__CHECKS_DONE__ = true;
console.log('[decompose]\n' + lines.join('\n'));
return;
}
const phaseArg = params.get('phase'); const phaseArg = params.get('phase');
const phases = phaseArg ? phaseArg.split(',').map(Number) : null; const phases = phaseArg ? phaseArg.split(',').map(Number) : null;
const skipSlow = params.get('slow') !== '1'; const skipSlow = params.get('slow') !== '1';

217
flow-state/src/checks/metadata.js Normal file
View File

@ -0,0 +1,217 @@
// Measuring every visualizer, and writing the answers back into the repo.
//
// The library has always had two kinds of fact about a scene. Declared ones —
// family, traits, `consumes` — which say what the scene is FOR, and measured
// ones, which say what it actually does when rendered. Declared facts belong in
// the scene file. Measured facts do not: hand-written, they drift the moment a
// shader changes, and nine scenes declaring `surface: 'canvas'` while painting
// under a third of the frame is what that drift looks like.
//
// So the measured half lives in scenes/metadata.json, generated from here,
// tracked in git, and stamped with a fingerprint of everything that could
// change it. When the fingerprint stops matching, the numbers are stale and the
// phase 12 gate says so — the file is a cache of a render, and a cache nobody
// can tell is stale is worse than no cache.
//
// The measurement is the GALLERY's: six songs, sampled parameters, real
// identities and palettes — a scene as it is actually cast, not as it renders
// at default parameters. The difference is not academic. Salt Flat paints 65%
// of the frame at defaults and 32% across six real songs, and the generator
// chooses grounds with this number.
import { Engine } from '../engine/Engine.js';
import { scenes } from '../scenes/registry.js';
import { galleryContexts, renderScene, THUMB } from './gallery.js';
import { readsHistory } from '../params/schema.js';
import { GROUND_BIAS, groundTemperamentFrom, groundPersonalityFrom } from '../scenes/surface.js';
import metadata from '../scenes/metadata.json';
/**
* What invalidates the measurements.
*
* Deliberately NOT every file under src/, which is what the gallery cache
* fingerprints: that changes when the UI changes, and it would mark the
* metadata stale for edits that cannot move a single number. What can move one
* is the scenes themselves, the contract they are compiled against, the
* identities and palettes they are handed, and the metric definitions — so
* those, and nothing else.
*
* Globbed rather than listed wherever a whole directory qualifies, because the
* file that invalidates a measurement is exactly the one nobody remembers to
* add to a list.
*/
const SOURCES = {
...import.meta.glob('/src/scenes/**/*.js', { query: '?raw', import: 'default', eager: true }),
...import.meta.glob('/src/checks/variety/descriptors.js', { query: '?raw', import: 'default', eager: true }),
...import.meta.glob('/src/checks/variety/signature.js', { query: '?raw', import: 'default', eager: true }),
...import.meta.glob('/src/checks/gallery.js', { query: '?raw', import: 'default', eager: true }),
...import.meta.glob('/src/checks/metadata.js', { query: '?raw', import: 'default', eager: true }),
...import.meta.glob('/src/engine/shader-contract.js', { query: '?raw', import: 'default', eager: true }),
...import.meta.glob('/src/look/Identity.js', { query: '?raw', import: 'default', eager: true }),
...import.meta.glob('/src/look/Personality.js', { query: '?raw', import: 'default', eager: true }),
...import.meta.glob('/src/look/palette.js', { query: '?raw', import: 'default', eager: true }),
...import.meta.glob('/src/audio/songbank.js', { query: '?raw', import: 'default', eager: true }),
};
/** The version of the measurement itself. Bump to force a refresh of everything. */
export const SCHEMA = 4;
export function metricsFingerprint() {
let h = 0x811c9dc5 >>> 0;
const mix = (str) => {
for (let i = 0; i < str.length; i++) {
h ^= str.charCodeAt(i);
h = Math.imul(h, 0x01000193) >>> 0;
}
};
mix(`schema:${SCHEMA}`);
for (const path of Object.keys(SOURCES).sort()) {
mix(path);
mix(SOURCES[path]);
}
return h.toString(16).padStart(8, '0');
}
/** Whether the checked-in metadata was measured from the code that is here now. */
export function metadataIsFresh() {
return metadata.fingerprint === metricsFingerprint();
}
/** Anything a viewer would read as painted rather than as backdrop. */
function litFraction(pixels) {
let lit = 0;
for (let i = 0; i < pixels.length; i += 4) {
if (pixels[i] + pixels[i + 1] + pixels[i + 2] > 90) lit++;
}
return lit / (pixels.length / 4);
}
const round = (x, places = 4) => Number(x.toFixed(places));
/** Mean of the six per-context descriptors, block by block. */
function meanProfile(descriptors) {
const out = {};
for (const block of Object.keys(descriptors[0])) {
const length = descriptors[0][block].length;
const acc = new Array(length).fill(0);
for (const d of descriptors) {
for (let i = 0; i < length; i++) acc[i] += d[block][i] / descriptors.length;
}
out[block] = acc.map((v) => round(v));
}
return out;
}
/**
* Measure the whole library.
*
* Per scene: how much frame it paints, how much it changes between songs, and
* its mean structural profile — the descriptor the variety harness compares
* videos with, averaged over the six renders. The profile is what makes this
* more than a list of numbers: two profiles can be compared, so the generator
* can ask whether a shot and the thing under it are the same picture twice.
*
* @returns {object} the metadata file's contents
*/
export function measureLibrary({ onScene = null, contexts = null } = {}) {
const ctx = contexts || galleryContexts(6);
// The same six songs, sampled the way a BED is. Coverage is mostly a
// function of a scene's parameters, so "how much does this paint" has two
// answers and the generator needs both: one for the budget, and one for
// whether it may be a ground at all. See GROUND_BIAS.
const bedCtx = ctx.map((c) => ({
...c,
bias: { ...c.bias, ...GROUND_BIAS },
personality: {
...groundPersonalityFrom(c.personality),
temperament: groundTemperamentFrom(c.personality.temperament),
},
}));
const engine = new Engine({ ...THUMB });
const out = {};
try {
const list = scenes.filter((m) => m.kind === 'fragment');
for (const module of list) {
const { thumbs, variety, byBlock, descriptors, error } = renderScene(engine, module, ctx);
const bed = renderScene(engine, module, bedCtx);
out[module.name] = {
coverage: round(thumbs.reduce((s, px) => s + litFraction(px), 0) / thumbs.length, 3),
// The WORST of the six, not the mean. What a ground has to
// promise is a filled frame in the video it lands in, and the
// spread across identities is enormous: a track whose ink
// treatment is `hollow` draws outlines instead of fills, so a
// scene that paints 61% averaged over six songs paints 2% in
// the one that asked for outlines — measured, and it is how a
// section with a ground under it still rendered near-black.
// A mean cannot make a promise; a minimum can.
bedCoverage: round(Math.min(...bed.thumbs.map(litFraction)), 3),
bedCoverageMean: round(
bed.thumbs.reduce((s, px) => s + litFraction(px), 0) / bed.thumbs.length, 3),
variety: round(variety, 3),
blocks: Object.fromEntries(
Object.entries(byBlock).map(([b, v]) => [b, round(v, 3)])),
// Declared, not measured, and carried here anyway: it is a fact
// about the scene that the composition rules read, and having
// every compositional input in one file is the point.
readsHistory: readsHistory(module),
profile: meanProfile(descriptors),
...(error ? { error } : {}),
};
if (onScene) onScene(Object.keys(out).length, list.length, module.name);
}
} finally {
engine.dispose();
}
return {
fingerprint: metricsFingerprint(),
schema: SCHEMA,
measured: new Date().toISOString().slice(0, 10),
contexts: ctx.map((c) => c.name),
scenes: out,
};
}
/** What moved against the checked-in file. */
export function metadataDrift(fresh) {
// Tolerates a missing or half-written file on purpose: this runs on the way
// to REPLACING it, and refusing to report because the thing being replaced
// is malformed is the least useful moment to be strict.
const previous = (metadata && metadata.scenes) || {};
const moved = [];
for (const [name, row] of Object.entries(fresh.scenes)) {
const was = previous[name];
if (!was) {
moved.push({ name, note: 'new' });
continue;
}
const delta = row.coverage - was.coverage;
if (Math.abs(delta) > 0.02) {
moved.push({
name, note: `coverage ${(was.coverage * 100).toFixed(0)}% → ${(row.coverage * 100).toFixed(0)}%`,
delta,
});
}
}
const gone = Object.keys(previous).filter((n) => !fresh.scenes[n]);
return {
moved: moved.sort((a, b) => Math.abs(b.delta || 0) - Math.abs(a.delta || 0)),
gone,
};
}
/**
* Ask the dev server to write the file back into the source tree.
*
* Dev-only by construction — the endpoint is a middleware in vite.config.js. A
* built page has no source tree to write to, and failing there is correct.
*/
export async function writeMetadata(fresh) {
const response = await fetch('/__metadata', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(fresh, null, 2) + '\n',
});
if (!response.ok) throw new Error(`${response.status} ${await response.text()}`);
return response.text();
}

10
flow-state/src/checks/phase10.js
View File

@ -37,6 +37,7 @@ import { frameDistance, frameLuminance } from '../engine/hash.js';
import { battery as timbreBattery } from './phase3.js'; import { battery as timbreBattery } from './phase3.js';
import { grainEnvelope } from '../look/grain.js'; import { grainEnvelope } from '../look/grain.js';
import { signatureUniforms } from '../look/Personality.js'; import { signatureUniforms } from '../look/Personality.js';
import { subjectOf, overlaysOf } from '../look/stack.js';
/** /**
* Several tracks that genuinely differ in what they sound like — the population * Several tracks that genuinely differ in what they sound like — the population
@ -124,8 +125,8 @@ check(10, 'one scene looks different in two different videos', () => {
// track's temperament — the question is what this scene WOULD // track's temperament — the question is what this scene WOULD
// look like in that video, and falling back to defaults would // look like in that video, and falling back to defaults would
// compare two identical parameter sets and prove nothing. // compare two identical parameter sets and prove nothing.
const stack = stacksOf(look).find((s) => s[0].module === module); const stack = stacksOf(look).find((s) => subjectOf(s).module === module);
const params = stack ? stack[0].params : sampleValues( const params = stack ? subjectOf(stack).params : sampleValues(
module, module,
new Rng(look.seed ^ 0x51ed270b), new Rng(look.seed ^ 0x51ed270b),
look.sections[0].bias, look.sections[0].bias,
@ -187,7 +188,7 @@ check(10, 'overlays happen sometimes and not always', () => {
const look = generateLook(track, { seed: 900 + s * 5231 }); const look = generateLook(track, { seed: 900 + s * 5231 });
for (const stack of stacksOf(look)) { for (const stack of stacksOf(look)) {
stacks++; stacks++;
const overlay = stack.slice(1).find((l) => l.module.role !== 'accent'); const overlay = overlaysOf(stack)[0];
if (overlay) { if (overlay) {
withOverlay++; withOverlay++;
blends.add(overlay.blend); blends.add(overlay.blend);
@ -214,8 +215,7 @@ check(10, 'an overlay never hides the shot underneath it', () => {
for (const { track } of battery()) { for (const { track } of battery()) {
for (let s = 0; s < 4; s++) { for (let s = 0; s < 4; s++) {
for (const stack of stacksOf(generateLook(track, { seed: 1300 + s * 8641 }))) { for (const stack of stacksOf(generateLook(track, { seed: 1300 + s * 8641 }))) {
for (const layer of stack.slice(1)) { for (const layer of overlaysOf(stack)) {
if (layer.module.role === 'accent') continue;
if (layer.opacity > 0.6) { if (layer.opacity > 0.6) {
problems.push(`${layer.module.name} at ${layer.opacity.toFixed(2)}`); problems.push(`${layer.module.name} at ${layer.opacity.toFixed(2)}`);
} }

272
flow-state/src/checks/phase11.js
View File

@ -21,10 +21,12 @@ import { shiftPalette, paletteContrast } from '../look/palette.js';
import { ArcDriver } from '../look/ArcDriver.js'; import { ArcDriver } from '../look/ArcDriver.js';
import { Engine } from '../engine/Engine.js'; import { Engine } from '../engine/Engine.js';
import { featureProviderFor } from '../audio/FeatureTrack.js'; import { featureProviderFor } from '../audio/FeatureTrack.js';
import { sampleValues, clampValue } from '../params/schema.js'; import { sampleValues, clampValue, canBackground } from '../params/schema.js';
import { Rng } from '../engine/rng.js'; import { Rng } from '../engine/rng.js';
import { frameDistance, frameLuminance } from '../engine/hash.js'; import { frameDistance, frameLuminance } from '../engine/hash.js';
import { SHOT_SIZES, SHOT_SIZE_NAMES, describeFraming } from '../look/framing.js'; import { SHOT_SIZES, SHOT_SIZE_NAMES, describeFraming } from '../look/framing.js';
import { reachFor, CURVE_NAMES } from '../look/Camera.js';
import { subjectOf } from '../look/stack.js';
/** Tracks at several tempos: shot length is measured in bars, so tempo is the axis. */ /** Tracks at several tempos: shot length is measured in bars, so tempo is the axis. */
let cached = null; let cached = null;
@ -188,11 +190,11 @@ check(11, 'every scene in the library is reachable', () => {
} }
} }
// Accents are cast by role rather than by family and are always eligible. // Accents are cast by role rather than by family and are always eligible.
const missing = scenes.filter((m) => m.role !== 'accent' && !reachable.has(m.name)); const missing = scenes.filter((m) => canBackground(m) && !reachable.has(m.name));
return expect(missing.length === 0, return expect(missing.length === 0,
missing.length ? `unreachable: ${missing.map((m) => m.name).join(', ')}` missing.length ? `unreachable: ${missing.map((m) => m.name).join(', ')}`
: `all ${reachable.size} non-accent scenes reachable across ${DIRECTORS.length} directors`); : `all ${reachable.size} castable scenes reachable across ${DIRECTORS.length} directors`);
}); });
check(11, 'no director starves a section kind', () => { check(11, 'no director starves a section kind', () => {
@ -204,7 +206,7 @@ check(11, 'no director starves a section kind', () => {
for (const [kind, families] of Object.entries(d.families)) { for (const [kind, families] of Object.entries(d.families)) {
const pool = new Set(); const pool = new Set();
for (const f of families) { for (const f of families) {
for (const m of scenesInFamily(f)) if (m.role !== 'accent') pool.add(m.name); for (const m of scenesInFamily(f)) if (canBackground(m)) pool.add(m.name);
} }
if (pool.size < 12) problems.push(`${d.name}/${kind}: only ${pool.size}`); if (pool.size < 12) problems.push(`${d.name}/${kind}: only ${pool.size}`);
} }
@ -272,7 +274,7 @@ check(11, 'two tracks do not agree on what a section kind looks like', () => {
directors.add(look.director); directors.add(look.director);
look.sections.forEach((section) => { look.sections.forEach((section) => {
const set = seen.get(section.kind) || new Set(); const set = seen.get(section.kind) || new Set();
(section.variants || [section.layers]).forEach((v) => set.add(v[0].module.family)); (section.variants || [section.layers]).forEach((v) => set.add(subjectOf(v).module.family));
seen.set(section.kind, set); seen.set(section.kind, set);
}); });
} }
@ -298,10 +300,10 @@ check(11, 'a track shows more of the library than it used to', () => {
for (let s = 0; s < 4; s++) { for (let s = 0; s < 4; s++) {
const look = generateLook(t, { seed: (i * 2654435761 + s * 40503) >>> 0 }); const look = generateLook(t, { seed: (i * 2654435761 + s * 40503) >>> 0 });
look.sections.forEach((section) => look.sections.forEach((section) =>
(section.variants || [section.layers]).forEach((v) => cast.add(v[0].module.name))); (section.variants || [section.layers]).forEach((v) => cast.add(subjectOf(v).module.name)));
} }
}); });
const pool = scenes.filter((m) => m.role !== 'accent').length; const pool = scenes.filter(canBackground).length;
return expect(cast.size >= pool * 0.4, return expect(cast.size >= pool * 0.4,
`${cast.size}/${pool} scenes cast across 12 tracks (floor ${Math.ceil(pool * 0.4)})`); `${cast.size}/${pool} scenes cast across 12 tracks (floor ${Math.ceil(pool * 0.4)})`);
}); });
@ -491,8 +493,11 @@ check(11, 'the slow axis is a journey rather than a cycle', () => {
try { try {
const cue = arc.cues[0]; const cue = arc.cues[0];
const at = (time) => arc._paramsAt(cue, 0, time, t.at(Math.round(time * 60))); // The shot, not the bed under it — the slow axis is a claim about the
const spec = arc._specFor(cue.sectionIndex, cue.variant, 0); // scene the section is about.
const slot = arc._subjectSlot(cue);
const at = (time) => arc._paramsAt(cue, slot, time, t.at(Math.round(time * 60)));
const spec = arc._specFor(cue.sectionIndex, cue.variant, slot);
const axis = arc._slowAxisFor(spec.module); const axis = arc._slowAxisFor(spec.module);
const start = at(t.duration * 0.05); const start = at(t.duration * 0.05);
@ -556,7 +561,29 @@ function meanAbs(a, b) {
return sum / a.length / 255; return sum / a.length / 255;
} }
/** How much the declared axis moves a scene, against how much it moves anyway. */ /**
* How much the declared axis moves a scene, against the metric's own noise.
*
* The experiment holds TIME constant and varies only the axis, which the first
* version of this check did not — and getting that wrong made it measure
* something else entirely.
*
* That version compared the scene at thirty seconds against the scene at two
* and a half minutes, and divided by what the scene did between those points
* with its parameters frozen. But those are two different places in the song,
* so the denominator was full of audio reactivity: the check was really asking
* "does the slow axis move this scene MORE than the music does", which is a
* different and much harder question, and one no scene should have to pass.
* Circuit Bloom failed it at 0.92x while its mean luminance was moving 5.6x
* across the axis — a large structural change, scored as nothing.
*
* So: one frame window, two parameter sets, and a control that is the same
* parameters over the next window — the residual churn the ten-second average
* failed to cancel, which is exactly the noise floor this measurement has to
* beat. With the axis disabled the two parameter sets are identical and the
* ratio is 0.00 by construction, which is what makes the companion check below
* a real discriminator rather than a formality.
*/
function axisRatio(engine, arc, look, track, module, { withAxis }) { function axisRatio(engine, arc, look, track, module, { withAxis }) {
const paramsAt = (seconds) => { const paramsAt = (seconds) => {
const out = sampleValues(module, new Rng(77), look.sections[0].bias, const out = sampleValues(module, new Rng(77), look.sections[0].bias,
@ -570,18 +597,16 @@ function axisRatio(engine, arc, look, track, module, { withAxis }) {
} }
return out; return out;
}; };
// The control is the same scene with its parameters held: whatever it does
// on its own between these two points in the track.
const frozenA = averagedFrame(engine, look, module,
sampleValues(module, new Rng(77), look.sections[0].bias, look.personality.temperament), 1800);
const frozenB = averagedFrame(engine, look, module,
sampleValues(module, new Rng(77), look.sections[0].bias, look.personality.temperament), 9000);
const movedA = averagedFrame(engine, look, module, paramsAt(30), 1800);
const movedB = averagedFrame(engine, look, module, paramsAt(150), 9000);
const own = meanAbs(frozenA, frozenB); const FRAME = 1800;
const moved = meanAbs(movedA, movedB); const WINDOW = 90 * 7;
return { own, moved, ratio: moved / Math.max(1e-6, own) }; const early = averagedFrame(engine, look, module, paramsAt(0), FRAME);
const late = averagedFrame(engine, look, module, paramsAt(track.duration), FRAME);
const control = averagedFrame(engine, look, module, paramsAt(0), FRAME + WINDOW);
const moved = meanAbs(early, late);
const noise = meanAbs(early, control);
return { moved, noise, ratio: moved / Math.max(1e-6, noise) };
} }
check(11, 'a declared slow axis actually changes the scene', () => { check(11, 'a declared slow axis actually changes the scene', () => {
@ -604,8 +629,8 @@ check(11, 'a declared slow axis actually changes the scene', () => {
const r = axisRatio(engine, arc, look, t, module, { withAxis: true }); const r = axisRatio(engine, arc, look, t, module, { withAxis: true });
detail.push(`${module.name} ${r.ratio.toFixed(2)}x`); detail.push(`${module.name} ${r.ratio.toFixed(2)}x`);
if (r.ratio < 1.25) { if (r.ratio < 1.25) {
problems.push(`${module.name}: axis moved ${r.moved.toFixed(4)} against ` + problems.push(`${module.name}: axis moved ${r.moved.toFixed(4)} against a ` +
`${r.own.toFixed(4)} on its own — only ${r.ratio.toFixed(2)}x`); `${r.noise.toFixed(4)} noise floor — only ${r.ratio.toFixed(2)}x`);
} }
} }
if (!declared.length) problems.push('no scene declares a slow axis'); if (!declared.length) problems.push('no scene declares a slow axis');
@ -701,8 +726,21 @@ check(11, 'framing stays inside the library\'s headroom', () => {
if (f.scale < 0.55 || f.scale > 2.2) { if (f.scale < 0.55 || f.scale > 2.2) {
problems.push(`§${cue.sectionIndex}#${cue.shotIndex}: scale ${f.scale.toFixed(2)}`); problems.push(`§${cue.sectionIndex}#${cue.shotIndex}: scale ${f.scale.toFixed(2)}`);
} }
if (Math.abs(f.shift[0]) > 0.3 || Math.abs(f.shift[1]) > 0.3) { // The gaze is live now — it is not on cue.framing, it is
problems.push(`§${cue.sectionIndex}#${cue.shotIndex}: shift ${f.shift.map((v) => v.toFixed(2))}`); // wherever the camera has travelled to by a given frame —
// so the bound has to be measured at frames rather than
// read off the plan. Its ceiling is the reach the shot's
// own size allows: see Camera.reachFor, which is why a
// close-up is permitted further off centre than a wide.
const ceiling = look.camera
? reachFor(f.scale, look.camera) + 1e-6 : 0.31;
for (const fr of [cue.startFrame, (cue.startFrame + cue.endFrame) >> 1,
cue.endFrame - 1]) {
const shift = arc._framingAt(cue.index, fr).shift;
if (Math.hypot(shift[0], shift[1]) > ceiling) {
problems.push(`§${cue.sectionIndex}#${cue.shotIndex}@${fr}: `
+ `|shift| ${Math.hypot(shift[0], shift[1]).toFixed(3)} > ${ceiling.toFixed(3)}`);
}
} }
} }
} finally { } finally {
@ -731,9 +769,20 @@ check(11, 'framing is identical on a seek and on playback', () => {
for (let i = 0; i < a.cues.length; i++) { for (let i = 0; i < a.cues.length; i++) {
const fa = a.cues[i].framing; const fa = a.cues[i].framing;
const fb = b.cues[i].framing; const fb = b.cues[i].framing;
if (fa && fb && (fa.scale !== fb.scale if (fa && fb && fa.scale !== fb.scale) {
|| fa.shift[0] !== fb.shift[0] || fa.shift[1] !== fb.shift[1])) { problems.push(`cue ${i}: scale ${fa.scale} vs ${fb.scale}`);
problems.push(`cue ${i}: ${JSON.stringify(fa)} vs ${JSON.stringify(fb)}`); }
// The gaze is a position in time, so agreeing on the plan is not
// enough — two drivers have to agree on where the camera IS at a
// frame. This is the property a seek depends on.
const cue = a.cues[i];
for (const fr of [cue.startFrame, (cue.startFrame + cue.endFrame) >> 1,
cue.endFrame - 1]) {
const sa = a._framingAt(i, fr).shift;
const sb = b._framingAt(i, fr).shift;
if (sa[0] !== sb[0] || sa[1] !== sb[1]) {
problems.push(`cue ${i}@${fr}: gaze ${sa} vs ${sb}`);
}
} }
} }
} finally { } finally {
@ -745,6 +794,105 @@ check(11, 'framing is identical on a seek and on playback', () => {
: `${a.cues.length} cues carry the same framing in both drivers`); : `${a.cues.length} cues carry the same framing in both drivers`);
}); });
/**
* Everything the camera exists to do, measured on one sweep of the battery.
*
* Stated as numbers because the device this replaces LOOKED right in the source
* and did nothing on screen: a recentre of a median 0.029 of a half-frame, at a
* fresh random angle every shot so successive offsets cancelled. Nothing in the
* old gates caught that, because the only bound was a ceiling — and a device
* doing nothing passes a ceiling comfortably.
*
* So each of these has a FLOOR. That is the lesson from the old one.
*/
function gazeStats() {
const offsets = []; const jumps = []; const travels = [];
const cameras = new Set(); const curves = new Set();
for (const { track: t } of tempoBattery()) {
for (let s = 0; s < 6; s++) {
const look = generateLook(t, { seed: 21400 + s * 5077 });
if (!look.camera) continue;
cameras.add(look.camera.name);
const arc = new ArcDriver(look, t);
try {
let prevEnd = null;
arc.cues.forEach((cue, i) => {
curves.add(arc.gaze[i].curve);
const start = arc._framingAt(i, cue.startFrame).shift;
const end = arc._framingAt(i, cue.endFrame - 1).shift;
offsets.push(Math.hypot(end[0], end[1]));
travels.push(Math.hypot(end[0] - start[0], end[1] - start[1]));
if (prevEnd) {
jumps.push(Math.hypot(start[0] - prevEnd[0], start[1] - prevEnd[1]));
}
prevEnd = end;
});
} finally {
arc.dispose();
}
}
}
const median = (a) => {
const v = a.slice().sort((x, y) => x - y);
return v.length ? v[v.length >> 1] : 0;
};
return {
offsets, jumps, travels, cameras, curves,
medOffset: median(offsets), medJump: median(jumps), medTravel: median(travels),
};
}
check(11, 'the camera moves far enough at a cut to be seen', () => {
// The floor is the whole point. The device this replaces had a median jump
// of 0.014 of a half-frame — present in every video, visible in none.
//
// Measured over the REFRAMES rather than over every cut. A match cut is a
// deliberate zero and there are enough of them to drag a plain median down;
// averaging the cuts that chose not to move together with the ones that did
// would let the reframes shrink to nothing without the number noticing,
// which is the exact failure this whole gate exists to catch.
const r = gazeStats();
const reframes = r.jumps.filter((j) => j >= 0.01);
const sorted = reframes.slice().sort((a, b) => a - b);
const med = sorted.length ? sorted[sorted.length >> 1] : 0;
return expect(med > 0.09 && reframes.length > 0,
`median reframe ${med.toFixed(3)} of a half-frame across ${reframes.length} reframes ` +
`· median offset ${r.medOffset.toFixed(3)} ` +
`(the old device measured 0.014 jump / 0.029 offset)`);
});
check(11, 'the camera moves DURING a shot, not only at cuts', () => {
// Framing used to be constant within a shot by design. The recentre is not
// framing — a camera that only ever steps is a slideshow of stills.
const r = gazeStats();
const moving = r.travels.filter((t) => t > 0.02).length;
const share = moving / Math.max(1, r.travels.length);
return expect(r.medTravel > 0.02 && share > 0.5,
`${moving}/${r.travels.length} shots travel (${(share * 100).toFixed(0)}%) · ` +
`median travel ${r.medTravel.toFixed(3)}`);
});
check(11, 'the camera cuts through as well as jumping', () => {
// A camera that relocates at EVERY cut is as much a single rule as one that
// never does. The match cut — where the gaze walks through the change and
// two scenes read as one place — has to survive as a real minority.
const r = gazeStats();
const matched = r.jumps.filter((j) => j < 0.01).length;
const share = matched / Math.max(1, r.jumps.length);
return expect(share > 0.04 && share < 0.6,
`${matched}/${r.jumps.length} cuts are match cuts (${(share * 100).toFixed(0)}%)`);
});
check(11, 'the library uses more than one camera and more than one curve', () => {
// Same argument directors.js makes about families: one camera applied to
// every track is how a library ends up with one look.
const r = gazeStats();
return expect(r.cameras.size >= 3 && r.curves.size >= 3,
`${r.cameras.size} cameras (${[...r.cameras].join(', ')}) · ` +
`${r.curves.size}/${CURVE_NAMES.length} curves used`);
});
check(11, 'the axis measurement would notice if the axis stopped working', () => { check(11, 'the axis measurement would notice if the axis stopped working', () => {
// EPIC-2.md §4 names this failure mode by name: a gate that measures the // EPIC-2.md §4 names this failure mode by name: a gate that measures the
// wrong thing. This one has already happened once here — the first version // wrong thing. This one has already happened once here — the first version
@ -782,3 +930,71 @@ check(11, 'the axis measurement would notice if the axis stopped working', () =>
problems.length ? problems.join(' · ') problems.length ? problems.join(' · ')
: `axis disabled reads ${detail.join(', ')} — the measurement tracks the axis`); : `axis disabled reads ${detail.join(', ')} — the measurement tracks the axis`);
}, { slow: true }); }, { slow: true });
check(11, 'framing visibly changes every scene in the library', () => {
// The gate the framing work shipped without, and the reason this revisit
// found anything. The other framing checks assert that the PLAN differs —
// cue sizes, distribution, headroom — and a plan that never reaches the
// image passes all of them. It did: framing was applied inside sigCamera,
// which is gated on the `camera` personality trait, so any scene that
// declined the track's drift silently declined the shot size too. Scan
// Tear, Pylon Grid, Pitch Shatter and the 3D layer rendered identically at
// every size and nothing noticed.
//
// Measured relative to how much image there is, for the reason Phase 10
// documents: most of this library is mostly dark.
const PALETTE = [[1, 0.3, 0.2], [0.2, 0.8, 1], [1, 0.9, 0.3],
[0.6, 0.3, 1], [0.2, 1, 0.6], [1, 1, 1]];
const PERSON = {
shape: { sides: 0, roundness: 0.25, elongation: 1, tilt: 0 },
camera: { driftAngle: 0, driftRate: 0, sway: 0, swayRate: 0.1, spin: 0, breathe: 0 },
space: { horizon: 0.5, depth: 0.5, washAngle: 0, wash: 0.2 },
style: { lineWeight: 0.5, softness: 0.5, texture: 0, symmetry: 1 },
};
const engine = new Engine({ width: 160, height: 90 });
engine.timeline.setDuration(60);
const problems = [];
let weakest = Infinity;
let weakestName = '';
try {
const shot = (module, framing) => {
engine.setLayerSpecs([{
module, params: sampleValues(module, new Rng(31), {}, null),
seed: 9, opacity: 1, blend: 'normal', palette: PALETTE, personality: PERSON,
}]);
// Framing is pushed onto live layers, the way the arc driver does it.
const layers = engine.compositor.layers || [];
layers.forEach((l) => l.setFraming && l.setFraming(framing));
engine.prime(600);
engine.compositor.reset();
layers.forEach((l) => l.setFraming && l.setFraming(framing));
return Uint8Array.from(engine.readPixels(engine.renderFrame(600)));
};
for (const module of scenes) {
const normal = shot(module, { size: 'normal', scale: 1, shift: [0, 0] });
const close = shot(module, { size: 'close', scale: SHOT_SIZES.close.scale, shift: [0.05, 0.02] });
const wide = shot(module, { size: 'wide', scale: SHOT_SIZES.wide.scale, shift: [0.05, 0.02] });
const brightness = Math.max(1e-3,
(frameLuminance(normal) + frameLuminance(close)) * 0.5);
const best = Math.max(frameDistance(normal, close), frameDistance(normal, wide))
/ brightness;
if (best < weakest) { weakest = best; weakestName = module.name; }
if (best < 0.05) {
problems.push(`${module.name}${module.kind === 'layer3d' ? ' [3D]' : ''}: ` +
`${best.toFixed(3)} — renders the same at every shot size`);
}
}
} finally {
engine.dispose();
}
return expect(problems.length === 0,
problems.length ? problems.join(' · ')
: `${scenes.length} scenes all respond to framing · weakest ` +
`${weakestName} at ${weakest.toFixed(2)} of its own brightness`);
}, { slow: true });

437
flow-state/src/checks/phase12.js Normal file
View File

@ -0,0 +1,437 @@
// Phase 12 — the SEED VARIETY TEST.
//
// Every other phase asks whether one video is correct. This one asks whether
// two videos are different, which is the failure the rest of the suite is
// structurally unable to see: a generator that ignores its seed passes
// determinism, flash safety, liveness and end-to-end rendering perfectly.
//
// The measurement lives in checks/variety/. The two checks that come first here
// are not about the generator at all — they are about the instrument. A
// structural variety score is worthless unless it can be shown to ignore the
// cheap axes (recolour, rotate) and to react to the expensive one (a different
// scene), so those are gated before any number derived from them is trusted.
//
// The library sweep in the middle is the map the rest is drawn on: the seed can
// only produce as much variety as the library holds, so "how many structurally
// distinct looks are there" is measured before "how many does a seed reach".
import { check, expect } from './framework.js';
import { song } from '../audio/songbank.js';
import { Show } from '../Show.js';
import { generateLook } from '../look/LookGenerator.js';
import { scenes } from '../scenes/registry.js';
import { canBackground } from '../params/schema.js';
import {
GROUND_MIN, canGround, isMeasured, surfaceOf, METADATA,
coverageOf as declaredCoverage, structuralDistance,
} from '../scenes/surface.js';
import { metadataIsFresh, metricsFingerprint } from './metadata.js';
import { RESTFUL_FAMILIES } from '../look/directors.js';
import { stackCoverage } from '../look/stack.js';
import { frameDescriptor, rotate90, recolour } from './variety/descriptors.js';
import { descriptorDistance, signatureDistance } from './variety/signature.js';
import {
measureVariety, measureSpecDiversity, signatureForScene, librarySweep,
} from './variety/report.js';
// `centre` is the bank's null hypothesis: the middle of every axis, a full
// six-stage arrangement. The gates used to run on a two-section synthetic whose
// only kinds were intro and outro, which left half the scene library unreachable
// and made every number here a measurement of that rather than of the seed.
export const varietyTrack = () => song('centre').track;
// The library sweep is the most expensive thing in the suite — every scene
// rendered — and three checks read it. Computed once per page load.
let cachedSweep = null;
export function librarySweepCached() {
if (!cachedSweep) cachedSweep = librarySweep(varietyTrack(), { probes: 3 });
return cachedSweep;
}
/** One frame of one seed, rendered the normal way. */
function sampleFrame(seed = 12001, width = 160, height = 90) {
const track = varietyTrack();
const show = new Show({ width, height });
try {
show.useTrack(track, generateLook(track, { seed }));
const section = show.look.sections[Math.floor(show.look.sections.length / 2)];
const frame = section.startFrame + Math.floor((section.endFrame - section.startFrame) / 2);
show.engine.compositor.reset();
for (let f = Math.max(0, frame - 20); f < frame; f++) show.renderFrame(f);
return Uint8Array.from(show.readPixels(show.renderFrame(frame)));
} finally {
show.dispose();
}
}
check(12, 'seed variety · the metric ignores colour and rotation', () => {
const w = 160, h = 90;
const pixels = sampleFrame(12001, w, h);
const base = frameDescriptor(pixels, w, h);
// Hue-rotated and brightened: the same picture in a different palette, which
// is precisely the difference we refuse to count as variety.
const graded = descriptorDistance(base, frameDescriptor(recolour(pixels), w, h));
// Turned ninety degrees. Layout is EXCLUDED here by design — where structure
// sits in the frame is real information, and a metric blind to it could not
// see a library that centres everything.
const spun = rotate90(pixels, w, h);
const turned = descriptorDistance(base, frameDescriptor(spun.pixels, spun.width, spun.height));
const structural = ['scale', 'orient', 'texture'];
const worstGrade = Math.max(...structural.concat('layout').map((b) => graded[b]));
const worstTurn = Math.max(...structural.map((b) => turned[b]));
return expect(worstGrade < 0.05 && worstTurn < 0.02 && graded.colour > 0.2,
`recolour moves structure ${worstGrade.toFixed(4)} (colour block ${graded.colour.toFixed(3)}) · ` +
`rotate moves structure ${worstTurn.toFixed(4)}`);
});
check(12, 'seed variety · the metric separates different scenes from the same scene', () => {
// The instrument's other half: it must react to the thing that IS a
// difference. Two renders of one scene must land far below two renders of
// two scenes, or a low variety score would just be a blind metric.
const track = varietyTrack();
const pool = scenes.filter((m) => canBackground(m) && m.kind === 'fragment');
const a = pool[0], b = pool[Math.floor(pool.length / 2)], c = pool[pool.length - 1];
const sigA = signatureForScene(track, a, 777, { probes: 2 });
const sigA2 = signatureForScene(track, a, 777, { probes: 2 });
const sigB = signatureForScene(track, b, 777, { probes: 2 });
const sigC = signatureForScene(track, c, 777, { probes: 2 });
const same = signatureDistance(sigA, sigA2).total;
const diff = Math.min(
signatureDistance(sigA, sigB).total,
signatureDistance(sigA, sigC).total,
signatureDistance(sigB, sigC).total,
);
return expect(same < 0.01 && diff > same * 8,
`same scene ${same.toFixed(4)} · different scenes ${diff.toFixed(4)} ` +
`(${a.name} / ${b.name} / ${c.name})`);
}, { slow: true });
check(12, 'seed variety · every visualization in the library is a distinct look', () => {
// Runs against ALL of them, not a sample. The existing per-scene gate does
// a version of this on raw pixels, where two scenes that are the same image
// in different colours pass comfortably; here colour is not counted, so a
// structural twin is visible as one.
//
// Twins are reported rather than failed on a fixed distance: what matters is
// that the library does not contain a CLUSTER of scenes that are one look
// wearing several names, because the casting code will happily "vary" a
// video by rotating between them.
const sweep = librarySweepCached();
const bigTwins = sweep.twins.filter((g) => g.length >= 3);
return expect(bigTwins.length === 0,
(bigTwins.length ? `structural twin groups: ${bigTwins.map((g) => g.join('≈')).join(' · ')} — ` : '') +
`${sweep.scenes.length} scenes · median distance ${sweep.median.toFixed(3)} · ` +
`closest ${sweep.closestPairs[0].name}≈${sweep.closestPairs[0].nearest} ` +
`at ${sweep.closestPairs[0].distance.toFixed(3)}`);
}, { slow: true });
check(12, 'seed variety · the generator casts a different show for a different seed', () => {
// Cheap, GPU-free, and the first thing to read when the rendered score
// drops: this says whether the generator ever DECIDED to make two different
// videos, before asking whether the pixels came out different.
const d = measureSpecDiversity(varietyTrack(), { seeds: 24 });
const problems = [];
if (d.sceneSetDistance < 0.6) problems.push(`scene sets only ${d.sceneSetDistance.toFixed(2)} apart`);
if (d.identicalCasts > 0) problems.push(`${d.identicalCasts} seed pairs cast identically`);
if (d.libraryCoverage < 0.6) {
problems.push(`only ${(d.libraryCoverage * 100).toFixed(0)}% of the library used ` +
`(never cast: ${d.uncast.slice(0, 6).join(', ')})`);
}
if (d.director.unique < 2) problems.push('one director for every seed');
return expect(problems.length === 0,
problems.length ? problems.join(' · ')
: `cast distance ${d.sceneSetDistance.toFixed(2)} · ` +
`library ${(d.libraryCoverage * 100).toFixed(0)}% · ` +
`${d.director.unique} directors · ${d.paletteScheme.unique} schemes · ` +
`${d.signature.unique} signatures · ${d.grain.unique} grain modes`);
});
check(12, 'seed variety · different seeds render structurally different videos', () => {
// THE gate. The threshold is a target the generator does not currently meet
// — a failure here is the known open defect, not a flaky check. The detail
// line carries the full breakdown so a run can be compared against the last
// one while the number is being moved.
const r = measureVariety(varietyTrack(), { seeds: 6, refScenes: 4, probes: 4 });
const problems = [];
// Against the floor: two seeds must differ by more than one seed differs
// from itself across its own sections. Below this the seed is decoration.
if (r.separation < 0.35) {
problems.push(`separation ${r.separation.toFixed(2)} (floor ${r.floor.toFixed(3)}, ` +
`observed ${r.observed.toFixed(3)}, ceiling ${r.ceiling.toFixed(3)})`);
}
// No seed may be shadowed by another. A healthy mean hides pairs that are
// the same video, and a viewer only ever sees the pair.
const shadowed = r.nearest.filter((d) => d < r.floor * 0.75).length;
if (shadowed) problems.push(`${shadowed} seeds shadowed by another seed`);
// Per block: it is not enough for the total to pass on colour-adjacent
// motion while every frame is composed the same way.
for (const [name, b] of Object.entries(r.byBlock)) {
if (name === 'colour') continue;
if (b.ratio < 0.25) problems.push(`${name} at ${(b.ratio * 100).toFixed(0)}% of achievable`);
}
const blocks = Object.entries(r.byBlock)
.map(([n, b]) => `${n} ${(b.ratio * 100).toFixed(0)}%`).join(' · ');
return expect(problems.length === 0,
(problems.length ? problems.join(' · ') + ' — ' : '') +
`separation ${r.separation.toFixed(2)} · ${blocks}`);
}, { slow: true });
/**
* The metadata is measured from the code that is actually here.
*
* scenes/metadata.json is a cache of a render — how much frame each scene
* paints, how much it changes between songs, what it looks like structurally —
* and the generator composes with it: which scenes may ground a section, and
* which pairs are unalike enough to be worth stacking. A stale file therefore
* does not produce a stale REPORT, it produces wrong videos, silently.
*
* So the file carries a fingerprint of everything that can move a number in it
* — the scenes, the shader contract, the identities and palettes they are
* handed, and the metric definitions — and this fails when it stops matching.
* The fix is not code: open gallery.html and press "refresh metadata".
*/
check(12, 'metadata · the measurements match the code that produced them', () => {
const fresh = metadataIsFresh();
const rows = Object.keys(METADATA.scenes || {}).length;
const unmeasured = scenes
.filter((m) => m.kind === 'fragment' && !isMeasured(m))
.map((m) => m.name);
return expect(fresh && unmeasured.length === 0,
(fresh ? '' : `stale: measured against ${METADATA.fingerprint}, code is ${metricsFingerprint()} — ` +
'rebuild it from gallery.html → refresh metadata · ') +
(unmeasured.length ? `never measured: ${unmeasured.slice(0, 5).join(', ')} · ` : '') +
`${rows} scenes measured ${METADATA.measured} · ` +
`${scenes.filter(canGround).length} can ground a section`);
});
/**
* The labels the measurements produce are usable ones.
*
* Not a re-measurement — the check above covers staleness. This asks whether
* the library, as labelled, can still cast a video: every section needs a
* ground, quiet sections are held to the restful families, so a threshold that
* left one ground in the whole library would pass every other gate here and
* make every video identical underneath.
*/
check(12, 'metadata · the measured labels leave enough grounds to cast with', () => {
const grounds = scenes.filter(canGround);
const byFamily = {};
for (const m of grounds) byFamily[m.family] = (byFamily[m.family] || 0) + 1;
const restful = RESTFUL_FAMILIES.reduce((n, f) => n + (byFamily[f] || 0), 0);
const problems = [];
// Low bars, and they are where the library actually is rather than where it
// ought to be. Seven scenes can ground a section — five geometric, two
// organic — and the two organic ones carry every quiet section of every
// video, because no minimal or flow canvas in the library fills half the
// frame without reading prev(). That is the library's largest hole and it
// is scene work, not generator work. The numbers are printed on every run
// so it stays visible instead of becoming the way things are.
if (grounds.length < 6) problems.push(`only ${grounds.length} grounds in the library`);
if (restful < 2) problems.push(`only ${restful} restful grounds for quiet sections`);
return expect(problems.length === 0,
(problems.length ? problems.join(' · ') + ' — ' : '') +
`${grounds.length} grounds · ` +
Object.entries(byFamily).map(([f, n]) => `${f} ${n}`).join(' · '));
});
/**
* COMPOSITION — what every section's frame is made of.
*
* Two rules, and they are the two ends of the same idea: a section always has a
* filled picture underneath it, and it never stacks up more than two frames'
* worth of material on top of that. Between them they rule out both failures
* the layer stack can produce — a shot floating on black, and four passes of
* texture over each other.
*
* Measured across the song bank rather than one track, because both rules are
* decided per director, per kind and per point in the story, and a single song
* exercises one director's opinion about six sections.
*/
check(12, 'composition · every section stands on a canvas, under a budget', () => {
const problems = [];
const budgets = [];
let stacks = 0;
let grounded = 0;
for (const name of ['centre', 'drone', 'lattice']) {
const track = song(name).track;
for (let s = 0; s < 4; s++) {
const look = generateLook(track, { seed: 4000 + s * 7717 });
for (const section of look.sections) {
for (const stack of section.variants || [section.layers]) {
stacks++;
const base = stack[0];
const cover = declaredCoverage(base.module);
if (surfaceOf(base.module) !== 'canvas' || cover < GROUND_MIN) {
problems.push(`${section.kind} stands on ${base.module.name} at ${(cover * 100).toFixed(0)}%`);
} else {
grounded++;
}
if (base.opacity < 1 || base.blend !== 'normal') {
problems.push(`${section.kind} ground ${base.module.name} is ${base.blend} at ${base.opacity.toFixed(2)}`);
}
const total = stackCoverage(stack);
budgets.push(total);
if (total > 2.0001) {
problems.push(`${section.kind} paints ${(total * 100).toFixed(0)}% — over budget`);
}
}
}
}
}
const mean = budgets.reduce((a, b) => a + b, 0) / Math.max(1, budgets.length);
const max = Math.max(...budgets);
return expect(problems.length === 0,
(problems.length ? problems.slice(0, 4).join(' · ') + ' — ' : '') +
`${grounded}/${stacks} grounded · coverage mean ${(mean * 100).toFixed(0)}% ` +
`· peak ${(max * 100).toFixed(0)}% of the 200% ceiling`);
});
/**
* The rendered frame, at BOTH ends.
*
* The spec-level check above says the generator intended a filled frame. This
* renders the middle of every section of twelve videos and looks at what came
* out, which is the only statement that matters to someone watching.
*
* Two failures, and they are opposites that arrive by the same route:
*
* too black a section that is a few bright things on nothing. Measured at
* 9 of 40 sampled frames under 20% painted before grounds, the
* darkest at 0.3%.
* too white a section clipped to paper. Measured at a median 24% of pixels
* at full white and entire sections at 100% when the shot was
* screened over its ground and the feedback loop was still an
* accumulator. Clipped white is not brightness, it is missing
* information: every difference inside it has been deleted.
*
* `ink` is counted at a low threshold on purpose — the question is whether the
* frame has anything IN it, not whether it is bright, and a legitimately dark
* scene is not a failure. `blown` is counted at near-full white on all three
* channels, which no grade should produce over a quarter of a frame.
*/
check(12, 'composition · a rendered section is neither black nor blown out', () => {
const dark = [];
const white = [];
const ink = [];
const blown = [];
const lum = [];
for (const name of ['centre', 'drone', 'lattice', 'glare', 'murk', 'runner']) {
const track = song(name).track;
for (const seed of [4001, 9931]) {
const show = new Show({ width: 160, height: 90 });
try {
show.useTrack(track, generateLook(track, { seed }));
for (const section of show.look.sections) {
const f = Math.round(section.startFrame
+ (section.endFrame - section.startFrame) * 0.5);
show.engine.compositor.reset();
for (let k = Math.max(0, f - 12); k < f; k++) show.renderFrame(k);
const px = Uint8Array.from(show.readPixels(show.renderFrame(f)));
let inked = 0, clipped = 0, light = 0;
const n = px.length / 4;
for (let i = 0; i < px.length; i += 4) {
const r = px[i], g = px[i + 1], b = px[i + 2];
const l = 0.2126 * r + 0.7152 * g + 0.0722 * b;
if (l > 13) inked++;
if (r > 238 && g > 238 && b > 238) clipped++;
light += l;
}
const where = `${name}/${seed.toString(16)} ${section.kind}`;
ink.push(inked / n);
blown.push(clipped / n);
lum.push(light / n / 255);
if (inked / n < 0.3) dark.push(`${where} only ${((inked / n) * 100).toFixed(0)}% painted`);
// Per section: the hard cap only. A deliberate blaze is
// allowed to be bright; nothing is allowed to be gone.
if (clipped / n > 0.5) white.push(`${where} ${((clipped / n) * 100).toFixed(0)}% clipped white`);
if (light / n / 255 > 0.9) white.push(`${where} mean luminance ${((light / n / 255) * 100).toFixed(0)}%`);
}
} finally {
show.dispose();
}
}
}
// A BLAZE is allowed; blazing by default is not.
//
// The ceiling is therefore two numbers rather than one. No single section
// may be wholly gone — past about half the frame at full white there is no
// picture left to read — and only a minority of them may be hot at all. The
// second is the one that matters: screening every shot over its ground blew
// a median quarter of EVERY frame, which is not a director choosing to peak,
// it is a pipeline with no headroom. See blazeOf in look/directors.js.
const hot = blown.filter((b) => b > 0.25).length;
const hotShare = hot / Math.max(1, blown.length);
if (hotShare > 0.2) {
white.push(`${(hotShare * 100).toFixed(0)}% of sections clipped past 25% — blazing is the default, not a choice`);
}
const problems = [...dark, ...white];
const mean = (a) => a.reduce((x, y) => x + y, 0) / Math.max(1, a.length);
return expect(problems.length === 0,
(problems.length ? problems.slice(0, 4).join(' · ') + ' — ' : '') +
`${ink.length} sections · painted mean ${(mean(ink) * 100).toFixed(0)}% ` +
`darkest ${(Math.min(...ink) * 100).toFixed(0)}% · ` +
`clipped mean ${(mean(blown) * 100).toFixed(0)}% worst ${(Math.max(...blown) * 100).toFixed(0)}% · ` +
`${hot} of ${blown.length} sections blazing`);
}, { slow: true });
/**
* A stack is two things happening, not one thing twice.
*
* The point of the measured profiles: family labels say a `flow` scene and an
* `organic` scene are different, and the render can disagree — two of them can
* sit 0.04 apart, which stacked is one texture at double density. This asks
* whether the generator's stacks are actually made of unalike material.
*
* A mean rather than a per-stack floor, because a lean is what the generator
* applies. Demanding every pair clear a bar would be demanding a particular
* draw, and the seed is supposed to be able to make an ordinary choice.
*/
check(12, 'composition · stacked layers are structurally unalike', () => {
const pairs = [];
for (const name of ['centre', 'drone', 'lattice']) {
const track = song(name).track;
for (let s = 0; s < 4; s++) {
const look = generateLook(track, { seed: 5200 + s * 3931 });
for (const section of look.sections) {
for (const stack of section.variants || [section.layers]) {
for (let i = 0; i < stack.length; i++) {
for (let j = i + 1; j < stack.length; j++) {
const d = structuralDistance(stack[i].module, stack[j].module);
if (d !== null) pairs.push(d);
}
}
}
}
}
}
const mean = pairs.reduce((a, b) => a + b, 0) / Math.max(1, pairs.length);
const twins = pairs.filter((d) => d < 0.03).length;
// Against the library's own median distance, so this measures the CHOOSING
// rather than the library — a bar in absolute units would drift every time
// a scene was added.
return expect(pairs.length > 0 && mean > 0.08 && twins / pairs.length < 0.05,
`${pairs.length} stacked pairs · mean distance ${mean.toFixed(3)} · ` +
`${twins} near-twins (${((twins / Math.max(1, pairs.length)) * 100).toFixed(0)}%)`);
});

247
flow-state/src/checks/phase13.js Normal file
View File

@ -0,0 +1,247 @@
// Phase 13 gate — the story.
//
// Phase 8 gave a track cuts, Phase 9 an identity, Phase 10 a hand on the dials.
// All three are properties of a video at a MOMENT. This phase is about the one
// property that only exists over its whole length: that it is going somewhere,
// and that where it is going follows the song rather than the seed.
//
// Everything here runs on the analysis and the look, without a GPU. The
// rendered half of the question — does the arc reach the IMAGE — is the
// `direction` statistic in the variety report (checks/variety/signature.js),
// because answering it requires probing real frames.
//
// The three things this gate is actually protecting, in order of how bad the
// failure would be:
//
// 1. the story never overrides the song. A plot that declares a climax where
// the track is quiet is worse than no plot, and a tension that lifts a
// breakdown into a drop breaks the quiet-kind coupling every earlier phase
// depends on.
// 2. the same kind twice is not the same twice. This is the whole feature.
// 3. it stays pure and continuous in the frame, or the export stops matching
// the preview and the boundary pops.
import { check, expect } from './framework.js';
import { generateLook } from '../look/LookGenerator.js';
import { FeatureTrack } from '../audio/FeatureTrack.js';
import { synthesizeSectioned } from '../audio/synth.js';
import { song } from '../audio/songbank.js';
import { storyStateAt, STORY_VARS, PLOT_NAMES } from '../look/Story.js';
import { subjectOf } from '../look/stack.js';
/**
* Bank songs rather than the two-section synthetics the other phases use.
*
* A story is a property of a SEQUENCE of sections, so a track that segments into
* two has nothing for this gate to look at — measured, the synthetic battery
* states no kind twice in any of its four tracks, which is precisely the case
* the central check here exists to measure. Four minutes each, across the bank's
* range, so every track has a real arrangement under it.
*/
const NAMES = ['ember', 'centre', 'elegy', 'lattice'];
let cachedBattery = null;
function battery() {
if (!cachedBattery) {
cachedBattery = NAMES.map((name) => ({ name, track: song(name, { duration: 220 }).track }));
}
return cachedBattery;
}
const looks = (seed0) => battery().map(({ track, name }, i) => ({
name, track, look: generateLook(track, { seed: (seed0 + i * 7919) >>> 0 }),
}));
check(13, 'the climax is where the song is loudest', () => {
// The one non-negotiable. Every moment in a story is READ off the section
// energies rather than placed by the seed, so this is a check that the
// derivation still says what it claims and has not drifted into being
// decorative — a story that puts its peak somewhere the audio does not is
// the failure mode that would make the whole layer worth deleting.
const bad = [];
for (const { name, track, look } of looks(0x5709)) {
const energies = track.sections.map((s) => s.energy || 0);
const peak = Math.max(...energies);
const at = look.story.moments.climax;
if (energies[at] < peak - 1e-9) bad.push(`${name}: climax §${at} at ${energies[at].toFixed(3)} vs peak ${peak.toFixed(3)}`);
}
return expect(bad.length === 0,
bad.length ? bad.join(' · ') : `${battery().length} tracks · climax on the loudest section in each`);
});
check(13, 'the story does not lift a quiet section into a loud one', () => {
// Tension moves the parameter bias, and it is bounded so that it can only
// decide WHICH drop this is, never whether a breakdown is one. If this ever
// fails, the family coupling in directors.js is next: an intro at drop
// energy is exactly the strobing opening that coupling exists to prevent.
const QUIET = new Set(['intro', 'breakdown', 'outro']);
let worstQuiet = 0;
let loudest = 0;
let checked = 0;
for (const { look } of looks(0x1a71)) {
for (const section of look.sections) {
checked++;
if (QUIET.has(section.kind)) worstQuiet = Math.max(worstQuiet, section.bias.energy);
if (section.kind === 'drop') loudest = Math.max(loudest, section.bias.energy);
}
}
return expect(worstQuiet < 0.62,
`${checked} sections · loudest quiet-kind bias ${worstQuiet.toFixed(3)} ` +
`(limit 0.62) · loudest drop bias ${loudest.toFixed(3)}`);
});
check(13, 'the second time a kind happens is not the first time again', () => {
// The feature, as a number. Take every track that states a kind twice and
// require the two occurrences to differ in something a viewer could name:
// which scene opens them, or how hard their parameters are pushed.
//
// Both halves count, because either alone is achievable and neither alone
// is the point — a different scene at the same intensity is a shuffle, and
// the same scene at a different intensity is a fade.
const seen = [];
for (const { name, look } of looks(0x2b0c)) {
const byKind = new Map();
for (const section of look.sections) {
if (!byKind.has(section.kind)) byKind.set(section.kind, []);
byKind.get(section.kind).push(section);
}
for (const [kind, list] of byKind) {
if (list.length < 2) continue;
const first = list[0];
const last = list[list.length - 1];
const sceneChanged = subjectOf(first.layers).module.name !== subjectOf(last.layers).module.name;
const pushed = Math.abs(last.story.tension - first.story.tension);
seen.push({ name, kind, sceneChanged, pushed });
}
}
if (!seen.length) {
return expect(false, 'no track in the battery states a kind twice — the check measured nothing');
}
const moved = seen.filter((s) => s.sceneChanged || s.pushed > 0.08);
return expect(moved.length === seen.length,
`${seen.length} repeated kinds · ${moved.length} differ · ` +
seen.map((s) => `${s.name}/${s.kind}: ${s.sceneChanged ? 'recast' : 'same scene'} ` +
`Δtension ${s.pushed.toFixed(2)}`).join(' · '));
});
check(13, 'the journey travels, and travels in stages', () => {
// Two failures at once. A journey that does not move is the old blind slow
// axis with extra machinery; a journey that slides continuously is a slow
// zoom, which is an effect rather than a narrative. So it has to cover most
// of its range AND spend most of the video not moving at all.
const rows = [];
for (const { name, track, look } of looks(0x3f11)) {
let lo = 1, hi = 0, moving = 0, frames = 0, maxStep = 0;
let previous = null;
for (let f = 0; f < track.frameCount; f += 5) {
const j = storyStateAt(look.story, f).journey;
lo = Math.min(lo, j); hi = Math.max(hi, j);
if (previous !== null) {
const step = Math.abs(j - previous);
maxStep = Math.max(maxStep, step);
if (step > 1e-4) moving++;
}
previous = j;
frames++;
}
rows.push({ name, travel: hi - lo, held: 1 - moving / Math.max(1, frames - 1), maxStep });
}
const worstTravel = Math.min(...rows.map((r) => r.travel));
const worstHeld = Math.min(...rows.map((r) => r.held));
const worstStep = Math.max(...rows.map((r) => r.maxStep));
return expect(worstTravel > 0.5 && worstHeld > 0.5 && worstStep < 0.1,
rows.map((r) => `${r.name}: travel ${r.travel.toFixed(2)} held ${(r.held * 100).toFixed(0)}% ` +
`worst step ${r.maxStep.toFixed(3)}`).join(' · '));
});
check(13, 'the story is a pure function of the frame', () => {
// Same rule as everything else on the render path: a seek must land on the
// story position playback would have reached, or the export stops matching
// the preview. Sampled forwards, then backwards, then at random.
const { track, look } = looks(0x4c33)[1];
const forward = [];
for (let f = 0; f < track.frameCount; f += 37) forward.push(storyStateAt(look.story, f));
let mismatch = null;
for (let i = forward.length - 1; i >= 0 && !mismatch; i--) {
const again = storyStateAt(look.story, i * 37);
for (const key of [...STORY_VARS, 'journey']) {
if (again[key] !== forward[i][key]) {
mismatch = `frame ${i * 37} ${key}: ${again[key]} vs ${forward[i][key]}`;
break;
}
}
}
return expect(!mismatch, mismatch || `${forward.length} frames sampled in both directions, identical`);
});
check(13, 'two tracks do not tell the same story', () => {
// A story layer that gave every video the same arc would be the exact
// failure directors.js was written to fix, one level up. Population check,
// like every variety measurement here: what matters is the spread, not any
// single track's plot.
const plots = new Map();
let total = 0;
for (let s = 0; s < 6; s++) {
for (const { look } of looks(0x7000 + s * 104729)) {
plots.set(look.story.plot, (plots.get(look.story.plot) || 0) + 1);
total++;
}
}
const reached = plots.size;
const commonest = Math.max(...plots.values()) / total;
return expect(reached >= 3 && commonest < 0.6,
`${total} looks · ${reached}/${PLOT_NAMES.length} plots reached · ` +
`commonest ${(commonest * 100).toFixed(0)}% · ` +
[...plots].map(([k, v]) => `${k} ${v}`).join(', '));
});
check(13, 'a recapitulation actually recapitulates', () => {
// When the story says the outro answers the intro, it has to be the same
// scene — and the sections still have to be told apart by everything else,
// or a "return" is just the video repeating itself.
const rows = [];
for (let s = 0; s < 8; s++) {
for (const { look } of looks(0x9100 + s * 15485863)) {
const intro = look.sections.find((x) => x.kind === 'intro');
const outro = look.sections.find((x) => x.kind === 'outro');
if (!look.story.recap || !intro || !outro) continue;
// Not journey alone. The plot most likely to ask for a recap is
// 'return', whose whole shape is an arch that comes BACK — so its
// outro sits near the intro on the journey by design, and it is the
// residue on everything else that makes the ending a return rather
// than a rewind.
rows.push({
same: subjectOf(intro.layers).module.name === subjectOf(outro.layers).module.name,
moved: Math.max(...['journey', 'reveal', 'tension'].map((k) =>
Math.abs(outro.story[k] - intro.story[k]))),
});
}
}
if (!rows.length) return expect(true, 'no recap track in this sample — nothing to check');
const kept = rows.filter((r) => r.same).length;
// 0.15 rather than something ambitious, and measured rather than guessed:
// across the sample the median recap arrives a full 1.0 from where it
// started and the closest — a 'return' on a four-section track, where the
// arch has the least room — lands at 0.18. What this rules out is an outro
// that is bit-identical to the intro, which is the failure worth gating.
const travelled = rows.filter((r) => r.moved > 0.15).length;
return expect(kept === rows.length && travelled === rows.length,
`${rows.length} recap tracks · ${kept} re-cast the opening scene · ` +
`${travelled} arrive at it changed · ` +
`closest ${Math.min(...rows.map((r) => r.moved)).toFixed(2)}`);
});
check(13, 'a track with no structure gets no story', () => {
// The degradation path. Two sections is not enough to carry an arc, and
// forcing one produces a video that lurches rather than one that
// progresses — so the whole layer fades toward neutral instead.
const track = FeatureTrack.fromAudioBuffer(
synthesizeSectioned({ bpm: 120, duration: 40, changeAt: 20 }), { fps: 60 });
const look = generateLook(track, { seed: 0xd15a });
const n = look.sections.length;
const worst = Math.max(...look.sections.map((s) =>
Math.max(...STORY_VARS.map((k) => Math.abs(s.story[k] - 0.5)))));
return expect(n > 3 || worst < 0.35,
`${n} sections · strength ${look.story.strength.toFixed(2)} · ` +
`furthest any variable travels from neutral ${worst.toFixed(2)}`);
});

21
flow-state/src/checks/phase2.js
View File

@ -9,11 +9,13 @@
import { check, expect } from './framework.js'; import { check, expect } from './framework.js';
import { Engine } from '../engine/Engine.js'; import { Engine } from '../engine/Engine.js';
import { scenes, FAMILIES } from '../scenes/registry.js'; import { scenes, FAMILIES } from '../scenes/registry.js';
import { defaultValues, sweepValues, validateModule } from '../params/schema.js'; import { defaultValues, sweepValues, validateModule, canBackground } from '../params/schema.js';
import { serializeParams, deserializeParams } from '../params/serialize.js'; import { serializeParams, deserializeParams } from '../params/serialize.js';
import { ParamPanel } from '../ui/ParamPanel.js'; import { ParamPanel } from '../ui/ParamPanel.js';
import { frameLuminance, frameVariance } from '../engine/hash.js'; import { frameLuminance, frameVariance } from '../engine/hash.js';
import { AUDIO_UNIFORMS, SIGNATURE_UNIFORMS } from '../engine/shader-contract.js'; import {
AUDIO_UNIFORMS, SIGNATURE_UNIFORMS, IDENTITY_UNIFORMS,
} from '../engine/shader-contract.js';
import { featureProviderFor } from '../audio/FeatureTrack.js'; import { featureProviderFor } from '../audio/FeatureTrack.js';
import { testTrack } from './phase1.js'; import { testTrack } from './phase1.js';
@ -52,6 +54,7 @@ check(2, 'declared uniforms and shader sources agree both ways', () => {
'u_seed', 'u_opacity', 'u_colors', 'u_colorCount', 'u_prev', 'u_hasPrev', 'u_seed', 'u_opacity', 'u_colors', 'u_colorCount', 'u_prev', 'u_hasPrev',
...AUDIO_UNIFORMS, ...AUDIO_UNIFORMS,
...Object.keys(SIGNATURE_UNIFORMS), ...Object.keys(SIGNATURE_UNIFORMS),
...Object.keys(IDENTITY_UNIFORMS),
]); ]);
const problems = []; const problems = [];
for (const module of scenes) { for (const module of scenes) {
@ -84,9 +87,11 @@ check(2, 'every scene compiles and renders', () => {
const pixels = engine.readPixels(engine.renderFrame(1200)); const pixels = engine.readPixels(engine.renderFrame(1200));
const lum = frameLuminance(pixels); const lum = frameLuminance(pixels);
const variance = frameVariance(pixels); const variance = frameVariance(pixels);
// Accent scenes composite over a background; most of their frame is // A scene that cannot stand alone composites over a background;
// legitimately black, so only variance is meaningful for them. // most of its frame is legitimately black, so only variance is
if (module.role !== 'accent' && !(lum > 0.001)) problems.push(`${module.name}: black frame`); // meaningful for it. Note this is NOT every composable scene — a
// sparse scene that can still carry a section owes us a picture.
if (canBackground(module) && !(lum > 0.001)) problems.push(`${module.name}: black frame`);
if (variance < 0.002) problems.push(`${module.name}: flat (var ${variance.toFixed(4)})`); if (variance < 0.002) problems.push(`${module.name}: flat (var ${variance.toFixed(4)})`);
} catch (err) { } catch (err) {
problems.push(`${module.name}: ${err.message}`); problems.push(`${module.name}: ${err.message}`);
@ -116,9 +121,9 @@ check(2, 'param range sweep produces no dead or blown frames', () => {
const lum = frameLuminance(pixels); const lum = frameLuminance(pixels);
const variance = frameVariance(pixels); const variance = frameVariance(pixels);
const label = `${module.name}.${name}=${JSON.stringify(value)}`; const label = `${module.name}.${name}=${JSON.stringify(value)}`;
const accent = module.role === 'accent'; const accent = !canBackground(module);
// An accent at brightness 0 really is black, and that is a // An overlay-only scene at brightness 0 really is black, and
// legitimate value — judge those on variance alone. // that is a legitimate value — judge those on variance alone.
if (lum > 0.985) problems.push(`${label} blown (lum ${lum.toFixed(3)})`); if (lum > 0.985) problems.push(`${label} blown (lum ${lum.toFixed(3)})`);
if (!accent && lum < 0.0008) problems.push(`${label} black (lum ${lum.toFixed(5)})`); if (!accent && lum < 0.0008) problems.push(`${label} black (lum ${lum.toFixed(5)})`);
if (!accent && variance < 0.0015) problems.push(`${label} flat (var ${variance.toFixed(4)})`); if (!accent && variance < 0.0015) problems.push(`${label} flat (var ${variance.toFixed(4)})`);

41
flow-state/src/checks/phase3.js
View File

@ -14,6 +14,7 @@ import { generateLook, rerollSection, rerollLook, describeLook } from '../look/L
import { paletteContrast, relativeLuminance } from '../look/palette.js'; import { paletteContrast, relativeLuminance } from '../look/palette.js';
import { frameDistance, frameLuminance, frameVariance } from '../engine/hash.js'; import { frameDistance, frameLuminance, frameVariance } from '../engine/hash.js';
import { testTrack } from './phase1.js'; import { testTrack } from './phase1.js';
import { subjectOf } from '../look/stack.js';
/** Four deliberately different tracks: the differentiation gate needs real spread. */ /** Four deliberately different tracks: the differentiation gate needs real spread. */
let cachedBattery = null; let cachedBattery = null;
@ -34,15 +35,22 @@ export function battery() {
export function renderLookFrame(engine, track, look, frame) { export function renderLookFrame(engine, track, look, frame) {
const section = look.sections[track.sectionIndexAt(frame)] || look.sections[0]; const section = look.sections[track.sectionIndexAt(frame)] || look.sections[0];
const layer = section.layers[0]; // The WHOLE stack, not the subject alone.
engine.setLayerSpecs([{ //
// This rendered `layers[0]` back when that was the section's only layer,
// and rendering one layer of a composed section answers a question nobody
// asked: a composable shot on its own is a few bright things on black,
// which is exactly what it is not supposed to be shown as. Measured, it
// reported a dead frame — luminance 0.0006 — for a section that renders
// perfectly well with the ground it was built with underneath it.
engine.setLayerSpecs(section.layers.map((layer) => ({
module: layer.module, module: layer.module,
params: layer.params, params: layer.params,
seed: layer.seed, seed: layer.seed,
opacity: layer.opacity, opacity: layer.opacity,
blend: layer.blend, blend: layer.blend,
palette: look.palette, palette: look.palette,
}]); })));
engine.compositor.setPost(look.post).setFeedback(look.feedback); engine.compositor.setPost(look.post).setFeedback(look.feedback);
engine.compositor.reset(); engine.compositor.reset();
return Uint8Array.from(engine.readPixels(engine.renderFrame(frame))); return Uint8Array.from(engine.readPixels(engine.renderFrame(frame)));
@ -55,9 +63,10 @@ check(3, 'the same audio always produces the same look', () => {
const b = generateLook(track, { samples }); const b = generateLook(track, { samples });
if (a.seed !== b.seed) return expect(false, `seeds differ: ${a.seed} vs ${b.seed}`); if (a.seed !== b.seed) return expect(false, `seeds differ: ${a.seed} vs ${b.seed}`);
const sameScenes = a.sections.every((s, i) => s.layers[0].module.name === b.sections[i].layers[0].module.name); const sameScenes = a.sections.every((s, i) =>
const sameParams = JSON.stringify(a.sections.map((s) => s.layers[0].params)) subjectOf(s.layers).module.name === subjectOf(b.sections[i].layers).module.name);
=== JSON.stringify(b.sections.map((s) => s.layers[0].params)); const sameParams = JSON.stringify(a.sections.map((s) => subjectOf(s.layers).params))
=== JSON.stringify(b.sections.map((s) => subjectOf(s.layers).params));
const samePalette = JSON.stringify(a.palette) === JSON.stringify(b.palette); const samePalette = JSON.stringify(a.palette) === JSON.stringify(b.palette);
return expect(sameScenes && sameParams && samePalette, return expect(sameScenes && sameParams && samePalette,
@ -144,7 +153,7 @@ check(3, 'different tracks get different looks at the same seed', () => {
const palettes = looks.map(({ look }) => look.palette.map(relativeLuminance).join(',')); const palettes = looks.map(({ look }) => look.palette.map(relativeLuminance).join(','));
const uniquePalettes = new Set(palettes).size; const uniquePalettes = new Set(palettes).size;
const sceneSets = looks.map(({ look }) => const sceneSets = looks.map(({ look }) =>
[...new Set(look.sections.map((s) => s.layers[0].module.name))].sort().join('+')); [...new Set(look.sections.map((s) => subjectOf(s.layers).module.name))].sort().join('+'));
return expect(uniquePalettes === looks.length, return expect(uniquePalettes === looks.length,
`${uniquePalettes}/${looks.length} distinct palettes at a fixed seed · ` + `${uniquePalettes}/${looks.length} distinct palettes at a fixed seed · ` +
@ -173,7 +182,7 @@ check(3, 'every generated look renders a live frame on every track', () => {
const lum = frameLuminance(pixels); const lum = frameLuminance(pixels);
const variance = frameVariance(pixels); const variance = frameVariance(pixels);
if (lum < 0.0008 || lum > 0.99 || variance < 0.0015) { if (lum < 0.0008 || lum > 0.99 || variance < 0.0015) {
problems.push(`${name} s${s} ${section.kind}/${section.layers[0].module.name}: ` + problems.push(`${name} s${s} ${section.kind}/${subjectOf(section.layers).module.name}: ` +
`lum ${lum.toFixed(4)} var ${variance.toFixed(4)}`); `lum ${lum.toFixed(4)} var ${variance.toFixed(4)}`);
} }
} }
@ -194,7 +203,7 @@ check(3, 'sections of the same kind share a scene', () => {
const byKind = new Map(); const byKind = new Map();
let violations = 0; let violations = 0;
for (const s of look.sections) { for (const s of look.sections) {
const name = s.layers[0].module.name; const name = subjectOf(s.layers).module.name;
if (byKind.has(s.kind) && byKind.get(s.kind) !== name) violations++; if (byKind.has(s.kind) && byKind.get(s.kind) !== name) violations++;
byKind.set(s.kind, name); byKind.set(s.kind, name);
} }
@ -205,14 +214,14 @@ check(3, 'sections of the same kind share a scene', () => {
check(3, 'reroll changes a section and respects locks', () => { check(3, 'reroll changes a section and respects locks', () => {
const track = testTrack(); const track = testTrack();
const look = generateLook(track, { seed: 555 }); const look = generateLook(track, { seed: 555 });
const before = JSON.stringify(look.sections[0].layers[0].params); const before = JSON.stringify(subjectOf(look.sections[0].layers).params);
rerollSection(look, track, 0, 1); rerollSection(look, track, 0, 1);
const afterUnlocked = JSON.stringify(look.sections[0].layers[0].params); const afterUnlocked = JSON.stringify(subjectOf(look.sections[0].layers).params);
look.sections[0].locked = true; look.sections[0].locked = true;
rerollSection(look, track, 0, 2); rerollSection(look, track, 0, 2);
const afterLocked = JSON.stringify(look.sections[0].layers[0].params); const afterLocked = JSON.stringify(subjectOf(look.sections[0].layers).params);
return expect(before !== afterUnlocked && afterUnlocked === afterLocked, return expect(before !== afterUnlocked && afterUnlocked === afterLocked,
`changed when unlocked: ${before !== afterUnlocked}, held when locked: ${afterUnlocked === afterLocked}`); `changed when unlocked: ${before !== afterUnlocked}, held when locked: ${afterUnlocked === afterLocked}`);
@ -222,12 +231,12 @@ check(3, 'a whole-track reroll preserves locked sections', () => {
const track = testTrack(); const track = testTrack();
const look = generateLook(track, { seed: 777 }); const look = generateLook(track, { seed: 777 });
look.sections[0].locked = true; look.sections[0].locked = true;
const lockedScene = look.sections[0].layers[0].module.name; const lockedScene = subjectOf(look.sections[0].layers).module.name;
const lockedParams = JSON.stringify(look.sections[0].layers[0].params); const lockedParams = JSON.stringify(subjectOf(look.sections[0].layers).params);
const next = rerollLook(look, track, 888); const next = rerollLook(look, track, 888);
return expect( return expect(
next.sections[0].layers[0].module.name === lockedScene && subjectOf(next.sections[0].layers).module.name === lockedScene &&
JSON.stringify(next.sections[0].layers[0].params) === lockedParams, JSON.stringify(subjectOf(next.sections[0].layers).params) === lockedParams,
`locked section survived a full reroll (${lockedScene})`); `locked section survived a full reroll (${lockedScene})`);
}); });

42
flow-state/src/checks/phase4.js
View File

@ -14,6 +14,7 @@ import { generateLook } from '../look/LookGenerator.js';
import { frameDistance, frameLuminance } from '../engine/hash.js'; import { frameDistance, frameLuminance } from '../engine/hash.js';
import { FeatureTrack } from '../audio/FeatureTrack.js'; import { FeatureTrack } from '../audio/FeatureTrack.js';
import { synthesizeSectioned } from '../audio/synth.js'; import { synthesizeSectioned } from '../audio/synth.js';
import { frameMaxDelta } from '../engine/hash.js';
let cached = null; let cached = null;
function arcTrack() { function arcTrack() {
@ -115,7 +116,22 @@ check(4, 'transitions produce no pops or black frames', () => {
const s = track.sections[i]; const s = track.sections[i];
const before = interior(cueAt(s.startFrame - 1)); const before = interior(cueAt(s.startFrame - 1));
const after = interior(cueAt(s.startFrame)); const after = interior(cueAt(s.startFrame));
const control = Math.max(before.peak, after.peak);
// BOTH shots' motion, not the busier one's. For most of the window
// the two stacks are on screen simultaneously — four layers, two of
// them accents in overlay or screen — and a nonlinear blend of two
// moving stacks genuinely moves more than either does alone. Taking
// the max says a dissolve may not exceed its busier half, which is
// not a property a dissolve has.
//
// Measured when four calm minimal scenes were added to the library:
// the boundary peak at frame 4528 was 0.124 with them and 0.130
// without — unchanged — while the control fell from 0.119 to 0.072
// because the mid-shot windows now landed on the calm new scenes.
// The metric was reading the denominator, not a pop. A real pop is
// still an order of magnitude clear of this: a cut to black and back
// measures ~1.0 against a control of ~0.2.
const control = before.peak + after.peak;
// The window starts just before the boundary rather than a second // The window starts just before the boundary rather than a second
// before it. The transition runs FORWARD from the boundary, so a // before it. The transition runs FORWARD from the boundary, so a
@ -250,8 +266,28 @@ check(4, 'the arc-driven render is still deterministic', () => {
const ha = hashes(a); const ha = hashes(a);
const hb = hashes(b); const hb = hashes(b);
const mismatches = ha.filter((h, i) => h !== hb[i]).length; const mismatches = ha.filter((h, i) => h !== hb[i]).length;
return expect(mismatches === 0, if (mismatches === 0) return expect(true, '60/60 frames bit-identical across two shows');
`${mismatches}/60 frames differed between two independently built shows`);
// Two independently built shows are two WebGL contexts, and bit-exactness
// is a same-context guarantee — engine/hash.js says so at the top. This
// check got away with the stricter test for a long time because the
// scenes it happened to cast were bit-stable; a casting change put
// smooth-gradient scenes in the frame and the last bit started moving.
//
// So the cross-context comparison is a distance with a threshold, and it
// is deliberately tight: three levels out of 255 is invisible, and
// anything that is actually non-deterministic will blow past it rather
// than creep up to it. Bit-exactness within ONE show is still demanded,
// by the check immediately below this one.
let worst = 0;
for (let f = 4400; f < 4460; f++) {
worst = Math.max(worst, frameMaxDelta(
Uint8Array.from(a.readPixels(a.renderFrame(f))),
Uint8Array.from(b.readPixels(b.renderFrame(f)))));
}
return expect(worst <= 4,
`${mismatches}/60 frames differed between two WebGL contexts, ` +
`worst channel delta ${worst}/255 (ceiling 4)`);
} finally { } finally {
a.dispose(); b.dispose(); a.dispose(); b.dispose();
} }

3
flow-state/src/checks/phase5.js
View File

@ -18,6 +18,7 @@ import { peakFlashRate } from '../engine/flash.js';
import { particleField } from '../scenes/layers3d/particles.js'; import { particleField } from '../scenes/layers3d/particles.js';
import { nebula } from '../scenes/shader/nebula.js'; import { nebula } from '../scenes/shader/nebula.js';
import { BLEND_MODES } from '../engine/Layer.js'; import { BLEND_MODES } from '../engine/Layer.js';
import { subjectOf } from '../look/stack.js';
const PALETTE = [ const PALETTE = [
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95], [0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
@ -192,7 +193,7 @@ check(5, 'generated looks stay within the flash-rate ceiling', () => {
} }
const rate = peakFlashRate(luminance, 60); const rate = peakFlashRate(luminance, 60);
const section = show.look.sections[cue.sectionIndex]; const section = show.look.sections[cue.sectionIndex];
const scene = (section.variants[cue.variant] || section.layers)[0].module.name; const scene = subjectOf(section.variants[cue.variant] || section.layers).module.name;
const label = `seed ${s} ${section.kind}/${scene}`; const label = `seed ${s} ${section.kind}/${scene}`;
if (rate > worst) { worst = rate; worstLabel = label; } if (rate > worst) { worst = rate; worstLabel = label; }
if (rate > 3) problems.push(`${label}: ${rate}/s`); if (rate > 3) problems.push(`${label}: ${rate}/s`);

7
flow-state/src/checks/phase7.js
View File

@ -10,12 +10,13 @@ import { check, expect, expectBelow } from './framework.js';
import { Engine } from '../engine/Engine.js'; import { Engine } from '../engine/Engine.js';
import { Show } from '../Show.js'; import { Show } from '../Show.js';
import { scenes, FAMILIES, scenesInFamily } from '../scenes/registry.js'; import { scenes, FAMILIES, scenesInFamily } from '../scenes/registry.js';
import { defaultValues, sampleValues } from '../params/schema.js'; import { defaultValues, sampleValues, canBackground } from '../params/schema.js';
import { Rng } from '../engine/rng.js'; import { Rng } from '../engine/rng.js';
import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js'; import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js';
import { synthesizeSectioned } from '../audio/synth.js'; import { synthesizeSectioned } from '../audio/synth.js';
import { generateLook } from '../look/LookGenerator.js'; import { generateLook } from '../look/LookGenerator.js';
import { frameDistance, frameMaxDelta, frameLuminance, frameVariance } from '../engine/hash.js'; import { frameDistance, frameMaxDelta, frameLuminance, frameVariance } from '../engine/hash.js';
import { subjectOf } from '../look/stack.js';
const PALETTE = [ const PALETTE = [
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95], [0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
@ -107,7 +108,7 @@ check(7, 'every scene stays live across seeds and section energies', () => {
rendered++; rendered++;
const lum = frameLuminance(pixels); const lum = frameLuminance(pixels);
const variance = frameVariance(pixels); const variance = frameVariance(pixels);
const accent = module.role === 'accent'; const accent = !canBackground(module);
const dead = accent ? variance < 0.0008 const dead = accent ? variance < 0.0008
: (lum < 0.0008 || lum > 0.99 || variance < 0.0015); : (lum < 0.0008 || lum > 0.99 || variance < 0.0015);
if (dead) { if (dead) {
@ -247,7 +248,7 @@ check(7, 'quiet sections now get minimal scenes', () => {
for (const section of look.sections) { for (const section of look.sections) {
if (!(section.kind in kinds)) continue; if (!(section.kind in kinds)) continue;
total++; total++;
const family = section.layers[0].module.family; const family = subjectOf(section.layers).module.family;
if (restful.has(family)) restfulCount++; if (restful.has(family)) restfulCount++;
if (family === 'minimal') minimalCount++; if (family === 'minimal') minimalCount++;
} }

3
flow-state/src/checks/phase8.js
View File

@ -12,6 +12,7 @@ import { synthesizeSectioned } from '../audio/synth.js';
import { generateLook } from '../look/LookGenerator.js'; import { generateLook } from '../look/LookGenerator.js';
import { MIN_SHOT_SECONDS, MAX_SHOT_SECONDS, HARD_CUT_ENERGY } from '../look/shots.js'; import { MIN_SHOT_SECONDS, MAX_SHOT_SECONDS, HARD_CUT_ENERGY } from '../look/shots.js';
import { frameDistance } from '../engine/hash.js'; import { frameDistance } from '../engine/hash.js';
import { subjectOf } from '../look/stack.js';
let cached = null; let cached = null;
function track8() { function track8() {
@ -131,7 +132,7 @@ check(8, 'the same section kind reuses the same roster', () => {
for (const look of looks()) { for (const look of looks()) {
const byKind = new Map(); const byKind = new Map();
for (const section of look.sections) { for (const section of look.sections) {
const roster = (section.variants || [section.layers]).map((v) => v[0].module.name).join('+'); const roster = (section.variants || [section.layers]).map((v) => subjectOf(v).module.name).join('+');
const seen = byKind.get(section.kind); const seen = byKind.get(section.kind);
if (seen && seen !== roster) problems.push(`${section.kind}: ${seen} vs ${roster}`); if (seen && seen !== roster) problems.push(`${section.kind}: ${seen} vs ${roster}`);
byKind.set(section.kind, roster); byKind.set(section.kind, roster);

44
flow-state/src/checks/phase9.js
View File

@ -13,13 +13,14 @@
import { check, expect } from './framework.js'; import { check, expect } from './framework.js';
import { Engine } from '../engine/Engine.js'; import { Engine } from '../engine/Engine.js';
import { scenes } from '../scenes/registry.js'; import { scenes } from '../scenes/registry.js';
import { defaultValues } from '../params/schema.js'; import { defaultValues, canBackground } from '../params/schema.js';
import { TRAITS, generatePersonality, sceneHonours, MIN_ELIGIBLE_SCENES } from '../look/Personality.js'; import { TRAITS, generatePersonality, sceneHonours, MIN_ELIGIBLE_SCENES } from '../look/Personality.js';
import { generateLook } from '../look/LookGenerator.js'; import { generateLook } from '../look/LookGenerator.js';
import { Rng } from '../engine/rng.js'; import { Rng } from '../engine/rng.js';
import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js'; import { FeatureTrack, featureProviderFor } from '../audio/FeatureTrack.js';
import { synthesizeSectioned } from '../audio/synth.js'; import { synthesizeSectioned } from '../audio/synth.js';
import { frameMaxDelta } from '../engine/hash.js'; import { frameMaxDelta } from '../engine/hash.js';
import { subjectOf } from '../look/stack.js';
const PALETTE = [ const PALETTE = [
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95], [0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
@ -40,7 +41,7 @@ function looks(count = 8) {
return Array.from({ length: count }, (_, i) => generateLook(track, { seed: 3000 + i * 6841 })); return Array.from({ length: count }, (_, i) => generateLook(track, { seed: 3000 + i * 6841 }));
} }
const castOf = (look) => look.sections.flatMap((s) => (s.variants || [s.layers]).map((v) => v[0].module)); const castOf = (look) => look.sections.flatMap((s) => (s.variants || [s.layers]).map((v) => subjectOf(v).module));
/** Two personalities differing in exactly one trait, for the "does it show" checks. */ /** Two personalities differing in exactly one trait, for the "does it show" checks. */
function pairDifferingIn(trait) { function pairDifferingIn(trait) {
@ -111,7 +112,7 @@ check(9, 'every trait has enough scenes to build a track from', () => {
// The casting rule only works if the library can staff it. A trait declared // The casting rule only works if the library can staff it. A trait declared
// by three scenes cannot carry a track — the rosters would collapse and every // by three scenes cannot carry a track — the rosters would collapse and every
// section would show the same two images, which is Phase 8 undone. // section would show the same two images, which is Phase 8 undone.
const counts = TRAITS.map((t) => [t, scenes.filter((m) => m.role !== 'accent' const counts = TRAITS.map((t) => [t, scenes.filter((m) => canBackground(m)
&& sceneHonours(m, [t])).length]); && sceneHonours(m, [t])).length]);
const thin = counts.filter(([, n]) => n < MIN_ELIGIBLE_SCENES); const thin = counts.filter(([, n]) => n < MIN_ELIGIBLE_SCENES);
return expect(thin.length === 0, return expect(thin.length === 0,
@ -119,20 +120,39 @@ check(9, 'every trait has enough scenes to build a track from', () => {
(thin.length ? ` — too thin: ${thin.map(([t]) => t).join(', ')}` : '')); (thin.length ? ` — too thin: ${thin.map(([t]) => t).join(', ')}` : ''));
}); });
check(9, 'no scene is cast in a track it cannot express', () => { check(9, 'casting leans hard on the track\'s signature', () => {
// The whole point. A scene that ignores the trait a track is built on is the // This used to demand that EVERY cast scene honour the signature, because
// shot that was obviously filmed somewhere else. // the signature was a hard filter. It is a weight now — measured, the filter
const problems = []; // was the largest single cause of sameness in the generator, disqualifying
// eleven scenes into half of all videos and leaving a third of the library
// uncastable. See look/Personality.js signatureWeight.
//
// So the claim being checked changes shape. A track may reach outside its
// signature; what it must not do is stop leaning on it. Honouring scenes
// should dominate the cast by a wide margin, and the anchor of each section
// — the scene that opens it and returns most often — should almost always
// honour it.
let honoured = 0, total = 0, anchorsHonoured = 0, anchors = 0;
for (const look of looks()) { for (const look of looks()) {
const signature = look.personality.signature; const signature = look.personality.signature;
for (const module of castOf(look)) { for (const module of castOf(look)) {
if (!sceneHonours(module, signature)) { total++;
problems.push(`${module.name} cast in a ${signature.join('+')} track`); if (sceneHonours(module, signature)) honoured++;
}
for (const section of look.sections) {
anchors++;
if (sceneHonours(subjectOf(section.layers).module, signature)) anchorsHonoured++;
} }
} }
} const share = total ? honoured / total : 0;
return expect(problems.length === 0, const anchorShare = anchors ? anchorsHonoured / anchors : 0;
[...new Set(problems)].slice(0, 4).join(' · ') || 'every scene honours its track');
// A signature of two traits leaves roughly a quarter of the library eligible,
// so chance alone would land near 0.25. Anything close to that means the
// weighting has stopped meaning anything.
return expect(share >= 0.55 && anchorShare >= 0.6,
`${(share * 100).toFixed(0)}% of cast scenes honour the signature ` +
`(chance ~25%), ${(anchorShare * 100).toFixed(0)}% of section anchors do`);
}); });
check(9, 'the signature still leaves a track enough scenes to cut between', () => { check(9, 'the signature still leaves a track enough scenes to cut between', () => {

51
flow-state/src/checks/scene-gate.js
View File

@ -23,6 +23,7 @@ import { synthesizeSectioned } from '../audio/synth.js';
import { frameLuminance, frameVariance, frameMaxDelta } from '../engine/hash.js'; import { frameLuminance, frameVariance, frameMaxDelta } from '../engine/hash.js';
import { peakFlashRate } from '../engine/flash.js'; import { peakFlashRate } from '../engine/flash.js';
import { generatePersonality } from '../look/Personality.js'; import { generatePersonality } from '../look/Personality.js';
import { generateIdentity } from '../look/Identity.js';
const PALETTE = [ const PALETTE = [
[0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95], [0.06, 0.03, 0.16], [0.85, 0.15, 0.55], [0.15, 0.75, 0.95],
@ -133,6 +134,56 @@ export function runSceneGate(name) {
const rate = peakFlashRate(luminance, 60); const rate = peakFlashRate(luminance, 60);
record(rate <= 3, 'flash rate', `${rate}/s at aggressive settings (ceiling 3)`); record(rate <= 3, 'flash rate', `${rate}/s at aggressive settings (ceiling 3)`);
// --- identity response -----------------------------------------------
// The migration gate. A scene that declares it consumes the cast must
// produce a DIFFERENT PICTURE when the song's cast changes — otherwise
// `consumes` is a comment and the whole inversion is unverifiable at
// library scale. Deliberately a much larger threshold than the trait
// check: a trait may be honoured subtly, but content is the subject.
for (const artifact of module.consumes || []) {
const other = generatePersonality(SUMMARY, new Rng(9001));
// Two identities as far apart as the generator can make them.
const alt = generateIdentity(
{ ...SUMMARY, meanFlatness: 0.6, meanCentroid: 0.85, bpm: 168 },
new Rng(31337), 6);
// Force the always-visible ink decisions on. A field scene's whole
// migration may be `inkValue`, and posterisation is off for most
// identities — without pinning it the probe would sometimes hand the
// scene two identities that ask it for the same picture and then
// fail it for complying.
alt.ink = { ...alt.ink, posterize: 4, fill: 'hatch', outline: 0.8, weight: 0.8 };
other.identity = alt;
if (artifact === 'cast') {
// The protagonist's geometry is read from the signature form.
other.shape = { sides: 8, roundness: 0.02, elongation: 1.4, tilt: 0.9 };
}
if (artifact === 'form') {
// Pin an assembly that is unmistakably not the fallback profile:
// a five-fold radial with a limb carved out of the body. A scene
// that renders this the same as a plain extrusion is treating
// the solid as a modifier, which is what the gate is for.
alt.form = {
symmetry: 'radial', symmetryN: 5, blend: 0.22, depth: 1.5,
chorus: { count: 2, symmetry: 'mirror', symmetryN: 3, flat: 1.5, thin: 0.6 },
parts: [
{ kind: 'prism', op: 'union', offset: [0, 0, 0], scale: [0.8, 0.7, 0.5], yaw: 0.4, pitch: 0.2, round: 0.1 },
{ kind: 'capsule', op: 'blend', offset: [0.6, 0.25, 0.1], scale: [0.3, 0.5, 0.3], yaw: 1.1, pitch: -0.4, round: 0.2 },
{ kind: 'torus', op: 'carve', offset: [0, 0.1, 0], scale: [0.55, 0.3, 0.4], yaw: 0.2, pitch: 0.8, round: 0 },
],
};
}
engine.setLayerSpecs([{
module, params: defaultValues(module), seed: 4242,
opacity: 1, blend: 'normal', palette: PALETTE, personality: other,
}]);
engine.compositor.reset();
const changed = frameMaxDelta(base,
Uint8Array.from(engine.readPixels(engine.renderFrame(600))));
record(changed > 24, `consumes: ${artifact}`,
`delta ${changed}/255 (floor 24)`);
}
// --- personality response -------------------------------------------- // --- personality response --------------------------------------------
// Every declared trait must move the image; a trait declared and ignored // Every declared trait must move the image; a trait declared and ignored
// gets the scene cast in tracks it cannot express. // gets the scene cast in tracks it cannot express.

241
flow-state/src/checks/scene-sweep.js Normal file
View File

@ -0,0 +1,241 @@
// One scene, across a wide grid of songs and seeds.
//
// The gallery answers "which scenes repeat themselves" by showing every scene
// six times. This answers the question you have immediately afterwards, which it
// cannot: WHY does this one repeat, and what actually moves it?
//
// Six thumbnails is enough to rank sixty-eight scenes and far too few to study
// one. Six songs at one seed each confounds the two inputs — a scene that looks
// the same six times might be ignoring the song, or ignoring the seed, and the
// gallery cannot tell you which. So this separates them onto the two axes of a
// grid:
//
// rows the SONG — different audio, so different features, sections,
// biases and sampled parameter centres.
// columns the SEED — same audio, so the same everything except the roll:
// a different identity, palette and parameter draw.
//
// A row that is uniform means the seed does nothing for this scene. A column
// that is uniform means the song does nothing. A grid that is uniform means the
// scene is a constant, and the diagonal being varied while the rows are flat is
// the shape you get when the palette is doing all the work.
//
// Everything is a real generator output — the same generateLook the exporter
// runs — so the grid shows what would actually be produced, with only the scene
// held fixed instead of chosen.
import { Rng, hashString } from '../engine/rng.js';
import { featureProviderFor } from '../audio/FeatureTrack.js';
import { song, SONGS } from '../audio/songbank.js';
import { generateLook } from '../look/LookGenerator.js';
import { describeIdentity } from '../look/Identity.js';
import { sampleValues } from '../params/schema.js';
import { frameDescriptor, motionDescriptor } from './variety/descriptors.js';
import { descriptorDistance, STRUCTURAL } from './variety/signature.js';
import { THUMB } from './gallery.js';
/** How many songs the bank can offer. The grid cannot ask for more. */
export const MAX_SONGS = SONGS.length;
/** Grid sizes the buttons offer, widest last. */
export const SWEEP_SIZES = [
{ label: '4 × 4', songs: 4, seeds: 4 },
{ label: '6 × 6', songs: 6, seeds: 6 },
{ label: '8 × 8', songs: 8, seeds: 8 },
{ label: '12 × 8', songs: 12, seeds: 8 },
{ label: 'every song × 10', songs: MAX_SONGS, seeds: 10 },
];
// Analysing a song is the slow part of building a grid — seconds each, against
// milliseconds for a look and a render. Re-rendering with a different seed count
// or a different scene must not pay for it twice, so tracks are held for the
// life of the page.
const trackCache = new Map();
/** Which bank entries a grid of this width uses — the spread songBank would pick. */
function specsFor(count) {
const n = Math.min(count, SONGS.length);
if (n >= SONGS.length) return SONGS;
return Array.from({ length: n }, (_, i) => SONGS[Math.round((i * (SONGS.length - 1)) / (n - 1))]);
}
/**
* Analyse the songs a grid needs, one at a time, reporting as each lands.
*
* Analysis is seconds per song and everything after it is milliseconds, so the
* widest grid spends most of its wall time here. Done in one call it froze the
* page for the better part of a minute with a status line nobody could see
* update; one song per turn of the event loop costs nothing and keeps the count
* moving. Cached for the life of the page, so only the first grid pays.
*/
export async function prepareSongs(count, onProgress) {
const specs = specsFor(count);
const tracks = [];
for (let i = 0; i < specs.length; i++) {
const name = specs[i].name;
if (!trackCache.has(name)) trackCache.set(name, song(name));
tracks.push(trackCache.get(name));
if (onProgress) onProgress(i + 1, specs.length, name);
await new Promise((r) => setTimeout(r, 0));
}
return tracks;
}
function tracksFor(count) {
return specsFor(count).map((s) => {
if (!trackCache.has(s.name)) trackCache.set(s.name, song(s.name));
return trackCache.get(s.name);
});
}
/**
* The grid's cells, in row-major order: song by song, seed by seed.
*
* `seedOffset` walks the grid onto a fresh set of seeds without changing its
* shape, which is how you check whether a flat-looking scene is flat or merely
* unlucky — ten more draws is a much better answer to that than staring harder
* at the same ten.
*
* @returns {{cells: object[], songNames: string[], seedLabels: string[]}}
*/
export function sweepGrid({ songs = 6, seeds = 6, seedOffset = 0 } = {}) {
const bank = tracksFor(Math.min(songs, MAX_SONGS));
const cells = [];
const seedLabels = [];
for (let s = 0; s < seeds; s++) seedLabels.push(`#${seedOffset + s}`);
for (const entry of bank) {
for (let s = 0; s < seeds; s++) {
// The SAME seed all the way down a column, deliberately.
//
// The first version hashed the song name in as well, so that column
// zero would not be one roll repeated down the grid. That reads
// better and measures nothing: with the seed varying on both axes,
// "across songs" and "across seeds" are the same comparison, and
// measured, they came back within 0.001 of each other for every
// scene tried — two numbers that could never disagree. Holding the
// seed fixed down a column is what makes that column attributable
// to the music.
//
// The identity still differs down a column, because it is derived
// from the track's summary as well as the seed. That is the point:
// the difference that survives a fixed roll is the song's doing.
const seed = hashString(`sweep:${seedOffset + s}`);
const look = generateLook(entry.track, { seed });
// The busiest section: where the scene is asked for the most, and
// where two draws are likeliest to converge on the same picture.
const section = look.sections.reduce(
(best, x) => (x.bias.energy > best.bias.energy ? x : best), look.sections[0]);
cells.push({
name: `${entry.name}${seedLabels[s]}`,
song: entry.name,
seedIndex: seedOffset + s,
seed,
track: entry.track,
palette: look.palette,
personality: look.personality,
bias: section.bias,
frame: section.startFrame + Math.floor((section.endFrame - section.startFrame) * 0.5),
identity: describeIdentity(look.personality.identity),
});
}
}
return { cells, songNames: bank.map((e) => e.name), seedLabels };
}
/**
* Render one scene over every cell, reporting each as it lands.
*
* Progressive on purpose: a wide grid is a hundred and seventy renders plus the
* audio analysis in front of it, and a page that shows nothing until the end of
* that reads as broken.
*/
export async function sweepScene({ engine, module, cells, onCell }) {
const thumbs = [];
const descriptors = [];
for (let i = 0; i < cells.length; i++) {
const ctx = cells[i];
engine.timeline.setDuration(ctx.track.duration);
engine.setFeatureProvider(featureProviderFor(ctx.track));
// Params come from the cell's own draw, exactly as the generator would
// sample them: the song's section bias and the seed's temperament.
const rng = new Rng(ctx.seed ^ hashString(module.name));
const params = sampleValues(module, rng, ctx.bias, ctx.personality.temperament);
engine.setLayerSpecs([{
module, params, seed: rng.int(0, 0x7fffffff),
opacity: 1, blend: 'normal',
palette: ctx.palette, personality: ctx.personality,
}]);
engine.compositor.reset();
for (let f = ctx.frame - 6; f < ctx.frame; f++) engine.renderFrame(f);
const pixels = Uint8Array.from(engine.readPixels(engine.renderFrame(ctx.frame)));
const moved = Uint8Array.from(engine.readPixels(engine.renderFrame(ctx.frame + 5)));
thumbs.push(pixels);
const still = frameDescriptor(pixels, THUMB.width, THUMB.height);
const motion = motionDescriptor(pixels, moved, THUMB.width, THUMB.height);
descriptors.push({ ...still, motion: motion.scale.concat(motion.layout) });
if (onCell) await onCell(i, cells.length, pixels);
}
return { thumbs, ...scoreGrid(descriptors, cells) };
}
/**
* Score the grid three ways, because one number cannot say what is wrong.
*
* overall every cell against every other — comparable to the gallery's
* variety score, on the same structural descriptor.
* bySeed cells of the SAME song, different seeds. Low means the seed does
* nothing here: the scene ignores the identity.
* bySong cells at the same seed index, different songs. Low means the song
* does nothing: the scene ignores the music.
*
* Those two are the diagnosis the gallery's single number cannot give, and they
* point at different fixes — a flat seed axis is a scene not reading the cast,
* a flat song axis is a scene not reading its features or its bias.
*/
function scoreGrid(descriptors, cells) {
const byBlock = {};
let total = 0, pairs = 0;
let seedSum = 0, seedPairs = 0;
let songSum = 0, songPairs = 0;
let closest = null;
for (let i = 0; i < descriptors.length; i++) {
for (let j = i + 1; j < descriptors.length; j++) {
const d = descriptorDistance(descriptors[i], descriptors[j]);
const structural = STRUCTURAL.reduce((a, b) => a + (d[b] || 0), 0) / STRUCTURAL.length;
for (const b of [...STRUCTURAL, 'colour']) byBlock[b] = (byBlock[b] || 0) + (d[b] || 0);
total += structural;
pairs++;
if (cells[i].song === cells[j].song) { seedSum += structural; seedPairs++; }
if (cells[i].seedIndex === cells[j].seedIndex) { songSum += structural; songPairs++; }
// The two cells that are most alike. On a wide grid this is the only
// practical way to find the pair worth looking at — a hundred and
// seventy thumbnails is past what an eye will compare.
if (!closest || structural < closest.distance) {
closest = { distance: structural, a: i, b: j };
}
}
}
for (const b of Object.keys(byBlock)) byBlock[b] /= pairs || 1;
return {
variety: pairs ? total / pairs : 0,
bySeed: seedPairs ? seedSum / seedPairs : 0,
bySong: songPairs ? songSum / songPairs : 0,
byBlock,
closest,
};
}

423
flow-state/src/checks/variety/descriptors.js Normal file
View File

@ -0,0 +1,423 @@
// A frame reduced to a STRUCTURAL descriptor: what shape the image is, not what
// colour it is or which way up.
//
// The problem this exists to solve: two seeds can produce videos that a pixel
// difference calls wildly different — one is teal and rotating left, the other
// is magenta and rotating right — while a viewer calls them the same video.
// Any metric built on raw pixels rewards exactly the variation we do not care
// about. So the descriptor is built to be blind to the cheap axes and sensitive
// to the expensive ones:
//
// blind to brightness, contrast, hue, saturation, global rotation
// sensitive to feature SCALE (fine texture vs big soft blobs), ORIENTATION
// structure (grid vs radial vs stripes), LAYOUT (centred glow vs
// full-frame vs banded), and TEXTURE STATISTICS (how many
// distinct elements, how sparse, how symmetric)
//
// Colour is still measured, but it is kept in its own block and excluded from
// the structural total — so a report can say "your palettes vary, your
// structure does not", which is the distinction the whole exercise is about.
//
// Everything here is plain Float32 maths on a 96x96 luma image. No FFT: a
// Laplacian pyramid gives the radial power spectrum directly, and a magnitude-
// weighted gradient histogram gives the angular one.
export const SIZE = 96;
/** RGBA bytes → square Float32 luma in 0..1, resampled to SIZE x SIZE. */
export function toLuma(pixels, width, height, size = SIZE) {
const out = new Float32Array(size * size);
for (let y = 0; y < size; y++) {
const sy = Math.min(height - 1, Math.floor((y + 0.5) * height / size));
for (let x = 0; x < size; x++) {
const sx = Math.min(width - 1, Math.floor((x + 0.5) * width / size));
const i = (sy * width + sx) * 4;
out[y * size + x] =
(0.2126 * pixels[i] + 0.7152 * pixels[i + 1] + 0.0722 * pixels[i + 2]) / 255;
}
}
return out;
}
/**
* Zero mean, unit standard deviation, in place.
*
* This is where exposure and contrast leave the metric. Two renders of the same
* structure at different brightness are the same image after this; that is the
* point, and it is why the descriptor cannot be fooled by a palette reroll.
*/
export function standardize(img) {
let mean = 0;
for (let i = 0; i < img.length; i++) mean += img[i];
mean /= img.length;
let variance = 0;
for (let i = 0; i < img.length; i++) variance += (img[i] - mean) ** 2;
const sd = Math.sqrt(variance / img.length) || 1e-6;
for (let i = 0; i < img.length; i++) img[i] = (img[i] - mean) / sd;
return img;
}
/** 2x2 box downsample. */
function halve(src, w, h) {
const ow = w >> 1, oh = h >> 1;
const out = new Float32Array(ow * oh);
for (let y = 0; y < oh; y++) {
for (let x = 0; x < ow; x++) {
const i = (y * 2) * w + x * 2;
out[y * ow + x] = (src[i] + src[i + 1] + src[i + w] + src[i + w + 1]) * 0.25;
}
}
return out;
}
/** Nearest-neighbour 2x upsample — good enough as the pyramid's low-pass. */
function double(src, w, h) {
const ow = w * 2, oh = h * 2;
const out = new Float32Array(ow * oh);
for (let y = 0; y < oh; y++) {
for (let x = 0; x < ow; x++) out[y * ow + x] = src[(y >> 1) * w + (x >> 1)];
}
return out;
}
/**
* Laplacian pyramid: one band-pass image per octave.
*
* Band k holds the detail that lives at roughly 2^k pixels. The energy across
* bands IS the radial power spectrum, which is the single most useful thing to
* know about an abstract image: fine grain, mid-scale filigree and slow blobs
* are structurally different pictures no matter what colour they are.
*/
export function pyramid(img, size = SIZE, levels = 5) {
const bands = [];
let cur = img, w = size, h = size;
for (let k = 0; k < levels; k++) {
const low = halve(cur, w, h);
const up = double(low, w >> 1, h >> 1);
const band = new Float32Array(cur.length);
for (let i = 0; i < cur.length; i++) band[i] = cur[i] - up[i];
bands.push({ data: band, w, h });
cur = low; w >>= 1; h >>= 1;
}
return bands;
}
function rms(a) {
let s = 0;
for (let i = 0; i < a.length; i++) s += a[i] * a[i];
return Math.sqrt(s / a.length);
}
/** Scale block: the radial spectrum, normalised to its own sum so it describes
* the SHAPE of the spectrum rather than how contrasty the frame was. */
function scaleBlock(bands) {
const v = bands.map((b) => rms(b.data));
const total = v.reduce((a, x) => a + x, 0) || 1e-6;
return v.map((x) => x / total);
}
const ORIENT_BINS = 12;
/**
* Orientation block, made rotation-invariant.
*
* A magnitude-weighted histogram of gradient direction (mod pi) says whether
* the image is a grid (two peaks 90 apart), stripes (one peak), radial
* (flat-ish), or a quasicrystal (five peaks). Rotating the image circularly
* SHIFTS that histogram, so taking the magnitude of its DFT throws the shift
* away and keeps the pattern. That is exactly the "not just rotation" property
* we need: turn a scene 30 degrees and this block does not move.
*/
function orientBlock(band) {
const { data, w, h } = band;
const hist = new Float64Array(ORIENT_BINS);
for (let y = 1; y < h - 1; y++) {
for (let x = 1; x < w - 1; x++) {
const i = y * w + x;
const gx = data[i + 1] - data[i - 1];
const gy = data[i + w] - data[i - w];
const mag = Math.hypot(gx, gy);
if (mag < 1e-6) continue;
let a = Math.atan2(gy, gx);
if (a < 0) a += Math.PI; // direction, not sign
if (a >= Math.PI) a -= Math.PI;
hist[Math.min(ORIENT_BINS - 1, Math.floor(a / Math.PI * ORIENT_BINS))] += mag;
}
}
const dc = hist.reduce((a, x) => a + x, 0) || 1e-6;
const out = [];
for (let k = 1; k <= 6; k++) {
let re = 0, im = 0;
for (let n = 0; n < ORIENT_BINS; n++) {
const th = -2 * Math.PI * k * n / ORIENT_BINS;
re += hist[n] * Math.cos(th);
im += hist[n] * Math.sin(th);
}
out.push(Math.hypot(re, im) / dc);
}
return out;
}
/**
* Layout block: where in the frame the structure actually is, on a 4x4 grid.
*
* Deliberately NOT rotation-invariant, and kept separate for that reason.
* "Everything the generator makes is a bright thing in the middle of a dark
* frame" is a composition failure, and it is invisible to every other block
* here — the scale and orientation spectra of two differently-composed frames
* can match perfectly.
*/
function layoutBlock(band, cells = 4) {
const { data, w, h } = band;
const grid = new Float64Array(cells * cells);
const counts = new Float64Array(cells * cells);
for (let y = 0; y < h; y++) {
const gy = Math.min(cells - 1, Math.floor(y * cells / h));
for (let x = 0; x < w; x++) {
const gx = Math.min(cells - 1, Math.floor(x * cells / w));
const v = data[y * w + x];
grid[gy * cells + gx] += v * v;
counts[gy * cells + gx]++;
}
}
let total = 0;
for (let i = 0; i < grid.length; i++) {
grid[i] = Math.sqrt(grid[i] / (counts[i] || 1));
total += grid[i];
}
return Array.from(grid, (x) => x / (total || 1e-6));
}
function corr(a, b) {
let num = 0, da = 0, db = 0;
for (let i = 0; i < a.length; i++) { num += a[i] * b[i]; da += a[i] * a[i]; db += b[i] * b[i]; }
return num / (Math.sqrt(da * db) || 1e-6);
}
/**
* Texture block: how many things are in the frame, how sparse they are, and how
* symmetric the composition is.
*
* Symmetry earns its place: mirror and 180-degree self-correlation is what
* separates a kaleidoscope from a drift, and a library that quietly funnels
* every seed into radially symmetric imagery will show up here as three numbers
* that never move, when nothing else in the descriptor notices.
*/
function textureBlock(band) {
const { data, w, h } = band;
// Zero crossings per row and per column: a proxy for element count that
// costs nothing and does not care about contrast.
let rowCross = 0, colCross = 0;
for (let y = 0; y < h; y++) {
for (let x = 1; x < w; x++) {
if ((data[y * w + x] > 0) !== (data[y * w + x - 1] > 0)) rowCross++;
}
}
for (let x = 0; x < w; x++) {
for (let y = 1; y < h; y++) {
if ((data[y * w + x] > 0) !== (data[(y - 1) * w + x] > 0)) colCross++;
}
}
const sd = rms(data) || 1e-6;
let sparse = 0, tail = 0;
for (let i = 0; i < data.length; i++) {
const m = Math.abs(data[i]);
if (m > sd) sparse++;
if (m > sd * 2.5) tail++;
}
const mirrorH = new Float32Array(data.length);
const mirrorV = new Float32Array(data.length);
const rot = new Float32Array(data.length);
for (let y = 0; y < h; y++) {
for (let x = 0; x < w; x++) {
mirrorH[y * w + x] = data[y * w + (w - 1 - x)];
mirrorV[y * w + x] = data[(h - 1 - y) * w + x];
rot[y * w + x] = data[(h - 1 - y) * w + (w - 1 - x)];
}
}
// Horizontal and vertical measurements are folded into a sum and an
// absolute difference rather than reported as a pair. A quarter turn SWAPS
// the two, and a descriptor that changed under a quarter turn would be
// measuring orientation twice — once here, unintentionally, on top of the
// orient block that handles it properly. Folded this way the anisotropy
// survives (a striped frame still reads as anisotropic) but its direction
// does not.
const cx = rowCross / (w * h), cy = colCross / (w * h);
const mh = corr(data, mirrorH), mv = corr(data, mirrorV);
return [
cx + cy, Math.abs(cx - cy),
sparse / data.length, tail / data.length,
(mh + mv) / 2, Math.abs(mh - mv), corr(data, rot),
];
}
/**
* Colour block. Reported, never counted in the structural score.
*
* Its job in the report is adversarial: it is the number that proves a high
* pixel-difference between two seeds was only ever a palette swap.
*/
function colourBlock(pixels) {
const hist = new Float64Array(6);
let sat = 0, val = 0, n = 0;
for (let i = 0; i < pixels.length; i += 4) {
const r = pixels[i] / 255, g = pixels[i + 1] / 255, b = pixels[i + 2] / 255;
const max = Math.max(r, g, b), min = Math.min(r, g, b);
const d = max - min;
const s = max <= 0 ? 0 : d / max;
sat += s; val += max; n++;
if (d < 1e-4) continue;
let hue;
if (max === r) hue = ((g - b) / d + 6) % 6;
else if (max === g) hue = (b - r) / d + 2;
else hue = (r - g) / d + 4;
hist[Math.min(5, Math.floor(hue))] += s * max;
}
const total = hist.reduce((a, x) => a + x, 0) || 1e-6;
return [...Array.from(hist, (x) => x / total), sat / n, val / n];
}
/**
* Region block: how differently the parts of the frame behave from each other.
*
* The gap `layout` leaves. Layout says WHERE the energy is, normalised, so a
* uniform field and a field with a subject in it can normalise to nearly the
* same answer — measured, every full-frame scene sat at 0.004 there no matter
* what was done to it. What distinguishes them is not the distribution of
* energy but whether one part of the picture is doing something different from
* another. A pattern spread evenly has nothing to look at precisely because
* every region is the same region.
*
* So each region is characterised in its own right — how much detail, how
* anisotropic, how many elements — and then expressed as its DEVIATION from the
* frame's average. A uniform field deviates by nothing everywhere; a subject
* shows up as a region behaving unlike its neighbours, and which region and how
* is something the song can change.
*/
function regionBlock(band, cells = 3) {
const { data, w, h } = band;
const rw = Math.floor(w / cells), rh = Math.floor(h / cells);
const rows = [];
for (let gy = 0; gy < cells; gy++) {
for (let gx = 0; gx < cells; gx++) {
let energy = 0, gxx = 0, gyy = 0, cross = 0, n = 0;
for (let y = gy * rh + 1; y < (gy + 1) * rh - 1; y++) {
for (let x = gx * rw + 1; x < (gx + 1) * rw - 1; x++) {
const i = y * w + x;
const v = data[i];
energy += v * v;
const dx = data[i + 1] - data[i - 1];
const dy = data[i + w] - data[i - w];
gxx += dx * dx;
gyy += dy * dy;
if ((v > 0) !== (data[i - 1] > 0)) cross++;
n++;
}
}
n = n || 1;
rows.push([
Math.sqrt(energy / n), // how much detail
(gxx - gyy) / (gxx + gyy + 1e-6), // which way it runs
cross / n, // how many elements
]);
}
}
// Deviation from the frame's own average, so this measures HETEROGENEITY
// rather than overall level — a brighter or busier frame does not register
// as a more varied one.
const mean = [0, 1, 2].map((k) => rows.reduce((a, r) => a + r[k], 0) / rows.length);
const scale = [0, 1, 2].map((k) => Math.max(1e-4, Math.abs(mean[k])) );
return rows.flatMap((r) => [
(r[0] - mean[0]) / scale[0],
r[1] - mean[1],
(r[2] - mean[2]) / scale[2],
]);
}
/**
* Full descriptor for one frame.
*
* @param {Uint8Array} pixels RGBA readback
* @returns {{scale:number[], orient:number[], layout:number[], texture:number[], colour:number[]}}
*/
export function frameDescriptor(pixels, width, height) {
const luma = standardize(toLuma(pixels, width, height));
const bands = pyramid(luma);
// Band 1 (~4px detail) is the working band for orientation, layout and
// texture: band 0 is dominated by grain and compression-scale noise, and
// the coarse bands are too small to have a layout worth reading.
const band = bands[1];
return {
scale: scaleBlock(bands),
orient: orientBlock(band),
layout: layoutBlock(band),
region: regionBlock(band),
texture: textureBlock(band),
colour: colourBlock(pixels),
};
}
/**
* Descriptor of what MOVED between two frames.
*
* The absolute difference image, run through the same machinery. This is the
* block that catches "every scene in the library is a slow full-frame drift":
* two videos can differ in every static frame and still move identically, and
* motion is a large part of what a viewer reads as the identity of a visual.
*/
export function motionDescriptor(a, b, width, height) {
const diff = new Uint8Array(a.length);
for (let i = 0; i < a.length; i += 4) {
diff[i] = Math.abs(a[i] - b[i]);
diff[i + 1] = Math.abs(a[i + 1] - b[i + 1]);
diff[i + 2] = Math.abs(a[i + 2] - b[i + 2]);
diff[i + 3] = 255;
}
const d = frameDescriptor(diff, width, height);
let energy = 0;
for (let i = 0; i < diff.length; i += 4) energy += diff[i] + diff[i + 1] + diff[i + 2];
energy = energy / ((diff.length / 4) * 3 * 255);
return { ...d, energy };
}
// --- transforms used to VALIDATE the metric ------------------------------
// A structural metric that has never been shown to ignore colour and rotation
// is a claim, not a measurement. These let the checks prove it.
/** Rotate an RGBA frame by 90 degrees. Returns {pixels, width, height}. */
export function rotate90(pixels, width, height) {
const out = new Uint8Array(pixels.length);
for (let y = 0; y < height; y++) {
for (let x = 0; x < width; x++) {
const si = (y * width + x) * 4;
const di = (x * height + (height - 1 - y)) * 4;
out[di] = pixels[si]; out[di + 1] = pixels[si + 1];
out[di + 2] = pixels[si + 2]; out[di + 3] = pixels[si + 3];
}
}
return { pixels: out, width: height, height: width };
}
/** Rotate the hue of every pixel by `turns` and scale brightness. */
export function recolour(pixels, turns = 0.33, gain = 1.25) {
const out = new Uint8Array(pixels.length);
const c = Math.cos(turns * 2 * Math.PI), s = Math.sin(turns * 2 * Math.PI);
// YIQ hue rotation — cheap, and it leaves luma structure exactly alone.
const m = [
0.299 + 0.701 * c + 0.168 * s, 0.587 - 0.587 * c + 0.330 * s, 0.114 - 0.114 * c - 0.497 * s,
0.299 - 0.299 * c - 0.328 * s, 0.587 + 0.413 * c + 0.035 * s, 0.114 - 0.114 * c + 0.292 * s,
0.299 - 0.300 * c + 1.250 * s, 0.587 - 0.588 * c - 1.050 * s, 0.114 + 0.886 * c - 0.203 * s,
];
for (let i = 0; i < pixels.length; i += 4) {
const r = pixels[i], g = pixels[i + 1], b = pixels[i + 2];
out[i] = Math.max(0, Math.min(255, (m[0] * r + m[1] * g + m[2] * b) * gain));
out[i + 1] = Math.max(0, Math.min(255, (m[3] * r + m[4] * g + m[5] * b) * gain));
out[i + 2] = Math.max(0, Math.min(255, (m[6] * r + m[7] * g + m[8] * b) * gain));
out[i + 3] = pixels[i + 3];
}
return out;
}

429
flow-state/src/checks/variety/print.js Normal file
View File

@ -0,0 +1,429 @@
// The variety report as text.
//
// Kept apart from the measurement so the gate and the diagnostic read the same
// numbers. The layout is chosen so the first three lines answer "is this bad",
// and everything below answers "where did the variety go" — which is the only
// question worth printing a table for.
import { song } from '../../audio/songbank.js';
import {
measureVariety, measureSpecDiversity, librarySweep,
measureSongVariety, measureFingerprint, measureDecomposition,
} from './report.js';
import { generateLook, describeLook } from '../../look/LookGenerator.js';
const bar = (v, width = 24) => {
const n = Math.max(0, Math.min(width, Math.round(v * width)));
return '█'.repeat(n) + '·'.repeat(width - n);
};
const pct = (v) => `${(v * 100).toFixed(0)}%`.padStart(4);
export async function varietyReportLines({ seeds = 8, probes = 5, library = true } = {}) {
// A real bank entry, not the two-section synthetic this used to run on.
// That one segmented into intro and outro only, and quiet kinds are
// restricted to the restful families for every director — a third of the
// library was unreachable and the score was measuring that, not the seed.
const track = song('centre').track;
const lines = [];
const spec = measureSpecDiversity(track, { seeds: Math.max(seeds, 24) });
// Yield so the "measuring…" line paints before the GPU work blocks.
await new Promise((r) => setTimeout(r, 0));
const r = measureVariety(track, { seeds, refScenes: 5, probes });
const ok = r.ceilingValid && r.separation >= 0.35;
const headline = r.ceilingValid
? `seed separation ${r.separation.toFixed(2)} (${ok ? 'acceptable' : 'TOO LOW'}) · ` +
`${pct(r.identity)} of what the library can express`
: `observed ${r.observed.toFixed(3)} vs floor ${r.floor.toFixed(3)} · ` +
'reference below the floor, separation not computable';
lines.push('SEED VARIETY — one track, one analysis, only the seed changes');
lines.push('');
lines.push(` floor ${r.floor.toFixed(4)} one video against itself, across its own sections`);
lines.push(` observed ${r.observed.toFixed(4)} two seeds against each other`);
lines.push(` ceiling ${r.ceiling.toFixed(4)} same pipeline, casts that share no scenes at all`);
lines.push('');
lines.push(r.ceilingValid
? ` separation ${bar(r.separation)} ${r.separation.toFixed(2)}`
: ' separation — not computable: the reference landed below the floor,\n' +
' which means real seeds already differ by more than videos\n' +
' built from casts that share no scenes.');
lines.push(' 0 = the seed changes nothing a viewer could name');
lines.push(' 1 = two seeds as unalike as two randomly assembled videos');
lines.push('');
lines.push('DIRECTION — of the floor above, how much is the video GOING somewhere');
lines.push('');
lines.push(` direction ${bar((r.direction + 1) / 2)} ${r.direction.toFixed(2)}` +
` (arcless reference ${r.directionFloor.toFixed(2)})`);
lines.push(' rank correlation between how far apart two probes are in');
lines.push(' time and how far apart they look. A video with an arc is');
lines.push(' most unlike itself at its two ends; one that rotates a');
lines.push(' roster is as unlike itself after ten seconds as after four');
lines.push(' minutes. Both score the same floor. Read them together:');
lines.push(' a floor that rises WITH direction is a story, and a floor');
lines.push(' that rises without one is a shuffle.');
lines.push('');
lines.push('WHERE THE VARIETY IS — per structural block, as a fraction of achievable');
lines.push('');
for (const [name, b] of Object.entries(r.byBlock)) {
const note = name === 'colour' ? ' (not counted — this is the axis that lies)' : '';
lines.push(` ${name.padEnd(8)} ${bar(b.ratio)} ${pct(b.ratio)}` +
` ${b.between.toFixed(3)} of ${b.ceiling.toFixed(3)}${note}`);
}
lines.push('');
lines.push(' scale feature size — fine texture vs large soft forms');
lines.push(' orient grid vs radial vs stripes, measured rotation-blind');
lines.push(' layout where in the frame the structure sits');
lines.push(' texture element count, sparsity, mirror and radial symmetry');
lines.push(' motion what moves and where, not how much');
lines.push('');
lines.push('CLOSEST SIBLING PER SEED — a seed below the floor is a duplicate video');
lines.push('');
r.nearest.forEach((d, i) => {
const flag = d < r.floor * 0.75 ? ' ← shadowed' : d < r.floor ? ' ← thin' : '';
lines.push(` seed ${r.seeds[i].toString(16).padStart(8, '0')} ${bar(d / Math.max(r.ceiling, 1e-6))} ${d.toFixed(3)}${flag}`);
});
if (r.worstPair) {
const w = r.worstPair;
const blocks = Object.entries(w.byBlock)
.map(([n, v]) => `${n} ${v.toFixed(3)}`).join(' · ');
lines.push('');
lines.push(` closest pair: seeds ${w.a} and ${w.b} at ${w.distance.toFixed(3)} — ${blocks}`);
}
lines.push('');
lines.push('THE DECISIONS BEHIND IT — spec-level, no rendering');
lines.push(' (high here + low above = the generator decides freely and the render flattens it;');
lines.push(' low here = casting is the bottleneck, and shader work will not fix it)');
lines.push('');
lines.push(` scene-set distance ${bar(spec.sceneSetDistance)} ${spec.sceneSetDistance.toFixed(2)} how differently ${spec.seeds} seeds cast`);
lines.push(` library coverage ${bar(spec.libraryCoverage)} ${pct(spec.libraryCoverage)} of castable scenes ever chosen`);
lines.push(` identical casts ${spec.identicalCasts} seed pairs`);
for (const key of ['director', 'paletteScheme', 'signature', 'grain', 'framing', 'paletteArc', 'anchorScenes']) {
const e = spec[key];
lines.push(` ${key.padEnd(20)} ${bar(e.normalized)} ${String(e.unique).padStart(3)} distinct (entropy ${e.entropy.toFixed(2)} bits)`);
}
lines.push('');
if (library) {
// The map the whole test is drawn on: a seed cannot reach more variety
// than the library holds, so a low separation is only the generator's
// fault once the library is known to hold distinct looks.
const sweep = librarySweep(track, { probes: 3 });
lines.push('THE LIBRARY — all ' + sweep.scenes.length + ' visualizations, every pair, colour not counted');
lines.push('');
lines.push(` median pair distance ${sweep.median.toFixed(3)} ` +
`(seed test ceiling for reference: ${r.ceiling.toFixed(3)})`);
lines.push(` closest 5% under ${sweep.p05.toFixed(3)}`);
lines.push('');
lines.push(` structural twins — every pair inside a group is under ${sweep.twinAt.toFixed(3)}:`);
if (sweep.twins.length === 0) {
lines.push(' no group of three or more; see the closest pairs below');
} else {
for (const group of sweep.twins) lines.push(` ${group.join(' ≈ ')}`);
}
lines.push('');
lines.push(' closest pairs in the library:');
for (const p of sweep.closestPairs) {
lines.push(` ${p.distance.toFixed(3)} ${p.name.padEnd(22)} ≈ ${p.nearest} (${p.family})`);
}
lines.push('');
if (spec.uncast.length) {
lines.push(` never cast in ${spec.seeds} seeds: ${spec.uncast.join(', ')}`);
lines.push('');
}
}
lines.push('SAMPLE LOOKS');
lines.push('');
for (const seed of r.seeds.slice(0, 6)) {
lines.push(` ${describeLook(generateLook(track, { seed }))}`);
}
return { lines, ok, headline };
}
/**
* The SONG variety report.
*
* Deliberately laid out in the order the questions have to be answered. Are two
* songs different at all; is that difference DERIVED from the music or merely
* random; and what does every output have in common regardless — which is the
* one that speaks to "you can tell what made it".
*/
export async function songVarietyReportLines({ songs = 6, probes = 5 } = {}) {
const lines = [];
await new Promise((r) => setTimeout(r, 0));
const r = measureSongVariety({ songs, probes });
const f = measureFingerprint({ songs: Math.min(songs, 5), probes: 3 });
const ok = r.ceilingValid && r.separation >= 0.45 && r.coupling >= 0.3;
const headline = (r.ceilingValid ? `song separation ${r.separation.toFixed(2)}` :
`observed ${r.observed.toFixed(3)} vs floor ${r.floor.toFixed(3)} (no valid ceiling)`) +
` · coupling ${r.coupling.toFixed(2)} (${ok ? 'acceptable' : 'TOO LOW'})`;
lines.push('SONG VARIETY — different songs, each with its own audio-derived seed');
lines.push('');
for (const b of r.bank) lines.push(` ${b.name.padEnd(9)} ${b.kinds.join(' ')}`);
lines.push('');
lines.push(` floor ${r.floor.toFixed(4)} one video against itself, across its own sections`);
lines.push(` observed ${r.observed.toFixed(4)} two songs against each other`);
lines.push(` ceiling ${r.ceiling.toFixed(4)} same pipeline, casts that share no scenes at all`);
lines.push('');
lines.push(r.ceilingValid
? ` separation ${bar(r.separation)} ${r.separation.toFixed(2)}`
: ' separation — not computable: reference below the floor');
lines.push(` coupling ${bar(Math.max(0, r.coupling))} ${r.coupling.toFixed(2)}`);
lines.push(' coupling is how strongly musical distance predicts visual distance.');
lines.push(' near 0 means the look is unrelated to the song — separation without');
lines.push(' it is not variety, it is noise with a different seed per file.');
lines.push('');
lines.push('WHERE THE VARIETY IS');
lines.push('');
for (const [name, b] of Object.entries(r.byBlock)) {
const note = name === 'colour' ? ' (not counted)' : '';
lines.push(` ${name.padEnd(8)} ${bar(b.ratio)} ${pct(b.ratio)}` +
` ${b.between.toFixed(3)} of ${b.ceiling.toFixed(3)}${note}`);
}
lines.push('');
lines.push('CLOSEST SONG PAIRS — two songs that came out as one video');
lines.push('');
for (const p of r.pairs.slice(0, 6)) {
lines.push(` ${p.total.toFixed(3)} ${p.a.padEnd(9)} ≈ ${p.b.padEnd(9)}` +
` (they sound ${p.musical < 0.25 ? 'alike' : p.musical > 0.5 ? 'nothing alike' : 'somewhat alike'}` +
`, musical distance ${p.musical.toFixed(2)})`);
}
lines.push('');
lines.push('THE HOUSE FINGERPRINT — what every output has in common');
lines.push(' (variance across our videos as a fraction of variance across random ones;');
lines.push(' a low number is a constant the generator imposes on everything it makes)');
lines.push('');
for (const t of f.tells) {
lines.push(` ${t.block.padEnd(8)} ${bar(Math.min(1, t.ratio))} ${pct(Math.min(1, t.ratio))}` +
` ${t.frozen}/${t.dims} dimensions effectively frozen`);
}
lines.push('');
return { lines, ok, headline };
}
/**
* The Epic 3 experiment: does a video built from STAGES beat one built from
* comparable legacy scenes?
*
* Same songs, same instrument, same number of scenes available — the only
* difference is whether those scenes draw the song's cast in the song's ink, or
* their own content. Three arms, because two would not distinguish "stages are
* better" from "a small pool is better".
*/
export async function experimentReportLines({ songs = 6, probes = 4, repeats = 3 } = {}) {
const { scenes } = await import('../../scenes/registry.js');
const byName = (n) => scenes.find((m) => m.name === n);
const arms = [
{
label: 'STAGES — draw the song\'s cast and ink',
pool: ['Procession', 'Constellation', 'Soloist', 'Swarm'].map(byName),
},
{
label: 'LEGACY — four comparable element-placing scenes',
pool: ['Floating Geometry', 'Firefly Drift', 'Scale Mosaic', 'Metaballs'].map(byName),
},
{
label: 'FULL — the unrestricted generator, for reference',
pool: null,
},
];
const lines = [];
lines.push('EPIC 3 EXPERIMENT — container vs content');
lines.push('');
lines.push(' Same songs, same instrument, same pool size. The stages own arrangement');
lines.push(' and nothing else: what is on screen comes from the song. The legacy arm');
lines.push(' is four scenes that each invent their own content.');
lines.push('');
lines.push(' Lower floor = a video that looks like itself over its own length.');
lines.push(' Higher observed = two songs that look like different work.');
lines.push('');
// Repeats, with error bars. The first run of this comparison used seven
// songs and put stages 14% ahead; at twelve songs the ordering flipped. A
// difference that changes sign with the sample is a difference that has to
// be reported with its spread or not at all.
const results = [];
for (const arm of arms) {
const runs = [];
for (let k = 0; k < repeats; k++) {
await new Promise((r) => setTimeout(r, 0));
runs.push(measureSongVariety({
songs, probes, pool: arm.pool, seedSalt: k * 7919,
}));
}
results.push({ arm, runs, r: runs[0] });
}
const mean = (a) => a.reduce((x, y) => x + y, 0) / a.length;
const half = (a) => (Math.max(...a) - Math.min(...a)) / 2;
lines.push(` arm floor observed spread (${repeats} runs)`);
lines.push(' ' + '-'.repeat(80));
for (const { arm, runs } of results) {
const spreads = runs.map((x) => x.observed - x.floor);
lines.push(` ${arm.label.padEnd(44)}${mean(runs.map((x) => x.floor)).toFixed(4)}` +
` ${mean(runs.map((x) => x.observed)).toFixed(4)}` +
` ${mean(spreads) >= 0 ? '+' : ''}${mean(spreads).toFixed(4)} ±${half(spreads).toFixed(4)}`);
}
lines.push('');
const gap = mean(results[0].runs.map((x) => x.observed - x.floor))
- mean(results[1].runs.map((x) => x.observed - x.floor));
const noise = Math.max(
half(results[0].runs.map((x) => x.observed - x.floor)),
half(results[1].runs.map((x) => x.observed - x.floor)));
lines.push(` stages minus legacy: ${gap >= 0 ? '+' : ''}${gap.toFixed(4)} against a noise band of ±${noise.toFixed(4)}`);
lines.push(` → ${Math.abs(gap) > noise * 2 ? 'a real difference' : 'INDISTINGUISHABLE at this sample size'}`);
lines.push('');
for (const { arm, runs, r } of results) {
lines.push(` ${arm.label}`);
for (const [name] of Object.entries(r.byBlock)) {
const v = runs.map((x) => x.byBlock[name].between);
lines.push(` ${name.padEnd(8)} between ${mean(v).toFixed(3)} ±${half(v).toFixed(3)}`);
}
const c = runs.map((x) => x.coupling);
lines.push(` coupling ${mean(c).toFixed(2)} ±${half(c).toFixed(2)}`);
lines.push('');
}
const ok = gap > noise * 2;
const headline = `stages ${gap >= 0 ? '+' : ''}${gap.toFixed(4)} vs legacy, noise ±${noise.toFixed(4)} — ` +
(Math.abs(gap) > noise * 2 ? (gap > 0 ? 'the inversion helps' : 'the inversion hurts')
: 'indistinguishable');
return { lines, ok, headline };
}
/**
* How big should a track's casting pool be?
*
* The Epic 3 experiment found the roster size, not the content sharing, was
* carrying most of the improvement — so the number deserves to be measured
* rather than picked. Reported as spread, because both ends of it matter: a
* one-scene pool would score perfectly on the floor and be unwatchable.
*/
export async function poolSweepLines({ songs = 6, probes = 4, sizes = null, repeats = 1 } = {}) {
const list = sizes || [3, 4, 6, 8, 12, 18, 24, 40];
const lines = [];
lines.push('CASTING POOL SWEEP — how many scenes one track may draw on');
lines.push('');
lines.push(' floor one video against itself, across its own sections');
lines.push(' observed two songs against each other');
lines.push(' spread the gap between them — the thing worth maximising');
lines.push('');
lines.push(' pool floor observed spread ratio distinct scenes/video');
lines.push(' ' + '-'.repeat(66));
const { generateLook } = await import('../../look/LookGenerator.js');
const { song } = await import('../../audio/songbank.js');
const track = song('centre').track;
// Repeats exist because the first run of this sweep was pure noise: spread
// was non-monotonic in pool size and peaked at the LARGEST pool, which is
// the opposite of what the Epic 3 arms suggested. Each size draws a
// different random pool per song, so a single run measures which scenes
// happened to come up as much as it measures the size. Repeating with a
// different draw and reporting the range is how to tell those apart.
let best = null;
for (const size of list) {
await new Promise((r) => setTimeout(r, 0));
const runs = [];
for (let k = 0; k < repeats; k++) {
const rk = measureSongVariety({
songs, probes, poolSize: size, seedSalt: k * 7919,
});
runs.push(rk);
}
const r = runs[0];
const spreads = runs.map((x) => x.observed - x.floor);
const spread = spreads.reduce((a, b) => a + b, 0) / spreads.length;
const range = repeats > 1
? ` ±${((Math.max(...spreads) - Math.min(...spreads)) / 2).toFixed(4)}` : '';
// How many distinct scenes a video actually ends up showing, which is
// the number a viewer experiences rather than the pool it was drawn from.
let distinct = 0;
for (let s = 0; s < 4; s++) {
const look = generateLook(track, { seed: 900 + s * 7919, poolSize: size });
distinct += new Set(look.sections.flatMap(
(sec) => sec.variants.flatMap((v) => v.map((l) => l.module.name)))).size / 4;
}
lines.push(` ${String(size).padStart(4)} ${r.floor.toFixed(4)} ${r.observed.toFixed(4)}` +
` ${spread >= 0 ? '+' : ''}${spread.toFixed(4)} ${(r.observed / r.floor).toFixed(3)}` +
` ${distinct.toFixed(1)}${range}`);
if (!best || spread > best.spread) best = { size, spread };
}
lines.push('');
lines.push(` widest spread at pool ${best.size} (+${best.spread.toFixed(4)})`);
lines.push('');
lines.push(' Read the whole column, not the winner. A very small pool wins this');
lines.push(' metric by making every video repetitive, which the metric cannot see');
lines.push(' and a viewer cannot miss — pick the knee, not the peak.');
return { lines, ok: true, headline: `widest spread at pool ${best.size}` };
}
/**
* Identity versus container, measured separately.
*
* The one experiment that says whether Epic 3 can work at all: hold the stage
* fixed and vary only the song's identity, then hold the identity fixed and
* vary only the stage.
*/
export async function decomposeReportLines({ songs = 6, probes = 3 } = {}) {
await new Promise((r) => setTimeout(r, 0));
const d = measureDecomposition({ songs, probes });
const lines = [];
lines.push('IDENTITY vs CONTAINER — where visual difference actually comes from');
lines.push('');
lines.push(` ${d.songs} songs · one fixed track · ${d.stageCount} cast-consuming scenes`);
lines.push('');
lines.push(` identity only ${bar(Math.min(1, d.identityOnly * 5))} ${d.identityOnly.toFixed(4)}`);
lines.push(` one stage (${d.stage}), each song's cast, ink and lattice`);
lines.push('');
lines.push(` container only ${bar(Math.min(1, d.containerOnly * 5))} ${d.containerOnly.toFixed(4)}`);
lines.push(' one identity, four different stages');
lines.push('');
lines.push(` both ${bar(Math.min(1, d.both * 5))} ${d.both.toFixed(4)}`);
lines.push(' what the generator actually does');
lines.push('');
lines.push(` neither ${bar(Math.min(1, d.sameBoth * 5))} ${d.sameBoth.toFixed(4)}`);
lines.push(' the same thing rendered twice — the instrument\'s own noise floor');
lines.push('');
const ratio = d.containerOnly > 1e-6 ? d.identityOnly / d.containerOnly : 0;
lines.push(` identity is worth ${(ratio * 100).toFixed(0)}% of what the container is worth.`);
lines.push('');
if (d.identityOnly < d.sameBoth * 3) {
lines.push(' IDENTITY IS NOT REACHING THE FRAME. Swapping every design decision a song');
lines.push(' makes moves the picture barely more than rendering the same thing twice.');
lines.push(' More registers cannot help until this number moves.');
} else if (ratio < 0.5) {
lines.push(' Identity reaches the frame but the container still dominates. The stages');
lines.push(' express what they are given only weakly — worth fixing the stages before');
lines.push(' adding registers for them to ignore.');
} else {
lines.push(' Identity carries as much as the container does. The inversion is working');
lines.push(' at the frame level, and the place to look next is whether the harness');
lines.push(' aggregates it away.');
}
lines.push('');
return { lines, ok: ratio >= 0.5, headline: `identity ${d.identityOnly.toFixed(4)} vs container ${d.containerOnly.toFixed(4)} (${(ratio * 100).toFixed(0)}%)` };
}

702
flow-state/src/checks/variety/report.js Normal file
View File

@ -0,0 +1,702 @@
// The variety measurement: does changing the seed actually change the video?
//
// A raw distance between two seeds means nothing on its own — 0.14 is not
// interpretable. It is only a number once it sits between two references that
// the same instrument produced:
//
// FLOOR how far one video travels from ITSELF over its own length (drift).
// Two seeds that differ by less than this are, in the only sense that
// matters, the same video shown twice.
// CEILING how far two videos are when the same pipeline is run with the
// design deliberately thrown away — every layer recast at random.
// This is the most variety the library can express, so it is what the
// generator is measured against, not some abstract 1.0.
//
// separation = (between-seed - floor) / (ceiling - floor)
//
// 0 means the seed does nothing a viewer could name. 1 means two seeds are as
// unalike as two randomly assembled videos. The honest target is somewhere well below
// 1 — a generator with a house style SHOULD land under the ceiling — but it has
// to clear the floor by a wide margin, and the per-block breakdown is what says
// where the missing variety went.
//
// Alongside the pixel measurement there is a SPEC measurement, which needs no
// GPU: how much the generator's own decisions differ across seeds. Pixels
// measure the symptom, the spec measures the cause. If spec diversity is high
// and pixel variety is low, the generator is deciding freely and the renderer
// or the post chain is flattening it. If spec diversity is also low, the
// casting is the bottleneck and no amount of shader work will fix it.
import { Show } from '../../Show.js';
import { generateLook } from '../../look/LookGenerator.js';
import { scenes } from '../../scenes/registry.js';
import { defaultValues, sampleValues, canBackground } from '../../params/schema.js';
import { Rng } from '../../engine/rng.js';
import { videoSignature, signatureDistance, STRUCTURAL } from './signature.js';
import { songBank } from '../../audio/songbank.js';
import { hashString } from '../../engine/rng.js';
import { subjectOf, isGround } from '../../look/stack.js';
import { canGround } from '../../scenes/surface.js';
const RENDER = { width: 160, height: 90 };
/** Signature for one seed, rendered through the whole normal pipeline. */
export function signatureForSeed(track, seed, options = {}) {
const { pool = null, poolSize, ...rest } = options;
const show = new Show({ ...RENDER });
try {
show.useTrack(track, generateLook(track, {
seed: seed >>> 0, pool, ...(poolSize ? { poolSize } : {}),
}));
return videoSignature(show, rest);
} finally {
show.dispose();
}
}
/**
* The ceiling reference: a video with the design thrown away.
*
* Same pipeline, same shot planning, same post — but every layer is recast to a
* scene picked uniformly at random and its parameters resampled without regard
* for the section. Two of these agree about nothing, so the distance between
* them is the most this library and this renderer can express. That is the
* honest thing to measure the generator against: not 1.0, which no pipeline
* reaches, and not two default-parameter scenes either — that was the first
* attempt, and it produced videos so internally uneventful that real seeds
* scored ABOVE the supposed ceiling on three blocks out of five.
*/
export function signatureForChaos(track, seed, options = {}) {
const show = new Show({ ...RENDER });
try {
const look = generateLook(track, { seed: seed >>> 0 });
const rng = new Rng((seed * 2246822519) >>> 0);
const pool = scenes.filter(canBackground);
// The ground keeps its JOB when its identity is thrown away. Recasting
// it from the whole library would give the reference videos thin,
// half-black frames no real video can have any more, and a ceiling
// measured on those is a ceiling for a pipeline that does not exist —
// it fell below the floor the first time this ran.
const groundPool = scenes.filter(canGround);
const temperament = look.personality && look.personality.temperament;
for (const section of look.sections) {
for (const variant of section.variants) {
for (const layer of variant) {
layer.module = rng.pick(isGround(layer) ? groundPool : pool);
layer.params = sampleValues(layer.module, rng, section.bias, temperament);
layer.seed = rng.int(0, 0x7fffffff);
}
}
section.layers = section.variants[0];
}
show.useTrack(track, look);
return videoSignature(show, options);
} finally {
show.dispose();
}
}
/**
* Signature for a video forced onto ONE library scene.
*
* `sampled` swaps default parameters for a seeded draw, which is what the
* ceiling wants: defaults are the middle of every range and make a scene look
* tamer than the generator would ever cast it.
*/
export function signatureForScene(track, module, seed, options = {}) {
const { sampled = false, ...rest } = options;
const show = new Show({ ...RENDER });
try {
const look = generateLook(track, { seed: seed >>> 0 });
const prng = new Rng((seed * 40503) >>> 0);
for (const section of look.sections) {
const layers = [{
module,
params: sampled
? sampleValues(module, prng, section.bias,
look.personality && look.personality.temperament)
: defaultValues(module),
seed: seed >>> 0,
blend: 'normal',
opacity: 1,
}];
section.variants = [layers];
section.layers = layers;
for (const shot of section.shots || []) shot.variant = 0;
}
show.useTrack(track, look);
return videoSignature(show, rest);
} finally {
show.dispose();
}
}
/**
* The ceiling: videos with the same STRUCTURE as real ones, cast from pools that
* share no scenes at all.
*
* Getting this reference right took three attempts, and both failures were
* instructive enough to record.
*
* 1. Recast every layer at random. Averaging a dozen random scenes converges
* on the same generic busy image every time, so two "chaos" videos came out
* closer to each other than two real ones.
* 2. One scene per video, no rotation. That collapsed the other way: a video
* that never changes scene has almost no internal variation, so the ceiling
* landed BELOW the floor, which is a within-video quantity.
*
* The reference has to match what it is bounding. These keep the real pipeline —
* rosters, shots, per-section sampling, so a reference video rotates between
* three or four scenes exactly as a real one does — while the pools they draw
* from are disjoint slices of the library. Same complexity, nothing in common.
*/
export function ceilingSignatures(track, { count = 4, probes = 4, seed = 0xbadc0de } = {}) {
const rng = new Rng(seed >>> 0);
const pool = rng.shuffle(scenes.filter(canBackground));
const slice = Math.max(4, Math.floor(pool.length / count));
const out = [];
for (let i = 0; i < count; i++) {
const mine = pool.slice(i * slice, (i + 1) * slice);
if (mine.length < 4) break;
const show = new Show({ ...RENDER });
try {
// The real generator, given a restricted cast. An earlier version
// reached in and reassigned every layer at random instead, and that
// averaged a dozen scenes per video — the more sections a track had,
// the more its references converged on the same generic image, so
// the ceiling fell BELOW the floor on any track with five sections.
show.useTrack(track, generateLook(track, {
seed: (seed + i * 40503) >>> 0,
pool: mine,
}));
out.push(videoSignature(show, { probes }));
} finally {
show.dispose();
}
}
return out;
}
/**
* Every visualization in the library, measured structurally, against every
* other one.
*
* This is the map the seed test is drawn on. A library of sixty names is not a
* library of sixty looks: two scenes built from different maths can land on the
* same image — the same feature scale, the same composition, the same kind of
* movement — and once they do, no amount of casting variety can produce a
* different-looking video by choosing between them.
*
* Note what this catches that the existing per-scene "distinct" gate cannot.
* That one compares raw pixels, so two scenes that are structurally the same
* image in different colours pass it comfortably. Here colour is not counted at
* all, so structural twins have nowhere to hide.
*
* Default parameters throughout: the question is what a scene inherently looks
* like, and sampled parameters would make the answer depend on which roll it
* got.
*/
export function librarySweep(track, { probes = 3, onProgress = null } = {}) {
const pool = scenes.filter(canBackground);
const sigs = [];
for (let i = 0; i < pool.length; i++) {
sigs.push(signatureForScene(track, pool[i], 4242, { probes }));
if (onProgress) onProgress(i + 1, pool.length, pool[i].name);
}
const n = pool.length;
const matrix = Array.from({ length: n }, () => new Float64Array(n));
for (let i = 0; i < n; i++) {
for (let j = i + 1; j < n; j++) {
const d = signatureDistance(sigs[i], sigs[j]).total;
matrix[i][j] = d;
matrix[j][i] = d;
}
}
const all = [];
for (let i = 0; i < n; i++) for (let j = i + 1; j < n; j++) all.push(matrix[i][j]);
all.sort((a, b) => a - b);
const nearest = pool.map((module, i) => {
let best = Infinity, at = -1;
for (let j = 0; j < n; j++) {
if (i !== j && matrix[i][j] < best) { best = matrix[i][j]; at = j; }
}
return { name: module.name, family: module.family, nearest: pool[at].name, distance: best };
});
// COMPLETE-link clustering at the twin threshold: a scene joins a group only
// if it is close to every member, not merely to one of them.
//
// Single link was the first attempt and it lied. Structural distance is
// chainable — A near B, B near C, C near D — so it reported fourteen scenes
// as one look when what actually existed was a chain of overlapping pairs.
// A group here means every pair inside it is a twin, which is a claim worth
// acting on.
const twinAt = all[Math.floor(all.length * 0.02)]; // the closest 2% of pairs
const order = [];
for (let i = 0; i < n; i++) for (let j = i + 1; j < n; j++) order.push([matrix[i][j], i, j]);
order.sort((a, b) => a[0] - b[0]);
const groupOf = new Array(n).fill(-1);
const clusters = [];
for (const [d, i, j] of order) {
if (d > twinAt) break;
const gi = groupOf[i], gj = groupOf[j];
const fits = (member, group) => group.every((k) => matrix[member][k] <= twinAt);
if (gi < 0 && gj < 0) {
groupOf[i] = groupOf[j] = clusters.length;
clusters.push([i, j]);
} else if (gi >= 0 && gj < 0 && fits(j, clusters[gi])) {
clusters[gi].push(j); groupOf[j] = gi;
} else if (gj >= 0 && gi < 0 && fits(i, clusters[gj])) {
clusters[gj].push(i); groupOf[i] = gj;
}
}
const groups = clusters.map((c) => c.map((i) => pool[i].name));
return {
scenes: pool.map((m) => m.name),
matrix,
nearest,
median: all[Math.floor(all.length / 2)],
mean: mean(all),
p05: all[Math.floor(all.length * 0.05)],
twinAt,
twins: groups.filter((g) => g.length > 1).sort((a, b) => b.length - a.length),
twinPairs: order.filter(([d]) => d <= twinAt)
.map(([d, i, j]) => ({ a: pool[i].name, b: pool[j].name, distance: d })),
closestPairs: nearest.slice().sort((a, b) => a.distance - b.distance).slice(0, 8),
};
}
function pairwise(items, fn) {
const out = [];
for (let i = 0; i < items.length; i++) {
for (let j = i + 1; j < items.length; j++) out.push(fn(items[i], items[j], i, j));
}
return out;
}
const mean = (a) => (a.length ? a.reduce((x, y) => x + y, 0) / a.length : 0);
/**
* The full measurement.
*
* @param {FeatureTrack} track one analysed track — held FIXED, so the only
* variable is the seed. Measuring across different audio would confound
* "the generator ignores its seed" with "these songs are alike".
* @param {object} options
* @returns {object} report
*/
export function measureVariety(track, {
seeds = 8, refScenes = 5, probes = 5, seed0 = 0x5eed,
} = {}) {
const seedList = [];
for (let i = 0; i < seeds; i++) seedList.push((seed0 + i * 2654435761) >>> 0);
const sigs = seedList.map((s) => signatureForSeed(track, s, { probes }));
// --- floor: a video against itself, later ---------------------------
const floor = mean(sigs.map((s) => s.drift));
// --- between-seed ---------------------------------------------------
const between = pairwise(sigs, (a, b) => signatureDistance(a, b));
const observed = mean(between.map((d) => d.total));
// --- ceiling: single-scene videos, each on a different scene ---------
const refSigs = ceilingSignatures(track, { count: refScenes, probes });
const ceilingPairs = pairwise(refSigs, (a, b) => signatureDistance(a, b));
const ceiling = mean(ceilingPairs.map((d) => d.total));
// The same statistic on the single-scene references — a video with no story
// in it at all. It is the zero this measurement is read against, rather than
// a theoretical 0: probes are not evenly spaced and a slow scene drifts on
// its own, so an arcless video does not score exactly nothing.
const directionFloor = mean(refSigs.map((s) => s.direction ?? 0));
// If the reference is not above the floor it is not a ceiling, and the
// ratio built on it is meaningless rather than large. Say so instead of
// printing four digits of nonsense.
const valid = ceiling > floor * 1.05;
const separation = valid ? (observed - floor) / (ceiling - floor) : NaN;
// Per block, the same three numbers — this is the diagnosis.
const byBlock = {};
for (const block of [...STRUCTURAL, 'colour']) {
const b = mean(between.map((d) => d.byBlock[block] ?? 0));
const c = mean(ceilingPairs.map((d) => d.byBlock[block] ?? 0));
byBlock[block] = {
between: b,
ceiling: c,
ratio: c > 1e-6 ? b / c : 0,
};
}
// Nearest-neighbour collapse: for each seed, how close is its closest
// sibling? A healthy generator has no seed that another seed shadows. A mean
// can look acceptable while two of eight seeds are visually the same video.
const nearest = sigs.map((_, i) => {
let best = 1;
for (let j = 0; j < sigs.length; j++) {
if (i === j) continue;
best = Math.min(best, signatureDistance(sigs[i], sigs[j]).total);
}
return best;
});
return {
seeds: seedList,
floor,
// Of that floor, how much is a video GOING somewhere rather than merely
// changing. The floor alone cannot tell the two apart, and a video with
// a story raises it on purpose. See variety/signature.js directionOf.
direction: mean(sigs.map((s) => s.direction ?? 0)),
directionFloor,
ceiling,
observed,
separation,
ceilingValid: valid,
identity: ceiling > 1e-6 ? observed / ceiling : 0,
byBlock,
nearest,
worstPair: worstPairOf(seedList, between),
motion: mean(sigs.map((s) => s.motion)),
};
}
function worstPairOf(seedList, between) {
let idx = 0, best = Infinity, k = 0;
for (let i = 0; i < seedList.length; i++) {
for (let j = i + 1; j < seedList.length; j++) {
if (between[k].total < best) { best = between[k].total; idx = k; }
k++;
}
}
k = 0;
for (let i = 0; i < seedList.length; i++) {
for (let j = i + 1; j < seedList.length; j++) {
if (k === idx) return { a: i, b: j, distance: best, byBlock: between[k].byBlock };
k++;
}
}
return null;
}
// --- spec-level diversity -------------------------------------------------
// No GPU, milliseconds to run. This is the one to run first when the score
// drops, because it says whether the generator ever MEANT to make two different
// videos.
function entropy(values) {
const counts = new Map();
for (const v of values) counts.set(v, (counts.get(v) || 0) + 1);
let h = 0;
for (const c of counts.values()) {
const p = c / values.length;
h -= p * Math.log2(p);
}
const max = Math.log2(Math.max(2, counts.size));
return { unique: counts.size, entropy: h, normalized: max > 0 ? h / Math.log2(values.length) : 0 };
}
export function measureSpecDiversity(track, { seeds = 32, seed0 = 0x5eed } = {}) {
const looks = [];
for (let i = 0; i < seeds; i++) {
looks.push(generateLook(track, { seed: (seed0 + i * 2654435761) >>> 0 }));
}
// Overlay-only scenes excluded on both sides of the ratio. They were
// counted in the numerator and not the denominator, which reported 102%
// coverage once the casting pool started reaching them.
const sceneSets = looks.map((l) => [...new Set(
l.sections.flatMap((s) => s.variants.flatMap(
(v) => v.filter((layer) => canBackground(layer.module))
.map((layer) => layer.module.name))),
)].sort());
const usable = scenes.filter(canBackground);
const covered = new Set(sceneSets.flat());
const uncast = usable.filter((m) => !covered.has(m.name)).map((m) => m.name);
// Jaccard distance between the scene SETS of two seeds: the most direct
// possible statement of "did the generator cast a different show".
const jaccard = pairwise(sceneSets, (a, b) => {
const A = new Set(a), B = new Set(b);
let inter = 0;
for (const x of A) if (B.has(x)) inter++;
const union = A.size + B.size - inter;
return union ? 1 - inter / union : 0;
});
return {
seeds,
libraryCoverage: covered.size / usable.length,
uncast,
sceneSetDistance: mean(jaccard),
identicalCasts: pairwise(sceneSets, (a, b) => (a.join('|') === b.join('|') ? 1 : 0))
.reduce((x, y) => x + y, 0),
director: entropy(looks.map((l) => l.director)),
paletteScheme: entropy(looks.map((l) => l.paletteScheme)),
signature: entropy(looks.map((l) => l.personality.signature.join('+'))),
grain: entropy(looks.map((l) => l.grain.mode)),
framing: entropy(looks.map((l) => l.framing.mode)),
paletteArc: entropy(looks.map((l) => l.paletteArc.mode)),
anchorScenes: entropy(looks.map((l) => l.sections.map((s) => subjectOf(s.layers).module.name).join('>'))),
};
}
// --- the SONG variety test -------------------------------------------------
//
// The seed test holds the song fixed and varies the seed, which answers "does
// the generator's randomness do anything". This varies the SONG, which is the
// question that actually matters: two different tracks should not obviously
// come out of the same software.
//
// It needs one thing the seed test does not. Separation alone can be reached by
// a generator that ignores the audio entirely and hashes the file — that would
// score perfectly and be completely wrong, because the video would have nothing
// to do with the music. So coupling is measured alongside it: songs that sound
// alike should look alike, and songs that sound different should look different.
// A high separation with zero coupling is not variety, it is noise.
/** Distance between two tracks as MUSIC, on the statistics the generator reads. */
function musicalDistance(a, b) {
const axes = [
[(t) => t.summary.bpm, 120],
[(t) => t.summary.meanCentroid, 0.7],
[(t) => t.summary.meanFlatness, 0.8],
[(t) => t.summary.dynamicRange, 0.7],
[(t) => t.sections.length, 6],
];
let d = 0;
for (const [of_, span] of axes) d += Math.min(1, Math.abs(of_(a) - of_(b)) / span);
return d / axes.length;
}
/** Spearman rank correlation — monotone association, robust to the scales. */
function spearman(xs, ys) {
const rank = (values) => {
const order = values.map((v, i) => [v, i]).sort((p, q) => p[0] - q[0]);
const r = new Array(values.length);
order.forEach(([, i], k) => { r[i] = k; });
return r;
};
const rx = rank(xs), ry = rank(ys);
const n = xs.length;
let sum = 0;
for (let i = 0; i < n; i++) sum += (rx[i] - ry[i]) ** 2;
return 1 - (6 * sum) / (n * (n * n - 1) || 1);
}
/**
* @param {object} options
* @returns {object} report
*/
export function measureSongVariety({
songs = 6, probes = 5, refScenes = 4, pool = null, poolSize = null, seedSalt = 0,
} = {}) {
const bank = songBank({ count: songs });
// The seed is derived from the audio in the real pipeline, so each song must
// get its own — deriving it from the name is the same relationship without
// needing the samples.
const sigs = bank.map((entry) =>
signatureForSeed(entry.track, (hashString(entry.name) + seedSalt) >>> 0,
{ probes, pool, poolSize }));
const floor = mean(sigs.map((s) => s.drift));
const between = [];
const musical = [];
const visual = [];
for (let i = 0; i < bank.length; i++) {
for (let j = i + 1; j < bank.length; j++) {
const d = signatureDistance(sigs[i], sigs[j]);
between.push({ ...d, a: bank[i].name, b: bank[j].name });
musical.push(musicalDistance(bank[i].track, bank[j].track));
visual.push(d.total);
}
}
const observed = mean(visual);
// Ceiling on the same songs, so it is not a different measurement.
const refSigs = ceilingSignatures(bank[0].track, { count: refScenes, probes });
const ceilingPairs = [];
for (let i = 0; i < refSigs.length; i++) {
for (let j = i + 1; j < refSigs.length; j++) {
ceilingPairs.push(signatureDistance(refSigs[i], refSigs[j]));
}
}
const ceiling = mean(ceilingPairs.map((d) => d.total));
const byBlock = {};
for (const block of [...STRUCTURAL, 'colour']) {
const b = mean(between.map((d) => d.byBlock[block] ?? 0));
const c = mean(ceilingPairs.map((d) => d.byBlock[block] ?? 0));
byBlock[block] = { between: b, ceiling: c, ratio: c > 1e-6 ? b / c : 0 };
}
const nearest = sigs.map((_, i) => {
let best = 1, at = i;
for (let j = 0; j < sigs.length; j++) {
if (i === j) continue;
const d = signatureDistance(sigs[i], sigs[j]).total;
if (d < best) { best = d; at = j; }
}
return { song: bank[i].name, nearest: bank[at].name, distance: best };
});
return {
bank: bank.map((b) => ({ name: b.name, covers: b.covers, kinds: b.track.sections.map((s) => s.kind) })),
floor,
ceiling,
observed,
separation: ceiling > floor * 1.05 ? (observed - floor) / (ceiling - floor) : NaN,
ceilingValid: ceiling > floor * 1.05,
identity: ceiling > 1e-6 ? observed / ceiling : 0,
coupling: spearman(musical, visual),
byBlock,
nearest,
pairs: between.map((d, k) => ({ ...d, musical: musical[k] }))
.sort((x, y) => x.total - y.total),
};
}
/**
* The house fingerprint: which descriptor dimensions never move.
*
* This is the direct measurement of "you can tell it came from the same
* software". A dimension whose variance across real outputs is a small fraction
* of its variance across randomly assembled ones is a constant the generator
* imposes on every video it makes — and constants are exactly what a viewer
* learns to recognise. Reported per block, in the order they most give the game
* away.
*/
export function measureFingerprint({ songs = 6, probes = 4, refScenes = 5 } = {}) {
const bank = songBank({ count: songs });
const ours = bank.map((entry) =>
signatureForSeed(entry.track, hashString(entry.name), { probes }));
const refs = ceilingSignatures(bank[0].track, { count: refScenes, probes, seed: 0x1337 });
// One vector per VIDEO, not per probe. Pooling probes mixes in how much each
// video varies over its own length, which is large for everything and washed
// the answer out to a flat 100% — the measurement said nothing was frozen
// while the separation score said almost everything was.
const flatten = (sigs, block) => sigs.map((s) => {
const rows = s.probes.map((p) => p[block]);
const out = new Array(rows[0].length).fill(0);
for (const r of rows) for (let i = 0; i < r.length; i++) out[i] += r[i] / rows.length;
return out;
});
const variance = (rows) => {
if (!rows.length) return [];
const n = rows[0].length;
const out = new Array(n).fill(0);
for (let d = 0; d < n; d++) {
const col = rows.map((r) => r[d]);
const m = col.reduce((a, b) => a + b, 0) / col.length;
out[d] = col.reduce((a, b) => a + (b - m) ** 2, 0) / col.length;
}
return out;
};
const tells = [];
for (const block of [...STRUCTURAL, 'colour']) {
const vo = variance(flatten(ours, block));
const vr = variance(flatten(refs, block));
const ratios = vo.map((v, d) => (vr[d] > 1e-12 ? v / vr[d] : 1));
const blockRatio = mean(ratios);
tells.push({
block,
ratio: blockRatio,
frozen: ratios.filter((r) => r < 0.15).length,
dims: ratios.length,
});
}
return { tells: tells.sort((a, b) => a.ratio - b.ratio) };
}
// --- where does visual difference actually come from? ----------------------
//
// The identity census settled one question and opened a better one. Across
// twelve songs the identities are genuinely far apart — 0.43 mean distance,
// no near-identical pairs, every fill, lattice, form and scale used — while the
// videos separate by about half of what the reference reaches. So the
// bottleneck is not the identity's range. Either the stages fail to turn
// identity differences into different frames, or the instrument cannot see the
// difference when they do.
//
// Those are opposite problems with opposite fixes, and one experiment separates
// them: hold the container fixed and vary only the identity, then hold the
// identity fixed and vary only the container.
/**
* @returns {{identityOnly:number, containerOnly:number, both:number, sameBoth:number}}
*/
export function measureDecomposition({ songs = 6, probes = 3, stageNames = null } = {}) {
const bank = songBank({ count: songs });
const track = bank[0].track;
// Every scene that actually draws the cast, not a hardcoded list — so this
// number tracks the migration instead of being pinned to the four scenes
// that happened to be written first.
const stages = stageNames
? stageNames.map((n) => scenes.find((m) => m.name === n)).filter(Boolean)
: scenes.filter((m) => (m.consumes || []).includes('cast') && canBackground(m));
// Each song's identity, lifted off its own look so it can be transplanted.
const looks = bank.map((e) => generateLook(e.track, { seed: hashString(e.name) }));
/** One stage, on one fixed track, wearing a given song's identity. */
const render = (stage, look, seed) => {
const show = new Show({ ...RENDER });
try {
const base = generateLook(track, { seed: seed >>> 0, pool: [stage] });
// Transplant the identity AND the signature form it reads from —
// the protagonist's geometry lives in personality.shape.
base.personality = {
...base.personality,
identity: look.personality.identity,
shape: look.personality.shape,
};
show.useTrack(track, base);
return videoSignature(show, { probes });
} finally {
show.dispose();
}
};
const dist = (list) => {
const out = [];
for (let i = 0; i < list.length; i++) {
for (let j = i + 1; j < list.length; j++) out.push(signatureDistance(list[i], list[j]).total);
}
return mean(out);
};
// A: one container, many identities. This is the whole point of Epic 3 —
// if it is near zero, the inversion cannot work no matter how many
// registers get added.
const oneStage = stages[1] || stages[0];
const identityOnly = dist(looks.map((l) => render(oneStage, l, 4242)));
// B: one identity, many containers. The old lever, measured on its own.
const containerOnly = dist(stages.map((st) => render(st, looks[0], 4242)));
// C: both vary, which is what the generator actually does.
const both = dist(looks.map((l, i) => render(stages[i % stages.length], l, 4242 + i)));
// D: nothing varies — the noise floor of the instrument itself.
const sameBoth = dist([
render(oneStage, looks[0], 4242),
render(oneStage, looks[0], 4242),
]);
return {
identityOnly, containerOnly, both, sameBoth,
stage: oneStage.name, songs: bank.length, stageCount: stages.length,
};
}

310
flow-state/src/checks/variety/signature.js Normal file
View File

@ -0,0 +1,310 @@
// A whole video reduced to one signature, and the distance between two of them.
//
// A single frame is not a video. Two seeds could open on identical-looking
// frames and diverge completely by the drop, or — the failure we actually
// suspect — differ frame by frame while following the same arc from the same
// kind of image to the same kind of image. So a signature samples the track at
// several points, keeps the mean (what this video looks like) AND the spread
// (how much it changes over its own length), and carries a motion block.
//
// The spread is not decoration. It gives the metric its floor: if two different
// seeds are no further apart than one seed is from itself five minutes later,
// then seed has stopped being a meaningful input, and that comparison is the
// honest way to say so.
import { frameDescriptor, motionDescriptor } from './descriptors.js';
/** Blocks that count toward the structural score. Colour is measured, not counted. */
export const STRUCTURAL = ['scale', 'orient', 'layout', 'region', 'texture', 'motion'];
export const ALL_BLOCKS = [...STRUCTURAL, 'colour'];
/**
* Per-block weights.
*
* Flat, on purpose. Every weighting we could justify would be a guess about
* which kind of sameness matters most, and the report breaks the score down by
* block anyway — so the diagnosis survives even if the single number is
* weighted wrong.
*/
const WEIGHTS = { scale: 1, orient: 1, layout: 1, region: 1, texture: 1, motion: 1 };
/**
* Chi-square distance, 0..1, for the blocks that are normalised histograms.
*
* Cosine was the first choice and it was wrong here: on all-positive histograms
* every pair scores as similar, and two genuinely unrelated scenes came out at
* 0.05 — a range too compressed to tell "somewhat alike" from "identical".
* Chi-square weights a difference by how small the bins involved are, which is
* what makes a shift of energy from fine detail to coarse blobs read as the
* large change it looks like.
*/
function chiSquare(a, b) {
let sa = 0, sb = 0;
for (let i = 0; i < a.length; i++) { sa += a[i]; sb += b[i]; }
sa = sa || 1e-9; sb = sb || 1e-9;
let d = 0;
for (let i = 0; i < a.length; i++) {
const x = a[i] / sa, y = b[i] / sb;
const den = x + y;
if (den > 1e-12) d += (x - y) ** 2 / den;
}
return Math.max(0, Math.min(1, d / 2));
}
// Per-dimension spans for the texture block, whose entries are not a histogram:
// crossing rates, sparsity fractions, and three correlations in -1..1.
const TEXTURE_SPAN = [1, 0.5, 0.5, 0.2, 2, 2, 2];
function scaledL1(a, b, spans) {
let d = 0;
for (let i = 0; i < a.length; i++) d += Math.min(1, Math.abs(a[i] - b[i]) / spans[i]);
return d / a.length;
}
// Region entries are signed deviations from a frame mean, in three repeating
// kinds, so they need spans rather than a histogram distance.
const REGION_SPAN = [1.2, 1.0, 1.2];
function blockDistance(name, a, b) {
if (name === 'texture') return scaledL1(a, b, TEXTURE_SPAN);
if (name === 'region') {
return scaledL1(a, b, a.map((_, i) => REGION_SPAN[i % 3]));
}
if (name === 'colour') {
// Hue distribution, then saturation and brightness, weighted so a hue
// rotation reads as the large colour change it is.
return 0.7 * chiSquare(a.slice(0, 6), b.slice(0, 6)) +
0.3 * scaledL1(a.slice(6), b.slice(6), [1, 1]);
}
return chiSquare(a, b);
}
/** Distance between two single-frame (or motion) descriptors, per block. */
export function descriptorDistance(a, b) {
const byBlock = {};
for (const block of ALL_BLOCKS) {
if (!a[block] || !b[block]) continue;
byBlock[block] = blockDistance(block, a[block], b[block]);
}
return byBlock;
}
/**
* Probe points across the track, each labelled with WHAT it is.
*
* Taken at section midpoints: a section boundary is where the look is designed
* to change, so sampling across boundaries is what makes the signature describe
* the video rather than one shot of it.
*
* The label is what lets two DIFFERENT songs be compared. They have different
* section counts, so probe 3 of one is not probe 3 of the other and matching by
* index would compare a drop against an outro and call the difference variety.
* Keyed by kind and by which occurrence of that kind it is, the comparison is
* this song's second drop against that song's second drop — the only pairing a
* viewer would accept as fair.
*/
export function probeFrames(show, count = 6) {
const sections = show.look.sections;
const step = Math.max(1, sections.length / count);
const seen = new Map();
const probes = [];
for (let i = 0; i < sections.length && probes.length < count; i += step) {
const section = sections[Math.floor(i)];
if (!section) continue;
const ordinal = seen.get(section.kind) || 0;
seen.set(section.kind, ordinal + 1);
probes.push({
frame: section.startFrame + Math.floor((section.endFrame - section.startFrame) * 0.5),
key: `${section.kind}#${ordinal}`,
kind: section.kind,
});
}
// A short track can run out of sections before it runs out of probe budget.
// Rather than resample the same midpoints, spread extra probes inside the
// sections it does have — a two-section song still has a beginning, a middle
// and an end worth measuring.
for (let pass = 1; probes.length < count && pass < 4; pass++) {
for (const section of sections) {
if (probes.length >= count) break;
const span = section.endFrame - section.startFrame;
const at = section.startFrame + Math.floor(span * (pass / (pass + 2)));
probes.push({ frame: at, key: `${section.kind}@${pass}`, kind: section.kind });
}
}
return probes.slice(0, count);
}
/**
* Render a loaded Show and reduce it to a signature.
*
* `warmup` frames are rendered before each probe so feedback and stateful
* layers are converged — an unwarmed probe measures the trail of a black frame,
* which is a structure all seeds share and would flatten the metric on its own.
*
* @returns {{blocks: object, probes: object[], drift: number, motion: number}}
*/
export function videoSignature(show, { probes = 6, gap = 5, warmup = 20 } = {}) {
const width = show.engine.width ?? show.engine.renderer.width;
const height = show.engine.height ?? show.engine.renderer.height;
const frames = probeFrames(show, probes);
const perProbe = [];
for (const probe of frames) {
const frame = probe.frame;
show.engine.compositor.reset();
const start = Math.max(0, frame - warmup);
for (let f = start; f < frame; f++) show.renderFrame(f);
const a = Uint8Array.from(show.readPixels(show.renderFrame(frame)));
const b = Uint8Array.from(show.readPixels(show.renderFrame(frame + gap)));
const still = frameDescriptor(a, width, height);
const moved = motionDescriptor(a, b, width, height);
perProbe.push({
...still,
motion: moved.scale.concat(moved.layout),
energy: moved.energy,
key: probe.key,
kind: probe.kind,
});
}
// Self-distance: how far this video travels from itself over its own length.
//
// Note what this number cannot tell you, and what `direction` below is for:
// it is the same whether the video went somewhere or merely kept changing.
let drift = 0, pairs = 0;
const gaps = [];
const dists = [];
for (let i = 0; i < perProbe.length; i++) {
for (let j = i + 1; j < perProbe.length; j++) {
const d = blockMean(descriptorDistance(perProbe[i], perProbe[j]));
drift += d;
pairs++;
gaps.push(Math.abs(frames[j].frame - frames[i].frame));
dists.push(d);
}
}
return {
frames,
probes: perProbe,
drift: pairs ? drift / pairs : 0,
// Does the video's self-distance grow with TIME? See directionOf.
direction: directionOf(gaps, dists),
motion: perProbe.reduce((a, p) => a + p.energy, 0) / perProbe.length,
};
}
/**
* DIRECTION: whether a video is travelling or merely wandering.
*
* `drift` measures how far a video gets from itself and cannot distinguish the
* two, which matters because they are opposite outcomes. A generator that
* shuffles unrelated images scores exactly like one that tells a story, and the
* story is the one anyone wants — so a video with a narrative arc will RAISE the
* floor the seed-variety test wants low, and without this statistic that reads
* as a regression.
*
* The split is rank correlation between how far apart two probes are in TIME and
* how far apart they are structurally. A video with an arc is most unlike itself
* at its two ends: distance grows with separation, ρ approaches 1. A video that
* rotates through a roster is as unlike itself at ten seconds as at four
* minutes: ρ sits at 0. Both can have identical drift.
*
* Spearman rather than Pearson: only the ORDER is meaningful. Nothing here
* claims the arc is linear, and it should not be — see look/Story.js on why the
* curves are staged.
*/
export function directionOf(gaps, dists) {
const n = gaps.length;
if (n < 3) return 0;
const rank = (xs) => {
const order = xs.map((v, i) => [v, i]).sort((a, b) => a[0] - b[0]);
const r = new Array(n);
for (let i = 0; i < n;) {
let j = i;
while (j + 1 < n && order[j + 1][0] === order[i][0]) j++;
const tied = (i + j) / 2 + 1; // mean rank across a tie group
for (let k = i; k <= j; k++) r[order[k][1]] = tied;
i = j + 1;
}
return r;
};
const a = rank(gaps);
const b = rank(dists);
const mean = (xs) => xs.reduce((s, x) => s + x, 0) / xs.length;
const ma = mean(a), mb = mean(b);
let num = 0, da = 0, db = 0;
for (let i = 0; i < n; i++) {
num += (a[i] - ma) * (b[i] - mb);
da += (a[i] - ma) ** 2;
db += (b[i] - mb) ** 2;
}
return da > 0 && db > 0 ? num / Math.sqrt(da * db) : 0;
}
function blockMean(byBlock) {
let sum = 0, weight = 0;
for (const block of STRUCTURAL) {
if (byBlock[block] === undefined) continue;
sum += byBlock[block] * WEIGHTS[block];
weight += WEIGHTS[block];
}
return weight ? sum / weight : 0;
}
/**
* Structural distance between two video signatures, 0..1.
*
* MATCHED probes, not averaged descriptors. The track is held fixed while the
* seed varies, so probe i is the same moment of the same song in both videos,
* and comparing them is the closest thing to sitting two renders side by side.
*
* The first version of this averaged each video's probes into one descriptor
* and compared those. It made the score meaningless: averaging six moments
* washes out exactly the structure being measured, so two seeds scored as
* closer to each other than one seed scored to ITSELF five minutes later — the
* floor came out above the ceiling. Matched probes put the between-seed
* distance and the within-video drift on the same scale, which is the only
* reason the ratio between them means anything.
*
* `byBlock` is the whole point of the return value — a bad total is only
* actionable once you know which block collapsed.
*/
export function signatureDistance(a, b) {
// Pair probes by their LABEL, not their position: drop#0 against drop#0.
// Two seeds of one song produce the same labels in the same order, so this
// is identical to index matching there — it only starts mattering when the
// songs differ, which is exactly when index matching would compare a drop
// against an outro and score the mismatch as variety.
const byKey = new Map(b.probes.map((p) => [p.key, p]));
const pairs = [];
for (const probe of a.probes) {
const other = byKey.get(probe.key);
if (other) pairs.push([probe, other]);
}
// Too few shared labels to average over. A two-section ambient track shares
// only its outro with a five-section club track, and a distance computed
// from one probe pair is not comparable to one computed from five — which
// silently made the comparison depend on how alike the two ARRANGEMENTS
// were, the very thing being measured. Below three matches, fall back to
// position so every pair is averaged over the same number of probes.
if (pairs.length < 3) {
pairs.length = 0;
const n = Math.min(a.probes.length, b.probes.length);
for (let i = 0; i < n; i++) pairs.push([a.probes[i], b.probes[i]]);
}
const sums = {};
for (const [x, y] of pairs) {
const d = descriptorDistance(x, y);
for (const [block, v] of Object.entries(d)) {
sums[block] = (sums[block] || 0) + v / pairs.length;
}
}
return { total: blockMean(sums), byBlock: sums, matched: pairs.length };
}
export { blockMean };

39
flow-state/src/engine/Layer.js
View File

@ -1,5 +1,8 @@
import * as THREE from 'three'; import * as THREE from 'three';
import { VERTEX_SHADER, buildFragmentShader, AUDIO_UNIFORMS, SIGNATURE_UNIFORMS } from './shader-contract.js'; import {
VERTEX_SHADER, buildFragmentShader, AUDIO_UNIFORMS,
SIGNATURE_UNIFORMS, IDENTITY_UNIFORMS, IDENTITY_ARRAY_UNIFORMS,
} from './shader-contract.js';
import { signatureUniforms, NEUTRAL_UNIFORMS } from '../look/Personality.js'; import { signatureUniforms, NEUTRAL_UNIFORMS } from '../look/Personality.js';
import { clampValue } from '../params/schema.js'; import { clampValue } from '../params/schema.js';
@ -40,10 +43,17 @@ export function buildShaderUniforms(module, baseParams, seed) {
u_hasPrev: { value: 0 }, u_hasPrev: { value: 0 },
}; };
for (const name of AUDIO_UNIFORMS) uniforms[name] = { value: 0 }; for (const name of AUDIO_UNIFORMS) uniforms[name] = { value: 0 };
for (const [name, type] of Object.entries(SIGNATURE_UNIFORMS)) { for (const [name, type] of Object.entries({ ...SIGNATURE_UNIFORMS, ...IDENTITY_UNIFORMS })) {
const v = NEUTRAL_UNIFORMS[name]; const v = NEUTRAL_UNIFORMS[name];
uniforms[name] = { value: type === 'vec2' ? new THREE.Vector2(v[0], v[1]) : v }; uniforms[name] = { value: type === 'vec2' ? new THREE.Vector2(v[0], v[1]) : v };
} }
// The assembly rides as an array of rows — see IDENTITY_ARRAY_UNIFORMS. The
// vectors are allocated once and written in place per frame, like u_colors.
for (const [name, def] of Object.entries(IDENTITY_ARRAY_UNIFORMS)) {
uniforms[name] = {
value: Array.from({ length: def.length }, () => new THREE.Vector4()),
};
}
for (const [name, def] of Object.entries(module.params || {})) { for (const [name, def] of Object.entries(module.params || {})) {
if (!def.uniform || def.type === 'palette') continue; if (!def.uniform || def.type === 'palette') continue;
@ -93,11 +103,18 @@ export function setFrameUniforms(layer, renderer, target, ctx) {
} }
const signature = signatureUniforms(layer.personality, layer.module); const signature = signatureUniforms(layer.personality, layer.module);
for (const [name, type] of Object.entries(SIGNATURE_UNIFORMS)) { for (const [name, type] of Object.entries({ ...SIGNATURE_UNIFORMS, ...IDENTITY_UNIFORMS })) {
const v = signature[name]; const v = signature[name];
if (type === 'vec2') u[name].value.set(v[0], v[1]); if (type === 'vec2') u[name].value.set(v[0], v[1]);
else u[name].value = v; else u[name].value = v;
} }
for (const [name, def] of Object.entries(IDENTITY_ARRAY_UNIFORMS)) {
const rows = signature[name] || [];
for (let i = 0; i < def.length; i++) {
const r = rows[i];
u[name].value[i].set(r ? r[0] : 0, r ? r[1] : 0, r ? r[2] : 0, r ? r[3] : 0);
}
}
// Framing is per shot and wins over the personality's neutral defaults — // Framing is per shot and wins over the personality's neutral defaults —
// it is the operator's hand on a shot that is already set up, not a trait // it is the operator's hand on a shot that is already set up, not a trait
@ -145,9 +162,13 @@ export class Layer {
} }
/** /**
* This shot's FRAMING — a per-shot scale/recentre pushed by the arc driver, * This shot's FRAMING — a per-shot scale/recentre pushed by the arc driver.
* applied inside sigCamera. A layer that is never framed renders at the *
* neutral (full-frame) scale, so nothing predating framing changes. * A fragment layer has it applied for it, in the shader epilogue, to the
* coordinate every scene is handed. A 3D layer receives it in `update` and
* honours it as a camera move, because only it knows what its camera means.
* A layer that is never framed renders at the neutral full-frame scale, so
* nothing predating framing changes.
*/ */
setFraming(framing) { setFraming(framing) {
this.framing = framing || null; this.framing = framing || null;
@ -273,6 +294,12 @@ export class SceneLayer extends Layer {
params: resolved, params: resolved,
palette: this.palette, palette: this.palette,
personality: this.personality, personality: this.personality,
// A 3D layer has a literal camera, so framing reaches it as a
// camera move rather than as a coordinate transform. Fragment
// scenes get it applied for them in the shader epilogue; this one
// has to honour it itself, because only it knows what its camera
// means. See look/framing.js.
framing: this.framing,
opacity: this.opacity, opacity: this.opacity,
THREE, THREE,
}); });

52
flow-state/src/engine/passes.js
View File

@ -30,6 +30,23 @@ void main() {
else if (u_mode == 3) result = mix(base.rgb, base.rgb * src.rgb, a); // multiply else if (u_mode == 3) result = mix(base.rgb, base.rgb * src.rgb, a); // multiply
else if (u_mode == 4) result = mix(base.rgb, blendOverlay(base.rgb, src.rgb), a); else if (u_mode == 4) result = mix(base.rgb, blendOverlay(base.rgb, src.rgb), a);
else if (u_mode == 5) result = mix(base.rgb, blendSoftLight(base.rgb, src.rgb), a); else if (u_mode == 5) result = mix(base.rgb, blendSoftLight(base.rgb, src.rgb), a);
// LUMAKEY — the source's own brightness is its alpha.
//
// For a layer that is a PICTURE with black around it, which is what a
// composable scene is: it replaces the base where it paints and leaves it
// where it does not. Screen was doing this job and screen is a lightening
// operator — over a filled ground it drives everything toward white, which
// measured as a median 24% of the frame clipped and whole sections at
// 100%. This keeps the shot's own colour instead of adding it to the bed's.
//
// The key is smoothstepped rather than raw luma so a dark-but-present shot
// does not dissolve into the ground, and gamma-ish weighted to keep thin
// bright lines opaque.
else if (u_mode == 6) {
float key = dot(src.rgb, vec3(0.2126, 0.7152, 0.0722));
key = smoothstep(0.02, 0.32, key);
result = mix(base.rgb, src.rgb, key * a);
}
else result = mix(base.rgb, src.rgb, a); // normal else result = mix(base.rgb, src.rgb, a); // normal
gl_FragColor = vec4(result, max(base.a, a)); gl_FragColor = vec4(result, max(base.a, a));
@ -65,7 +82,26 @@ void main() {
vec3 hist = texture2D(u_history, clamp(warped, 0.0, 1.0)).rgb; vec3 hist = texture2D(u_history, clamp(warped, 0.0, 1.0)).rgb;
// Decay strictly below 1 keeps the loop convergent; the 10k-frame stability // Decay strictly below 1 keeps the loop convergent; the 10k-frame stability
// check in tools/ verifies it neither saturates to white nor dies to black. // check in tools/ verifies it neither saturates to white nor dies to black.
vec3 outC = cur + hist * u_decay * u_amount; //
// NORMALISED, which it was not. cur + hist * decay * amount is an
// accumulator: a static image settles at 1/(1 - decay*amount) times its own
// brightness, which is 2.3x at the settings the generator hands out. That
// was survivable while a frame was a few bright things on black and stopped
// being survivable the moment every section stood on a filled ground —
// measured, entire sections rendered as pure white, and turning feedback off
// took the same frame from 100% blown to 34% mean luminance.
//
// Dividing by the gain keeps the trail — moving content still smears, which
// is the whole point — instead of stacking exposures.
//
// PARTIALLY, at 0.6, rather than all the way. Full normalisation is the
// mathematically tidy answer and it takes the lift out with the blowout:
// measured over 74 sections, the median frame went from 95% painted to 71%
// and fourteen fell under the black-frame floor. Feedback contributing SOME
// brightness is part of what the looks were built around. At 0.6 a still
// frame settles about 1.2x its drawn brightness instead of 2.3x.
float gain = u_decay * u_amount;
vec3 outC = (cur + hist * gain) / (1.0 + gain * 0.6);
gl_FragColor = vec4(min(outC, vec3(4.0)), 1.0); gl_FragColor = vec4(min(outC, vec3(4.0)), 1.0);
} }
`; `;
@ -155,6 +191,19 @@ void main() {
float v = 1.0 - u_vignette * dot(dir, dir) * 2.0; float v = 1.0 - u_vignette * dot(dir, dir) * 2.0;
col *= clamp(v, 0.0, 1.0); col *= clamp(v, 0.0, 1.0);
// HIGHLIGHT SHOULDER — the top end rolls off instead of clipping.
//
// Everything under the knee is untouched, so the image keeps its contrast;
// above it, values compress toward but never reach 1. Without this, bloom
// plus a filled ground plus a bright palette clips large areas to pure
// white — measured at 38% of a drop's frame — and clipped white is not
// bright, it is missing: every difference inside it is gone.
//
// Cheap, per channel, and deliberately not a full filmic curve. The job is
// to stop the frame flattening out at the top, not to grade it.
vec3 over = max(col - 0.75, vec3(0.0));
col = min(col, vec3(0.75)) + over / (1.0 + over * 4.0);
// Deterministic grain: keyed on frame index, never on a random source. // Deterministic grain: keyed on frame index, never on a random source.
// //
// Cell size and refresh rate are separate on purpose. Fine-and-boiling is // Cell size and refresh rate are separate on purpose. Fine-and-boiling is
@ -204,4 +253,5 @@ void main() { gl_FragColor = texture2D(u_tex, vUv); }
export const BLEND_MODE_IDS = { export const BLEND_MODE_IDS = {
normal: 0, add: 1, screen: 2, multiply: 3, overlay: 4, softlight: 5, normal: 0, add: 1, screen: 2, multiply: 3, overlay: 4, softlight: 5,
lumakey: 6,
}; };

715
flow-state/src/engine/shader-contract.js
View File

@ -89,6 +89,77 @@ export const SIGNATURE_UNIFORMS = {
u_sigFrameShift: 'vec2', // recentre, in scene units u_sigFrameShift: 'vec2', // recentre, in scene units
}; };
/**
* The song's CAST and INK — Epic 3's content and style artifacts.
*
* These differ from the signature uniforms above in kind, not degree. A
* signature uniform is a modifier on an image the shader already had, which is
* why a scene is free to ignore one. A cast uniform IS the image: a stage that
* ignores it has nothing to draw. See look/Identity.js.
*/
export const IDENTITY_UNIFORMS = {
u_castSides: 'float', // protagonist: 0 = round, else polygon sides
u_castRound: 'float',
u_castElong: 'float',
u_castTilt: 'float',
u_castNotchN: 'float', // notches cut into the boundary, 0 = none
u_castNotchD: 'float',
u_castHollow: 'float', // >0 makes it an annulus — a form with a hole
u_chorusSides: 'float', // the second member: a relative, not a stranger
u_chorusRound: 'float',
u_chorusElong: 'float',
u_chorusTilt: 'float',
u_chorusNotchN: 'float',
u_chorusNotchD: 'float',
u_chorusHollow: 'float',
u_inkWeight: 'float', // stroke width
u_inkEdge: 'float', // 0 = soft/airbrushed, 1 = hard vector
u_inkFill: 'float', // index into Identity.FILLS
u_inkHatchAngle: 'float',
u_inkHatchScale: 'float',
u_inkOutline: 'float', // 0..1 outline strength on top of the fill
u_inkPosterize: 'float', // 0 = off, else levels
u_latKind: 'float', // index into Identity.LATTICES
u_latJitter: 'float', // how far off the lattice things sit
u_latSpread: 'float', // how much of the frame it occupies
u_latScaleSpread: 'float', // 0 = all one size, 1 = a few large, many small
u_latScaleBias: 'float', // + puts the large ones in the middle
u_latScale: 'float', // the song's element size, ~0.1 tiny .. ~0.9 huge
u_focusN: 'float', // how many focal points, 0 = none
u_focusR: 'float', // their reach, in scene units
u_focusPull: 'float', // + draws the field in, - opens a void
u_impact: 'float', // index into Identity.IMPACTS
u_formCount: 'float', // parts in the assembly, 0 = fall back to the profile
u_formSym: 'float', // index into Identity.SYMMETRIES
u_formSymN: 'float', // its fold/repeat count
u_formBlend: 'float', // how far a `blend` op melts two parts together
u_formDepth: 'float', // how deep the solid is relative to how wide
// The chorus solid: the same parts, fewer of them, squashed differently.
u_formChorusN: 'float',
u_formChorusSym: 'float',
u_formChorusSymN: 'float',
u_formChorusFlat: 'float', // its height against the protagonist's
u_formChorusThin: 'float', // its depth against the protagonist's
};
/**
* The assembly, as fixed-width rows. See look/Identity.js formPartRows.
*
* Separate from the scalars above because it is an ARRAY, and three.js needs an
* array of Vector4 rather than a number — the one place the uniform plumbing
* has to know the difference. Sized for MAX_FORM_PARTS × 3 rows and read only
* through castSDF3, so a scene never touches it directly.
*/
export const IDENTITY_ARRAY_UNIFORMS = {
u_formPart: { type: 'vec4', length: 18 },
};
export const FRAME_UNIFORMS = [ export const FRAME_UNIFORMS = [
'u_time', 'u_frame', 'u_progress', 'u_seed', 'u_time', 'u_frame', 'u_progress', 'u_seed',
'u_resolution', 'u_aspect', 'u_pixelScale', 'u_opacity', 'u_resolution', 'u_aspect', 'u_pixelScale', 'u_opacity',
@ -113,6 +184,10 @@ ${AUDIO_UNIFORMS.map((u) => `uniform float ${u};`).join('\n')}
${Object.entries(SIGNATURE_UNIFORMS).map(([u, t]) => `uniform ${t} ${u};`).join('\n')} ${Object.entries(SIGNATURE_UNIFORMS).map(([u, t]) => `uniform ${t} ${u};`).join('\n')}
${Object.entries(IDENTITY_UNIFORMS).map(([u, t]) => `uniform ${t} ${u};`).join('\n')}
${Object.entries(IDENTITY_ARRAY_UNIFORMS).map(([u, d]) => `uniform ${d.type} ${u}[${d.length}];`).join('\n')}
uniform sampler2D u_prev; uniform sampler2D u_prev;
uniform int u_hasPrev; uniform int u_hasPrev;
@ -224,10 +299,6 @@ float sigForm(vec2 p, vec2 centre, float size) {
*/ */
vec2 sigCamera(vec2 p) { vec2 sigCamera(vec2 p) {
float t = u_time; float t = u_time;
// Framing first: everything below is the operator's hand on a shot that has
// already been set up, so it composes on top of the framing rather than
// fighting it.
p = p / max(u_sigFrameScale, 0.05) + u_sigFrameShift;
p = rot(u_sigSpin * t) * p; p = rot(u_sigSpin * t) * p;
p *= 1.0 - u_sigBreathe * sin(u_barPhase * 6.28318530718); p *= 1.0 - u_sigBreathe * sin(u_barPhase * 6.28318530718);
p += vec2(sin(t * u_sigSwayRate), cos(t * u_sigSwayRate * 0.83)) * u_sigSway; p += vec2(sin(t * u_sigSwayRate), cos(t * u_sigSwayRate * 0.83)) * u_sigSway;
@ -272,6 +343,317 @@ vec3 sigAir(vec3 col, vec2 p, float distance01) {
return col; return col;
} }
// --- the cast --------------------------------------------------------------
// The song's own forms. A stage that places discrete elements places THESE, and
// that is what makes two stages in one video look like one video — and two
// videos of different songs look like different work.
/** Signed distance to a cast member, radius ~1 at size 1. */
float castSDF(vec2 q, float sides, float rnd, float elong, float tilt,
float notchN, float notchD, float hollow) {
q = rot(tilt) * q;
q.x /= max(elong, 0.05);
float r = length(q);
float a = atan(q.y, q.x);
float d;
if (sides < 2.5) {
d = r - 1.0;
} else {
float seg = 6.28318530718 / sides;
float folded = cos(mod(a + seg * 0.5, seg) - seg * 0.5);
float poly = r * folded - cos(seg * 0.5);
d = mix(poly, r - 1.0, clamp(rnd, 0.0, 1.0));
}
// Notches scallop the boundary. Approximate as a radial perturbation — it
// is not a true distance any more, but every use here is a thresholded mask
// and the error is far below a pixel at the sizes these are drawn.
if (notchN > 0.5) d += notchD * cos(notchN * a);
// A hole through the middle. Cheap, and the single most recognisable thing
// a generated form can have.
if (hollow > 0.001) d = abs(d) - hollow * 0.35;
return d;
}
/** The protagonist, centred, radius ~1. */
float castMain(vec2 q) {
return castSDF(q, u_castSides, u_castRound, u_castElong, u_castTilt,
u_castNotchN, u_castNotchD, u_castHollow);
}
/** The chorus member — many of these, small. */
float castChorus(vec2 q) {
return castSDF(q, u_chorusSides, u_chorusRound, u_chorusElong, u_chorusTilt,
u_chorusNotchN, u_chorusNotchD, u_chorusHollow);
}
// --- the staging -----------------------------------------------------------
// Where things go. Shared, so two stages in one video agree about composition —
// and so the SIZE HIERARCHY is a decision the song makes once rather than one
// each stage makes for itself. The first four stages all placed similarly-sized
// elements, which left feature scale out of the measurement entirely.
/** Node i of n on the song's lattice: xy position, z scale multiplier. */
vec3 stageNode(float i, float n) {
vec2 h = hash22(vec2(i * 1.37 + 3.1, i * 0.71 + 7.7));
float total = max(n, 1.0);
vec2 pos;
if (u_latKind < 0.5) { // grid
float cols = max(1.0, floor(sqrt(total) + 0.5));
float rows = max(1.0, ceil(total / cols));
pos = vec2((mod(i, cols) / max(cols - 1.0, 1.0) - 0.5) * 2.0,
(floor(i / cols) / max(rows - 1.0, 1.0) - 0.5) * 2.0);
} else if (u_latKind < 1.5) { // radial rings
float rings = max(1.0, floor(sqrt(total * 0.5) + 0.5));
float ring = mod(i, rings) + 1.0;
float a = (i / total) * 6.28318530718 * 3.0;
pos = vec2(cos(a), sin(a)) * (ring / rings);
} else if (u_latKind < 2.5) { // spiral, golden angle
float a = i * 2.39996323;
pos = vec2(cos(a), sin(a)) * sqrt(i / total);
} else if (u_latKind < 3.5) { // scatter
pos = (h - 0.5) * 2.0;
} else { // strata
float rows = max(1.0, floor(total / 4.0 + 0.5));
pos = vec2((h.x - 0.5) * 2.0,
(mod(i, rows) / max(rows - 1.0, 1.0) - 0.5) * 2.0);
}
pos += (h - 0.5) * u_latJitter;
pos *= u_latSpread;
// Sits on the same ground every other scene in the track sits on.
pos.y += sigHorizonY() * 0.3;
// A power law when the song wants a hierarchy, near-uniform when it does
// not. Biased toward the middle or the edges.
float u = max(hash11(i * 7.13 + 1.7), 0.001);
float size = mix(1.0, pow(u, 1.0 + u_latScaleSpread * 2.5) * 2.4, u_latScaleSpread);
size *= 1.0 + u_latScaleBias * (0.5 - length(pos) * 0.5);
// The song's own element size, relative to the neutral 0.35. A stage
// multiplies its own size param by this rather than choosing outright, so
// one song is made of a few huge forms and another of many small ones.
size *= u_latScale / 0.35;
return vec3(pos, max(size, 0.05));
}
/** The song's element size as a multiplier a stage applies to its own size. */
float stageScale() { return u_latScale / 0.35; }
// --- the focus -------------------------------------------------------------
// Somewhere for a full-frame field to be ABOUT.
//
// An edge-to-edge texture has no composition to vary: measured, two songs of
// Voronoi Shatter differ by 0.004 in the layout block, which is nothing, and
// that is honest rather than a metric failure — the cells change and the
// arrangement does not. These give the field one to three points to organise
// itself around, drawn from the same lattice everything else is placed on, so a
// scene using them is composing in the song's terms rather than inventing a
// centre of its own.
/** Where focal point i sits, in scene units. */
vec2 focusAt(float i) {
return stageNode(i * 3.0 + 1.0, max(u_focusN, 1.0) * 3.0).xy * 0.8;
}
/**
* Influence at p: 1 at a focal point, falling to 0 at its reach.
* Zero everywhere when the song asked for no focus.
*/
float focusField(vec2 p) {
if (u_focusN < 0.5) return 0.0;
float best = 0.0;
for (int i = 0; i < 3; i++) {
if (float(i) >= u_focusN) break;
float d = length(p - focusAt(float(i)));
best = max(best, smoothstep(u_focusR, u_focusR * 0.15, d));
}
return best;
}
/**
* Warp a coordinate toward the focus, or away from it.
*
* The amount argument is the scene's own appetite for it. A field applies this to the
* coordinate it tiles in, and the tiling densifies or opens up around the point
* without the scene needing to know where the point is or why.
*/
vec2 focusWarp(vec2 p, float amount) {
if (u_focusN < 0.5 || amount <= 0.0) return p;
vec2 q = p;
for (int i = 0; i < 3; i++) {
if (float(i) >= u_focusN) break;
vec2 d = p - focusAt(float(i));
float r = length(d);
float w = smoothstep(u_focusR, 0.0, r);
q += normalize(d + 1e-5) * w * u_focusPull * amount * u_focusR * 0.45;
}
return q;
}
// --- the ink ---------------------------------------------------------------
// How the cast is drawn. Changes every pixel of every stage at once, and does
// it structurally rather than chromatically — which is the point, since colour
// was already the only register doing any work.
// The ink and the STYLE TRAIT are one decision applied at two levels, so both
// feed the same two numbers.
//
// They were separate until surface treatment moved out of the scenes: sigGrain
// went to the post chain and sigEdge became inkStroke, which between them left
// the style trait with almost nothing to express — measured, a style+shape
// signature had two eligible scenes left in the whole library. The trait is not
// obsolete, it is now carried by the ink, and a scene that draws in the song's
// hand is honouring the track's line weight by construction.
float inkWeight() { return 0.004 + mix(u_inkWeight, u_sigLine, 0.4) * 0.055; }
float inkSoft() { return mix(0.03, 0.0015, clamp(u_inkEdge * 0.6 + (1.0 - u_sigSoft) * 0.4, 0.0, 1.0)); }
/** The fill treatment as a 0..1 coverage pattern. 1 everywhere when flat. */
float inkPattern(vec2 uv) {
int mode = int(u_inkFill + 0.5);
if (mode == 2) { // hatch
vec2 h = rot(u_inkHatchAngle) * uv * u_inkHatchScale;
return smoothstep(0.3, 0.7, 0.5 + 0.5 * sin(h.y));
}
if (mode == 3) { // stipple
return step(0.42, hash12(floor(uv * u_inkHatchScale * 2.0)));
}
if (mode == 4) { // halftone
vec2 g = fract(uv * u_inkHatchScale * 0.25) - 0.5;
return smoothstep(0.38, 0.28, length(g));
}
return 1.0;
}
/**
* Ink coverage for a signed distance: the fill in the track's treatment, plus
* its outline. The 'hollow' fill treatment draws the outline only.
*/
float inkMask(float d, vec2 uv) {
float soft = inkSoft();
int mode = int(u_inkFill + 0.5);
float fillA = smoothstep(soft, -soft, d) * inkPattern(uv);
if (mode == 5) fillA = 0.0;
float w = inkWeight();
float strength = (mode == 5) ? 1.0 : u_inkOutline;
float line = smoothstep(w + soft, w - soft, abs(d)) * strength;
return clamp(max(fillA, line), 0.0, 1.0);
}
/**
* Outline only, in the song's hand. The drop-in replacement for sigEdge.
*
* sigEdge draws a line at the track's line weight; this draws it at the
* identity's, which is the same idea one layer up. Kept separate from inkMask
* because a scene that only ever wanted an edge should not suddenly acquire a
* fill when it migrates.
*/
float inkStroke(float d) {
float w = inkWeight();
float soft = inkSoft();
return smoothstep(w + soft, w - soft, abs(d));
}
/**
* The protagonist as a filled, inked mask at a point. Replaces sigForm.
*
* Same signature as the thing it supersedes so the substitution is mechanical
* across the library — see MIGRATION.md. uv is recomputed here rather than
* passed, so the call site does not have to change shape.
*/
float castForm(vec2 p, vec2 centre, float size) {
float s = max(size, 1e-3);
vec2 uv = vec2(p.x / u_aspect, p.y) * 0.5 + 0.5;
return inkMask(castMain((p - centre) / s) * s, uv);
}
/** The track's value structure. Off unless the identity asked for it. */
vec3 inkValue(vec3 col) {
if (u_inkPosterize < 1.5) return col;
float n = u_inkPosterize;
return floor(col * n + 0.5) / n;
}
// The subject lives below the ink because subjectEdge draws with inkStroke,
// and GLSL has no forward declarations — placed above, the whole preamble
// failed to compile and every scene using it rendered black.
/**
* The song's protagonist, standing at a focal point, as a signed distance.
*
* The missing piece for full-frame fields. A pattern spread evenly over the
* screen has nothing to watch and nothing to track: there is no subject, so the
* eye has nowhere to rest and no way to tell one song's version from another's.
*
* Warping the field around a focus was the previous attempt and it did not fix
* this. Measured on Voronoi Shatter it raised orientation variety from 0.140 to
* 0.194 while layout stayed at 0.004 — the cells changed shape and the frame's
* energy stayed exactly as evenly spread as before, which is the same thing as
* saying it still had no subject. A subject has to occupy part of the frame and
* leave the rest alone.
*
* The index argument picks which focal point. Returns a large positive distance when the
* song asked for no focus, so a scene can add it unconditionally.
*/
float subjectSDF(vec2 p, float index, float size) {
if (u_focusN < 0.5) return 1e3;
float s = max(size * stageScale(), 1e-3);
return castMain((p - focusAt(index)) / s) * s;
}
/** The nearest subject, for scenes that just want "is there one here". */
float subjectSDF(vec2 p, float size) {
float d = 1e3;
for (int i = 0; i < 3; i++) {
if (float(i) >= u_focusN) break;
d = min(d, subjectSDF(p, float(i), size));
}
return d;
}
/** 0..1 inside the subject, with a soft edge. */
float subjectMask(vec2 p, float size) {
float d = subjectSDF(p, size);
return smoothstep(0.012, -0.012, d);
}
/** The subject's rim, for a line the eye can follow. */
float subjectEdge(vec2 p, float size) {
return inkStroke(subjectSDF(p, size));
}
/**
* Displace a coordinate around the subject: the field bends near the form and
* is untouched away from it.
*
* This is the WARP impact, and it is what a scene reaches for when it wants the
* pattern itself disturbed rather than replaced.
*/
vec2 subjectWarp(vec2 p, float size, float amount) {
if (u_focusN < 0.5 || amount <= 0.0) return p;
vec2 q = p;
for (int i = 0; i < 3; i++) {
if (float(i) >= u_focusN) break;
vec2 c = focusAt(float(i));
vec2 d = p - c;
float r = length(d);
float s = max(size * stageScale(), 1e-3);
// Strongest at the form's edge, nothing at its centre or far away.
float w = smoothstep(s * 2.2, s * 0.9, r) * smoothstep(0.0, s * 0.6, r);
q += normalize(d + 1e-5) * w * amount * s * 0.9;
}
return q;
}
/** Which impact the song chose. 0 shift, 1 warp, 2 punch, 3 morph, 4 overlay. */
bool impactIs(float which) { return abs(u_impact - which) < 0.5; }
vec3 prev(vec2 uv) { vec3 prev(vec2 uv) {
if (u_hasPrev == 0) return vec3(0.0); if (u_hasPrev == 0) return vec3(0.0);
return texture2D(u_prev, uv).rgb; return texture2D(u_prev, uv).rgb;
@ -279,6 +661,309 @@ vec3 prev(vec2 uv) {
float sat(float x) { return clamp(x, 0.0, 1.0); } float sat(float x) { return clamp(x, 0.0, 1.0); }
vec3 sat3(vec3 x) { return clamp(x, 0.0, 1.0); } vec3 sat3(vec3 x) { return clamp(x, 0.0, 1.0); }
/**
* The framed equivalent of uv, for scenes that build their image in uv space.
*
* Screen-space scenes — a scan tear, a vertical transposition — slice the FRAME
* and are right to do so: a signal artefact happens to the signal, not to the
* world behind it. But the imagery behind the slicing is still a subject, and a
* subject can be filmed wide or close. So the slice grid stays on raw uv while
* the field it displaces is built from this, which carries the shot's framing.
*/
vec2 framedUv(vec2 p) { return vec2(p.x / u_aspect, p.y) * 0.5 + 0.5; }
`;
/**
* The 3D cast helpers, compiled ONLY into scenes that declare
* `consumes: ['form']`.
*
* They belong to the contract like everything else, but not to the preamble.
* A raymarch is a fixed-bound loop, GLSL compilers unroll those, and an
* unrolled march carries a copy of the assembly SDF per step — so leaving
* these in the shared preamble made all 68 scenes pay a large compile for
* something 3 of them call. Measured, that was enough to make the check page
* look hung: every scene gate compiles the whole library for its distinctness
* comparison.
*/
export const FORM_PREAMBLE = `
// --- the cast in three dimensions -------------------------------------------
// The protagonist as a SOLID rather than as a silhouette. See Identity.js
// generateForm for what an assembly is and why it is worth having.
//
// The short version: an outline is the same picture from every angle, so a
// scene that turns one is showing you the same shape rotated. A solid's
// outline changes as it turns, and that is a variety axis the library did not
// previously have — but only for a scene that actually moves the camera or the
// object, which is why the helpers below take a ray rather than a point.
//
// Everything here is a distance function of position alone: no state, no
// integration, so a seek lands exactly where playback would. Same rule as the
// rest of the contract.
mat3 formRot(float yaw, float pitch) {
float cy = cos(yaw), sy = sin(yaw);
float cp = cos(pitch), sp = sin(pitch);
return mat3(cy, 0.0, -sy, sy * sp, cp, cy * sp, sy * cp, -sp, cy * cp);
}
float sminForm(float a, float b, float k) {
if (k <= 0.0001) return min(a, b);
float h = clamp(0.5 + 0.5 * (b - a) / k, 0.0, 1.0);
return mix(b, a, h) - k * h * (1.0 - h);
}
/** The song's own profile, extruded. The part kind that keeps the cast's face. */
float formPrism(vec3 q, vec3 r) {
vec2 rr = max(r.xy, vec2(1e-3));
// Scaled back by the SMALLER axis: a non-uniform scale is not a distance,
// and taking the larger one overestimates, which the march then overshoots
// into pinholes along the profile's edge.
float d2 = castMain(q.xy / rr) * min(rr.x, rr.y);
float dz = abs(q.z) - max(r.z, 1e-3) * u_formDepth;
return min(max(d2, dz), 0.0) + length(max(vec2(d2, dz), 0.0));
}
float formBox(vec3 q, vec3 r) {
vec3 d = abs(q) - max(r, vec3(1e-3));
return min(max(d.x, max(d.y, d.z)), 0.0) + length(max(d, 0.0));
}
float formCapsule(vec3 q, vec3 r) {
float rad = max(min(r.x, r.z), 1e-3);
float h = max(r.y, 1e-3);
q.y -= clamp(q.y, -h, h);
return length(q) - rad;
}
float formTorus(vec3 q, vec3 r) {
vec2 c = vec2(length(q.xz) - max(r.x, 1e-3), q.y);
return length(c) - max(r.z * 0.45, 1e-3);
}
/**
* Fold a point so the parts repeat. This is what turns an assembly from debris
* into an object — see Identity.js SYMMETRIES.
*/
vec3 formFold(vec3 q, float k, float folds) {
if (k < 0.5) return q;
if (k < 1.5) { q.x = abs(q.x); return q; }
if (k < 2.5) { // radial about the up axis
float n = max(folds, 2.0);
float seg = 6.28318530718 / n;
float a = atan(q.z, q.x);
float r = length(q.xz);
a = abs(mod(a + seg * 0.5, seg) - seg * 0.5);
return vec3(cos(a) * r, q.y, sin(a) * r);
}
// stack: a bounded repeat up the form's own axis, so it is a column of a
// known height rather than an infinite one the ray never escapes.
float n = max(folds, 2.0);
float h = 1.6 / n;
float lim = (n - 1.0) * 0.5;
q.y -= h * clamp(floor(q.y / h + 0.5), -lim, lim);
return q;
}
/**
* Signed distance to an assembly, radius ~1.
*
* Shared by the protagonist and the chorus, which differ only in how many parts
* they take, how those parts repeat, and their proportions — see Identity.js.
* The prop argument scales each part's height and depth, which is what makes the
* chorus a relative of the protagonist rather than a smaller copy of it.
*
* Falls back to the flat profile extruded when the song brought no assembly, so
* a scene may call this unconditionally and still draw the right character.
*/
float formSDF(vec3 q, float count, float sym, float symN, vec2 prop) {
q = formFold(q, sym, symN);
if (count < 0.5) return formPrism(q, vec3(1.0, prop.x, 0.6 * prop.y));
float d = 1e3;
for (int i = 0; i < 6; i++) {
if (float(i) >= count) break;
vec4 row0 = u_formPart[i * 3];
vec4 row1 = u_formPart[i * 3 + 1];
vec4 row2 = u_formPart[i * 3 + 2];
vec3 p = formRot(row2.x, row2.y) * (q - row0.xyz);
vec3 r = row1.xyz * vec3(1.0, prop.x, prop.y);
float pd;
if (row0.w < 0.5) pd = formPrism(p, r);
else if (row0.w < 1.5) pd = formBox(p, r);
else if (row0.w < 2.5) pd = formCapsule(p, r);
else if (row0.w < 3.5) pd = formTorus(p, r);
else pd = length(p / max(r, vec3(1e-3))) * min(r.x, min(r.y, r.z))
- min(r.x, min(r.y, r.z));
pd -= row2.z * 0.15;
if (i == 0) d = pd;
else if (row1.w < 0.5) d = min(d, pd);
else if (row1.w < 1.5) d = sminForm(d, pd, u_formBlend);
else d = max(d, -pd);
}
return d;
}
/** The protagonist as a solid, radius ~1. */
float castSDF3(vec3 q) {
return formSDF(q, u_formCount, u_formSym, u_formSymN, vec2(1.0));
}
/**
* The chorus member as a solid — many of these, small. The 3D counterpart of
* castChorus, and what a scene fills a space with.
*/
float castChorus3(vec3 q) {
return formSDF(q, u_formChorusN, u_formChorusSym, u_formChorusSymN,
vec2(u_formChorusFlat, u_formChorusThin));
}
// Surface normals, by the tetrahedron trick: four samples rather than the six
// central differences take. The assembly SDF is the expensive call in this
// contract — up to six primitives per evaluation — so two saved samples per
// shaded pixel is worth more here than the marginal accuracy, and at the sizes
// these are drawn the difference is not visible.
vec3 castNormal3(vec3 q) {
vec2 k = vec2(1.0, -1.0);
float e = 0.0025;
return normalize(k.xyy * castSDF3(q + k.xyy * e) + k.yyx * castSDF3(q + k.yyx * e)
+ k.yxy * castSDF3(q + k.yxy * e) + k.xxx * castSDF3(q + k.xxx * e));
}
vec3 castChorusNormal3(vec3 q) {
vec2 k = vec2(1.0, -1.0);
float e = 0.004;
return normalize(k.xyy * castChorus3(q + k.xyy * e) + k.yyx * castChorus3(q + k.yyx * e)
+ k.yxy * castChorus3(q + k.yxy * e) + k.xxx * castChorus3(q + k.xxx * e));
}
/** A rotation a scene can hand the instance helpers. */
mat3 castTurn(float yaw, float pitch) { return formRot(yaw, pitch); }
/**
* March a ray at the solid. Returns the hit distance, or -1 for a miss, and
* writes the surface normal.
*
* The canned version exists so a scene that wants the song's object in its world
* costs five lines rather than a raymarcher — the same bargain castForm makes in
* two dimensions, and the reason the library has sixty scenes.
*/
float castMarch(vec3 ro, vec3 rd, float far, out vec3 n) {
n = vec3(0.0, 0.0, 1.0);
float t = 0.0;
float hit = -1.0;
for (int i = 0; i < 64; i++) {
float d = castSDF3(ro + rd * t);
if (d < 0.0012) { hit = t; break; }
// Slightly under-relaxed: the boolean ops are not true distances at the
// seams, and a full step overshoots them into visible pinholes.
t += d * 0.82;
if (t > far) break;
}
// The normal is taken AFTER the loop, never inside it. GLSL compilers
// unroll a fixed-bound march, so a normal in the loop body multiplies four
// more copies of the assembly SDF by the step count — which compiled fine
// for one call site and made a fourteen-instance scene take minutes to
// build, with the check page sitting there looking hung.
if (hit < 0.0) return -1.0;
n = castNormal3(ro + rd * hit);
return hit;
}
/**
* The solid shaded in the track's palette and value structure.
*
* Lighting is a look decision, so it belongs here rather than in each scene:
* two stages that both march the protagonist should agree about which way the
* key light points, exactly as they agree about the lattice.
*/
/**
* One INSTANCE of the solid, drawn where a 2D scene was stamping the profile.
*
* The local argument is the pixel in the instance's own frame — (p - centre) /
* size, so an existing scene passes exactly what it already computes for
* castMain. Returns depth into the object, or -1 for a miss, and writes the
* world-space normal.
*
* Orthographic, and that is the point: a scene drawing thirty small objects
* wants each one to look solid, not to share one perspective camera it does not
* have. The bounding-sphere reject before the march is what makes thirty of them
* affordable — cost follows the instances a pixel actually touches rather than
* the instances in the frame, so a sparse field costs almost nothing.
*/
// The span of the ray that can possibly be inside the object: the chord of the
// bounding sphere, from a camera two radii back. Starting the march at the
// SPHERE rather than at the camera is what makes many instances affordable —
// the first draft marched the empty two radii in front of every object, spent
// most of its step budget there, and made a fourteen-element field slow enough
// that the scene gate stopped finishing.
#define CAST_SPHERE_R2 1.3
float castSolid(vec2 local, mat3 turn, out vec3 n) {
n = vec3(0.0, 0.0, -1.0);
float h = CAST_SPHERE_R2 - dot(local, local);
if (h <= 0.0) return -1.0;
float half_ = sqrt(h);
vec3 ro = turn * vec3(local, -2.0);
vec3 rd = turn * vec3(0.0, 0.0, 1.0);
float t = 2.0 - half_;
float far = 2.0 + half_;
float hit = -1.0;
for (int i = 0; i < 24; i++) {
float d = castSDF3(ro + rd * t);
if (d < 0.004) { hit = t; break; }
t += d * 0.82;
if (t > far) break;
}
if (hit < 0.0) return -1.0;
n = castNormal3(ro + rd * hit) * turn;
return hit;
}
/** The same, for a chorus member. Fewer steps: these are drawn small. */
float castChorusSolid(vec2 local, mat3 turn, out vec3 n) {
n = vec3(0.0, 0.0, -1.0);
float h = CAST_SPHERE_R2 - dot(local, local);
if (h <= 0.0) return -1.0;
float half_ = sqrt(h);
vec3 ro = turn * vec3(local, -2.0);
vec3 rd = turn * vec3(0.0, 0.0, 1.0);
float t = 2.0 - half_;
float far = 2.0 + half_;
float hit = -1.0;
for (int i = 0; i < 12; i++) {
float d = castChorus3(ro + rd * t);
if (d < 0.008) { hit = t; break; }
t += d * 0.82;
if (t > far) break;
}
if (hit < 0.0) return -1.0;
n = castChorusNormal3(ro + rd * hit) * turn;
return hit;
}
vec3 castLit(vec3 n, vec3 rd) {
vec3 key = normalize(vec3(0.45, 0.75, 0.5));
float diff = max(dot(n, key), 0.0);
float fill = max(dot(n, -key), 0.0) * 0.35;
float rim = pow(1.0 - max(dot(n, -rd), 0.0), 2.5);
vec3 col = mix(pal(1) * 0.22, pal(2), diff);
col += pal(0) * fill;
col += pal(3) * rim * 0.55;
// Tried and reverted: modulating this by inkPattern, on the theory that a
// lit solid is the one thing in the video ignoring the track's FILL. It
// reads well and measures nothing — Pylon Grid 0.1324 to 0.1322, Effigy
// 0.2382 to 0.2373, both inside the last digit that means anything. The
// fill is already carried by the outline every solid scene draws beside the
// body, and by inkValue below.
return inkValue(col);
}
`; `;
const EPILOGUE = ` const EPILOGUE = `
@ -286,6 +971,23 @@ void main() {
vec2 uv = vUv; vec2 uv = vUv;
vec2 p = (uv - 0.5) * 2.0; vec2 p = (uv - 0.5) * 2.0;
p.x *= u_aspect; p.x *= u_aspect;
// FRAMING is applied here, to the coordinate every scene is handed, rather
// than inside sigCamera where it started out.
//
// sigCamera is gated on the camera personality trait: a scene that does
// not want the track's drift and sway simply never calls it. That is
// correct for a TRAIT and wrong for framing, which is not one — it is where
// the camera is standing for this shot, and no scene should be exempt from
// it because of an unrelated art-direction decision. Measured, that
// accident left Scan Tear and Pylon Grid completely unframed, and the two
// are otherwise perfectly good candidates for a close-up.
//
// uv is deliberately NOT framed. It is screen space: prev() reads the
// feedback buffer with it and sigGrain speckles in it, and both of those
// belong to the output image rather than to the scene being filmed.
p = p / max(u_sigFrameScale, 0.05) + u_sigFrameShift;
vec4 col = scene(uv, p); vec4 col = scene(uv, p);
gl_FragColor = vec4(col.rgb, col.a * u_opacity); gl_FragColor = vec4(col.rgb, col.a * u_opacity);
} }
@ -305,8 +1007,13 @@ export function buildFragmentShader(sceneModule) {
paramUniforms.push(`uniform ${glslType} ${def.uniform}; // param: ${name}`); paramUniforms.push(`uniform ${glslType} ${def.uniform}; // param: ${name}`);
} }
// The solid helpers are opt-in: see FORM_PREAMBLE for why they are not in
// the shared preamble.
const wantsForm = (sceneModule.consumes || []).includes('form');
return [ return [
PREAMBLE, PREAMBLE,
wantsForm ? FORM_PREAMBLE : '',
paramUniforms.join('\n'), paramUniforms.join('\n'),
'\n// ---- scene ----\n', '\n// ---- scene ----\n',
sceneModule.shader, sceneModule.shader,

341
flow-state/src/look/ArcDriver.js
View File

@ -2,8 +2,13 @@ import { createLayer } from '../engine/Layer.js';
import { Rng } from '../engine/rng.js'; import { Rng } from '../engine/rng.js';
import { clampValue } from '../params/schema.js'; import { clampValue } from '../params/schema.js';
import { paletteShiftAt } from './paletteArc.js'; import { paletteShiftAt } from './paletteArc.js';
import { shiftPalette } from './palette.js'; import { shiftPalette, lerpPalettes } from './palette.js';
import { frameShot, neutralFraming } from './framing.js'; import { frameShot, neutralFraming } from './framing.js';
import { planGaze, gazeAt } from './Camera.js';
import { storyStateAt, NEUTRAL_STATE } from './Story.js';
import { subjectIndexOf, isGround } from './stack.js';
import { groundPersonalityFrom } from '../scenes/surface.js';
import { derivePalettePlan, directorByName } from './directors.js';
/** /**
* Drives the look across the song. * Drives the look across the song.
@ -43,6 +48,8 @@ export class ArcDriver {
this.framingStyle = (look.framing && look.framing.mode !== 'locked') this.framingStyle = (look.framing && look.framing.mode !== 'locked')
? look.framing : null; ? look.framing : null;
this._planFraming(); this._planFraming();
this._planGaze();
this._planPalette();
this.state = { sectionIndex: 0, shotIndex: 0, crossfade: 0, incoming: null }; this.state = { sectionIndex: 0, shotIndex: 0, crossfade: 0, incoming: null };
} }
@ -63,12 +70,75 @@ export class ArcDriver {
for (const cue of this.cues) { for (const cue of this.cues) {
const section = this.look.sections[cue.sectionIndex]; const section = this.look.sections[cue.sectionIndex];
const energy = (section.bias && section.bias.energy) || 0; const energy = (section.bias && section.bias.energy) || 0;
const framing = frameShot(this.framingStyle, previous, energy, rng); // Shot size is where the story's `closeness` lands: a video that is
// approaching its subject does it at the cuts, because that is the
// only place a size is allowed to change. See look/framing.js.
const closeness = (section.story || NEUTRAL_STATE).closeness;
const framing = frameShot(this.framingStyle, previous, energy, rng, closeness);
cue.framing = framing; cue.framing = framing;
previous = framing; previous = framing;
} }
} }
/**
* Plan where the camera looks, across the whole video.
*
* After framing, because the reach available to a move depends on the shot
* size it is made at — a close-up is inside the composition and can travel
* across it; a wide already sees the whole thing. See Camera.reachFor.
*
* Unlike framing this does NOT stop at a locked-off track: locked is a
* decision about SIZE, and a video that never changes size can still be one
* whose attention moves. Only a look with no camera at all — hand-built, or
* a check constructing sections directly — goes without.
*/
_planGaze() {
if (!this.look.camera) return;
const rng = new Rng((this.look.seed ^ 0x2f9c1d4b) >>> 0);
this.gaze = planGaze(this.cues, this.look.sections, this.look.camera, rng);
}
/**
* Plan which palette is on screen for each cue.
*
* The director picks the progression; the palette set determines how many
* choices there are. Stored on the look so it survives serialisation and so
* the HUD can describe it.
*/
_planPalette() {
const director = directorByName(this.look.director);
const palettes = this.look.palettes || [this.look.palette];
const rng = new Rng((this.look.seed ^ 0x7a1b3c9d) >>> 0);
this.look.palettePlan = derivePalettePlan(
this.cues, this.look.sections, director, rng, palettes);
}
/** Where the camera is looking at `frame`, for the cue at `cueIndex`. */
_gazeAt(cueIndex, frame) {
if (!this.gaze) return null;
const cue = this.cues[cueIndex];
const move = this.gaze[cueIndex];
if (!cue || !move) return null;
return gazeAt(move, frame - cue.startFrame);
}
/**
* The framing a cue is played with at `frame` — its size, plus wherever the
* gaze has travelled to by now.
*
* Size comes off the plan and never changes within the shot; the shift is
* live. Returning a fresh object each call is deliberate: Layer copies the
* values into uniforms immediately, and a shared mutable framing would make
* the outgoing half of a crossfade read the incoming half's position.
*/
_framingAt(cueIndex, frame) {
const cue = this.cues[cueIndex];
const base = (cue && cue.framing) || neutralFraming();
const shift = this._gazeAt(cueIndex, frame);
if (!shift) return base;
return { size: base.size, scale: base.scale, shift };
}
dispose() { dispose() {
for (const layer of this.layerCache.values()) layer.dispose(); for (const layer of this.layerCache.values()) layer.dispose();
this.layerCache.clear(); this.layerCache.clear();
@ -100,6 +170,10 @@ export class ArcDriver {
startFrame: shot.startFrame, startFrame: shot.startFrame,
endFrame: shot.endFrame, endFrame: shot.endFrame,
atSectionStart, atSectionStart,
// Carried onto the cue as well as folded into fadeFrames:
// the camera reads it, because a straight cut earns a
// bigger reframe than a dissolve. See Camera.jumpFor.
hardCut: !!shot.hardCut && !atSectionStart,
fadeFrames: shot.hardCut && !atSectionStart fadeFrames: shot.hardCut && !atSectionStart
? Math.max(2, Math.round(this.track.fps * 0.06)) ? Math.max(2, Math.round(this.track.fps * 0.06))
: this._dissolveFrames(energy, span), : this._dissolveFrames(energy, span),
@ -245,6 +319,15 @@ export class ArcDriver {
// a random draw finds the second kind almost every time. // a random draw finds the second kind almost every time.
const declared = eligible.filter(([, def]) => def.slowAxis); const declared = eligible.filter(([, def]) => def.slowAxis);
// Which WAY the video travels is the track's decision, not the scene's.
//
// The sign used to be an independent coin flip per scene, so a five
// minute video routinely had one scene growing denser while the next one
// thinned out — movement with no direction, which is the difference
// between a video that goes somewhere and one that merely changes.
// Magnitude stays per scene; the sign is shared. See look/Story.js.
const sign = this.look.story ? this.look.story.axisSign : (rng.bool() ? 1 : -1);
const axis = []; const axis = [];
if (declared.length) { if (declared.length) {
for (const [name, def] of declared) { for (const [name, def] of declared) {
@ -256,7 +339,7 @@ export class ArcDriver {
// Most of the range. This param was chosen because moving it // Most of the range. This param was chosen because moving it
// is what the scene looks like changing, so a timid walk // is what the scene looks like changing, so a timid walk
// wastes the one lever that works. // wastes the one lever that works.
travel: (hi - lo) * rng.range(0.45, 0.7) * (rng.bool() ? 1 : -1), travel: (hi - lo) * rng.range(0.45, 0.7) * sign,
}); });
} }
} else { } else {
@ -272,7 +355,7 @@ export class ArcDriver {
name, name,
def, def,
declared: false, declared: false,
travel: (hi - lo) * rng.range(0.25, 0.5) * (rng.bool() ? 1 : -1), travel: (hi - lo) * rng.range(0.25, 0.5) * sign,
}); });
} }
} }
@ -284,16 +367,28 @@ export class ArcDriver {
* Base params for a section at a given time: the look's sampled values, plus * Base params for a section at a given time: the look's sampled values, plus
* the slow axis, plus drift, plus the lookahead ramp toward what comes next. * the slow axis, plus drift, plus the lookahead ramp toward what comes next.
*/ */
_paramsAt(cue, slot, time, features) { _paramsAt(cue, slot, time, features, story = null) {
const spec = this._specFor(cue.sectionIndex, cue.variant, slot); const spec = this._specFor(cue.sectionIndex, cue.variant, slot);
const out = { ...spec.params }; const out = { ...spec.params };
// --- slow axis ------------------------------------------------------ // --- slow axis ------------------------------------------------------
// Eased rather than linear, so the travel is slowest at the head and // How far along the journey this frame is. The story owns this: its
// tail. A video should not open mid-move. // curve holds inside a section and moves at the boundary, so the axis
// travels in STAGES rather than sliding continuously for five minutes —
// a scene of a story rather than a slow zoom. See look/Story.js.
//
// With no story it falls back to the eased progress ramp this was
// before, which is also what Story.js emits for a track too short to
// carry one: slowest at the head and tail, because a video should not
// open mid-move.
let journey;
if (story) {
journey = story.journey;
} else {
const duration = Math.max(1e-6, this.track.duration); const duration = Math.max(1e-6, this.track.duration);
const p = Math.max(0, Math.min(1, time / duration)); const p = Math.max(0, Math.min(1, time / duration));
const journey = p * p * (3 - 2 * p); journey = p * p * (3 - 2 * p);
}
for (const item of this._slowAxisFor(spec.module)) { for (const item of this._slowAxisFor(spec.module)) {
const base = out[item.name]; const base = out[item.name];
if (typeof base !== 'number') continue; if (typeof base !== 'number') continue;
@ -372,36 +467,151 @@ export class ArcDriver {
* scenes is a flash, and the flash meter is not decorative. * scenes is a flash, and the flash meter is not decorative.
*/ */
/** /**
* The track's palette, moved to where this frame sits in the arc. * The track's palette, moved to where this frame sits in the arc and in the
* palette plan.
* *
* Recomputed once per frame rather than once per layer, and memoised on the * Two levels: the director's coarse choice of which base palette is on screen
* rounded shift: the movement is slow by design, so consecutive frames * for this cue (from `look.palettes` / `look.palettePlan`), plus the fine
* almost always want the same colours and the OKLCH round trip is wasted * `paletteArc` drift inside that palette. Blends between two palettes in
* work. Rounding also makes the cache key stable under a seek, which keeps * OKLCH when the plan's transition is `blend`, timed to the cue's own
* the frame-exactness guarantee — a seeked frame gets bit-identical colours * crossfade so colour and image move together.
* to a played one rather than merely similar ones. *
* Recomputed once per frame and memoised on the rounded state.
*/ */
_paletteAt(frame, features) { _paletteAt(frame, features, story) {
const arc = this.look.paletteArc; const arc = this.look.paletteArc;
if (!arc || arc.mode === 'static') return this.look.palette; const plan = this.look.palettePlan;
const palettes = this.look.palettes || [this.look.palette];
const cueIndex = this._cueIndexAt(frame);
const cue = this.cues[cueIndex] || null;
// Coarse palette for this cue (fallback to the single palette when no plan)
let baseIndex = 0;
if (plan && plan.cues && plan.cues.length > cueIndex) {
baseIndex = plan.cues[cueIndex];
}
baseIndex = Math.max(0, Math.min(palettes.length - 1, baseIndex | 0));
// During a blend transition, lerp from the previous cue's palette to this
// one's over the cue's fadeFrames, in OKLCH so hue travel stays perceptual.
let base = palettes[baseIndex] || palettes[0];
let blendT = 0;
let blendFrom = -1;
if (plan && plan.transition === 'blend' && cue && cueIndex > 0) {
const prevIndex = plan.cues[cueIndex - 1];
if (prevIndex !== baseIndex) {
const into = frame - cue.startFrame;
if (into >= 0 && into < cue.fadeFrames) {
const t = into / Math.max(1, cue.fadeFrames);
blendT = t * t * (3 - 2 * t);
blendFrom = prevIndex;
const a = palettes[prevIndex] || palettes[0];
const b = base;
base = lerpPalettes(a, b, blendT);
}
}
}
if (!arc || arc.mode === 'static') {
// Still memoise so a seek returns the same object identity for the
// compositor's layer-change check, but key off palette selection too.
const key = `p${baseIndex}|f${blendFrom}:${blendT.toFixed(3)}`;
if (this._paletteKey === key) return this._palette;
this._paletteKey = key;
this._palette = base;
return this._palette;
}
const shift = paletteShiftAt(arc, { const shift = paletteShiftAt(arc, {
progress: frame / Math.max(1, this.track.frameCount - 1), progress: frame / Math.max(1, this.track.frameCount - 1),
sectionKind: this.track.sectionAt(frame).kind, sectionKind: this.track.sectionAt(frame).kind,
features, features,
story,
}); });
const key = `${shift.hue.toFixed(3)}|${shift.chroma.toFixed(3)}|${shift.lightness.toFixed(3)}`; const key = `p${baseIndex}|f${blendFrom}:${blendT.toFixed(3)}|${shift.hue.toFixed(3)}|${shift.chroma.toFixed(3)}|${shift.lightness.toFixed(3)}`;
if (this._paletteKey === key) return this._palette; if (this._paletteKey === key) return this._palette;
this._paletteKey = key; this._paletteKey = key;
this._palette = shiftPalette(this.look.palette, shift); this._palette = shiftPalette(base, shift);
return this._palette; return this._palette;
} }
/**
* The track's personality, with as much of its identity SHOWN as the story
* has reached.
*
* This is where a story reaches Epic 3's content registers, and it needed no
* new uniform to do it: `setPersonality` is already called on every layer
* every frame, and the cast/ink/lattice uniforms are derived from the object
* it is handed. Scaling the features that make the song's forms specific —
* the notches, the hole through the middle, the outline, the size hierarchy
* — means the cast literally ARRIVES over the video instead of being fully
* stated in the first shot and merely repeated after that.
*
* Only the specificity moves, never the identity itself: the protagonist has
* the same number of sides at thirty seconds as at four minutes. A form that
* changed its shape would be a different character rather than the same one
* seen more clearly.
*
* Memoised on the rounded reveal, exactly as _paletteAt memoises on the
* rounded shift and for the same two reasons: consecutive frames want the
* same value, and rounding is what keeps a seeked frame bit-identical to a
* played one rather than merely close.
*/
_personalityAt(story) {
const base = this.look.personality;
if (!base || !base.identity || !story) return base;
const reveal = Math.max(0, Math.min(1, story.reveal));
const key = Math.round(reveal * 50);
if (!this._personalityCache) this._personalityCache = new Map();
const cached = this._personalityCache.get(key);
if (cached) return cached;
// Never all the way to nothing. A cast erased to plain circles is a
// different track's cast, not this one's withheld — the video still has
// to look like itself in its first thirty seconds.
const shown = 0.35 + (key / 50) * 0.65;
const id = base.identity;
const member = (m) => ({
...m,
notchDepth: m.notchDepth * shown,
hollow: m.hollow * shown,
});
const moved = {
...base,
identity: {
...id,
cast: { protagonist: member(id.cast.protagonist), chorus: member(id.cast.chorus) },
ink: {
...id.ink,
outline: id.ink.outline * shown,
// Posterisation is a value structure rather than an amount,
// so it arrives whole at a threshold instead of fading in.
posterize: shown > 0.6 ? id.ink.posterize : 0,
},
lattice: { ...id.lattice, scaleSpread: id.lattice.scaleSpread * shown },
},
};
this._personalityCache.set(key, moved);
return moved;
}
update(frame, features) { update(frame, features) {
const time = frame / this.track.fps; const time = frame / this.track.fps;
const palette = this._paletteAt(frame, features); // Where the video is in its story. One lookup per frame, handed to
// everything below rather than recomputed — and a pure function of the
// frame, so a seek lands on the same story position as playback.
// A look with no story at all — hand-built by a check, or generated
// before this existed — passes null rather than the neutral state, so
// everything below takes its own pre-story path. The neutral state's
// `journey` is 0.5, and handing that to the slow axis would park it at
// the middle of its travel for the whole video rather than ramping.
const story = this.look.story ? storyStateAt(this.look.story, frame) : null;
const palette = this._paletteAt(frame, features, story);
const personality = this._personalityAt(story);
const cueIndex = this._cueIndexAt(frame); const cueIndex = this._cueIndexAt(frame);
const cue = this.cues[cueIndex]; const cue = this.cues[cueIndex];
if (!cue) return this.activeLayers; if (!cue) return this.activeLayers;
@ -416,8 +626,15 @@ export class ArcDriver {
// The shot being played INTO carries its own framing; the shot fading // The shot being played INTO carries its own framing; the shot fading
// out keeps the framing it was filmed with, so a cut changes the size // out keeps the framing it was filmed with, so a cut changes the size
// exactly when the cut changes the image rather than half a beat after. // exactly when the cut changes the image rather than half a beat after.
const framing = cue.framing || neutralFraming(); //
const outgoingFraming = previous ? (previous.framing || neutralFraming()) : framing; // The outgoing shot is evaluated at the SAME frame, on its own move —
// it is still on screen, and freezing its gaze at the cut would stop
// the old image dead half a second before it disappears. A camera that
// was travelling when the edit arrived keeps travelling as it fades.
const framing = this._framingAt(cueIndex, frame);
const outgoingFraming = previous
? this._framingAt(cueIndex - 1, frame)
: framing;
const layers = []; const layers = [];
@ -437,11 +654,11 @@ export class ArcDriver {
for (let slot = 0; slot < this._stackSize(previous); slot++) { for (let slot = 0; slot < this._stackSize(previous); slot++) {
const spec = this._specFor(previous.sectionIndex, previous.variant, slot); const spec = this._specFor(previous.sectionIndex, previous.variant, slot);
const layer = this._layerFor(previous.sectionIndex, previous.variant, slot); const layer = this._layerFor(previous.sectionIndex, previous.variant, slot);
layer.setParams(this._paramsAt(previous, slot, time, outgoingFeatures)); layer.setParams(this._paramsAt(previous, slot, time, outgoingFeatures, story));
layer.opacity = slot === 0 ? 1 : spec.opacity; layer.opacity = slot === 0 ? 1 : spec.opacity;
layer.blend = slot === 0 ? 'normal' : spec.blend; layer.blend = slot === 0 ? 'normal' : spec.blend;
layer.setPalette(palette); layer.setPalette(palette);
layer.setPersonality(this.look.personality); layer.setPersonality(isGround(spec) ? groundPersonalityFrom(personality) : personality);
layer.setFraming(outgoingFraming); layer.setFraming(outgoingFraming);
layers.push(layer); layers.push(layer);
} }
@ -450,11 +667,11 @@ export class ArcDriver {
for (let slot = 0; slot < this._stackSize(cue); slot++) { for (let slot = 0; slot < this._stackSize(cue); slot++) {
const spec = this._specFor(cue.sectionIndex, cue.variant, slot); const spec = this._specFor(cue.sectionIndex, cue.variant, slot);
const layer = this._layerFor(cue.sectionIndex, cue.variant, slot); const layer = this._layerFor(cue.sectionIndex, cue.variant, slot);
layer.setParams(this._paramsAt(cue, slot, time, features)); layer.setParams(this._paramsAt(cue, slot, time, features, story));
layer.opacity = (slot === 0 ? 1 : spec.opacity) * (fading ? eased : 1); layer.opacity = (slot === 0 ? 1 : spec.opacity) * (fading ? eased : 1);
layer.blend = slot === 0 ? 'normal' : spec.blend; layer.blend = slot === 0 ? 'normal' : spec.blend;
layer.setPalette(palette); layer.setPalette(palette);
layer.setPersonality(this.look.personality); layer.setPersonality(isGround(spec) ? groundPersonalityFrom(personality) : personality);
layer.setFraming(framing); layer.setFraming(framing);
layers.push(layer); layers.push(layer);
} }
@ -466,18 +683,36 @@ export class ArcDriver {
variant: cue.variant, variant: cue.variant,
kind: section.kind, kind: section.kind,
crossfade: fading ? eased : 0, crossfade: fading ? eased : 0,
sceneName: this._specFor(cue.sectionIndex, cue.variant, 0).module.name, // The SHOT's name, not the ground's. Every stack starts with a bed
// now, and the HUD naming it would report the same handful of
// canvases for every section of every video.
sceneName: this._specFor(
cue.sectionIndex, cue.variant, this._subjectSlot(cue)).module.name,
buildSlope: features ? features.buildSlope || 0 : 0, buildSlope: features ? features.buildSlope || 0 : 0,
// Where the story is, for the HUD and the checks. A video that is
// supposed to be going somewhere should be able to say where.
act: story.act,
tension: story.tension,
reveal: story.reveal,
journey: story.journey,
}; };
this.activeLayers = layers; this.activeLayers = layers;
return layers; return layers;
} }
_stackSize(cue) { /** Which slot of a cue's stack is the shot. See look/stack.js. */
_subjectSlot(cue) {
return subjectIndexOf(this._stackFor(cue));
}
_stackFor(cue) {
const section = this.look.sections[cue.sectionIndex]; const section = this.look.sections[cue.sectionIndex];
const stack = (section.variants && section.variants[cue.variant]) || section.layers; return (section.variants && section.variants[cue.variant]) || section.layers;
return stack.length; }
_stackSize(cue) {
return this._stackFor(cue).length;
} }
/** Layers changed identity — the compositor needs the new list. */ /** Layers changed identity — the compositor needs the new list. */
@ -498,6 +733,9 @@ export class ArcDriver {
// A reroll re-plans the section's shots, so the cue list is stale too. // A reroll re-plans the section's shots, so the cue list is stale too.
this.cues = this._buildCues(); this.cues = this._buildCues();
this._planFraming(); this._planFraming();
this._planGaze();
this._planPalette();
this._paletteKey = null;
this._slopeCache = null; this._slopeCache = null;
} }
@ -506,6 +744,9 @@ export class ArcDriver {
this.driftPlans.clear(); this.driftPlans.clear();
this.cues = this._buildCues(); this.cues = this._buildCues();
this._planFraming(); this._planFraming();
this._planGaze();
this._planPalette();
this._paletteKey = null;
this._slopeCache = null; this._slopeCache = null;
} }
@ -527,9 +768,49 @@ export class ArcDriver {
return this; return this;
} }
/**
* Which palette of the set is on screen for a frame. Mirrors the coarse
* selection in _paletteAt without the OKLCH lerp or the fine shift — what
* the panel highlights as "active".
*/
paletteIndexAt(frame) {
const plan = this.look.palettePlan;
const palettes = this.look.palettes || [this.look.palette];
if (!plan || !plan.cues || !plan.cues.length) return 0;
const cueIndex = this._cueIndexAt(frame);
let idx = plan.cues[cueIndex];
if (idx === undefined) idx = plan.cues[plan.cues.length - 1] || 0;
return Math.max(0, Math.min(palettes.length - 1, idx | 0));
}
/**
* Whether a frame is inside a palette blend window.
* Returns {from,to,t} while the two palettes are interpolating, else null.
*/
paletteBlendAt(frame) {
const plan = this.look.palettePlan;
if (!plan || plan.transition !== 'blend') return null;
const cueIndex = this._cueIndexAt(frame);
if (cueIndex === 0) return null;
const cue = this.cues[cueIndex];
if (!cue) return null;
const cur = plan.cues[cueIndex];
const prev = plan.cues[cueIndex - 1];
if (cur === prev) return null;
const into = frame - cue.startFrame;
if (into >= 0 && into < cue.fadeFrames) {
return { from: prev, to: cur, t: into / Math.max(1, cue.fadeFrames) };
}
return null;
}
/** Push a palette change through without rebuilding layers. */ /** Push a palette change through without rebuilding layers. */
setPalette(palette) { setPalette(palette) {
this.look.palette = palette; this.look.palette = palette;
// Keep the full set's alias in sync — `look.palette` is always palettes[0].
if (this.look.palettes && this.look.palettes.length) {
this.look.palettes[0] = palette;
}
// The moved palette is memoised on the SHIFT, so a new base palette at // The moved palette is memoised on the SHIFT, so a new base palette at
// an unchanged point in the arc would otherwise keep serving the old // an unchanged point in the arc would otherwise keep serving the old
// colours until the arc happened to move. // colours until the arc happened to move.

396
flow-state/src/look/Camera.js Normal file
View File

@ -0,0 +1,396 @@
// The GAZE: where in the scene the frame is looking, and how it gets there.
//
// This is the director's camera department. `Story.js` says what the video is
// doing — tension, closeness, order, which act a section is in — in terms that
// are deliberately imagery-free. Something has to turn that into a picture, and
// until now every consumer did its own ad-hoc translation: shots.js reads
// tension for the cut rate, framing.js reads closeness for the shot size,
// LookGenerator reads population for the overlay chance. Nobody owned the
// camera, and it showed.
//
// WHAT WAS WRONG
//
// The recentre existed — `framing.shift`, applied in the shader epilogue as
// `p / scale + shift` — and it was inert. Measured across 121 cues:
//
// median |shift| from centre 0.029 (2.9% of a half-frame)
// median |jump| at a cut 0.014 (1.4%)
// largest jump seen 0.120
//
// Three causes, and the amplitude was only one of them:
//
// 1. `amount = spec.drift * style.range * rng.range(0.3, 1)` capped the offset
// at 0.10, because a comment worried that pushing off centre would find the
// scenes' empty corners.
// 2. The direction was `rng.range(0, 2π)` — a fresh uniform angle every shot.
// No axis, no continuity, no intent. That is why the median JUMP is smaller
// than the median OFFSET: consecutive shots mostly cancelled each other.
// 3. Nothing about the song reached it at all. Shot SIZE got the story's
// `closeness`; the recentre got a per-track constant and a die roll.
//
// WHAT THIS DOES INSTEAD
//
// A gaze is a path, planned once over the whole cue list, so the video's
// attention travels rather than jittering. Per cue it carries a move:
//
// from → to the two points, in scene units
// delay how long the shot holds before it starts moving
// travel how long the move takes
// curve how it accelerates
//
// The story decides all four. A climax jumps far and arrives hard; a resolution
// drifts a short way back toward centre and never quite stops; a low-`order`
// section throws the gaze off its axis. Between two points the gaze is a pure
// function of (plan, cue, frames-into-cue), the same rule the rest of the render
// path follows — a seek lands on the frame playback would have shown.
//
// NOTE ON THE PER-SHOT RULE. framing.js says framing is constant within a shot,
// and that a move inside a shot "would fight the drift LFO and the slow axis,
// both of which already own continuous motion". That was right about SIZE and
// wrong about the recentre: a zoom that creeps during a shot is an effect, but a
// camera that settles onto its subject is how shots have always worked. Size
// still steps at the cut and only at the cut. The gaze moves.
const clamp01 = (x) => Math.max(0, Math.min(1, x));
const lerp = (a, b, t) => a + (b - a) * t;
/**
* How the gaze accelerates between two points.
*
* These are the reason a move reads as a decision rather than as a tween. A
* `snap` and a `glide` cover the same distance in the same time and say
* completely different things about the section they are in.
*/
export const CURVES = {
// Hard out of the gate, decelerating into the target. The edit-room move:
// it feels like the camera was already going when the cut happened.
snap: (t) => 1 - (1 - t) ** 3,
// Slow away, slow in. The default, and what a calm section wants.
glide: (t) => t * t * (3 - 2 * t),
// Constant speed, and it does NOT arrive — see `travel` below, which is
// allowed to exceed the shot. A drifting camera that gets cut away from
// mid-move is the most alive of these.
drift: (t) => t,
// Overshoots and comes back. Used sparingly: it is the only curve here that
// is legible as a flourish, so it belongs at moments and not everywhere.
//
// The standard back-ease-out, written out rather than improvised. The
// improvised version evaluated to 2.0 at t=0 instead of 0 — every `settle`
// shot therefore STARTED at twice its target, well outside the reach the
// plan had clamped it to, and the headroom gate caught offsets of 0.61
// against a 0.385 ceiling. An easing curve has two values that are not
// negotiable, f(0)=0 and f(1)=1, and this one had neither.
settle: (t) => {
const c1 = 1.70158;
const u = t - 1;
return 1 + (c1 + 1) * u * u * u + c1 * u * u;
},
};
export const CURVE_NAMES = Object.keys(CURVES);
/**
* A director's point of view about its camera.
*
* Same rationale as the family table in directors.js: a fixed rule applied to
* every track is how a library ends up with one camera, and the camera is
* exactly the register a viewer reads as "who shot this".
*
* reach multiplier on how far the gaze is allowed from centre
* pace multiplier on travel time — low is restless, high is patient
* curves weights over CURVE_NAMES, in order
* axial how much the gaze prefers to move along one axis rather than
* anywhere. High reads as composed; low reads as searching.
*/
export const CAMERAS = {
// Patient and composed. Long moves, mostly horizontal, rarely hurried.
contemplative: { reach: 0.85, pace: 1.45, curves: [1, 4, 3, 1], axial: 0.75 },
// Cuts with the camera already moving. The closest to an edited music video.
kinetic: { reach: 1.20, pace: 0.60, curves: [5, 2, 1, 1], axial: 0.35 },
// Locked-off until it is not. Holds, then commits to one large move.
deliberate: { reach: 1.10, pace: 0.85, curves: [3, 3, 1, 2], axial: 0.85 },
// Never settles. Long drifts that get cut away from mid-travel.
roaming: { reach: 0.95, pace: 1.70, curves: [1, 2, 5, 1], axial: 0.25 },
// Small, exact, and it always arrives. The one that stays near centre.
precise: { reach: 0.70, pace: 1.00, curves: [2, 5, 1, 2], axial: 0.90 },
};
export const CAMERA_NAMES = Object.keys(CAMERAS);
/**
* The furthest the gaze may sit from centre, in scene units.
*
* The old comment's worry was real but it was stated as a constant when it is a
* function of the shot size. The visible half-frame at scale s is 1/s scene
* units, so a close-up is looking at a small piece of the composition and can
* move a long way across it before reaching anywhere empty; a wide is already
* seeing everything there is and moving off centre only finds the edges.
*
* The scale term is the ADDITION rather than the whole thing, which the first
* version got wrong: `(0.12 + 0.56 * headroom)` gave a close-up 0.35 and every
* other shot 0.12, and since headroom is zero at any scale ≤ 1 that meant the
* normal and wide shots — most of the video — were still capped at barely more
* than the old inert 0.10. The base has to be worth seeing on its own.
*
* Now: `normal` and `wide` allow 0.20–0.34 depending on the camera, `close`
* 0.32–0.45. Against a measured median offset of 0.029 and a hard old ceiling
* of 0.10, a typical move is roughly six times what it was.
*
* The absolute cap is what keeps this inside the answer to "how far should it
* roam" — moderate, with close-ups furthest.
*/
export const MAX_REACH = 0.45;
export function reachFor(scale, camera) {
const headroom = Math.max(0, 1 - 1 / Math.max(scale, 0.05));
return Math.min(MAX_REACH, (0.28 + 0.42 * headroom) * camera.reach);
}
/** Pick a camera for a track. The director leans, the seed decides. */
export function deriveCamera(director, summary, rng) {
const preferred = (director && director.camera) || null;
const weights = CAMERA_NAMES.map((name) => (name === preferred ? 4 : 1));
const name = rng.pickWeighted(CAMERA_NAMES, weights);
return {
name,
...CAMERAS[name],
// The track's own axis. A video whose gaze moves along one line reads as
// composed even when the line is arbitrary — what reads as sloppy is a
// different direction every time, which is precisely what the uniform
// random angle was doing.
axis: rng.range(0, Math.PI * 2),
};
}
/**
* How far this cut should jump, 0..1 of the available reach.
*
* This is the whole "driven by the song" requirement in one function, so the
* mapping is stated rather than buried:
*
* tension the main term. A wound-up section reframes hard.
* act the climax gets the biggest move in the video, and the
* resolution gets the smallest — a video that keeps flinging its
* camera after the peak has nothing left to say with it.
* hardCut a straight cut earns a bigger reframe than a dissolve. The two
* devices are already gated on energy together (shots.js), so this
* compounds deliberately.
* order low order widens the spread, so a section coming apart is also
* less predictable about where it looks.
*/
function jumpFor(story, hardCut, rng) {
const tension = story ? story.tension : 0.5;
const order = story ? story.order : 0.5;
let base = 0.25 + tension * 0.55;
if (story) {
if (story.act === 'climax') base = Math.max(base, 0.85);
else if (story.act === 'turn') base = Math.max(base, 0.6);
else if (story.act === 'resolution') base = Math.min(base, 0.3);
else if (story.act === 'setup') base = Math.min(base, 0.45);
}
if (hardCut) base = Math.min(1, base * 1.25);
// Disorder widens the draw rather than raising it: a broken section is less
// predictable, not uniformly bigger.
const spread = 0.2 + (1 - order) * 0.55;
return clamp01(base * rng.range(1 - spread, 1 + spread * 0.6));
}
/**
* Where the gaze goes next.
*
* Direction is the track's axis, plus a wander that `order` controls and the
* camera's `axial` bounds. The one hard rule is that a move must not simply
* undo the last one — reversing along the same line is how the old uniform
* angle produced offsets that cancelled, and it is why nothing appeared to
* move even at the amplitudes it did reach.
*/
function targetFor(from, distance, camera, story, rng, previousDir) {
const order = story ? story.order : 0.5;
const wander = (1 - camera.axial) * (0.35 + (1 - order) * 0.65);
// Both ends of the axis are legitimate; which one is a coin flip biased
// away from wherever we already are, so the gaze crosses the frame rather
// than orbiting one side of it.
const along = camera.axis + (rng.bool() ? 0 : Math.PI);
let dir = along + rng.range(-Math.PI, Math.PI) * wander;
if (previousDir !== null) {
// Within 35° of a straight reversal, nudge it off. A reversal is a
// legitimate move; an exact retrace is the thing that reads as jitter.
const delta = Math.abs(normalizeAngle(dir - (previousDir + Math.PI)));
if (delta < 0.6) dir += (delta < 0.3 ? 1 : -1) * 0.9;
}
return {
point: [from[0] + Math.cos(dir) * distance, from[1] + Math.sin(dir) * distance],
dir,
};
}
function normalizeAngle(a) {
let x = a % (Math.PI * 2);
if (x > Math.PI) x -= Math.PI * 2;
if (x < -Math.PI) x += Math.PI * 2;
return x;
}
/**
* How long the move takes, and how long the shot waits first.
*
* The "different speeds in different sections" requirement. A loud section
* moves fast and is done; a quiet one takes most of the shot to arrive. Travel
* is allowed to exceed the shot length — that is not a bug, it is what makes a
* `drift` read as a camera that was going somewhere when the edit cut away.
*/
function timingFor(spanFrames, story, energy, camera, curve, rng) {
const tension = story ? story.tension : 0.5;
const urgency = clamp01(energy * 0.6 + tension * 0.4);
// Fraction of the shot spent moving. Fast material arrives in the first
// third; slow material is still arriving at the cut.
let travel = lerp(1.15, 0.28, urgency) * camera.pace * rng.range(0.8, 1.25);
// A drift is defined by not arriving, so it always outruns its shot.
if (curve === 'drift') travel = Math.max(travel, 1.2);
// How long it holds first. A `deliberate` camera earns its name here: the
// hold is what makes the move that follows read as a decision.
const delay = lerp(0.22, 0.02, urgency) * rng.range(0.4, 1.3);
const usable = Math.max(1, spanFrames);
return {
delayFrames: Math.round(clamp01(delay) * usable),
travelFrames: Math.max(1, Math.round(travel * usable)),
};
}
/**
* Whether the cut RELOCATES the camera or the camera walks through it.
*
* Both are real edits and they say opposite things. A cut that lands on a new
* part of the scene is a reframe — the loud one, the one you notice. A cut the
* camera walks through is a match cut, and it is what makes two scenes read as
* one continuous place, which is the effect worth keeping.
*
* So this is a decision per cut, not a mode. Hard cuts relocate, dissolves
* mostly do not, and tension raises the odds everywhere: a wound-up section
* jumps around inside itself, a resolution stops doing that.
*/
function relocatesAt(cue, story, rng) {
if (!cue.hardCut && cue.atSectionStart) return true; // a new section is a new place
const tension = story ? story.tension : 0.5;
let chance = 0.12 + tension * 0.45;
if (cue.hardCut) chance += 0.3;
if (story) {
if (story.act === 'climax') chance += 0.2;
else if (story.act === 'resolution') chance *= 0.4;
}
return rng.bool(clamp01(chance));
}
/**
* Plan the gaze across a whole video.
*
* Walks the cues in order. Each move starts from where the gaze ACTUALLY was
* when the cut arrived — not from the target of the previous move, which may
* never have been reached: a `drift` is defined by outrunning its shot, and
* resuming from its unreached target is a silent teleport at every cut. That
* was measurable as a median jump of 0.000 with a p90 of 0.274, which is a
* camera that mostly does nothing and occasionally lurches.
*
* @param {Array} cues ArcDriver cues, each already carrying `framing`
* @param {Array} sections look.sections, for bias and story state
* @param {object} camera from deriveCamera
* @param {Rng} rng
* @returns {Array} one move per cue, index-aligned
*/
export function planGaze(cues, sections, camera, rng) {
const moves = [];
let at = [0, 0];
let previousDir = null;
for (const cue of cues) {
const section = sections[cue.sectionIndex] || {};
const story = section.story || null;
const energy = (section.bias && section.bias.energy) || 0;
const scale = (cue.framing && cue.framing.scale) || 1;
const reach = reachFor(scale, camera);
const span = Math.max(1, cue.endFrame - cue.startFrame);
const curve = rng.pickWeighted(CURVE_NAMES, camera.curves);
// THE CUT. Either the camera is somewhere new when the image changes,
// or it walks through the change and the two shots read as one place.
//
// Carried-over positions are re-clamped, because reach belongs to the
// SHOT: walking a close-up's 0.45 offset into a wide would start that
// wide further off centre than a wide is ever allowed to be. The small
// snap this causes lands exactly on a cut, which is where the eye is
// least able to see it.
let from = clampToReach(at, reach);
if (moves.length && relocatesAt(cue, story, rng)) {
const hop = jumpFor(story, !!cue.hardCut, rng) * reach;
const jumped = targetFor(from, hop, camera, story, rng, previousDir);
from = clampToReach(jumped.point, reach);
previousDir = jumped.dir;
}
// THE MOVE. Where it travels during the shot, from wherever the cut
// left it. Deliberately smaller than the cut's hop — the reframe is the
// statement and the move is the camera living inside it.
const distance = jumpFor(story, false, rng) * reach * 0.7;
const { point, dir } = targetFor(from, distance, camera, story, rng, previousDir);
// Clamp the TARGET to the reach disc rather than the step, so a move
// that would leave the frame is shortened instead of being redirected —
// redirecting is what makes a bounded random walk orbit its boundary.
const to = clampToReach(point, reach);
const { delayFrames, travelFrames } = timingFor(span, story, energy, camera, curve, rng);
const move = { from, to, curve, delayFrames, travelFrames, reach };
moves.push(move);
// Where the gaze actually ends up when this shot is cut away from.
at = gazeAt(move, span);
previousDir = dir;
}
return moves;
}
function clampToReach(p, reach) {
const d = Math.hypot(p[0], p[1]);
if (d <= reach || d === 0) return p;
const k = reach / d;
return [p[0] * k, p[1] * k];
}
/**
* Where the gaze is, `frames` into a cue.
*
* Pure in (move, frames): this is what keeps a seek frame-identical to
* playback, and it is why the plan holds points and durations rather than a
* running position.
*/
export function gazeAt(move, frames) {
if (!move) return [0, 0];
const t = clamp01((frames - move.delayFrames) / move.travelFrames);
const eased = CURVES[move.curve] ? CURVES[move.curve](t) : t;
const p = [
lerp(move.from[0], move.to[0], eased),
lerp(move.from[1], move.to[1], eased),
];
// The reach bound is enforced HERE and not only on the endpoints, because a
// curve is allowed to leave the segment between them: `settle` overshoots
// its target by about 10% on purpose. Clamping the plan is not the same as
// clamping the path, and only the path is what reaches the screen.
return move.reach ? clampToReach(p, move.reach) : p;
}
/** One line for the HUD, the look panel and check output. */
export function describeCamera(camera) {
if (!camera) return 'camera: locked';
return `camera: ${camera.name} (reach ${camera.reach.toFixed(2)}, `
+ `pace ${camera.pace.toFixed(2)})`;
}

464
flow-state/src/look/Identity.js Normal file
View File

@ -0,0 +1,464 @@
// The song's IDENTITY: the content it is made of, and the hand it is drawn in.
//
// This is the Epic 3 inversion. A scene used to be self-contained — it knew how
// to make metaballs and needed nothing from the track but a palette — and the
// personality could only ever be a set of modifiers layered on an image the
// shader already had. Measured, that is why two songs came out about as
// different from each other as one video is from itself: the generator was
// varying the CONTAINER and never the content.
//
// So the song generates content first, and a stage is a way of arranging
// content it is given.
//
// CAST what is on screen. A protagonist and a chorus, as actual forms with
// sides, notches and hollows, not as hints a shader may consult.
// INK how they are drawn. Weight, edge, fill treatment, outline,
// posterisation — the hand, which changes every pixel of every stage
// at once and does it structurally rather than chromatically.
//
// Both travel as uniforms, which makes them data rather than code and means any
// stage can consume them without knowing anything about any other stage. The
// rule that keeps this honest, from EPIC-3.md §2:
//
// An artifact is content a stage could not have invented for itself.
//
// A stage that renders acceptably with the cast replaced by a default is using
// it as a modifier and will drift back into ignoring it, exactly the way most of
// the library ignores u_sigSides today.
/**
* Solids a 3D cast member is assembled from, as the part's kind index.
*
* `prism` is the 2D cast profile extruded — the protagonist given a body — and
* it is deliberately first, because a form built only from library primitives
* would be a shape the song did not choose. The other four are what a profile
* cannot be: something that bulges, something that tapers, something with a
* hole you can see through from an angle.
*/
export const SOLIDS = ['prism', 'box', 'capsule', 'torus', 'sphere'];
/**
* How the parts repeat, as the shader's `u_formSym` index.
*
* This is the load-bearing half of the assembly. Parts unioned at random
* positions read as debris; the same parts under a symmetry read as DESIGNED,
* and a designed object is the only kind worth calling a protagonist. It is
* also what keeps the form recognisable from any angle, which is the whole
* reason for giving it a third dimension.
*/
export const SYMMETRIES = ['none', 'mirror', 'radial', 'stack'];
/** Booleans a part can join with, as the part's op index. */
export const FORM_OPS = ['union', 'blend', 'carve'];
/** How many parts an assembly can have. The shader loops to exactly this. */
export const MAX_FORM_PARTS = 6;
/** Fill treatments, as the shader's `u_inkFill` index. */
export const FILLS = ['flat', 'ramp', 'hatch', 'stipple', 'halftone', 'hollow'];
/** Lattices, as the shader's `u_latKind` index. */
export const LATTICES = ['grid', 'radial', 'spiral', 'scatter', 'strata'];
/**
* How the song's subject IMPACTS a field, as the shader's `u_impact` index.
*
* A pattern spread evenly over the screen has nothing to watch: no subject, so
* nowhere for the eye to rest and no way to tell one song's version from
* another's. Warping the field around a focal point was the first attempt and it
* changed the cells without changing where the frame's energy was, which is the
* same thing as still having no subject.
*
* So the subject is a form that does something TO the field, and which thing is
* the song's decision — the same form punching a hole, bending the pattern, or
* running it at a different rate are three different videos.
*/
export const IMPACTS = ['shift', 'warp', 'punch', 'morph', 'overlay'];
const clamp01 = (x) => Math.max(0, Math.min(1, x));
/**
* One cast member.
*
* `notches` and `hollow` are what take this past the existing signature form.
* A rounded pentagon is a shape; a pentagon with six notches cut into it and a
* hole through the middle is a CHARACTER — recognisable across stages, which is
* the entire point of sharing it.
*/
function castMember(rng, { angular, intricate, solid }) {
const sides = rng.pickWeighted(
[0, 3, 4, 5, 6, 8],
[3 + (1 - angular) * 5, 1 + angular * 2, 2 + angular * 2,
1 + angular * 2, 1.5 + angular * 2, 0.5 + angular * 1.5]);
return {
sides,
round: clamp01(rng.range(0.05, 0.55) * (1.3 - angular * 0.6)),
elong: rng.range(0.8, 1.5),
tilt: rng.range(0, Math.PI),
// A notched form reads as made rather than as found.
notchCount: rng.bool(0.25 + intricate * 0.5)
? rng.pickWeighted([3, 4, 5, 6, 8, 12], [2, 3, 2, 3, 2, 1]) : 0,
notchDepth: rng.range(0.06, 0.1 + intricate * 0.22),
// Hollow forms are the difference between a blob library and a
// recognisable one, and they cost nothing to draw.
hollow: rng.bool(0.45 - solid * 0.3) ? rng.range(0.15, 0.6) : 0,
};
}
/**
* The protagonist as a SOLID: a small assembly of parts, joined by booleans
* under a symmetry.
*
* The 2D cast is a silhouette, and a silhouette is the same picture from every
* angle — which means a scene that turns one is not showing you anything new,
* it is showing you the same outline rotated. That is the ceiling this exists
* to lift: an assembly's outline CHANGES as it turns, so a shot of it has
* somewhere to go over eight bars without the scene inventing motion.
*
* It is content rather than a modifier by the EPIC-3 §4 test: a stage handed a
* default assembly draws a plain extruded profile, and no stage could have
* invented "a five-fold radial of carved prisms with a torus through it" for
* itself. The parts stay tied to the 2D cast — `prism` parts ARE the song's
* profile — so the solid and the silhouette are the same character rather than
* two unrelated generators running side by side.
*/
function generateForm(rng, { angular, intricate, solid }) {
const count = rng.pickWeighted([2, 3, 4, 5, 6],
[3, 3 + intricate, 1.5 + intricate * 3, 0.5 + intricate * 3, 0.2 + intricate * 2]);
const symmetry = rng.pickWeighted(SYMMETRIES, [
1, // none — rare, and it shows
2 + angular, // mirror
2 + (1 - angular) * 2, // radial
1 + angular * 1.5, // stack
]);
// A radial fold of 2 is a mirror by another name, and a stack of 6 is a
// column rather than an object, so the two symmetries want different counts.
const symmetryN = symmetry === 'radial' ? rng.int(3, 8) : rng.int(2, 5);
const parts = [];
for (let i = 0; i < count; i++) {
// The first part is the body and always positive: an assembly whose
// opening move is a subtraction has nothing to subtract from.
const op = i === 0 ? 'union' : rng.pickWeighted(FORM_OPS, [
2, // union
1 + (1 - angular) * 2.5, // blend — smooth, and the soft look
0.6 + intricate * 2, // carve — holes, and the made look
]);
const kind = rng.pickWeighted(SOLIDS, [
// The profile leads, so the solid keeps the song's own outline.
4,
1 + angular * 2, // box
1 + (1 - angular) * 1.5, // capsule
0.6 + intricate * 1.6, // torus
1 + (1 - angular), // sphere
]);
// Parts near the origin build a body; parts far out build limbs. The
// first one is centred so there is always something at the middle.
const reach = i === 0 ? 0 : rng.range(0.15, 0.85) * (0.6 + intricate * 0.7);
const dir = rng.range(0, Math.PI * 2);
const size = (i === 0 ? rng.range(0.55, 0.95) : rng.range(0.2, 0.6))
* (1.15 - intricate * 0.35);
parts.push({
kind, op,
offset: [Math.cos(dir) * reach, rng.range(-0.7, 0.7) * reach, Math.sin(dir) * reach],
scale: [size, size * rng.range(0.6, 1.5), size * rng.range(0.35, 1.2)],
yaw: rng.range(0, Math.PI * 2),
pitch: rng.range(-0.9, 0.9),
// Rounding the part's own surface, on top of the ink's edge. A
// solid track wants blunt parts; a dynamic one wants sharp ones.
round: rng.range(0, 0.35) * (0.4 + solid * 1.2),
});
}
return {
parts, symmetry, symmetryN,
// THE CHORUS SOLID: the protagonist's body plan, simplified.
//
// A relative rather than a stranger, for the same reason the 2D chorus
// is: a frame full of both has to read as one production. So it is the
// first few parts of the same assembly under its own symmetry and its
// own proportions — which is what a supporting character IS, structurally.
//
// It costs no extra part rows. A second full assembly would have doubled
// the uniform array for something that must not look like a different
// object anyway, and "fewer parts, squashed differently" is both cheaper
// and a better description of the thing.
chorus: {
count: Math.min(parts.length, rng.int(1, 3)),
symmetry: rng.bool(0.55) ? symmetry : rng.pick(['none', 'mirror']),
symmetryN: rng.int(2, 5),
// Squashed and thinned against the protagonist. A chorus that is
// merely a smaller protagonist adds numbers and no information.
flat: rng.range(0.5, 1.7),
thin: rng.range(0.45, 1.5),
},
// How far a `blend` op melts two parts into one. Low reads as welded
// hard edges, high as a single lump — both are legible, and the middle
// is where an assembly stops looking like parts at all.
blend: rng.range(0.04, 0.3) * (1.4 - angular * 0.8),
// How deep the solid is relative to how wide. A track can be built on
// slabs or on columns, and that decision is visible before anything else.
depth: rng.range(0.45, 1.6),
};
}
/**
* @param {object} summary FeatureTrack summary
* @param {Rng} rng
* @param {number} sections
*/
export function generateIdentity(summary, rng, sections = 4) {
const bright = summary.meanCentroid ?? 0.5;
const noisy = Math.min(1, (summary.meanFlatness ?? 0.2) * 3);
const fast = clamp01(((summary.bpm ?? 120) - 80) / 80);
const dynamic = clamp01(summary.dynamicRange ?? 0.5);
const busy = clamp01((sections - 2) / 5);
// The audio sets the centre of each decision and the seed picks within it —
// the same arrangement the personality values use, and for the same reason:
// deriving outright would buy coupling by destroying seed variety.
const angular = clamp01(noisy * 0.6 + fast * 0.3 + rng.range(-0.25, 0.25));
const intricate = clamp01(busy * 0.5 + bright * 0.3 + rng.range(-0.3, 0.3));
const solid = clamp01(0.5 - dynamic * 0.4 + rng.range(-0.25, 0.25));
const protagonist = castMember(rng.fork('protagonist'), { angular, intricate, solid });
// The chorus is a relative of the protagonist, not a stranger: it shares the
// family and differs in proportion, which is what makes a frame full of them
// read as one production rather than as two libraries stacked.
const chorusRng = rng.fork('chorus');
const chorus = {
...castMember(chorusRng, { angular, intricate, solid }),
sides: chorusRng.bool(0.6) ? protagonist.sides : chorusRng.pick([0, 3, 4, 6]),
tiltOffset: chorusRng.range(-0.6, 0.6),
};
const ink = {
// Line weight and edge hardness: the two decisions a viewer reads as
// "what this was drawn with".
weight: clamp01(0.15 + noisy * 0.35 + rng.range(-0.15, 0.35)),
edge: clamp01(0.3 + bright * 0.3 + rng.range(-0.3, 0.4)),
fill: rng.pickWeighted(FILLS, [
3, // flat
3, // ramp
1 + intricate * 3, // hatch
1 + noisy * 2.5, // stipple
1 + bright * 2.5, // halftone
1 + (1 - solid) * 2, // hollow — outline only
]),
hatchAngle: rng.range(0, Math.PI),
hatchScale: rng.range(40, 160) * (0.6 + intricate * 0.9),
outline: rng.bool(0.45 + angular * 0.3) ? rng.range(0.3, 1) : 0,
// Posterisation is a value-structure decision, and it is the cheapest
// way to make one track look printed and another look lit.
posterize: rng.bool(0.3) ? rng.int(3, 6) : 0,
};
// STAGING: where things go, and how their sizes are distributed.
//
// The first four stages separated songs almost entirely on layout while
// their feature SCALE collapsed — every stage placed similarly-sized
// elements, so size stopped being a variable at all. A shared lattice fixes
// both halves: the stages agree with each other about placement, which is
// what makes a video look like itself, and the size hierarchy becomes a
// decision the song makes rather than one each stage makes for itself.
const lattice = {
kind: rng.pickWeighted(LATTICES, [
1 + angular * 3, // grid
1 + (1 - angular) * 2, // radial
1 + (1 - angular) * 2, // spiral
2, // scatter
1 + angular * 2, // strata
]),
jitter: clamp01(rng.range(0, 0.5) + (1 - angular) * 0.3),
spread: rng.range(0.5, 0.85) + dynamic * 0.35,
// A few large and many small, or all one size. Busy material earns the
// hierarchy; a sparse track wants its elements to be equals.
scaleSpread: clamp01(0.15 + intricate * 0.6 + rng.range(-0.2, 0.25)),
// Whether the big ones sit in the middle or around the edges.
scaleBias: rng.range(-1, 1),
// How big the song's elements are AT ALL — a per-song decision rather
// than a per-stage one.
//
// This is the block the measurements kept pointing at. Sharing a cast
// lowered the floor as predicted, but `scale` — feature size — came out
// consistently WORSE than the legacy arm (0.033 against 0.053, well
// outside the noise), because every stage still chose its own element
// size from its own param range and every song therefore landed in the
// same place. A song made of six huge forms and a song made of four
// hundred tiny ones are different videos before anything else is
// decided; that decision belongs here.
elementScale: 0.35 * 2 ** rng.range(-1.4, 1.4) * (1.25 - intricate * 0.5),
// FOCUS: one to three points the field is disturbed around.
//
// A full-frame texture has no composition — measured, its layout
// distance between two songs is 0.004, which is nothing, because
// edge-to-edge content sits in the same place however its cells fall.
// That is not the metric failing; there genuinely is nothing to compose.
// A focal point gives the field somewhere to be about, and because the
// point moves with the song it gives the layout something to say.
focusCount: rng.pickWeighted([1, 2, 3], [4, 2, 1]),
focusRadius: rng.range(0.35, 1.1),
// Positive draws the field in and densifies it; negative opens a void.
focusPull: (rng.bool(0.65) ? 1 : -1) * rng.range(0.3, 1),
impact: rng.pick(IMPACTS),
};
// The solid the protagonist is, as opposed to the outline it casts. Forked
// rather than drawn inline so adding it does not shift every decision made
// after it — an identity generated today has to stay the identity it was.
const form = generateForm(rng.fork('form'), { angular, intricate, solid });
return {
cast: { protagonist, chorus }, ink, lattice, form,
character: { angular, intricate, solid },
};
}
/** Neutral values, so a layer built without an identity renders as it always did. */
export const NEUTRAL_IDENTITY_UNIFORMS = {
u_castSides: 0, u_castRound: 0.25, u_castElong: 1, u_castTilt: 0,
u_castNotchN: 0, u_castNotchD: 0, u_castHollow: 0,
u_chorusSides: 0, u_chorusRound: 0.25, u_chorusElong: 1, u_chorusTilt: 0,
u_chorusNotchN: 0, u_chorusNotchD: 0, u_chorusHollow: 0,
u_inkWeight: 0.3, u_inkEdge: 0.5, u_inkFill: 0, u_inkHatchAngle: 0,
u_inkHatchScale: 80, u_inkOutline: 0, u_inkPosterize: 0,
u_latKind: 3, u_latJitter: 0.5, u_latSpread: 0.9,
u_latScaleSpread: 0.3, u_latScaleBias: 0, u_latScale: 0.35,
u_focusN: 0, u_focusR: 0.6, u_focusPull: 0, u_impact: 0,
// A count of zero is the shader's instruction to fall back to the 2D cast
// extruded, so a scene that marches the solid still draws the right
// character when it is handed no identity at all.
u_formCount: 0, u_formSym: 0, u_formSymN: 3, u_formBlend: 0.12, u_formDepth: 0.8,
u_formChorusN: 0, u_formChorusSym: 0, u_formChorusSymN: 3,
u_formChorusFlat: 1, u_formChorusThin: 1,
u_formPart: Array.from({ length: MAX_FORM_PARTS * 3 }, () => [0, 0, 0, 0]),
};
/**
* The assembly, packed for the shader: three vec4 per part.
*
* A part is eleven numbers, and eleven scalar uniforms times six parts is
* sixty-six declarations nobody would keep in step with the generator. Packed
* rows are indexed by the loop counter instead, which is the one array access
* GLSL ES 1.0 allows and is why the layout is fixed-width rather than tight.
*
* row 0 offset.xyz | solid index
* row 1 scale.xyz | boolean op index
* row 2 yaw, pitch, round | unused
*/
function chorusUniforms(chorus) {
if (!chorus) {
return {
u_formChorusN: 0, u_formChorusSym: 0, u_formChorusSymN: 3,
u_formChorusFlat: 1, u_formChorusThin: 1,
};
}
return {
u_formChorusN: chorus.count,
u_formChorusSym: SYMMETRIES.indexOf(chorus.symmetry),
u_formChorusSymN: chorus.symmetryN,
u_formChorusFlat: chorus.flat,
u_formChorusThin: chorus.thin,
};
}
function formPartRows(form) {
const rows = [];
for (let i = 0; i < MAX_FORM_PARTS; i++) {
const p = form && form.parts[i];
if (!p) { rows.push([0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]); continue; }
rows.push([p.offset[0], p.offset[1], p.offset[2], SOLIDS.indexOf(p.kind)]);
rows.push([p.scale[0], p.scale[1], p.scale[2], FORM_OPS.indexOf(p.op)]);
rows.push([p.yaw, p.pitch, p.round, 0]);
}
return rows;
}
/**
* @param {object} identity
* @param {object} shape the personality's signature form
*
* The protagonist's GEOMETRY is the signature form, read live rather than
* copied at generation time. The two were separate decisions in the first
* draft, which meant a track built on hexagons could have a round protagonist —
* the signature form said one thing and the thing actually on screen said
* another, and the shape trait stopped meaning anything for stages. The cast is
* the signature form made concrete: same sides, same rounding, same tilt, plus
* the notches and hollows that turn a shape into a character.
*/
export function identityUniforms(identity, shape = null) {
if (!identity) return { ...NEUTRAL_IDENTITY_UNIFORMS };
const { protagonist: p, chorus: b } = identity.cast;
const a = shape ? {
...p,
sides: shape.sides,
round: shape.roundness,
elong: shape.elongation,
tilt: shape.tilt,
} : p;
const ink = identity.ink;
return {
u_castSides: a.sides, u_castRound: a.round, u_castElong: a.elong, u_castTilt: a.tilt,
u_castNotchN: a.notchCount, u_castNotchD: a.notchCount ? a.notchDepth : 0,
u_castHollow: a.hollow,
// The chorus stays a relative of the protagonist: it inherits the
// signature form's tilt and usually its sides.
u_chorusSides: b.sides, u_chorusRound: b.round, u_chorusElong: b.elong,
u_chorusTilt: a.tilt + b.tiltOffset,
u_chorusNotchN: b.notchCount, u_chorusNotchD: b.notchCount ? b.notchDepth : 0,
u_chorusHollow: b.hollow,
u_inkWeight: ink.weight, u_inkEdge: ink.edge,
u_inkFill: FILLS.indexOf(ink.fill),
u_inkHatchAngle: ink.hatchAngle, u_inkHatchScale: ink.hatchScale,
u_inkOutline: ink.outline, u_inkPosterize: ink.posterize,
u_latKind: LATTICES.indexOf(identity.lattice.kind),
u_latJitter: identity.lattice.jitter,
u_latSpread: identity.lattice.spread,
u_latScaleSpread: identity.lattice.scaleSpread,
u_latScaleBias: identity.lattice.scaleBias,
u_latScale: identity.lattice.elementScale,
u_focusN: identity.lattice.focusCount,
u_focusR: identity.lattice.focusRadius,
u_focusPull: identity.lattice.focusPull,
u_impact: IMPACTS.indexOf(identity.lattice.impact),
u_formCount: identity.form ? identity.form.parts.length : 0,
u_formSym: identity.form ? SYMMETRIES.indexOf(identity.form.symmetry) : 0,
u_formSymN: identity.form ? identity.form.symmetryN : 3,
u_formBlend: identity.form ? identity.form.blend : 0.12,
u_formDepth: identity.form ? identity.form.depth : 0.8,
u_formPart: formPartRows(identity.form),
// Defaulted rather than assumed. A check harness builds identities by
// hand to probe a scene, and reading through a missing sub-object here
// throws inside a uniform getter — where the only symptom is a page that
// never finishes and never says why. Cost of the guard: nothing.
...chorusUniforms(identity.form && identity.form.chorus),
};
}
const SHAPE_NAMES = { 0: 'round', 3: 'triangular', 4: 'square', 5: 'pentagonal', 6: 'hexagonal', 8: 'octagonal' };
export function describeIdentity(identity) {
if (!identity) return 'no identity';
const { protagonist: a, chorus: b } = identity.cast;
const form = (m) => `${SHAPE_NAMES[m.sides] || `${m.sides}-sided`}` +
`${m.notchCount ? `/${m.notchCount}-notch` : ''}${m.hollow ? '/hollow' : ''}`;
const ink = identity.ink;
const solid = identity.form
? ` · solid ${identity.form.parts.length}-part/${identity.form.symmetry}` +
`${identity.form.symmetry === 'none' ? '' : identity.form.symmetryN}`
: '';
return `cast ${form(a)} + ${form(b)}${solid} · ink ${ink.fill}` +
`${ink.outline ? '+outline' : ''}${ink.posterize ? `/${ink.posterize}-tone` : ''}` +
` w${ink.weight.toFixed(2)} · on ${identity.lattice.kind}` +
` · ${identity.lattice.focusCount} focus/${identity.lattice.impact}` +
` at ${identity.lattice.elementScale < 0.2 ? 'tiny' :
identity.lattice.elementScale > 0.6 ? 'huge' : 'mid'} scale`;
}

750
flow-state/src/look/LookGenerator.js
View File

@ -5,15 +5,26 @@
// the decoded audio, so a given file always renders the same video. // the decoded audio, so a given file always renders the same video.
import { Rng, hashSamples } from '../engine/rng.js'; import { Rng, hashSamples } from '../engine/rng.js';
import { AudioPalette, generateUsablePalette } from './palette.js'; import { AudioPalette, generateUsablePalette, rgbToOklch } from './palette.js';
import { scenes, scenesInFamily, FAMILIES } from '../scenes/registry.js'; import { scenes, scenesInFamily, FAMILIES } from '../scenes/registry.js';
import { sampleValues, defaultValues } from '../params/schema.js'; import { sampleValues, defaultValues, canBackground } from '../params/schema.js';
import {
canGround, surfaceOf, structuralDistance, groundBiasFrom, groundTemperamentFrom,
coverageOf as sceneCoverage, GROUND_MIN,
} from '../scenes/surface.js';
import { GROUND, subjectOf } from './stack.js';
import { planShots } from './shots.js'; import { planShots } from './shots.js';
import { generatePersonality, sceneHonours, describePersonality } from './Personality.js'; import {
generatePersonality, sceneHonours, signatureWeight, describePersonality,
} from './Personality.js';
import { deriveGrain, describeGrain, applyGrainToPost } from './grain.js'; import { deriveGrain, describeGrain, applyGrainToPost } from './grain.js';
import { pickDirector, directorByName } from './directors.js'; import {
pickDirector, directorByName, crowdOf, blazeOf, RESTFUL_FAMILIES, QUIET_KINDS,
} from './directors.js';
import { derivePaletteArc, describePaletteArc } from './paletteArc.js'; import { derivePaletteArc, describePaletteArc } from './paletteArc.js';
import { deriveFramingStyle, describeFraming } from './framing.js'; import { deriveFramingStyle, describeFraming } from './framing.js';
import { deriveCamera, describeCamera } from './Camera.js';
import { deriveStory, storyForSection, NEUTRAL_STATE } from './Story.js';
// Which families suit which section kind now comes from the track's DIRECTOR // Which families suit which section kind now comes from the track's DIRECTOR
// (look/directors.js) rather than from a constant here. The coupling it // (look/directors.js) rather than from a constant here. The coupling it
@ -33,10 +44,18 @@ const KIND_ENERGY = {
* and `energy` up; a breakdown pulls them down. Seed variation still dominates, * and `energy` up; a breakdown pulls them down. Seed variation still dominates,
* so two tracks with the same structure do not converge on the same look. * so two tracks with the same structure do not converge on the same look.
*/ */
function biasFor(section, summary) { function biasFor(section, summary, motion = null, story = null) {
const kindEnergy = KIND_ENERGY[section.kind] ?? 0.5; const kindEnergy = KIND_ENERGY[section.kind] ?? 0.5;
const measured = Math.min(1, section.energy / Math.max(1e-6, summary.meanLoudness * 1.6)); const measured = Math.min(1, section.energy / Math.max(1e-6, summary.meanLoudness * 1.6));
const energy = kindEnergy * 0.6 + measured * 0.4; // Where this section sits in the story moves the bias, and is deliberately
// the smallest term in it. The kind decides what a section IS — the spread
// between an intro and a drop is 0.7 of the range — and the story decides
// which drop this is, worth a tenth of that. Bounded rather than trusted:
// a breakdown at maximum tension is still, unambiguously, a breakdown, and
// the quiet-kind coupling in directors.js depends on it staying that way.
const tension = story ? story.tension : 0.5;
const population = story ? story.population : 0.5;
const energy = clamp01(kindEnergy * 0.6 + measured * 0.4 + (tension - 0.5) * 0.16);
// 60bpm → 0, 180bpm → 1. Tempo, not energy, is what a viewer reads as // 60bpm → 0, 180bpm → 1. Tempo, not energy, is what a viewer reads as
// "this is moving too fast for the song": a slow track can have a huge drop // "this is moving too fast for the song": a slow track can have a huge drop
@ -45,19 +64,118 @@ function biasFor(section, summary) {
// biased to 0.9 motion and scenes that skittered over it. // biased to 0.9 motion and scenes that skittered over it.
const tempo = clamp01((summary.bpm - 60) / 120); const tempo = clamp01((summary.bpm - 60) / 120);
// The track's motion CHARACTER, on top of its tempo. Tempo alone compresses
// — 124 and 138bpm are the same number to a viewer — and motion was the
// weakest axis in every measurement because it was the only lever. Stillness
// is allowed to halve the animation rate or half again raise it, which is a
// difference anyone can see, and it is a property of the track rather than
// of the section. See look/Personality.js.
const still = motion ? motion.stillness : 0.5;
const churn = motion ? motion.churn : 0.25;
return { return {
energy, energy,
density: Math.min(1, energy * 0.7 + section.flux * 1.2), density: clamp01(energy * 0.7 + section.flux * 1.2 + (population - 0.5) * 0.2),
motion: clamp01(0.12 + tempo * 0.55 + energy * 0.28), motion: clamp01(0.12 + tempo * 0.4 + energy * 0.2 + (1 - still) * 0.35),
// Applied on top of every `rate: true` param, so absolute animation // Applied on top of every `rate: true` param, so absolute animation
// speed scales with the song rather than only its sampled position in // speed scales with the song rather than only its sampled position in
// a range. Bounded well short of a stop or a blur. See params/schema.js. // a range. Bounded well short of a stop or a blur. See params/schema.js.
rateScale: 0.45 + tempo * 0.95, rateScale: (0.45 + tempo * 0.95) * (1.35 - still * 0.75) * (1 + churn * 0.25),
}; };
} }
const clamp01 = (x) => Math.max(0, Math.min(1, x)); const clamp01 = (x) => Math.max(0, Math.min(1, x));
/**
* Derive the set of palettes for this track. The director's palette.count
* names how many distinct schemes the video will cut between; the actual
* colours stay audio-tilted (same warmth/energy) so they read as one track's
* world rather than as unrelated palettes shuffled together.
*/
function derivePaletteSet(summary, rng, director) {
const cfg = (director && director.palette) || {};
const spec = cfg.count;
let count = 1;
if (Array.isArray(spec)) {
const lo = Math.max(1, spec[0] | 0);
const hi = Math.max(lo, spec[1] | 0);
count = lo === hi ? lo : rng.int(lo, hi);
} else if (typeof spec === 'number') {
count = Math.max(1, spec | 0);
} else {
// No palette config — single palette, like before
count = 1;
}
count = Math.min(4, Math.max(1, count));
const palettes = [];
const paletteSchemes = [];
const hueRefs = [];
for (let k = 0; k < count; k++) {
let palette = null;
let scheme = null;
let hue = 0;
// Try a few forks so the set is diverse in scheme and hue, not just
// repeated draws that happen to land on the same scheme.
for (let attempt = 0; attempt < 12; attempt++) {
const fork = rng.fork(`palette:${k}:${attempt}`);
const src = new AudioPalette(summary, fork);
const cand = generateUsablePalette(src, 6);
const candScheme = src.lastScheme || 'unknown';
let candHue = 0;
try { candHue = rgbToOklch(cand[0])[2]; } catch { candHue = fork.range(0, Math.PI * 2); }
const schemeDup = paletteSchemes.includes(candScheme);
let tooClose = false;
for (const h2 of hueRefs) {
let dh = Math.abs(candHue - h2) % (2 * Math.PI);
if (dh > Math.PI) dh = 2 * Math.PI - dh;
if (dh < 0.5) { tooClose = true; break; }
}
// Keep trying while we have attempts left and the candidate would
// make the set harder to tell apart. Relax after a few tries.
if (attempt < 8 && schemeDup) continue;
if (attempt < 6 && tooClose) continue;
palette = cand;
scheme = candScheme;
hue = candHue;
break;
}
if (!palette) {
const src = new AudioPalette(summary, rng.fork(`palette:${k}:final`));
palette = generateUsablePalette(src, 6);
scheme = src.lastScheme || 'unknown';
try { hue = rgbToOklch(palette[0])[2]; } catch { hue = 0; }
}
palettes.push(palette);
paletteSchemes.push(scheme);
hueRefs.push(hue);
}
return { palettes, paletteSchemes };
}
/**
* The track's temperament, moved to where this section sits in the story.
*
* Temperament is the hand on every parameter dial and it was constant for the
* whole video, which is why two occurrences of a kind sampled around the same
* point however far apart they were. Scaling `extremity` by tension is the
* ratchet: the same scene, sampled nearer the ends of its own ranges the later
* it appears. Bounded by the range extremity is drawn from — this moves where a
* track sits inside its own character, it does not give it a different one.
*/
function temperamentFor(temperament, state) {
if (!temperament || !state) return temperament;
const tension = state.tension;
return {
...temperament,
intensity: Math.max(-1, Math.min(1, temperament.intensity + (tension - 0.5) * 0.5)),
extremity: clamp01(temperament.extremity * (0.82 + tension * 0.4)),
};
}
/** /**
* Scenes eligible for a section kind, weighted by how well the family fits. * Scenes eligible for a section kind, weighted by how well the family fits.
* *
@ -66,26 +184,90 @@ const clamp01 = (x) => Math.max(0, Math.min(1, x));
* on is not a worse choice, it is the shot that was clearly filmed somewhere * on is not a worse choice, it is the shot that was clearly filmed somewhere
* else. See look/Personality.js. * else. See look/Personality.js.
*/ */
function candidatesForKind(kind, used, signature = [], director) { /**
* The scenes THIS TRACK is allowed to cast from — a seeded subset of the
* library, not the whole thing.
*
* There is a real tension here, and the first attempt at fixing the signature
* gate walked straight into it. The hard trait filter was doing two jobs at
* once: it was collapsing the library onto eleven over-declared scenes, which
* was the bug, and it was also giving each track a DIFFERENT pool to cast from,
* which was load-bearing. Replacing it with a soft weight fixed the collapse and
* removed the differentiation — every track then drew from the same weighted
* library, and measured song separation went from 0.03 to -0.15. Two songs came
* out more alike than before.
*
* So the differentiation is kept and its bias removed. Every track gets its own
* pool of about a third of the library, sampled without replacement, weighted by
* the signature so the track still has a point of view. What changes is that a
* scene declaring two traits is now merely less likely to be drawn than one
* declaring four, instead of being ineligible for six tracks in seven.
*/
/**
* How many of the library's scenes one track is allowed to draw on.
*
* Swept directly — checks.html?sweep=1 — across 4, 8, 16 and 32, over twelve
* songs with three pool draws each. The answer is that it does not matter:
*
* pool 4 spread +0.0061 ±0.0035
* pool 8 spread +0.0090 ±0.0040
* pool 16 spread +0.0102 ±0.0046
* pool 32 spread +0.0035 ±0.0032
*
* The differences are the same size as the run-to-run noise. This corrects a
* claim made when the Epic 3 arms first came in: those arms appeared to show
* that a small roster was the largest available win, but they varied two things
* at once — the pool was smaller AND it was the same pool for every song — and
* the sweep isolating size finds nothing.
*
* So 8 is chosen on grounds the metric cannot see. It puts about nine distinct
* scenes in a video rather than seventeen, and a video a viewer can hold in
* their head is worth having even when the instrument is indifferent.
*/
export const POOL_SIZE = 8;
function castingPool(rng, signature, size = POOL_SIZE) {
const pool = scenes.filter(canBackground);
const remaining = pool.slice();
const weights = remaining.map((m) => signatureWeight(m, signature));
const picked = [];
const target = Math.min(size, remaining.length);
while (picked.length < target && remaining.length) {
const chosen = rng.pickWeighted(remaining, weights);
const at = remaining.indexOf(chosen);
remaining.splice(at, 1);
weights.splice(at, 1);
picked.push(chosen);
}
return picked;
}
function candidatesForKind(kind, used, signature = [], director, pool = null) {
const families = director.families[kind] || Object.keys(FAMILIES); const families = director.families[kind] || Object.keys(FAMILIES);
const allowed = pool ? new Set(pool.map((m) => m.name)) : null;
const candidates = []; const candidates = [];
for (const family of families) { for (const family of families) {
const inFamily = scenesInFamily(family) const inFamily = scenesInFamily(family)
.filter((m) => m.role !== 'accent' && sceneHonours(m, signature)); .filter((m) => canBackground(m) && (!allowed || allowed.has(m.name)));
// Weight by family preference order, and push down anything already // Weight by family preference order, and push down anything already
// used so a five-section track doesn't show one scene five times. // used so a five-section track doesn't show one scene five times.
const weight = families.length - families.indexOf(family); const weight = families.length - families.indexOf(family);
for (const scene of inFamily) { for (const scene of inFamily) {
candidates.push({ scene, weight: weight * (used.has(scene.name) ? 0.15 : 1) }); candidates.push({
scene,
// The signature is a lean now, not a wall. See
// Personality.signatureWeight for why it had to stop being one.
weight: weight * signatureWeight(scene, signature)
* (used.has(scene.name) ? 0.15 : 1),
});
} }
} }
if (!candidates.length) { if (!candidates.length) {
// Every family for this kind was emptied by the signature filter. Widen // This track's pool holds nothing in the families the director wants for
// to the whole library, still honouring the signature; only if that is // this kind. Widen to the pool, then to the library — the track keeps
// empty too does the personality lose and the video keep its scenes. // its scenes either way.
const anywhere = scenes.filter((m) => m.role !== 'accent' && sceneHonours(m, signature)); const fallback = (pool && pool.length) ? pool : scenes.filter(canBackground);
const pool = anywhere.length ? anywhere : scenes.filter((m) => m.role !== 'accent'); return fallback.map((scene) => ({ scene, weight: signatureWeight(scene, signature) }));
return pool.map((scene) => ({ scene, weight: 1 }));
} }
return candidates; return candidates;
} }
@ -94,7 +276,7 @@ function candidatesForKind(kind, used, signature = [], director) {
* How many stage visuals a kind rotates between. Busy material takes more. * How many stage visuals a kind rotates between. Busy material takes more.
* *
* Sized against the library rather than picked out of the air: a kind draws * Sized against the library rather than picked out of the air: a kind draws
* from three families, which is seven to nine non-accent scenes, so a roster of * from three families, which is seven to nine castable scenes, so a roster of
* four still leaves the weighting room to avoid what other kinds already took. * four still leaves the weighting room to avoid what other kinds already took.
* Variants a section never reaches cost nothing — layers are built per cue, so * Variants a section never reaches cost nothing — layers are built per cue, so
* only the ones its shots actually show are ever compiled. * only the ones its shots actually show are ever compiled.
@ -103,6 +285,120 @@ function rosterSizeFor(kind) {
return (KIND_ENERGY[kind] ?? 0.5) > 0.5 ? 4 : 3; return (KIND_ENERGY[kind] ?? 0.5) > 0.5 ? 4 : 3;
} }
/**
* The most painted frame a stack is allowed to add up to.
*
* 1.0 is one filled picture. 2.0 is two of them stacked, which is where the
* compositor's blends stop producing depth and start producing mud — past it
* the layers are no longer readable as separate things, so nothing is gained
* by the third pass except cost.
*/
const MAX_COVERAGE = 2.0;
/**
* How full THIS section's frame is allowed to get, in painted coverage.
*
* Three inputs, in the order they matter:
*
* the director — how much this point of view lets happen at once. A
* brutalist video is one large thing everywhere in it; a
* corrupt one is everything over everything. See crowdOf.
* the song — a loud, dense section carries more than a quiet one.
* the stage — where the story is. `population` is literally how crowded
* this point in the video wants to be, and layering is the
* one lever on it that needs no cooperation from the scenes.
*
* The floor is GROUND_MIN because the ground is not optional: a section always
* pays for its bed first, and the budget governs what may be stacked on it.
*/
function coverageBudgetFor(bias, story, director) {
const stage = story ? story.population * 0.6 + story.tension * 0.4 : 0.5;
const want = 0.6 + bias.energy * 0.5 + bias.density * 0.2 + (stage - 0.5) * 0.5;
return Math.max(GROUND_MIN, Math.min(MAX_COVERAGE, want * crowdOf(director)));
}
/** The budget a KIND is planned against, before a section's own bias exists. */
function kindBudget(kind, director) {
return coverageBudgetFor(
{ energy: KIND_ENERGY[kind] ?? 0.5, density: 0.5 }, null, director);
}
/**
* The GROUND a kind's sections stand on: a canvas that paints at least half the
* frame, cast once per kind so a section's cuts change the shot without moving
* the video to another world.
*
* Most of the library cannot do this job and is not supposed to — two thirds of
* it is composable, which means it reads as elements ON something and has
* nothing of its own behind them. Those scenes were being cast as backgrounds
* anyway, which is why a section could be a few bright things on black for
* ninety seconds. The ground is what they are on.
*
* Chosen against the kind's budget rather than at random: a scene that paints
* 98% of the frame is a legitimate ground for a drop and the wrong bed for an
* intro, because everything the intro puts on it has to remain visible.
*/
function castGround(kind, roster, rng, signature, director, used) {
const pool = scenes.filter(canGround);
if (!pool.length) return null;
const families = director.families[kind] || Object.keys(FAMILIES);
const quiet = QUIET_KINDS.includes(kind);
// What the shots standing on it will paint, so the ground leaves room for
// the section it is under.
const reserve = roster.length
? roster.reduce((sum, m) => sum + sceneCoverage(m), 0) / roster.length
: 0.2;
const headroom = kindBudget(kind, director) - reserve;
const weights = pool.map((m) => {
const at = families.indexOf(m.family);
// Off-family grounds stay reachable — the ground is a bed, not the
// director's statement — but the director still leads.
let w = at >= 0 ? families.length - at : 0.35;
// The quiet-kind rule applies to the floor as well. An intro standing on
// a strobing glitch canvas is the mistake that rule exists to prevent,
// and it is worse underneath than on top because nothing hides it.
if (quiet && !RESTFUL_FAMILIES.includes(m.family)) w *= 0.15;
w *= signatureWeight(m, signature);
// Overshooting the budget is allowed and discouraged: the ground is
// mandatory, so an oversized one is spent frame the shot cannot use.
w /= 1 + 4 * Math.max(0, sceneCoverage(m) - headroom);
// The bed has to be unlike the things standing on it, or the section is
// one texture at double density. Measured against the whole roster,
// because every member of it will be shot against this ground.
w *= contrastWeight(m, roster.map((r) => ({ module: r })));
// A video returns to its world rather than visiting six of them.
if (used.has(m.name)) w *= 3;
// Nothing stands on itself. If the kind's own anchor is groundable it
// will be its own ground in buildStack, and this pick is for the rest.
if (roster.some((r) => r.name === m.name)) w *= 0.1;
return Math.max(1e-4, w);
});
const ground = rng.pickWeighted(pool, weights);
used.add(ground.name);
return ground;
}
/** One ground per section kind. See castGround. */
function assignGroundsByKind(rosterByKind, rng, signature, director) {
const grounds = new Map();
const used = new Set();
// Loud kinds first, for the same reason rosters are assigned that way: they
// are what the video is remembered for, so they choose their world first.
const priority = ['drop', 'sustain', 'build', 'breakdown', 'intro', 'outro'];
const kinds = [...rosterByKind.keys()]
.sort((a, b) => priority.indexOf(a) - priority.indexOf(b));
for (const kind of kinds) {
grounds.set(kind, castGround(
kind, rosterByKind.get(kind) || [], rng.fork(`ground:${kind}`),
signature, director, used));
}
return grounds;
}
/** /**
* Scenes are chosen per section KIND, not per section — and a kind gets a * Scenes are chosen per section KIND, not per section — and a kind gets a
* ROSTER of two or three, not one. * ROSTER of two or three, not one.
@ -116,7 +412,7 @@ function rosterSizeFor(kind) {
* Variation between two sections of the same kind comes from their parameter * Variation between two sections of the same kind comes from their parameter
* sets, from where their shots fall, and from the arc driver's drift. * sets, from where their shots fall, and from the arc driver's drift.
*/ */
function assignRostersByKind(sections, rng, signature = [], director) { function assignRostersByKind(sections, rng, signature = [], director, pool = null) {
const byKind = new Map(); const byKind = new Map();
const used = new Set(); const used = new Set();
@ -131,7 +427,7 @@ function assignRostersByKind(sections, rng, signature = [], director) {
const size = rosterSizeFor(kind); const size = rosterSizeFor(kind);
for (let slot = 0; slot < size; slot++) { for (let slot = 0; slot < size; slot++) {
const pool = candidatesForKind(kind, used, signature, director) const options = candidatesForKind(kind, used, signature, director, pool)
.filter((c) => !roster.includes(c.scene)) .filter((c) => !roster.includes(c.scene))
.map((c) => ({ .map((c) => ({
scene: c.scene, scene: c.scene,
@ -140,9 +436,10 @@ function assignRostersByKind(sections, rng, signature = [], director) {
// whole visual language. // whole visual language.
weight: c.weight * (roster.length && c.scene.family === roster[0].family ? 3 : 1), weight: c.weight * (roster.length && c.scene.family === roster[0].family ? 3 : 1),
})); }));
if (!pool.length) break; if (!options.length) break;
const chosen = rng.pickWeighted(pool.map((c) => c.scene), pool.map((c) => c.weight)); const chosen = rng.pickWeighted(
options.map((c) => c.scene), options.map((c) => c.weight));
roster.push(chosen); roster.push(chosen);
used.add(chosen.name); used.add(chosen.name);
} }
@ -152,6 +449,50 @@ function assignRostersByKind(sections, rng, signature = [], director) {
return byKind; return byKind;
} }
/**
* The RECAPITULATION: the outro re-casts what the intro opened on.
*
* The oldest device in the form and the cheapest one available here — the scene
* is already in the roster and already compiled, and all that changes is which
* member of it anchors. What makes it read as a return rather than as a repeat
* is that the outro plays it with the parameters the story has arrived at:
* the same scene, four minutes further along its slow axis, at the story's
* closing tension. See look/Story.js.
*/
function applyRecap(rosterByKind) {
const intro = rosterByKind.get('intro');
const outro = rosterByKind.get('outro');
if (!intro || !outro || !intro.length || !outro.length) return;
const opener = intro[0];
// Keep the outro's own roster behind the recapped anchor, minus a duplicate:
// the section still cuts away from it, it just opens and closes there.
const rest = outro.filter((m) => m.name !== opener.name);
rosterByKind.set('outro', [opener, ...rest]);
}
/**
* Which member of the kind's roster anchors THIS section.
*
* `roster[0]` opened every section of its kind, so a track's biggest visual was
* spent in the first fifteen seconds of the first drop and then spent again,
* identically, at every drop after it. Reserving it makes the anchor something
* the video arrives at: earlier occurrences open on a companion, and the
* anchor's own section is the one the story calls the climax.
*
* The roster itself does not change — the section still cuts between all of it,
* which is what keeps the kind's identity — only which member it opens on.
*/
function anchorOrder(roster, state) {
if (roster.length < 2 || !state) return roster;
// The climax, the resolution and any single occurrence get the real anchor.
const earned = state.act === 'climax' || state.act === 'resolution'
|| state.ordinalOf < 2 || state.reveal > 0.66;
if (earned) return roster;
const companion = 1 + (state.ordinal % (roster.length - 1));
return [roster[companion], ...roster.filter((_, i) => i !== companion)];
}
/** /**
* Post-processing and feedback derived from track character. * Post-processing and feedback derived from track character.
* Ambient material gets more feedback and bloom; dense club material gets * Ambient material gets more feedback and bloom; dense club material gets
@ -191,49 +532,212 @@ function derivePost(summary, rng, grain) {
} }
/** /**
* One layer stack: a background scene, sometimes a second scene composited over * How much a candidate would add to a stack, structurally.
* it, sometimes an accent on top of that.
* *
* Three deliberately different jobs: * The question a stack has to answer is not "are these two scenes different
* things" but "will a viewer see two things". Those come apart: the measured
* distance between two scenes' structural profiles is what a viewer reads, and
* it does not follow the family labels. So a candidate is weighted by how far
* it sits from everything already in the stack, taking the CLOSEST such
* distance — one near-twin in the stack is enough to make the addition read as
* more of the same, however unlike the other layers it is.
* *
* background — the shot. Always present, always opaque. * A lean, not a filter, and for the same reason the signature weighting is:
* overlay — a SECOND full scene at partial opacity. Not always: this is the * measured distances are a description of the library as it is today, and a
* generator that obeyed them exactly would cast the same handful of contrasts
* in every video. Unmeasured scenes score neutral rather than zero — never
* having been rendered is not evidence of sameness.
*/
const CONTRAST_NEUTRAL = 0.12;
function contrastWeight(candidate, stack) {
let closest = Infinity;
for (const layer of stack) {
const d = structuralDistance(candidate, layer.module);
if (d !== null) closest = Math.min(closest, d);
}
if (closest === Infinity) closest = CONTRAST_NEUTRAL;
// 0.02 apart (twins) → 0.25; 0.12 (typical) → 1.0; 0.30 (unalike) → 2.1.
return Math.max(0.15, Math.min(2.5, 0.15 + (closest / CONTRAST_NEUTRAL) * 0.85));
}
/**
* One layer stack: the ground, the shot standing on it, and sometimes a pass
* or two composited over both.
*
* ground — a canvas painting at least half the frame. Always present,
* always opaque, and usually NOT the scene the section is about:
* two thirds of the library is composable, and a composable
* scene on its own is a few bright things on black. It is the
* one layer the section does not choose freely — see castGround.
* When the shot is itself a full canvas it IS the ground, because
* two canvases stacked is two pictures fighting.
* shot — what the section is about. Screened over the ground rather
* than replacing it, so what it does not paint is the ground
* rather than black. `subjectOf` finds it; see look/stack.js.
* overlay — a composable scene at partial opacity. Not always: this is the
* variation valve, and a stack that always doubled up would read * variation valve, and a stack that always doubled up would read
* as permanently cluttered rather than as occasionally layered. * as permanently cluttered rather than as occasionally layered.
* Drawn from a different family so the two images argue instead * Drawn from a different family so the two images argue instead
* of blurring, and kept off scenes that are already busy. * of blurring, and kept off scenes that are already busy.
* accent — the depth pass. Mostly-empty by design (role: 'accent'),
* additive, low opacity.
* *
* Quiet material mostly goes without either — an intro is supposed to be sparse. * There used to be a third slot, `accent`, reserved for scenes declaring
* `role: 'accent'`. Exactly one scene ever declared it, and the overlay path
* above required a `composable` label no scene carried — so the reserved slot
* was the only layering that ever happened, and every layered stack in every
* song was the same particle field. One path, one roster: what goes on top is
* whatever is composable, which is now a third of the library.
*
* Quiet material mostly goes without any — an intro is supposed to be sparse.
*/ */
function buildStack(module, accentRoster, overlayRoster, bias, rng, temperament) { function buildStack(module, overlayRoster, bias, rng, temperament, story = null,
const layers = [{ { ground = null, director = null, kind = null } = {}) {
const sectionKind = kind;
// A shot that fills the frame by itself is its own ground; anything else
// gets one under it.
const standsAlone = canGround(module) || !ground;
// --- the blaze ------------------------------------------------------
// Whether THIS section is one the director lets bloom out: the shot added
// to its ground rather than keyed onto it, so the two brightnesses sum and
// the highlights go to paper.
//
// A decision, and a rationed one. Screening every shot over its ground is
// how a median quarter of every frame in every video ended up clipped —
// the effect was not wrong, being the default was. It has to be earned:
// the director's appetite, times a loud section, times a late point in the
// story. Quiet kinds never blaze; a breakdown that goes white is not a
// decision, it is a bug with a rationale.
const blaze = !standsAlone
&& !QUIET_KINDS.includes(sectionKind)
&& rng.bool(blazeOf(director) * clamp01(bias.energy * 1.2)
* (story ? 0.4 + story.tension * 0.9 : 0.7));
// The shot is sampled first and from the caller's rng, so a stack draws the
// same shot it always did and the ground arrives underneath it rather than
// in front of it in the seed stream.
const shot = {
module, module,
params: sampleValues(module, rng, bias, temperament), params: sampleValues(module, rng, bias, temperament),
seed: rng.int(0, 0x7fffffff), seed: rng.int(0, 0x7fffffff),
// Keyed over the ground on its own brightness by default: what the shot
// leaves unpainted is then the ground rather than black, which is the
// whole point of standing it on one, and what it DOES paint stays its
// own colour. 'screen' is the blaze — see above, and passes.js.
blend: standsAlone ? 'normal' : (blaze ? 'screen' : 'lumakey'),
// Carried so the HUD, the checks and a later pass over the look can all
// tell a deliberate bloom-out from a broken one.
blaze,
opacity: 1,
};
// --- ground ---------------------------------------------------------
// Sampled calmer and sparser than it would be as a shot, because a bed the
// shot cannot be read against is not a bed.
const layers = [];
if (!standsAlone) {
const groundRng = rng.fork(`ground:${ground.name}`);
layers.push({
module: ground,
role: GROUND,
// Calmed, but NOT thinned — see GROUND_BIAS in scenes/surface.js,
// which is also the bias the ground was MEASURED at. The first
// version subtracted 0.3 from density here, which is exactly
// backwards for a bed: an intro is already biased sparse, so the
// ground came out at density zero and the section was thin again
// for a new reason. Three of forty rendered sections fell under 30%
// painted with a ground under every one of them.
params: sampleValues(ground, groundRng, groundBiasFrom(bias),
groundTemperamentFrom(temperament)),
seed: groundRng.int(0, 0x7fffffff),
blend: 'normal', blend: 'normal',
opacity: 1, opacity: 1,
}]; });
}
layers.push(shot);
// --- the budget -----------------------------------------------------
// What is on the frame so far, and how much more this section is allowed
// to put on it. See coverageBudgetFor: the ground and the shot are not
// negotiable, so the budget governs the passes over them — a quiet intro
// spends everything on its bed and stacks nothing, a crowded drop under a
// director with an appetite for it gets two passes.
const budget = coverageBudgetFor(bias, story, director);
let spent = layers.reduce((sum, l) => sum + sceneCoverage(l.module), 0);
// --- overlay -------------------------------------------------------- // --- overlay --------------------------------------------------------
// Roughly a third of stacks on busy material, rarely on quiet material, and // Roughly a third of stacks on busy material, rarely on quiet material, and
// never on a background that is itself a full-frame glitch — two competing // never on a background that is itself a full-frame glitch — two competing
// corruption passes is noise, not depth. // corruption passes is noise, not depth.
// Layering is much more likely now that what goes on top is guaranteed to
// leave the shot underneath visible.
// `population` is how crowded the story wants this point in the video to
// be, and layering is the only lever on that which does not need the scene's
// cooperation: a lone form in an empty frame and the same form under two
// more passes are the sparse and crowded ends of one video.
const crowd = story ? (story.population - 0.5) * 0.5 : 0;
// The base rate was tuned when this branch was dead and layering only ever
// came from the reserved accent slot. With a third of the library eligible
// it lands on half of all stacks, which is the "permanently cluttered" the
// comment above warns about — and it costs seed separation, because a video
// where everything is doubled up looks like every other video where
// everything is doubled up.
const overlayChance = module.family === 'glitch' const overlayChance = module.family === 'glitch'
? 0.05 ? 0.1
: 0.12 + bias.energy * 0.35 + (temperament ? Math.max(0, temperament.detail) * 0.2 : 0); : 0.2 + bias.energy * 0.35 + crowd
+ (temperament ? Math.max(0, temperament.detail) * 0.2 : 0);
const overlays = overlayRoster.filter((m) => m.family !== module.family && m.name !== module.name); // Only COMPOSABLE scenes go on top. A second canvas over the first is two
if (overlays.length && rng.bool(Math.min(0.6, overlayChance))) { // pictures fighting rather than one picture with depth, and it is what the
const overlay = rng.pick(overlays); // library did for as long as every scene was treated as interchangeable.
//
// The other half of the trade: a composable scene alone is a few bright
// things on black, which scores well for variety and is thin to watch.
// Layering is what turns both halves into one image.
let available = overlayRoster.filter((m) => m.name !== module.name
&& surfaceOf(m) === 'composable');
// Two passes at the same slot rather than two differently-named slots. The
// second is rarer and only on loud, crowded material — that is where the
// old accent pass used to land, and it is the difference between a shot with
// something over it and a shot with a texture and a shimmer over it.
const chances = [
Math.min(0.65, overlayChance),
Math.min(0.25, overlayChance * bias.energy * 0.5),
];
for (const [pass, chance] of chances.entries()) {
// The budget is a wall, and it is also a lean: as the frame fills up
// the odds of adding to it fall away before the wall is reached, so a
// stack that is already nearly full rarely gets a token last pass.
const headroom = budget - spent;
const fits = available.filter((m) => sceneCoverage(m) <= headroom);
if (!fits.length || !rng.bool(chance * clamp01(headroom / 0.35))) break;
available = fits;
// Prefer a different family so the images argue instead of blurring —
// and then, within that, prefer the ones that MEASURE different.
//
// Family is a label somebody typed; structural distance is what the
// gallery saw when it rendered the two scenes under the same six songs.
// They disagree often enough to matter: two 'geometric' scenes can be
// 0.31 apart and a 'flow' and an 'organic' scene 0.04, and stacking the
// second pair is one picture at double density rather than a picture
// with something happening in it. See scenes/surface.js.
const offFamily = available.filter((m) => m.family !== module.family);
const pool = offFamily.length ? offFamily : available;
const overlay = rng.pickWeighted(pool, pool.map((m) => contrastWeight(m, layers)));
spent += sceneCoverage(overlay);
available = available.filter((m) => m.name !== overlay.name
&& m.family !== overlay.family);
// Screen and add keep the background readable underneath; softlight and // Screen and add keep the background readable underneath; softlight and
// overlay tint it instead. All four preserve the shot; 'normal' would // overlay tint it instead. All four preserve the shot; 'normal' would
// simply replace it, which is what the shot cut is for. // simply replace it, which is what the shot cut is for.
const blend = rng.pickWeighted(['screen', 'add', 'softlight', 'overlay'], [3, 2, 2, 1]); const blend = rng.pickWeighted(['screen', 'add', 'softlight', 'overlay'], [3, 2, 2, 1]);
// A second pass sits lighter than the first, so what accumulates is
// depth rather than a third opaque picture.
const fade = pass === 0 ? 1 : 0.6;
layers.push({ layers.push({
module: overlay, module: overlay,
params: sampleValues(overlay, rng.fork(`overlay:${overlay.name}`), { params: sampleValues(overlay, rng.fork(`overlay:${pass}:${overlay.name}`), {
// An overlay reads as texture over the shot, so it is sampled // An overlay reads as texture over the shot, so it is sampled
// sparser and calmer than it would be as a background. // sparser and calmer than it would be as a background.
...bias, ...bias,
@ -242,20 +746,7 @@ function buildStack(module, accentRoster, overlayRoster, bias, rng, temperament)
}, temperament), }, temperament),
seed: rng.int(0, 0x7fffffff), seed: rng.int(0, 0x7fffffff),
blend, blend,
opacity: blend === 'add' ? rng.range(0.18, 0.42) : rng.range(0.25, 0.55), opacity: (blend === 'add' ? rng.range(0.18, 0.42) : rng.range(0.25, 0.55)) * fade,
});
}
// --- accent ---------------------------------------------------------
if (accentRoster.length && rng.bool(bias.energy * 0.8)) {
const eligible = accentRoster.filter((m) => m.family !== module.family);
const accent = rng.pick(eligible.length ? eligible : accentRoster);
layers.push({
module: accent,
params: sampleValues(accent, rng.fork('accent'), bias, temperament),
seed: rng.int(0, 0x7fffffff),
blend: rng.pickWeighted(['add', 'screen'], [2, 1]),
opacity: rng.range(0.18, 0.5),
}); });
} }
return layers; return layers;
@ -266,7 +757,10 @@ function buildStack(module, accentRoster, overlayRoster, bias, rng, temperament)
* @param {object} options * @param {object} options
* @returns {object} LookSpec * @returns {object} LookSpec
*/ */
export function generateLook(track, { seed = null, samples = null, overrides = null } = {}) { export function generateLook(track, {
seed = null, samples = null, overrides = null,
pool: poolOverride = null, poolSize = POOL_SIZE,
} = {}) {
const resolvedSeed = seed !== null const resolvedSeed = seed !== null
? seed >>> 0 ? seed >>> 0
: samples ? hashSamples(samples) : 0x9e3779b9; : samples ? hashSamples(samples) : 0x9e3779b9;
@ -274,20 +768,43 @@ export function generateLook(track, { seed = null, samples = null, overrides = n
const rng = new Rng(resolvedSeed); const rng = new Rng(resolvedSeed);
const summary = track.summary; const summary = track.summary;
const paletteSource = new AudioPalette(summary, rng.fork('palette'));
const palette = generateUsablePalette(paletteSource, 6);
// The production design, decided before a single scene is cast — casting // The production design, decided before a single scene is cast — casting
// depends on it. See look/Personality.js. // depends on it. See look/Personality.js.
const personality = generatePersonality(summary, rng.fork('personality'), (signature) => const personality = generatePersonality(summary, rng.fork('personality'), (signature) =>
scenes.filter((m) => m.role !== 'accent' && sceneHonours(m, signature)).length); scenes.filter((m) => canBackground(m) && sceneHonours(m, signature)).length,
track.sections.length);
// The track's point of view about what a song looks like. Cast before any // The track's point of view about what a song looks like. Cast before any
// scene is, because it decides which scenes are even candidates. // scene is, because it decides which scenes are even candidates.
const director = pickDirector(summary, rng.fork('director')); const director = pickDirector(summary, rng.fork('director'));
const { palettes, paletteSchemes } = derivePaletteSet(
summary, rng.fork('palettes'), director);
// Back-compat alias: the first palette is still the track's palette.
const palette = palettes[0];
const paletteSource = { lastScheme: paletteSchemes[0] };
// This track's cast, drawn before any section is assigned. See castingPool.
//
// `poolOverride` exists for the variety harness, which needs reference
// videos that share no scenes with each other but are otherwise built by
// exactly this code — a reference assembled by any other path stops being
// comparable to the thing it is bounding.
const pool = poolOverride && poolOverride.length
? poolOverride
: castingPool(rng.fork('pool'), personality.signature, poolSize);
// What HAPPENS over the track, as opposed to what it is made of. Derived
// before casting because it decides which member of a roster anchors which
// section, and whether the outro answers the intro. See look/Story.js.
const story = deriveStory(track, summary, rng.fork('story'));
const rosterByKind = assignRostersByKind( const rosterByKind = assignRostersByKind(
track.sections, rng.fork('scenes'), personality.signature, director); track.sections, rng.fork('scenes'), personality.signature, director, pool);
if (story.recap) applyRecap(rosterByKind);
// What each kind's sections stand on. After the recap, so the outro is
// grounded against the roster it actually ends up with.
const groundByKind = assignGroundsByKind(
rosterByKind, rng.fork('grounds'), personality.signature, director);
// The grain treatment: usually none, and when present described rather than // The grain treatment: usually none, and when present described rather than
// dialled. See look/grain.js. // dialled. See look/grain.js.
const grain = deriveGrain(summary, rng.fork('grain')); const grain = deriveGrain(summary, rng.fork('grain'));
@ -295,40 +812,53 @@ export function generateLook(track, { seed = null, samples = null, overrides = n
const paletteArc = derivePaletteArc(summary, rng.fork('paletteArc')); const paletteArc = derivePaletteArc(summary, rng.fork('paletteArc'));
// Whether shots change SIZE at the cut, and how boldly. See look/framing.js. // Whether shots change SIZE at the cut, and how boldly. See look/framing.js.
const framing = deriveFramingStyle(summary, rng.fork('framing')); const framing = deriveFramingStyle(summary, rng.fork('framing'));
// WHERE the camera looks, and how it travels there. The director leans
// toward one camera the way it leans toward one family per kind, and the
// seed decides — see look/Camera.js.
const camera = deriveCamera(director, summary, rng.fork('camera'));
const { post, feedback } = derivePost(summary, rng.fork('post'), grain); const { post, feedback } = derivePost(summary, rng.fork('post'), grain);
// Scenes that declare role 'accent' composite over a background rather than
// being one — most of their frame is empty by design. They are never chosen
// as a section's primary scene.
// Accents honour the signature too where they can. If none can, the track
// goes without depth layers rather than putting an off-design element into
// every stack.
const accentRoster = scenes.filter((m) => m.role === 'accent'
&& sceneHonours(m, personality.signature));
// Scenes eligible to be composited OVER a background. Same casting rule as // Scenes eligible to be composited OVER a background. Same casting rule as
// everything else — an overlay is on screen as much as the shot under it, // everything else — an overlay is on screen as much as the shot under it,
// so an off-design one would be just as visible. // so an off-design one would be just as visible.
const overlayRoster = scenes.filter((m) => m.role !== 'accent' //
&& sceneHonours(m, personality.signature)); // Widened past the casting pool with the scenes that exist only to sit on
// top: those are never drawn as a section's primary scene, so the pool —
// which is built out of background candidates — would never contain them.
const overlayOnly = scenes.filter((m) => !canBackground(m));
const overlayRoster = (pool.length >= 4 ? pool : scenes.filter(canBackground))
.concat(overlayOnly);
const sections = track.sections.map((section) => { const sections = track.sections.map((section) => {
const roster = rosterByKind.get(section.kind) || [scenes[0]]; const state = storyForSection(story, section.index);
const kindRoster = rosterByKind.get(section.kind) || [scenes[0]];
// The kind's roster, opened on the member this point in the story has
// earned. Same set, different anchor. See anchorOrder.
const roster = anchorOrder(kindRoster, state);
const sectionRng = rng.fork(`section:${section.index}:${roster[0].name}`); const sectionRng = rng.fork(`section:${section.index}:${roster[0].name}`);
const bias = biasFor(section, summary); const bias = biasFor(section, summary, personality.motion, state);
// The RATCHET: how hard the track pushes its dials is a property of the
// track (Personality.temperament) scaled by where in the story it is,
// so the last occurrence of a kind samples further out than the first.
const temperament = temperamentFor(personality.temperament, state);
const ground = groundByKind.get(section.kind) || null;
const variants = roster.map((module, v) => buildStack( const variants = roster.map((module, v) => buildStack(
module, accentRoster, overlayRoster, bias, module, overlayRoster, bias,
sectionRng.fork(`variant:${section.index}:${v}`), personality.temperament, sectionRng.fork(`variant:${section.index}:${v}`), temperament, state,
{ ground, director, kind: section.kind },
)); ));
const shots = planShots( const shots = planShots(
section, track, bias, variants.length, sectionRng.fork(`shots:${section.index}`), section, track, bias, variants.length,
sectionRng.fork(`shots:${section.index}`), state,
); );
return { return {
index: section.index, index: section.index,
kind: section.kind, kind: section.kind,
story: state,
startFrame: section.startFrame, startFrame: section.startFrame,
endFrame: section.endFrame, endFrame: section.endFrame,
start: section.start, start: section.start,
@ -347,11 +877,16 @@ export function generateLook(track, { seed = null, samples = null, overrides = n
const look = { const look = {
seed: resolvedSeed, seed: resolvedSeed,
palette, palette,
// The full set the director cuts between — ArcDriver reads this per cue.
palettes,
paletteSchemes,
personality, personality,
paletteScheme: paletteSource.lastScheme, paletteScheme: paletteSource.lastScheme,
director: director.name, director: director.name,
story,
paletteArc, paletteArc,
framing, framing,
camera,
grain, grain,
post, post,
feedback, feedback,
@ -369,32 +904,41 @@ export function rerollSection(look, track, sectionIndex, salt = 0) {
const rng = new Rng((look.seed ^ (sectionIndex * 0x9e3779b9) ^ (salt * 0x85ebca6b)) >>> 0); const rng = new Rng((look.seed ^ (sectionIndex * 0x9e3779b9) ^ (salt * 0x85ebca6b)) >>> 0);
const signature = (look.personality && look.personality.signature) || []; const signature = (look.personality && look.personality.signature) || [];
const families = directorByName(look.director).families[section.kind] || Object.keys(FAMILIES); const director = directorByName(look.director);
const families = director.families[section.kind] || Object.keys(FAMILIES);
// A reroll re-draws this section's cast from the same kind of pool the track
// was built with, weighted by the signature rather than filtered by it.
let candidates = families.flatMap((f) => scenesInFamily(f)) let candidates = families.flatMap((f) => scenesInFamily(f))
.filter((m) => m.role !== 'accent' && sceneHonours(m, signature)); .filter(canBackground);
if (!candidates.length) { if (!candidates.length) candidates = scenes.filter(canBackground);
candidates = scenes.filter((m) => m.role !== 'accent' && sceneHonours(m, signature));
}
// Re-roll the whole roster, not just the anchor: the section's shots cut // Re-roll the whole roster, not just the anchor: the section's shots cut
// between all of them, so replacing one would leave the section half old. // between all of them, so replacing one would leave the section half old.
const size = Math.min(rosterSizeFor(section.kind), Math.max(1, candidates.length)); const size = Math.min(rosterSizeFor(section.kind), Math.max(1, candidates.length));
const roster = []; const roster = [];
while (roster.length < size) { while (roster.length < size) {
const pool = candidates.filter((m) => !roster.includes(m)); const options = candidates.filter((m) => !roster.includes(m));
if (!pool.length) break; if (!options.length) break;
roster.push(rng.pick(pool)); roster.push(rng.pickWeighted(options, options.map((m) => signatureWeight(m, signature))));
} }
if (!roster.length) roster.push(scenes[0]); if (!roster.length) roster.push(scenes[0]);
const accentRoster = scenes.filter((m) => m.role === 'accent'); const overlayRoster = castingPool(rng.fork('pool'), signature)
const overlayRoster = scenes.filter((m) => m.role !== 'accent' && sceneHonours(m, signature)); .concat(scenes.filter((m) => !canBackground(m)));
// A reroll changes what this section is made of. Where it sits in the story
// is a property of the song, so it survives untouched.
const state = section.story || NEUTRAL_STATE;
// The section is re-cast, so its ground is re-cast with it — a reroll that
// kept the old bed under new shots would be answering half the question.
const ground = castGround(
section.kind, roster, rng.fork('ground'), signature, director, new Set());
section.variants = roster.map((module, v) => buildStack( section.variants = roster.map((module, v) => buildStack(
module, accentRoster, overlayRoster, section.bias, rng.fork(`variant:${v}`), module, overlayRoster, section.bias, rng.fork(`variant:${v}`),
look.personality && look.personality.temperament, temperamentFor(look.personality && look.personality.temperament, state), state,
{ ground, director, kind: section.kind },
)); ));
section.shots = planShots( section.shots = planShots(
section, track, section.bias, section.variants.length, rng.fork('shots'), section, track, section.bias, section.variants.length, rng.fork('shots'), state,
); );
section.layers = section.variants[0]; section.layers = section.variants[0];
return look; return look;
@ -413,14 +957,27 @@ export function rerollLook(look, track, newSeed) {
} }
function applyOverrides(look, overrides) { function applyOverrides(look, overrides) {
if (overrides.palette) look.palette = overrides.palette; if (overrides.palette) {
look.palette = overrides.palette;
// Keep the full set in sync if the caller replaced the single alias.
if (look.palettes && look.palettes.length) look.palettes[0] = overrides.palette;
if (overrides.palettes) {
look.palettes = overrides.palettes;
look.palette = look.palettes[0] || look.palette;
}
}
if (overrides.palettes) {
look.palettes = overrides.palettes;
look.palette = look.palettes[0] || look.palette;
}
if (overrides.paletteSchemes) look.paletteSchemes = overrides.paletteSchemes;
if (overrides.post) look.post = { ...look.post, ...overrides.post }; if (overrides.post) look.post = { ...look.post, ...overrides.post };
if (overrides.feedback) look.feedback = { ...look.feedback, ...overrides.feedback }; if (overrides.feedback) look.feedback = { ...look.feedback, ...overrides.feedback };
if (overrides.sections) { if (overrides.sections) {
overrides.sections.forEach((o, i) => { overrides.sections.forEach((o, i) => {
if (!look.sections[i]) return; if (!look.sections[i]) return;
if (o.locked !== undefined) look.sections[i].locked = o.locked; if (o.locked !== undefined) look.sections[i].locked = o.locked;
if (o.params) Object.assign(look.sections[i].layers[0].params, o.params); if (o.params) Object.assign(subjectOf(look.sections[i].layers).params, o.params);
}); });
} }
return look; return look;
@ -428,10 +985,17 @@ function applyOverrides(look, overrides) {
/** Compact description, used by the HUD and by check output. */ /** Compact description, used by the HUD and by check output. */
export function describeLook(look) { export function describeLook(look) {
const kinds = look.sections.map((s) => `${s.kind}:${s.layers[0].module.name}`); const kinds = look.sections.map((s) => `${s.kind}:${subjectOf(s.layers).module.name}`);
return `seed ${look.seed.toString(16)} · ${look.director} · ${look.paletteScheme} · ` + const palTag = look.paletteSchemes && look.paletteSchemes.length > 1
? look.paletteSchemes.join('→')
: look.paletteScheme;
const planTag = look.palettePlan
? ` · palettes:${look.palettePlan.progression}/${look.palettePlan.transition}`
: '';
return `seed ${look.seed.toString(16)} · ${look.director} · ${palTag} · ` +
`${describePersonality(look.personality)} · ${describeGrain(look.grain)} · ` + `${describePersonality(look.personality)} · ${describeGrain(look.grain)} · ` +
`${describePaletteArc(look.paletteArc)} · ${describeFraming(look.framing)} · ` + `${describePaletteArc(look.paletteArc)} · ${describeFraming(look.framing)} · ` +
`${describeCamera(look.camera)}${planTag} · ` +
`${[...new Set(kinds)].join(', ')}`; `${[...new Set(kinds)].join(', ')}`;
} }

181
flow-state/src/look/Personality.js
View File

@ -38,8 +38,23 @@
// Everything here is seeded off the look seed, so a track's personality is as // Everything here is seeded off the look seed, so a track's personality is as
// reproducible as everything else. // reproducible as everything else.
import {
generateIdentity, identityUniforms, describeIdentity, NEUTRAL_IDENTITY_UNIFORMS,
} from './Identity.js';
export const TRAITS = ['shape', 'camera', 'space', 'style']; export const TRAITS = ['shape', 'camera', 'space', 'style'];
const clamp01 = (x) => Math.max(0, Math.min(1, x));
// Box-Muller standard normal from the seeded rng. Two uniform draws per
// sample; the mode is at 0 and the tails are rare, which is the whole
// point — uniform boxes put as many tracks at the wild edge as near level.
function gauss(rng) {
const u1 = Math.max(rng.next(), 1e-7);
const u2 = rng.next();
return Math.sqrt(-2 * Math.log(u1)) * Math.cos(2 * Math.PI * u2);
}
/** /**
* Traits eligible to be a track's signature, and how often. * Traits eligible to be a track's signature, and how often.
* *
@ -51,6 +66,38 @@ export const TRAITS = ['shape', 'camera', 'space', 'style'];
*/ */
const SIGNATURE_WEIGHTS = { shape: 4, space: 3, camera: 2, style: 2 }; const SIGNATURE_WEIGHTS = { shape: 4, space: 3, camera: 2, style: 2 };
/**
* How the audio tilts the choice of signature.
*
* The signature decides which scenes a track can even cast, so if it is picked
* from the seed alone then the single most consequential decision in the whole
* generator has no relationship to the music. Measured, that is exactly what
* happened: across seven songs the correlation between how different two tracks
* SOUND and how different their videos LOOK was -0.03. The videos differed; the
* differences just had nothing to do with the songs.
*
* A tilt, not a rule. Every trait stays reachable for every track — a mapping
* rigid enough to predict is the failure this layer exists to avoid — but a
* track built on transients leans toward shape, a spacious one toward space, a
* fast one toward camera and a noisy one toward style.
*/
function signatureTilt(summary, sections) {
const bright = summary.meanCentroid ?? 0.5;
const noisy = Math.min(1, (summary.meanFlatness ?? 0.2) * 3);
const fast = clamp01(((summary.bpm ?? 120) - 80) / 80);
const dynamic = clamp01(summary.dynamicRange ?? 0.5);
// A track that keeps changing what it is doing has structure to draw on;
// one that states an idea and holds it has a place instead.
const busy = clamp01((sections - 2) / 5);
return {
shape: 0.5 + busy * 1.2 + (1 - noisy) * 0.5,
space: 0.5 + dynamic * 1.1 + (1 - busy) * 0.6,
camera: 0.5 + fast * 1.2,
style: 0.5 + noisy * 1.3 + bright * 0.4,
};
}
/** Minimum scenes that must survive the signature filter for it to be usable. */ /** Minimum scenes that must survive the signature filter for it to be usable. */
export const MIN_ELIGIBLE_SCENES = 6; export const MIN_ELIGIBLE_SCENES = 6;
@ -68,38 +115,69 @@ export const MIN_ELIGIBLE_SCENES = 6;
* How many scenes would survive a given signature. Injected rather than * How many scenes would survive a given signature. Injected rather than
* imported so this module never has to know the registry exists. * imported so this module never has to know the registry exists.
*/ */
export function generatePersonality(summary, rng, countEligible = null) { export function generatePersonality(summary, rng, countEligible = null, sections = 4) {
const bright = summary.meanCentroid; const bright = summary.meanCentroid;
const noisy = Math.min(1, summary.meanFlatness * 3); const noisy = Math.min(1, summary.meanFlatness * 3);
const fast = Math.min(1, Math.max(0, (summary.bpm - 80) / 80)); const fast = Math.min(1, Math.max(0, (summary.bpm - 80) / 80));
const loud = Math.min(1, (summary.dynamicRange ?? 0.5) + (summary.meanLoudness ?? 0.3)); const loud = Math.min(1, (summary.dynamicRange ?? 0.5) + (summary.meanLoudness ?? 0.3));
const dynamic = clamp01(summary.dynamicRange ?? 0.5);
// Audio sets the CENTRE of each distribution and the seed picks within it.
//
// The alternative — deriving values from the audio outright — buys coupling
// by destroying seed variety, since two seeds on one song would then agree
// about everything. Centring keeps both: two songs land in different regions
// of the space, two seeds land in different places inside one region. The
// window is deliberately wide enough that the mapping cannot be read off a
// finished video.
const around = (centre, window, lo, hi) =>
Math.max(lo, Math.min(hi, centre + rng.range(-window, window)));
const shape = { const shape = {
// 0 sides means round. Everything else is a polygon the whole track // 0 sides means round. Everything else is a polygon the whole track
// shares — the single most recognisable thing here. // shares — the single most recognisable thing here. Noise and transients
sides: rng.pickWeighted([0, 3, 4, 5, 6, 8], [3, 2, 3, 2, 3, 1]), // earn corners; tonal, smooth material stays round.
roundness: rng.range(0.05, 0.5), sides: rng.pickWeighted(
elongation: rng.range(0.85, 1.45), [0, 3, 4, 5, 6, 8],
[3 + (1 - noisy) * 4, 1 + noisy * 2, 2 + noisy * 2,
1 + noisy * 1.5, 2 + noisy * 2, 0.5 + noisy * 1.5]),
roundness: around(0.28 - noisy * 0.15, 0.18, 0.05, 0.5),
elongation: around(1.15, 0.3, 0.85, 1.45),
tilt: rng.range(0, Math.PI), tilt: rng.range(0, Math.PI),
}; };
// Camera: bell-curve over level vs. wild. Most tracks stay near
// level — a little drift, a little sway, almost no roll — and only
// the tail really swings. Uniform boxes put as many tracks at the
// wild edge as near level, which is why the grid read as wild so
// often; a Gaussian puts the mode at level and rarity in the tails.
// Audio still tilts the centre (fast → more drift/sway/spin,
// dynamic → more breathe) but the seed picks within a bell around it.
const gaussAround = (centre, sigma, lo, hi) =>
Math.max(lo, Math.min(hi, centre + gauss(rng) * sigma));
const camera = { const camera = {
driftAngle: rng.range(0, Math.PI * 2), driftAngle: rng.range(0, Math.PI * 2),
// A slow track should not be filmed from a moving car. // A slow track should not be filmed from a moving car. Centre near
driftRate: rng.range(0.01, 0.06) * (0.6 + fast * 0.8), // level (0.016 + tempo), sigma narrow so wild drift is a tail event.
sway: rng.range(0.0, 0.06), driftRate: gaussAround(0.016 + fast * 0.016, 0.010, 0.004, 0.065),
swayRate: rng.range(0.05, 0.22), sway: gaussAround(0.010 + fast * 0.018, 0.009, 0, 0.055),
spin: rng.range(-0.05, 0.05), swayRate: gaussAround(0.09 + fast * 0.06, 0.028, 0.05, 0.22),
// Spin centred at zero — most tracks stay level, only the tails roll.
// Spread scales with tempo so a fast track has a wider bell, not a
// displaced one. Sign comes from the Gaussian itself.
spin: gaussAround(0, 0.012 + fast * 0.012, -0.05, 0.05),
// Breathing is locked to the bar, so it is the one camera move that // Breathing is locked to the bar, so it is the one camera move that
// reads as musical rather than as drifting. // reads as musical rather than as drifting. A dynamic track breathes.
breathe: rng.range(0.0, 0.05), breathe: gaussAround(dynamic * 0.022, 0.010, 0, 0.048),
}; };
const space = { const space = {
horizon: rng.range(0.32, 0.62), // A bright track sits high in its frame and a dark one sits low; a
depth: rng.range(0.2, 0.9), // dynamic one has depth to fall away into.
horizon: around(0.36 + bright * 0.2, 0.1, 0.32, 0.62),
depth: around(0.25 + dynamic * 0.5, 0.25, 0.2, 0.9),
washAngle: rng.range(0, Math.PI * 2), washAngle: rng.range(0, Math.PI * 2),
wash: rng.range(0.1, 0.5), wash: around(0.18 + (1 - bright) * 0.2, 0.15, 0.1, 0.5),
}; };
const style = { const style = {
@ -115,7 +193,24 @@ export function generatePersonality(summary, rng, countEligible = null) {
// Fold counts stay low and are usually off. Symmetry is the fastest way // Fold counts stay low and are usually off. Symmetry is the fastest way
// to make a library look like one series and also the fastest way to // to make a library look like one series and also the fastest way to
// make every track look like a screensaver. // make every track look like a screensaver.
symmetry: rng.pickWeighted([1, 1, 2, 3, 4, 6], [6, 4, 2, 2, 2, 1]), symmetry: rng.pickWeighted([1, 1, 2, 3, 4, 6],
[6, 4, 1 + (1 - noisy) * 2, 1 + (1 - noisy) * 2, 1 + (1 - noisy) * 2, 0.5 + (1 - noisy)]),
};
// How this track MOVES, as a character rather than a rate.
//
// Motion was the weakest block in every measurement — 47% of achievable
// across seeds — because the only lever on it was a tempo-derived rate
// multiplier, and tempo compresses. Two tracks at 124 and 138bpm got
// essentially the same movement. Stillness is the missing axis: some music
// wants an image that hangs almost motionless and some wants one that never
// settles, and that is not the same question as how fast it animates.
const motion = {
// 0 = hangs, 1 = never settles. Dynamic, spacious material earns the
// stillness; dense, fast material does not get it.
stillness: clamp01(around(0.5 + dynamic * 0.35 - fast * 0.45, 0.3, 0, 1)),
// Whether the movement is steady or agitated, independent of its speed.
churn: clamp01(around(0.25 + noisy * 0.5, 0.3, 0, 1)),
}; };
// How hard this track pushes every scene it casts. Deliberately wide, and // How hard this track pushes every scene it casts. Deliberately wide, and
@ -137,9 +232,14 @@ export function generatePersonality(summary, rng, countEligible = null) {
extremity: rng.range(0.45, 1.0), extremity: rng.range(0.45, 1.0),
}; };
const signature = pickSignature(rng, countEligible); const signature = pickSignature(rng, countEligible, signatureTilt(summary, sections));
return { signature, shape, camera, space, style, temperament }; // The song's cast and ink. Generated here so everything downstream — layer
// uniforms, the HUD, presets — reaches it the same way it reaches the rest
// of the production design. See look/Identity.js.
const identity = generateIdentity(summary, rng.fork('identity'), sections);
return { signature, shape, camera, space, style, motion, temperament, identity };
} }
/** /**
@ -150,10 +250,11 @@ export function generatePersonality(summary, rng, countEligible = null) {
* rosters from, fall back to the stronger of the two rather than shipping a * rosters from, fall back to the stronger of the two rather than shipping a
* track whose every section is forced onto the same two scenes. * track whose every section is forced onto the same two scenes.
*/ */
function pickSignature(rng, countEligible) { function pickSignature(rng, countEligible, tilt = null) {
const primary = rng.pickWeighted(TRAITS, TRAITS.map((t) => SIGNATURE_WEIGHTS[t])); const weightOf = (t) => SIGNATURE_WEIGHTS[t] * (tilt ? tilt[t] : 1);
const primary = rng.pickWeighted(TRAITS, TRAITS.map(weightOf));
const rest = TRAITS.filter((t) => t !== primary); const rest = TRAITS.filter((t) => t !== primary);
const secondary = rng.pickWeighted(rest, rest.map((t) => SIGNATURE_WEIGHTS[t])); const secondary = rng.pickWeighted(rest, rest.map(weightOf));
const pair = [primary, secondary]; const pair = [primary, secondary];
if (!countEligible || countEligible(pair) >= MIN_ELIGIBLE_SCENES) return pair; if (!countEligible || countEligible(pair) >= MIN_ELIGIBLE_SCENES) return pair;
@ -169,6 +270,35 @@ export function sceneHonours(module, signature) {
return signature.every((t) => traits.includes(t)); return signature.every((t) => traits.includes(t));
} }
/**
* How well a scene fits the signature, 0..1 — and the casting WEIGHT that
* follows from it.
*
* `sceneHonours` was a hard filter, and as a filter it was the single largest
* cause of sameness in the generator. Measured across the library: a scene
* declaring all four traits is eligible for every track and opens half of all
* videos, while a scene declaring two is eligible for one track in fourteen.
* Eleven scenes out of sixty-one carried nearly every video, five of them from
* the same family, and that was the house style — not a decision anyone made,
* just an artefact of which scenes happened to declare four traits.
*
* A lean rather than a wall. Honouring the whole signature is worth six times
* the weight of honouring none of it, which is more than enough for the track to
* read as one production, while leaving the rest of the library reachable
* instead of disqualified.
*/
export function signatureAffinity(module, signature) {
if (!signature || !signature.length) return 1;
const traits = module.traits || [];
let hit = 0;
for (const t of signature) if (traits.includes(t)) hit++;
return hit / signature.length;
}
export function signatureWeight(module, signature) {
return 1 + signatureAffinity(module, signature) * 5;
}
/** /**
* Flatten to the uniform values the shader contract expects. * Flatten to the uniform values the shader contract expects.
* *
@ -210,10 +340,13 @@ export function signatureUniforms(personality, module = null) {
// every frame. Neutral here so a layer built without one is unframed. // every frame. Neutral here so a layer built without one is unframed.
u_sigFrameScale: 1, u_sigFrameScale: 1,
u_sigFrameShift: [0, 0], u_sigFrameShift: [0, 0],
...identityUniforms(personality.identity, personality.shape),
}; };
} }
export const NEUTRAL_UNIFORMS = { export const NEUTRAL_UNIFORMS = {
...NEUTRAL_IDENTITY_UNIFORMS,
u_sigSides: 0, u_sigSides: 0,
u_sigRound: 0.25, u_sigRound: 0.25,
u_sigElong: 1, u_sigElong: 1,
@ -245,6 +378,12 @@ export function describePersonality(personality) {
SHAPE_NAMES[shape.sides] || `${shape.sides}-sided`, SHAPE_NAMES[shape.sides] || `${shape.sides}-sided`,
]; ];
if (style.symmetry > 1) parts.push(`${style.symmetry}-fold`); if (style.symmetry > 1) parts.push(`${style.symmetry}-fold`);
if (personality.identity) parts.push(describeIdentity(personality.identity));
if (personality.motion) {
const m = personality.motion;
parts.push(`${m.stillness > 0.6 ? 'still' : m.stillness < 0.3 ? 'restless' : 'moving'}` +
`${m.churn > 0.6 ? '+churn' : ''}`);
}
if (personality.temperament) { if (personality.temperament) {
const t = personality.temperament; const t = personality.temperament;
parts.push(`${t.intensity >= 0 ? 'hot' : 'cool'} ${t.extremity.toFixed(2)} bold`); parts.push(`${t.intensity >= 0 ? 'hot' : 'cool'} ${t.extremity.toFixed(2)} bold`);

404
flow-state/src/look/Story.js Normal file
View File

@ -0,0 +1,404 @@
// The song's STORY: where a section sits in the video, as opposed to what kind
// of section it is.
//
// Everything that decides what this generator puts on screen is keyed on
// section KIND — the roster (LookGenerator.assignRostersByKind), the parameter
// bias (KIND_ENERGY), the cutting rhythm (shots.rhythmFor), the colour offset
// (paletteArc.kindHue). Kinds recur. So the fourth drop is cast from the same
// roster, biased to the same energy, cut at the same rate and tinted the same
// hue as the first one, and nothing in the video can tell you which of them you
// are watching.
//
// The one exception was ArcDriver's slow axis, which travels one way across the
// whole track — and it is blind: it does not know where the drop is, and its
// direction is a coin flip per scene, so two scenes in one video routinely
// travel against each other.
//
// That is a song structure without a story: recurrence without consequence.
// This module adds the missing coordinate.
//
// POSITION which occurrence of its kind a section is, and which act it is in.
// `ordinal` alone — "the 3rd of 4 drops" — unlocks most of what
// follows, and nothing downstream had it.
// MOMENTS the four frames the video turns on, FOUND in the audio rather
// than placed by the seed. A plot that declares a climax where the
// track is quiet is worse than no plot at all.
// PLOT what the track does with them, as one of a few coherent narrative
// shapes, chosen the way look/directors.js chooses a director.
//
// The plot is expressed as five variables that everything downstream reads.
// They are STAGED, not ramped: they hold flat inside a section and move at its
// boundary. A story advances in scenes; a smooth ramp over five minutes is a
// slow zoom, which is an effect rather than a narrative.
//
// Everything here is a pure function of (story, frame). No state, no random
// source past derivation — the same rule the rest of the render path follows,
// and what keeps a seek frame-identical to playback.
const clamp01 = (x) => Math.max(0, Math.min(1, x));
const smooth = (x) => { const t = clamp01(x); return t * t * (3 - 2 * t); };
const lerp = (a, b, t) => a + (b - a) * t;
/** The variables a plot moves. Neutral is 0.5 for all of them. */
export const STORY_VARS = ['tension', 'reveal', 'closeness', 'population', 'order'];
/**
* What the story reads as when there is none — a look built by hand, or a check
* that constructs sections directly. Everything downstream must render exactly
* what it rendered before this module existed when it sees these.
*/
export const NEUTRAL_STATE = {
tension: 0.5, reveal: 0.5, closeness: 0.5, population: 0.5, order: 0.5,
journey: 0.5, act: 'development',
};
/**
* The narrative shapes.
*
* One coherent point of view about what happens over a song, the same way a
* director is one point of view about what a song looks like. Weighted, with
* the audio tilting the odds and never deciding — a fixed mapping from measured
* features to narrative is how a library ends up with one story per genre.
*
* `curve` receives a section's narrative position and the track's seeded curve
* shape, and returns the five variables plus the journey the slow axis follows.
*
* ctx:
* p 0..1 through the section list
* rise 0..1 approach to the climax (1 from the climax onward)
* after 0..1 through the aftermath (0 up to the climax)
* ord 0..1 which occurrence of this kind it is
* energy 0..1 the section's measured energy, against the track's loudest
*/
export const PLOTS = [
{
name: 'emergence',
weight: 3,
// Almost nothing, then something. The reveal rises and stays up: what
// was uncovered does not go back in the box.
curve: (ctx, sh) => {
const r = smooth(ctx.rise * 0.85 + ctx.p * 0.15);
return {
tension: 0.2 + 0.7 * r,
reveal: lerp(0.12, 1, r),
population: 0.25 + 0.6 * r,
closeness: 0.35 + 0.4 * r * sh.closeDir,
order: 0.6 - 0.1 * r,
journey: r,
};
},
},
{
name: 'escalation',
weight: 3,
// Each recurrence goes further than the last. A ratchet rather than a
// curve — this is the plot that needs `ordinal`, and the one a viewer
// names as "it kept going somewhere".
curve: (ctx, sh) => {
const r = clamp01(ctx.rise * 0.5 + ctx.ord * 0.5);
return {
tension: 0.25 + 0.75 * r,
reveal: 0.3 + 0.6 * r,
population: 0.3 + 0.6 * r,
closeness: 0.35 + 0.45 * r * sh.closeDir,
order: 0.7 - 0.35 * r,
journey: clamp01(r * 0.8 + ctx.p * 0.2),
};
},
},
{
name: 'collapse',
weight: 2,
// Order into entropy. The climax is the thing coming apart rather than
// the thing at its peak, so `order` is monotone down and everything else
// follows the energy.
curve: (ctx, sh) => {
const r = smooth(ctx.rise);
return {
tension: 0.3 + 0.6 * r,
reveal: 0.35 + 0.5 * ctx.p,
population: clamp01(0.35 + 0.5 * r - 0.35 * ctx.after),
closeness: 0.3 + 0.45 * ctx.p * sh.closeDir,
order: 0.9 - 0.8 * smooth(ctx.p * 0.7 + ctx.rise * 0.3),
journey: ctx.p,
};
},
},
{
name: 'return',
weight: 2,
// ABA. Everything arches up to the climax and comes back down — but not
// all the way, and not to the same place: the small residue on `after`
// is what makes it a return rather than a loop. The outro re-casts the
// intro's scene; see LookGenerator and `recap`.
curve: (ctx, sh) => {
const arch = clamp01(smooth(ctx.rise) * (1 - smooth(ctx.after)));
return {
tension: 0.2 + 0.7 * arch,
reveal: clamp01(0.25 + 0.6 * arch + 0.18 * ctx.after),
population: 0.3 + 0.55 * arch,
closeness: 0.35 + 0.45 * arch * sh.closeDir,
order: 0.65 - 0.2 * arch,
journey: clamp01(arch * 0.85 + ctx.p * 0.15),
};
},
},
{
name: 'unveiling',
weight: 2,
// The song's protagonist is withheld and then it is all there is.
// Population falls as reveal rises for that reason: a frame full of
// chorus is what was hiding it.
curve: (ctx, sh) => {
const shown = smooth((ctx.rise - sh.holdOut) / Math.max(0.15, 1 - sh.holdOut));
return {
tension: 0.25 + 0.6 * smooth(ctx.rise),
reveal: 0.1 + 0.9 * shown,
population: 0.6 - 0.35 * shown,
closeness: 0.3 + 0.6 * shown * sh.closeDir,
order: 0.55 + 0.15 * shown,
journey: clamp01(shown * 0.75 + ctx.p * 0.25),
};
},
},
];
export const PLOT_NAMES = PLOTS.map((p) => p.name);
export function plotByName(name) {
return PLOTS.find((p) => p.name === name) || PLOTS[0];
}
/**
* Cast the plot.
*
* The tilts are deliberately loose: a track that keeps changing what it is
* doing has recurrences to ratchet, a dynamic one has somewhere to come back
* from, a noisy one comes apart more readily than it builds. Every plot stays
* reachable for every track.
*/
function pickPlot(summary, sectionCount, rng) {
const noisy = Math.min(1, (summary.meanFlatness ?? 0.2) * 3);
const dynamic = clamp01(summary.dynamicRange ?? 0.5);
const busy = clamp01((sectionCount - 2) / 5);
const weights = PLOTS.map((plot) => {
let w = plot.weight;
if (plot.name === 'escalation') w *= 0.5 + busy * 1.8;
if (plot.name === 'collapse') w *= 0.5 + noisy * 1.8;
if (plot.name === 'return') w *= 0.5 + dynamic * 1.4;
if (plot.name === 'emergence') w *= 0.6 + (1 - busy) * 1.2;
if (plot.name === 'unveiling') w *= 0.6 + (1 - noisy) * 1.0;
return w;
});
return rng.pickWeighted(PLOTS, weights);
}
/**
* The frames the story turns on.
*
* Every one of these is read off the section energies `segment.js` already
* produced. Nothing here may invent a moment the audio does not have: the story
* follows the song, and the one failure mode worse than no story is a story
* that declares its climax where the track is quiet.
*
* A short or flat track collapses several of these onto the same section, which
* is correct rather than degenerate — a song with no structure gets no story.
*/
function findMoments(sections) {
const n = sections.length;
const energies = sections.map((s) => s.energy || 0);
const peak = Math.max(...energies, 1e-6);
// The climax is the loudest section, ties broken toward the later one: when
// a track states the same peak twice, the second one is the one that means
// something, because the first has already happened.
let climax = 0;
for (let i = 0; i < n; i++) if (energies[i] >= energies[climax]) climax = i;
// The arrival is the video's first "here it is" — the first section that
// clears most of the way to the peak.
let arrival = null;
for (let i = 0; i < n; i++) {
if (energies[i] >= peak * 0.6) { arrival = i; break; }
}
if (arrival === null) arrival = Math.min(climax, n - 1);
// The turn is the largest fall, and it has to happen after something has
// arrived — a quiet opening followed by a quieter one is not a reversal.
let turn = null;
let worst = 0;
for (let i = arrival + 1; i < n; i++) {
const fall = energies[i - 1] - energies[i];
if (fall > worst) { worst = fall; turn = i; }
}
// The resolution is where the track stops trying to top itself.
let resolution = null;
for (let i = climax + 1; i < n; i++) {
if (energies[i] < energies[climax] * 0.85) { resolution = i; break; }
}
if (resolution === null && climax < n - 1) resolution = n - 1;
return { arrival, turn, climax, resolution };
}
function actFor(index, moments) {
if (index === moments.climax) return 'climax';
if (moments.resolution !== null && index >= moments.resolution) return 'resolution';
if (moments.turn !== null && index === moments.turn) return 'turn';
if (moments.arrival !== null && index < moments.arrival) return 'setup';
return 'development';
}
/**
* Derive the story for a track.
*
* @param {FeatureTrack} track
* @param {object} summary track.summary
* @param {Rng} rng
* @returns {object} plain data — no functions, so a look stays serialisable
*/
export function deriveStory(track, summary, rng) {
const sections = track.sections || [];
const n = sections.length;
const plot = pickPlot(summary, n, rng);
// The track's own version of its plot. Two tracks telling the same story
// still have to differ, for the reason directors.js gives about house
// styles — so the shape of the curves is seeded even when the plot is not.
const shape = {
// Whether this video moves toward its subject or pulls away from it.
// Toward, usually: a video that ends further away than it started is a
// real choice and a rarer one.
closeDir: rng.bool(0.75) ? 1 : -0.6,
// How long `unveiling` keeps its protagonist back.
holdOut: rng.range(0.45, 0.8),
// How hard this track commits to its plot at all.
bite: rng.range(0.7, 1.0),
};
const moments = findMoments(sections);
// How much story a track has room for. Under about four sections the curves
// have nowhere to travel, and forcing them produces a video that lurches
// rather than one that progresses, so the whole layer fades toward neutral.
const strength = clamp01((n - 1) / 3) * shape.bite;
const peak = Math.max(...sections.map((s) => s.energy || 0), 1e-6);
const climax = moments.climax;
const seenKind = new Map();
const countKind = new Map();
for (const s of sections) countKind.set(s.kind, (countKind.get(s.kind) || 0) + 1);
const states = sections.map((section, i) => {
const ordinal = seenKind.get(section.kind) || 0;
seenKind.set(section.kind, ordinal + 1);
const ordinalOf = countKind.get(section.kind) || 1;
const ctx = {
p: n > 1 ? i / (n - 1) : 0.5,
rise: climax > 0 ? clamp01(i / climax) : 1,
after: i > climax && n - 1 > climax ? clamp01((i - climax) / (n - 1 - climax)) : 0,
ord: ordinalOf > 1 ? ordinal / (ordinalOf - 1) : (n > 1 ? i / (n - 1) : 0.5),
energy: clamp01((section.energy || 0) / peak),
};
const raw = plot.curve(ctx, shape);
// The plot proposes and the song disposes. Blending the section's
// measured energy back in is what stops a story-driven tension from
// overriding a quiet section — the narrative may say "further than
// before", it may not say "loud" where the track is not.
raw.tension = raw.tension * 0.75 + ctx.energy * 0.25;
const state = { index: i, kind: section.kind, ordinal, ordinalOf, act: actFor(i, moments) };
for (const key of STORY_VARS) {
state[key] = clamp01(lerp(0.5, clamp01(raw[key]), strength));
}
// The journey keeps its full travel — it is the axis the whole video
// moves along, and halving it on a four-section track is the same as
// not having it. Its NEUTRAL is the plain progress ramp the slow axis
// used before this module existed, so a thin track degrades to exactly
// the old behaviour rather than to a flat line.
state.journey = clamp01(lerp(ctx.p, clamp01(raw.journey), strength));
return state;
});
// How long a story variable takes to arrive at its new value. Roughly a
// bar: long enough that nothing pops mid-crossfade (the failure buildSlope
// already taught us about at a boundary), short enough that against a
// five-minute video it still reads as a step rather than a ramp.
const barSeconds = track.tempo
? (track.tempo.period * track.tempo.beatsPerBar) / track.fps : 2;
const blendFrames = Math.max(6, Math.round(barSeconds * track.fps));
return {
plot: plot.name,
shape,
moments,
strength,
blendFrames,
// All of a video's scenes should travel the same WAY. Magnitude stays
// per scene, but the sign is the track's, because two scenes drifting
// against each other is what made the slow axis read as wobble rather
// than as a direction.
axisSign: rng.bool() ? 1 : -1,
// The oldest device available and the cheapest one here: the outro
// re-casts the intro's scene, played with the parameters the story has
// arrived at rather than the ones it opened with.
recap: plot.name === 'return' || rng.bool(0.15),
sections: states,
frames: sections.map((s) => ({ startFrame: s.startFrame, endFrame: s.endFrame })),
};
}
/** The story state of one section, or the neutral read. */
export function storyForSection(story, index) {
if (!story || !story.sections || !story.sections[index]) return { ...NEUTRAL_STATE };
return story.sections[index];
}
/**
* The story state at a frame.
*
* Held flat inside a section and blended over `blendFrames` after each boundary.
* Continuous in frame on purpose: the outgoing layer of a crossfade is still on
* screen when the boundary passes, and stepping its inputs there is a visible
* pop at exactly the moment the edit is trying to hide.
*/
export function storyStateAt(story, frame) {
if (!story || !story.sections || !story.sections.length) return { ...NEUTRAL_STATE };
const frames = story.frames;
let lo = 0;
let hi = frames.length - 1;
while (lo < hi) {
const mid = (lo + hi + 1) >> 1;
if (frames[mid].startFrame <= frame) lo = mid; else hi = mid - 1;
}
const current = story.sections[lo];
if (lo === 0) return current;
const into = frame - frames[lo].startFrame;
if (into >= story.blendFrames) return current;
const previous = story.sections[lo - 1];
const t = smooth(into / story.blendFrames);
const out = { index: current.index, kind: current.kind, act: current.act,
ordinal: current.ordinal, ordinalOf: current.ordinalOf };
for (const key of STORY_VARS) out[key] = lerp(previous[key], current[key], t);
out.journey = lerp(previous.journey, current.journey, t);
return out;
}
/** One line for the HUD, the look panel and check output. */
export function describeStory(story) {
if (!story) return 'story: none';
const m = story.moments;
const at = (i) => (i === null || i === undefined ? '–' : `§${i}`);
return `story: ${story.plot}${story.recap ? ' + recap' : ''} · `
+ `arrival ${at(m.arrival)} turn ${at(m.turn)} climax ${at(m.climax)} `
+ `resolution ${at(m.resolution)}`;
}

222
flow-state/src/look/directors.js
View File

@ -35,7 +35,34 @@
* when it is being quiet. * when it is being quiet.
*/ */
export const RESTFUL_FAMILIES = ['minimal', 'flow', 'organic']; export const RESTFUL_FAMILIES = ['minimal', 'flow', 'organic'];
const QUIET_KINDS = ['intro', 'breakdown', 'outro']; export const QUIET_KINDS = ['intro', 'breakdown', 'outro'];
/**
* How full a director lets the frame get.
*
* Scales the coverage budget in LookGenerator — the painted-area ceiling a
* section's stack is built against, where 1.0 is one filled frame and 2.0 is
* the hard cap. A point of view about what a song looks like includes how much
* is allowed to be happening at once, and it is the difference between a
* brutalist drop (one big thing) and a corrupt one (everything, over
* everything). Absent, a director is read as 1.2.
*/
const DEFAULT_CROWD = 1.2;
/**
* How willing a director is to let the frame BLAZE.
*
* A blaze is the shot composited additively over its ground instead of keyed
* onto it, so the two brightnesses sum and the highlights bloom out. It is a
* real effect and worth having — a drop that goes white for eight bars reads as
* the song peaking — and it was, until it was made a decision, simply what
* every section did: screen over a filled ground blew a median quarter of every
* frame to paper, all the time, in every video.
*
* So it is rationed. This is the appetite; the section still has to be loud and
* late in the story to earn one. Absent, a director is read as 0.15.
*/
const DEFAULT_BLAZE = 0.15;
/** /**
* Each director maps every section kind to three families, most-preferred * Each director maps every section kind to three families, most-preferred
@ -45,6 +72,17 @@ const QUIET_KINDS = ['intro', 'breakdown', 'outro'];
export const DIRECTORS = [ export const DIRECTORS = [
{ {
name: 'ambient', name: 'ambient',
// Space, not glare. A blaze here is the exception that proves it.
blaze: 0.1,
// Space is the subject, so the frame stays owed to it: one
// picture, and a pass over it only when the song is at its loudest.
crowd: 1.1,
// Patient camera to match: long moves, mostly along one line. See
// look/Camera.js — a director's point of view now includes how it
// shoots, not only what it points at.
camera: 'contemplative',
// Colour stays slow — two palettes that blend into one another.
palette: { count: [2, 2], progression: 'sequential', transition: 'blend' },
// The original table. A drop resolves into geometry; everything quiet is // The original table. A drop resolves into geometry; everything quiet is
// minimal. Still the most broadly applicable, so it keeps the most weight. // minimal. Still the most broadly applicable, so it keeps the most weight.
weight: 3, weight: 3,
@ -59,6 +97,15 @@ export const DIRECTORS = [
}, },
{ {
name: 'brutalist', name: 'brutalist',
// Mass does not glow. Almost never.
blaze: 0.05,
// One large thing, seen whole. Layering hides mass, which is the
// only thing this director is interested in.
crowd: 1.05,
// Holds, then commits to one large move. Architecture is looked AT.
camera: 'deliberate',
// Two palettes, story-led — the drop arrives in the second palette.
palette: { count: [2, 2], progression: 'storyLed', transition: 'cut' },
// Everything is architecture. Quiet means empty rather than soft, so it // Everything is architecture. Quiet means empty rather than soft, so it
// leads on minimal and reaches for organic last. // leads on minimal and reaches for organic last.
weight: 2, weight: 2,
@ -73,6 +120,15 @@ export const DIRECTORS = [
}, },
{ {
name: 'organicist', name: 'organicist',
// Light through leaves — bloom belongs to this world.
blaze: 0.2,
// Growth accumulates. Things overlap here because that is what
// living material does — nothing in this world is a single clean plate.
crowd: 1.45,
// Never settles, because nothing here is ever finished settling.
camera: 'roaming',
// Three palettes mapped by kind — each kind of section keeps a colour.
palette: { count: [3, 3], progression: 'kindLed', transition: 'blend' },
// Nothing is ever built; things grow and dissolve. Deliberately never // Nothing is ever built; things grow and dissolve. Deliberately never
// reaches for glitch — a point of view is defined by what it refuses. // reaches for glitch — a point of view is defined by what it refuses.
weight: 2, weight: 2,
@ -87,6 +143,15 @@ export const DIRECTORS = [
}, },
{ {
name: 'corrupt', name: 'corrupt',
// Overload is the subject. Half its drops go white.
blaze: 0.5,
// Everything over everything. The damage is the subject and it is
// never confined to one plate.
crowd: 1.7,
// Cuts with the camera already moving.
camera: 'kinetic',
// Most palettes, most contrast — the damage is legible as colour too.
palette: { count: [3, 4], progression: 'contrast', transition: 'cut' },
// The signal is damaged and the damage is the subject — everywhere the // The signal is damaged and the damage is the subject — everywhere the
// damage is allowed to be. Its quiet sections lead on flow, so the calm // damage is allowed to be. Its quiet sections lead on flow, so the calm
// reads as signal drifting rather than as rest. // reads as signal drifting rather than as rest.
@ -102,6 +167,16 @@ export const DIRECTORS = [
}, },
{ {
name: 'geometer', name: 'geometer',
// Exact, and occasionally exact and incandescent.
blaze: 0.15,
// Pattern on pattern is a moiré, which is a pattern. Layers are
// welcome as long as they are exact.
crowd: 1.3,
// Small, exact, always arrives — the pattern is the subject and the
// camera does not editorialise about it.
camera: 'precise',
// Three palettes traversed as a pong — the pattern's colour answers its form.
palette: { count: [2, 3], progression: 'pong', transition: 'blend' },
// Pattern first, everywhere, at every energy. The drop is not an // Pattern first, everywhere, at every energy. The drop is not an
// explosion, it is the pattern at its densest. // explosion, it is the pattern at its densest.
weight: 2, weight: 2,
@ -143,6 +218,151 @@ export function pickDirector(summary, rng) {
return rng.pickWeighted(DIRECTORS, weights); return rng.pickWeighted(DIRECTORS, weights);
} }
/** How full this director lets the frame get. See DEFAULT_CROWD. */
export function crowdOf(director) {
return (director && director.crowd) || DEFAULT_CROWD;
}
/** How willing this director is to let a section blaze. See DEFAULT_BLAZE. */
export function blazeOf(director) {
return (director && director.blaze !== undefined) ? director.blaze : DEFAULT_BLAZE;
}
export function directorByName(name) { export function directorByName(name) {
return DIRECTORS.find((d) => d.name === name) || DIRECTORS[0]; return DIRECTORS.find((d) => d.name === name) || DIRECTORS[0];
} }
// ── palette progression ───────────────────────────────────────────────
/**
* Build a per-cue palette assignment.
*
* `cues` is ArcDriver's flat list of (section,shot) spans; `sections` carries
* the story state each cue belongs to. `palettes` is the set LookGenerator built.
* The director's `palette.progression` names which strategy to use.
*
* Pure in (cues, sections, director, palettes, rng) — the seed makes the same
* song always traverse its palettes the same way.
*/
export function derivePalettePlan(cues, sections, director, rng, palettes) {
const count = palettes ? palettes.length : 1;
if (!cues || !cues.length || count <= 1) {
return {
name: 'single',
paletteCount: Math.max(1, count),
progression: 'single',
transition: 'cut',
cues: (cues || []).map(() => 0),
};
}
const cfg = (director && director.palette) || {};
const progression = cfg.progression || 'sequential';
const transition = cfg.transition || 'blend';
const n = cues.length;
const indices = new Array(n);
if (progression === 'sequential') {
for (let i = 0; i < n; i++) indices[i] = i % count;
} else if (progression === 'pong') {
const cycle = count > 1 ? 2 * count - 2 : 1;
for (let i = 0; i < n; i++) {
const k = i % cycle;
indices[i] = k < count ? k : cycle - k;
}
} else if (progression === 'kindLed') {
// Stable kind → palette map, shuffled per track so two directors with the
// same progression don't map identically.
const kinds = ['intro', 'build', 'drop', 'sustain', 'breakdown', 'outro'];
const offset = rng ? rng.int(0, Math.max(1, count) - 1) : 0;
// Seeded shuffle of palette indices for the kind mapping
const order = [...Array(count).keys()];
if (rng) {
for (let i = order.length - 1; i > 0; i--) {
const j = rng.int(0, i);
const t = order[i]; order[i] = order[j]; order[j] = t;
}
}
const kindMap = new Map();
for (let k = 0; k < kinds.length; k++) {
// Round-robin through shuffled palette order, with offset
kindMap.set(kinds[k], order[(k + offset) % count]);
}
for (let i = 0; i < n; i++) {
const cue = cues[i];
const sec = sections[cue.sectionIndex];
const kind = sec ? sec.kind : 'intro';
indices[i] = kindMap.has(kind) ? kindMap.get(kind) : (i % count);
}
} else if (progression === 'storyLed') {
for (let i = 0; i < n; i++) {
const cue = cues[i];
const sec = sections[cue.sectionIndex];
const story = sec ? sec.story : null;
if (story) {
if (story.act === 'climax') indices[i] = count - 1;
else if (story.act === 'resolution') indices[i] = Math.max(0, Math.floor(count / 2));
else {
const t = Math.max(0, Math.min(1, story.tension ?? 0.5));
indices[i] = Math.min(count - 1, Math.floor(t * count));
}
} else {
const p = n > 1 ? i / (n - 1) : 0.5;
indices[i] = Math.min(count - 1, Math.floor(p * count));
}
}
// Ensure at least one switch happens — storyLed on flat material can collapse
const uniq = new Set(indices);
if (uniq.size < 2 && count >= 2) {
// Nudge the middle cue to the other palette
const mid = Math.floor(n / 2);
indices[mid] = count > 2 ? 1 : 1;
}
} else if (progression === 'contrast') {
// Start seeded, then always pick the palette furthest in hue from the
// previous. Hue of the first colour in OKLCH is the representative.
const toHue = ([r, g, b]) => {
const lin = (x) => {
const v = Math.max(0, Math.min(1, x));
return v <= 0.04045 ? v / 12.92 : Math.pow((v + 0.055) / 1.055, 2.4);
};
const R = lin(r), G = lin(g), B = lin(b);
const l = 0.4122214708 * R + 0.5363325363 * G + 0.0514459929 * B;
const m = 0.2119034982 * R + 0.6806995451 * G + 0.1073969566 * B;
const s_ = 0.0883024619 * R + 0.2817188376 * G + 0.6299787005 * B;
const l_ = Math.cbrt(l), m_ = Math.cbrt(m), s = Math.cbrt(s_);
const a = 1.9779984951 * l_ - 2.4285922050 * m_ + 0.4505937099 * s;
const bb = 0.0259040371 * l_ + 0.7827717662 * m_ - 0.8086757660 * s;
return Math.atan2(bb, a);
};
const hues = palettes.map((pal) => { try { return toHue(pal[0]); } catch { return 0; } });
const hueDist = (a, b) => {
let dh = Math.abs(a - b) % (2 * Math.PI);
if (dh > Math.PI) dh = 2 * Math.PI - dh;
return dh;
};
indices[0] = rng ? rng.int(0, count - 1) : 0;
for (let i = 1; i < n; i++) {
const prev = indices[i - 1];
let best = (prev + 1) % count;
let bestD = -1;
for (let c = 0; c < count; c++) {
if (c === prev && count > 1) continue;
const d = hueDist(hues[c], hues[prev]);
if (d > bestD) { bestD = d; best = c; }
}
indices[i] = best;
}
} else {
for (let i = 0; i < n; i++) indices[i] = i % count;
}
return {
name: progression,
paletteCount: count,
progression,
transition,
cues: indices,
};
}

52
flow-state/src/look/framing.js
View File

@ -11,9 +11,15 @@
// rather than being a magnified 720p frame, which is why this is a coordinate // rather than being a magnified 720p frame, which is why this is a coordinate
// transform and not a post pass. It is also why it costs nothing at 4K. // transform and not a post pass. It is also why it costs nothing at 4K.
// //
// Framing is per shot and constant within it. A zoom that moves during a shot // Shot SIZE is per shot and constant within it. A zoom that moves during a shot
// is a different device — one that would fight the drift LFO and the slow axis, // is a different device — one that would fight the drift LFO and the slow axis,
// both of which already own continuous motion. // both of which already own continuous motion.
//
// The RECENTRE is not: where the camera is looking moves during a shot, and
// that lives in look/Camera.js. This module used to own both and treated them
// the same way, which is how the recentre ended up as a per-shot constant of
// about 3% of a half-frame at a random angle — a device that was present in
// every frame of every video and visible in none of them.
/** /**
* The shot sizes, as multipliers on the scene's coordinate scale. * The shot sizes, as multipliers on the scene's coordinate scale.
@ -24,9 +30,9 @@
* measured by pushing until the image stopped being worth looking at. * measured by pushing until the image stopped being worth looking at.
*/ */
export const SHOT_SIZES = { export const SHOT_SIZES = {
wide: { scale: 0.62, drift: 0.06 }, wide: { scale: 0.62 },
normal: { scale: 1.0, drift: 0.05 }, normal: { scale: 1.0 },
close: { scale: 1.7, drift: 0.10 }, close: { scale: 1.7 },
}; };
export const SHOT_SIZE_NAMES = Object.keys(SHOT_SIZES); export const SHOT_SIZE_NAMES = Object.keys(SHOT_SIZES);
@ -61,8 +67,10 @@ export function deriveFramingStyle(summary, rng) {
* @param {object|null} previous the previous shot's framing * @param {object|null} previous the previous shot's framing
* @param {number} energy section energy, 0..1 * @param {number} energy section energy, 0..1
* @param {Rng} rng * @param {Rng} rng
* @param {number} closeness 0..1 from the story — how near this point in the
* video wants to be to its subject
*/ */
export function frameShot(style, previous, energy, rng) { export function frameShot(style, previous, energy, rng, closeness = 0.5) {
if (style.mode === 'locked') return neutralFraming(); if (style.mode === 'locked') return neutralFraming();
const keep = previous && !rng.bool(style.changeChance); const keep = previous && !rng.bool(style.changeChance);
@ -71,10 +79,18 @@ export function frameShot(style, previous, energy, rng) {
// Loud material earns the close-ups; quiet material earns the wides. This // Loud material earns the close-ups; quiet material earns the wides. This
// is a lean rather than a rule, so an intro can still land on a close and // is a lean rather than a rule, so an intro can still land on a close and
// read as intimate instead of empty. // read as intimate instead of empty.
//
// The story tilts the same draw across the video: a track that has been
// approaching its subject for four minutes should not answer its climax
// with a wide just because the dice said so. Still a tilt — both sizes stay
// reachable everywhere, because a story told by never cutting wide again is
// one shot type held for five minutes, which is what framing was added to
// stop.
const near = Math.max(0, Math.min(1, closeness));
const weights = [ const weights = [
1 + (1 - energy) * 2.5, // wide (1 + (1 - energy) * 2.5) * (1.4 - near * 0.9), // wide
2, // normal 2, // normal
1 + energy * 2.5, // close (1 + energy * 2.5) * (0.6 + near * 0.9), // close
]; ];
let size = rng.pickWeighted(SHOT_SIZE_NAMES, weights); let size = rng.pickWeighted(SHOT_SIZE_NAMES, weights);
@ -90,17 +106,17 @@ export function frameShot(style, previous, energy, rng) {
// than the same sizes drawn less often. // than the same sizes drawn less often.
const scale = 1 + (spec.scale - 1) * style.range; const scale = 1 + (spec.scale - 1) * style.range;
// Recentring is what stops a close-up being a centre crop of the wide. Held // Recentring used to be decided here, as `spec.drift * style.range` at a
// small: the scenes are centred compositions and pushing far off centre // uniform random angle — capped at 0.10 of a half-frame, with a fresh
// finds their empty corners. // direction every shot. Measured over 121 cues it moved the frame by a
const angle = rng.range(0, Math.PI * 2); // median of 0.029 and successive shots mostly cancelled, so the device was
const amount = spec.drift * style.range * rng.range(0.3, 1); // inert. It is now the camera's, planned as a continuous path across the
// whole video and driven by the story. See look/Camera.js.
return { //
size, // `shift` stays on the returned framing so a look built without a camera —
scale, // a hand-made one, or a check constructing framings directly — still has
shift: [Math.cos(angle) * amount, Math.sin(angle) * amount], // the field every consumer already reads.
}; return { size, scale, shift: [0, 0] };
} }
export function neutralFraming() { export function neutralFraming() {

27
flow-state/src/look/palette.js
View File

@ -299,3 +299,30 @@ export function toHex([r, g, b]) {
const c = (v) => Math.round(Math.max(0, Math.min(1, v)) * 255).toString(16).padStart(2, '0'); const c = (v) => Math.round(Math.max(0, Math.min(1, v)) * 255).toString(16).padStart(2, '0');
return `#${c(r)}${c(g)}${c(b)}`; return `#${c(r)}${c(g)}${c(b)}`;
} }
/**
* Lerp two palettes per-colour in OKLCH, so hue travel stays perceptual and
* lightness doesn't wobble the way an RGB lerp does. Hue takes the short way
* round the wheel.
*/
export function lerpPalettes(a, b, t) {
const n = Math.max(a.length, b.length);
const tt = Math.max(0, Math.min(1, t));
const out = [];
for (let i = 0; i < n; i++) {
const ca = a[i % a.length];
const cb = b[i % b.length];
const [La, Ca, ha] = rgbToOklch(ca);
const [Lb, Cb, hb] = rgbToOklch(cb);
let dh = hb - ha;
dh = ((dh + Math.PI) % (2 * Math.PI)) - Math.PI;
// Near-grey colours have unstable hue — fade hue influence with chroma.
// For now lerp directly; OKLCH hue of a near-zero chroma is still stable
// enough at 6-colour palette sizes.
const L = La + (Lb - La) * tt;
const C = Ca + (Cb - Ca) * tt;
const h = ha + dh * tt;
out.push(oklchToRgb(L, Math.max(0, C), h));
}
return out;
}

32
flow-state/src/look/paletteArc.js
View File

@ -24,7 +24,7 @@ export const MAX_HUE_ROTATION = 0.6; // radians, ~34 degrees
export const MAX_CHROMA_SCALE = 0.35; // ±35% saturation export const MAX_CHROMA_SCALE = 0.35; // ±35% saturation
export const MAX_LIGHT_SHIFT = 0.07; // OKLCH lightness export const MAX_LIGHT_SHIFT = 0.07; // OKLCH lightness
export const ARC_MODES = ['static', 'drift', 'sections', 'lift']; export const ARC_MODES = ['static', 'drift', 'sections', 'lift', 'narrative'];
/** /**
* How this track's colour moves. * How this track's colour moves.
@ -34,7 +34,12 @@ export const ARC_MODES = ['static', 'drift', 'sections', 'lift'];
* track whose colour holds is a legitimate choice and one in six or so gets it. * track whose colour holds is a legitimate choice and one in six or so gets it.
*/ */
export function derivePaletteArc(summary, rng) { export function derivePaletteArc(summary, rng) {
const mode = rng.pickWeighted(ARC_MODES, [1.5, 3, 3, 2.5]); // 'narrative' is the same movement as 'drift', keyed on where the story is
// rather than on how much of the file has elapsed. It is weighted highest
// because it is the only mode whose colour change lands WITH something: a
// clock has no reason to turn the picture warmer at three minutes, and the
// arrival of a climax does.
const mode = rng.pickWeighted(ARC_MODES, [1.5, 2.5, 3, 2.5, 3.5]);
const dir = rng.bool() ? 1 : -1; const dir = rng.bool() ? 1 : -1;
return { return {
@ -72,8 +77,11 @@ export function derivePaletteArc(summary, rng) {
* @param {number} ctx.progress 0..1 through the track * @param {number} ctx.progress 0..1 through the track
* @param {string} ctx.sectionKind kind of the section this frame is in * @param {string} ctx.sectionKind kind of the section this frame is in
* @param {object} ctx.features FeatureTrack row * @param {object} ctx.features FeatureTrack row
* @param {object} ctx.story story state for this frame, see look/Story.js
*/ */
export function paletteShiftAt(arc, { progress = 0, sectionKind = '', features = null } = {}) { export function paletteShiftAt(arc, {
progress = 0, sectionKind = '', features = null, story = null,
} = {}) {
if (!arc) return { hue: 0, chroma: 1, lightness: 0 }; if (!arc) return { hue: 0, chroma: 1, lightness: 0 };
// The slow underlying travel, present in every mode. Eased rather than // The slow underlying travel, present in every mode. Eased rather than
@ -97,6 +105,20 @@ export function paletteShiftAt(arc, { progress = 0, sectionKind = '', features =
chroma = 1 + arc.chromaLift * (drive * 2 - 1); chroma = 1 + arc.chromaLift * (drive * 2 - 1);
lightness = arc.lightLift * (drive - 0.35); lightness = arc.lightLift * (drive - 0.35);
hue += arc.kindHue[sectionKind] * 0.4 || 0; hue += arc.kindHue[sectionKind] * 0.4 || 0;
} else if (arc.mode === 'narrative') {
// Colour follows the STORY rather than the clock. The journey carries
// the hue, tension opens and closes the saturation, and what has been
// revealed lifts the value — so the frame at the climax is a colour the
// opening implied and the resolution comes back off it.
//
// A track with no story degrades to 'drift': journey is the plain
// progress ramp when Story.js has nothing to work with, and the two
// expressions are then identical.
const s = story || {};
const journey = s.journey ?? eased;
hue = arc.hueTravel * journey;
chroma = 1 + arc.chromaLift * ((s.tension ?? 0.5) * 2 - 1);
lightness = arc.lightLift * ((s.reveal ?? 0.5) - 0.4);
} }
return { return {
@ -113,5 +135,9 @@ export function describePaletteArc(arc) {
if (!arc || arc.mode === 'static') return 'colour: held'; if (!arc || arc.mode === 'static') return 'colour: held';
if (arc.mode === 'drift') return `colour: drift ${(arc.hueTravel * 57.3).toFixed(0)}°`; if (arc.mode === 'drift') return `colour: drift ${(arc.hueTravel * 57.3).toFixed(0)}°`;
if (arc.mode === 'lift') return `colour: lift ±${(arc.chromaLift * 100).toFixed(0)}% sat`; if (arc.mode === 'lift') return `colour: lift ±${(arc.chromaLift * 100).toFixed(0)}% sat`;
if (arc.mode === 'narrative') {
return `colour: narrative ${(arc.hueTravel * 57.3).toFixed(0)}° / `
+ `±${(arc.chromaLift * 100).toFixed(0)}% sat`;
}
return `colour: per-section ±${(Math.max(...Object.values(arc.kindHue).map(Math.abs)) * 57.3).toFixed(0)}°`; return `colour: per-section ±${(Math.max(...Object.values(arc.kindHue).map(Math.abs)) * 57.3).toFixed(0)}°`;
} }

37
flow-state/src/look/shots.js
View File

@ -51,7 +51,17 @@ export const HARD_CUT_ENERGY = 0.66;
* Every entry is a power-of-two bar count, so a cut is always on a phrase line * Every entry is a power-of-two bar count, so a cut is always on a phrase line
* of some depth even before it is snapped to a downbeat. * of some depth even before it is snapped to a downbeat.
*/ */
function rhythmFor(energy, rng) { function rhythmFor(energy, rng, story = null) {
// Where the section sits in the story shifts which band it cuts in. A song
// does not only get louder toward its climax, it gets more urgent, and edit
// rate is the one register that says urgency without changing the image at
// all. The resolution goes the other way and holds — the last thing a video
// should do is keep cutting at the pace of the thing that just ended.
if (story) {
const urgency = (story.tension - 0.5) * 0.22;
energy = Math.max(0, Math.min(1, energy + urgency));
if (story.act === 'resolution') energy = Math.min(energy, 0.44);
}
if (energy > 0.72) { if (energy > 0.72) {
// Loud material: quick cuts, but still answered by a longer hold. // Loud material: quick cuts, but still answered by a longer hold.
return rng.pick([[4, 4, 8], [8, 4, 4], [4, 4, 4, 8], [8, 8, 4, 4], [4, 8, 4, 4]]); return rng.pick([[4, 4, 8], [8, 4, 4], [4, 4, 4, 8], [8, 8, 4, 4], [4, 8, 4, 4]]);
@ -116,17 +126,29 @@ function snapCut(from, ideal, downbeats, tolerance) {
* @param {object} bias the section's bias, for energy * @param {object} bias the section's bias, for energy
* @param {number} variantCount how many stage visuals the section has * @param {number} variantCount how many stage visuals the section has
* @param {Rng} rng * @param {Rng} rng
* @param {object|null} story this section's story state, see look/Story.js
* @returns {Array<{index,startFrame,endFrame,variant,hardCut}>} * @returns {Array<{index,startFrame,endFrame,variant,hardCut}>}
*/ */
export function planShots(section, track, bias, variantCount, rng) { export function planShots(section, track, bias, variantCount, rng, story = null) {
const fps = track.fps; const fps = track.fps;
const duration = Math.max(0, section.end - section.start); const duration = Math.max(0, section.end - section.start);
const barSeconds = (track.tempo.period * track.tempo.beatsPerBar) / fps; const barSeconds = (track.tempo.period * track.tempo.beatsPerBar) / fps;
const lengths = fitPattern(rhythmFor(bias.energy, rng), barSeconds); const lengths = fitPattern(rhythmFor(bias.energy, rng, story), barSeconds);
const shortest = Math.min(...lengths); const shortest = Math.min(...lengths);
// A section with only one visual to show has nothing to cut to. // A section with only one visual to show has nothing to cut TO. It still
// gets shot boundaries: a boundary changes the FRAMING as well as the
// image, so the same visual filmed again at another size is a shot, and the
// ceiling is about how long one image is held rather than about how many
// images a section has.
//
// This used to skip cut planning entirely, and the exemption swallowed the
// ceiling with it — measured, a 150 BPM track held one visual for 151.98
// seconds, seven times the limit, which is exactly the complaint this whole
// level of hierarchy exists to answer. What it does still suppress is the
// hard cut: cutting straight between two framings of one image is a jump
// cut, so those boundaries always dissolve.
const single = variantCount < 2; const single = variantCount < 2;
// Walk the pattern, laying shots end to end from the section start. Each cut // Walk the pattern, laying shots end to end from the section start. Each cut
@ -138,7 +160,7 @@ export function planShots(section, track, bias, variantCount, rng) {
const downbeats = track.tempo.downbeats || []; const downbeats = track.tempo.downbeats || [];
const cuts = []; const cuts = [];
if (!single) { {
let at = section.start; let at = section.start;
for (let k = 0; k < 512; k++) { for (let k = 0; k < 512; k++) {
const raw = at + lengths[k % lengths.length]; const raw = at + lengths[k % lengths.length];
@ -182,7 +204,10 @@ export function planShots(section, track, bias, variantCount, rng) {
// edited, but on anything calmer it reads as a glitch, so cuts are // edited, but on anything calmer it reads as a glitch, so cuts are
// gated on real energy rather than sprinkled everywhere: nothing below // gated on real energy rather than sprinkled everywhere: nothing below
// the threshold ever cuts, and only the loudest material cuts often. // the threshold ever cuts, and only the loudest material cuts often.
hardCut: bias.energy > HARD_CUT_ENERGY // A single-visual section changes framing, not image. Cutting hard
// between two framings of the same thing is a jump cut, so those
// boundaries always dissolve.
hardCut: !single && bias.energy > HARD_CUT_ENERGY
&& rng.bool(Math.min(0.85, (bias.energy - HARD_CUT_ENERGY) * 2.5)), && rng.bool(Math.min(0.85, (bias.energy - HARD_CUT_ENERGY) * 2.5)),
}); });
} }

55
flow-state/src/look/stack.js Normal file
View File

@ -0,0 +1,55 @@
// What a layer stack is made of, and how to ask it questions.
//
// A stack used to be "the scene, then whatever was composited over it", so
// `layers[0]` meant the section's scene everywhere in the program. It no longer
// does: every stack now starts with a GROUND — a filled canvas the shot happens
// on — and the scene the section is ABOUT sits above it.
//
// Everything that used to reach for `layers[0]` wants the subject, not the
// ground: the param panel edits it, the HUD names it, and the variety report
// counts it. Those all go through `subjectOf` now. Reading the ground as the
// section's scene would be actively wrong for the measurements — grounds come
// from a twenty-scene pool, so a report that counted them would show a library
// three times smaller than the one actually on screen.
import { coverageOf } from '../scenes/surface.js';
/** Layers carrying this role are the bed, not the shot. */
export const GROUND = 'ground';
export function isGround(layer) {
return !!layer && layer.role === GROUND;
}
/** Index of the layer the section is about. */
export function subjectIndexOf(stack) {
const at = stack.findIndex((l) => !isGround(l));
return at < 0 ? 0 : at;
}
/** The layer the section is about — the shot, as opposed to what it stands on. */
export function subjectOf(stack) {
return stack[subjectIndexOf(stack)];
}
/** The ground under a stack, or null if the subject is its own ground. */
export function groundOf(stack) {
return stack.find(isGround) || null;
}
/** Everything above the subject: the passes composited over the shot. */
export function overlaysOf(stack) {
return stack.slice(subjectIndexOf(stack) + 1);
}
/**
* How much painted frame a stack adds up to, in units of one filled frame.
*
* Deliberately a SUM and not a union: two layers each painting 60% do not add
* up to 120% of a screen, but they do add up to two things happening at once,
* and that is the quantity the budget is about. 2.0 is the ceiling — see
* MAX_COVERAGE in LookGenerator.
*/
export function stackCoverage(stack) {
return stack.reduce((sum, l) => sum + coverageOf(l.module), 0);
}

150
flow-state/src/main.js
View File

@ -5,8 +5,10 @@ import { formatTime } from './audio/decode.js';
import { describeLook } from './look/LookGenerator.js'; import { describeLook } from './look/LookGenerator.js';
import { toHex } from './look/palette.js'; import { toHex } from './look/palette.js';
import { applyGrainToPost, describeGrain, GRAIN_MASKS, GRAIN_MODES } from './look/grain.js'; import { applyGrainToPost, describeGrain, GRAIN_MASKS, GRAIN_MODES } from './look/grain.js';
import { renderClickTrack, audioBufferToWavBlob } from './audio/clicktrack.js'; import { renderClickTrack, audioBufferToWavBlob } from './audio/metronome.js';
import { Exporter, exportSegment, downloadBlob, PRESETS, isSupported } from './export/Exporter.js'; import { Exporter, exportSegment, downloadBlob, PRESETS, isSupported } from './export/Exporter.js';
import { subjectOf, groundOf, overlaysOf, stackCoverage } from './look/stack.js';
import { coverageOf as sceneCoverage } from './scenes/surface.js';
const QUALITY = { const QUALITY = {
draft: 0.5, // half resolution — for scrubbing heavy stacks draft: 0.5, // half resolution — for scrubbing heavy stacks
@ -56,7 +58,7 @@ const paramPanel = new ParamPanel(document.createElement('div'), onParamChange);
// ---------------------------------------------------------------- loading // ---------------------------------------------------------------- loading
async function loadFile(file) { async function loadFile(file, { seed = null } = {}) {
if (state.busy) return; if (state.busy) return;
state.busy = true; state.busy = true;
stopPlayback(); stopPlayback();
@ -76,7 +78,7 @@ async function loadFile(file) {
await state.show.load(file, (stage, fraction) => { await state.show.load(file, (stage, fraction) => {
dom.thStep.textContent = stage; dom.thStep.textContent = stage;
dom.thFill.style.width = `${Math.round((fraction || 0) * 100)}%`; dom.thFill.style.width = `${Math.round((fraction || 0) * 100)}%`;
}); }, { seed });
dom.audio.src = URL.createObjectURL(file); dom.audio.src = URL.createObjectURL(file);
dom.thLabel.textContent = 'loaded track'; dom.thLabel.textContent = 'loaded track';
@ -113,6 +115,35 @@ if (dom.changeTrack) dom.changeTrack.addEventListener('click', () => dom.fileInp
dom.fileInput.addEventListener('change', (e) => { dom.fileInput.addEventListener('change', (e) => {
if (e.target.files[0]) loadFile(e.target.files[0]); if (e.target.files[0]) loadFile(e.target.files[0]);
}); });
/**
* `?song=centre` — open a bank song straight from a debug page.
*
* The filmstrip reduces a song to nine stills; the only way to argue with that
* reading is to watch the thing move, and until now that meant remembering
* which of seventeen near-identically-named wavs to drag in. `&seed=` carries
* the strip's seed across so the video you watch is the one it measured.
*
* Dev-only in practice: `test/songs/` is served by vite from the project root
* and is not part of a build. The name is matched against a bare word rather
* than the bank list so the app does not have to import the synth.
*/
async function loadBankSong(name, seed) {
if (!/^[a-z0-9_-]+$/i.test(name)) return;
const url = `/test/songs/${name}.wav`;
try {
const res = await fetch(url);
if (!res.ok) throw new Error(`${res.status} ${res.statusText}`);
const blob = await res.blob();
await loadFile(new File([blob], `${name}.wav`, { type: 'audio/wav' }), { seed });
} catch (err) {
dom.thLabel.textContent = 'no such song';
dom.thName.hidden = false;
dom.thName.textContent = name;
dom.thStep.textContent = `${url}: ${err.message} — run npm run build:songs`;
console.error(err);
}
}
document.addEventListener('dragover', (e) => e.preventDefault()); document.addEventListener('dragover', (e) => e.preventDefault());
document.addEventListener('drop', (e) => { document.addEventListener('drop', (e) => {
e.preventDefault(); e.preventDefault();
@ -265,7 +296,8 @@ function renderPanel() {
if (state.tab === 'scene') { if (state.tab === 'scene') {
paramPanel.container = dom.panelBody; paramPanel.container = dom.panelBody;
paramPanel.build(section.layers[0].module, section.layers[0].params); const subject = subjectOf(section.layers);
paramPanel.build(subject.module, subject.params);
// The section's stage visuals, with the one currently on screen marked. // The section's stage visuals, with the one currently on screen marked.
// Params above edit the anchor (variant 0) — the image the section opens // Params above edit the anchor (variant 0) — the image the section opens
@ -278,20 +310,29 @@ function renderPanel() {
list.innerHTML = `<div class="pp-sub">stage visuals · ${shots.length} shots</div>` + list.innerHTML = `<div class="pp-sub">stage visuals · ${shots.length} shots</div>` +
section.variants.map((stack, v) => section.variants.map((stack, v) =>
`<div class="pp-react-row${v === active ? ' current' : ''}">` + `<div class="pp-react-row${v === active ? ' current' : ''}">` +
`<span>${v === 0 ? '&#9679;' : '&#9675;'} ${stack[0].module.name}</span>` + `<span>${v === 0 ? '&#9679;' : '&#9675;'} ${subjectOf(stack).module.name}</span>` +
`<span class="pp-feature">${shots.filter((s) => s.variant === v).length}&times;</span>` + `<span class="pp-feature">${shots.filter((s) => s.variant === v).length}&times;</span>` +
`</div>`).join(''); `</div>`).join('');
dom.panelBody.appendChild(list); dom.panelBody.appendChild(list);
} }
if (section.layers.length > 1) { // The rest of the stack, named by the job each layer is doing rather
// than by its index: what is under the shot and what is over it are
// different questions, and the panel used to call both "layered over".
const ground = groundOf(section.layers);
const overlays = overlaysOf(section.layers);
if (ground || overlays.length) {
const row = (l, label) =>
`<div class="pp-react-row"><span>${l.module.name}</span>` +
`<span class="pp-feature">${label}</span>` +
`<span class="pp-amount">${(sceneCoverage(l.module) * 100).toFixed(0)}%</span></div>`;
const note = document.createElement('div'); const note = document.createElement('div');
note.className = 'pp-reactive'; note.className = 'pp-reactive';
note.innerHTML = '<div class="pp-sub">accent layer</div>' + note.innerHTML =
section.layers.slice(1).map((l) => `<div class="pp-sub">stack · ${(stackCoverage(section.layers) * 100).toFixed(0)}% painted</div>` +
`<div class="pp-react-row"><span>${l.module.name}</span>` + (ground ? row(ground, 'ground') : '') +
`<span class="pp-feature">${l.blend}</span>` + row(subject, 'shot') +
`<span class="pp-amount">${l.opacity.toFixed(2)}</span></div>`).join(''); overlays.map((l) => row(l, l.blend)).join('');
dom.panelBody.appendChild(note); dom.panelBody.appendChild(note);
} }
return; return;
@ -302,6 +343,14 @@ function renderPanel() {
// The track's production design. Scenes that cannot express what it is // The track's production design. Scenes that cannot express what it is
// built on were never cast — see look/Personality.js. // built on were never cast — see look/Personality.js.
const personality = show.look.personality; const personality = show.look.personality;
const palettes = show.look.palettes || [show.look.palette];
const schemes = show.look.paletteSchemes || [show.look.paletteScheme];
const plan = show.look.palettePlan;
const activeIdx = show.arc ? show.arc.paletteIndexAt(show.timeline.frame) : 0;
const blend = show.arc ? show.arc.paletteBlendAt(show.timeline.frame) : null;
const planLabel = plan
? `${plan.progression}/${plan.transition} · ${palettes.length} palettes`
: `${schemes[0] || ''}`;
dom.panelBody.innerHTML = ` dom.panelBody.innerHTML = `
<div class="pp-heading"><span class="pp-name">${show.fileName || 'track'}</span></div> <div class="pp-heading"><span class="pp-name">${show.fileName || 'track'}</span></div>
<div class="kv"><span>seed</span><b>${show.look.seed.toString(16)}</b></div> <div class="kv"><span>seed</span><b>${show.look.seed.toString(16)}</b></div>
@ -309,7 +358,7 @@ function renderPanel() {
<div class="kv"><span>tempo conf.</span><b>${show.track.tempo.confidence.toFixed(2)}</b></div> <div class="kv"><span>tempo conf.</span><b>${show.track.tempo.confidence.toFixed(2)}</b></div>
<div class="kv"><span>duration</span><b>${formatTime(show.duration)}</b></div> <div class="kv"><span>duration</span><b>${formatTime(show.duration)}</b></div>
<div class="kv"><span>sections</span><b>${show.track.sections.length}</b></div> <div class="kv"><span>sections</span><b>${show.track.sections.length}</b></div>
<div class="kv"><span>scheme</span><b>${show.look.paletteScheme}</b></div> <div class="kv"><span>director</span><b>${show.look.director}${plan ? ` · ${plan.progression}` : ''}</b></div>
<div class="kv"><span>built on</span><b>${personality.signature.join(' + ') || 'nothing'}</b></div> <div class="kv"><span>built on</span><b>${personality.signature.join(' + ') || 'nothing'}</b></div>
<div class="kv"><span>form</span><b>${personality.shape.sides || 'round'}${ <div class="kv"><span>form</span><b>${personality.shape.sides || 'round'}${
personality.shape.sides ? '-sided' : ''}</b></div> personality.shape.sides ? '-sided' : ''}</b></div>
@ -319,19 +368,33 @@ function renderPanel() {
? personality.style.symmetry + '-fold · ' : ''}line ${personality.style.lineWeight.toFixed(2)}</b></div> ? personality.style.symmetry + '-fold · ' : ''}line ${personality.style.lineWeight.toFixed(2)}</b></div>
<div class="kv"><span>brightness</span><b>${summary.meanCentroid.toFixed(3)}</b></div> <div class="kv"><span>brightness</span><b>${summary.meanCentroid.toFixed(3)}</b></div>
<div class="kv"><span>dynamics</span><b>${summary.dynamicRange.toFixed(3)}</b></div> <div class="kv"><span>dynamics</span><b>${summary.dynamicRange.toFixed(3)}</b></div>
<div class="swatches">${show.look.palette.map((c) => <div class="pp-sub">palettes · ${planLabel}</div>
<div id="palette-list">${palettes.map((pal, i) => {
const sc = schemes[i] || schemes[0] || '';
const isActive = i === activeIdx;
const isBlendSrc = blend && (i === blend.from || i === blend.to);
const cls = isActive ? 'active' : (isBlendSrc ? 'blending' : '');
const tag = isActive
? (blend ? `● blend ${(blend.t * 100).toFixed(0)}%` : '● active')
: (isBlendSrc ? '○ blend' : '');
return `<div class="pal-row ${cls}" data-pal="${i}">
<div class="pal-meta"><span class="pal-idx">#${i + 1}</span><span class="pal-scheme">${sc}</span><span class="pal-tag">${tag}</span></div>
<div class="swatches pal-swatches">${pal.map((c) =>
`<span class="sw" style="background:${toHex(c)}" title="${toHex(c)}"></span>`).join('')}</div> `<span class="sw" style="background:${toHex(c)}" title="${toHex(c)}"></span>`).join('')}</div>
</div>`;
}).join('')}</div>
<div id="palette-blend-hint" class="hint"${blend ? '' : ' hidden'}>${blend ? `blending #${blend.from + 1} → #${blend.to + 1} · ${(blend.t * 100).toFixed(0)}% through cut` : ''}</div>
<div class="pp-sub">sections</div> <div class="pp-sub">sections</div>
${show.look.sections.map((s, i) => ` ${show.look.sections.map((s, i) => `
<div class="kv ${i === index ? 'current' : ''}"> <div class="kv ${i === index ? 'current' : ''}">
<span>${s.kind}${s.locked ? ' &#128274;' : ''} <span>${s.kind}${s.locked ? ' &#128274;' : ''}
${s.shots ? `<i class="dim">${s.shots.length} shots</i>` : ''}</span> ${s.shots ? `<i class="dim">${s.shots.length} shots</i>` : ''}</span>
<b>${(s.variants || [s.layers]).map((v) => v[0].module.name).join(' / ')}</b> <b>${(s.variants || [s.layers]).map((v) => subjectOf(v).module.name).join(' / ')}</b>
</div>`).join('')} </div>`).join('')}
<button id="btn-clicktrack" class="wide">download click track</button> <button id="btn-metronome" class="wide">download click track</button>
<div class="hint">Mixes clicks onto the detected beat grid. If they don't sit on <div class="hint">Mixes clicks onto the detected beat grid. If they don't sit on
the beat, tempo detection is wrong and everything downstream inherits it.</div>`; the beat, tempo detection is wrong and everything downstream inherits it.</div>`;
document.getElementById('btn-clicktrack').addEventListener('click', downloadClickTrack); document.getElementById('btn-metronome').addEventListener('click', downloadClickTrack);
return; return;
} }
@ -562,11 +625,11 @@ document.getElementById('btn-segment').addEventListener('click', () => runExport
async function downloadClickTrack() { async function downloadClickTrack() {
if (!state.show.ready) return; if (!state.show.ready) return;
const button = document.getElementById('btn-clicktrack'); const button = document.getElementById('btn-metronome');
button.textContent = 'rendering…'; button.textContent = 'rendering…';
try { try {
const buffer = await renderClickTrack(state.show.audioBuffer, state.show.track.tempo); const buffer = await renderClickTrack(state.show.audioBuffer, state.show.track.tempo);
downloadBlob(audioBufferToWavBlob(buffer), `${state.show.fileName}-clicktrack.wav`); downloadBlob(audioBufferToWavBlob(buffer), `${state.show.fileName}-metronome.wav`);
button.textContent = 'download click track'; button.textContent = 'download click track';
} catch (err) { } catch (err) {
button.textContent = `failed: ${err.message}`; button.textContent = `failed: ${err.message}`;
@ -590,6 +653,37 @@ window.addEventListener('resize', resize);
let lastPanelSection = -1; let lastPanelSection = -1;
let lastRenderedFrame = -1; let lastRenderedFrame = -1;
function syncPalettesLive() {
if (state.tab !== 'look' || !state.show.ready || !state.show.arc) return;
const palettes = state.show.look.palettes;
if (!palettes || palettes.length <= 1) return;
const list = document.getElementById('palette-list');
if (!list) return;
const frame = state.show.timeline.frame;
const active = state.show.arc.paletteIndexAt(frame);
const blend = state.show.arc.paletteBlendAt(frame);
for (const row of list.querySelectorAll('.pal-row')) {
const idx = Number(row.dataset.pal);
const isActive = idx === active;
const isBlendSrc = !!blend && (idx === blend.from || idx === blend.to) && !isActive;
row.classList.toggle('active', isActive);
row.classList.toggle('blending', isBlendSrc);
const tag = row.querySelector('.pal-tag');
if (tag) {
if (isActive) tag.textContent = blend ? `● blend ${(blend.t * 100).toFixed(0)}%` : '● active';
else if (isBlendSrc) tag.textContent = '○ blend';
else tag.textContent = '';
}
}
const hint = document.getElementById('palette-blend-hint');
if (hint) {
if (blend) {
hint.hidden = false;
hint.textContent = `blending #${blend.from + 1} → #${blend.to + 1} · ${(blend.t * 100).toFixed(0)}% through cut`;
} else hint.hidden = true;
}
}
function frame(now) { function frame(now) {
requestAnimationFrame(frame); requestAnimationFrame(frame);
const show = state.show; const show = state.show;
@ -631,6 +725,8 @@ function frame(now) {
if (state.tab === 'scene' || state.tab === 'look') renderPanel(); if (state.tab === 'scene' || state.tab === 'look') renderPanel();
} }
syncPalettesLive();
if (state.hudVisible) { if (state.hudVisible) {
const f = show.track.at(show.timeline.frame); const f = show.track.at(show.timeline.frame);
dom.hud.innerHTML = dom.hud.innerHTML =
@ -638,6 +734,12 @@ function frame(now) {
`<div>${show.engine.width}×${show.engine.height} · ${state.quality}</div>` + `<div>${show.engine.width}×${show.engine.height} · ${state.quality}</div>` +
`<div>section ${sectionIndex} ${arc.kind} · ${arc.sceneName}</div>` + `<div>section ${sectionIndex} ${arc.kind} · ${arc.sceneName}</div>` +
`<div>layers ${show.arc.activeLayers.length} · build ${(f.buildSlope || 0).toFixed(2)}</div>` + `<div>layers ${show.arc.activeLayers.length} · build ${(f.buildSlope || 0).toFixed(2)}</div>` +
// Where the video is in its story, so "is it going anywhere" is
// something you can read off the HUD rather than infer. See
// look/Story.js.
`<div>${show.look.story ? show.look.story.plot : 'no story'} · ${arc.act || ''} · ` +
`tension ${(arc.tension ?? 0.5).toFixed(2)} reveal ${(arc.reveal ?? 0.5).toFixed(2)} ` +
`journey ${(arc.journey ?? 0).toFixed(2)}</div>` +
`<div>loud ${f.loudness.toFixed(2)} low ${f.bandLow.toFixed(2)} high ${f.bandHigh.toFixed(2)}</div>` + `<div>loud ${f.loudness.toFixed(2)} low ${f.bandLow.toFixed(2)} high ${f.bandHigh.toFixed(2)}</div>` +
`<div>beat ${f.beat.toFixed(2)} bar ${f.barPhase.toFixed(2)} flux ${f.flux.toFixed(2)}</div>`; `<div>beat ${f.beat.toFixed(2)} bar ${f.barPhase.toFixed(2)} flux ${f.flux.toFixed(2)}</div>`;
} }
@ -652,3 +754,15 @@ if (import.meta.env && import.meta.env.DEV) {
requestAnimationFrame(frame); requestAnimationFrame(frame);
resize(); resize();
// Tell the boot guard in index.html that the module graph made it all the way
// through. Without this the guard cannot distinguish "still starting" from
// "never going to start", and the failure it exists to catch is exactly the one
// that produces no error at all in the page.
window.__FLOW_STATE_READY__ = true;
const query = new URLSearchParams(location.search);
if (query.get('song')) {
const seed = query.get('seed');
loadBankSong(query.get('song'), seed === null ? null : (Number(seed) >>> 0));
}

67
flow-state/src/params/schema.js
View File

@ -53,6 +53,58 @@ export const REACTIVE_FEATURES = [
export const REACTIVE_RESPONSES = ['linear', 'spike', 'smooth', 'inverse']; export const REACTIVE_RESPONSES = ['linear', 'spike', 'smooth', 'inverse'];
/**
* Identity artifacts a scene can consume. See look/Identity.js and EPIC-3.md.
*
* A trait is a modifier a scene may honour; an artifact is CONTENT the scene
* draws. Declaring one is a commitment the gate enforces: swap the song's
* identity and a scene that claims `cast` must produce a different picture.
*
* `form` is `cast` in three dimensions — the protagonist as an assembly of
* solids rather than as an outline. A scene declaring it marches the object,
* which is what makes its silhouette change as the shot moves; a scene that
* only stamps the flat profile declares `cast` and not this.
*/
export const ARTIFACT_NAMES = ['cast', 'ink', 'staging', 'form'];
/**
* Whether a scene's image depends on the FRAME BEFORE IT.
*
* Read off the shader rather than declared, because the declaration would be a
* second copy of something the source already says exactly: a scene reads
* history if and only if it calls `prev()`.
*
* What it decides is where a scene may sit in a stack. `prev()` returns the
* whole composited previous frame — everything, including whatever was layered
* ON TOP of this scene — so a datamosh or a time smear underneath a shot is not
* grounding it, it is recycling it. Two things follow, and both were measured
* the moment such a scene became a bed: the render stops being reproducible from
* a seek (state carries across frames that a seek has not rendered), and small
* numeric differences compound frame over frame instead of staying put — 91/255
* between two WebGL contexts, against a ceiling of 4.
*
* These scenes are exactly right as a shot or as a pass over one. They are only
* wrong as the thing underneath. See canGround in scenes/surface.js.
*/
export function readsHistory(module) {
return typeof module.shader === 'string' && /\bprev\s*\(/.test(module.shader);
}
/**
* Whether a scene can carry a section on its own.
*
* Composable and background-capable are NOT opposites: most sparse scenes are
* complete pictures that happen to leave room — a skyline against night sky
* paints a quarter of the frame and is still a shot. Coverage says how much is
* painted, not whether what's painted is a picture, so it cannot decide this.
*
* Opt out with `background: false` only when a scene is genuinely nothing but
* elements — points in empty space, with no ground of its own to stand on.
*/
export function canBackground(module) {
return module.background !== false;
}
/** /**
* Personality traits a scene can honour. See look/Personality.js. * Personality traits a scene can honour. See look/Personality.js.
* *
@ -231,6 +283,21 @@ export function validateModule(module) {
if (!TRAIT_NAMES.includes(t)) errors.push(`${id}: unknown trait '${t}'`); if (!TRAIT_NAMES.includes(t)) errors.push(`${id}: unknown trait '${t}'`);
} }
} }
// `surface` was a declaration and is now measured — see scenes/surface.js.
// Rejected rather than ignored, so a scene file carrying a stale one is a
// loud error instead of a line that quietly means nothing.
if (module.surface !== undefined) {
errors.push(`${id}: 'surface' is derived from scenes/metadata.json — remove the declaration`);
}
if (module.consumes !== undefined) {
if (!Array.isArray(module.consumes)) {
errors.push(`${id}: \`consumes\` must be an array of ${ARTIFACT_NAMES.join('/')}`);
} else {
for (const a of module.consumes) {
if (!ARTIFACT_NAMES.includes(a)) errors.push(`${id}: unknown artifact '${a}'`);
}
}
}
if (module.texture !== undefined if (module.texture !== undefined
&& (typeof module.texture !== 'number' || module.texture < 0 || module.texture > 2)) { && (typeof module.texture !== 'number' || module.texture < 0 || module.texture > 2)) {
errors.push(`${id}: \`texture\` must be a number 0..2 — how much of the track's ` + errors.push(`${id}: \`texture\` must be a number 0..2 — how much of the track's ` +

31
flow-state/src/scenes/layers3d/particles.js
View File

@ -12,9 +12,16 @@ export const particleField = {
family: 'flow', family: 'flow',
kind: 'layer3d', kind: 'layer3d',
// Composited over a background, never used as one: most of the frame is // Composited over a background, never used as one: most of the frame is
// legitimately black, so it is judged on variance rather than luminance and // legitimately black, so it is judged on variance rather than luminance.
// the look generator only picks it as an accent layer. //
role: 'accent', // This used to say `role: 'accent'`, which was a privileged slot rather than
// a description. It made this scene the ONLY thing that could ever sit on
// top — every layered stack in every song was these particles — while the
// general overlay path sat dead because nothing else was labelled. It is one
// composable scene among many now.
// Points in empty space with no ground of their own — the one scene in the
// library that genuinely cannot carry a section alone. See schema.canBackground.
background: false,
// Personality: see look/Personality.js. A point cloud cannot draw the // Personality: see look/Personality.js. A point cloud cannot draw the
// signature form and has no horizon, so it claims only the camera — which // signature form and has no horizon, so it claims only the camera — which
// it can honour exactly, being the one scene with a real one. // it can honour exactly, being the one scene with a real one.
@ -80,7 +87,7 @@ export const particleField = {
return { points, geometry, material, positions, colors, phases, max }; return { points, geometry, material, positions, colors, phases, max };
}, },
update({ instance, camera, timeline, features, params, palette, personality }) { update({ instance, camera, timeline, features, params, palette, personality, framing }) {
const { geometry, material, positions, colors, phases, max } = instance; const { geometry, material, positions, colors, phases, max } = instance;
const count = Math.min(max, Math.round(params.count)); const count = Math.min(max, Math.round(params.count));
const t = timeline.time; const t = timeline.time;
@ -133,19 +140,27 @@ export const particleField = {
// the same slow returning pan, sway and roll every shader scene fakes in // the same slow returning pan, sway and roll every shader scene fakes in
// its coordinate space. Bounded and periodic, so a seek still lands on // its coordinate space. Bounded and periodic, so a seek still lands on
// the same frame as sequential playback. // the same frame as sequential playback.
// The shot's framing, applied to the only literal camera in the
// library. Every fragment scene gets this as a coordinate scale in the
// shader epilogue; here it is what it actually is — the camera standing
// closer or further back. Dividing the distance by the scale matches
// the fragment behaviour, where the coordinate is divided by it.
const frame = framing || { scale: 1, shift: [0, 0] };
const dolly = 4 / Math.max(frame.scale, 0.05);
const cam = personality ? personality.camera : null; const cam = personality ? personality.camera : null;
if (cam) { if (cam) {
const pan = 20 * Math.sin(t * 0.05); const pan = 20 * Math.sin(t * 0.05);
camera.position.set( camera.position.set(
Math.cos(cam.driftAngle) * cam.driftRate * pan Math.cos(cam.driftAngle) * cam.driftRate * pan
+ Math.sin(t * cam.swayRate) * cam.sway, + Math.sin(t * cam.swayRate) * cam.sway + frame.shift[0],
Math.sin(cam.driftAngle) * cam.driftRate * pan Math.sin(cam.driftAngle) * cam.driftRate * pan
+ Math.cos(t * cam.swayRate * 0.83) * cam.sway, + Math.cos(t * cam.swayRate * 0.83) * cam.sway + frame.shift[1],
4, dolly,
); );
camera.rotation.z = cam.spin * t; camera.rotation.z = cam.spin * t;
} else { } else {
camera.position.set(0, 0, 4); camera.position.set(frame.shift[0], frame.shift[1], dolly);
camera.rotation.z = 0; camera.rotation.z = 0;
} }
camera.lookAt(camera.position.x, camera.position.y, -depth * 0.4); camera.lookAt(camera.position.x, camera.position.y, -depth * 0.4);

8190
flow-state/src/scenes/metadata.json Normal file
View File

File diff suppressed because it is too large Load Diff

58
flow-state/src/scenes/registry.js
View File

@ -1,5 +1,13 @@
import { validateModule } from '../params/schema.js'; import { validateModule } from '../params/schema.js';
// STAGES — scenes that draw the song's cast rather than their own content.
// See scenes/stage/README.md and EPIC-3.md.
import { procession } from './stage/procession.js';
import { constellation } from './stage/constellation.js';
import { soloist } from './stage/soloist.js';
import { effigy } from './stage/effigy.js';
import { swarm } from './stage/swarm.js';
import { nebula } from './shader/nebula.js'; import { nebula } from './shader/nebula.js';
import { classicWave } from './shader/classic-wave.js'; import { classicWave } from './shader/classic-wave.js';
import { floatingGeometry } from './shader/floating-geometry.js'; import { floatingGeometry } from './shader/floating-geometry.js';
@ -42,6 +50,27 @@ import { eclipseField } from './shader/eclipse-field.js';
import { girderLattice } from './shader/girder-lattice.js'; import { girderLattice } from './shader/girder-lattice.js';
import { quasicrystal } from './shader/quasicrystal.js'; import { quasicrystal } from './shader/quasicrystal.js';
import { timeSmear } from './shader/time-smear.js'; import { timeSmear } from './shader/time-smear.js';
import { rainColumn } from './shader/rain-column.js';
import { magnetLines } from './shader/magnet-lines.js';
import { karmanStreet } from './shader/karman-street.js';
import { turingBloom } from './shader/turing-bloom.js';
import { myceliumWeb } from './shader/mycelium-web.js';
import { scaleMosaic } from './shader/scale-mosaic.js';
import { pendulumTrace } from './shader/pendulum-trace.js';
import { contourMap } from './shader/contour-map.js';
import { shojiGrid } from './shader/shoji-grid.js';
import { balanceStack } from './shader/balance-stack.js';
import { suspensionSpan } from './shader/suspension-span.js';
import { stairwellDescent } from './shader/stairwell-descent.js';
import { aqueductMarch } from './shader/aqueduct-march.js';
import { dataAisle } from './shader/data-aisle.js';
import { voronoiShatter } from './shader/voronoi-shatter.js';
import { apollonianGasket } from './shader/apollonian-gasket.js';
import { isometricBlocks } from './shader/isometric-blocks.js';
import { halftoneMisprint } from './shader/halftone-misprint.js';
import { drosteFeedback } from './shader/droste-feedback.js';
import { analogWow } from './shader/analog-wow.js';
import { mountainFlight } from './shader/mountain-flight.js';
/** /**
* The scene library. Families exist so the arc driver can choose by section * The scene library. Families exist so the arc driver can choose by section
@ -109,6 +138,35 @@ const MODULES = [
girderLattice, girderLattice,
quasicrystal, quasicrystal,
timeSmear, timeSmear,
rainColumn,
magnetLines,
karmanStreet,
turingBloom,
myceliumWeb,
scaleMosaic,
pendulumTrace,
contourMap,
shojiGrid,
balanceStack,
suspensionSpan,
stairwellDescent,
aqueductMarch,
dataAisle,
voronoiShatter,
apollonianGasket,
isometricBlocks,
halftoneMisprint,
drosteFeedback,
analogWow,
// Stages. Registered alongside the scenes so every existing gate covers
// them; what makes them different is `consumes`, not where they live.
procession,
constellation,
soloist,
swarm,
effigy,
mountainFlight,
]; ];
const errors = []; const errors = [];

98
flow-state/src/scenes/shader/analog-wow.js Normal file
View File

@ -0,0 +1,98 @@
// Glitch family: tape. The transport is not quite steady, so every line of the
// image is displaced by where the tape was when that line was written — a
// continuous horizontal warp, chroma trailing behind luma, and the occasional
// dropout where the oxide has worn through.
//
// Scan Tear and Block Mosh are digital faults: they are quantised, blocky, and
// they happen to whole regions at once. This is analogue and wet — nothing has
// an edge, the error varies smoothly down the frame, and the colour lags the
// picture instead of being replaced by it. Wow and flutter, not corruption.
//
// No declared slow axis: the dropouts and the head smear put two windows of
// identical parameters 0.041 apart, the second-highest noise floor in the
// library, and chroma bleed against that measured 0.02x.
export const analogWow = {
name: 'Analog Wow',
family: 'glitch',
kind: 'fragment',
texture: 0.9,
consumes: ['ink'],
traits: ['camera', 'style'],
params: {
wow: { type: 'float', range: [0, 0.12], default: 0.035, uniform: 'u_wow' },
flutter: { type: 'float', range: [0, 0.03], default: 0.008, uniform: 'u_flutter' },
bleed: { type: 'float', range: [0, 0.05], default: 0.015, uniform: 'u_bleed' },
dropout: { type: 'float', range: [0, 0.6], default: 0.2, uniform: 'u_dropout' },
bars: { type: 'float', range: [1, 9], default: 3.5, uniform: 'u_bars', bias: 'density' },
speed: { type: 'float', range: [0.05, 0.7], default: 0.2, uniform: 'u_speed', bias: 'motion', rate: true },
palette: { type: 'palette', count: 5 },
},
reactive: {
dropout: { feature: 'flux', amount: 0.3, response: 'spike' },
wow: { feature: 'bandLow', amount: 0.25, response: 'smooth' },
},
shader: `
// What was recorded: broad soft bars of colour, the kind of thing that shows
// tape damage clearly because it has no detail of its own to hide it behind.
vec3 programme(vec2 q, float t) {
float band = q.y * u_bars + fbm(q * 1.4 + vec2(t * 0.3, 0.0), 3) * 1.2;
vec3 col = palRamp(0.1 + band * 0.12);
col *= 0.55 + 0.45 * sat(0.5 + 0.5 * sin(band * 3.14159 + t));
col += pal(4) * smoothstep(0.85, 1.0, sat(0.5 + 0.5 * sin(q.x * 3.0 - t * 1.3))) * 0.25;
// The boundary between bands is drawn in the track's weight, so even a
// damaged signal is damaged in the video's own hand.
col += pal(4) * inkStroke(fract(band) - 0.5) * (0.15 + u_sigLine * 0.9);
return col * 0.7;
}
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_speed + u_seed;
p = sigCamera(p);
// Transport error for THIS line: a slow wow, a faster flutter, and a little
// noise on top. It is a function of y, which is what makes the frame skew
// rather than slide — every line was written at a different moment.
float line = p.y * 30.0;
float err = sin(p.y * 2.1 - t * 1.7) * u_wow
+ sin(p.y * 11.0 + t * 5.3) * u_flutter
+ (vnoise(vec2(line, t * 3.0)) - 0.5) * u_flutter * 2.0;
vec2 q = p + vec2(err, 0.0);
// Luma is where it should be; chroma trails it. Sampling the programme
// three times at three displacements is exactly what a chroma delay does.
vec3 col;
col.r = programme(q + vec2(u_bleed, 0.0), t).r;
col.g = programme(q, t).g;
col.b = programme(q - vec2(u_bleed * 0.7, 0.0), t).b;
// Dropouts: bands where the tape has lost contact. They travel slowly down
// the frame, and they replace the picture with the tape's own noise floor
// rather than with black — a black band is a cut, this is a fault.
float bandId = floor(p.y * 8.0 + t * 2.0);
float hit = step(1.0 - u_dropout * 0.35, hash11(bandId * 1.7 + floor(t * 3.0)));
float band = smoothstep(0.5, 0.15, abs(fract(p.y * 8.0 + t * 2.0) - 0.5)) * hit;
vec3 noise = pal(1) * (0.25 + 0.5 * hash12(vec2(uv.x * 400.0, bandId)));
col = mix(col, noise, band * 0.7);
// Head smear: the previous frame, dragged sideways by the same transport
// error, is what gives tape its characteristic horizontal ghosting. The
// programme above stands on its own, so a seek recovers immediately.
vec3 past = prev(uv - vec2(err * 0.4 + 0.004, 0.0));
col = mix(col, max(col, past), 0.7 * 0.55);
// Head-switching noise at the bottom of the frame — the one part of the
// image that is always damaged.
float sw = smoothstep(0.06, 0.0, uv.y) * u_dropout;
col = mix(col, pal(2) * hash12(vec2(uv.x * 300.0, floor(t * 12.0))), sw * 0.6);
return vec4(inkValue(col), 1.0);
}
`,
};
export default analogWow;

103
flow-state/src/scenes/shader/apollonian-gasket.js Normal file
View File

@ -0,0 +1,103 @@
// Geometric family: an Apollonian packing — a circle filled with circles, each
// gap filled again with smaller ones, forever. Built by inversion rather than by
// drawing, so the nesting is exact at every scale the frame can resolve.
//
// Quasicrystal is the other infinitely-detailed geometric and its detail is
// additive: waves summed until the interference looks complicated. This detail
// is structural — every disc has a definite size and place, and zooming finds
// more of them rather than finer noise. Nothing here is a texture, which is why
// it holds up when the shot pushes in.
export const apollonianGasket = {
name: 'Apollonian Gasket',
family: 'geometric',
kind: 'fragment',
// Drawn geometry; grain only muddies the small discs.
texture: 0.3,
consumes: ['ink'],
traits: ['camera', 'style'],
params: {
depth: { type: 'int', range: [3, 12], default: 7, uniform: 'u_depth', bias: 'density' },
pack: { type: 'float', range: [0.9, 1.6], default: 1.2, uniform: 'u_pack' },
offset: { type: 'float', range: [0, 0.35], default: 0.2, uniform: 'u_offset' },
zoom: { type: 'float', range: [0.4, 2.2], default: 1.0, uniform: 'u_zoom', slowAxis: true },
rim: { type: 'float', range: [0, 1.5], default: 0.7, uniform: 'u_rim', bias: 'energy' },
churn: { type: 'float', range: [0.004, 0.05], default: 0.015, uniform: 'u_churn', bias: 'motion', rate: true },
palette: { type: 'palette', count: 5 },
},
reactive: {
rim: { feature: 'bandHigh', amount: 0.35, response: 'smooth' },
offset: { feature: 'bandLow', amount: 0.12, response: 'smooth' },
},
shader: `
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_churn + u_seed;
p = sigCamera(p);
p /= max(u_zoom, 0.05);
// The Kleinian fold: reflect into a fundamental domain, then invert through
// the unit circle, over and over. Each pass divides the scale by the amount
// the inversion contracted, and the accumulated scale is what turns the
// whole recursion back into a single distance at the end.
float scale = 1.0;
vec2 q = p;
float touched = 0.0;
// lint: fixed-cost — u_depth inversions, bounded at fourteen
for (int i = 0; i < 14; i++) {
if (i >= u_depth) break;
// Stop when the next generation would be finer than the frame can
// resolve. Without this the deep discs alias into a speckle that reads
// as noise, which is the opposite of what an exact packing is for.
if (scale > 45.0) break;
// Fold into the strip: this is what makes the packing infinite in
// every direction rather than one circle's worth.
q = mod(q + 1.0, 2.0) - 1.0;
q -= vec2(u_offset * sin(t + float(i) * 0.7), u_offset * cos(t * 0.8)) * 0.25;
float r2 = dot(q, q);
float k = u_pack / max(r2, 1e-4);
q *= k;
scale *= k;
touched += 1.0;
}
// Back to a screen-space distance. Without dividing by the accumulated
// scale the small discs would be drawn with the same line weight as the
// large ones and the image would be a solid mat of edges.
float d = (length(q) - 1.0) / scale;
float w = 0.006 + u_sigLine * 0.03;
float edge = smoothstep(w * 2.0, w * 0.3, abs(d));
float glow = exp(-abs(d) * 26.0);
// The deepest generations are held back rather than cut off. A hard depth
// limit makes discs pop in and out as the fold slides across it, and a
// frame-wide population of discs appearing at once measured as a strobe.
float lod = smoothstep(45.0, 18.0, scale);
edge *= mix(0.25, 1.0, lod);
glow *= mix(0.15, 1.0, lod);
// Colour by how deep the point fell before it settled, so the generations
// of the packing are legible as bands rather than all being one hue.
float gen = fract(log2(max(scale, 1e-6)) * 0.15 + 0.5);
vec3 col = pal(0) * 0.06;
col += palRamp(gen) * edge * (0.55 + u_rim * 0.7);
col += palRamp(gen + 0.25) * glow * u_rim * 0.4;
col += pal(4) * inkStroke(d) * 0.5;
// Fill the interiors faintly so the discs are objects and not just outlines.
col += palRamp(gen + 0.5) * smoothstep(0.0, -0.06, d) * 0.12;
col *= 0.7 + 0.3 * exp(-dot(p, p) * 0.2);
return vec4(inkValue(col), 1.0);
}
`,
};
export default apollonianGasket;

126
flow-state/src/scenes/shader/aqueduct-march.js Normal file
View File

@ -0,0 +1,126 @@
// Structural family: an aqueduct marching away across the frame — a lower tier
// of wide piers, an upper tier of small fast ones, and daylight through every
// opening.
//
// Pylon Grid is open frame: you see through the whole structure and it has no
// mass. This is masonry. The wall is solid, the light comes through holes cut in
// it, and the rhythm changes tier to tier — two arches up top for every one
// below, which is the thing that makes a real aqueduct read as marching rather
// than repeating. The openings are the track's signature form, so a hexagonal
// video is an aqueduct of hexagons.
//
// No declared slow axis, for the reason Contour Map and Balance Stack give: the
// candidates were measured — tier height 0.91x, opening size 0.62x, tier count
// 0.17x — and none of them beats the camera's own drift in Phase 11's
// ten-second average. The structure is legible because it is high-contrast and
// mostly still, and that is exactly the kind of image where a slow parameter
// walk moves less of the frame than a slow pan does.
export const aqueductMarch = {
name: 'Aqueduct March',
family: 'structural',
kind: 'fragment',
texture: 0.9,
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'space', 'style'],
params: {
bays: { type: 'float', range: [1.2, 6], default: 2.2, uniform: 'u_bays', bias: 'density' },
opening: { type: 'float', range: [0.18, 0.6], default: 0.42, uniform: 'u_opening' },
tiers: { type: 'int', range: [1, 4], default: 2, uniform: 'u_tiers' },
rise: { type: 'float', range: [0.3, 1.3], default: 0.7, uniform: 'u_rise' },
recede: { type: 'float', range: [0, 0.8], default: 0.35, uniform: 'u_recede' },
sun: { type: 'float', range: [0, 1.5], default: 0.7, uniform: 'u_sunlight', bias: 'energy' },
march: { type: 'float', range: [0.005, 0.2], default: 0.04, uniform: 'u_march', bias: 'motion', rate: true },
palette: { type: 'palette', count: 5 },
},
reactive: {
sun: { feature: 'loudness', amount: 0.3, response: 'smooth' },
recede: { feature: 'bandLow', amount: 0.15, response: 'smooth' },
},
shader: `
// One tier of masonry: a wall with a row of openings cut through it. Returns
// how much of this pixel is stone.
float tier(vec2 q, float bays, float top, float bottom, float shift) {
if (q.y > top || q.y < bottom) return 0.0;
float h = top - bottom;
float g = q.x * bays + shift;
float cx = (fract(g) - 0.5) / bays;
// The opening is sized as a FRACTION OF ITS BAY in both directions, not as
// a circle of some radius: a bay is much wider than it is tall once there
// are three of them across the frame, and a round hole in it is a porthole
// rather than an arch. Sized this way the same number reads as the same
// opening whatever the bay count is.
float cw = 0.5 / bays; // half a bay across
float ch = h * 0.4; // half an opening up
float k = u_opening * 2.4;
vec2 local = vec2(cx / max(cw * k, 1e-4),
(q.y - (bottom + h * 0.46)) / max(ch * k, 1e-4));
// The opening sits low in the tier, leaving a solid band above it to carry
// the next tier — which is the whole structural point of an aqueduct.
float hole = castMain(local) * min(cw, ch) * k;
return smoothstep(-0.004, 0.004, hole);
}
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_march + u_seed;
p = sigCamera(p);
float horizon = sigHorizonY() * 0.35 - 0.1;
// Sky behind the structure, and the ground it stands on.
vec3 col = mix(pal(1) * 0.22, pal(0) * 0.05, sat((p.y - horizon) * 0.7 + 0.2));
col += pal(3) * exp(-abs(p.y - horizon) * 5.0) * u_sunlight * 0.5;
col = mix(col, pal(0) * 0.1, smoothstep(horizon + 0.01, horizon - 0.05, p.y));
// Two ranges of the same structure: the far one smaller, slower and hazier,
// so the aqueduct crosses a valley rather than standing in a diagram.
// lint: fixed-cost — a near range and a far one
for (int k = 0; k < 2; k++) {
float fk = float(k);
float scale = mix(1.0, 0.35, fk * u_recede * 2.0);
float lift = horizon + fk * u_recede * 0.25;
vec2 q = vec2(p.x / max(scale, 0.15), (p.y - lift) / max(scale, 0.15));
float shift = t * mix(1.0, 0.35, fk);
float stone = 0.0;
float base = 0.0;
// lint: fixed-cost — at most four tiers
for (int i = 0; i < 4; i++) {
if (i >= u_tiers) break;
float fi = float(i);
float bottom = fi * u_rise;
float top = bottom + u_rise;
// Each tier up is twice as fine and half as tall a rhythm.
stone = max(stone, tier(q, u_bays * pow(2.0, fi), top, bottom,
shift * pow(2.0, fi) + fi * 0.31));
base = max(base, step(bottom, q.y) * step(q.y, top));
}
// Stone: lit from the sun side, shadowed on the other, courses picked
// out by a horizontal banding that survives the perspective.
// Courses are drawn in the track's hand: heavy for a track with a thick
// line, barely there for a fine one.
float course = 0.5 + 0.5 * sin(q.y * 60.0);
course = mix(course, smoothstep(0.35, 0.75, course), u_sigLine);
vec3 masonry = mix(pal(2) * 0.35, pal(4) * 0.8, sat(q.x * 0.4 + 0.5));
masonry *= 0.7 + (0.15 + u_sigLine * 0.5) * course;
masonry += pal(3) * u_sunlight * 0.22 * sat(q.x * 0.5 + 0.5);
masonry += pal(4) * inkStroke(0.5 - abs(fract(q.x * u_bays) - 0.5) - 0.03) * (0.1 + u_sigLine * 0.9);
float hazed = mix(1.0, 0.55, fk * u_recede);
col = mix(col, masonry * hazed, stone * base);
}
col = sigAir(col, p, smoothstep(-0.2, 0.9, p.y - horizon));
return vec4(inkValue(col), 1.0);
}
`,
};
export default aqueductMarch;

9
flow-state/src/scenes/shader/aurora-veil.js
View File

@ -13,6 +13,7 @@ export const auroraVeil = {
name: 'Aurora Veil', name: 'Aurora Veil',
family: 'flow', family: 'flow',
kind: 'fragment', kind: 'fragment',
consumes: ['ink'],
traits: ['camera', 'space', 'style'], traits: ['camera', 'space', 'style'],
params: { params: {
@ -20,13 +21,11 @@ export const auroraVeil = {
height: { type: 'float', range: [0.4, 1.8], default: 1.0, uniform: 'u_height' }, height: { type: 'float', range: [0.4, 1.8], default: 1.0, uniform: 'u_height' },
fold: { type: 'float', range: [0.1, 1.4], default: 0.55, uniform: 'u_fold', bias: 'density' }, fold: { type: 'float', range: [0.1, 1.4], default: 0.55, uniform: 'u_fold', bias: 'density' },
speed: { type: 'float', range: [0.03, 0.5], default: 0.15, uniform: 'u_speed', bias: 'motion', rate: true }, speed: { type: 'float', range: [0.03, 0.5], default: 0.15, uniform: 'u_speed', bias: 'motion', rate: true },
glow: { type: 'float', range: [0, 1.4], default: 0.5, uniform: 'u_glow', bias: 'energy' },
ground: { type: 'float', range: [0, 0.8], default: 0.3, uniform: 'u_ground' }, ground: { type: 'float', range: [0, 0.8], default: 0.3, uniform: 'u_ground' },
palette: { type: 'palette', count: 5 }, palette: { type: 'palette', count: 5 },
}, },
reactive: { reactive: {
glow: { feature: 'bandHigh', amount: 0.4, response: 'smooth' },
fold: { feature: 'bandLow', amount: 0.3, response: 'smooth' }, fold: { feature: 'bandLow', amount: 0.3, response: 'smooth' },
}, },
@ -61,8 +60,7 @@ vec4 scene(vec2 uv, vec2 p) {
float sheet = exp(-d * d / max(widthAt * widthAt, 1e-5)) * rise; float sheet = exp(-d * d / max(widthAt * widthAt, 1e-5)) * rise;
vec3 tint = palRamp(fract(s) * 0.5 + above * 0.12 + 0.1); vec3 tint = palRamp(fract(s) * 0.5 + above * 0.12 + 0.1);
col += tint * sheet * (0.55 + u_glow * 0.6); col += tint * sheet * (0.55 + 0.5 * 0.6);
col += tint * exp(-d * 5.0) * rise * u_glow * 0.12;
} }
// Ground: the curtains reflected, dim and compressed. // Ground: the curtains reflected, dim and compressed.
@ -72,8 +70,7 @@ vec4 scene(vec2 uv, vec2 p) {
} }
col = sigAir(col, p, smoothstep(0.0, 1.6, abs(p.x))); col = sigAir(col, p, smoothstep(0.0, 1.6, abs(p.x)));
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

124
flow-state/src/scenes/shader/balance-stack.js Normal file
View File

@ -0,0 +1,124 @@
// Minimal family: four or five of the track's signature form stacked one on
// another on an empty plain, leaning further than they should, with a low sun
// throwing the whole stack across the ground.
//
// Salt Flat is the other minimal with a horizon and an object, and its object is
// one, tiny and miles away: the subject there is the emptiness. Here the subject
// is close, it is the only thing in the frame, and it is a structure with a
// problem — it leans, and the low sun throws it across the whole plain. Nearly
// still, and the stillness is tense rather than calm.
//
// No declared slow axis. The sun crossing is the one that should qualify — it
// regrades the entire sky — and it measures 0.93x against the noise floor,
// because in a frame this empty the camera's drift moves the average about as
// much as anything in the scene can. Left undeclared rather than overclaimed.
export const balanceStack = {
name: 'Balance Stack',
family: 'minimal',
kind: 'fragment',
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'space', 'style'],
params: {
count: { type: 'int', range: [2, 6], default: 4, uniform: 'u_count', bias: 'density' },
lean: { type: 'float', range: [-0.5, 0.5], default: 0.12, uniform: 'u_lean' },
block: { type: 'float', range: [0.08, 0.4], default: 0.2, uniform: 'u_block' },
taper: { type: 'float', range: [0.5, 1.0], default: 0.82, uniform: 'u_taper' },
sun: { type: 'float', range: [-1.0, 1.0], default: -0.6, uniform: 'u_sun' },
shadow: { type: 'float', range: [0, 1], default: 0.6, uniform: 'u_shadow' },
rim: { type: 'float', range: [0, 1.5], default: 0.7, uniform: 'u_rim', bias: 'energy' },
sway: { type: 'float', range: [0.01, 0.2], default: 0.04, uniform: 'u_sway', bias: 'motion', rate: true },
palette: { type: 'palette', count: 5 },
},
reactive: {
rim: { feature: 'loudness', amount: 0.3, response: 'smooth' },
shadow: { feature: 'bandLow', amount: 0.2, response: 'smooth' },
},
shader: `
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_sway + u_seed;
p = sigCamera(p);
float ground = sigHorizonY() * 0.4 - 0.35;
float above = p.y - ground;
// Sky and plain. Two flat tones and a line: everything else in the frame is
// the stack.
// The sun's position lights the whole frame, not just the stack: the glow
// sits where it is on the horizon and the plain takes its colour from it.
// That is what makes the sun crossing the sky the scene's long journey.
float sunX = u_sun * 1.2;
float toSun = abs(p.x - sunX);
// The sky is graded ACROSS the frame toward the sun as well as up it, so
// where the sun stands recolours everything rather than adding a spot.
vec3 sky = mix(pal(1) * 0.22, pal(0) * 0.07, sat(above * 0.5 + 0.1));
vec3 warm = mix(pal(3) * 0.3, pal(2) * 0.22, sat(above * 0.6 + 0.2));
vec3 col = mix(sky, warm, sat(1.0 - toSun * 0.55));
col += pal(3) * exp(-abs(above) * 8.0) * exp(-toSun * 0.8) * 0.9;
col += pal(3) * exp(-abs(above) * 2.0) * exp(-toSun * 1.6) * 0.35;
if (above < 0.0) {
col = mix(pal(2) * 0.16, pal(0) * 0.09, sat(-above * 1.6));
col += pal(3) * exp(above * 4.0) * exp(-toSun * 1.2) * 0.5;
}
// The sun is low and to one side; everything about the light comes from it.
vec2 lightDir = normalize(vec2(u_sun, 0.55));
// The shadow first, so the stack sits on top of its own.
float shade = 0.0;
float y = ground;
float x = 0.0;
// lint: fixed-cost — at most six blocks in the stack
for (int i = 0; i < 6; i++) {
if (i >= u_count) break;
float fi = float(i);
float size = u_block * pow(u_taper, fi);
// Stacked by accumulation, so each block actually rests on the one
// below whatever the taper is doing.
y += size;
x += u_lean * size * 1.6 + sin(t * 1.3 + fi * 0.7) * 0.012 * fi;
float h = y;
// Cast along the ground, stretched by how high the block sits.
vec2 at = vec2(x - lightDir.x * (h - ground) * 2.2, ground - 0.02);
vec2 q = (p - at) / vec2(size * 2.4, size * 0.32);
shade = max(shade, exp(-dot(q, q) * 1.6));
y += size;
}
col = mix(col, pal(0) * 0.05, shade * u_shadow * smoothstep(0.02, -0.02, above));
// The stack. Drawn back to front, which for a vertical pile is bottom up.
y = ground;
x = 0.0;
// lint: fixed-cost — the same six blocks
for (int i = 0; i < 6; i++) {
if (i >= u_count) break;
float fi = float(i);
float size = u_block * pow(u_taper, fi);
y += size;
x += u_lean * size * 1.6 + sin(t * 1.3 + fi * 0.7) * 0.012 * fi;
vec2 at = vec2(x, y);
float d = castMain((p - at) / size) * size;
// Body: shaded across the form, so it has a lit side and a dark side.
float lit = sat(dot(normalize(p - at + 1e-4), lightDir) * 0.5 + 0.5);
vec3 body = mix(pal(0) * 0.08, pal(2) * 0.4, lit) * (0.65 + 0.35 * (1.0 - fi * 0.12));
col = mix(col, body, smoothstep(0.004, -0.004, d));
// Rim: the sun catching the edge, in the track's line weight.
col += pal(4) * inkStroke(d) * (0.3 + u_rim * 0.9) * (0.4 + lit * 0.8);
y += size;
}
col = sigAir(col, p, smoothstep(0.1, 1.5, abs(p.x) * 0.7 + sat(above * 0.5)));
return vec4(inkValue(col), 1.0);
}
`,
};
export default balanceStack;

8
flow-state/src/scenes/shader/block-mosh.js
View File

@ -11,11 +11,11 @@ export const blockMosh = {
family: 'glitch', family: 'glitch',
kind: 'fragment', kind: 'fragment',
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
consumes: ['ink'],
traits: ['camera', 'style'], traits: ['camera', 'style'],
params: { params: {
block: { type: 'float', range: [4, 48], default: 22, uniform: 'u_blocks', bias: 'density' }, block: { type: 'float', range: [4, 48], default: 22, uniform: 'u_blocks', bias: 'density' },
smear: { type: 'float', range: [0, 0.5], default: 0.18, uniform: 'u_smear', bias: 'energy' },
quantize: { type: 'float', range: [1, 8], default: 4, uniform: 'u_quantize' }, quantize: { type: 'float', range: [1, 8], default: 4, uniform: 'u_quantize' },
bleed: { type: 'float', range: [0, 1], default: 0.45, uniform: 'u_bleed' }, bleed: { type: 'float', range: [0, 1], default: 0.45, uniform: 'u_bleed' },
speed: { type: 'float', range: [0.1, 1.4], default: 0.5, uniform: 'u_speed', bias: 'motion', rate: true }, speed: { type: 'float', range: [0.1, 1.4], default: 0.5, uniform: 'u_speed', bias: 'motion', rate: true },
@ -24,7 +24,6 @@ export const blockMosh = {
}, },
reactive: { reactive: {
smear: { feature: 'bandHigh', amount: 0.3, response: 'smooth' },
bleed: { feature: 'flux', amount: 0.2, response: 'spike' }, bleed: { feature: 'flux', amount: 0.2, response: 'spike' },
burst: { feature: 'beat', amount: 0.25, response: 'smooth' }, burst: { feature: 'beat', amount: 0.25, response: 'smooth' },
}, },
@ -59,7 +58,7 @@ vec4 scene(vec2 uv, vec2 p) {
// Each block gets one fixed stroke per step: a direction and a reach. // Each block gets one fixed stroke per step: a direction and a reach.
vec2 stroke = (hash22(cell + st * 31.0) - 0.5) * 2.0; vec2 stroke = (hash22(cell + st * 31.0) - 0.5) * 2.0;
float reach = hash12(cell + st * 17.0); float reach = hash12(cell + st * 17.0);
vec2 dst = clamp(uv + stroke * u_smear * (0.3 + reach), 0.0, 1.0); vec2 dst = clamp(uv + stroke * 0.18 * (0.3 + reach), 0.0, 1.0);
// The mosh pulls the PREVIOUS frame along the stroke and layers it over the // The mosh pulls the PREVIOUS frame along the stroke and layers it over the
// fresh field. The previous frame is itself moshed, so smear accrues. // fresh field. The previous frame is itself moshed, so smear accrues.
@ -73,8 +72,7 @@ vec4 scene(vec2 uv, vec2 p) {
float blk = hash12(cell + st * 7.0); float blk = hash12(cell + st * 7.0);
col += pal(2) * u_burst * (0.5 + 0.5 * blk) * (0.5 + 0.5 * beat); col += pal(2) * u_burst * (0.5 + 0.5 * blk) * (0.5 + 0.5 * beat);
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

8
flow-state/src/scenes/shader/cargo-belt.js
View File

@ -15,6 +15,7 @@ export const cargoBelt = {
kind: 'fragment', kind: 'fragment',
// Takes the track's surface grain, but lightly — this is drawn, not filmed. // Takes the track's surface grain, but lightly — this is drawn, not filmed.
texture: 0.4, texture: 0.4,
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'style'], traits: ['shape', 'camera', 'style'],
params: { params: {
@ -65,11 +66,11 @@ vec4 scene(vec2 uv, vec2 p) {
if (rnd > u_gap) { if (rnd > u_gap) {
vec2 local = vec2((withinCell - 0.5) * 2.0, dy / max(span * 0.5, 1e-3)); vec2 local = vec2((withinCell - 0.5) * 2.0, dy / max(span * 0.5, 1e-3));
float size = u_crateSize * (0.7 + rnd * 0.5); float size = u_crateSize * (0.7 + rnd * 0.5);
float d = sigShape(local / max(size, 1e-3)) * size; float d = castMain(local / max(size, 1e-3)) * size;
vec3 crateColor = pal(int(mod(cell + fi, 4.0)) + 1); vec3 crateColor = pal(int(mod(cell + fi, 4.0)) + 1);
col = mix(col, crateColor * (0.35 + rnd * 0.5), smoothstep(0.02, -0.02, d)); col = mix(col, crateColor * (0.35 + rnd * 0.5), smoothstep(0.02, -0.02, d));
col += crateColor * sigEdge(d) * (0.4 + u_lamp * 0.5); col += crateColor * inkStroke(d) * (0.4 + u_lamp * 0.5);
// Lamp on a minority of crates, pulsing on the beat. Local, not // Lamp on a minority of crates, pulsing on the beat. Local, not
// whole-frame: a per-crate blink is not a flash. // whole-frame: a per-crate blink is not a flash.
@ -81,8 +82,7 @@ vec4 scene(vec2 uv, vec2 p) {
col += pal(2) * rail * u_rails * 0.5; col += pal(2) * rail * u_rails * 0.5;
} }
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

11
flow-state/src/scenes/shader/cell-divide.js
View File

@ -15,6 +15,7 @@ export const cellDivide = {
name: 'Cell Divide', name: 'Cell Divide',
family: 'organic', family: 'organic',
kind: 'fragment', kind: 'fragment',
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'style'], traits: ['shape', 'camera', 'style'],
params: { params: {
@ -23,14 +24,12 @@ export const cellDivide = {
speed: { type: 'float', range: [0.05, 1.0], default: 0.22, uniform: 'u_speed', bias: 'motion', rate: true }, speed: { type: 'float', range: [0.05, 1.0], default: 0.22, uniform: 'u_speed', bias: 'motion', rate: true },
wall: { type: 'float', range: [0.01, 0.14],default: 0.045,uniform: 'u_wall' }, wall: { type: 'float', range: [0.01, 0.14],default: 0.045,uniform: 'u_wall' },
nucleus: { type: 'float', range: [0.0, 0.5], default: 0.22, uniform: 'u_nucleus' }, nucleus: { type: 'float', range: [0.0, 0.5], default: 0.22, uniform: 'u_nucleus' },
glow: { type: 'float', range: [0, 1.5], default: 0.5, uniform: 'u_glow', bias: 'energy' },
split: { type: 'float', range: [0, 1], default: 0.4, uniform: 'u_split' }, split: { type: 'float', range: [0, 1], default: 0.4, uniform: 'u_split' },
palette: { type: 'palette', count: 5 }, palette: { type: 'palette', count: 5 },
}, },
reactive: { reactive: {
nucleus: { feature: 'beat', amount: 0.2, response: 'spike' }, nucleus: { feature: 'beat', amount: 0.2, response: 'spike' },
glow: { feature: 'bandMid', amount: 0.3, response: 'smooth' },
}, },
shader: ` shader: `
@ -90,18 +89,16 @@ vec4 scene(vec2 uv, vec2 p) {
// The wall itself. // The wall itself.
float wall = smoothstep(u_wall * (0.5 + u_sigLine), 0.0, membrane); float wall = smoothstep(u_wall * (0.5 + u_sigLine), 0.0, membrane);
col = mix(col, pal(4), wall * 0.8); col = mix(col, pal(4), wall * 0.8);
col += pal(3) * exp(-membrane * 26.0) * u_glow * 0.3;
// Nucleus, in the track's signature form. // Nucleus, in the track's signature form.
if (u_nucleus > 0.01) { if (u_nucleus > 0.01) {
float size = u_nucleus * 0.35; float size = u_nucleus * 0.35;
float d = sigShape((p - nearestAt) / max(size, 1e-3)) * size; float d = castMain((p - nearestAt) / max(size, 1e-3)) * size;
col = mix(col, pal(1), smoothstep(0.008, -0.008, d) * 0.85); col = mix(col, pal(1), smoothstep(0.008, -0.008, d) * 0.85);
col += pal(2) * sigEdge(d) * (0.35 + u_glow * 0.4); col += pal(2) * inkStroke(d) * (0.35 + 0.5 * 0.4);
} }
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

33
flow-state/src/scenes/shader/circuit-bloom.js
View File

@ -15,6 +15,7 @@ export const circuitBloom = {
kind: 'fragment', kind: 'fragment',
// Takes the track's surface grain, but lightly — this is drawn, not filmed. // Takes the track's surface grain, but lightly — this is drawn, not filmed.
texture: 0.4, texture: 0.4,
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'style'], traits: ['shape', 'camera', 'style'],
params: { params: {
@ -25,6 +26,16 @@ export const circuitBloom = {
pulse: { type: 'float', range: [0, 1.5], default: 0.6, uniform: 'u_pulse', bias: 'energy' }, pulse: { type: 'float', range: [0, 1.5], default: 0.6, uniform: 'u_pulse', bias: 'energy' },
speed: { type: 'float', range: [0.05, 1.2], default: 0.35, uniform: 'u_speed', bias: 'motion', rate: true }, speed: { type: 'float', range: [0.05, 1.2], default: 0.35, uniform: 'u_speed', bias: 'motion', rate: true },
fill: { type: 'float', range: [0.2, 0.95], default: 0.6, uniform: 'u_fill', bias: 'density' }, fill: { type: 'float', range: [0.2, 0.95], default: 0.6, uniform: 'u_fill', bias: 'density' },
// How far the board has been routed out from its centre. This is the
// slow axis: the board BLOOMS across the track, which is what the scene
// is named for and what it never actually did.
// Range stops at 1.6 on purpose: the frame's far corner is about 1.4
// units out, so anything past that is a board that already covers
// everything, and axis travel spent up there changes nothing.
// Floor raised from 0.25: below about half, the growth front has not
// reached far enough for any pad to be drawn and the frame is empty.
grown: { type: 'float', range: [0.55, 1.6], default: 1.0, uniform: 'u_grown', slowAxis: true },
edge: { type: 'float', range: [0.2, 1], default: 0.5, uniform: 'u_edge' },
palette: { type: 'palette', count: 5 }, palette: { type: 'palette', count: 5 },
}, },
@ -56,8 +67,15 @@ vec4 scene(vec2 uv, vec2 p) {
float dH = abs(f.y); float dH = abs(f.y);
float dV = abs(f.x); float dV = abs(f.x);
// Distance from the centre of the board, used to gate growth outward. // How far the board has been routed. This used to be a fixed vignette
float reach = sat(1.4 - length(p) * 0.5); // (a plain falloff on length(p)), which meant the layout was identical at
// every moment of the song — the packets ran, the pads blinked, and the
// BOARD never changed. Growing it is the scene's one piece of large-scale
// structure, and the thing its name was always promising.
//
// Floored rather than closed: an unrouted region keeps a trace of copper so
// a small board is sparse rather than an empty frame.
float reach = mix(0.04, 1.0, sat((u_grown - length(p)) * (0.6 + u_edge * 3.0)));
float traceMask = 0.0; float traceMask = 0.0;
if (horizontal > 0.5) traceMask += smoothstep(width, width * 0.35, dH); if (horizontal > 0.5) traceMask += smoothstep(width, width * 0.35, dH);
@ -76,13 +94,16 @@ vec4 scene(vec2 uv, vec2 p) {
// Pads sit where both traces meet, stamped in the signature form. // Pads sit where both traces meet, stamped in the signature form.
if (horizontal > 0.5 && vertical > 0.5 && rnd2 > 1.0 - u_pads) { if (horizontal > 0.5 && vertical > 0.5 && rnd2 > 1.0 - u_pads) {
float d = sigShape(f / max(u_padSize, 1e-3)) * u_padSize; float d = castMain(f / max(u_padSize, 1e-3)) * u_padSize;
col += pal(2) * smoothstep(0.01, -0.01, d) * reach * 0.7; col += pal(2) * smoothstep(0.01, -0.01, d) * reach * 0.7;
col += pal(3) * sigEdge(d) * reach * (0.4 + u_pulse * 0.4); // Filled as well as stroked. At the bottom of the 'grown' range the pads
// were a hairline and nothing else, and the ink's stroke can be thinner
// than the edge it replaced — which left the frame empty at grown=0.25.
col = mix(col, pal(3), inkMask(d, uv) * reach * 0.6);
col += pal(3) * inkStroke(d) * reach * (0.4 + u_pulse * 0.4);
} }
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

24
flow-state/src/scenes/shader/classic-wave.js
View File

@ -7,9 +7,9 @@ export const classicWave = {
family: 'flow', family: 'flow',
kind: 'fragment', kind: 'fragment',
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
// Grain is this scene's ONLY expression of the style trait — it draws no // Smooth concentric colour: grain only mutes the ramp it is built on.
// hard edges to weight — so it keeps a share of it rather than opting out. texture: 0,
texture: 0.35, consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'style'], traits: ['shape', 'camera', 'style'],
params: { params: {
@ -39,7 +39,7 @@ vec4 scene(vec2 uv, vec2 p) {
p = sigCamera(p); p = sigCamera(p);
// The rings take the track's signature form: round tracks get circles, // The rings take the track's signature form: round tracks get circles,
// hexagonal tracks get hexagonal rings, and it costs one call. // hexagonal tracks get hexagonal rings, and it costs one call.
float d = sigShape(p) + 1.0; float d = castMain(p) + 1.0;
float angle = atan(p.y, p.x); float angle = atan(p.y, p.x);
float t = u_time * u_speed + u_seed; float t = u_time * u_speed + u_seed;
@ -47,18 +47,26 @@ vec4 scene(vec2 uv, vec2 p) {
float spoke = u_spokes > 0 ? sin(angle * float(u_spokes) + t) * u_beat : 0.0; float spoke = u_spokes > 0 ? sin(angle * float(u_spokes) + t) * u_beat : 0.0;
float v = 0.5 + 0.5 * sin(wave + spoke); float v = 0.5 + 0.5 * sin(wave + spoke);
v = mix(v, smoothstep(0.2, 0.8, v), u_softness);
vec3 col = palRamp(t * u_colorRoll + d * 0.25) * v; // TRAIT style: the track's hand on the crests. The scene's own u_softness
// decides HOW MUCH the wave is contrasted; the track decides what that
// contrast looks like — u_sigSoft widens the transition, u_sigLine tightens
// it. Centred on 0.5 so changing the hand does not change the exposure.
float edge = mix(0.06, 0.42, sat(u_sigSoft)) * mix(1.3, 0.6, sat(u_sigLine));
v = mix(v, smoothstep(0.5 - edge, 0.5 + edge, v), u_softness);
// ...and independently of u_softness, which a track is free to sample at
// zero: a heavier line concentrates the crest rather than letting it bloom
// across the whole ring.
vec3 col = palRamp(t * u_colorRoll + d * 0.25) * pow(v, mix(1.0, 2.2, sat(u_sigLine)));
// Core glow, the part that reads as the "hit". // Core glow, the part that reads as the "hit".
col += pal(0) * (1.0 - smoothstep(0.0, 0.7, d)) * u_bloomCore * 0.6; col += pal(0) * (1.0 - smoothstep(0.0, 0.7, d)) * u_bloomCore * 0.6;
// Keep the corners from clipping to flat colour. // Keep the corners from clipping to flat colour.
col *= 0.6 + 0.4 * (1.0 - smoothstep(0.8, 1.8, d)); col *= 0.6 + 0.4 * (1.0 - smoothstep(0.8, 1.8, d));
col += sigGrain(uv);
return vec4(col, 1.0); return vec4(inkValue(col), 1.0);
} }
`, `,
}; };

96
flow-state/src/scenes/shader/contour-map.js Normal file
View File

@ -0,0 +1,96 @@
// Minimal family: a survey drawing of a landscape — nested contour lines on a
// ground plane running to the horizon, every fifth one heavier, and water
// standing in the low ground.
//
// Horizon Lines is a bundle of parallel rules and Ridge Terrain is stacked
// silhouettes: both describe a landscape by what it hides. Contours describe it
// by what it measures, so the lines are closed, nested and never cross, and the
// shape of the land is legible from the spacing alone. The water level is the
// slow tide of the piece — the same terrain drowning and draining.
export const contourMap = {
name: 'Contour Map',
family: 'minimal',
kind: 'fragment',
// Drafting, not photography.
texture: 0.3,
consumes: ['ink'],
traits: ['camera', 'space', 'style'],
params: {
levels: { type: 'float', range: [4, 30], default: 12, uniform: 'u_levels', bias: 'density' },
relief: { type: 'float', range: [0.4, 3.0], default: 1.2, uniform: 'u_relief' },
sea: { type: 'float', range: [0.1, 0.9], default: 0.42, uniform: 'u_sea' },
indexN: { type: 'int', range: [3, 8], default: 5, uniform: 'u_indexN' },
weight: { type: 'float', range: [0.03, 0.3], default: 0.1, uniform: 'u_weight' },
shore: { type: 'float', range: [0, 1.4], default: 0.6, uniform: 'u_shore', bias: 'energy' },
drift: { type: 'float', range: [0.002, 0.03], default: 0.008, uniform: 'u_drift', bias: 'motion', rate: true },
palette: { type: 'palette', count: 5 },
},
reactive: {
shore: { feature: 'loudness', amount: 0.3, response: 'smooth' },
relief: { feature: 'bandLow', amount: 0.2, response: 'smooth' },
},
shader: `
// The land. Slow enough that the survey is being redrawn rather than animated.
float land(vec2 g, float t) {
return fbm(g * u_relief + vec2(t, t * 0.6), 4);
}
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_drift + u_seed;
p = sigCamera(p);
float horizon = sigHorizonY() * 0.4 + 0.35;
float below = horizon - p.y;
// Sky: the paper the survey is drawn on, lit from the horizon.
vec3 col = mix(pal(1) * 0.2, pal(0) * 0.07, sat((p.y - horizon) * 1.4 + 0.15));
col += pal(3) * exp(-abs(p.y - horizon) * 10.0) * 0.15;
if (below > 0.002) {
float depth = 1.0 / max(below, 0.03);
vec2 g = vec2(p.x * depth, depth) * 0.5;
float h = land(g, t);
// Contours. The band is measured in HEIGHT, not in pixels, which is
// what makes the lines crowd on steep ground and spread on flat ground
// without anything having to compute a gradient.
float f = h * u_levels;
float band = abs(fract(f) - 0.5) * 2.0;
float index = abs(fract(f / float(u_indexN)) - 0.5) * 2.0;
float w = u_weight * (0.35 + u_sigLine * 1.8);
float line = smoothstep(w, w * 0.2, band);
float heavy = smoothstep(w * 1.6, w * 0.3, index);
vec3 ground = pal(0) * 0.1;
ground += palRamp(0.3 + h * 0.3) * line * 0.5;
ground += pal(4) * heavy * 0.35;
// Water standing in everything below the level. A flat tone, because
// water has no contours — which is exactly what makes the level legible.
float wet = smoothstep(u_sea + 0.02, u_sea - 0.02, h);
vec3 water = mix(pal(2) * 0.7, pal(3) * 0.45, sat(below));
// Water takes the sky, so a flooded frame is a bright one and the level
// is legible from across the room.
water += pal(3) * exp(-below * 1.6) * u_shore * 0.5;
ground = mix(ground, water, wet);
// Shoreline: the one bright edge in the drawing.
float edge = smoothstep(0.05, 0.0, abs(h - u_sea));
ground += pal(3) * edge * u_shore * 0.8;
col = mix(col, ground, smoothstep(0.0, 0.03, below));
}
col = sigAir(col, p, sat(1.0 - below * 0.9));
return vec4(inkValue(col), 1.0);
}
`,
};
export default contourMap;

37
flow-state/src/scenes/shader/curl-flow.js
View File

@ -9,6 +9,7 @@ export const curlFlow = {
family: 'flow', family: 'flow',
kind: 'fragment', kind: 'fragment',
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
consumes: ['ink'],
traits: ['camera', 'space', 'style'], traits: ['camera', 'space', 'style'],
params: { params: {
@ -17,12 +18,19 @@ export const curlFlow = {
streak: { type: 'float', range: [0, 1], default: 0.55, uniform: 'u_streak' }, streak: { type: 'float', range: [0, 1], default: 0.55, uniform: 'u_streak' },
contrast: { type: 'float', range: [0.5, 4], default: 1.6, uniform: 'u_contrast' }, contrast: { type: 'float', range: [0.5, 4], default: 1.6, uniform: 'u_contrast' },
veins: { type: 'float', range: [1, 12], default: 5.0, uniform: 'u_veins', bias: 'density' }, veins: { type: 'float', range: [1, 12], default: 5.0, uniform: 'u_veins', bias: 'density' },
glow: { type: 'float', range: [0, 1.2], default: 0.35, uniform: 'u_glow', bias: 'energy' }, // The large-scale structure — see the shader. `channel` is the slow
// axis rather than `channelAt`, and the difference was measured: MOVING
// the band preserves the frame's total energy, so a ten-second average
// of it is nearly the same image wherever it sits. Opening and closing
// the band changes how much of the frame is lit at all, which is what
// the eye actually reads as the image changing.
channel: { type: 'float', range: [0, 1], default: 0.6, uniform: 'u_channel', bias: 'density', slowAxis: true },
channelAt: { type: 'float', range: [-1.1, 1.1], default: 0.0, uniform: 'u_channelAt' },
channelAngle: { type: 'float', range: [0, 6.283], default: 1.2, uniform: 'u_channelAngle' },
palette: { type: 'palette', count: 5 }, palette: { type: 'palette', count: 5 },
}, },
reactive: { reactive: {
glow: { feature: 'beat', amount: 0.3, response: 'spike' },
veins: { feature: 'bandMid', amount: 0.25 }, veins: { feature: 'bandMid', amount: 0.25 },
streak: { feature: 'flux', amount: 0.2, response: 'smooth' }, streak: { feature: 'flux', amount: 0.2, response: 'smooth' },
}, },
@ -40,18 +48,37 @@ vec4 scene(vec2 uv, vec2 p) {
float veins = 1.0 - abs(sin(n * u_veins + t * 2.0)); float veins = 1.0 - abs(sin(n * u_veins + t * 2.0));
veins = pow(sat(veins), u_contrast); veins = pow(sat(veins), u_contrast);
// A CURRENT: one broad band of the frame carries the filaments and the rest
// runs comparatively bare.
//
// Without this the scene is one noise field at one scale, which means it is
// statistically identical everywhere and at every moment. Measured, that is
// exactly what made it read as static however fast it moved: the eye finds
// the statistics in about two seconds and then there is nothing left. Every
// pixel was moving and the IMAGE never changed.
//
// The band never closes fully — 0.22 at its darkest — because an empty half
// frame is a different failure from a busy one.
float along = dot(p, vec2(cos(u_channelAngle), sin(u_channelAngle)));
float band = (along - u_channelAt) * (0.9 + u_channel * 2.0);
float current = mix(1.0, 0.10 + 0.90 * exp(-band * band), u_channel);
veins *= current;
vec3 col = mix(pal(0) * 0.12, pal(1), veins); vec3 col = mix(pal(0) * 0.12, pal(1), veins);
col += pal(2) * pow(veins, 3.0) * u_glow;
col = mix(col, pal(3), sat(length(flow) * 0.4) * 0.35); col = mix(col, pal(3), sat(length(flow) * 0.4) * 0.35);
// Feedback trails: the previous frame, pulled slightly along the flow. // Feedback trails: the previous frame, pulled slightly along the flow.
vec3 trail = prev(uv - flow * 0.004); vec3 trail = prev(uv - flow * 0.004);
col = max(col, trail * u_streak); col = max(col, trail * u_streak);
// The current gates the FINAL image rather than only the vein term. The
// feedback trail and the flow tint both fill the frame on their own, so
// structuring the veins alone left the composite as uniform as it was.
col *= mix(1.0, 0.16 + 0.84 * current, u_channel);
col *= 0.6 + 0.4 * exp(-dot(p, p) * 0.35); col *= 0.6 + 0.4 * exp(-dot(p, p) * 0.35);
col = sigAir(col, p, smoothstep(0.0, 1.7, length(p))); col = sigAir(col, p, smoothstep(0.0, 1.7, length(p)));
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

113
flow-state/src/scenes/shader/data-aisle.js Normal file
View File

@ -0,0 +1,113 @@
// Structural family: a cold aisle. Two walls of equipment running to a
// vanishing point, indicator lights on every unit, a lit floor between them.
//
// Neon City is the other lit structure and it is exterior, vast and far: the
// lights are windows in buildings a mile off. This is interior and close enough
// to touch — two flat walls a couple of metres apart, the geometry is a corridor
// rather than a skyline, and the lights are not decoration but the only thing
// telling you the machines are running. What moves is the camera down the aisle.
export const dataAisle = {
name: 'Data Aisle',
family: 'structural',
kind: 'fragment',
texture: 0.6,
consumes: ['ink'],
traits: ['camera', 'space', 'style'],
params: {
width: { type: 'float', range: [0.25, 1.1], default: 0.55, uniform: 'u_aisle' },
racks: { type: 'float', range: [1, 8], default: 3, uniform: 'u_racks', bias: 'density' },
units: { type: 'float', range: [3, 20], default: 9, uniform: 'u_units', bias: 'density' },
leds: { type: 'float', range: [0, 1.5], default: 0.7, uniform: 'u_leds', bias: 'energy' },
activity:{ type: 'float', range: [0, 1], default: 0.45, uniform: 'u_activity' },
run: { type: 'float', range: [0.02, 0.5], default: 0.12, uniform: 'u_run', bias: 'motion', rate: true },
palette: { type: 'palette', count: 5 },
},
reactive: {
leds: { feature: 'bandHigh', amount: 0.35, response: 'smooth' },
activity: { feature: 'flux', amount: 0.25, response: 'spike' },
},
shader: `
// One wall of racks. 'along' is metres down the aisle, 'up' is height on the
// wall, and the return is the wall's colour.
vec3 wall(vec2 at, float side, float t, float lightAmount) {
// Rack bays, and the gap between two racks.
float bay = at.x * u_racks;
float bayId = floor(bay);
float bx = fract(bay) - 0.5;
float unit = at.y * u_units;
float unitId = floor(unit);
float uy = fract(unit) - 0.5;
// The metalwork: a dark case with a seam at every unit and every bay.
float seam = min(0.5 - abs(bx), 0.5 - abs(uy));
vec3 col = mix(pal(0) * 0.3, pal(2) * 0.42, sat(at.y * 0.6 + 0.2));
float seamW = 0.03 + u_sigLine * 0.12;
col = mix(col, pal(0) * 0.12, smoothstep(seamW, seamW * 0.2, seam));
// Indicators: a row per unit, most of them steady, a few of them working.
float id = hash12(vec2(bayId * 7.1 + unitId, side + floor(at.x * 0.5)));
float slot = fract(bx * 8.0 + 0.5);
float led = smoothstep(0.34, 0.16, abs(slot - 0.5)) * smoothstep(0.3, 0.12, abs(uy));
// Blinking is quantised onto the bar grid rather than free-running: on a
// wall of two hundred lights, free-running flicker is a fizz, and stepping
// it keeps the whole wall under the flash ceiling as well.
float step8 = floor(u_barPhase * 8.0) + floor(at.x * 3.0);
float busy = step(1.0 - u_activity, hash12(vec2(id * 31.0, step8)));
float bright = mix(0.25, 1.0, busy);
col += palRamp(0.55 + id * 0.35) * led * bright * lightAmount * 1.8;
col += pal(4) * inkStroke(seam - seamW) * (0.15 + u_sigLine * 0.9);
return col;
}
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_run + u_seed;
p = sigCamera(p);
float horizon = sigHorizonY() * 0.25;
vec2 c = vec2(p.x, p.y - horizon);
// Corridor: the nearest of four planes. Distance down the aisle is 1/|c|,
// which is the whole of the perspective.
float dx = abs(c.x), dy = abs(c.y);
bool onWall = dx * (1.0 / max(u_aisle, 0.05)) > dy * 2.4;
float alongDist = onWall ? u_aisle / max(dx, 0.004) : 0.42 / max(dy, 0.004);
float along = alongDist * 0.5 + t * 6.0;
vec3 col;
if (onWall) {
// Height on the wall, corrected for distance so the racks stay upright.
float up = c.y * alongDist / u_aisle * 0.5 + 0.5;
col = wall(vec2(along, sat(up)), sign(c.x), t, 0.6 + u_leds * 0.9);
} else if (c.y < 0.0) {
// Floor: a lit strip down the middle, and the spill off both walls.
float across = c.x * alongDist / 0.42;
col = pal(0) * 0.16 + pal(2) * 0.12;
col += pal(3) * exp(-abs(across) * 2.5) * (0.45 + u_leds * 0.5);
col += pal(1) * 0.05 * step(0.5, fract(along * 0.5));
} else {
// Ceiling: cable trays and the occasional fitting.
float across = c.x * alongDist / 0.42;
col = pal(0) * 0.07;
col += pal(1) * 0.12 * smoothstep(0.35, 0.2, abs(fract(across * 1.5) - 0.5));
col += pal(3) * smoothstep(0.12, 0.0, abs(fract(along * 0.25) - 0.5)) * u_leds * 0.25;
}
// Distance haze, and the vanishing point itself, which is where the aisle
// ends and the only part of the frame that never resolves.
float far = sat(1.0 - alongDist * 0.06);
col = mix(col, pal(1) * 0.12, sat(far * 0.7));
col = sigAir(col, p, far);
return vec4(inkValue(col), 1.0);
}
`,
};
export default dataAisle;

82
flow-state/src/scenes/shader/droste-feedback.js Normal file
View File

@ -0,0 +1,82 @@
// Glitch family: video feedback. A camera pointed at its own monitor — the
// frame is redrawn inside itself slightly smaller and slightly turned, so a
// small motif at the centre becomes an endless corridor of copies of itself.
//
// Time Smear is the other feedback glitch and it TRANSLATES the previous frame,
// which leaves a comet trail behind a moving subject. This scales it. The
// difference is the whole scene: a translated feedback drifts off the edge and
// clears, a scaled one never clears, because everything the frame has ever
// contained is still in there, one ring further in.
//
// A live motif is drawn every frame, so the image exists before the loop has
// anything in it and comes back after a seek.
//
// No declared slow axis. What the corridor holds depends on where the loop has
// been, so two ten-second windows of the same parameters are 0.020 apart on
// their own and the persistence axis moved 0.004 — the same measurement Turing
// Bloom and Voronoi Shatter record. A feedback scene's history is its structure,
// and Phase 11 cannot separate a parameter from it.
export const drosteFeedback = {
name: 'Droste Feedback',
family: 'glitch',
kind: 'fragment',
texture: 0.5,
consumes: ['ink'],
traits: ['camera', 'style'],
params: {
zoom: { type: 'float', range: [0.93, 1.02], default: 0.975, uniform: 'u_zoomStep' },
turn: { type: 'float', range: [-0.12, 0.12], default: 0.04, uniform: 'u_turn' },
persist: { type: 'float', range: [0.9, 0.995], default: 0.985, uniform: 'u_persist' },
motif: { type: 'float', range: [0.04, 0.35], default: 0.14, uniform: 'u_motif' },
tint: { type: 'float', range: [0, 0.5], default: 0.15, uniform: 'u_tint' },
pace: { type: 'float', range: [0.05, 0.8], default: 0.25, uniform: 'u_pace', bias: 'motion', rate: true },
palette: { type: 'palette', count: 5 },
},
reactive: {
motif: { feature: 'beat', amount: 0.2, response: 'spike' },
},
shader: `
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_pace + u_seed;
// Folded as well as filmed: a feedback loop through a mirror is still a
// feedback loop, and the fold is the track's, not this scene's.
p = sigFolded(sigCamera(p));
// The motif. Small, live, and always drawn: this is the thing the loop eats.
vec2 at = vec2(sin(t * 0.6), cos(t * 0.47)) * 0.42;
float d = length(p - at) - u_motif;
float ring = abs(d) - u_motif * 0.35;
vec3 col = pal(0) * 0.05;
col += palRamp(0.2 + fbm(p * 2.0 + t, 3) * 0.2) * smoothstep(0.02, -0.02, ring) * (0.5 + 0.6 * 0.8);
col += pal(4) * inkStroke(ring) * (0.3 + 0.6 * 0.9);
// The loop: read the previous frame from a slightly LARGER, slightly turned
// sample of itself, so last frame's image comes back shrunk toward the
// middle. Contracting is what nests — expanding pushes every copy off the
// edge and leaves a plume instead of a corridor. Scaling about the centre in
// screen space is what keeps the copies concentric rather than sliding.
vec2 c = uv - 0.5;
c.x *= u_aspect;
c = rot(u_turn) * c / max(u_zoomStep, 0.5);
c.x /= u_aspect;
vec3 past = prev(c + 0.5);
// Each generation is tinted a step further round the palette, so depth into
// the corridor is legible as colour rather than only as size.
past = mix(past, palRamp(0.6) * (past.r + past.g + past.b) * 0.5, u_tint);
col = max(col, past * u_persist);
// Vignette, which also stops the corner content being recycled forever.
col *= 0.75 + 0.25 * exp(-dot(p, p) * 0.35);
return vec4(inkValue(col), 1.0);
}
`,
};
export default drosteFeedback;

6
flow-state/src/scenes/shader/dust-chamber.js
View File

@ -13,6 +13,7 @@ export const dustChamber = {
name: 'Dust Chamber', name: 'Dust Chamber',
family: 'minimal', family: 'minimal',
kind: 'fragment', kind: 'fragment',
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'space', 'style'], traits: ['shape', 'camera', 'space', 'style'],
params: { params: {
@ -77,13 +78,12 @@ vec4 scene(vec2 uv, vec2 p) {
float lit = exp(-pow((at.x - axis) / max(halfWidth, 1e-3), 2.0) * 2.0); float lit = exp(-pow((at.x - axis) / max(halfWidth, 1e-3), 2.0) * 2.0);
float pulse = 0.55 + 0.45 * sin(t * 2.0 + s * 3.0); float pulse = 0.55 + 0.45 * sin(t * 2.0 + s * 3.0);
float m = sigForm(p, at, u_moteSize * (0.6 + hash11(s + 3.3))); float m = castForm(p, at, u_moteSize * (0.6 + hash11(s + 3.3)));
col += pal(3) * m * lit * pulse * (0.5 + beam); col += pal(3) * m * lit * pulse * (0.5 + beam);
} }
col = sigAir(col, p, smoothstep(0.0, 1.5, length(p))); col = sigAir(col, p, smoothstep(0.0, 1.5, length(p)));
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

12
flow-state/src/scenes/shader/eclipse-field.js
View File

@ -18,6 +18,11 @@ export const eclipseField = {
kind: 'fragment', kind: 'fragment',
// Crisp line work: the track's surface grain would only fur the edges. // Crisp line work: the track's surface grain would only fur the edges.
texture: 0, texture: 0,
consumes: ['cast', 'ink'],
// No `style`: this scene opts out of surface grain (texture: 0) and its only
// other style evidence was sigEdge, which the migration replaced with the
// ink. Traits and artifacts are different layers, so taking the ink does not
// earn the trait back. See MIGRATION.md.
traits: ['shape', 'camera', 'space', 'style'], traits: ['shape', 'camera', 'space', 'style'],
params: { params: {
@ -47,7 +52,7 @@ vec4 scene(vec2 uv, vec2 p) {
vec2 discAt = vec2(0.0, sigHorizonY() * 0.25); vec2 discAt = vec2(0.0, sigHorizonY() * 0.25);
// Distance to the occluder's edge, in the track's form. // Distance to the occluder's edge, in the track's form.
float d = sigShape((p - discAt) / max(u_size, 1e-3)) * u_size; float d = castMain((p - discAt) / max(u_size, 1e-3)) * u_size;
// The source behind it. // The source behind it.
float toLight = length(p - lightAt); float toLight = length(p - lightAt);
@ -76,11 +81,10 @@ vec4 scene(vec2 uv, vec2 p) {
// The occluder: not pure black, so it reads as an object rather than a hole. // The occluder: not pure black, so it reads as an object rather than a hole.
col = mix(col, pal(0) * 0.12, smoothstep(0.004, -0.004, d)); col = mix(col, pal(0) * 0.12, smoothstep(0.004, -0.004, d));
col += pal(3) * sigEdge(d) * (0.4 + u_corona * 0.5); col += pal(3) * inkStroke(d) * (0.4 + u_corona * 0.5);
col = sigAir(col, p, smoothstep(0.0, 1.7, length(p))); col = sigAir(col, p, smoothstep(0.0, 1.7, length(p)));
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

21
flow-state/src/scenes/shader/firefly-drift.js
View File

@ -10,19 +10,18 @@ export const fireflyDrift = {
family: 'flow', family: 'flow',
kind: 'fragment', kind: 'fragment',
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
consumes: ['cast', 'ink', 'staging'],
traits: ['camera', 'style'], traits: ['camera', 'style'],
params: { params: {
count: { type: 'float', range: [6, 48], default: 22, uniform: 'u_count', bias: 'density' }, count: { type: 'float', range: [6, 48], default: 22, uniform: 'u_count', bias: 'density' },
speed: { type: 'float', range: [0.05, 0.8], default: 0.3, uniform: 'u_speed', bias: 'motion', rate: true }, speed: { type: 'float', range: [0.05, 0.8], default: 0.3, uniform: 'u_speed', bias: 'motion', rate: true },
glow: { type: 'float', range: [0, 1.2], default: 0.5, uniform: 'u_glow', bias: 'energy' },
jitter: { type: 'float', range: [0, 0.4], default: 0.12, uniform: 'u_jitter' }, jitter: { type: 'float', range: [0, 0.4], default: 0.12, uniform: 'u_jitter' },
spread: { type: 'float', range: [0.4, 1.3], default: 0.95, uniform: 'u_spread' }, spread: { type: 'float', range: [0.4, 1.3], default: 0.95, uniform: 'u_spread' },
palette: { type: 'palette', count: 4 }, palette: { type: 'palette', count: 4 },
}, },
reactive: { reactive: {
glow: { feature: 'beat', amount: 0.3, response: 'spike' },
jitter: { feature: 'bandHigh', amount: 0.2, response: 'smooth' }, jitter: { feature: 'bandHigh', amount: 0.2, response: 'smooth' },
}, },
@ -38,25 +37,27 @@ vec4 scene(vec2 uv, vec2 p) {
if (float(i) >= u_count) break; if (float(i) >= u_count) break;
float fi = float(i); float fi = float(i);
// Seed the mote to a stable place in the frame (deterministic per i). // The mote's home is the song's lattice; the drift is this scene's own.
vec2 grid = vec2(hash12(vec2(fi, 1.7)), hash12(vec2(fi, 9.1))); vec3 node = stageNode(fi, u_count);
vec2 base = (grid - 0.5) * 2.0 * u_spread * vec2(1.0, 0.7); vec2 base = node.xy * u_spread * vec2(1.0, 0.7);
// Advect along the curl field plus a bounded wander term on top. // Advect along the curl field plus a bounded wander term on top.
vec2 flow = curl(grid * 3.0 + vec2(0.0, t * 0.15), t * 0.5); vec2 flow = curl(node.xy * 3.0 + vec2(0.0, t * 0.15), t * 0.5);
vec2 pos = base + flow * 1.3 vec2 pos = base + flow * 1.3
+ vec2(sin(t * (0.4 + fract(fi * 0.13))), + vec2(sin(t * (0.4 + fract(fi * 0.13))),
cos(t * (0.5 + fract(fi * 0.29)))) * u_jitter; cos(t * (0.5 + fract(fi * 0.29)))) * u_jitter;
// A mote is one of the chorus, drawn small — the glow is the scene's
// contribution, the form is the song's.
float size = 0.045 * node.z;
float sd = castChorus((p - pos) / max(size, 1e-3)) * size;
float d = length(p - pos); float d = length(p - pos);
vec3 hc = pal(int(mod(fi, 4.0))); vec3 hc = pal(int(mod(fi, 4.0)));
col += hc * (exp(-d * d / (0.015 + u_glow * 0.04)) * (1.0 + u_glow * 0.6)); col = mix(col, hc, inkMask(sd, uv));
col += hc * exp(-d * d * 60.0);
} }
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

73
flow-state/src/scenes/shader/floating-geometry.js
View File

@ -1,6 +1,14 @@
// Ported from party-stage's "Floating Geometry". Shape count, size and motion // Ported from party-stage's "Floating Geometry". Shape count, size and motion
// were fixed constants in the original; they are the whole point of the scene, // were fixed constants in the original; they are the whole point of the scene,
// so they are now params the look generator can move. // so they are now params the look generator can move.
//
// The bodies are SOLID. This scene is the one in the library whose whole
// premise is objects adrift with nothing under them, and adrift is a thing that
// only reads in three dimensions: a flat silhouette rotating is a sticker
// turning, and no amount of drift makes it a body. Each element is now an
// instance of the song's assembly, marched in its own frame — so a shape hides
// its own far side, and two of them at different angles are visibly the same
// object seen twice rather than two copies of one outline. See castSolid.
export const floatingGeometry = { export const floatingGeometry = {
name: 'Floating Geometry', name: 'Floating Geometry',
@ -10,6 +18,7 @@ export const floatingGeometry = {
// of shapes, so `shape` is the trait it exists to express. // of shapes, so `shape` is the trait it exists to express.
// Takes the track's surface grain, but lightly — this is drawn, not filmed. // Takes the track's surface grain, but lightly — this is drawn, not filmed.
texture: 0.4, texture: 0.4,
consumes: ['form', 'ink', 'staging'],
traits: ['shape', 'camera', 'style'], traits: ['shape', 'camera', 'style'],
params: { params: {
@ -18,14 +27,12 @@ export const floatingGeometry = {
drift: { type: 'float', range: [0.1, 1.5], default: 0.4, uniform: 'u_drift', bias: 'motion', rate: true }, drift: { type: 'float', range: [0.1, 1.5], default: 0.4, uniform: 'u_drift', bias: 'motion', rate: true },
spin: { type: 'float', range: [0, 2], default: 0.6, uniform: 'u_spin', rate: true }, spin: { type: 'float', range: [0, 2], default: 0.6, uniform: 'u_spin', rate: true },
variety: { type: 'float', range: [0, 0.6], default: 0.25, uniform: 'u_variety' }, variety: { type: 'float', range: [0, 0.6], default: 0.25, uniform: 'u_variety' },
aura: { type: 'float', range: [0, 1], default: 0.25,uniform: 'u_aura', bias: 'energy' },
spread: { type: 'float', range: [0.3, 1.2], default: 0.8, uniform: 'u_spread' }, spread: { type: 'float', range: [0.3, 1.2], default: 0.8, uniform: 'u_spread' },
palette: { type: 'palette', count: 5 }, palette: { type: 'palette', count: 5 },
}, },
reactive: { reactive: {
size: { feature: 'beat', amount: 0.18, response: 'spike' }, size: { feature: 'beat', amount: 0.18, response: 'spike' },
aura: { feature: 'bandHigh', amount: 0.4 },
}, },
shader: ` shader: `
@ -36,35 +43,69 @@ vec4 scene(vec2 uv, vec2 p) {
// Background wash from the two darkest palette entries. // Background wash from the two darkest palette entries.
vec3 col = mix(pal(0) * 0.18, pal(1) * 0.24, sin(t) * 0.5 + 0.5); vec3 col = mix(pal(0) * 0.18, pal(1) * 0.24, sin(t) * 0.5 + 0.5);
// Whether a body already owns this pixel. The elements all float at the same
// depth, so which one wins where they overlap was always arbitrary — it used
// to be whichever came last. Making it whichever comes FIRST costs nothing
// visually and bounds the work at roughly one march per pixel: at the top of
// the size range fourteen bodies cover the frame several deep, and marching
// every one of them at every pixel was slow enough that the scene gate ran
// for minutes without finishing.
float painted = 0.0;
for (int i = 0; i < 14; i++) { for (int i = 0; i < 14; i++) {
if (i >= u_count) break; if (i >= u_count) break;
float fi = float(i); float fi = float(i);
float s = u_seed + fi * 123.456; float s = u_seed + fi * 123.456;
vec2 pos = vec2( // The song's lattice sets where they hang; this scene keeps the float.
sin(t * 0.5 + s) * u_spread, vec3 node = stageNode(fi, float(u_count));
cos(t * 0.3 + s * 1.1) * u_spread * 0.55 vec2 pos = node.xy * u_spread * vec2(1.0, 0.55)
); + vec2(sin(t * 0.5 + s), cos(t * 0.3 + s * 1.1)) * 0.18;
vec2 sp = rot(t * (0.2 + fract(s) * u_spin)) * (p - pos); float size = u_size * node.z * (0.6 + fract(s * 0.7) * 0.8);
float size = u_size * (0.6 + fract(s * 0.7) * 0.8); // Every element is the song's protagonist. This scene used to pick
// Every element is the track's signature form. This scene used to pick
// between a box and a circle per element, which is precisely the choice // between a box and a circle per element, which is precisely the choice
// the production design should be making — one video, one cast. // the production design should be making — one video, one cast.
// The variety param only scales them apart; it never changes what they are. // The variety param only scales them apart; it never changes what they are.
float scale = size * (1.0 + (fract(s * 0.37) - 0.5) * u_variety); float scale = size * (1.0 + (fract(s * 0.37) - 0.5) * u_variety);
float d = sigShape(sp / max(scale, 1e-3)) * scale;
// Each body turns on its own two axes, at its own rate. The spread of
// angles is what makes the field read as one object seen from several
// sides rather than as a row of identical stamps — the thing a flat
// silhouette could not do however fast it span.
mat3 turn = castTurn(t * (0.2 + fract(s) * u_spin) + s,
sin(t * 0.4 + s * 2.1) * 0.9);
vec2 local = (p - pos) / max(scale, 1e-3);
// Nothing this element can contribute to this pixel. Cheap to ask, and
// it is what keeps the cost proportional to the elements a pixel
// actually touches rather than to the elements in the frame — without
// it the slice distance below is evaluated fourteen times per pixel.
if (dot(local, local) > 1.6 || painted > 0.5) continue;
vec3 n;
float hit = castSolid(local, turn, n);
vec3 shapeColor = pal(i + int(floor(t * 0.3))); vec3 shapeColor = pal(i + int(floor(t * 0.3)));
float intensity = smoothstep(0.012, 0.0, d) + sigEdge(d) * 0.35; if (hit > 0.0) {
col = mix(col, shapeColor, intensity * 0.85); painted = 1.0;
col += shapeColor * (1.0 - smoothstep(0.0, size * 2.2, abs(d))) * u_aura; // Lit in the track's key light, then pulled toward this element's
// own palette entry so the field keeps the colour rhythm it had.
col = mix(castLit(n, vec3(0.0, 0.0, 1.0)), shapeColor, 0.35);
} }
col += sigGrain(uv); // The halo and the rim survive the migration: the halo is what stops a
return vec4(col, 1.0); // dark body on a dark wash from disappearing, and the rim is the body's
// outline drawn in the track's line weight. Both read off the object's
// own slice at this angle rather than off a separate flat profile, so
// there is still exactly one shape on screen.
float d = castSDF3(turn * vec3(local, 0.0)) * scale;
col += shapeColor * (1.0 - smoothstep(0.0, size * 2.2, abs(d))) * 0.25;
col = mix(col, shapeColor, inkStroke(d) * 0.7);
}
return vec4(inkValue(col), 1.0);
} }
`, `,
}; };

8
flow-state/src/scenes/shader/flora.js
View File

@ -12,6 +12,7 @@ export const flora = {
family: 'organic', family: 'organic',
kind: 'fragment', kind: 'fragment',
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'style'], traits: ['shape', 'camera', 'style'],
params: { params: {
@ -48,7 +49,7 @@ float petal(vec2 p, vec2 node, vec2 dir, float len, float width) {
float across = dot(off, perp); float across = dot(off, perp);
float taper = 0.6 + 0.4 * sat(along / max(len, 1e-3)); float taper = 0.6 + 0.4 * sat(along / max(len, 1e-3));
vec2 q = vec2(along / max(len, 1e-3), across / max(width * taper, 1e-3)); vec2 q = vec2(along / max(len, 1e-3), across / max(width * taper, 1e-3));
float form = sigForm(q * 0.5, vec2(0.0), 0.5); float form = castForm(q * 0.5, vec2(0.0), 0.5);
float clip = smoothstep(0.0, -0.1, along) * smoothstep(len + 0.12, len * 0.72, along); float clip = smoothstep(0.0, -0.1, along) * smoothstep(len + 0.12, len * 0.72, along);
return form * clip; return form * clip;
} }
@ -105,13 +106,12 @@ vec4 scene(vec2 uv, vec2 p) {
for (int k = 0; k < 6; k++) { for (int k = 0; k < 6; k++) {
float a = 6.2831853 * (float(k) + 0.5) / 6.0 + s; float a = 6.2831853 * (float(k) + 0.5) / 6.0 + s;
vec2 off = vec2(cos(a), sin(a)) * u_leaf * (0.30 + 0.2 * beat); vec2 off = vec2(cos(a), sin(a)) * u_leaf * (0.30 + 0.2 * beat);
bloom += sigForm(p - vec2(tipX, tipY), off, u_leaf * 0.5); bloom += castForm(p - vec2(tipX, tipY), off, u_leaf * 0.5);
} }
col = mix(col, pal(2), sat(bloom) * u_bud * (0.6 + 0.4 * beat)); col = mix(col, pal(2), sat(bloom) * u_bud * (0.6 + 0.4 * beat));
} }
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

6
flow-state/src/scenes/shader/gate-corridor.js
View File

@ -16,6 +16,7 @@ export const gateCorridor = {
kind: 'fragment', kind: 'fragment',
// Takes the track's surface grain, but lightly — this is drawn, not filmed. // Takes the track's surface grain, but lightly — this is drawn, not filmed.
texture: 0.4, texture: 0.4,
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'space', 'style'], traits: ['shape', 'camera', 'space', 'style'],
params: { params: {
@ -62,7 +63,7 @@ vec4 scene(vec2 uv, vec2 p) {
float phase = fract((fi / float(u_gates)) + t); float phase = fract((fi / float(u_gates)) + t);
float scale = u_aperture * exp(phase * 3.2) * 0.35; float scale = u_aperture * exp(phase * 3.2) * 0.35;
float d = abs(sigShape(q / max(scale, 1e-3)) * max(scale, 1e-3)); float d = abs(castMain(q / max(scale, 1e-3)) * max(scale, 1e-3));
// Near gates are drawn thicker and brighter: the only depth cue that // Near gates are drawn thicker and brighter: the only depth cue that
// matters once the geometry is right. // matters once the geometry is right.
@ -82,8 +83,7 @@ vec4 scene(vec2 uv, vec2 p) {
} }
col = sigAir(col, p, 1.0 - smoothstep(0.0, 1.2, length(q))); col = sigAir(col, p, 1.0 - smoothstep(0.0, 1.2, length(q)));
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

11
flow-state/src/scenes/shader/girder-lattice.js
View File

@ -17,6 +17,7 @@ export const girderLattice = {
kind: 'fragment', kind: 'fragment',
// Crisp line work: the track's surface grain would only fur the edges. // Crisp line work: the track's surface grain would only fur the edges.
texture: 0, texture: 0,
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'space', 'style'], traits: ['shape', 'camera', 'space', 'style'],
params: { params: {
@ -57,8 +58,7 @@ vec4 scene(vec2 uv, vec2 p) {
if (above <= 0.001) { if (above <= 0.001) {
col = sigAir(col, p, 1.0); col = sigAir(col, p, 1.0);
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
float best = 1e3; float best = 1e3;
@ -94,9 +94,9 @@ vec4 scene(vec2 uv, vec2 p) {
} }
if (u_plate > 0.004) { if (u_plate > 0.004) {
float pd = sigShape((p - vec2(halfW0, y0)) / u_plate) * u_plate; float pd = castMain((p - vec2(halfW0, y0)) / u_plate) * u_plate;
plateHit = min(plateHit, pd); plateHit = min(plateHit, pd);
pd = sigShape((p - vec2(-halfW0, y0)) / u_plate) * u_plate; pd = castMain((p - vec2(-halfW0, y0)) / u_plate) * u_plate;
plateHit = min(plateHit, pd); plateHit = min(plateHit, pd);
} }
@ -109,7 +109,7 @@ vec4 scene(vec2 uv, vec2 p) {
float steel = smoothstep(weight, weight * 0.3, best); float steel = smoothstep(weight, weight * 0.3, best);
col = mix(col, pal(2) * (0.3 + above * 0.5), steel); col = mix(col, pal(2) * (0.3 + above * 0.5), steel);
col += pal(3) * sigEdge(best - weight) * 0.4; col += pal(3) * inkStroke(best - weight) * 0.4;
// Joint plates. // Joint plates.
col = mix(col, pal(4) * 0.7, smoothstep(0.006, -0.006, plateHit)); col = mix(col, pal(4) * 0.7, smoothstep(0.006, -0.006, plateHit));
@ -118,7 +118,6 @@ vec4 scene(vec2 uv, vec2 p) {
col += pal(4) * exp(-length(p - vec2(0.0, vanishY + nearest * 1.4)) * 14.0) * u_lamp; col += pal(4) * exp(-length(p - vec2(0.0, vanishY + nearest * 1.4)) * 14.0) * u_lamp;
col = sigAir(col, p, 1.0 - smoothstep(0.0, 1.4, above)); col = sigAir(col, p, 1.0 - smoothstep(0.0, 1.4, above));
col += sigGrain(uv);
return vec4(col, 1.0); return vec4(col, 1.0);
} }
`, `,

102
flow-state/src/scenes/shader/halftone-misprint.js Normal file
View File

@ -0,0 +1,102 @@
// Glitch family: a four-colour press run with the plates out of register. Each
// ink is screened at its own angle, and each one lands a little off, so the
// image is fringed and the dots moiré against each other.
//
// Every other glitch scene here corrupts a signal — tearing it, freezing it,
// blocking it up. This one is a printing fault rather than an electronic one:
// the image is never damaged, it is simply separated and reassembled wrong.
// Nothing else in the library is made of dots, and the dots are what carry it.
export const halftoneMisprint = {
name: 'Halftone Misprint',
family: 'glitch',
kind: 'fragment',
// The paper's tooth is the point; take the track's grain in full.
texture: 1.2,
consumes: ['ink'],
traits: ['camera', 'style'],
params: {
screen: { type: 'float', range: [18, 90], default: 42, uniform: 'u_screen', bias: 'density' },
slip: { type: 'float', range: [0, 0.06], default: 0.02, uniform: 'u_slip' },
angle: { type: 'float', range: [0, 1.6], default: 0.5, uniform: 'u_angle' },
ink: { type: 'float', range: [0.3, 1.5], default: 0.9, uniform: 'u_ink', slowAxis: true },
art: { type: 'float', range: [0.5, 4], default: 1.6, uniform: 'u_art', bias: 'density' },
press: { type: 'float', range: [0.02, 0.4], default: 0.09, uniform: 'u_press', bias: 'motion', rate: true },
palette: { type: 'palette', count: 5 },
},
reactive: {
slip: { feature: 'flux', amount: 0.3, response: 'spike' },
ink: { feature: 'bandLow', amount: 0.25, response: 'smooth' },
},
shader: `
// The artwork being printed: a soft, slow field, deliberately simple. What the
// scene is about is the separation, not the picture.
float artwork(vec2 q, float t) {
float a = fbm(q * u_art + vec2(t * 0.5, -t * 0.3), 4);
float b = 0.5 + 0.5 * sin(q.x * u_art * 2.0 + a * 4.0 + t);
return sat(a * 0.7 + b * 0.5);
}
// One screened separation: sample the artwork where THIS plate landed, then
// threshold it against a rotated dot grid. The dot grows with ink coverage,
// which is what a halftone is.
float plate(vec2 q, float ang, vec2 slip, float t, float gain) {
float density = sat(artwork(q + slip, t) * gain);
vec2 r = rot(ang) * (q + slip);
// Ruling is a count of dots ACROSS THE FRAME, not a size in pixels — which
// is what keeps a 720p preview and a 4K export the same print rather than
// the same dot pitch on a bigger sheet.
vec2 cell = fract(r * u_screen * 0.5) - 0.5;
// Dot edge hardness is the track's: a sharp track prints a crisp dot on
// coated stock, a soft one lets the ink spread into the fibre.
float soft = 0.04 + u_sigSoft * 0.45;
float radius = sqrt(max(density, 0.0)) * 0.62;
return smoothstep(radius, radius - soft, length(cell));
}
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_press + u_seed;
p = sigCamera(p);
// Registration error. It steps on the bar rather than drifting: a press
// slips, settles, and slips again, and a continuously crawling offset reads
// as a wobble rather than as a fault.
float era = floor(u_barPhase * 4.0) + floor(t * 2.0) * 4.0;
vec2 slipC = (hash22(vec2(era, 1.0)) - 0.5) * u_slip;
vec2 slipM = (hash22(vec2(era, 2.0)) - 0.5) * u_slip;
vec2 slipY = (hash22(vec2(era, 3.0)) - 0.5) * u_slip;
// Classic screen angles: 15°, 75°, 0°, spread by the angle parameter so a
// track can have them in near-register or wildly rosetted.
float a1 = 0.26 * u_angle, a2 = 1.31 * u_angle, a3 = 0.0;
float c = plate(p, a1, slipC, t, u_ink);
float m = plate(p, a2, slipM, t, u_ink);
float y = plate(p, a3, slipY, t, u_ink);
// Subtractive-ish: each ink takes light out of the paper, in its own hue.
vec3 col = pal(1) * 0.85;
col -= pal(2) * c * 0.5;
col -= pal(3) * m * 0.5;
col -= pal(4) * y * 0.45;
// The black plate, printed last and in register, which is what stops the
// whole thing turning to mud.
float k = plate(p, 0.65 * u_angle, vec2(0.0), t, u_ink * 0.7);
col = mix(col, pal(0) * 0.12, k * 0.55);
// Ink edges take the track's line weight — a hard press or a soft one.
col += pal(0) * sigEdge(0.5 - c) * (0.05 + u_sigLine * 0.35);
col = max(col, vec3(0.0));
return vec4(inkValue(col), 1.0);
}
`,
};
export default halftoneMisprint;

33
flow-state/src/scenes/shader/horizon-lines.js
View File

@ -9,6 +9,7 @@ export const horizonLines = {
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
// Crisp line work: the track's surface grain would only fur the edges. // Crisp line work: the track's surface grain would only fur the edges.
texture: 0, texture: 0,
consumes: ['ink'],
traits: ['space', 'camera', 'style'], traits: ['space', 'camera', 'style'],
params: { params: {
@ -17,13 +18,11 @@ export const horizonLines = {
bend: { type: 'float', range: [0, 1.2], default: 0.35, uniform: 'u_bend' }, bend: { type: 'float', range: [0, 1.2], default: 0.35, uniform: 'u_bend' },
speed: { type: 'float', range: [0.02, 0.4], default: 0.12, uniform: 'u_speed', bias: 'motion', rate: true }, speed: { type: 'float', range: [0.02, 0.4], default: 0.12, uniform: 'u_speed', bias: 'motion', rate: true },
spread: { type: 'float', range: [0.2, 1.4], default: 0.9, uniform: 'u_spread' }, spread: { type: 'float', range: [0.2, 1.4], default: 0.9, uniform: 'u_spread' },
glow: { type: 'float', range: [0, 1], default: 0.3, uniform: 'u_glow', bias: 'energy' },
palette: { type: 'palette', count: 4 }, palette: { type: 'palette', count: 4 },
}, },
reactive: { reactive: {
bend: { feature: 'bandLow', amount: 0.35, response: 'smooth' }, bend: { feature: 'bandLow', amount: 0.35, response: 'smooth' },
glow: { feature: 'beat', amount: 0.3, response: 'spike' },
}, },
shader: ` shader: `
@ -39,7 +38,10 @@ vec4 scene(vec2 uv, vec2 p) {
float offset = sin(p.x * 2.2 + t * 2.0) * u_bend * envelope; float offset = sin(p.x * 2.2 + t * 2.0) * u_bend * envelope;
vec3 col = pal(0) * 0.06; vec3 col = pal(0) * 0.06;
float total = 0.0;
// One pixel, in the scene units p is measured in. p spans [-1, 1] vertically
// regardless of output size, so this is the exact height of a pixel.
float px = 2.0 / u_resolution.y;
for (int i = 0; i < 40; i++) { for (int i = 0; i < 40; i++) {
if (float(i) >= u_count) break; if (float(i) >= u_count) break;
@ -49,19 +51,34 @@ vec4 scene(vec2 uv, vec2 p) {
float y = slot + offset * (0.4 + fract(fi * 0.37)); float y = slot + offset * (0.4 + fract(fi * 0.37));
float d = abs(p.y - y); float d = abs(p.y - y);
float line = smoothstep(u_thickness * (0.5 + u_sigLine), 0.0, d); // Analytic coverage rather than a smoothstep to zero.
float halo = exp(-d * 26.0) * u_glow; //
// The width here is genuinely sub-pixel: 0.008 scene units against a
// 0.028-unit pixel at 720p is a line under a third of a pixel wide, and
// the range goes down to a fourteenth. Ramping from full to nothing
// across a third of a pixel is a near-vertical cliff, and a cliff turns
// any float wobble in d into a whole byte of colour — which is what
// made this the one scene in the library that failed the determinism
// gate, intermittently, at 2/255.
//
// Coverage fixes the cause rather than the symptom. The transition
// always spans exactly one pixel, so nothing sits on a discontinuity;
// and a line thinner than a pixel now comes out DIM rather than being
// drawn at full brightness wherever a pixel centre happens to land on
// it, which is both correct and the end of the shimmer it used to have.
// Width stays in scene units, so resolution independence is unchanged.
float w = u_thickness * (0.5 + u_sigLine);
float line = sat((w - d) / px + 0.5);
float halo = exp(-d * 26.0) * 0.3;
vec3 c = pal(i); vec3 c = pal(i);
col += c * (line + halo * 0.55); col += c * (line + halo * 0.55);
total += line;
} }
// Keep the far edges dark so the lines read as a subject, not wallpaper. // Keep the far edges dark so the lines read as a subject, not wallpaper.
col *= 0.55 + 0.45 * exp(-dot(p, p) * 0.5); col *= 0.55 + 0.45 * exp(-dot(p, p) * 0.5);
col = sigAir(col, p, smoothstep(0.0, 1.6, abs(p.y))); col = sigAir(col, p, smoothstep(0.0, 1.6, abs(p.y)));
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

4
flow-state/src/scenes/shader/ink-bleed.js
View File

@ -13,6 +13,7 @@ export const inkBleed = {
name: 'Ink Bleed', name: 'Ink Bleed',
family: 'organic', family: 'organic',
kind: 'fragment', kind: 'fragment',
consumes: ['ink'],
traits: ['camera', 'space', 'style'], traits: ['camera', 'space', 'style'],
params: { params: {
@ -72,8 +73,7 @@ vec4 scene(vec2 uv, vec2 p) {
col = max(col, soaked * u_soak); col = max(col, soaked * u_soak);
col = sigAir(col, p, smoothstep(0.0, 1.7, length(p))); col = sigAir(col, p, smoothstep(0.0, 1.7, length(p)));
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

164
flow-state/src/scenes/shader/isometric-blocks.js Normal file
View File

@ -0,0 +1,164 @@
// Geometric family: a field of extruded blocks seen isometrically, rising and
// falling on the low end. A solid, lit surface with a light direction — three
// faces per block, and the whole grid reads as a landscape rather than a chart.
//
// Floating Geometry is bodies adrift in space with nothing under them. This is
// the opposite proposition: everything is on one grid, everything touches the
// ground, and the only freedom any block has is its height. The plan of each
// block is the track's signature form, so a hexagonal video gets a honeycomb of
// columns rather than a chessboard of them.
export const isometricBlocks = {
name: 'Isometric Blocks',
family: 'geometric',
kind: 'fragment',
texture: 0.6,
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'style'],
params: {
grid: { type: 'float', range: [2, 10], default: 4.5, uniform: 'u_grid', bias: 'density', slowAxis: true },
height: { type: 'float', range: [0.1, 0.7], default: 0.35, uniform: 'u_height', bias: 'energy' },
// Displacement: how far each block wanders from its slot on the grid.
//
// Every block sat exactly on the lattice, so the plan of the field was
// the same plan in every song and only the heights moved — which is
// what the gallery was reporting as a layout distance of 0.005, near
// enough to nothing. Displacing them per cell puts the SONG in the
// arrangement rather than only in the elevation.
scatter: { type: 'float', range: [0, 0.85], default: 0.3, uniform: 'u_scatter', bias: 'density' },
// Extra disturbance AT the song's focal points, on top of the scatter
// everywhere — not instead of it.
//
// It was a trade at first, spending the scatter budget near the focus
// and calming the rest, and that measured worse than no focus at all:
// 0.049 down to 0.037. Most of the field went back onto the rigid
// lattice and took the orientation variety with it, 0.164 to 0.105. The
// focal point has to be something the field gains, not something the
// rest of it pays for.
gather: { type: 'float', range: [0, 1.6], default: 0.8, uniform: 'u_gather' },
plan: { type: 'float', range: [0.2, 0.5], default: 0.4, uniform: 'u_plan' },
tilt: { type: 'float', range: [0.35, 0.75], default: 0.55, uniform: 'u_isoTilt' },
light: { type: 'float', range: [0, 1.4], default: 0.65, uniform: 'u_light' },
wave: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_wave' },
roll: { type: 'float', range: [0.01, 0.08], default: 0.035, uniform: 'u_roll', bias: 'motion', rate: true },
palette: { type: 'palette', count: 5 },
},
// The low end deliberately does NOT drive the block heights. Measured at
// aggressive settings that read as 4 flashes a second: every tower in the
// frame grows and shrinks together, which is a whole-frame luminance cycle
// on the kick and exactly what the flash ceiling exists to stop. The beat
// lands on a quarter of the tops instead — same reading, local swing.
reactive: {
light: { feature: 'loudness', amount: 0.25, response: 'smooth' },
},
shader: `
// How tall the block at this cell stands, in screen units. A travelling wave
// plus a fixed per-cell character, so the field has a rhythm running through it
// and a skyline that is still recognisable between beats.
float columnHeight(vec2 cell, float t) {
float own = hash12(cell + u_seed);
float wave = sin(dot(cell, vec2(0.7, 0.5)) - t * 1.1) * 0.5 + 0.5;
// Squared, so most of the field lies flat and the tall ones stand out as
// individual towers. A linear height makes every column tall enough to
// occlude its neighbours, and the field turns into vertical stripes.
return u_height * pow(mix(own, wave, u_wave), 2.4);
}
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_roll + u_seed;
p = sigCamera(p);
float tilt = max(u_isoTilt, 0.1);
float cw = 1.0 / u_grid; // one cell across, on screen
vec3 col = mix(pal(0) * 0.06, pal(1) * 0.14, sat(p.y * 0.4 + 0.5));
// A block standing on row r covers screen rows ABOVE r's ground line, so a
// pixel can only belong to a row at or behind it. Walking a fixed number of
// rows from the back forward and overwriting as we go is the painter's
// algorithm, and it is the whole of the depth sorting an isometric field
// needs — no sorting, no z buffer, just the order of the loop.
float colX = floor(p.x / cw);
float fx = (p.x / cw - colX - 0.5) * cw; // screen offset within the column
// Alternate columns are staggered half a row, which is what interlocks the
// field into a lattice instead of leaving it a set of independent stacks.
float stagger = mod(colX, 2.0) * 0.5;
float rowHere = floor(p.y / (tilt * cw) - stagger);
// Rows FURTHER AWAY stand higher up the frame, so a pixel can only belong
// to a block whose ground line is at or below it — row <= rowHere. Counting
// down from there draws the far ones first and lets the near ones paint
// over them.
// lint: fixed-cost — eight rows of depth, back to front
for (int j = 7; j >= 0; j--) {
float row = rowHere - float(7 - j);
vec2 cell = vec2(colX, row);
// Where this block would stand if nothing disturbed it, so the focus can
// be asked about the cell rather than about the pixel.
float restY = (row + stagger) * tilt * cw;
float focus = focusField(vec2((colX + 0.5) * cw - 0.5, restY));
// Each cell steps off its slot by a fixed amount of its own, and how far
// depends on how close it is to what the song is looking at. Spread
// evenly this is noise on a grid; concentrated, the lattice stands
// except where it is disturbed, and the field acquires a subject.
//
// Bounded under half a cell: past that blocks cross each other and the
// isometric read — which is the whole point of the scene — breaks down.
float amount = u_scatter * (1.0 + focus * u_gather * 2.0);
vec2 offset = (hash22(cell + u_seed + 3.7) - 0.5) * amount * cw * 0.9;
float groundY = restY + offset.y; // where this cell meets the ground
// Blocks rise toward the focus as well as scattering around it, so the
// disturbance is something you can see in the skyline rather than only
// in the plan — and rising changes WHERE the frame's energy is, which
// scattering alone did not.
float h = columnHeight(cell, t)
* (1.0 + offset.x * 2.0)
* (1.0 + focus * u_gather * u_focusPull * 1.3);
float dy = p.y - groundY; // height up the block face
// Side wall: the plan swept up from the ground line to the top face.
float inX = smoothstep(u_plan * cw, u_plan * cw * 0.85, abs(fx - offset.x));
float wall = inX * step(0.0, dy) * step(dy, h);
float lit = 0.5 + 0.5 * sat(fx / (u_plan * cw) * 0.8 + 0.5);
// The wall darkens toward the ground, which is the only shading cue
// that tells a tall block from a short one at a glance.
// Neither face takes the loudness. Anything that multiplies a face
// colour multiplies most of the frame, and a field of blocks brightening
// together on the kick measured at 5 flashes a second. The light level
// is expressed on the EDGES, which are a few percent of the image.
vec3 side = mix(pal(0) * 0.5, pal(2) * 0.8, lit) * 0.75;
side *= 0.45 + 0.55 * sat(dy / max(h, 1e-3));
col = mix(col, side, wall);
// Top face: the track's signature form, lying flat, lifted by h and
// squashed by the tilt — so every block in the field is the same
// silhouette as every other subject in the video.
vec2 q = vec2(fx, (p.y - groundY - h) / tilt);
float dTop = castMain(q / (u_plan * cw)) * (u_plan * cw);
float topMask = smoothstep(0.004, -0.004, dTop);
vec3 top = mix(pal(2), pal(3), sat(h / max(u_height, 1e-3))) * 0.8;
col = mix(col, top, topMask);
// A quarter of the blocks answer the beat, chosen by cell rather than
// by anything global, so the field has a pulse without the frame having
// a flash.
float picked = step(0.75, hash12(cell * 3.17 + 5.0));
col += pal(3) * topMask * picked * u_beat * 0.45;
col += pal(4) * inkStroke(dTop) * (0.2 + u_light * 1.2) * step(0.0, dy);
}
col *= 0.75 + 0.25 * exp(-dot(p, p) * 0.3);
return vec4(inkValue(col), 1.0);
}
`,
};
export default isometricBlocks;

23
flow-state/src/scenes/shader/kaleido-tunnel.js
View File

@ -6,9 +6,9 @@ export const kaleidoTunnel = {
family: 'geometric', family: 'geometric',
kind: 'fragment', kind: 'fragment',
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
// Grain is this scene's ONLY expression of the style trait — it draws no // Hard grid lines on flat colour: grain reads as dirt on the lens.
// hard edges to weight — so it keeps a share of it rather than opting out. texture: 0,
texture: 0.35, consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'style'], traits: ['shape', 'camera', 'style'],
params: { params: {
@ -17,12 +17,10 @@ export const kaleidoTunnel = {
speed: { type: 'float', range: [0.1, 1.5], default: 0.45, uniform: 'u_speed', bias: 'motion', rate: true }, speed: { type: 'float', range: [0.1, 1.5], default: 0.45, uniform: 'u_speed', bias: 'motion', rate: true },
twist: { type: 'float', range: [0, 2], default: 0.5, uniform: 'u_twist' }, twist: { type: 'float', range: [0, 2], default: 0.5, uniform: 'u_twist' },
rings: { type: 'float', range: [2, 24], default: 8, uniform: 'u_rings', bias: 'density' }, rings: { type: 'float', range: [2, 24], default: 8, uniform: 'u_rings', bias: 'density' },
glow: { type: 'float', range: [0, 1.5], default: 0.5, uniform: 'u_glow', bias: 'energy' },
palette: { type: 'palette', count: 6 }, palette: { type: 'palette', count: 6 },
}, },
reactive: { reactive: {
glow: { feature: 'beat', amount: 0.45, response: 'spike' },
twist: { feature: 'bandLow', amount: 0.3 }, twist: { feature: 'bandLow', amount: 0.3 },
rings: { feature: 'bandHigh', amount: 0.2 }, rings: { feature: 'bandHigh', amount: 0.2 },
}, },
@ -38,7 +36,7 @@ vec4 scene(vec2 uv, vec2 p) {
// Cross-section measured in the signature form: the tunnel mouth is the // Cross-section measured in the signature form: the tunnel mouth is the
// track's shape rather than a circle. // track's shape rather than a circle.
float radius = max(sigShape(p) + 1.0, 1e-4); float radius = max(castMain(p) + 1.0, 1e-4);
vec2 folded = kaleido(p, sides); vec2 folded = kaleido(p, sides);
float angle = atan(folded.y, folded.x); float angle = atan(folded.y, folded.x);
@ -49,7 +47,13 @@ vec4 scene(vec2 uv, vec2 p) {
float ringLines = abs(fract(z * u_rings * 0.1) - 0.5) * 2.0; float ringLines = abs(fract(z * u_rings * 0.1) - 0.5) * 2.0;
float wallLines = abs(fract(wall * sides) - 0.5) * 2.0; float wallLines = abs(fract(wall * sides) - 0.5) * 2.0;
float grid = smoothstep(0.42, 0.0, ringLines) + smoothstep(0.42, 0.0, wallLines); // TRAIT style: the grid is the whole image, so the track draws it. The
// threshold was a bare 0.42 — a line weight with no name. u_sigLine sets how
// much of the cell the line occupies, u_sigSoft how far it feathers.
float weight = mix(0.22, 0.6, sat(u_sigLine));
float feather = weight * mix(0.15, 1.0, sat(u_sigSoft));
float grid = smoothstep(weight, max(weight - feather, 0.0), ringLines)
+ smoothstep(weight, max(weight - feather, 0.0), wallLines);
vec3 col = palRamp(z * 0.05 + wall * 0.2) * 0.35; vec3 col = palRamp(z * 0.05 + wall * 0.2) * 0.35;
col += pal(int(mod(floor(z * u_rings * 0.1), 6.0))) * grid * 0.7; col += pal(int(mod(floor(z * u_rings * 0.1), 6.0))) * grid * 0.7;
@ -57,10 +61,9 @@ vec4 scene(vec2 uv, vec2 p) {
// Depth cue: far end of the tunnel darkens, mouth glows. // Depth cue: far end of the tunnel darkens, mouth glows.
float fade = smoothstep(0.0, 1.1, radius); float fade = smoothstep(0.0, 1.1, radius);
col *= 0.25 + 0.9 * fade; col *= 0.25 + 0.9 * fade;
col += pal(3) * (1.0 - fade) * u_glow * 0.6; col += pal(3) * (1.0 - fade) * 0.5 * 0.6;
col += sigGrain(uv);
return vec4(col, 1.0); return vec4(inkValue(col), 1.0);
} }
`, `,
}; };

104
flow-state/src/scenes/shader/karman-street.js Normal file
View File

@ -0,0 +1,104 @@
// Flow family: a vortex street. One bluff body sits still in a moving stream and
// sheds vortices alternately off each shoulder; they drift downstream, growing
// and weakening, curling the streamlines as they pass.
//
// Every other flow scene moves the whole frame — Vortex Drift spins it, Curl
// Flow advects it, Aurora Veil sweeps it. This one has a fixed obstacle and a
// wake, so the image has an upstream and a downstream and a still point in it,
// and the motion is a consequence of the body rather than the state of the
// frame. The body is the track's signature form.
export const karmanStreet = {
name: 'Kármán Street',
family: 'flow',
kind: 'fragment',
texture: 0.7,
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'space', 'style'],
params: {
shed: { type: 'float', range: [0.03, 0.5], default: 0.12, uniform: 'u_shed', bias: 'motion', rate: true },
drift: { type: 'float', range: [0.15, 0.7], default: 0.42, uniform: 'u_drift' },
offset: { type: 'float', range: [0.05, 0.5], default: 0.22, uniform: 'u_offset' },
radius: { type: 'float', range: [0.06, 0.3], default: 0.15, uniform: 'u_radius' },
body: { type: 'float', range: [0.05, 0.28], default: 0.13, uniform: 'u_body' },
streams: { type: 'float', range: [4, 44], default: 16, uniform: 'u_streams', bias: 'density' },
warp: { type: 'float', range: [0, 0.5], default: 0.22, uniform: 'u_warp' },
wake: { type: 'float', range: [0, 1], default: 0.35, uniform: 'u_wake', slowAxis: true },
palette: { type: 'palette', count: 5 },
},
reactive: {
radius: { feature: 'bandLow', amount: 0.25, response: 'smooth' },
},
shader: `
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_shed + u_seed;
p = sigCamera(p);
// The stream runs along the track's horizon, so the wake sits in the same
// place the ground does in every other scene that has one.
float lane = sigHorizonY() * 0.35;
vec2 origin = vec2(-0.85, lane);
// Eight vortices in flight at once: one shed per unit of t, alternating
// sides, each one older and larger than the one behind it.
vec2 vel = vec2(0.0);
float vort = 0.0;
// lint: fixed-cost — eight vortices in flight, not a search
for (int i = 0; i < 8; i++) {
float age = fract(t) + float(i);
float idx = floor(t) - float(i);
float sgn = mod(idx, 2.0) < 0.5 ? 1.0 : -1.0;
vec2 c = origin + vec2(age * u_drift,
sgn * u_offset * (1.0 - exp(-age * 1.8)) + sin(age * 0.6 + idx) * 0.02);
float rad = u_radius * (0.55 + age * 0.16);
vec2 d = p - c;
float r2 = dot(d, d) + 1e-4;
float fall = exp(-r2 / (rad * rad));
// Induced velocity, bounded at the core so the centre does not blow up.
vel += sgn * vec2(-d.y, d.x) * (fall / (r2 + rad * rad)) * rad;
vort += sgn * fall * exp(-age * 0.25);
}
// Streamlines: a plain ruled field, dragged by the induced velocity. The
// curl in the image is entirely the vortices bending straight lines.
vec2 q = p - vel * u_warp;
float ruled = sin(q.y * u_streams) * 0.5 + 0.5;
float w = 0.25 + u_sigLine * 0.4;
float line = smoothstep(w, w * 0.2, abs(ruled - 0.5) * 2.0);
// Turbulent haze: everything downstream of the body is stirred, and the
// band of stirred water widens across the track. That is the scene's long
// journey — a clean stream at the top of the video and a churned one at the
// end — so the haze both lights the wake and dissolves the ruled lines
// inside it, rather than being a wash laid over an unchanged image.
float lanePos = (p.y - lane) / (0.1 + u_wake * 0.8);
float haze = exp(-lanePos * lanePos) * smoothstep(-0.15, 0.6, p.x - origin.x);
line *= 1.0 - 0.85 * haze * u_wake;
vec3 col = mix(pal(0) * 0.08, pal(1) * 0.14, uv.y);
col += palRamp(0.45 + q.y * 0.05) * line * (0.35 + 0.55 * 0.45);
col += pal(2) * haze * u_wake * 0.9;
col += pal(4) * haze * sat(abs(vort)) * u_wake * 0.5;
// The cores themselves, warm one way and cool the other.
col += pal(3) * sat(vort) * 0.55 * 0.5;
col += pal(2) * sat(-vort) * 0.55 * 0.5;
// The obstacle: still, solid, with the stream piling up on its nose.
float d = castMain((p - origin) / u_body) * u_body;
col = mix(col, pal(0) * 0.1, smoothstep(0.005, -0.005, d));
col += pal(4) * inkStroke(d) * (0.5 + 0.55 * 0.5);
col = sigAir(col, p, smoothstep(0.0, 1.6, length(p - origin) * 0.7));
return vec4(inkValue(col), 1.0);
}
`,
};
export default karmanStreet;

91
flow-state/src/scenes/shader/magnet-lines.js Normal file
View File

@ -0,0 +1,91 @@
// Flow family: the field lines of three drifting magnetic poles, drawn the way
// iron filings draw them — as closed loops leaving one pole and arriving at the
// next, thickening where the field is strong.
//
// Curl Flow and Smoke Column sample a noise field and let particles wander in
// it; nothing about the image is solved. Here the lines are the exact contours
// of the stream function of the poles, so they never cross, never fray, and
// reconfigure globally the instant a pole moves — a field with rules rather than
// a field with texture. The poles themselves are the track's signature form.
export const magnetLines = {
name: 'Magnet Lines',
family: 'flow',
kind: 'fragment',
// Line work. Grain furs it up, so take only a dusting.
texture: 0.5,
consumes: ['cast', 'ink'],
traits: ['shape', 'camera', 'style'],
params: {
lines: { type: 'int', range: [3, 26], default: 11, uniform: 'u_lines', bias: 'density' },
spread: { type: 'float', range: [0.15, 1.1], default: 0.55, uniform: 'u_spread', slowAxis: true },
orbit: { type: 'float', range: [0.01, 0.3], default: 0.05, uniform: 'u_orbit', bias: 'motion', rate: true },
core: { type: 'float', range: [0.02, 0.22], default: 0.09, uniform: 'u_core' },
filings: { type: 'float', range: [0, 1], default: 0.45, uniform: 'u_filings' },
palette: { type: 'palette', count: 5 },
},
reactive: {
core: { feature: 'beat', amount: 0.2, response: 'spike' },
},
shader: `
// Pole i, on a slow lissajous so the three never settle into a rotation.
vec2 poleAt(int i, float t) {
float fi = float(i);
float a = t + fi * 2.0944;
return vec2(cos(a), sin(a * 1.31 + fi)) * u_spread;
}
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_orbit + u_seed;
p = sigCamera(p);
// Stream function of the poles. Its contours ARE the field lines, which is
// why they close on themselves for free. Charges sum to +1 so the whole
// thing wraps once per turn at infinity, and an integer contour count keeps
// that wrap continuous instead of leaving a seam.
float psi = 0.0;
float energy = 0.0;
// lint: fixed-cost — three poles, not a search
for (int i = 0; i < 3; i++) {
vec2 d = p - poleAt(i, t);
float q = mod(float(i), 2.0) < 0.5 ? 1.0 : -1.0;
psi += q * atan(d.y, d.x);
energy += 1.0 / (dot(d, d) * 14.0 + 0.35);
}
float f = psi * float(u_lines) / 6.28318530718;
float band = abs(fract(f) - 0.5) * 2.0; // 0 on the line, 1 between lines
// Lines are thin where the field is weak and thick where it crowds, which
// is the whole visual signature of filings.
float w = (0.18 + u_sigLine * 0.5) * (0.35 + sat(energy) * 0.9);
float line = smoothstep(w, w * 0.15, band);
// Filings: the line is not solid, it is a queue of grains along it.
float grainAlong = vnoise(vec2(f * 40.0, psi * 3.0 + t));
line *= mix(1.0, smoothstep(0.25, 0.75, grainAlong), u_filings);
vec3 col = pal(0) * 0.06;
col += palRamp(0.15 + fract(f) * 0.15 + energy * 0.1) * line * (0.6 + 0.6 * 0.7);
col += pal(2) * sat(energy) * 0.12 * 0.6;
// The poles: the track's form, lit from inside, one hot and one cold.
// lint: fixed-cost — the same three poles
for (int i = 0; i < 3; i++) {
vec2 c = poleAt(i, t);
float d = castMain((p - c) / u_core) * u_core;
vec3 tint = mod(float(i), 2.0) < 0.5 ? pal(3) : pal(4);
col = mix(col, tint * 0.25, smoothstep(0.004, -0.004, d));
col += tint * inkStroke(d) * (0.5 + 0.6 * 0.6);
}
col *= 0.7 + 0.3 * exp(-dot(p, p) * 0.3);
return vec4(inkValue(col), 1.0);
}
`,
};
export default magnetLines;

24
flow-state/src/scenes/shader/metaballs.js
View File

@ -8,6 +8,7 @@ export const metaballs = {
family: 'organic', family: 'organic',
kind: 'fragment', kind: 'fragment',
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
consumes: ['cast', 'ink', 'staging'],
traits: ['shape', 'camera', 'style'], traits: ['shape', 'camera', 'style'],
params: { params: {
@ -39,16 +40,18 @@ vec4 scene(vec2 uv, vec2 p) {
float fi = float(i); float fi = float(i);
float s = u_seed + fi * 71.3; float s = u_seed + fi * 71.3;
vec2 centre = vec2( // Homes on the song's lattice, orbited by this scene's own motion.
sin(t * (0.7 + fract(s * 0.13)) + s) * u_spread, vec3 node = stageNode(fi, float(u_count));
cos(t * (0.5 + fract(s * 0.29)) + s * 1.7) * u_spread * 0.62 vec2 centre = node.xy * u_spread * vec2(1.0, 0.62)
); + vec2(sin(t * (0.7 + fract(s * 0.13)) + s),
cos(t * (0.5 + fract(s * 0.29)) + s * 1.7)) * 0.25;
// Distance measured in the track's form rather than as a circle: for a // Distance measured in the song's FORM rather than as a circle: for a
// round personality this is exactly length(p - centre), and for a // round cast this is exactly length(p - centre), and for a hexagonal
// hexagonal one the blobs merge as hexagons. // one the blobs merge as hexagons.
float d = sigShape((p - centre) / max(u_radius, 1e-3)) * u_radius + u_radius; float radius = u_radius * node.z;
float contribution = (u_radius * u_radius) / max(d * d, 1e-4); float d = castMain((p - centre) / max(radius, 1e-3)) * radius + radius;
float contribution = (radius * radius) / max(d * d, 1e-4);
field += contribution; field += contribution;
tint += pal(i) * contribution; tint += pal(i) * contribution;
} }
@ -62,9 +65,8 @@ vec4 scene(vec2 uv, vec2 p) {
vec3 col = pal(0) * 0.06; vec3 col = pal(0) * 0.06;
col = mix(col, tint, surface); col = mix(col, tint, surface);
col += tint * rim * u_rim; col += tint * rim * u_rim;
col += sigGrain(uv);
return vec4(col, 1.0); return vec4(inkValue(col), 1.0);
} }
`, `,
}; };

48
flow-state/src/scenes/shader/moire-grid.js
View File

@ -12,9 +12,16 @@ export const moireGrid = {
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
// Crisp line work: the track's surface grain would only fur the edges. // Crisp line work: the track's surface grain would only fur the edges.
texture: 0, texture: 0,
consumes: ['ink'],
traits: ['camera', 'style'], traits: ['camera', 'style'],
params: { params: {
// The subject: the song's protagonist standing in the field, with the
// interference running at a different phase inside it. A window rather
// than a shape laid on top — the pattern is still the picture, it just
// has somewhere to be about, and the edge gives the eye a line to track.
subject: { type: 'float', range: [0, 1], default: 0.7, uniform: 'u_subject' },
subjSize: { type: 'float', range: [0.2, 0.75], default: 0.42, uniform: 'u_subjSize' },
density: { type: 'float', range: [6, 60], default: 22, uniform: 'u_density', bias: 'density' }, density: { type: 'float', range: [6, 60], default: 22, uniform: 'u_density', bias: 'density' },
offset: { type: 'float', range: [0.0, 0.5], default: 0.08, uniform: 'u_offset' }, offset: { type: 'float', range: [0.0, 0.5], default: 0.08, uniform: 'u_offset' },
rotate: { type: 'float', range: [0, 0.25], default: 0.04, uniform: 'u_rotate', bias: 'motion', rate: true }, rotate: { type: 'float', range: [0, 0.25], default: 0.04, uniform: 'u_rotate', bias: 'motion', rate: true },
@ -23,13 +30,11 @@ export const moireGrid = {
// redefinition error that renders the scene as a black frame. // redefinition error that renders the scene as a black frame.
width: { type: 'float', range: [0.06, 0.5], default: 0.2, uniform: 'u_lineWidth' , slowAxis: true }, width: { type: 'float', range: [0.06, 0.5], default: 0.2, uniform: 'u_lineWidth' , slowAxis: true },
warp: { type: 'float', range: [0, 1], default: 0.25, uniform: 'u_warp' }, warp: { type: 'float', range: [0, 1], default: 0.25, uniform: 'u_warp' },
glow: { type: 'float', range: [0, 1.2], default: 0.35, uniform: 'u_glow', bias: 'energy' },
palette: { type: 'palette', count: 4 }, palette: { type: 'palette', count: 4 },
}, },
reactive: { reactive: {
offset: { feature: 'bandLow', amount: 0.35 }, offset: { feature: 'bandLow', amount: 0.35 },
glow: { feature: 'beat', amount: 0.35, response: 'spike' },
warp: { feature: 'flux', amount: 0.2, response: 'smooth' }, warp: { feature: 'flux', amount: 0.2, response: 'smooth' },
}, },
@ -56,18 +61,49 @@ vec4 scene(vec2 uv, vec2 p) {
float a = grid(rot(t * 6.28318530718) * q, u_density, weight); float a = grid(rot(t * 6.28318530718) * q, u_density, weight);
float b = grid(rot(-t * 6.28318530718 + u_offset * 3.14159) * (q + u_offset), u_density, weight); float b = grid(rot(-t * 6.28318530718 + u_offset * 3.14159) * (q + u_offset), u_density, weight);
// The subject does something TO the field, and which thing is the song's
// decision rather than this scene's — the same form punching a hole,
// bending the grid or running it at another rate are three different
// videos. See Identity.IMPACTS.
float d = subjectSDF(p, u_subjSize);
float inside = smoothstep(0.01, -0.01, d) * u_subject;
// WARP bends the pattern around the form instead of replacing it inside,
// so it is applied to the coordinate everything below is drawn from.
if (impactIs(1.0)) q = subjectWarp(q, u_subjSize, u_subject * 1.2);
vec2 qi = q + vec2(u_offset * 0.7, -u_offset * 0.4);
float ai = grid(rot(t * 6.28318530718 * 1.35) * qi, u_density * 1.18, weight);
float bi = grid(rot(-t * 6.28318530718 * 0.7 + u_offset * 3.14159) * (qi + u_offset), u_density * 0.86, weight);
// The interference term is the point: where both grids land, it peaks. // The interference term is the point: where both grids land, it peaks.
float interference = a * b; float interference = a * b;
float either = max(a, b); float either = max(a, b);
if (impactIs(0.0)) { // shift: a different beat inside
interference = mix(interference, ai * bi, inside);
either = mix(either, max(ai, bi), inside);
} else if (impactIs(2.0)) { // punch: the field stops at the form
interference *= 1.0 - inside;
either *= 1.0 - inside;
} else if (impactIs(3.0)) { // morph: one grid survives inside
interference = mix(interference, a * 0.35, inside);
either = mix(either, a, inside);
} else if (impactIs(4.0)) { // overlay: the form reads as solid
either = max(either, inside);
interference = max(interference, inside * 0.6);
}
vec3 col = pal(0) * 0.05; vec3 col = pal(0) * 0.05;
col += pal(1) * either * 0.35; col += pal(1) * either * 0.35;
col += pal(2) * interference * (0.8 + u_glow); col += pal(2) * interference * (0.8 + 0.35);
col += pal(3) * pow(interference, 3.0) * u_glow;
// The rim, so the form has an edge to follow rather than only a change of
// texture. Drawn in the song's hand like everything else.
col = mix(col, pal(3), inkStroke(d) * u_subject);
col *= 0.6 + 0.4 * exp(-dot(p, p) * 0.3); col *= 0.6 + 0.4 * exp(-dot(p, p) * 0.3);
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

142
flow-state/src/scenes/shader/mountain-flight.js Normal file
View File

@ -0,0 +1,142 @@
// Structural family: a flight through mountains. A real perspective camera
// travelling forward over a raymarched heightfield — peaks rise, pass to either
// side and occlude what is behind them, with a snowline near the top and the
// ridge silhouettes drawn in the song's hand.
//
// Ridge Terrain is the neighbour, and it is the flat version of this: parallax
// bands of 1D noise that slide sideways and can never hide each other, because
// there is no depth for them to hide in. Here the image is a 3D field sampled
// along the ray, so the motion is INTO the frame rather than across it, valleys
// open and close as the camera passes through them, and a near peak eats a far
// one. The massifs themselves are the song's protagonist, extruded: a hollow
// cast makes calderas, a notched one makes star-shaped ridges.
export const mountainFlight = {
name: 'Mountain Flight',
family: 'structural',
kind: 'fragment',
// Personality: see look/Personality.js. Identity: see look/Identity.js.
consumes: ['cast', 'ink', 'staging'],
traits: ['camera', 'space', 'style'],
params: {
relief: { type: 'float', range: [0.15, 1.4], default: 0.55, uniform: 'u_relief', bias: 'energy' },
rugged: { type: 'float', range: [0.15, 1.1], default: 0.45, uniform: 'u_rug', bias: 'density' },
ridges: { type: 'int', range: [2, 5], default: 4, uniform: 'u_oct', bias: 'density' },
massif: { type: 'float', range: [0.1, 0.9], default: 0.5, uniform: 'u_massif' },
span: { type: 'float', range: [4, 16], default: 8.0, uniform: 'u_span', bias: 'density' },
altitude: { type: 'float', range: [0.15, 2.2], default: 0.7, uniform: 'u_alt', slowAxis: true },
snowline: { type: 'float', range: [0.15, 1.1], default: 0.55, uniform: 'u_snow', bias: 'energy' },
speed: { type: 'float', range: [0.1, 2.5], default: 0.8, uniform: 'u_speed', bias: 'motion', rate: true },
palette: { type: 'palette', count: 6 },
},
reactive: {
relief: { feature: 'bandLow', amount: 0.22, response: 'smooth' },
snowline: { feature: 'bandHigh', amount: 0.2, response: 'smooth' },
altitude: { feature: 'bandSub', amount: 0.15, response: 'smooth' },
},
shader: `
// The massif standing in the world cell around w: the song's protagonist, used
// as a PLAN rather than a picture — its footprint is extruded into a mountain.
// Kept strictly inside its own cell so one sample of one cell is the whole
// answer, which is what makes it affordable inside the march.
float massifH(vec2 w) {
vec2 cell = floor(w / u_span);
vec2 local = w - (cell + 0.5) * u_span;
// Which node of the song's lattice this cell got, and how big it stands.
vec3 node = stageNode(floor(hash12(cell) * 7.0), 8.0);
vec2 at = clamp(node.xy, -1.0, 1.0) * u_span * 0.18;
float s = u_span * 0.2 * clamp(u_massif * node.z, 0.08, 1.1);
float d = castMain((local - at) / s) * s;
// Flanks fall away from the footprint's edge; the summit is its middle.
float m = sat(-d / max(s * 0.9, 1e-3));
return pow(m, 0.7) * u_relief * 1.7;
}
float terrainH(vec2 w) {
// The song's element size is the size of the whole landscape's vocabulary:
// a few enormous massifs, or a crowd of small ones.
vec2 q = w * u_rug / max(stageScale(), 0.25);
float sum = 0.0, norm = 0.0, amp = 1.0;
for (int i = 0; i < 5; i++) {
if (i >= u_oct) break;
// Ridged, not billowy: mountains have crests.
float n = 1.0 - abs(vnoise(q) * 2.0 - 1.0);
sum += n * n * amp;
norm += amp;
q = rot(0.63) * q * 2.03;
amp *= 0.45;
}
return (sum / max(norm, 1e-3)) * u_relief + massifH(w);
}
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_speed + u_seed;
p = sigCamera(p);
// The camera rides a fixed height over whatever it is flying across, so a
// tall massif lifts the flight path instead of swallowing it.
vec3 ro = vec3(u_seed * 3.7, 0.0, t * 4.0);
ro.y = terrainH(ro.xz) + u_alt * (1.0 + stageScale() * 0.3);
// The track's horizon decides how far down the camera is looking.
vec3 rd = normalize(vec3(p.x, p.y - sigHorizonY() * 0.5, 1.6));
float dist = 0.4;
float hit = 0.0;
float gap = 0.0;
float grazed = 1e3; // closest the ray came, per unit distance
// A march ends on its hit or its far plane, not on a param.
// lint: fixed-cost
for (int i = 0; i < 72; i++) {
vec3 at = ro + rd * dist;
gap = at.y - terrainH(at.xz);
grazed = min(grazed, gap / max(dist, 1.0));
if (gap < 0.002 * dist) { hit = 1.0; break; }
// Cone stepping: safe near the surface, cheap out at the far end.
dist += max(gap * 0.5, 0.03 + dist * 0.02);
if (dist > 46.0) break;
}
float fog = sat(dist / 46.0);
vec3 col;
if (hit > 0.5) {
vec3 at = ro + rd * dist;
float e = 0.02 + dist * 0.006;
vec3 nrm = normalize(vec3(
terrainH(at.xz - vec2(e, 0.0)) - terrainH(at.xz + vec2(e, 0.0)),
2.0 * e,
terrainH(at.xz - vec2(0.0, e)) - terrainH(at.xz + vec2(0.0, e))));
vec3 sun = normalize(vec3(0.55, 0.42, -0.5));
float diff = sat(dot(nrm, sun));
float elev = sat(at.y / max(u_relief * 2.2, 0.2));
// Snow lies high, and only where the slope will hold it.
float snow = smoothstep(u_snow, u_snow + 0.16, elev * (0.45 + 0.75 * nrm.y));
vec3 rock = mix(pal(2), pal(3), sat(elev * 1.3));
col = mix(rock, pal(5), snow);
col *= 0.22 + 0.9 * diff;
col += pal(1) * 0.12 * sat(nrm.y); // sky bounce into the flats
} else {
col = mix(pal(1) * 0.45, pal(0) * 0.22, sat(rd.y * 2.2 + 0.15));
fog = 0.85;
// The ridgeline the sky is cut against, drawn at the song's weight.
col += pal(4) * inkStroke(grazed * 0.6) * 0.5;
}
col = sigAir(col, p, fog);
return vec4(inkValue(col), 1.0);
}
`,
};
export default mountainFlight;

99
flow-state/src/scenes/shader/mycelium-web.js Normal file
View File

@ -0,0 +1,99 @@
// Organic family: a fungal mat seen in perspective, creeping toward the camera.
// Hyphae branch across the substrate, brighten where two threads cross, and the
// colony spreads outward from where it started.
//
// Flora grows upright, symmetric and toward the light; every organic scene in
// the library is a thing standing in a space. This one is a thing lying flat in
// one — a mat on the ground, receding to the track's horizon — so it reads as
// surface rather than as subject, and the branching is lateral and off-centre
// instead of radial.
export const myceliumWeb = {
name: 'Mycelium Web',
family: 'organic',
kind: 'fragment',
consumes: ['ink'],
traits: ['camera', 'space', 'style'],
params: {
threads: { type: 'float', range: [1.5, 9], default: 3.5, uniform: 'u_threads', bias: 'density' },
branch: { type: 'float', range: [0, 1.4], default: 0.6, uniform: 'u_branch' },
fine: { type: 'float', range: [0.02, 0.35], default: 0.18, uniform: 'u_fine' },
front: { type: 'float', range: [0.4, 6], default: 1.6, uniform: 'u_front', slowAxis: true },
nodes: { type: 'float', range: [0, 1.4], default: 0.6, uniform: 'u_nodes', bias: 'energy' },
creep: { type: 'float', range: [0.01, 0.35], default: 0.07, uniform: 'u_creep', bias: 'motion', rate: true },
damp: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_damp' },
palette: { type: 'palette', count: 5 },
},
reactive: {
nodes: { feature: 'bandHigh', amount: 0.35, response: 'smooth' },
branch: { feature: 'bandMid', amount: 0.25, response: 'smooth' },
damp: { feature: 'loudness', amount: 0.2, response: 'smooth' },
},
shader: `
// Hyphae: ridged noise warped by itself, so threads run in bundles and fork
// rather than lying in a regular weave.
float hyphae(vec2 g, float scale, float warp) {
vec2 w = vec2(fbm(g * scale * 0.7 + 3.1, 3), fbm(g * scale * 0.7 - 7.4, 3)) - 0.5;
float n = vnoise(g * scale + w * warp * 3.0);
return 1.0 - abs(n * 2.0 - 1.0);
}
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_creep + u_seed;
p = sigCamera(p);
float horizon = sigHorizonY() * 0.45 + 0.25;
float below = horizon - p.y;
// Air above the mat: nothing but the track's wash.
vec3 col = mix(pal(1) * 0.16, pal(0) * 0.06, sat((p.y - horizon) * 1.2 + 0.2));
if (below > 0.001) {
// Ground plane. Everything from here on is drawn in substrate
// coordinates, so the mat lies flat and the weave compresses with
// distance instead of being a flat texture pinned to the frame.
float depth = 1.0 / max(below, 0.02);
vec2 g = vec2(p.x * depth, depth) * 0.6;
g.y -= t * 2.0; // the colony creeps forward
float coarse = hyphae(g, u_threads, u_branch);
float fine = hyphae(g * 2.7 + 11.0, u_threads, u_branch * 0.6);
float w = u_fine * (0.4 + u_sigLine * 1.6);
float thread = smoothstep(w, w * 0.15, 1.0 - coarse);
thread += smoothstep(w * 0.6, 0.0, 1.0 - fine) * 0.5;
// The colony has a front: it is dense where it started and thins out at
// the edge of where it has reached.
float reach = smoothstep(u_front, u_front * 0.25, length(vec2(g.x, g.y + t * 2.0)) * 0.5);
thread *= mix(0.04, 1.0, reach);
// Anastomosis — where two hyphae meet and fuse. Those junctions are the
// only bright points in the mat, so the eye reads it as a network.
float node = sat(coarse * fine * 1.8 - 0.55) * reach;
vec3 mat = mix(pal(0) * 0.12, pal(2) * 0.95, sat(thread));
mat += pal(3) * sat(thread) * 0.22;
mat += palRamp(0.6 + node) * node * u_nodes * 1.6;
mat += pal(4) * sigEdge(1.0 - coarse - w) * u_nodes * 0.2;
// Wet substrate underneath, so the mat is not floating on black.
mat += pal(1) * 0.1 * (1.0 - sat(thread)) * u_damp * smoothstep(0.0, 0.6, below);
// Outside the front there is substrate and nothing else, so how far the
// colony has spread is legible as area rather than as brightness.
mat *= mix(0.3, 1.0, reach);
col = mix(col, mat, smoothstep(0.0, 0.04, below));
}
col = sigAir(col, p, sat(1.0 - below * 1.1));
return vec4(inkValue(col), 1.0);
}
`,
};
export default myceliumWeb;

8
flow-state/src/scenes/shader/nebula.js
View File

@ -9,20 +9,19 @@ export const nebula = {
family: 'organic', family: 'organic',
kind: 'fragment', kind: 'fragment',
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
consumes: ['ink'],
traits: ['camera', 'space', 'style'], traits: ['camera', 'space', 'style'],
params: { params: {
scale: { type: 'float', range: [4, 24], default: 12, uniform: 'u_scale', bias: 'density' }, scale: { type: 'float', range: [4, 24], default: 12, uniform: 'u_scale', bias: 'density' },
swirl: { type: 'float', range: [0, 3], default: 1.0, uniform: 'u_swirl', bias: 'motion' }, swirl: { type: 'float', range: [0, 3], default: 1.0, uniform: 'u_swirl', bias: 'motion' },
rings: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_rings' }, rings: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_rings' },
glow: { type: 'float', range: [0, 1.5], default: 0.5, uniform: 'u_glow', bias: 'energy' },
speed: { type: 'float', range: [0.1, 1.5],default: 0.5, uniform: 'u_speed', bias: 'motion', rate: true }, speed: { type: 'float', range: [0.1, 1.5],default: 0.5, uniform: 'u_speed', bias: 'motion', rate: true },
depth: { type: 'float', range: [0, 1], default: 0.4, uniform: 'u_depth' }, depth: { type: 'float', range: [0, 1], default: 0.4, uniform: 'u_depth' },
palette: { type: 'palette', count: 4 }, palette: { type: 'palette', count: 4 },
}, },
reactive: { reactive: {
glow: { feature: 'beat', amount: 0.35, response: 'spike' },
swirl: { feature: 'bandLow', amount: 0.20 }, swirl: { feature: 'bandLow', amount: 0.20 },
rings: { feature: 'flux', amount: 0.25, response: 'smooth' }, rings: { feature: 'flux', amount: 0.25, response: 'smooth' },
}, },
@ -50,16 +49,15 @@ vec4 scene(vec2 uv, vec2 p) {
// Core glow. // Core glow.
float edge = 1.0 - smoothstep(0.1, 0.9, r); float edge = 1.0 - smoothstep(0.1, 0.9, r);
col += pal(3) * edge * u_glow * 0.5; col += pal(3) * edge * 0.5 * 0.5;
// Vignette the far field so the frame has a subject. // Vignette the far field so the frame has a subject.
col *= 0.5 + 0.5 * (1.0 - smoothstep(0.6, 1.6, r)); col *= 0.5 + 0.5 * (1.0 - smoothstep(0.6, 1.6, r));
// The location's air, and its surface. // The location's air, and its surface.
col = sigAir(col, p, smoothstep(0.0, 1.6, r)); col = sigAir(col, p, smoothstep(0.0, 1.6, r));
col += sigGrain(uv);
return vec4(col, 1.0); return vec4(inkValue(col), 1.0);
} }
`, `,
}; };

9
flow-state/src/scenes/shader/neon-city.js
View File

@ -12,6 +12,7 @@ export const neonCity = {
family: 'structural', family: 'structural',
kind: 'fragment', kind: 'fragment',
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
consumes: ['ink'],
traits: ['space', 'camera', 'style'], traits: ['space', 'camera', 'style'],
params: { params: {
@ -20,10 +21,8 @@ export const neonCity = {
height: { type: 'float', range: [0.2, 2.5], default: 1.0, uniform: 'u_height', bias: 'energy' }, height: { type: 'float', range: [0.2, 2.5], default: 1.0, uniform: 'u_height', bias: 'energy' },
speed: { type: 'float', range: [0.02, 0.4], default: 0.08, uniform: 'u_speed', bias: 'motion', rate: true }, speed: { type: 'float', range: [0.02, 0.4], default: 0.08, uniform: 'u_speed', bias: 'motion', rate: true },
horizon: { type: 'float', range: [-0.3, 0.3], default: 0.0, uniform: 'u_horizon' }, horizon: { type: 'float', range: [-0.3, 0.3], default: 0.0, uniform: 'u_horizon' },
haze: { type: 'float', range: [0, 1], default: 0.45, uniform: 'u_haze' },
reflect: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_reflect' }, reflect: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_reflect' },
pulse: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_pulse' }, pulse: { type: 'float', range: [0, 1], default: 0.5, uniform: 'u_pulse' },
glow: { type: 'float', range: [0, 1.5], default: 0.4, uniform: 'u_glow', bias: 'energy' },
palette: { type: 'palette', count: 4 }, palette: { type: 'palette', count: 4 },
}, },
@ -99,7 +98,7 @@ vec4 scene(vec2 uv, vec2 p) {
// Ground-couple: darker, breath where the tower meets the street. // Ground-couple: darker, breath where the tower meets the street.
fut *= 0.7 + 0.3 * smoothstep(0.0, 0.5, gy); fut *= 0.7 + 0.3 * smoothstep(0.0, 0.5, gy);
// Haze pushes the far rings into the sky, the way distance actually does. // Haze pushes the far rings into the sky, the way distance actually does.
col = mix(col, fut, tower * (1.0 - u_haze * (1.0 - depth))); col = mix(col, fut, tower * (1.0 - 0.45 * (1.0 - depth)));
} }
// Street: a slate floor with a molten reflection of the nearest towers. // Street: a slate floor with a molten reflection of the nearest towers.
@ -111,11 +110,9 @@ vec4 scene(vec2 uv, vec2 p) {
} }
// Horizon haze bloom, the city's glow pooling where towers meet the sky. // Horizon haze bloom, the city's glow pooling where towers meet the sky.
col += pal(2) * exp(-abs(p.y - horizon) * 9.0) * u_glow;
col = sigAir(col, p, smoothstep(0.0, 1.4, length(p))); col = sigAir(col, p, smoothstep(0.0, 1.4, length(p)));
col += sigGrain(uv);
return vec4(col, 1.0); return vec4(inkValue(col), 1.0);
} }
`, `,
}; };

91
flow-state/src/scenes/shader/pendulum-trace.js Normal file
View File

@ -0,0 +1,91 @@
// Minimal family: one hairline drawing a harmonograph figure — two pendulums
// per axis, slightly out of tune with each other — over its own slowly fading
// ghost. Nothing else is in the frame.
//
// Silk Ribbon is the other minimal with a single moving element, and it is a
// wide band: an area with two edges. This is a line with no width to speak of,
// and it is the only scene in the library where what you are looking at is the
// HISTORY of a moving point rather than the point. The figure never repeats
// exactly, because the two frequencies are deliberately not a whole ratio, so
// the drawing precesses instead of closing.
export const pendulumTrace = {
name: 'Pendulum Trace',
family: 'minimal',
kind: 'fragment',
// Fine line work; grain only furs it up.
texture: 0.25,
consumes: ['ink'],
traits: ['camera', 'style'],
params: {
ratio: { type: 'float', range: [1.5, 5.5], default: 3.01, uniform: 'u_ratio' },
detune: { type: 'float', range: [0, 0.08], default: 0.03, uniform: 'u_detune' },
size: { type: 'float', range: [0.5, 1.5], default: 1.05, uniform: 'u_size' },
decay: { type: 'float', range: [0, 0.5], default: 0.15, uniform: 'u_decay' },
weight: { type: 'float', range: [0.002, 0.02], default: 0.006, uniform: 'u_weight' },
arc: { type: 'float', range: [1.0, 8], default: 3.5, uniform: 'u_arc', bias: 'density' },
persist: { type: 'float', range: [0.9, 0.996], default: 0.992, uniform: 'u_persist', slowAxis: true },
speed: { type: 'float', range: [0.05, 0.9], default: 0.3, uniform: 'u_speed', bias: 'motion', rate: true },
palette: { type: 'palette', count: 5 },
},
reactive: {
weight: { feature: 'beat', amount: 0.15, response: 'spike' },
arc: { feature: 'bandMid', amount: 0.2, response: 'smooth' },
},
shader: `
// Where the pen is at time s. Two pendulums per axis, the second slightly
// detuned from a whole-number ratio — which is the entire reason a harmonograph
// figure precesses instead of retracing one closed loop forever.
vec2 pen(float s) {
// The swing loses energy and is pushed again on the phrase. A real
// exponential decay in absolute time would be correct and useless: five
// minutes in, the figure would have shrunk to a dot.
float damp = 1.0 - u_decay * (1.0 - cos(u_phrasePhase * 6.28318530718)) * 0.5;
float x = sin(s) + 0.6 * sin(s * u_ratio + 1.7);
float y = cos(s * (1.0 + u_detune)) + 0.6 * cos(s * u_ratio * (1.0 - u_detune) + 0.4);
return vec2(x, y) * 0.5 * u_size * damp;
}
vec4 scene(vec2 uv, vec2 p) {
float t = u_time * u_speed + u_seed;
p = sigCamera(p);
vec3 col = pal(0) * 0.07;
// Vignette wash, so the empty frame is a lit space rather than black.
col += pal(1) * 0.13 * exp(-dot(p, p) * 0.6);
// The live stroke: the last u_arc radians of travel, sampled at a fixed
// resolution and reduced to the nearest point on the curve.
float best = 1e9;
float head = 0.0;
// lint: fixed-cost — 48 samples along a fixed arc, not a search
for (int i = 0; i < 48; i++) {
float f = float(i) / 47.0;
float s = t * 20.0 - u_arc * (1.0 - f);
float d = length(p - pen(s));
if (d < best) { best = d; head = f; }
}
float w = u_weight * (0.4 + u_sigLine * 2.5);
float line = smoothstep(w, w * 0.2, best);
// The tip is brightest and the tail falls away, so the line has a direction
// of travel even in a still frame.
vec3 ink = palRamp(0.2 + head * 0.35);
col += ink * line * (0.5 + head * 0.9) * (0.6 + 0.6 * 0.8);
// The ghost: everything the pen has already drawn, fading. Sampled straight
// rather than smeared, so old strokes stay in place and thin out instead of
// sliding around the frame.
col = max(col, prev(uv) * u_persist);
return vec4(inkValue(col), 1.0);
}
`,
};
export default pendulumTrace;

13
flow-state/src/scenes/shader/pitch-shatter.js
View File

@ -10,6 +10,7 @@ export const pitchShatter = {
family: 'glitch', family: 'glitch',
kind: 'fragment', kind: 'fragment',
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
consumes: ['ink'],
traits: ['camera', 'style'], traits: ['camera', 'style'],
params: { params: {
@ -17,7 +18,6 @@ export const pitchShatter = {
amp: { type: 'float', range: [0, 0.35], default: 0.1, uniform: 'u_amp', bias: 'energy' }, amp: { type: 'float', range: [0, 0.35], default: 0.1, uniform: 'u_amp', bias: 'energy' },
quantize: { type: 'float', range: [1, 8], default: 4, uniform: 'u_quantize' }, quantize: { type: 'float', range: [1, 8], default: 4, uniform: 'u_quantize' },
bleed: { type: 'float', range: [0, 0.9], default: 0.4, uniform: 'u_bleed' }, bleed: { type: 'float', range: [0, 0.9], default: 0.4, uniform: 'u_bleed' },
glow: { type: 'float', range: [0, 1], default: 0.3, uniform: 'u_glow', bias: 'energy' },
speed: { type: 'float', range: [0.05, 0.8], default: 0.25, uniform: 'u_speed', bias: 'motion', rate: true }, speed: { type: 'float', range: [0.05, 0.8], default: 0.25, uniform: 'u_speed', bias: 'motion', rate: true },
palette: { type: 'palette', count: 5 }, palette: { type: 'palette', count: 5 },
}, },
@ -25,7 +25,6 @@ export const pitchShatter = {
reactive: { reactive: {
amp: { feature: 'beat', amount: 0.4, response: 'spike' }, amp: { feature: 'beat', amount: 0.4, response: 'spike' },
bleed: { feature: 'flux', amount: 0.25, response: 'smooth' }, bleed: { feature: 'flux', amount: 0.25, response: 'smooth' },
glow: { feature: 'bandHigh', amount: 0.2 },
}, },
shader: ` shader: `
@ -46,7 +45,10 @@ vec4 scene(vec2 uv, vec2 p) {
// The base image is always sampled from the un-pitched field so the scene // The base image is always sampled from the un-pitched field so the scene
// has real content behind the displacement, even on its first frame. // has real content behind the displacement, even on its first frame.
vec2 baseUv = vec2(uv.x, fract(uv.y - pitch)); // Slices are screen space; the field they transpose is filmed at the
// shot's framing. See framedUv.
vec2 fuv = framedUv(p);
vec2 baseUv = vec2(fuv.x, fract(fuv.y - pitch));
float field = fbm(vec2(baseUv.x * 2.5, baseUv.y * 4.0) + t * 0.4, 4); float field = fbm(vec2(baseUv.x * 2.5, baseUv.y * 4.0) + t * 0.4, 4);
float ramp = fract(field * 2.0 + baseUv.y * 2.0 - t * 0.4); float ramp = fract(field * 2.0 + baseUv.y * 2.0 - t * 0.4);
vec3 col = palRamp(ramp * 0.7 + slice * 0.02); vec3 col = palRamp(ramp * 0.7 + slice * 0.02);
@ -63,9 +65,8 @@ vec4 scene(vec2 uv, vec2 p) {
float beam = exp(-abs(p.y - beamY * 0.6) * 6.0); float beam = exp(-abs(p.y - beamY * 0.6) * 6.0);
col += pal(3) * beam * u_slices * 0.001 * (0.5 + u_beat); col += pal(3) * beam * u_slices * 0.001 * (0.5 + u_beat);
col += pal(int(mod(slice, 5.0))) * (1.0 - sliceRand) * u_glow * 0.3 * sliceRand; col += pal(int(mod(slice, 5.0))) * (1.0 - sliceRand) * 0.3 * 0.3 * sliceRand;
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

7
flow-state/src/scenes/shader/plasma-bloom.js
View File

@ -7,6 +7,7 @@ export const plasmaBloom = {
family: 'organic', family: 'organic',
kind: 'fragment', kind: 'fragment',
// Personality: see look/Personality.js. // Personality: see look/Personality.js.
consumes: ['ink'],
traits: ['camera', 'space', 'style'], traits: ['camera', 'space', 'style'],
params: { params: {
@ -15,13 +16,11 @@ export const plasmaBloom = {
speed: { type: 'float', range: [0.02, 0.5], default: 0.1, uniform: 'u_speed', bias: 'motion', rate: true }, speed: { type: 'float', range: [0.02, 0.5], default: 0.1, uniform: 'u_speed', bias: 'motion', rate: true },
bands: { type: 'float', range: [1, 10], default: 3.5, uniform: 'u_bands' }, bands: { type: 'float', range: [1, 10], default: 3.5, uniform: 'u_bands' },
softness:{ type: 'float', range: [0, 1], default: 0.5, uniform: 'u_softness' }, softness:{ type: 'float', range: [0, 1], default: 0.5, uniform: 'u_softness' },
glow: { type: 'float', range: [0, 1.5], default: 0.4, uniform: 'u_glow', bias: 'energy' },
palette: { type: 'palette', count: 6 }, palette: { type: 'palette', count: 6 },
}, },
reactive: { reactive: {
warp: { feature: 'bandLow', amount: 0.35 }, warp: { feature: 'bandLow', amount: 0.35 },
glow: { feature: 'beat', amount: 0.35, response: 'spike' },
bands: { feature: 'centroid', amount: 0.2, response: 'smooth' }, bands: { feature: 'centroid', amount: 0.2, response: 'smooth' },
}, },
@ -41,14 +40,12 @@ vec4 scene(vec2 uv, vec2 p) {
vec3 col = palRamp(shaped * 0.6 + length(r) * 0.25); vec3 col = palRamp(shaped * 0.6 + length(r) * 0.25);
col *= 0.35 + 0.75 * shaped; col *= 0.35 + 0.75 * shaped;
col += pal(4) * pow(shaped, 5.0) * u_glow;
// Dark corners so the bloom has somewhere to sit. // Dark corners so the bloom has somewhere to sit.
col *= 0.55 + 0.45 * exp(-dot(p, p) * 0.4); col *= 0.55 + 0.45 * exp(-dot(p, p) * 0.4);
col = sigAir(col, p, smoothstep(0.0, 1.8, length(p))); col = sigAir(col, p, smoothstep(0.0, 1.8, length(p)));
col += sigGrain(uv); return vec4(inkValue(col), 1.0);
return vec4(col, 1.0);
} }
`, `,
}; };

Some files were not shown because too many files have changed in this diff Show More