diff --git a/SEASHELLS_README.md b/SEASHELLS_README.md new file mode 100644 index 0000000000..5501d0ad7f --- /dev/null +++ b/SEASHELLS_README.md @@ -0,0 +1,359 @@ +# Seashells.mjs: Complete Documentation + +A comprehensive guide to understanding, using, and remixing **seashells.mjs** — a bytebeat algorithmic synthesizer with visual feedback. + +--- + +## What Is Seashells? + +**Seashells** is an interactive generative music instrument that: + +1. **Synthesizes audio** using bytebeat (mathematical bit operations) +2. **Visualizes in real-time** by painting bytebeat patterns to the screen +3. **Feeds back visuals → audio** by sampling pixels and using them to modulate synthesis +4. **Sustains via hold sequences** — can play autonomously for 90+ minutes + +It's designed as a tape-like generative system: set it running and watch/listen to emergence unfold. + +--- + +## Quick Start + +### Launching +```bash +npm run ac # Start dev server +# Navigate to seashells in your browser +``` + +### Interactive Play +- **Touch anywhere** to start a voice at that position + - X-axis = base frequency (left=low, right=high) + - Y-axis = pitch multiplier (top=high, bottom=low) +- **Drag** to change frequency/pitch in real-time +- **Lift** to stop that voice + +### Hold Sequence (Autoplay) +- Press **H** to start autonomous voice generation +- Voices spawn at orbital positions every 2 seconds +- Each voice holds for 5-13 seconds +- New voices spawn before old ones fade → continuous audio +- Press **H** again to stop + +### For 90-Minute Tape +- Press **H** at start of session +- Let it run unattended +- The system will sustain audio continuously +- Visual feedback creates ongoing emergence + +--- + +## Files in This Documentation + +| File | Purpose | +|------|---------| +| `seashells_analysis.md` | Technical breakdown of how each component works | +| `seashells_conceptual_model.md` | High-level architecture, 4-layer model, remix framework | +| `seashells_variation_examples.md` | 5 concrete remix examples with copy-paste code | +| `SEASHELLS_README.md` | This file — quick reference | + +**Read in this order:** +1. This README (5 min) +2. Conceptual Model (20 min) — understand the 4 layers +3. Analysis (30 min) — deep dive into each layer +4. Variation Examples (60 min) — try specific remixes + +--- + +## The 4-Layer Architecture + +``` +┌────────────────────────────────────────────────────┐ +│ Layer 4: SEQUENCING │ +│ How voices spawn, sustain, and move │ +│ (Hold mechanism, orbital paths, voice lifecycle) │ +├────────────────────────────────────────────────────┤ +│ Layer 3: FEEDBACK LOOP │ +│ Audio ↔ Visual feedback │ +│ (Sample pixels, convert to audio modulation) │ +├────────────────────────────────────────────────────┤ +│ Layer 2: SYNTHESIS │ +│ Audio generation │ +│ (5 bytebeat patterns, blending, modulation) │ +├────────────────────────────────────────────────────┤ +│ Layer 1: SPATIAL MAPPING │ +│ Touch position → Audio parameters │ +│ (X→frequency, Y→pitch factor) │ +└────────────────────────────────────────────────────┘ +``` + +**Key insight:** Each layer is independent. Change one without breaking others. + +--- + +## What Makes It Work for 90 Minutes? + +1. **Algorithmic Complexity** + - 5 blending bytebeat patterns (not 1) + - Each pattern responds to 10+ modulation parameters + - Feedback loop creates continuous variation + +2. **Visual-Audio Feedback** + - Audio paints pixels + - Pixels influence audio via feedback + - Creates genuine emergence, not repetition + +3. **Voice Continuity** + - Hold sequences spawn new voices before old ones fade + - Ensures unbroken audio stream + - Voices never synchronize (different speeds) + +4. **State Decay** + - Interaction memory decays over 10 seconds + - Creates slow drift in parameters + - System explores new regions of parameter space + +--- + +## Made a Change? Test It Like This + +### 1-Minute Test +``` +Press H → listen for 60 seconds +Does audio continue without gaps? +Does timbre vary or is it repetitive? +``` + +### 5-Minute Test +``` +Press H → listen for 5 minutes +Do patterns feel structured or random? +Is visual complexity growing or settling? +Any obvious repetition loops? +``` + +### Tape Test (90 minutes) +``` +Press H → start recording +Leave running (no interaction) +Come back after 90 minutes +Listen back: would you put this on cassette? +``` + +**Tape quality checklist:** +- [ ] Audio never drops out (voices always sustain) +- [ ] Timbre evolves (5+ distinct characters over 90 min) +- [ ] No obvious repetition (don't hear same sequence twice) +- [ ] Visual patterns remain interesting (not just noise) +- [ ] Rhythm/pacing feels intentional, not random + +--- + +## Remix Quick Reference + +### Change Spatial Mapping (10 min) +- File: `seashells.mjs` +- Functions: `mapXToFrequency()`, `mapYToPitchFactor()` +- Examples: quantized scale, polar coords, grid snapping + +### Change Synthesis Patterns (30 min) +- File: `seashells.mjs` +- Location: `generator.bytebeat()` (line 61) +- Task: Add new pattern, integrate into blending + +### Change Feedback (30 min) +- File: `seashells.mjs` +- Function: `samplePixelFeedback()` (line 371) +- Task: Change what pixels are sampled, how they map to audio + +### Change Sequencing (30 min) +- File: `seashells.mjs` +- Functions: `spawnHoldVoice()`, `updateHoldVoices()` +- Task: Alter spawn timing, positions, movement patterns + +### Change Visuals (1 hour) +- File: `seashells.mjs` +- Function: `paint()` (line 525) +- Task: Different rendering (oscilloscope, spectrogram, particles) + +--- + +## Common Remix Patterns + +### "Make It Musical" +- Quantize X-axis to specific scale (pentatonic, chromatic) +- Use grid-based sequencing instead of orbital +- Reduce chaos injection +- Result: Harmonic, bell-like, more consonant + +### "Make It Chaotic" +- Increase feedback sensitivity to brightness/variance +- Add more chaos injection +- Increase pattern blending speed +- Result: Glitchy, algorithmic, harsh + +### "Make It Visual" +- Replace pixel column visualization with oscilloscope +- Add particles, trails, or fractal rendering +- Sync visual updates to audio beats +- Result: Visuals are primary, audio is secondary + +### "Make It Spacious" +- Reduce concurrent voices (max 3-4 instead of 6) +- Increase hold durations (10-30 seconds instead of 5-13) +- Reduce spawn rate (every 5-10 seconds instead of 2) +- Result: Sparse, contemplative, room to breathe + +### "Make It Dense" +- Increase concurrent voices (15-20 instead of 6) +- Decrease hold durations (2-5 seconds instead of 5-13) +- Increase spawn rate (every 1 second) +- Result: Dense, layered, orchestral + +--- + +## Performance Notes + +### If Synthesis Is CPU-Heavy +- Reduce waveform sample count (currently 512) +- Reduce pixel feedback sampling points (currently 12-20) +- Profile in DevTools to find bottleneck + +### If Visuals Fill with Noise +- Add slow screen wipe: `if (now % 45000 < 1000) wipe(0,0,0)` +- Reduce additive blending intensity +- Use opacity/fade instead of accumulation + +### If Hold Sequence Is Uneven +- Increase spawn interval (>2000ms) +- Reduce max concurrent voices (to 3-4) +- Make durations more consistent (reduce randomness) + +--- + +## Conceptual Symmetries (Design Patterns) + +These patterns appear throughout the code — exploit them: + +1. **Orbital Math** — Scanning, voice movement, visual sweeps all use cos/sin + - Use same orbit equations everywhere for coherence + +2. **Feedback Parameters** — Audio parameters match visual feedback sources + - High brightness → intensity + - High variance → chaos + - Exploit this for intuitive relationships + +3. **Time Scales** + - Sample-level: 44.1 kHz (bytebeat) + - Voice-level: 1-20 seconds (hold durations) + - System-level: 10+ seconds (state decay) + - Design remixes that respect these scales + +4. **Randomness** — Always constrained by feedback + - Voice spawn positions: random + orbital structure + - Hold durations: random ± base duration + - Pattern mixing: time-based + feedback bias + - Never pure noise, always quasi-musical + +--- + +## Key Files & Functions + +### Core Synthesis +- Line 61: `generator.bytebeat()` — The heart of audio generation +- Lines 82-97: Pattern definitions +- Lines 100-126: Pattern blending logic + +### Feedback +- Line 371: `samplePixelFeedback()` — Pixel → audio conversion +- Lines 379-424: Sampling strategy +- Lines 440-486: RGB → audio parameter mapping + +### Sequencing +- Line 40: `holdSequence` object initialization +- Line 370: `spawnHoldVoice()` — Create new voice +- Line 406: `updateHoldVoices()` — Update positions & durations +- Line 457: `toggleHoldSequence()` — Start/stop autoplay + +### Spatial Mapping +- Line 190: `mapXToFrequency()` — X pixel → Hz +- Line 198: `mapYToPitchFactor()` — Y pixel → pitch multiplier +- Line 205: `deriveVoiceFrequency()` — Combine into final frequency + +### Visuals +- Line 525: `paint()` — Main rendering function +- Lines 551-612: Pixel drawing logic +- Line 210: `drawTouchMapping()` — Grid visualization + +### Interaction +- Line 696: `sim()` — Per-frame updates +- Line 715: `act()` — Event handling (touch, keyboard) + +--- + +## Next Steps + +### To Understand Seashells +1. Read `seashells_conceptual_model.md` (understand 4 layers) +2. Read `seashells_analysis.md` (deep technical) +3. Try pressing H, making touches, observe behavior + +### To Remix Seashells +1. Pick one variation from `seashells_variation_examples.md` +2. Copy code into seashells.mjs +3. Test with `npm run ac` +4. Iterate one small change at a time + +### To Create Your Own Variation +1. Identify which layer(s) you want to change +2. Read the relevant functions in Analysis doc +3. Sketch the change on paper first +4. Implement in small steps +5. Test after each change + +--- + +## Philosophy + +Seashells is built on **layered independence**: +- Spatial mapping doesn't know about synthesis +- Synthesis doesn't know about visuals +- Visuals don't know about sequencing +- Sequencing is just a voice generator + +This means: +- You can modify any layer without breaking others +- Testing is incremental (change one thing, test) +- Remixes are combinatorial (stack changes) +- Future extensions are easy (add new layers) + +This is intentional design. Use it. + +--- + +## Questions? + +- **How does feedback work?** → See `samplePixelFeedback()` in Analysis +- **How can I add a new pattern?** → See "Add Your Own Pattern" in Conceptual Model +- **How do I make it more musical?** → See "Make It Musical" in Remix Patterns +- **What's the audio quality?** → Bytebeat, lo-fi by design (8-bit character) +- **Can I export audio?** → Use your browser's recording, or modify to write to AudioBuffer +- **Can I use this in my own piece?** → Yes, architecture is modular and reusable + +--- + +## Version History + +- **2025.6.13** — Initial Seashells release +- **2025.6.14** — Added hold mechanism, documentation suite + +**Created for:** 90-minute cassette tape experimentation + +**Best consumed as:** +- Interactive exploration (press H, make touches) +- Tape/long-form listening (press H, walk away) +- Educational dissection (read Analysis, remix patterns) +- Foundation for variations (remix, combine, extend) + +--- + +Made with care for emergence and modular design. Happy creating! diff --git a/SEASH_LAB_BENCH.md b/SEASH_LAB_BENCH.md new file mode 100644 index 0000000000..c7d2df9e99 --- /dev/null +++ b/SEASH_LAB_BENCH.md @@ -0,0 +1,400 @@ +# Seash — Lab Bench Proof of Concept + +A minimal, readable bytebeat synthesizer. ~350 lines of clear code. Perfect for understanding the mechanism and experimenting. + +--- + +## What It Does + +**Seash** is the distilled core of Seashells: +- Generate sound using bytebeat (simple mathematical synthesis) +- Auto-spawn voices on a loop (hold sequence) +- Play interactively by touching the screen +- Press H to toggle automatic playback + +No complex feedback loops. No dense visuals. Just the mechanism. + +--- + +## How It Works (Conceptual) + +### Layer 1: Sound Generation (Bytebeat) + +**The core idea:** Audio from bit operations on integers. + +```javascript +// Two patterns that blend over time +pattern1 = (t ^ (t >> 8) ^ (t >> 9)) & 255 // XOR: crisp, digital +pattern2 = ((t * harmonic) & (t >> 5) | (t >> 4)) & 255 // Melodic: pitched + +// Mix them based on time +finalPattern = pattern1 * (1 - blend) + pattern2 * blend +``` + +**Why it works:** Integer operations are deterministic. Same input → same output. But small changes in parameters create wildly different sounds. + +**Key insight:** The two patterns **never collide**. When one fades out, the other fades in. This creates a continuous, evolving texture. + +--- + +### Layer 2: Spatial Control (Touch → Frequency) + +**How position maps to sound:** + +```javascript +// X-axis (left to right) +X = 0% → 55 Hz (very low) +X = 50% → 220 Hz (middle) +X = 100% → 880 Hz (high) +// Logarithmic scale (musically natural) + +// Y-axis (top to bottom) +Y = 0% → 2.0x pitch multiplier (octave up) +Y = 50% → 1.0x pitch multiplier (normal) +Y = 100% → 0.5x pitch multiplier (octave down) +``` + +**Result:** Any (X, Y) position has a unique frequency. Move around = explore the frequency space. + +**Why logarithmic:** Our ears perceive pitch logarithmically. A 2x frequency jump feels the same from 100Hz→200Hz as from 1000Hz→2000Hz. + +--- + +### Layer 3: Multi-Voice Management + +**The system tracks multiple simultaneous voices:** + +```javascript +const touchVoices = new Map() // { pointerIndex → { sound, frequency, x, y } } +``` + +**When you touch:** +1. New voice spawns at that position +2. Frequency derived from X/Y +3. Voice added to map with unique pointer ID + +**When you move your finger:** +1. Voice position updates +2. Frequency updates smoothly (with lerp to avoid jumps) + +**When you lift:** +1. Voice fades out (0.08 second fade) +2. Removed from map + +**Volume balancing:** If you have N fingers down, each voice volume = 0.5 / sqrt(N) +- 1 voice: 0.5 volume +- 4 voices: 0.25 each (stays reasonable) +- 16 voices: 0.125 each + +--- + +### Layer 4: Auto-Voice Generation (Hold Sequence) + +**The magic mechanism:** + +```javascript +if (holdSequence.enabled) { + // Every 3 seconds, spawn a new voice + spawnHoldVoice() + + // Each voice: + // - Starts at orbital position + // - Drifts along orbit + // - Lasts 6±1.5 seconds + // - Fades out when duration expires + + // Result: continuous audio, voices never sync +} +``` + +**Why it works for 90 minutes:** +- New voice spawns before old one dies +- Each has different orbit speed (unique drifting path) +- No two voices follow same trajectory +- Even though only 2 patterns, constant variation + +**Why voices don't sync:** +- Orbit speeds differ by small amounts (0.0003–0.0005 rad/frame) +- After 30 seconds, they're all at different phases +- Mathematical: irrational multiples → infinite non-repetition + +--- + +## The Code (Line by Line) + +### Sections + +**Lines 1–20:** Constants and initialization +- `touchVoices`: Map of active voices +- `holdSequence`: State for auto-generation + +**Lines 22–64:** Bytebeat generator +- Takes frequency, time, and sample count +- Returns 512 audio samples +- Blends 2 patterns smoothly + +**Lines 66–80:** Utility functions +- `clamp()`: Constrain values to range +- `mapXToFrequency()`: X pixel → Hz (logarithmic) +- `mapYToPitch()`: Y pixel → pitch factor +- `deriveFrequency()`: Combine X+Y + +**Lines 82–130:** Voice lifecycle +- `createVoice()`: Initialize synthesizer +- `startTouchVoice()`: New voice from touch +- `updateTouchVoice()`: Move existing voice +- `stopTouchVoice()`: Fade and remove +- `rebalanceVolumes()`: Keep mix balanced + +**Lines 132–175:** Hold sequence +- `spawnHoldVoice()`: Create auto voice at orbital position +- `updateHoldVoices()`: Move orbits, spawn new, fade old +- `toggleHoldSequence()`: Start/stop auto mode + +**Lines 177–210:** Rendering +- `paint()`: Draw grid, labels, voice positions +- Minimal UI: just frequency readouts and status + +**Lines 212–247:** Input handling +- `act()`: Touch and keyboard events +- Support 8 simultaneous touches +- H key toggles hold sequence + +**Lines 249–253:** Per-frame updates +- `sim()`: Audio polling, hold sequence updates + +--- + +## How to Use It + +### Interactive Mode +``` +Touch screen → voice spawns at that position +Move finger → frequency changes in real-time +Lift finger → voice fades out + +Multi-touch: 8 fingers at once, each with own voice +``` + +### Autoplay (Tape Mode) +``` +Press H → hold sequence starts +Voices spawn automatically every 3 seconds +Each holds for 6±1.5 seconds +Press H again → stops + +Let run for 90 minutes → listen to emergence +``` + +### Observe the Mechanism +``` +Watch voice numbers increase and decrease +Notice frequency labels updating as voices move +See hold sequence spawn new voice before old fades +Listen to how patterns blend smoothly +``` + +--- + +## Why It's a Good Lab Bench + +### 1. **Readable** +- ~350 lines (vs 800+ for seashells.mjs) +- No visual feedback loop +- No complex state decay +- Direct cause-effect + +### 2. **Modifiable** +Each section is independent: + +**Change sound:** Edit `generator.bytebeat()` (line 22) +- Add pattern 3: just write another formula +- Integrate it into blending + +**Change spatial mapping:** Edit `mapXToFrequency()` (line 69) +- Try linear instead of logarithmic +- Try quantized to specific notes +- Try polar coordinates + +**Change auto-generation:** Edit `spawnHoldVoice()` (line 135) +- Different spawn positions (grid, random, bounded) +- Different orbit speeds +- Different durations + +**Change UI:** Edit `paint()` (line 179) +- Remove grid, add oscilloscope +- Add different text labels +- Show orbit paths visually + +### 3. **Testable** +- Add `console.log()` anywhere to debug +- Change one variable, test immediately +- No side effects (each voice is independent) + +### 4. **Minimal Dependencies** +- Only uses AC's `sound.synth()` API +- No external libraries +- No complex state machines + +--- + +## Experiments to Try + +### Experiment 1: Change the Patterns +**Goal:** Make it sound more chaotic + +```javascript +// In generator.bytebeat(), change p1: +const p1 = (t * t) & (t >> 4) & 255; // Multiplicative instead of XOR + +// Test: Press H, listen to how it differs +``` + +### Experiment 2: Change Frequency Range +**Goal:** Make it higher or lower pitched + +```javascript +// In mapXToFrequency(): +const minHz = 110; // Was 55 (raise minimum) +const maxHz = 440; // Was 880 (lower maximum) + +// Test: Press H, notice narrower frequency range +``` + +### Experiment 3: Change Auto-Spawn Rate +**Goal:** More or fewer voices + +```javascript +// In holdSequence: +spawnInterval: 1500, // Was 3000 (spawn every 1.5 sec instead of 3) + +// Test: More voices, denser texture +``` + +### Experiment 4: Change Voice Lifespan +**Goal:** Longer or shorter holds + +```javascript +// In spawnHoldVoice(): +const duration = 15000 + (Math.random() - 0.5) * 5000; // Was 6000 ± 1500 + +// Test: Slower, more meditative tape +``` + +### Experiment 5: Add a Third Pattern +**Goal:** More timbral variety + +```javascript +// In generator.bytebeat(), add: +const p3 = ((t >> 2) + (t >> 5)) & 255; // Additive pattern + +// Modify blending to include it: +let mixPhase = (time * 0.1 + freqScale * 0.5) % 3; // Was % 2 +if (mixPhase < 1) { /* p1 to p2 */ } +else if (mixPhase < 2) { /* p2 to p3 */ } +else { /* p3 to p1 */ } + +// Test: Three-way blend creates richer texture +``` + +--- + +## What's Removed (vs Seashells) + +### Removed: +- **Feedback loop** (pixels don't influence audio) +- **5 patterns** (reduced to 2 for clarity) +- **Complex UI** (minimal text only) +- **Grid visualization** (just dots) +- **State decay system** (removed unnecessary complexity) +- **Chaos injection** (simplified synthesis) +- **Harmonic Bell quantization** (bare version uses full spectrum) + +### Kept: +- **Core bytebeat synthesis** +- **Spatial frequency mapping** +- **Multi-voice management** +- **Hold sequence (auto-generation)** +- **Smooth blending between patterns** + +--- + +## Performance Notes + +**CPU usage:** Very light +- 2 patterns instead of 5 +- No pixel sampling +- No feedback loop +- Simple rendering + +**Audio quality:** 8-bit bytebeat character (lo-fi by design) + +**Latency:** Minimal — direct synthesis, no heavy processing + +--- + +## Next Steps + +### To Understand Deeper: +1. Run it and experiment with touches +2. Press H and listen for 5 minutes +3. Add `console.log()` to understand timing +4. Modify one parameter at a time + +### To Build Variations: +1. Copy seash.mjs to seashell_variation.mjs +2. Make ONE change +3. Test immediately +4. Document what changed and why +5. Stack changes incrementally + +### To Connect to Seashells: +Once you understand seashell: +1. Read seashells_conceptual_model.md +2. Understand how feedback loop works +3. Add feedback sampling to seashell +4. You've now built seashells from scratch! + +--- + +## Philosophy + +**Seashell** embodies these principles: + +1. **Clarity over cleverness** — Every line should make sense +2. **Mechanism over magic** — You can trace cause-effect +3. **Minimal abstraction** — One function per concept +4. **Modular independence** — Change one layer, others stay stable +5. **Testability** — Add print statements, change values, listen + +Use it as a **thinking tool**, not just a sound generator. + +--- + +## Quick Reference + +| What | Where | How | +|------|-------|-----| +| Change sound | `generator.bytebeat()` line 22 | Modify pattern formulas | +| Change frequency range | `mapXToFrequency()` line 69 | Edit minHz/maxHz | +| Change pitch modulation | `mapYToPitch()` line 75 | Edit exponent (2) | +| Change auto-spawn timing | `holdSequence.spawnInterval` line 24 | Edit milliseconds | +| Change voice duration | `spawnHoldVoice()` line 149 | Edit duration calculation | +| Change voice count limit | `updateHoldVoices()` line 161 | Edit `.length < 5` | +| Change rendering | `paint()` line 179 | Modify drawing code | +| Change audio response | `sim()` line 245 | Edit voice update logic | + +--- + +## Files to Compare + +- **seash.mjs** (this one) — bare bones, ~350 lines +- **seashells.mjs** — full version, ~900 lines, with feedback +- **seashells_conceptual_model.md** — 4-layer architecture explanation + +You can learn by: +1. Understanding seashell completely +2. Reading conceptual_model.md +3. Comparing against seashells.mjs to see what each layer adds + +Good lab bench = good foundation for understanding the full piece. diff --git a/SEASH_SLIDERS.md b/SEASH_SLIDERS.md new file mode 100644 index 0000000000..eabd91c87d --- /dev/null +++ b/SEASH_SLIDERS.md @@ -0,0 +1,311 @@ +# Seash Sliders — Lab Bench Control Panel + +Five interactive sliders at the bottom of seash for real-time parameter adjustment. + +--- + +## The Sliders + +All sliders appear at the bottom of the screen and are touchable in real-time. + +### 1. **Spawn** (Spawn Interval) +- **Range:** 500 ms — 8000 ms +- **Default:** 3000 ms (3 seconds) +- **What it does:** How often new voices are spawned in hold sequence +- **Try this:** + - Move left (500ms) = dense, overlapping voices + - Move right (8000ms) = sparse, distinct phrases + - Sweet spot: 2000-4000ms for continuous texture + +### 2. **Duration** (Voice Duration) +- **Range:** 2000 ms — 20000 ms +- **Default:** 6000 ms (6 seconds) +- **What it does:** How long each voice sustains before fading +- **Try this:** + - Left (2s) = staccato, quick pulses + - Right (20s) = long held tones, pad-like + - Quick changes: 2-4 seconds + - Sustained changes: 10-15 seconds + +### 3. **OrbitSpd** (Orbit Speed) +- **Range:** 0.0001 — 0.001 rad/frame +- **Default:** 0.0003 rad/frame +- **What it does:** How fast each voice drifts through frequency space +- **Try this:** + - Left = slow, lazy movement + - Right = fast, erratic movement + - Controls the "wandering" behavior of auto voices + +### 4. **BlendSpd** (Pattern Blend Speed) +- **Range:** 0.01 — 0.5 +- **Default:** 0.1 +- **What it does:** How fast patterns cross-fade into each other +- **Try this:** + - Left (0.01) = smooth, gradual transitions + - Right (0.5) = quick, snappy transitions + - Affects overall timbre evolution + +### 5. **MaxVoices** (Max Concurrent Auto Voices) +- **Range:** 1 — 10 +- **Default:** 5 voices +- **What it does:** Maximum number of simultaneous auto-generated voices +- **Try this:** + - 1 = single voice, monophonic + - 3-5 = balanced polyphony + - 8-10 = dense cluster + +--- + +## How to Use + +### Adjust a Slider +1. **Touch and drag** the slider handle (■) left/right +2. Value updates in real-time +3. Audio responds immediately +4. Release to stop dragging + +### Read the Values +Each slider shows its current value to the right: +``` +Spawn: ________■____ 3000 + (value displayed on right) +``` + +### Combine Sliders for Effects + +**Slow Pad:** +- Duration: 15000 (long holds) +- Spawn: 8000 (slow spawn) +- OrbitSpd: 0.0001 (minimal drift) +- BlendSpd: 0.01 (smooth transitions) + +**Textural Chaos:** +- Duration: 3000 (quick pulses) +- Spawn: 500 (rapid spawning) +- OrbitSpd: 0.001 (fast movement) +- BlendSpd: 0.3 (snappy blending) + +**Musical Melody:** +- Duration: 8000 (medium holds) +- Spawn: 4000 (structured spacing) +- OrbitSpd: 0.0003 (moderate drift) +- BlendSpd: 0.15 (balanced transitions) + +**Dense Chords:** +- MaxVoices: 10 (many simultaneous) +- Duration: 10000 (held chords) +- Spawn: 2000 (continuous) +- OrbitSpd: 0.0001 (minimal movement) + +--- + +## What's Happening Under the Hood + +### Spawn Interval +Controls `params.spawnInterval`, used in `updateHoldVoices()`: +```javascript +if (now - lastSpawnTime > params.spawnInterval && count < maxVoices) { + spawnHoldVoice(); // Create new voice +} +``` + +### Voice Duration +Controls `params.voiceDuration`, used when spawning: +```javascript +const duration = params.voiceDuration + (Math.random() - 0.5) * (duration * 0.5); +// Voice lasts this long before fading +``` + +### Orbit Speed +Controls `params.orbitSpeed`, used for voice position: +```javascript +hold.orbitPhase += params.orbitSpeed; +const x = Math.cos(hold.orbitPhase) * radius + center; +``` + +### Blend Speed +Controls `params.blendSpeed`, in the bytebeat generator: +```javascript +const mixPhase = (time * params.blendSpeed + freqScale * 0.5) % 2; +// Controls how fast patterns morph +``` + +### Max Voices +Controls `params.maxVoices`, checked during spawning: +```javascript +if (activeHolds.length < params.maxVoices) { + spawnHoldVoice(); // Only spawn if under limit +} +``` + +--- + +## Slider Layout + +``` +Screen height: + +[Main playing area - 80% of screen] + - Frequency grid + - Voice positions + - Touch interaction + +[Slider area - bottom 20% of screen] + Spawn: ___■_____ 3000 + Duration: ______■__ 6000 + OrbitSpd: _____■___ 0.0003 + BlendSpd: ____■____ 0.1 + MaxVoices: _■_______ 5 +``` + +Sliders are fully interactive while you play — adjust in real-time. + +--- + +## Tips for Experimentation + +### 1. **Change One at a Time** +- Adjust Spawn, listen for 10 seconds +- Adjust Duration next, observe how it interacts +- This trains your intuition for each parameter + +### 2. **Listen for Patterns** +- Try MaxVoices=1 (monophonic) with different spawn/duration combos +- Try MaxVoices=8 (dense) with same settings +- Notice how polyphony changes the effect + +### 3. **Map the Space** +Create a mental map: +``` +Slow, smooth ←→ Fast, chaotic +(all sliders left) (all sliders right) + +Spacious, minimal ←→ Dense, overlapping +(low spawn, max=1) (high spawn, max=10) +``` + +### 4. **Use as Tape Control** +- Set parameters how you like +- Press H to start recording +- Let it run 30 minutes +- Sliders capture your intended "composition" in 5 parameters + +### 5. **Keyboard + Slider Combo** +- H to toggle hold sequence +- Drag sliders while holding plays +- Create interactive performances + +--- + +## Slider Precision + +**Coarse adjustments:** +- Touch slider, drag all the way left or right +- Jumps to min/max + +**Fine adjustments:** +- Touch slider and drag slowly +- Values update frame-by-frame as you drag + +**Quantized sliders:** +- MaxVoices: steps of 1 (always integer) +- Others: continuous (floating point) + +--- + +## Getting the Values Right + +### For 90-Minute Tape +- **Spawn:** 3000-5000 (not too dense) +- **Duration:** 6000-10000 (medium holds) +- **OrbitSpd:** 0.0002-0.0004 (gentle movement) +- **BlendSpd:** 0.1-0.2 (smooth but alive) +- **MaxVoices:** 4-6 (polyphonic without muddy) + +### For Interactive Play +- **Spawn:** Don't matter (you're creating voices) +- **Duration/OrbitSpd:** Personal taste +- **BlendSpd:** 0.15 for snappy response +- **MaxVoices:** 6-8 (more bandwidth for your touches) + +### For Study +- **Spawn:** 3000 (baseline) +- **Duration:** 6000 (baseline) +- **OrbitSpd:** 0.0003 (baseline) +- **BlendSpd:** 0.1 (baseline) +- **MaxVoices:** 5 (baseline) + +Then change ONE, listen, understand, reset, repeat. + +--- + +## Not Working? + +**Slider doesn't respond:** +- Make sure you're touching on the blue slider line, not the label +- Drag horizontally (not vertically) +- Release and try again + +**Values not changing:** +- Confirm you're in seash (not seashells) +- Check that hold sequence is ON (press H) +- Values change immediately; listen for audio change + +**Want to reset:** +- Reload the page (F5) +- Or manually drag each slider back to default +- No "reset" button needed + +--- + +## Next: Combine with Touch Play + +You can: +1. Set sliders to your preferred hold sequence settings +2. **Also** touch the screen to add manual voices +3. Manual touches + auto holds mix together +4. Create layered performances + +The sliders control the "background" (auto voices), your touches are the "foreground" (interactive layer). + +--- + +## Example Sessions + +### Session A: Deep Listen (90 minutes) +``` +Spawn: 4000 (slower spawn) +Duration: 8000 (longer holds) +OrbitSpd: 0.0002 (minimal drift) +BlendSpd: 0.08 (smooth) +MaxVoices: 4 (sparse chords) + +Result: Meditative, slowly evolving texture +``` + +### Session B: Interactive Play (30 minutes) +``` +Spawn: 3000 +Duration: 6000 +OrbitSpd: 0.0003 +BlendSpd: 0.15 +MaxVoices: 5 + +Press H, then also make touches +Auto voices + your gestures = rich conversation +``` + +### Session C: Chaos Lab (15 minutes) +``` +Spawn: 800 (rapid!) +Duration: 3000 (quick bursts) +OrbitSpd: 0.0008 (fast movement) +BlendSpd: 0.3 (snappy blends) +MaxVoices: 8 (dense!) + +Result: Glitchy, algorithmic texture +``` + +--- + +Made for experimentation. Sliders = direct feedback loop between your intention and the sound. diff --git a/seashells_analysis.md b/seashells_analysis.md new file mode 100644 index 0000000000..3fa42908e8 --- /dev/null +++ b/seashells_analysis.md @@ -0,0 +1,334 @@ +# Seashells.mjs Analysis + +## Overview +**Seashells** is a bytebeat algorithmic synthesizer with visual feedback. It's designed as an interactive piece where touch positions generate audio, but converting it to autoplay requires significant architectural changes. + +--- + +## Core Audio Mechanism + +### Bytebeat Synthesis +The piece uses **5 algorithmic patterns** that blend over time: + +1. **Pattern 1 (XOR Cascade)** - Crisp, digital texture + ``` + (t ^ (t >> (8 + shiftMod1)) ^ (t >> (9 + shiftMod2))) & 255 + ``` + Classic bytebeat - creates sharp, glitchy sounds + +2. **Pattern 2 (Melodic Stepped)** - Harmonic content + ``` + ((t * harmonic) & (t >> (5 + bitMod1)) | (t >> (4 + bitMod2))) & 255 + ``` + Responsive to frequency scaling + +3. **Pattern 3 (Rhythmic)** - Complex polyrhythmic patterns + ``` + (t | (t >> rhythmMod | t >> 7)) * (t & (t >> 11 | t >> complexMod)) & 255 + ``` + Highly sensitive to frequency and feedback + +4. **Pattern 4 (Sierpinski-like)** - Fractal patterns + ``` + (t & (t >> (5 + sierpinskiMod) | t >> 8)) & 255 + ``` + Creates algorithmic complexity + +5. **Pattern 5 (Frequency-responsive Melodic)** + ``` + ((t * melodyScale) ^ (t >> 6)) & (t >> 8) & 255 + ``` + Strong pitch sensitivity + +### Pattern Blending +- Patterns cycle through 4-phase blend states (0→1→2→3→0) +- Blend speed is modifiable by feedback (`mixSpeed`) +- Blending is smooth and continuous (`blendIntensity` 0.3-1.0) + +### Feedback System (Audio ↔ Visual Loop) +The piece samples pixels from the screen and converts them into audio modulation parameters: + +**Sampling Strategy:** +- 4 corner samples +- 4 edge samples (mid-points of each edge) +- 4 diagonal sweeps +- 4+ orbital scanning samples (elliptical patterns that move with interaction memory) + +**Conversion to Audio Parameters:** +``` +Red channel → timeModulation, harmonicScale, colorMod.r +Green channel → rhythmScale, mixSpeed, colorMod.g +Blue channel → shiftMod2, patternBias, colorMod.b +Contrast → bitMod values (higher contrast = more bit operations) +Variance → chaosLevel (pixel unpredictability → audio chaos) +``` + +**Chaos Injection:** +```javascript +if (feedback.chaosLevel > 0.5) { + finalPattern = finalPattern ^ Math.floor(feedback.chaosLevel * 128); +} +``` + +--- + +## Visual Generation + +### Pixel Rendering +- **No wipe()** - pixels accumulate, creating permanent trails +- Bytebeat values map directly to Y positions +- Colors computed from bit patterns AND feedback color mods +- Additive blending for accumulation effects + +### Visual Elements +1. **Main column visualization** - One vertical line per X pixel, height = bytebeat value +2. **Bit pattern layers** - Each bit of the bytebeat value adds horizontal bands +3. **Vertical streaks** - Frequency-responsive vertical lines (every Nth column) +4. **Horizontal sweep** - Time-based horizontal scan line that moves down the screen + +### Interaction Visualization +- Touch overlays show active voices with colored circles +- Frequency label for each touch (Hz) +- Grid showing frequency/pitch mapping +- Help text when idle + +--- + +## Interaction State (The "Memory" System) + +The piece maintains persistent modulation state that decays over time: + +```javascript +interactionState = { + scanOffset: 0, // Orbital scan phase + scanVelocity: 0.003, // How fast it scans + scanSpread: 1.0, // Vertical spread of scan + orbit: 0, // Cumulative rotation bias + memory: 0, // Persistent touch "memory" (0-1) + chaosBias: 0, // How chaotic it gets + density: 1.0, // Sampling density + lastTouchAt: 0 // Timestamp of last interaction +} +``` + +**How Touch Influences State:** +```javascript +scanOffset += (nx * 0.11 + ny * 0.07) // Touch moves scan +orbit += (nx - 0.5) * 0.18 // Horizontal bias → rotation +scanSpread *= 0.9; scanSpread += (0.65 + ny) * 0.1 // Vertical → spread +memory *= 0.94; memory += 0.05 + |nx - 0.5| * 0.08 // Accumulates +chaosBias *= 0.9; chaosBias += |nx - 0.5| * 0.25 // Edges → chaos +``` + +**Decay Over Time:** +- Memory decays at 0.998/frame if touched recently, 0.992 if idle +- Chaos decays at 0.997 (active) / 0.985 (idle) +- Orbit decays at 0.992 +- After ~10 seconds idle: memory → 0, piece quiets down + +--- + +## Current Limitations for 90-Minute Tape + +### ❌ Problems +1. **No autoplay** - Requires manual touches to generate audio +2. **Silent when idle** - Help screen displays when no voices active +3. **Limited generative richness** - Only 8 simultaneous voices, all driven by touch +4. **Accumulation without clearing** - Visual system will eventually fill with noise +5. **No time-based voice generation** - No procedural voice triggering +6. **Memory decay** - State fades to silence after ~10 seconds of inactivity + +### ✅ Strengths (Why It Could Work) +1. **High algorithmic complexity** - 5 blending patterns × feedback × chaos injection = very large parameter space +2. **Feedback loop creates emergence** - Visual patterns influence audio, creating unpredictable evolution +3. **Deterministic** - Same pixel patterns always produce same audio (reproducible tape) +4. **Minimal repetition** - Bytebeat patterns are subtle and shift continuously via blending +5. **Scaling** - Can handle more simultaneous voices than needed (currently capped at 8) + +--- + +## Required Changes for Autoplay + +### Option 1: Procedural Voice Generation (Recommended) + +**Add time-based voice triggering in `sim()`:** + +```javascript +function sim({ sound, hud, screen }) { + // ... existing code ... + + // Procedural voice generation + const voiceTargetCount = Math.round(1 + sharedPixelFeedback.density * 4); + const currentVoiceCount = totalVoiceCount(); + + if (currentVoiceCount < voiceTargetCount && performance.now() - interactionState.lastAutoVoiceAt > 300) { + // Add voice at pseudo-random "musical" position + const nextX = (interactionState.autoVoicePhase * screen.width) % screen.width; + const nextY = (Math.sin(performance.now() * 0.0003) * 0.5 + 0.5) * screen.height; + + startTouchVoice({ + pointerIndex: 8 + currentVoiceCount, // Use high indices for auto voices + x: nextX, + y: nextY, + screenWidth: screen.width, + screenHeight: screen.height, + sound + }); + + interactionState.lastAutoVoiceAt = performance.now(); + interactionState.autoVoicePhase = (interactionState.autoVoicePhase + 0.31) % 1; // Golden ratio + } + + // Age out auto-voices slowly (don't kill, just quiet) + // This creates natural voice turnover instead of jumping in/out +} +``` + +**Adjustments needed:** +- Increase `maxTouchPointers` from 8 to ~20-30 for more voices +- Add `lastAutoVoiceAt` and `autoVoicePhase` to `interactionState` +- Modify voice volume calculation to account for mix of auto/touch voices + +### Option 2: "Hold" Mode (Simpler, More Controlled) + +**Add a single "master" voice that holds until changed:** + +```javascript +let holdState = { + x: null, + y: null, + holdUntil: 0, + nextChangeAt: 0 +}; + +function act({ event: e, sound, screen, pens }) { + // Existing touch handling... + + if (e.is("keyboard:down:h")) { + // Toggle hold mode + if (holdState.x === null) { + // Start holding at a specific position + holdState.x = screen.width * 0.5; + holdState.y = screen.height * 0.5; + holdState.holdUntil = performance.now() + 5000; // Hold for 5 sec + startTouchVoice({ + pointerIndex: 99, // Special hold voice + x: holdState.x, + y: holdState.y, + screenWidth: screen.width, + screenHeight: screen.height, + sound + }); + } else { + stopTouchVoice(99); + holdState.x = null; + } + } +} + +function sim({ sound, hud, screen }) { + // Auto-release hold if time expired + if (holdState.x !== null && performance.now() > holdState.holdUntil) { + stopTouchVoice(99); + holdState.x = null; + } + + // Or: continuously update hold position based on pixel feedback + if (holdState.x !== null) { + const feedback = sharedPixelFeedback; + holdState.x = (holdState.x + feedback.patternBias * 0.5) % screen.width; + holdState.y = (holdState.y + feedback.timeModulation * 0.0001) % screen.height; + updateTouchVoice({ + pointerIndex: 99, + x: holdState.x, + y: holdState.y, + screenWidth: screen.width, + screenHeight: screen.height, + sound + }); + } +} +``` + +### Option 3: Hybrid (Best for Tape) + +Combine procedural generation + controlled hold positions: +- Auto-voices spawn at intervals determined by pixel feedback +- Each voice holds for variable duration (3-15 seconds) +- Hold positions follow orbital patterns (music-like phrasing) +- User can still manually intervene + +--- + +## Viability for 90 Minutes + +### Without Changes +**⚠️ Not viable** - Needs manual interaction, would result in 90 minutes of silence + random touches + +### With Procedural Voices +**✅ Viable** - Could sustain audio, but: +- Voices may cluster in same regions without spatial variation +- Without user interaction, state may converge to stable patterns +- Visual accumulation could become monolithic + +### With Hold Mode + Orbital Sequencing +**✅ Very viable** - Could create: +- Phrased movements (voices move through parameter space) +- Natural emergence from pixel feedback +- Balance between predictability and surprise +- Tape-like "performance" quality + +### Recommended Hybrid Approach + +1. **Keep current touch system** for interactivity +2. **Add procedural voice spawning** that's influenced by feedback +3. **Add orbital "hold" sequences** that create musical phrasing +4. **Slowly wipe screen** (every 30-60 seconds) to prevent visual noise accumulation +5. **Map feedback more musically** - e.g., high variance → more voices, high brightness → faster tempo + +Example voice spawning pattern: +```javascript +// Spawn voices at orbital positions, Fibonacci intervals +const goldenRatio = 1.618; +const nextSpawn = Math.floor(baseInterval * Math.pow(goldenRatio, currentSpawnIndex)); +const orbitPhase = (performance.now() * 0.0001 + currentSpawnIndex * 0.31) % (Math.PI * 2); +const x = (Math.cos(orbitPhase) * 0.4 + 0.5) * screen.width; +const y = (Math.sin(orbitPhase) * 0.4 + 0.5) * screen.height; +``` + +--- + +## Memory & Emergence + +The **key strength** is that visual state influences audio via feedback sampling: + +1. Pixels accumulate → visual patterns become complex +2. Complex visuals → chaotic feedback parameters +3. Chaotic feedback → audio becomes more generative +4. Audio via painting → new visual patterns +5. Loop → increasing complexity over 90 minutes + +This is **genuine emergence**, not repetition. A 90-minute tape would document the system's exploration of its parameter space, gradually finding new combinations. + +--- + +## Suggested Implementation Priority + +If building autoplay version: + +1. **First** - Add slow screen wipe (every 45 sec) to prevent accumulation +2. **Second** - Add procedural voice spawning based on pixel variance +3. **Third** - Implement hold sequences (3-15 second voice holds at orbital positions) +4. **Fourth** - Map feedback more musically (high-brightness → voice clusters, etc.) +5. **Optional** - Add keyboard shortcuts for manual phase control (reset wipe, trigger voices, etc.) + +--- + +## Code Entry Points to Modify + +- `sim()` (line 696) - Add voice generation logic +- `interactionState` (line 25) - Add autoplay-specific state +- `act()` (line 625) - Add keyboard controls for autoplay +- `paint()` (line 490) - Add conditional wipe logic + +Would preserve all existing touch/visual mechanics while enabling tape-like continuous playback. diff --git a/seashells_conceptual_model.md b/seashells_conceptual_model.md new file mode 100644 index 0000000000..f8f591a15d --- /dev/null +++ b/seashells_conceptual_model.md @@ -0,0 +1,529 @@ +# Seashells: Conceptual Model & Variation Framework + +This document breaks down the architecture of **seashells.mjs** into conceptual components, so you can understand, remix, and create variations. + +--- + +## The Core Stack (4 Layers) + +``` +┌─────────────────────────────────┐ +│ SEQUENCING LAYER │ How voices spawn & interact +│ (Hold system, voice lifecycle) │ +├─────────────────────────────────┤ +│ SYNTHESIS LAYER │ How audio is generated +│ (5 bytebeat patterns, blending)│ +├─────────────────────────────────┤ +│ FEEDBACK LOOP LAYER │ Audio ↔ Visual feedback +│ (Pixel sampling → parameters) │ +├─────────────────────────────────┤ +│ SPATIAL MAPPING LAYER │ Touch → Frequency/Pitch +│ (X/Y to Hz, modulation axes) │ +└─────────────────────────────────┘ +``` + +Each layer is **independently modifiable**. You can swap out any component without breaking the others. + +--- + +## Layer 1: Spatial Mapping (Touch/Position → Audio Parameters) + +### Current Implementation +```javascript +X-axis: screen position → base frequency (80–1600 Hz logarithmic) +Y-axis: screen position → pitch multiplier (0.5x–2x linear) +``` + +**Functions involved:** +- `mapXToFrequency(x, width)` - Convert X pixel to frequency +- `mapYToPitchFactor(y, height)` - Convert Y pixel to pitch multiplier +- `deriveVoiceFrequency()` - Combine both into final frequency + +### Variations You Could Try + +**1. Polar Coordinate Mapping** +```javascript +// Instead of cartesian X/Y +const angle = Math.atan2(y - centerY, x - centerX); +const distance = Math.sqrt((x-centerX)² + (y-centerY)²); +const frequency = minHz * Math.pow(maxHz/minHz, distance/maxRadius); +const timbre = (angle + Math.PI) / (2 * Math.PI); // Map to 0-1 +``` + +**2. Vertical Strip Mapping** (like a piano keyboard) +```javascript +// Ignore X, only use Y for frequency +const frequency = 55 * Math.pow(2, y / screenHeight * 5); // 5 octaves +``` + +**3. Grid Quantization** (musical scale constraints) +```javascript +const notes = [55, 62, 69, 82, 110, 123, 147, 165, 196, 220]; // C minor pentatonic +const gridX = Math.round(x / screenWidth * (notes.length - 1)); +const octaveY = Math.round(y / screenHeight * 4); +const frequency = notes[gridX] * Math.pow(2, octaveY); +``` + +**4. Feedback-Influenced Mapping** (space changes based on audio) +```javascript +const baseFreq = mapXToFrequency(x, width); +const pitchMult = mapYToPitchFactor(y, height); +// Modulate by pixel feedback +const feedbackScale = 0.8 + sharedPixelFeedback.intensity * 0.4; +return baseFreq * pitchMult * feedbackScale; +``` + +--- + +## Layer 2: Synthesis (Audio Generation) + +### Current Architecture: 5 Blending Patterns + +The piece uses **5 independent bytebeat generators** that morph through each other: + +```javascript +pattern1 = (t ^ (t >> (8 + shiftMod1)) ^ (t >> (9 + shiftMod2))) & 255 +pattern2 = ((t * harmonic) & (t >> (5 + bitMod1)) | (t >> (4 + bitMod2))) & 255 +pattern3 = (t | (t >> rhythmMod | t >> 7)) * (t & (t >> 11 | t >> complexMod)) & 255 +pattern4 = (t & (t >> (5 + sierpinskiMod) | t >> 8)) & 255 +pattern5 = ((t * melodyScale) ^ (t >> 6)) & (t >> 8) & 255 +``` + +**Blending mechanism:** +- Time-based phase progresses through 5 states (0→1→2→3→4→0) +- Between states, linear interpolation smooths transitions +- Phase speed and intensity controlled by feedback + +### Understanding Each Pattern + +| Pattern | Type | Character | Key Insight | +|---------|------|-----------|-------------| +| **Pattern 1** | XOR Cascade | Digital, crisp, glitchy | Bit flips create harsh transitions | +| **Pattern 2** | Melodic | Pitched, harmonic | `t * harmonic` creates repeating cycles | +| **Pattern 3** | Rhythmic | Complex polyrhythm | Multiplication creates interference patterns | +| **Pattern 4** | Fractal | Sierpinski-like, algorithmic | Simple XOR creates complexity | +| **Pattern 5** | Frequency-Responsive | Pitch-sensitive melodic | Scale changes with input frequency | + +### Variation: Add Your Own Pattern + +**Step 1: Design a pattern** +```javascript +const pattern6 = (t * t) & (t >> (7 + feedback.complexity)) & 255; +``` + +**Step 2: Integrate into blending loop** +```javascript +let mixPhase = (time * 0.08 + freqScale * 0.5) % 6; // Changed from 5 to 6 +if (mixPhase < 1) { + finalPattern = pattern1 * (1 - blend) + pattern2 * blend; +} else if (mixPhase < 2) { + finalPattern = pattern2 * (1 - blend) + pattern3 * blend; +} // ... add more conditions ... +else if (mixPhase < 5) { + finalPattern = pattern5 * (1 - blend) + pattern6 * blend; +} +``` + +### Pattern Design Ideas + +**Additive (Smooth)** +```javascript +const patternSmooth = ((t >> 1) + (t >> 3) + (t >> 5)) & 255; +``` + +**Multiplicative (Complex)** +```javascript +const patternComplex = (t * (t >> 4) * (t >> 8)) & 255; +``` + +**Modulo-based (Rhythmic)** +```javascript +const patternModulo = (t % 128 + (t >> 8) % 128) & 255; +``` + +**Conditional (Structured)** +```javascript +const patternConditional = (t & 128) ? (t << 1) & 255 : (t >> 1) & 255; +``` + +--- + +## Layer 3: Feedback Loop (Visual → Audio Influence) + +### Current System: Pixel Sampling → Parameter Modulation + +**Sampling strategy:** 12-20 points strategically distributed +- 4 corners (detect extreme brightness) +- 4 edge midpoints (detect edge activity) +- 4 diagonal sweeps (detect diagonal patterns) +- 4+ orbital scans (detect center/rotation) + +**Conversion:** +``` +RED channel → Harmonic scaling, time modulation +GREEN channel → Rhythm scaling, mix speed +BLUE channel → Pattern bias, shift modulation +Brightness → Intensity, chaos injection +Contrast → Bit operations +Variance → Chaos level +``` + +### Feedback Parameters Affected + +```javascript +timeModulation: How the time variable shifts (larger jumps = more chaotic) +shiftMod1/2: XOR shift amounts (bigger shifts = less repetitive) +harmonicScale: How many cycles the melody completes +rhythmScale: Speed of rhythmic modulation +bitMod1/2: Bit operation amounts (chaos injection) +mixSpeed: How fast patterns cycle through +blendIntensity: How smooth transitions are +chaosLevel: XOR noise injection probability +colorMod (r,g,b): Color channel multipliers (affects visuals) +``` + +### Variation: Change What Pixels Affect + +**Current: RGB brightness → Audio parameters** + +**Alternative 1: Directional Gradient** +```javascript +// Sample top half vs bottom half +const topSamples = sampleRegion(0, 0, width, height/2); +const bottomSamples = sampleRegion(0, height/2, width, height); +const topBrightness = avgBrightness(topSamples); +const bottomBrightness = avgBrightness(bottomSamples); + +feedback.mixSpeed = 0.5 + (topBrightness / 255) * 2; +feedback.chaosLevel = (bottomBrightness / 255); +``` + +**Alternative 2: Edge Detection** +```javascript +// High contrast areas → more complexity +const contrast = maxBrightness - minBrightness; +feedback.complexity = contrast / 255; +``` + +**Alternative 3: Color-Specific Regions** +```javascript +// Sample only red-dominant pixels +const redRegions = samples.filter(s => s.r > s.g && s.r > s.b); +feedback.intensity = redRegions.length / samples.length; +``` + +### Variation: Change Visual Effects from Audio + +The piece also **paints bytebeat patterns** back to the screen: + +**Current:** +```javascript +// For each pixel column: +const bytebeat = pattern(...); +const y = (bytebeat / 255) * screenHeight; +screen.pixels[y * width + x] = color; +``` + +**Alternative: Oscilloscope Mode** +```javascript +// Draw audio waveform like an oscilloscope +const samples = generator.bytebeat({ frequency, sampleRate, time, samplesNeeded: 512 }); +for (let i = 0; i < samples.length; i++) { + const y = (samples[i] * 0.5 + 0.5) * screenHeight; + const x = (i / samples.length) * screenWidth; + screen.pixels[Math.round(y * width + x)] = 255; +} +``` + +**Alternative: Spectrogram Mode** +```javascript +// Show frequency content over time +const frequencies = fft(bytebeat_output); +for (let freq = 0; freq < frequencies.length; freq++) { + const brightness = frequencies[freq]; + const y = (freq / frequencies.length) * screenHeight; + screen.pixels[Math.round(y * width + sweepX)] = brightness; +} +``` + +--- + +## Layer 4: Sequencing (Voice Lifecycle & Hold Mechanism) + +### Current Architecture: Hold Sequence + +**States:** +- **Off** - No automatic voices, only touch interaction +- **On** - Periodically spawns voices at orbital positions, 5-13 second durations + +**Parameters:** +```javascript +spawnInterval: 2000ms (spawn every 2 seconds) +maxConcurrentHolds: 6 (never more than 6 at once) +baseDuration: 5000-13000ms (influenced by chaos feedback) +orbitSpeed: 0.0003-0.0006 rad/frame (varies per voice) +wobble: 0.15-0.35 (influenced by memory) +``` + +**Spawning logic:** +``` +Position = orbital path (cosine × radius, sine × radius) + Radius influenced by feedback.density + Phase influenced by time + randomness +Duration = base + (1 - chaos) bonus - (1 - quiet bonus) + Less chaos → longer holds + High memory → longer holds +Movement = orbital drift + wobble + Each voice has independent orbital speed + Memory makes movements more pronounced +``` + +### Variation: Different Sequencing Strategies + +**1. Fibonacci Interval Spawning** +```javascript +const goldenRatio = 1.618; +const intervals = []; +for (let i = 0; i < 10; i++) { + intervals.push(Math.floor(1000 * Math.pow(goldenRatio, i))); +} +// Spawn voices at fibonacci-spaced intervals +``` + +**2. Grid-Based Spawning** +```javascript +// Spawn voices at fixed grid positions, one per cell +for (let gx = 0; gx < gridWidth; gx++) { + for (let gy = 0; gy < gridHeight; gy++) { + const x = (gx + 0.5) / gridWidth * screenWidth; + const y = (gy + 0.5) / gridHeight * screenHeight; + spawnVoiceAt(x, y, sound); + } +} +``` + +**3. Random Walk Sequencing** +```javascript +// Each voice position is random walk from previous +const walk = { x: screenWidth * 0.5, y: screenHeight * 0.5 }; +for (let i = 0; i < voiceCount; i++) { + walk.x += (Math.random() - 0.5) * 200; + walk.y += (Math.random() - 0.5) * 200; + walk.x = clamp(walk.x, 0, screenWidth); + walk.y = clamp(walk.y, 0, screenHeight); + spawnVoiceAt(walk.x, walk.y, sound); +} +``` + +**4. Brightness-Following Sequencing** +```javascript +// Spawn voices at brightest regions of screen +const samples = samplePixels(screen, 20); +const sorted = samples.sort((a, b) => b.brightness - a.brightness); +sorted.slice(0, 5).forEach(sample => { + spawnVoiceAt(sample.x, sample.y, sound); +}); +``` + +**5. Phase-Locking to Audio** +```javascript +// Spawn new voices synchronized to audio beat +const audioEnergy = measureAudioEnergy(sound); +if (audioEnergy > threshold && (now - lastSpawn) > spawnDelay) { + spawnHoldVoice(screenWidth, screenHeight, sound); + lastSpawn = now; +} +``` + +--- + +## Remix Guide: Creating Variations + +### Quick Swaps (30 minutes) + +**1. Change the color palette** +- Modify `touchOverlayPalette` (line 14-23) +- Modify color generation in `paint()` (line 558-560) + +**2. Change spatial mapping** +- Replace `mapXToFrequency()` and `mapYToPitchFactor()` +- E.g., use only vertical axis, or add diagonal + +**3. Adjust hold sequence timing** +- Change `spawnInterval` (currently 2000ms) +- Change hold duration calculation (currently 5-13 seconds) +- Change max concurrent holds (currently 6) + +**4. Modify feedback sensitivity** +- Increase/decrease pixel sampling points +- Change RGB→parameter mappings +- Adjust decay rates in `sim()` + +--- + +### Medium Swaps (1-2 hours) + +**1. Add a 6th bytebeat pattern** +- Design new pattern formula +- Insert into blending loop (change mod 5 to mod 6) +- Adjust blend transitions + +**2. Implement alternative sequencing** +- Comment out `updateHoldVoices()` +- Write new spawning logic +- Re-export or call from `sim()` + +**3. Change visual rendering** +- Modify pixel drawing (lines 509-598) +- Swap from vertical columns to orbits/grids/waveforms +- Add new visual effects (trails, particles, etc.) + +**4. Implement new feedback strategy** +- Rewrite `samplePixelFeedback()` +- Change what gets sampled (edges, variance, specific colors) +- Change RGB→parameter mappings + +--- + +### Deep Remixes (3-6 hours) + +**1. Multi-Layer Synthesis** +- Have different hold voices use different pattern sets +- E.g., lower voices use pattern 1-2, higher voices use 4-5 + +**2. Envelope Shaping** +- Add ADSR envelopes to voices +- Make volume/timbre evolve over hold duration + +**3. Harmonic Relationships** +- Make voices respond to each other +- E.g., new voice spawned at harmonic of existing voices + +**4. Spatial Audio Evolution** +- Make voices' frequency change as they move through space +- Create "force fields" where certain regions repel/attract + +**5. Generative Visual System** +- Decouple visuals from audio synthesis +- Create independent generative visual patterns +- Use audio to modulate visual parameters + +--- + +## Code Landmarks for Modification + +### To understand a layer, read these functions: + +**Spatial Mapping:** +- `mapXToFrequency()` (line 190) +- `mapYToPitchFactor()` (line 198) +- `deriveVoiceFrequency()` (line 205) + +**Synthesis:** +- `generator.bytebeat()` (line 61) +- Pattern definitions (lines 82-97) +- Pattern blending (lines 100-126) + +**Feedback:** +- `samplePixelFeedback()` (line 371) +- Sampling strategy (lines 379-424) +- Parameter derivation (lines 440-486) + +**Sequencing:** +- `spawnHoldVoice()` (line 370) +- `updateHoldVoices()` (line 406) +- `toggleHoldSequence()` (line 457) +- Hold state initialization (line 40) + +**Visuals:** +- `paint()` (line 525) +- Pixel rendering (lines 551-612) +- Color computation (lines 558-560) + +--- + +## Conceptual Symmetries + +Notice these patterns: + +1. **Feedback flows upward**: Pixels → Audio → Pixels +2. **Time operates at multiple scales**: + - Sample-level: Bytebeat generation (44.1kHz) + - Voice-level: Hold durations (seconds) + - System-level: State decay (10+ seconds) +3. **Randomness is constrained**: Random values modulated by feedback +4. **Movement is orbital**: Scanning, voice drift, visual sweeps all use trig functions +5. **Colors derive from bits**: RGB computed from bytebeat pattern XORs + +These symmetries are **features** you can exploit in variations: +- Use same orbital math for voices and pixel sampling +- Use same bytebeat generators for audio and visuals +- Use same feedback parameters to shape multiple layers + +--- + +## Testing Your Variations + +When you remix, test these: + +1. **With no touches** (hold sequence only) + - Does it sustain audio continuously? + - Are voices distinguishable or do they blend? + - Does visual feedback remain varied? + +2. **With touches** (interactive) + - Do touch voices feel responsive? + - Does hold sequence coexist peacefully? + - Are there frequency collisions (too many same-pitch voices)? + +3. **After 5 minutes idle** + - Does it settle to silence or continue? + - Do visuals accumulate wisely or become noise? + +4. **After 90 minutes** + - Would you listen to this as a tape? + - Is there enough emergence/surprise? + - Does it feel like a composition or just random? + +--- + +## Example Variations to Try + +### Variation A: "Comb Filter Seashells" +- Keep synthesis/feedback as-is +- Change `spawnInterval` to 500ms (faster) +- Spawn voices at fixed frequency ratios (1x, 1.5x, 2x, 3x fundamental) +- Result: Harmonic relationships, bell-like tones + +### Variation B: "Noise Garden" +- Keep synthesis/feedback as-is +- Add 2-3 new chaotic bytebeat patterns +- Increase `chaosLevel` sensitivity 5x +- Result: More glitchy, algorithmic harshness + +### Variation C: "Visual Instruments" +- Keep synthesis as-is +- Change visual rendering to oscilloscope +- Scale oscilloscope based on voice frequency +- High voices = small tight spirals, low voices = large loose ones +- Result: Visual becomes the primary interface, audio is secondary + +### Variation D: "Memory Piece" +- Keep synthesis as-is +- Make spawn rate depend on accumulated visual memory +- Bright areas → more voices spawn nearby +- Result: Visuals "grow" audio in response + +--- + +## Final Note + +The beauty of this piece is that **every layer is independent**. You can: +- Change synthesis without touching sequencing +- Change sequencing without touching visuals +- Change feedback without touching synthesis +- Change mapping without touching anything else + +This independence is intentional. It means you can remix safely, testing one change at a time, without breaking the whole system. + +Happy remixing! diff --git a/seashells_variation_examples.md b/seashells_variation_examples.md new file mode 100644 index 0000000000..d0997c7fbb --- /dev/null +++ b/seashells_variation_examples.md @@ -0,0 +1,474 @@ +# Seashells: Concrete Variation Examples + +Four complete variation sketches with copy-paste code. Each is a self-contained remix you can test. + +--- + +## Example 1: "Harmonic Bell" — Constrained Pitch Mapping + +**Concept:** Instead of a continuous frequency space, voices snap to a musical scale. Creates bell-like tones. + +**Key change:** Replace the `mapXToFrequency()` function + +```javascript +// Replace mapXToFrequency() with this: +function mapXToFrequencyQuantized(x, width) { + const w = Math.max(1, width - 1); + const nx = clamp((x ?? w / 2) / w, 0, 1); + + // C minor pentatonic scale + const notes = [ + 55, // A1 + 66, // B1 + 82, // E2 + 110, // A2 + 123, // B2 + 165, // E3 + 220, // A3 + 247, // B3 + 330, // E4 + 440 // A4 + ]; + + const index = Math.floor(nx * (notes.length - 1)); + return notes[index]; +} + +// Then in deriveVoiceFrequency(): +function deriveVoiceFrequency({ x, y, screenWidth, screenHeight }) { + const base = mapXToFrequencyQuantized(x, screenWidth) * mapYToPitchFactor(y, screenHeight); + return clamp(base, 55, 1760); +} +``` + +**What this does:** +- X-axis snaps to 10 specific pitches (A minor pentatonic) +- Y-axis still modulates pitch up/down +- Result: Naturally harmonious, bells/resonators + +**To test:** Press H, touch the left side, right side, watch the pitch snap between specific notes. + +--- + +## Example 2: "Chaos Intensifier" — Feedback-Driven Synthesis + +**Concept:** High visual variance → more chaotic audio. Creates feedback loops where visual complexity breeds audio wildness. + +**Changes:** +1. Increase chaos sensitivity in `samplePixelFeedback()` +2. Add new "chaos patterns" to synthesis + +```javascript +// In samplePixelFeedback(), find this line: +// chaosLevel: Math.min(1.0, variance / 20000), + +// Replace with: +chaosLevel: clamp(Math.sqrt(variance / 10000), 0, 1), // More sensitive + +// Then in the generator.bytebeat() function, find the chaos injection: +// if (liveFeedback && liveFeedback.chaosLevel > 0.5) { +// finalPattern = finalPattern ^ Math.floor(liveFeedback.chaosLevel * 128); +// } + +// Replace with: +if (liveFeedback && liveFeedback.chaosLevel > 0.3) { + const chaosAmount = Math.floor(liveFeedback.chaosLevel * 200); + finalPattern = (finalPattern ^ chaosAmount) + (chaosAmount >> 2) & 255; +} +``` + +**What this does:** +- Chaos level becomes much more sensitive (square root scaling) +- Chaos injection affects more bits +- Bright, contrasty visuals → immediately more chaotic audio + +**To test:** Press H, make the screen bright/contrasty with touches, watch audio become glitchier. + +--- + +## Example 3: "Grid Voices" — Spatial Voice Quantization + +**Concept:** Hold sequence spawns voices on a grid, creating structured movement patterns. + +**Replace the `spawnHoldVoice()` function:** + +```javascript +// Grid configuration +const gridConfig = { + cols: 4, + rows: 3, + cellIndex: 0 +}; + +function spawnHoldVoice(screenWidth, screenHeight, sound) { + const voiceId = holdSequence.nextVoiceId++; + const feedback = sharedPixelFeedback; + + // Get next grid position (row-major order) + const cellIndex = gridConfig.cellIndex % (gridConfig.cols * gridConfig.rows); + const col = cellIndex % gridConfig.cols; + const row = Math.floor(cellIndex / gridConfig.cols); + gridConfig.cellIndex += 1; + + // Convert grid to screen coordinates (with padding) + const padding = 40; + const cellWidth = (screenWidth - padding * 2) / gridConfig.cols; + const cellHeight = (screenHeight - padding * 2) / gridConfig.rows; + + const x = padding + (col + 0.5) * cellWidth; + const y = padding + (row + 0.5) * cellHeight; + + // Duration varies by grid position + const baseDuration = 4000 + (col + row) * 1000; + const duration = baseDuration + (Math.random() - 0.5) * 1000; + + const hold = { + voiceId, + x, + y, + startTime: performance.now(), + duration, + orbitPhase: 0, + orbitSpeed: 0.0001 + col * 0.00005 // Different speeds per column + }; + + startTouchVoice({ + pointerIndex: voiceId, + x: Math.round(x), + y: Math.round(y), + screenWidth, + screenHeight, + sound + }); + + holdSequence.activeHolds.push(hold); + holdSequence.lastSpawnTime = performance.now(); +} + +// In updateHoldVoices(), replace the orbital movement with: +function updateHoldVoices(screenWidth, screenHeight, sound) { + if (!holdSequence.enabled) return; + + const now = performance.now(); + const feedback = sharedPixelFeedback; + + // Spawn new hold if interval exceeded + if (now - holdSequence.lastSpawnTime > holdSequence.spawnInterval && holdSequence.activeHolds.length < 12) { + spawnHoldVoice(screenWidth, screenHeight, sound); + } + + // Update positions - GRID movement only (subtle vibrato) + for (let i = holdSequence.activeHolds.length - 1; i >= 0; i--) { + const hold = holdSequence.activeHolds[i]; + const elapsed = now - hold.startTime; + + if (elapsed > hold.duration) { + stopTouchVoice(hold.voiceId, 0.15); + holdSequence.activeHolds.splice(i, 1); + continue; + } + + // Grid position stays fixed, but add vibrato + const vibratoAmount = 10 + Math.sin(now * 0.003 + hold.voiceId) * 8; + const vibratoX = Math.sin(now * 0.004 + hold.voiceId * 0.5) * vibratoAmount; + const vibratoY = Math.cos(now * 0.005 + hold.voiceId * 0.7) * vibratoAmount; + + const x = hold.x + vibratoX; + const y = hold.y + vibratoY; + + updateTouchVoice({ + pointerIndex: hold.voiceId, + x: Math.round(x), + y: Math.round(y), + screenWidth, + screenHeight, + sound + }); + } +} +``` + +**What this does:** +- Voices spawn in a 4×3 grid and fill it sequentially +- Each voice has a fixed position with subtle vibrato +- Creates structured, predictable movement +- Different columns have different modulation speeds + +**To test:** Press H, watch voices fill grid positions systematically. + +--- + +## Example 4: "Waveform Display" — Visual Audio Feedback + +**Concept:** Instead of bytebeat creating vertical lines, show actual waveform shapes. More "traditional" audio visualization. + +**Replace most of the `paint()` function (lines 551-612):** + +```javascript +// In paint(), replace the main pixel-manipulation loop with: + +if (totalVoiceCount() === 0) { + wipe(10, 14, 22); + drawTouchMapping({ ink, line, write, screen, emphasized: true }); + ink(210, 232, 255); + write("hold touches to play / press 'h' for hold sequence", { x: 2, y: Math.max(hudSafeTop + 2, screen.height - 16) }, undefined, undefined, false, uiFont); + write("x=base hz y=pitch mult", { x: 2, y: Math.max(hudSafeTop + 10, screen.height - 8) }, undefined, undefined, false, uiFont); + return; +} + +// FEEDBACK LOOP +sharedPixelFeedback = samplePixelFeedback(screen); +const feedback = sharedPixelFeedback; + +// Generate waveform samples +const samplesPerFrame = screen.width; +const samples = generator.bytebeat({ + frequency: currentFrequency, + sampleRate: 44100, + time: performance.now() * 0.001, + samplesNeeded: samplesPerFrame, + feedback +}); + +// Draw waveform as oscilloscope-style +const centerY = screen.height * 0.5; +const amplitude = screen.height * 0.35; + +for (let x = 0; x < samplesPerFrame - 1; x++) { + const sample1 = samples[x]; + const sample2 = samples[x + 1]; + + const y1 = centerY - sample1 * amplitude; + const y2 = centerY - sample2 * amplitude; + + // Draw line between consecutive samples + drawLineBresenham( + Math.round(x), + Math.round(y1), + Math.round(x + 1), + Math.round(y2), + screen, + [200, 150, 255, 255] + ); +} + +// Draw baseline +for (let x = 0; x < screen.width; x++) { + const pixelIndex = (centerY * screen.width + x) * 4; + screen.pixels[pixelIndex] = 80; + screen.pixels[pixelIndex + 1] = 80; + screen.pixels[pixelIndex + 2] = 100; + screen.pixels[pixelIndex + 3] = 255; +} + +// Helper: simple Bresenham line drawing +function drawLineBresenham(x0, y0, x1, y1, screen, color) { + const dx = Math.abs(x1 - x0); + const dy = Math.abs(y1 - y0); + const sx = x0 < x1 ? 1 : -1; + const sy = y0 < y1 ? 1 : -1; + let err = dx - dy; + + let x = x0, y = y0; + while (true) { + if (x >= 0 && x < screen.width && y >= 0 && y < screen.height) { + const pixelIndex = (y * screen.width + x) * 4; + screen.pixels[pixelIndex] = color[0]; + screen.pixels[pixelIndex + 1] = color[1]; + screen.pixels[pixelIndex + 2] = color[2]; + screen.pixels[pixelIndex + 3] = color[3]; + } + + if (x === x1 && y === y1) break; + const e2 = 2 * err; + if (e2 > -dy) err -= dy, x += sx; + if (e2 < dx) err += dx, y += sy; + } +} + +drawTouchMapping({ ink, line, write, screen, emphasized: false }); +drawTouchOverlays({ ink, line, circle, write, screen }); +``` + +**What this does:** +- Shows actual audio waveform like an oscilloscope +- Waveform updates in real-time based on synthesized samples +- Visual directly represents what you're hearing +- Feedback loop still influences timbre + +**To test:** Press H, watch the waveform shape change as pattern blending happens. + +--- + +## Example 5: "Memory Painter" — Voices Follow Visual Entropy + +**Concept:** Voices spawn where the screen is most chaotic, creating a feedback where audio "grows" from visual disturbance. + +**Modify `spawnHoldVoice()`:** + +```javascript +function spawnHoldVoice(screenWidth, screenHeight, sound) { + const voiceId = holdSequence.nextVoiceId++; + const feedback = sharedPixelFeedback; + + // Sample multiple regions and find the most chaotic + const samplePoints = 16; + let maxChaos = 0; + let spawnX = screenWidth * 0.5; + let spawnY = screenHeight * 0.5; + + for (let i = 0; i < samplePoints; i++) { + const x = Math.random() * screenWidth; + const y = Math.random() * screenHeight; + + // Measure local entropy (variance of nearby pixels) + const regionSamples = []; + for (let dx = -10; dx <= 10; dx += 5) { + for (let dy = -10; dy <= 10; dy += 5) { + const px = clamp(Math.round(x + dx), 0, screenWidth - 1); + const py = clamp(Math.round(y + dy), 0, screenHeight - 1); + const pixelIndex = (py * screenWidth + px) * 4; + const brightness = screen.pixels[pixelIndex] + + screen.pixels[pixelIndex + 1] + + screen.pixels[pixelIndex + 2]; + regionSamples.push(brightness); + } + } + + // Compute variance + const avg = regionSamples.reduce((a, b) => a + b, 0) / regionSamples.length; + const variance = regionSamples.reduce((sum, val) => sum + Math.pow(val - avg, 2), 0) / regionSamples.length; + + if (variance > maxChaos) { + maxChaos = variance; + spawnX = x; + spawnY = y; + } + } + + const baseDuration = 5000 + (1 - feedback.chaosLevel) * 8000; + const duration = baseDuration + (Math.random() - 0.5) * 2000; + + const hold = { + voiceId, + x: spawnX, + y: spawnY, + startTime: performance.now(), + duration, + orbitPhase: 0, + orbitSpeed: 0.0002 + }; + + startTouchVoice({ + pointerIndex: voiceId, + x: Math.round(spawnX), + y: Math.round(spawnY), + screenWidth, + screenHeight, + sound + }); + + holdSequence.activeHolds.push(hold); + holdSequence.lastSpawnTime = performance.now(); +} +``` + +**What this does:** +- Analyzes visual entropy (how chaotic pixels are) +- Spawns new voices in the most chaotic regions +- Creates positive feedback: audio → pixels → more audio +- Visual "disturbances" are musically rewarded + +**To test:** Press H, touch screen to create visual chaos, watch new voices spawn there. + +--- + +## How to Implement These + +1. **Backup original:** + ```bash + cp system/public/aesthetic.computer/disks/seashells.mjs seashells_original.mjs + ``` + +2. **Pick one variation** (say, Harmonic Bell) + +3. **Copy its code** into seashells.mjs, replacing the specified functions + +4. **Test in dev environment:** + ```bash + npm run ac + # Navigate to seashells in browser + # Press H to activate hold sequence + ``` + +5. **Iterate:** Once you get one variation working, try others + +--- + +## Combining Variations + +You can **stack these concepts**: + +- Grid Voices + Harmonic Bell = Structured harmonic grid +- Chaos Intensifier + Waveform Display = Visual feedback of audio chaos +- Memory Painter + Grid Voices = Chaos accumulates in grid cells +- All four = Complex emergent system + +The trick is testing each change in isolation first, then carefully combining them. + +--- + +## Debugging Tips + +**If synthesis breaks (no sound):** +- Check that `currentFrequency` is in 20–20000 Hz range +- Verify generator.bytebeat returns 512+ samples +- Check sound.synth() is being called with correct parameters + +**If hold sequence doesn't work:** +- Verify `holdSequence.enabled` is toggled by 'H' key +- Check `spawnHoldVoice()` is being called from `updateHoldVoices()` +- Make sure `voiceId` values don't collide with touch pointer IDs + +**If visuals freeze:** +- Pixel manipulation loops might be expensive +- Reduce `samplesPerFrame` or `gridConfig` cell count +- Profile in DevTools Performance tab + +**If feedback loop breaks:** +- Verify `sharedPixelFeedback` is being updated in paint() +- Check pixel sampling doesn't go out of bounds +- Ensure feedback parameters scale to expected ranges + +--- + +## What To Listen For + +### Harmonic Bell +- Should sound like struck bells or gongs +- Quantized pitches mean less dissonance +- Movement within grid feels musically constrained + +### Chaos Intensifier +- Silent visuals = subtle, calm tone +- Complex/bright visuals = harsh, glitchy audio +- Real feedback loop, not just cosmetic + +### Grid Voices +- Predictable, structured movement +- Different columns have different "personalities" (timbre) +- Feels like an instrument you could learn to play + +### Waveform Display +- You see exactly what you hear +- Blending between patterns visible as shape changes +- Useful for understanding bytebeat architecture + +### Memory Painter +- Audio grows out of visual "accidents" +- Touching creates short-term audio response +- Over time, visual noise accumulates + +--- + +Happy remixing! Each variation teaches you something about how the layers interact. diff --git a/system/public/aesthetic.computer/disks/seash.mjs b/system/public/aesthetic.computer/disks/seash.mjs new file mode 100644 index 0000000000..07bce97d96 --- /dev/null +++ b/system/public/aesthetic.computer/disks/seash.mjs @@ -0,0 +1,518 @@ +// Seashell — bare-bones bytebeat synthesizer +// Lab bench proof of concept: minimal code, maximum clarity + +/* Core mechanism: + - 2 simple bytebeat patterns + - Hold sequence: spawn voices automatically + - X-axis = frequency, Y-axis = pitch multiplier + - No visual feedback loop, no complexity + - ~350 lines, easy to understand and modify +*/ + +// Voice management +const touchVoices = new Map(); +const maxTouchPointers = 8; + +// Hold sequence: auto-spawn voices +const holdSequence = { + enabled: false, + activeHolds: [], + lastSpawnTime: 0, + spawnInterval: 3000, // 3 seconds between spawns + nextVoiceId: 100 +}; + +// Parameters (controlled by sliders) +const params = { + spawnInterval: 3000, // ms between voice spawns + voiceDuration: 6000, // ms each voice lasts + orbitSpeed: 0.0003, // rad/frame orbital speed + blendSpeed: 0.1, // pattern blend rate + maxVoices: 5 // max concurrent auto voices +}; + +// Slider UI +const sliders = [ + { + label: "Spawn", + key: "spawnInterval", + min: 500, + max: 8000, + step: 100, + x: 0, + y: 0, + width: 0, + height: 16, + dragging: false + }, + { + label: "Duration", + key: "voiceDuration", + min: 2000, + max: 20000, + step: 500, + x: 0, + y: 0, + width: 0, + height: 16, + dragging: false + }, + { + label: "OrbitSpd", + key: "orbitSpeed", + min: 0.0001, + max: 0.001, + step: 0.00005, + x: 0, + y: 0, + width: 0, + height: 16, + dragging: false + }, + { + label: "BlendSpd", + key: "blendSpeed", + min: 0.01, + max: 0.5, + step: 0.02, + x: 0, + y: 0, + width: 0, + height: 16, + dragging: false + }, + { + label: "MaxVoices", + key: "maxVoices", + min: 1, + max: 10, + step: 1, + x: 0, + y: 0, + width: 0, + height: 16, + dragging: false + } +]; + +// Simple bytebeat generator +const generator = { + bytebeat: ({ frequency, sampleRate, time, samplesNeeded, feedback = null }) => { + const samples = []; + const freqScale = frequency / 440; + const timeOffset = Math.floor(time * sampleRate * freqScale * 0.3); + + for (let i = 0; i < samplesNeeded; i++) { + const t = timeOffset + i; + + // Pattern 1: XOR cascade (crisp, digital) + const p1 = (t ^ (t >> 8) ^ (t >> 9)) & 255; + + // Pattern 2: Melodic (pitched, harmonic) + const harmonic = Math.max(1, Math.floor(freqScale * 2)); + const p2 = ((t * harmonic) & (t >> 5) | (t >> 4)) & 255; + + // Mix patterns based on time + const mixPhase = (time * 0.1 + freqScale * 0.5) % 2; + let finalPattern; + if (mixPhase < 1) { + const blend = mixPhase; + finalPattern = p1 * (1 - blend) + p2 * blend; + } else { + const blend = mixPhase - 1; + finalPattern = p2 * (1 - blend) + p1 * blend; + } + + // Convert to audio range + let sample = (finalPattern / 127.5) - 1; + sample *= 0.6; // Volume scaling + samples.push(sample); + } + return samples; + } +}; + +function clamp(value, low, high) { + return Math.max(low, Math.min(high, value)); +} + +// Spatial mapping +function mapXToFrequency(x, width) { + const nx = clamp(x / width, 0, 1); + const minHz = 55; + const maxHz = 880; + return minHz * Math.pow(maxHz / minHz, nx); +} + +function mapYToPitch(y, height) { + const ny = clamp(y / height, 0, 1); + return Math.pow(2, (0.5 - ny) * 2); // 0.5x to 2x +} + +function deriveFrequency(x, y, screenWidth, screenHeight) { + const base = mapXToFrequency(x, screenWidth) * mapYToPitch(y, screenHeight); + return clamp(base, 40, 2000); +} + +// Voice management +function createVoice({ sound, frequency, volume = 0.5 }) { + return sound.synth({ + type: "custom", + tone: frequency, + duration: "🔁", + volume, + generator: generator.bytebeat + }); +} + +function startTouchVoice({ pointerIndex, x, y, screenWidth, screenHeight, sound }) { + const key = `touch-${pointerIndex}`; + if (touchVoices.has(key)) return; + + const frequency = deriveFrequency(x, y, screenWidth, screenHeight); + const voice = createVoice({ sound, frequency, volume: 0.5 }); + + touchVoices.set(key, { sound: voice, x, y, frequency }); + rebalanceVolumes(); +} + +function updateTouchVoice({ pointerIndex, x, y, screenWidth, screenHeight }) { + const key = `touch-${pointerIndex}`; + const voice = touchVoices.get(key); + if (!voice) return; + + voice.x = x; + voice.y = y; + + const targetFrequency = deriveFrequency(x, y, screenWidth, screenHeight); + if (Math.abs(targetFrequency - voice.frequency) > 0.5) { + voice.frequency += (targetFrequency - voice.frequency) * 0.15; + voice.sound?.update?.({ tone: voice.frequency }); + } +} + +function stopTouchVoice(pointerIndex, fade = 0.1) { + const key = `touch-${pointerIndex}`; + const voice = touchVoices.get(key); + if (!voice) return; + voice.sound?.kill(fade); + touchVoices.delete(key); + rebalanceVolumes(); +} + +function stopAllVoices(fade = 0.1) { + for (const voice of touchVoices.values()) { + voice.sound?.kill(fade); + } + touchVoices.clear(); +} + +function rebalanceVolumes() { + const count = touchVoices.size; + if (count <= 0) return; + const baseVolume = clamp(0.5 / Math.sqrt(count), 0.15, 0.4); + for (const voice of touchVoices.values()) { + voice.sound?.update?.({ volume: baseVolume }); + } +} + +// Hold sequence: auto-voice generation +function spawnHoldVoice(screenWidth, screenHeight, sound) { + const voiceId = holdSequence.nextVoiceId++; + + // Orbital position + const angle = (performance.now() * 0.0001) + Math.random() * Math.PI * 2; + const radius = 0.35; + const x = (Math.cos(angle) * radius + 0.5) * screenWidth; + const y = (Math.sin(angle) * radius + 0.5) * screenHeight; + + // Duration with randomness (from params) + const duration = params.voiceDuration + (Math.random() - 0.5) * (params.voiceDuration * 0.5); + + const hold = { + voiceId, + x, + y, + startTime: performance.now(), + duration, + orbitPhase: angle, + orbitSpeed: params.orbitSpeed + Math.random() * (params.orbitSpeed * 0.5) + }; + + startTouchVoice({ + pointerIndex: voiceId, + x: Math.round(x), + y: Math.round(y), + screenWidth, + screenHeight, + sound + }); + + holdSequence.activeHolds.push(hold); + holdSequence.lastSpawnTime = performance.now(); +} + +function updateHoldVoices(screenWidth, screenHeight, sound) { + if (!holdSequence.enabled) return; + + const now = performance.now(); + + // Spawn new voice if needed (use params) + if (now - holdSequence.lastSpawnTime > params.spawnInterval && holdSequence.activeHolds.length < params.maxVoices) { + spawnHoldVoice(screenWidth, screenHeight, sound); + } + + // Update existing holds + for (let i = holdSequence.activeHolds.length - 1; i >= 0; i--) { + const hold = holdSequence.activeHolds[i]; + const elapsed = now - hold.startTime; + + if (elapsed > hold.duration) { + stopTouchVoice(hold.voiceId, 0.1); + holdSequence.activeHolds.splice(i, 1); + continue; + } + + // Orbital movement + hold.orbitPhase += hold.orbitSpeed; + const x = (Math.cos(hold.orbitPhase) * 0.3 + 0.5) * screenWidth; + const y = (Math.sin(hold.orbitPhase) * 0.3 + 0.5) * screenHeight; + + updateTouchVoice({ + pointerIndex: hold.voiceId, + x: Math.round(x), + y: Math.round(y), + screenWidth, + screenHeight + }); + } +} + +function toggleHoldSequence(screenWidth, screenHeight, sound) { + if (holdSequence.enabled) { + for (const hold of holdSequence.activeHolds) { + stopTouchVoice(hold.voiceId, 0.08); + } + holdSequence.activeHolds = []; + holdSequence.enabled = false; + } else { + holdSequence.enabled = true; + holdSequence.nextVoiceId = 100; + spawnHoldVoice(screenWidth, screenHeight, sound); + } +} + +// Slider helpers +function drawSliders({ ink, write, screen }) { + const sliderAreaHeight = sliders.length * 20 + 10; + const sliderY = screen.height - sliderAreaHeight; + + // Draw slider area background + ink(8, 10, 16); + for (let y = sliderY; y < screen.height; y++) { + for (let x = 0; x < screen.width; x++) { + // Just set background by clearing that area when we draw + } + } + + // Position sliders + const labelWidth = 55; + const sliderWidth = screen.width - labelWidth - 15; + + sliders.forEach((slider, i) => { + slider.y = sliderY + i * 20 + 5; + slider.x = labelWidth; + slider.width = sliderWidth; + slider.height = 14; + + // Draw label + ink(140, 160, 190); + write(slider.label, { x: 5, y: slider.y }, undefined, undefined, false, "MatrixChunky8"); + + // Draw slider background + ink(30, 40, 60); + for (let sx = slider.x; sx < slider.x + slider.width; sx++) { + write("_", { x: sx, y: slider.y }, undefined, undefined, false, "MatrixChunky8"); + } + + // Draw slider handle + const normalizedValue = (params[slider.key] - slider.min) / (slider.max - slider.min); + const handleX = slider.x + Math.floor(normalizedValue * slider.width); + + ink(100, 180, 220); + write("■", { x: handleX, y: slider.y }, undefined, undefined, false, "MatrixChunky8"); + + // Draw value + ink(180, 200, 230); + let displayValue = params[slider.key]; + if (slider.step < 1) { + displayValue = displayValue.toFixed(5); + } else { + displayValue = Math.round(displayValue); + } + write(`${displayValue}`, { x: slider.x + slider.width + 5, y: slider.y }, undefined, undefined, false, "MatrixChunky8"); + }); +} + +function checkSliderClick(x, y) { + for (let i = 0; i < sliders.length; i++) { + const slider = sliders[i]; + if (y >= slider.y && y < slider.y + slider.height && x >= slider.x && x < slider.x + slider.width) { + return i; + } + } + return -1; +} + +function updateSlider(sliderIndex, x) { + if (sliderIndex < 0 || sliderIndex >= sliders.length) return; + const slider = sliders[sliderIndex]; + const relativeX = clamp(x - slider.x, 0, slider.width); + const normalizedValue = relativeX / slider.width; + const newValue = slider.min + normalizedValue * (slider.max - slider.min); + params[slider.key] = newValue; +} + +// Rendering +function paint({ wipe, ink, write, screen, box }) { + const voiceCount = touchVoices.size; + + // Clear screen + wipe(10, 12, 18); + + // Draw simple grid (frequency reference) + ink(40, 50, 70); + for (let x = 0; x < screen.width; x += Math.floor(screen.width / 8)) { + for (let y = 0; y < screen.height - 110; y += Math.floor(screen.height / 8)) { + write("·", { x, y }, undefined, undefined, false, "MatrixChunky8"); + } + } + + // Draw frequency labels + ink(180, 200, 230); + write(`${Math.round(mapXToFrequency(0, screen.width))}Hz`, { x: 2, y: 2 }, undefined, undefined, false, "MatrixChunky8"); + write(`${Math.round(mapXToFrequency(screen.width, screen.width))}Hz`, { x: screen.width - 35, y: 2 }, undefined, undefined, false, "MatrixChunky8"); + + // Draw status + ink(220, 240, 255); + const status = holdSequence.enabled ? "HOLD: ON" : "HOLD: OFF"; + write(status, { x: 2, y: screen.height - 130 }, undefined, undefined, false, "MatrixChunky8"); + write(`Voices: ${voiceCount}`, { x: screen.width - 50, y: screen.height - 130 }, undefined, undefined, false, "MatrixChunky8"); + + // Draw voice positions + ink(100, 180, 220, 100); + for (const [key, voice] of touchVoices.entries()) { + const x = Math.round(voice.x); + const y = Math.round(voice.y); + + // Draw circle + ink(150, 200, 255, 150); + write("●", { x, y }, undefined, undefined, false, "MatrixChunky8"); + + // Draw frequency label + ink(200, 220, 255); + write(`${Math.round(voice.frequency)}`, { x: x + 3, y: y - 2 }, undefined, undefined, false, "MatrixChunky8"); + } + + // Help text + if (voiceCount === 0) { + ink(180, 200, 230); + write("Touch to play | Press H for hold", { x: 2, y: Math.floor(screen.height / 2) - 50 }, undefined, undefined, false, "MatrixChunky8"); + } + + // Draw sliders + drawSliders({ ink, write, screen }); +} + +// Input handling +function act({ event: e, sound, screen, pens }) { + // Keyboard: H toggles hold sequence + if (e.is("keyboard:down:h")) { + toggleHoldSequence(screen.width, screen.height, sound); + } + + // Slider interaction (check for slider touches first) + for (let i = 1; i <= maxTouchPointers; i++) { + if (e.is(`touch:${i}`)) { + const pointer = pens?.(i); + const x = pointer?.x ?? e.x; + const y = pointer?.y ?? e.y; + + const sliderIndex = checkSliderClick(x, y); + if (sliderIndex >= 0) { + sliders[sliderIndex].dragging = i; + updateSlider(sliderIndex, x); + return; // Don't create voice if touching slider + } + } + + if (e.is(`draw:${i}`)) { + const pointer = pens?.(i); + const x = pointer?.x ?? e.x; + const y = pointer?.y ?? e.y; + + // Check if this was a slider drag + if (sliders.some(s => s.dragging === i)) { + const sliderIndex = sliders.findIndex(s => s.dragging === i); + updateSlider(sliderIndex, x); + return; + } + + // Otherwise update touch voice + updateTouchVoice({ pointerIndex: i, x, y, screenWidth: screen.width, screenHeight: screen.height }); + } + + if (e.is(`lift:${i}`)) { + // Check if this was a slider + const sliderIndex = sliders.findIndex(s => s.dragging === i); + if (sliderIndex >= 0) { + sliders[sliderIndex].dragging = false; + return; + } + + // Otherwise stop touch voice + stopTouchVoice(i, 0.08); + } + } + + // Touch/mouse input (only if not on slider) + for (let i = 1; i <= maxTouchPointers; i++) { + if (e.is(`touch:${i}`)) { + const pointer = pens?.(i); + const x = pointer?.x ?? e.x; + const y = pointer?.y ?? e.y; + if (checkSliderClick(x, y) < 0) { + startTouchVoice({ pointerIndex: i, x, y, screenWidth: screen.width, screenHeight: screen.height, sound }); + } + } + } + + // Fallback for single-touch environments + if (e.is("touch")) { + if (checkSliderClick(e.x, e.y) < 0) { + startTouchVoice({ pointerIndex: 1, x: e.x, y: e.y, screenWidth: screen.width, screenHeight: screen.height, sound }); + } + } + if (e.is("draw")) { + updateTouchVoice({ pointerIndex: 1, x: e.x, y: e.y, screenWidth: screen.width, screenHeight: screen.height }); + } + if (e.is("lift")) { + stopTouchVoice(1, 0.08); + } +} + +// Per-frame updates +function sim({ sound, screen }) { + sound.speaker?.poll(); + updateHoldVoices(screen.width, screen.height, sound); +} + +// Initialization +function boot({ hud }) { + // Runs once at startup +} + +// Cleanup +function leave() { + stopAllVoices(0.05); +}