Clock {#code} Stample Feature — Design Report #

Date: January 26, 2026
Author: Generated by Copilot
Status: ✅ Implemented

---

Table of Contents #

1. Overview
2. Current Architecture
3. Proposed {#code} Syntax
4. Multi-Stample Support
5. Syntax Highlighting
6. Implementation Plan
7. Implementation Summary

---

Overview #

This report documents the design and implementation of {#code} support in the clock piece, allowing samples to be loaded from networked paintings using the same short-code system that stample already supports.

User Goal #

clock {#abc}cdefg {#xyz}abcd

• Load painting #abc as a stample, play notes cdefg using that sample
• Separate track using painting #xyz, play notes abcd using that sample

---

Current Architecture #

Stample in stample.mjs #

The stample piece already supports loading paintings by code:

// In boot():
if (decodedParam.startsWith("#")) {
  bitmapLoading = true;
  bitmapLoaded = false;
  await loadPaintingCode(decodedParam, { get, preload, sound });
}

// loadPaintingCode function:
async function loadPaintingCode(code, { get, preload, sound }) {
  const normalized = decodeURIComponent(code).replace(/^#/, "");
  
  // Fast path for short codes: /media/paintings/{code}.png
  if (normalized.length <= 6) {
    const directUrl = `${baseUrl}/media/paintings/${normalized}.png?t=${Date.now()}`;
    const directImg = await preload(directUrl, true);
    // Convert to audio via decodeBitmapToSample()
  }
  
  // Fallback: resolve via /api/painting-code → get.painting()
}

The encoding scheme uses RGB channels (3 samples per pixel):

• encodeSampleToBitmap(data) — float32 samples → RGBA pixels
• decodeBitmapToSample(bitmap) — RGBA pixels → float32 samples

Stample in clock.mjs #

Previously clock loaded one stample from store["stample:sample"]:

// In boot():
if (store?.retrieve) {
  const storedSample = await store.retrieve("stample:sample", "local:db");
  if (storedSample?.data?.length) {
    stampleSampleId = storedSample.id || "stample";
    stampleSampleData = storedSample.data;
    stampleSampleRate = storedSample.sampleRate;
    sound?.registerSample?.(storedId, storedSample.data, storedSample.sampleRate);
  }
}

When {stample} or {sample} waveform is used:

// In createManagedSound():
if (waveType === "stample" || waveType === "sample") {
  const sampleId = stampleSampleId || fallbackSfx;
  synthInstance = sound.play(sampleId, {
    volume, pitch: finalFreq, loop: !struck
  });
}

Melody Parser #

The melody parser already handles waveform syntax in braces:

// In parseMelody():
if (char === '{') {
  const content = melodyString.substring(i + 1, endBrace);
  
  // Check for waveform types
  if (['sine', 'sawtooth', 'square', 'triangle', 'noise-white', 
       'sample', 'stample', 'custom', 'bubble'].includes(contentLower)) {
    currentWaveType = contentLower;
  }
}

---

Proposed {#code} Syntax #

New Syntax #

{#abc123}           — Load painting code "abc123" as active stample
{#abc123:0.5}       — Load painting + set volume to 50%
{stample}           — Use last-loaded stample (existing behavior)
{sample}            — Alias for {stample}

Melody Examples #

clock {#abc123} cdefg                    — Load painting, play notes with that sample
clock {#abc123} ceg {#xyz789} dfa        — Switch samples mid-melody
clock {#abc123:0.8} cdefg {sine:0.5} ab  — Mix sample and synth

Parsing Changes #

In melody-parser.mjs, extend the {...} handler:

// Inside parseMelody() where char === '{'
if (contentLower.startsWith('#')) {
  // This is a painting code reference
  const [codePart, volumePart] = content.split(':');
  const paintingCode = codePart.slice(1); // Remove # prefix
  
  currentWaveType = 'stample';  // Set waveform type
  currentStampleCode = paintingCode; // NEW: Track which code to load
  
  if (volumePart) {
    const vol = parseFloat(volumePart);
    if (!isNaN(vol) && vol >= 0 && vol <= 1) {
      currentVolume = vol;
    }
  }
}

Each note object would then include:

{
  note: 'c',
  octave: 4,
  waveType: 'stample',
  stampleCode: 'abc123',  // NEW field
  volume: 0.8,
  // ...
}

---

Multi-Stample Support #

Challenge #

Currently clock has single global variables:

let stampleSampleId = null;
let stampleSampleData = null;
let stampleSampleRate = null;

For multiple codes, we need a cache map.

Proposed Solution #

// Replace single variables with a cache
const stampleCache = new Map(); // code -> { sampleId, data, sampleRate, loaded, loading }

// Track which codes are used in the current melody
let requiredStampleCodes = new Set();

// During boot, after parsing melody:
requiredStampleCodes = extractStampleCodes(melodyState);

// Preload all required stamples
for (const code of requiredStampleCodes) {
  loadStampleCode(code, { get, preload, sound });
}

Loading Function #

Adapt from stample.mjs:

async function loadStampleCode(code, { get, preload, sound }) {
  if (stampleCache.has(code)) return; // Already loaded or loading
  
  stampleCache.set(code, { loading: true, loaded: false });
  
  try {
    // Fast path: /media/paintings/{code}.png
    const directUrl = `/media/paintings/${code}.png?t=${Date.now()}`;
    const img = await preload(directUrl, true);
    const buffer = await imageToBuffer(img);
    
    // Decode RGB bitmap to audio samples
    const decoded = decodeBitmapToSample(buffer);
    const sampleId = `stample:${code}`;
    
    sound?.registerSample?.(sampleId, decoded, 48000);
    
    stampleCache.set(code, {
      sampleId,
      data: decoded,
      sampleRate: 48000,
      loaded: true,
      loading: false
    });
    
    console.log(`🎵 Clock: Loaded stample #${code}`);
  } catch (err) {
    console.warn(`🎵 Clock: Failed to load stample #${code}:`, err);
    stampleCache.set(code, { loading: false, loaded: false, error: err });
  }
}

Playback Changes #

In createManagedSound():

if (waveType === "stample" || waveType === "sample") {
  // Check for specific code, otherwise use default
  const code = note.stampleCode;
  let sampleId;
  
  if (code && stampleCache.has(code)) {
    const cached = stampleCache.get(code);
    sampleId = cached.loaded ? cached.sampleId : fallbackSfx;
  } else {
    sampleId = stampleSampleId || fallbackSfx;
  }
  
  if (sampleId) {
    synthInstance = sound.play(sampleId, {
      volume, pitch: finalFreq, loop: !struck
    });
  }
}

---

Syntax Highlighting #

Current Highlighting Colors #

From buildColoredMelodyStringUnified() in clock.mjs:

Element	Color	When Active
Note letters	Gray → RGB note color	Before timing starts → After
Current note	Red	While playing
Mutated notes	Goldenrod	When mutated
{ }	Yellow	Always
Waveform content	Cyan	Inside braces
Mutation *	Rainbow (animated)	Always
Spaces	Yellow/Gray	Separator
>	Cyan	Sequence separator
Loop count	Magenta	Before >
Disabled x	Brown	Group prefix

Proposed Highlighting for {#code} #

Element	Color	Condition
{ }	Yellow	Always
#	Orange	Code indicator
Code chars	Cyan (loading)	While loading
Code chars	Lime (loaded)	After successful load
Code chars	Red (error)	If load failed
:	Yellow	Volume separator
Volume	Cyan	Volume value

Implementation in buildColoredMelodyStringUnified() #

// Inside the waveform content handling
if (inWaveformForColoring) {
  if (waveformContent.startsWith('#')) {
    // Check load status of this code
    const code = waveformContent.slice(1).split(':')[0];
    const cached = stampleCache.get(code);
    
    if (cached?.loaded) {
      color = "lime";       // Successfully loaded
    } else if (cached?.loading) {
      color = "cyan";       // Still loading
    } else if (cached?.error) {
      color = "red";        // Failed to load
    } else {
      color = "orange";     // Not yet started loading
    }
  } else {
    color = "cyan";         // Regular waveform content
  }
}

Visual Feedback During Load #

Before:  clock {#abc123} cdefg     — #abc123 in orange
Loading: clock {#abc123} cdefg     — #abc123 in cyan (animated?)
Success: clock {#abc123} cdefg     — #abc123 in lime ✓
Error:   clock {#abc123} cdefg     — #abc123 in red ✗

---

Implementation Plan #

Phase 1: Core Infrastructure #

1. Add stampleCache map to clock.mjs
2. Create loadStampleCode() function (adapt from stample.mjs)
3. Copy helper functions from stample.mjs:
   • decodeBitmapToSample()
   • imageToBuffer()

Phase 2: Melody Parser #

4. Extend {...} parsing in melody-parser.mjs to detect #code syntax
5. Add stampleCode field to note objects
6. Export extractStampleCodes() helper to collect all codes from parsed melody

Phase 3: Boot & Preload #

7. In boot(): After parsing melody, preload all required stample codes
8. Track loading state for UI feedback

Phase 4: Playback #

9. Update createManagedSound() to use cached stample by code
10. Handle missing/loading samples gracefully (use fallback)

Phase 5: Syntax Highlighting #

11. Update buildColoredMelodyStringUnified() for #code coloring
12. Add load status colors (lime/cyan/red)
13. Trigger repaint when load status changes

Phase 6: Testing #

14. Test single {#code} usage
15. Test multiple different codes in one melody
16. Test switching between code and default stample
17. Test mixing codes with synth waveforms
18. Test error handling (invalid code, network failure)

---

Open Questions #

1. Preload vs Lazy Load? #

Option A: Preload all in boot (recommended)

• Pros: Samples ready before playback, no timing issues
• Cons: Longer boot time if many codes

Option B: Lazy load on first use

• Pros: Faster boot for unused codes
• Cons: Playback delay on first note, timing issues

Recommendation: Preload all during boot with progress indicator.

2. Cache Persistence? #

Should loaded stamples be cached to IndexedDB like regular stample?

Option A: Memory only

• Cleared on refresh
• Simpler implementation

Option B: Persist to store

• Survives refresh
• Uses more storage
• Key pattern: stample:code:{code}

3. What if code doesn't exist? #

Options:

1. Play silence (rest)
2. Use fallback synth (sine)
3. Use default stample (if any)
4. Show error in HUD

Implementation: Uses fallback synth with visual indication (red highlighting).

4. Maximum concurrent samples? #

Should we limit how many different codes can be loaded?

• Web Audio typically handles ~32 concurrent audio sources
• Memory concern for very long samples

Implementation: No hard limit, but uses lazy loading and caching.

5. Volume normalization? #

Different paintings may encode samples at different levels. Should we auto-normalize loaded samples?

Recommendation: Future enhancement - add optional {#code:0.5} volume multiplier per-code.

---

Implementation Summary #

Files Modified #

1. melody-parser.mjs #

• Added currentStampleCode variable to track active painting code
• Extended {...} brace parsing to detect #code prefix
• Added stampleCode field to all note objects (7 locations)
• Added extractStampleCodes(melodyState) export function

2. clock.mjs #

• Added extractStampleCodes to imports
• Added stampleCache Map for caching loaded samples
• Added netPreload and soundRef module references
• Added helper functions:
  • decodeBitmapToSample(bitmap, meta) — RGB→audio conversion
  • imageToBuffer(image) — Image→pixel buffer
  • loadStampleCode(code, {preload, sound}) — Async lazy loader
  • getStampleByCode(code, {preload, sound}) — Get cached or trigger load
• Updated createManagedSound() signature to include stampleCode parameter
• Updated stample playback logic to check painting code cache first
• Updated 4 createManagedSound() call sites to pass stampleCode
• Updated buildColoredMelodyStringUnified() for syntax highlighting:
  • Lime = loaded
  • Cyan = loading
  • Light red = error
  • Orange = not started

Usage #

clock {#abc}cdefg

Loads painting #abc from /media/paintings/abc.png, decodes RGB→audio, plays notes c-d-e-f-g using that sample.

clock {#abc}ceg {#xyz}dfa

Two parallel tracks: first uses painting #abc, second uses painting #xyz.

Cache Structure #

stampleCache = Map {
  "abc" => {
    sampleId: "stample-abc",
    data: Float32Array,
    sampleRate: 44100,
    loaded: true,
    loading: false,
    error: null
  },
  ...
}

Syntax Highlighting Colors #

State	Color
Loaded	Lime
Loading	Cyan
Error	Light Red
Not Started	Orange
Regular waveform	Cyan

---

Related Files #

File	Changes
clock.mjs	Core implementation ✅
melody-parser.mjs	Parse {#code} syntax ✅
stample.mjs	Reference implementation

---

References #

• Clock Syntax Reference — Current syntax documentation
• Clock Sigil Shortcodes Plan — *code shortcode system
• Stample piece source for bitmap encoding/decoding

---

Summary #

The {#code} feature has been implemented, enabling:

• ✅ Loading samples from networked paintings by short code
• ✅ Multiple different samples in a single melody (via parallel tracks)
• ✅ Visual feedback (syntax highlighting) for load status
• ✅ Lazy loading with session-persistent caching
• ✅ Seamless fallback to synth or default stample when loading/error

The implementation reuses the RGB encoding scheme from stample.mjs and integrates cleanly with the existing melody parser and clock playback infrastructure.