# Clock `{#code}` Stample Feature — Design Report

**Date:** January 26, 2026  
**Author:** Generated by Copilot  
**Status:** ✅ Implemented

---

## Table of Contents

1. [Overview](#overview)
2. [Current Architecture](#current-architecture)
3. [Proposed `{#code}` Syntax](#proposed-code-syntax)
4. [Multi-Stample Support](#multi-stample-support)
5. [Syntax Highlighting](#syntax-highlighting)
6. [Implementation Plan](#implementation-plan)
7. [Implementation Summary](#implementation-summary)

---

## Overview

This report documents the design and implementation of `{#code}` support in the clock piece, allowing samples to be loaded from networked paintings using the same short-code system that stample already supports.

### User Goal
```
clock {#abc}cdefg {#xyz}abcd
```
- Load painting `#abc` as a stample, play notes `cdefg` using that sample
- Separate track using painting `#xyz`, play notes `abcd` using that sample

---

## Current Architecture

### Stample in `stample.mjs`

The stample piece already supports loading paintings by code:

```javascript
// In boot():
if (decodedParam.startsWith("#")) {
  bitmapLoading = true;
  bitmapLoaded = false;
  await loadPaintingCode(decodedParam, { get, preload, sound });
}

// loadPaintingCode function:
async function loadPaintingCode(code, { get, preload, sound }) {
  const normalized = decodeURIComponent(code).replace(/^#/, "");
  
  // Fast path for short codes: /media/paintings/{code}.png
  if (normalized.length <= 6) {
    const directUrl = `${baseUrl}/media/paintings/${normalized}.png?t=${Date.now()}`;
    const directImg = await preload(directUrl, true);
    // Convert to audio via decodeBitmapToSample()
  }
  
  // Fallback: resolve via /api/painting-code → get.painting()
}
```

The encoding scheme uses RGB channels (3 samples per pixel):
- `encodeSampleToBitmap(data)` — float32 samples → RGBA pixels
- `decodeBitmapToSample(bitmap)` — RGBA pixels → float32 samples

### Stample in `clock.mjs`

Previously clock loaded **one** stample from `store["stample:sample"]`:

```javascript
// In boot():
if (store?.retrieve) {
  const storedSample = await store.retrieve("stample:sample", "local:db");
  if (storedSample?.data?.length) {
    stampleSampleId = storedSample.id || "stample";
    stampleSampleData = storedSample.data;
    stampleSampleRate = storedSample.sampleRate;
    sound?.registerSample?.(storedId, storedSample.data, storedSample.sampleRate);
  }
}
```

When `{stample}` or `{sample}` waveform is used:

```javascript
// In createManagedSound():
if (waveType === "stample" || waveType === "sample") {
  const sampleId = stampleSampleId || fallbackSfx;
  synthInstance = sound.play(sampleId, {
    volume, pitch: finalFreq, loop: !struck
  });
}
```

### Melody Parser

The melody parser already handles waveform syntax in braces:

```javascript
// In parseMelody():
if (char === '{') {
  const content = melodyString.substring(i + 1, endBrace);
  
  // Check for waveform types
  if (['sine', 'sawtooth', 'square', 'triangle', 'noise-white', 
       'sample', 'stample', 'custom', 'bubble'].includes(contentLower)) {
    currentWaveType = contentLower;
  }
}
```

---

## Proposed `{#code}` Syntax

### New Syntax

```
{#abc123}           — Load painting code "abc123" as active stample
{#abc123:0.5}       — Load painting + set volume to 50%
{stample}           — Use last-loaded stample (existing behavior)
{sample}            — Alias for {stample}
```

### Melody Examples

```
clock {#abc123} cdefg                    — Load painting, play notes with that sample
clock {#abc123} ceg {#xyz789} dfa        — Switch samples mid-melody
clock {#abc123:0.8} cdefg {sine:0.5} ab  — Mix sample and synth
```

### Parsing Changes

In `melody-parser.mjs`, extend the `{...}` handler:

```javascript
// Inside parseMelody() where char === '{'
if (contentLower.startsWith('#')) {
  // This is a painting code reference
  const [codePart, volumePart] = content.split(':');
  const paintingCode = codePart.slice(1); // Remove # prefix
  
  currentWaveType = 'stample';  // Set waveform type
  currentStampleCode = paintingCode; // NEW: Track which code to load
  
  if (volumePart) {
    const vol = parseFloat(volumePart);
    if (!isNaN(vol) && vol >= 0 && vol <= 1) {
      currentVolume = vol;
    }
  }
}
```

Each note object would then include:

```javascript
{
  note: 'c',
  octave: 4,
  waveType: 'stample',
  stampleCode: 'abc123',  // NEW field
  volume: 0.8,
  // ...
}
```

---

## Multi-Stample Support

### Challenge

Currently clock has single global variables:
```javascript
let stampleSampleId = null;
let stampleSampleData = null;
let stampleSampleRate = null;
```

For multiple codes, we need a **cache map**.

### Proposed Solution

```javascript
// Replace single variables with a cache
const stampleCache = new Map(); // code -> { sampleId, data, sampleRate, loaded, loading }

// Track which codes are used in the current melody
let requiredStampleCodes = new Set();

// During boot, after parsing melody:
requiredStampleCodes = extractStampleCodes(melodyState);

// Preload all required stamples
for (const code of requiredStampleCodes) {
  loadStampleCode(code, { get, preload, sound });
}
```

### Loading Function

Adapt from `stample.mjs`:

```javascript
async function loadStampleCode(code, { get, preload, sound }) {
  if (stampleCache.has(code)) return; // Already loaded or loading
  
  stampleCache.set(code, { loading: true, loaded: false });
  
  try {
    // Fast path: /media/paintings/{code}.png
    const directUrl = `/media/paintings/${code}.png?t=${Date.now()}`;
    const img = await preload(directUrl, true);
    const buffer = await imageToBuffer(img);
    
    // Decode RGB bitmap to audio samples
    const decoded = decodeBitmapToSample(buffer);
    const sampleId = `stample:${code}`;
    
    sound?.registerSample?.(sampleId, decoded, 48000);
    
    stampleCache.set(code, {
      sampleId,
      data: decoded,
      sampleRate: 48000,
      loaded: true,
      loading: false
    });
    
    console.log(`🎵 Clock: Loaded stample #${code}`);
  } catch (err) {
    console.warn(`🎵 Clock: Failed to load stample #${code}:`, err);
    stampleCache.set(code, { loading: false, loaded: false, error: err });
  }
}
```

### Playback Changes

In `createManagedSound()`:

```javascript
if (waveType === "stample" || waveType === "sample") {
  // Check for specific code, otherwise use default
  const code = note.stampleCode;
  let sampleId;
  
  if (code && stampleCache.has(code)) {
    const cached = stampleCache.get(code);
    sampleId = cached.loaded ? cached.sampleId : fallbackSfx;
  } else {
    sampleId = stampleSampleId || fallbackSfx;
  }
  
  if (sampleId) {
    synthInstance = sound.play(sampleId, {
      volume, pitch: finalFreq, loop: !struck
    });
  }
}
```

---

## Syntax Highlighting

### Current Highlighting Colors

From `buildColoredMelodyStringUnified()` in clock.mjs:

| Element | Color | When Active |
|---------|-------|-------------|
| Note letters | Gray → RGB note color | Before timing starts → After |
| Current note | Red | While playing |
| Mutated notes | Goldenrod | When mutated |
| `{` `}` | Yellow | Always |
| Waveform content | Cyan | Inside braces |
| Mutation `*` | Rainbow (animated) | Always |
| Spaces | Yellow/Gray | Separator |
| `>` | Cyan | Sequence separator |
| Loop count | Magenta | Before `>` |
| Disabled `x` | Brown | Group prefix |

### Proposed Highlighting for `{#code}`

| Element | Color | Condition |
|---------|-------|-----------|
| `{` `}` | Yellow | Always |
| `#` | Orange | Code indicator |
| Code chars | **Cyan** (loading) | While loading |
| Code chars | **Lime** (loaded) | After successful load |
| Code chars | **Red** (error) | If load failed |
| `:` | Yellow | Volume separator |
| Volume | Cyan | Volume value |

### Implementation in `buildColoredMelodyStringUnified()`

```javascript
// Inside the waveform content handling
if (inWaveformForColoring) {
  if (waveformContent.startsWith('#')) {
    // Check load status of this code
    const code = waveformContent.slice(1).split(':')[0];
    const cached = stampleCache.get(code);
    
    if (cached?.loaded) {
      color = "lime";       // Successfully loaded
    } else if (cached?.loading) {
      color = "cyan";       // Still loading
    } else if (cached?.error) {
      color = "red";        // Failed to load
    } else {
      color = "orange";     // Not yet started loading
    }
  } else {
    color = "cyan";         // Regular waveform content
  }
}
```

### Visual Feedback During Load

```
Before:  clock {#abc123} cdefg     — #abc123 in orange
Loading: clock {#abc123} cdefg     — #abc123 in cyan (animated?)
Success: clock {#abc123} cdefg     — #abc123 in lime ✓
Error:   clock {#abc123} cdefg     — #abc123 in red ✗
```

---

## Implementation Plan

### Phase 1: Core Infrastructure

1. **Add `stampleCache` map** to clock.mjs
2. **Create `loadStampleCode()` function** (adapt from stample.mjs)
3. **Copy helper functions** from stample.mjs:
   - `decodeBitmapToSample()`
   - `imageToBuffer()`

### Phase 2: Melody Parser

4. **Extend `{...}` parsing** in melody-parser.mjs to detect `#code` syntax
5. **Add `stampleCode` field** to note objects
6. **Export `extractStampleCodes()` helper** to collect all codes from parsed melody

### Phase 3: Boot & Preload

7. **In `boot()`**: After parsing melody, preload all required stample codes
8. **Track loading state** for UI feedback

### Phase 4: Playback

9. **Update `createManagedSound()`** to use cached stample by code
10. **Handle missing/loading samples** gracefully (use fallback)

### Phase 5: Syntax Highlighting

11. **Update `buildColoredMelodyStringUnified()`** for `#code` coloring
12. **Add load status colors** (lime/cyan/red)
13. **Trigger repaint** when load status changes

### Phase 6: Testing

14. Test single `{#code}` usage
15. Test multiple different codes in one melody
16. Test switching between code and default stample
17. Test mixing codes with synth waveforms
18. Test error handling (invalid code, network failure)

---

## Open Questions

### 1. Preload vs Lazy Load?

**Option A: Preload all in boot (recommended)**
- Pros: Samples ready before playback, no timing issues
- Cons: Longer boot time if many codes

**Option B: Lazy load on first use**
- Pros: Faster boot for unused codes
- Cons: Playback delay on first note, timing issues

**Recommendation:** Preload all during boot with progress indicator.

### 2. Cache Persistence?

Should loaded stamples be cached to IndexedDB like regular stample?

**Option A: Memory only**
- Cleared on refresh
- Simpler implementation

**Option B: Persist to store**
- Survives refresh
- Uses more storage
- Key pattern: `stample:code:{code}`

### 3. What if code doesn't exist?

Options:
1. Play silence (rest)
2. Use fallback synth (sine)
3. Use default stample (if any)
4. Show error in HUD

**Implementation:** Uses fallback synth with visual indication (red highlighting).

### 4. Maximum concurrent samples?

Should we limit how many different codes can be loaded?
- Web Audio typically handles ~32 concurrent audio sources
- Memory concern for very long samples

**Implementation:** No hard limit, but uses lazy loading and caching.

### 5. Volume normalization?

Different paintings may encode samples at different levels.
Should we auto-normalize loaded samples?

**Recommendation:** Future enhancement - add optional `{#code:0.5}` volume multiplier per-code.

---

## Implementation Summary

### Files Modified

#### 1. `melody-parser.mjs`
- Added `currentStampleCode` variable to track active painting code
- Extended `{...}` brace parsing to detect `#code` prefix
- Added `stampleCode` field to all note objects (7 locations)
- Added `extractStampleCodes(melodyState)` export function

#### 2. `clock.mjs`
- Added `extractStampleCodes` to imports
- Added `stampleCache` Map for caching loaded samples
- Added `netPreload` and `soundRef` module references
- Added helper functions:
  - `decodeBitmapToSample(bitmap, meta)` — RGB→audio conversion
  - `imageToBuffer(image)` — Image→pixel buffer
  - `loadStampleCode(code, {preload, sound})` — Async lazy loader
  - `getStampleByCode(code, {preload, sound})` — Get cached or trigger load
- Updated `createManagedSound()` signature to include `stampleCode` parameter
- Updated stample playback logic to check painting code cache first
- Updated 4 `createManagedSound()` call sites to pass `stampleCode`
- Updated `buildColoredMelodyStringUnified()` for syntax highlighting:
  - Lime = loaded
  - Cyan = loading
  - Light red = error
  - Orange = not started

### Usage

```
clock {#abc}cdefg
```
Loads painting `#abc` from `/media/paintings/abc.png`, decodes RGB→audio, plays notes c-d-e-f-g using that sample.

```
clock {#abc}ceg {#xyz}dfa
```
Two parallel tracks: first uses painting `#abc`, second uses painting `#xyz`.

### Cache Structure

```javascript
stampleCache = Map {
  "abc" => {
    sampleId: "stample-abc",
    data: Float32Array,
    sampleRate: 44100,
    loaded: true,
    loading: false,
    error: null
  },
  ...
}
```

### Syntax Highlighting Colors

| State | Color |
|-------|-------|
| Loaded | Lime |
| Loading | Cyan |
| Error | Light Red |
| Not Started | Orange |
| Regular waveform | Cyan |

---

## Related Files

| File | Changes |
|------|---------|
| [clock.mjs](../system/public/aesthetic.computer/disks/clock.mjs) | Core implementation ✅ |
| [melody-parser.mjs](../system/public/aesthetic.computer/lib/melody-parser.mjs) | Parse `{#code}` syntax ✅ |
| [stample.mjs](../system/public/aesthetic.computer/disks/stample.mjs) | Reference implementation |

---

## References

- [Clock Syntax Reference](./clock-syntax-and-sequence-advancement.md) — Current syntax documentation
- [Clock Sigil Shortcodes Plan](../plans/clock-sigil-shortcodes.md) — `*code` shortcode system
- Stample piece source for bitmap encoding/decoding

---

## Summary

The `{#code}` feature has been implemented, enabling:
- ✅ Loading samples from networked paintings by short code
- ✅ Multiple different samples in a single melody (via parallel tracks)
- ✅ Visual feedback (syntax highlighting) for load status
- ✅ Lazy loading with session-persistent caching
- ✅ Seamless fallback to synth or default stample when loading/error

The implementation reuses the RGB encoding scheme from stample.mjs and integrates cleanly with the existing melody parser and clock playback infrastructure.