autotune algorithms — non-neural offline pitch correction #

scope: the actual internals of pitch-correction, with code we can lift. neural systems (rvc, diffsinger, etc.) are the other agent's beat. this is dsp, c, and python — the stuff that runs on jeffrey's 8gb macbook without a gpu and finishes before lunch.

the problem with pop/bin/pitchsnap.mjs today: it slices a vocal by word and runs rubberband -p N per segment. that's a static shift — every sample inside the word slides up the same n semitones. the source's natural pitch contour rides along, which is the opposite of autotune. autotune clamps — it replaces f0 with a target curve.

1. what autotune actually does — five sentences #

step 1: estimate f0 (pitch) frame-by-frame on the input, ~5 ms hop. step 2: for each frame, snap the detected midi note to the nearest member of an allowed scale (with hysteresis so a wobble around the boundary doesn't flip notes every frame). step 3: compute a per-frame correction ratio target_f0 / source_f0. step 4: shift the audio by that ratio while preserving formants — so the timbre stays human, the pitch becomes mechanical. step 5: stitch frames back together with phase coherence (or, in psola/world world, resynth from decomposed parameters).

that's it. the magic is entirely in steps 1, 4, and 5 being good.

2. pitch detection — yin / pyin #

naive autocorrelation (what pitchsnap.mjs uses today) finds octave errors because the autocorrelation peak at lag t is matched at 2t, 3t, 4t, etc. any noise pushes the search to the wrong harmonic. yin (de cheveigné & kawahara, 2002) fixes this with the cumulative mean normalized difference function — the curve starts at 1 and only dips below when a true period is found, with the lower-lag (higher pitch) match preferred.

# from patriceguyot/Yin (mit license) — compact form
def differenceFunction(x, w, tau_max):
    # d(tau) = sum_i (x[i] - x[i+tau])^2, computed via fft for speed
    x_cumsum = np.concatenate(([0.], (x*x).cumsum()))
    fc = np.fft.rfft(x, size_pad)
    conv = np.fft.irfft(fc * fc.conjugate())[:tau_max]
    return x_cumsum[w:w-tau_max:-1] + x_cumsum[w] - x_cumsum[:tau_max] - 2*conv

def cmndf(df, N):
    # normalize so curve starts at 1, drops < 1 at true period
    return np.insert(df[1:] * range(1, N) / np.cumsum(df[1:]), 0, 1)

def getPitch(cmdf, tau_min, tau_max, harmo_th=0.1):
    tau = tau_min
    while tau < tau_max:
        if cmdf[tau] < harmo_th:
            while tau+1 < tau_max and cmdf[tau+1] < cmdf[tau]:
                tau += 1   # walk down to local minimum
            return tau
        tau += 1
    return 0  # no pitch found (unvoiced)

source: patriceguyot/Yin/yin.py, also vendored into nvidia/mellotron. pyin (mauch & dixon, 2014) extends yin with a probabilistic hmm over candidate periods — much more robust on noisy/breathy vocals. librosa.yin and librosa.pyin ship both.

3. pitch shifting — psola vs phase vocoder vs world resynth #

three families, each with a tradeoff:

td-psola (charpentier & stella, 1986). estimate f0, find pitch-period peaks, extract two-period hann-windowed grains, re-space the grains at the target period, overlap-add. key property: formants are intrinsic to the grain shape, so they survive untouched — no separate formant flag needed. weak on polyphonic / unvoiced material. clean reference: sannawag/TD-PSOLA:

def shift_pitch(signal, fs, f_ratio):
    peaks = find_peaks(signal, fs)        # autocorr-based pitch-mark detection
    return psola(signal, peaks, f_ratio)  # window+respace+overlap-add

phase vocoder (flanagan & golden, 1966; dolson 1986). stft → for each bin, estimate true frequency from the phase derivative across hops → resynthesize at scaled frame rate → resample to original duration. this is what librosa's effects.pitch_shift does (time-stretch via phase vocoder, then resample). also the engine inside rubber band's R2 mode. weakness: "phasiness" — bins drift apart, transients smear. fixes (laroche & dolson 1999, "phase locking") are what rubber band's R3 (--finer) mode adds.

world resynth (morise et al., 2016). decompose speech into three independent streams: f0, smoothed spectral envelope (cheaptrick), aperiodicity (d4c). modify any of them. resynth. this is the cleanest "replace the f0 curve" pipeline that exists outside the neural world — see section 4.

method	formants	best on	failure mode
td-psola	preserved (intrinsic)	clean monophonic vocals	breathy / unvoiced
phase vocoder	requires extra filter	broadband / polyphonic	phasiness, transient smear
world resynth	preserved (envelope split out)	speech / vocals	not free of artifacts on heavy distortion

4. world / pyworld — the f0-replace pipeline #

this is the one. pyworld is the python wrapper around morise's c library. install: pip install pyworld. has wheels for arm64 macos. the canonical autotune flow is six lines:

import pyworld as pw

# 1. decompose
_f0, t  = pw.dio(x, fs)             # raw f0 candidates (or pw.harvest for accuracy)
f0      = pw.stonemask(x, _f0, t, fs)   # refine f0 to sub-frame precision
sp      = pw.cheaptrick(x, f0, t, fs)   # smoothed spectral envelope (formants)
ap      = pw.d4c(x, f0, t, fs)          # aperiodicity (breath, fricatives)

# 2. modify f0 — this is where autotune happens
f0_corrected = quantize_to_scale(f0, scale_midi_notes)  # see section 5

# 3. resynthesize
y = pw.synthesize(f0_corrected, sp, ap, fs)

note what's not here: there is no pitch-shift step. you literally write the new f0 curve into the synthesizer. sp (formants) and ap (breath texture) are unchanged, so the speaker's identity survives perfectly. this is the api surface we want.

harvest is slower but recovers from the octave errors that bite dio on jeffrey's takes. for vocals always prefer harvest + stonemask.

5. scale quantization with hysteresis #

given an instantaneous f0 in hz and a target scale (set of midi numbers), the snap is straightforward — hz → midi via 69 + 12*log2(f/440), then nearest-member lookup. the trick is hysteresis: if you're sitting between two scale degrees and the f0 wobbles ±10 cents, naive nearest-snap will flip notes every frame, producing a digital trill. fix:

def quantize_to_scale(f0, scale_midi, prev_target=None,
                      hysteresis_cents=30, retain=1.0):
    out = np.zeros_like(f0)
    cur = prev_target
    for i, f in enumerate(f0):
        if f <= 0:                      # unvoiced — keep silent
            out[i] = 0; continue
        midi = 69 + 12*np.log2(f/440)
        # candidates sorted by distance to current frame
        cands = sorted(scale_midi, key=lambda n: abs(n - midi))
        nearest = cands[0]
        if cur is not None and nearest != cur:
            # only switch if we've moved > hysteresis cents past the boundary
            if abs(midi - nearest)*100 + hysteresis_cents > abs(midi - cur)*100:
                nearest = cur
        cur = nearest
        # `retain` interpolates between source pitch (0) and full snap (1)
        corrected_midi = midi + retain * (nearest - midi)
        out[i] = 440 * 2**((corrected_midi - 69)/12)
    return out

retain=1.0 is full hard auto-tune (the t-pain sound). retain=0.6 is melodyne-style — keeps human bend. hysteresis_cents=30 to 50 kills the trill without slowing legit pitch transitions.

6. shortlist — what we can plug in tonight #

lib	language	offline?	install	what we get
pyworld	python	yes	`pip install pyworld`	f0 extract + replace + resynth, formants intrinsic
psola	python	yes	`pip install psola`	td-psola via parselmouth/praat — needs target-pitch numpy array
librosa	python	yes	`pip install librosa`	`yin`, `pyin`, `effects.pitch_shift` (phase vocoder)
crepe	python	gpu-light	`pip install crepe`	neural pitch detector — slower but cleanest f0
autotalent	c (ladspa)	yes	source build	reference c implementation, gpl2
rubberband	c++	yes	`brew install rubberband` (already installed)	phase vocoder pitch shift, no f0-replace
sannawag/TD-PSOLA	python	yes	`git clone`	minimal readable td-psola reference

all of the python options install on apple silicon python 3.14 with native wheels in 2026. pyworld's only build dep is numpy + cython.

7. integration sketch — pitchsnap.mjs → world #

the right move is to delegate the per-word pitch step to a small python helper that does world-style f0 replacement. pitchsnap.mjs keeps its responsibilities (whisper alignment, score parsing, grid snap, slice extraction) and calls the helper for the actual correction:

# pop/bin/pitchsnap_world.py — called per-word from pitchsnap.mjs
import sys, numpy as np, pyworld as pw, soundfile as sf

in_wav, out_wav, target_midi, retain = sys.argv[1:5]
target_midi = float(target_midi); retain = float(retain)

x, fs = sf.read(in_wav, dtype="float64")
if x.ndim > 1: x = x.mean(axis=1)

f0_raw, t = pw.harvest(x, fs, f0_floor=80, f0_ceil=600)
f0  = pw.stonemask(x, f0_raw, t, fs)
sp  = pw.cheaptrick(x, f0, t, fs)
ap  = pw.d4c(x, f0, t, fs)

# replace f0 with target — hold the bend with `retain`
target_hz = 440 * 2**((target_midi - 69)/12)
voiced = f0 > 0
f0_new = np.where(voiced, np.exp((1-retain)*np.log(np.maximum(f0,1e-6))
                                  + retain*np.log(target_hz)), 0)

y = pw.synthesize(f0_new, sp, ap, fs)
sf.write(out_wav, y.astype(np.float32), fs)

then in pitchsnap.mjs, replace the rubberband call inside the per-word loop with:

spawnSync("python3", [
  resolve(import.meta.dirname, "pitchsnap_world.py"),
  sliceWav, shiftedWav, String(targetMidi), String(retain)
], { stdio: "inherit" });

next phase: instead of a single target_midi per word, write a per-frame target curve that interpolates between syllable notes — that's the syllable-glide we already plan in pitchsnap.mjs curve mode, but executed by world instead of cross-fading two rubberband renders.

recommendation: integrate pyworld first. it's the only option in this list that lets us replace the f0 curve instead of shifting it, formants stay intact for free, and the install is one line.

pip install pyworld soundfile numpy