diff --git a/docs/superpowers/specs/2026-07-06-markpub-content-design.md b/docs/superpowers/specs/2026-07-06-markpub-content-design.md new file mode 100644 index 0000000..53bf058 --- /dev/null +++ b/docs/superpowers/specs/2026-07-06-markpub-content-design.md @@ -0,0 +1,156 @@ +# Markpub content member — design + +**Date:** 2026-07-06 +**Status:** approved + +## Motivation + +The `site.standard.document` records ghoststandard writes carry no `content` +union member at all — only `textContent`, a plain-text extraction of the +Ghost post HTML. A standard.site viewer can show a preview but cannot render +the post body. + +markpub.at (`at.markpub.markdown` / `at.markpub.text`, repo AramZS/markpub.at) +is a community lexicon for exactly this slot, designed to nest inside +standard.site documents. gifthood.org adopted it on 2026-07-06 +(spec `2026-07-06-markpub-content-shape-design.md` in that repo); this brings +ghoststandard to the same contract so any markpub-aware reader can render +posts from either project. + +Unlike gifthood, this is purely additive: there is no legacy content shape to +keep reading, no read path at all (ghoststandard is a write-only bridge — +Ghost is the source of truth, deletes go through the local SQLite mapping), +and no blob problem (Ghost posts reference images by absolute URL on Ghost's +own hosting, so they survive as plain `![](url)` markdown). + +## The wrinkle: derived markdown + +Ghost's Admin API emits `html`, `lexical`/`mobiledoc`, and `plaintext` — never +markdown (since Ghost 5, markdown is just a card type inside Lexical). So the +emitted markdown is **derived** from `post.html` via turndown + the GFM +plugin. Consequences accepted during brainstorming: + +- Standard blog content (headings, prose, links, images, lists, code, tables) + converts cleanly. Exotic Ghost cards degrade: embed iframes vanish (no text + content), bookmark cards collapse to links, galleries flatten to image + lines. Acceptable — `textContent` has the same limits today. +- `flavor: "gfm"` is declared (turndown's GFM plugin output). +- `renderingRules` is **omitted** — ghoststandard never renders from this + markdown (Ghost renders from Lexical), and the lexicon says to leave the + field out in that case. This deliberately differs from gifthood, where + markdown is the source and `renderingRules: "marked"` is a true statement. + +## Decisions (settled during brainstorming) + +1. **Converter: turndown + turndown-plugin-gfm** — de-facto standard, + extensible rule system for Ghost's kg-cards. (Ghost emitting markdown + directly was checked and ruled out; see above.) +2. **Tests: mirror gifthood.** Vendor the two markpub lexicon JSONs as + fixtures and validate the emitted member with `@atproto/lexicon`. This + introduces the repo's first test setup (vitest). +3. **One targeted kg-card rule**, everything else turndown defaults. Ghost + wraps every editor card in `
`; a single + rule for captioned image figures covers the common case without a + per-card-type maintenance treadmill. +4. **Emitted fields:** `$type`, `text: { markdown }`, `flavor: "gfm"`. + `renderingRules`, `textBlob`, `facets`, `lenses`, `frontMatter` omitted + (all optional; nothing consumes them — YAGNI). +5. **Structure: a dedicated content-shape module** (`src/lib/content-shape.ts`), + matching gifthood's layout — one file owns the union member and the + turndown configuration, so future markpub evolution touches one place. + +## Out of scope + +- Any upstream markpub proposal (that conversation is driven from gifthood). +- Changes to `textContent`, cover-image handling, codeinjection, webhooks, or + any other document field — the `content` member is the sole record change. +- Rendering or reading markdown back (no read path exists). +- Per-site opt-out of the content member. + +## 1. New module: `src/lib/content-shape.ts` + +- `MARKPUB_MARKDOWN_TYPE = 'at.markpub.markdown'`. +- A module-level configured `TurndownService`: `headingStyle: 'atx'`, + `codeBlockStyle: 'fenced'`, GFM plugin applied (tables, strikethrough). +- One rule for Ghost captioned-image figures + (`
` containing `img` + `figcaption`): + image line followed by the caption on its own line. All other markup rides + turndown defaults. +- `buildMarkpubContent(html: string): Record | null` — + returns `null` when the input HTML is empty/whitespace-only, when the + *converted markdown* comes out empty/whitespace-only (e.g. an embed-only + post), **or if turndown throws** (conversion failure is logged and + swallowed; a conversion bug must never fail a sync — the record simply + omits `content`). Otherwise: + + ```ts + { + $type: 'at.markpub.markdown', + text: { markdown }, + flavor: 'gfm', + } + ``` + +## 2. Write side: `transform.ts` + +`ghostPostToRecord` builds the member from `post.html` and attaches it +alongside the other optional fields: + +```ts +const content = buildMarkpubContent(post.html ?? ''); +if (content) record.content = content; +``` + +A comment notes that the `site.standard.document` content union is +`closed: false`, so the member needs no lexicon registration — the value is +the shared `$type`. `textContent` stays exactly as-is (it serves +search/preview consumers; the markdown serves renderers). + +## 3. Dependencies and typing + +- Runtime deps: `turndown`, `turndown-plugin-gfm`. +- Dev deps: `@types/turndown`, `vitest`, `@atproto/lexicon` (versions pinned + from `npm view version` at implementation time). +- `turndown-plugin-gfm` ships no types: add an ambient declaration to the + existing `src/types.d.ts` (same loose-shim pattern as `@tryghost/admin-api` + and `html-to-text`). +- New package script: `"test": "vitest run"` — the repo's first. + +## 4. Vendored lexicons + tests + +- Copy `src/lexicons/markpub/{markdown,text}.json` and the fixtures README + verbatim from gifthood.org (source: AramZS/markpub.at, MIT, + `dist/lexicons/at/markpub/`, commit `9b53a3a8f93d`, fetched 2026-07-06). + Fixtures only — never loaded at runtime; a schema drift on refresh breaks + `npm test`, acting as a tripwire rather than a production dependency. +- New `src/lib/content-shape.test.ts`: + - loads the vendored schemas into a `Lexicons` instance and asserts + `buildMarkpubContent(html)` output passes validation for + `at.markpub.markdown`; + - conversion cases: prose/headings/links, kg-image-card figure with + caption, code card → fenced block, GFM table, embed iframe → dropped + (and an embed-only post → `null`), empty HTML → `null`. +- New `src/lib/transform.test.ts`: the record carries `content` for a post + with HTML and omits it for a post without. + +## 5. Migration: `gs backfill` + +No new mechanism. After deploy, run `gs backfill` once per site — it re-syncs +every published post via `putRecord` with stable rkeys (the Ghost post id), +so every record gains the `content` member in place. Spot-check afterward: +`content.$type`, `content.text.markdown` renders sensibly, `textContent` +unchanged. + +README gets one line in the "What ghoststandard does" bullet noting the post +body travels as `at.markpub.markdown` content. + +## 6. Error handling & testing posture + +No new failure modes: the writer emits a constant shape or nothing, and the +conversion guard in `buildMarkpubContent` means a turndown failure degrades +to today's behavior (record without `content`). Record size is a non-issue — +the markdown roughly doubles the existing `textContent` payload, far below +PDS record limits for blog-sized posts. + +Gates: `npm run typecheck` and `npm test` before every commit. Live pass: +dev publish → record shows the markpub member → `gs backfill` → spot-check.