diff --git a/README.md b/README.md index cf7a81e..cd8b790 100644 --- a/README.md +++ b/README.md @@ -73,6 +73,20 @@ Many events carry only a street address, no coordinates — so they never surfac **Backfilling an existing corpus.** The drip only trickles, so to resolve a backlog run the off-Cloudflare CLI against the deployed D1: `pnpm -C apps/web geocode:backfill --limit 50`. It reaches D1 over the REST API, so it needs `CLOUDFLARE_ACCOUNT_ID` / `CLOUDFLARE_API_TOKEN` / `D1_DATABASE_ID` and `MEILI_URL` / `MEILI_KEY` (plus `SEARCH_INDEX` if not `events`), and a LocationIQ `GEOCODER_URL` / `GEOCODER_KEY`. It refuses a bulk or uncapped run against public Nominatim (keyless is capped to `--limit 1..25`). Useful flags: `--limit N` (`0` = no cap), `--dry-run`, `--retry-negative` (re-attempt negatively-cached addresses), and `--allow-public-nominatim` (override the public-host guard). Like search itself, geocoding only helps once the sink is feeding the index, so run this after the rollout steps above. +## Load-more pagination + +Every paginated list — the home events feed, a profile's hosting and past events, a topic, and search — shares one continuation mechanism, so "load more" always resumes the same query on the same backend that produced page 1. + +**Why a self-describing token.** Load-more used to have the client echo back the page-1 query parameters and let the server re-derive which backend to use from the request shape ("is a search term set, and is Meilisearch configured?"). That inference broke whenever a page's first load and its load-more resolved to different backends. A D1 keyset fed to Meilisearch became `Number(base64url)`, which is `NaN`, which collapses to offset 0 — a relevance-reordered duplicate of page 1. A Meilisearch offset fed to D1 was ignored, silently dropping the upcoming and discoverable filters page 1 had applied. + +**The envelope.** The continuation cursor is now an opaque, self-contained token: `base64url(JSON { v, q, args?, raw })`. `q` names the server-side query (`events`, `hosting`, `past-events`, `topic`, `search-d1`, or `search-meili`). `args` carries only public-safe scope choices (a profile actor, a topic slug, the "popular" toggle). `raw` is the opaque backend-native cursor to resume from. The client treats the whole token as a blob and echoes it back unchanged. Load-more looks `q` up in a registry of resumers (`events-load-more.ts`) and re-runs that query with server-authoritative filter values; the client supplies neither the pipeline nor any filter. + +**Why that's safe.** Because every filter value lives server-side in the registry entry, a tampered token cannot widen what it sees — it can only name another already-public query or fail to decode. The unlisted-inclusive plain `listRecords` pipeline deliberately has no registry entry, so no cursor can reach it. `decodeCursor` never throws and returns null on anything malformed, so load-more simply ends pagination cleanly. A deep-linked `?cursor=` only resumes when the envelope was minted for the *same* query — same `q` **and** the same public-safe scope (topic slug, profile actor, or popular/all toggle); a keyset is specific to its result set, so a `technology` topic cursor, another actor's keyset, or a `popular` cursor under `?filter=all` all start a fresh page 1 rather than resuming a foreign position. Search (`search-d1`/`search-meili`) deep-links never resume: their defining term rides `?q=`, not the envelope, so an inbound cursor can't be proven to match the route's term. + +**Backend-native cursors.** `raw` stays opaque and backend-specific: a D1 keyset built inside `@atmo-dev/contrail`, or a Meilisearch offset that `tagCursor` prefixes as `meili:`. `tagCursor` and `parseCursor` in `cursor.ts` are that Meilisearch offset codec, also used by near-me; the envelope simply wraps whatever they produce. Cursors minted before the envelope existed (top-level `meili:` or `d1:` tags, or bare offsets) are tolerated as a deploy-window courtesy — they fail to decode and end pagination cleanly rather than resuming incorrectly — and that tolerance can be dropped once such cursors have drained. + +**The pieces.** `cursor.ts` handles envelope encode/decode and backend tagging. `events-load-more.ts` holds the resumer registry and the shared load-more handler. Each route's `+page.server.ts` mints the page-1 envelope from its own filters, and `EventList.svelte` echoes the token on "load more". Adding a query or backend means registering a resumer, not editing a branch. + ## contributing open for contributions by all :) diff --git a/apps/web/src/lib/components/EventList.svelte b/apps/web/src/lib/components/EventList.svelte index 8c6853b..abf88f8 100644 --- a/apps/web/src/lib/components/EventList.svelte +++ b/apps/web/src/lib/components/EventList.svelte @@ -9,14 +9,19 @@ cursor, handles = {}, actor = undefined, - fetchParams, + q = undefined, gridClass = 'grid gap-6 sm:grid-cols-2' }: { events: FlatEventRecord[]; cursor: string | null; handles?: Record; actor?: string | undefined; - fetchParams: Record; + // The cursor is now a fully opaque, self-describing continuation envelope: + // load-more POSTs only { cursor }, no client-echoed pipeline/ + // filters. The single exception is the free-text search TERM, which stays + // OUT of the envelope and rides here so the search page's load-more can + // re-run its query; other pages leave it undefined. + q?: string | undefined; gridClass?: string; } = $props(); @@ -43,19 +48,13 @@ loading = true; try { - const params: Record = {}; - for (const [key, value] of Object.entries(fetchParams)) { - if (key === 'limit' || key === 'rsvpsGoingCountMin' || key === 'rsvpsCountMin') { - params[key] = Number(value); - } else if (key === 'profiles') { - params[key] = value === 'true'; - } else { - params[key] = value; - } - } - params.cursor = currentCursor; - - const result = await loadMoreEvents(params as Parameters[0]); + // Opaque token in, opaque token out: the envelope names the server-side + // query, so there is no client-side query reconstruction to echo. Only + // the search term (when present) rides alongside the cursor. + const result = await loadMoreEvents({ + cursor: currentCursor, + ...(q !== undefined ? { q } : {}) + }); extraEvents = [...extraEvents, ...result.events]; currentCursor = result.cursor; diff --git a/apps/web/src/routes/(app)/events/+page.server.ts b/apps/web/src/routes/(app)/events/+page.server.ts index 787b2bb..15b64ee 100644 --- a/apps/web/src/routes/(app)/events/+page.server.ts +++ b/apps/web/src/routes/(app)/events/+page.server.ts @@ -3,7 +3,7 @@ import { getServerClient, listDiscoverableEventsFromContrail } from '$lib/contrail'; -import { parseCursor, tagCursor } from '$lib/contrail/cursor'; +import { nextCursor, rawForQuery } from '$lib/contrail/cursor'; import type { PageServerLoad } from './$types'; const PAGE_SIZE = 20; @@ -11,10 +11,10 @@ const PAGE_SIZE = 20; export const load: PageServerLoad = async ({ url, platform }) => { const client = getServerClient(platform!.env.DB); const now = new Date().toISOString(); - // Untag any inbound cursor (deep link) before handing the opaque keyset to D1; - // legacy untagged cursors pass through unchanged (om-7dbs). - const cursor = parseCursor(url.searchParams.get('cursor')).raw ?? undefined; const isPopular = url.searchParams.get('filter') !== 'all'; + // Deep-link ?cursor= resumes only an 'events' cursor minted for the same + // popular/all filter; anything else -> fresh page 1 (see rawForQuery). + const cursor = rawForQuery(url.searchParams.get('cursor'), 'events', { popular: isPopular }); const response = await listDiscoverableEventsFromContrail(client, { startsAtMin: now, @@ -36,8 +36,8 @@ export const load: PageServerLoad = async ({ url, platform }) => { return { events: flattenEventRecords(response.records), handles, - // Tag the first-page cursor so load-more routes back to this same D1 - // discoverable pipeline instead of re-inferring a backend (om-7dbs). - cursor: tagCursor('d1', response.cursor ?? null) + // A self-describing envelope: load-more re-runs THIS server-side query + // (discoverable + startsAtMin=now + the popular toggle), no client filters. + cursor: nextCursor('events', response.cursor ?? null, { popular: isPopular }) }; }; diff --git a/apps/web/src/routes/(app)/events/+page.svelte b/apps/web/src/routes/(app)/events/+page.svelte index 35b20f6..8a3142e 100644 --- a/apps/web/src/routes/(app)/events/+page.svelte +++ b/apps/web/src/routes/(app)/events/+page.svelte @@ -8,18 +8,6 @@ let filter = $derived(page.url.searchParams.get('filter') === 'all' ? 'all' : 'popular'); - let fetchParams = $derived({ - // load-more must re-run the discoverable pipeline + popular filter page 1 - // used, or unlisted / non-popular events leak onto later pages. - pipeline: 'discoverable', - startsAtMin: new Date().toISOString(), - profiles: 'true', - sort: 'startsAt', - order: 'asc', - limit: '20', - ...(filter === 'popular' ? { rsvpsCountMin: '2' } : {}) - }); - function setFilter(val: string) { const url = new URL(page.url); if (val === 'all') url.searchParams.set('filter', 'all'); @@ -67,6 +55,6 @@ {/if}

{:else} - + {/if} diff --git a/apps/web/src/routes/(app)/p/[actor]/hosting/+page.server.ts b/apps/web/src/routes/(app)/p/[actor]/hosting/+page.server.ts index 6486d82..35e75f4 100644 --- a/apps/web/src/routes/(app)/p/[actor]/hosting/+page.server.ts +++ b/apps/web/src/routes/(app)/p/[actor]/hosting/+page.server.ts @@ -5,7 +5,7 @@ import { getServerClient, listAuthoredEventsFromContrail } from '$lib/contrail'; -import { parseCursor, tagCursor } from '$lib/contrail/cursor'; +import { nextCursor, rawForQuery } from '$lib/contrail/cursor'; import { isActorIdentifier } from '@atcute/lexicons/syntax'; import { error } from '@sveltejs/kit'; @@ -20,8 +20,8 @@ export async function load({ params, url, platform }) { if (!did) throw error(404, 'Actor not found'); - // Untag any inbound cursor before the D1 read; legacy untagged passes through. - const cursor = parseCursor(url.searchParams.get('cursor')).raw ?? undefined; + // Deep-link ?cursor= resumes only a 'hosting' cursor for this actor; else fresh page 1. + const cursor = rawForQuery(url.searchParams.get('cursor'), 'hosting', { actor }); const now = new Date().toISOString(); const [profile, response] = await Promise.all([ @@ -39,8 +39,9 @@ export async function load({ params, url, platform }) { return { events: response ? flattenEventRecords(response.records) : [], - // Tag so load-more stays on this D1 authored pipeline (om-7dbs). - cursor: tagCursor('d1', response?.cursor ?? null), + // Self-describing envelope: load-more re-runs the authored + upcoming query + // scoped to this actor, server-side. + cursor: nextCursor('hosting', response?.cursor ?? null, { actor }), actorProfile: profile, actor, actorDid: did diff --git a/apps/web/src/routes/(app)/p/[actor]/hosting/+page.svelte b/apps/web/src/routes/(app)/p/[actor]/hosting/+page.svelte index dbd98b3..c55eee9 100644 --- a/apps/web/src/routes/(app)/p/[actor]/hosting/+page.svelte +++ b/apps/web/src/routes/(app)/p/[actor]/hosting/+page.svelte @@ -10,18 +10,6 @@ let hostAvatar = $derived( hostProfile?.value?.avatar ? getProfileBlobUrl(hostDid, hostProfile.value.avatar) : undefined ); - - let fetchParams: Record = $derived({ - // load-more must re-run the authored pipeline page 1 used, or conference - // talks (excluded by listAuthored) leak onto later pages. - pipeline: 'authored', - profiles: 'true', - sort: 'startsAt', - order: 'asc', - startsAtMin: new Date().toISOString(), - ...(data.actor ? { actor: data.actor } : {}), - limit: '20' - }); @@ -55,7 +43,6 @@ events={data.events ?? []} cursor={data.cursor ?? null} actor={data.actor} - {fetchParams} gridClass="space-y-3" /> {:else} diff --git a/apps/web/src/routes/(app)/p/[actor]/past-events/+page.server.ts b/apps/web/src/routes/(app)/p/[actor]/past-events/+page.server.ts index 56549b0..44fe029 100644 --- a/apps/web/src/routes/(app)/p/[actor]/past-events/+page.server.ts +++ b/apps/web/src/routes/(app)/p/[actor]/past-events/+page.server.ts @@ -5,7 +5,7 @@ import { getServerClient, listAuthoredEventsFromContrail } from '$lib/contrail'; -import { parseCursor, tagCursor } from '$lib/contrail/cursor'; +import { nextCursor, rawForQuery } from '$lib/contrail/cursor'; import { isActorIdentifier } from '@atcute/lexicons/syntax'; import { error } from '@sveltejs/kit'; @@ -20,8 +20,8 @@ export async function load({ params, url, platform }) { if (!did) throw error(404, 'Actor not found'); - // Untag any inbound cursor before the D1 read; legacy untagged passes through. - const cursor = parseCursor(url.searchParams.get('cursor')).raw ?? undefined; + // Deep-link ?cursor= resumes only a 'past-events' cursor for this actor; else fresh page 1. + const cursor = rawForQuery(url.searchParams.get('cursor'), 'past-events', { actor }); const now = new Date().toISOString(); const [profile, response] = await Promise.all([ @@ -44,8 +44,9 @@ export async function load({ params, url, platform }) { return { events, - // Tag so load-more stays on this D1 authored pipeline (om-7dbs). - cursor: tagCursor('d1', response?.cursor ?? null), + // Self-describing envelope: load-more re-runs the authored + past query + // (desc, startsAtMax=now) scoped to this actor, server-side. + cursor: nextCursor('past-events', response?.cursor ?? null, { actor }), actorProfile: profile, actor, actorDid: did diff --git a/apps/web/src/routes/(app)/p/[actor]/past-events/+page.svelte b/apps/web/src/routes/(app)/p/[actor]/past-events/+page.svelte index 3b73860..a2b76e1 100644 --- a/apps/web/src/routes/(app)/p/[actor]/past-events/+page.svelte +++ b/apps/web/src/routes/(app)/p/[actor]/past-events/+page.svelte @@ -10,18 +10,6 @@ let hostAvatar = $derived( hostProfile?.value?.avatar ? getProfileBlobUrl(hostDid, hostProfile.value.avatar) : undefined ); - - let fetchParams: Record = $derived({ - // load-more must re-run the authored pipeline page 1 used, or conference - // talks (excluded by listAuthored) leak onto later pages. - pipeline: 'authored', - profiles: 'true', - sort: 'startsAt', - order: 'desc', - startsAtMax: new Date().toISOString(), - ...(data.actor ? { actor: data.actor } : {}), - limit: '20' - }); @@ -55,7 +43,6 @@ events={data.events ?? []} cursor={data.cursor ?? null} actor={data.actor} - {fetchParams} gridClass="space-y-3" /> {:else} diff --git a/apps/web/src/routes/(app)/search/+page.server.ts b/apps/web/src/routes/(app)/search/+page.server.ts index 95eeb5e..9914d94 100644 --- a/apps/web/src/routes/(app)/search/+page.server.ts +++ b/apps/web/src/routes/(app)/search/+page.server.ts @@ -5,23 +5,32 @@ import { } from '$lib/contrail'; import { runEventSearchPage, searchBackendFromEnv } from '$lib/search/server/query'; import { SEARCH_PAGE_SIZE } from '$lib/search/constants'; +import { nextCursor } from '$lib/contrail/cursor'; import type { PageServerLoad } from './$types'; export const load: PageServerLoad = async ({ url, platform }) => { const client = getServerClient(platform!.env.DB); const q = url.searchParams.get('q')?.trim() || ''; - const cursor = url.searchParams.get('cursor') ?? undefined; if (!q) return { events: [], handles: {}, cursor: null, query: '' }; + // Search page 1 does NOT resume from ?cursor=: the term rides ?q=, not the + // envelope, so an inbound cursor can't be proven to match this route's term. + // (Load-more still resumes via the remote command, which carries the term.) + // Meilisearch ranks (typo tolerance, prefix, relevance); D1 supplies the // records. Falls back to the LIKE-based D1 path when the search backend is // unconfigured (local dev) or down. const backend = searchBackendFromEnv(platform?.env); if (backend) { try { - const page = await runEventSearchPage(backend, client, { q, cursor }); - return { events: page.events, handles: page.handles, cursor: page.cursor, query: q }; + const page = await runEventSearchPage(backend, client, { q }); + return { + events: page.events, + handles: page.handles, + cursor: nextCursor('search-meili', page.cursor), + query: q + }; } catch (err) { console.error('search backend failed, falling back to D1 search:', err); } @@ -36,8 +45,7 @@ export const load: PageServerLoad = async ({ url, platform }) => { startsAtMin: new Date().toISOString(), sort: 'startsAt', order: 'desc', - limit: SEARCH_PAGE_SIZE, - cursor + limit: SEARCH_PAGE_SIZE }); if (!response) return { events: [], handles: {}, cursor: null, query: q }; @@ -50,14 +58,11 @@ export const load: PageServerLoad = async ({ url, platform }) => { return { events: flattenEventRecords(response.records), handles, - // The D1 fallback is first-batch-only; don't hand its cursor back. Even - // with self-describing cursors (om-7dbs), the search fetchParams carry no - // `pipeline`, so a d1-tagged cursor would route load-more through plain - // listRecords — dropping the discoverable filter and startsAtMin this page - // applies — and later pages would drift into past and non-discoverable - // events. Re-enabling consistent D1 pagination here would mean threading the - // discoverable pipeline + filters through; deferred. Drop the cursor. - cursor: null, + // Self-describing 'search-d1' envelope: load-more re-runs the SAME + // discoverable + startsAtMin + desc query, with the search term from ?q=/ + // input — later pages stay upcoming-only and discoverable, no drift into + // past/non-discoverable events. + cursor: nextCursor('search-d1', response.cursor ?? null), query: q }; }; diff --git a/apps/web/src/routes/(app)/search/+page.svelte b/apps/web/src/routes/(app)/search/+page.svelte index ee540d7..87378b7 100644 --- a/apps/web/src/routes/(app)/search/+page.svelte +++ b/apps/web/src/routes/(app)/search/+page.svelte @@ -57,13 +57,7 @@ events={data.events} cursor={data.cursor} handles={data.handles} - fetchParams={{ - search: data.query, - profiles: 'true', - sort: 'startsAt', - order: 'desc', - limit: '20' - }} + q={data.query} /> {/if} {/if} diff --git a/apps/web/src/routes/(app)/search/page.server.test.ts b/apps/web/src/routes/(app)/search/page.server.test.ts index 2e2f491..4062a03 100644 --- a/apps/web/src/routes/(app)/search/page.server.test.ts +++ b/apps/web/src/routes/(app)/search/page.server.test.ts @@ -2,12 +2,13 @@ import { afterEach, describe, expect, it, vi } from 'vitest'; // The search load decides between two backends whose cursors are NOT // interchangeable: Meilisearch (offset cursor) and the D1 LIKE fallback (opaque -// cursor). loadMoreEvents re-routes to Meili whenever a backend is configured, -// so a D1 cursor handed back after a backend failure would be misread. And even -// with no backend at all, loadMoreEvents paginates via listRecords — without the -// discoverable filter or startsAtMin this load applies — so its cursor isn't -// safe to continue either. The D1 fallback is therefore first-batch-only. These -// tests pin that contract. +// keyset). Both now hand back a self-describing continuation ENVELOPE: the Meili +// path a 'search-meili' envelope, the D1 fallback a 'search-d1' envelope. The D1 +// fallback used to return cursor:null because load-more had no safe way to +// re-run the discoverable+startsAtMin query — the envelope closes that gap (and +// with it the earlier "search results stop after the first batch" limitation), +// so these tests now pin a REAL cursor on the D1 path, keyed to a query the +// load-more registry re-runs identically. vi.mock('$lib/contrail', () => ({ getServerClient: vi.fn(() => ({})), flattenEventRecords: vi.fn((records: unknown[]) => records), @@ -19,8 +20,9 @@ vi.mock('$lib/search/server/query', () => ({ })); import { load } from './+page.server'; -import { flattenEventRecords, listDiscoverableEventsFromContrail } from '$lib/contrail'; +import { listDiscoverableEventsFromContrail } from '$lib/contrail'; import { runEventSearchPage, searchBackendFromEnv } from '$lib/search/server/query'; +import { decodeCursor, encodeCursor } from '$lib/contrail/cursor'; const mockSearchBackendFromEnv = vi.mocked(searchBackendFromEnv); const mockRunEventSearchPage = vi.mocked(runEventSearchPage); @@ -54,23 +56,24 @@ describe('search page load', () => { expect(mockListDiscoverable).not.toHaveBeenCalled(); }); - it('serves the Meili page (offset cursor) when the backend succeeds', async () => { + it('serves the Meili page wrapped in a search-meili envelope when the backend succeeds', async () => { mockSearchBackendFromEnv.mockReturnValue({ url: 'https://meili.test', apiKey: 'k' }); mockRunEventSearchPage.mockResolvedValue({ events: [{ uri: 'at://did:plc:a/community.lexicon.calendar.event/1' }], handles: { 'did:plc:a': 'alice' }, - cursor: '20', + cursor: 'meili:20', distances: {} } as unknown as Awaited>); const result = await runLoad('kite'); - expect(result.cursor).toBe('20'); - expect(result.events).toHaveLength(1); + // The offset rides inside a self-describing envelope, so load-more re-runs + // the Meili path (not D1 listRecords) with the term from ?q=/input. + expect(decodeCursor(result.cursor)).toEqual({ v: 1, q: 'search-meili', raw: 'meili:20' }); expect(mockListDiscoverable).not.toHaveBeenCalled(); }); - it('drops the D1 cursor when a configured backend fails, so load-more cannot misroute it', async () => { + it('paginates the D1 fallback with a search-d1 envelope when a configured backend fails', async () => { mockSearchBackendFromEnv.mockReturnValue({ url: 'https://meili.test', apiKey: 'k' }); mockRunEventSearchPage.mockRejectedValue(new Error('meili down')); mockListDiscoverable.mockResolvedValue({ @@ -81,15 +84,13 @@ describe('search page load', () => { const result = await runLoad('kite'); - // Events still served from the D1 fallback... - expect(result.events).toHaveLength(1); expect(result.handles).toEqual({ 'did:plc:b': 'bob' }); - // ...but the incompatible D1 cursor is suppressed. - expect(result.cursor).toBeNull(); - expect(flattenEventRecords).toHaveBeenCalled(); + // The D1 cursor is now RESUMABLE: a search-d1 envelope whose load-more re-runs + // the same discoverable + startsAtMin + desc query. No more cursor:null. + expect(decodeCursor(result.cursor)).toEqual({ v: 1, q: 'search-d1', raw: 'd1-opaque-cursor' }); }); - it('drops the D1 cursor when no backend is configured, so load-more cannot drift past the first batch', async () => { + it('paginates the D1 fallback with a search-d1 envelope when no backend is configured', async () => { mockSearchBackendFromEnv.mockReturnValue(null); mockListDiscoverable.mockResolvedValue({ records: [{ uri: 'at://did:plc:c/community.lexicon.calendar.event/3' }], @@ -99,12 +100,56 @@ describe('search page load', () => { const result = await runLoad('kite'); - // First batch is served from the discoverable, upcoming-only D1 query... - expect(result.events).toHaveLength(1); - // ...but the cursor is suppressed: loadMoreEvents would paginate via - // listRecords without the discoverable filter or startsAtMin, drifting - // into past and non-discoverable events on later pages. - expect(result.cursor).toBeNull(); + // First batch is the discoverable, upcoming-only, desc D1 query... + const params = mockListDiscoverable.mock.calls[0][1]; + expect(params).toMatchObject({ search: 'kite', order: 'desc' }); + expect(typeof params.startsAtMin).toBe('string'); + // ...and later pages resume it via a real envelope, not cursor:null. + expect(decodeCursor(result.cursor)).toEqual({ v: 1, q: 'search-d1', raw: 'd1-opaque-cursor' }); expect(mockRunEventSearchPage).not.toHaveBeenCalled(); }); + + it('ends cleanly (cursor:null) only on a genuinely last D1 page', async () => { + mockSearchBackendFromEnv.mockReturnValue(null); + mockListDiscoverable.mockResolvedValue({ + records: [{ uri: 'at://did:plc:d/community.lexicon.calendar.event/4' }], + profiles: [], + cursor: null + } as unknown as Awaited>); + + const result = await runLoad('kite'); + expect(result.cursor).toBeNull(); + }); + + it('deep-link: does NOT resume even its OWN search-d1 envelope (term not in envelope)', async () => { + mockSearchBackendFromEnv.mockReturnValue(null); + mockListDiscoverable.mockResolvedValue({ + records: [], + profiles: [], + cursor: null + } as unknown as Awaited>); + + // The keyset was minted for SOME term, but the term rides ?q= and is absent + // from the envelope — a `dogs` keyset under ?q=kite would corrupt pagination, + // and the two are indistinguishable. So page 1 always starts fresh. + const inbound = encodeCursor({ v: 1, q: 'search-d1', raw: 'page2keyset' }); + await runLoad('kite', inbound); + + expect(mockListDiscoverable.mock.calls[0][1].cursor).toBeUndefined(); + }); + + it('deep-link: ignores a foreign-query envelope (fresh page 1, no resume)', async () => { + mockSearchBackendFromEnv.mockReturnValue(null); + mockListDiscoverable.mockResolvedValue({ + records: [], + profiles: [], + cursor: null + } as unknown as Awaited>); + + // An 'events' envelope deep-linked into /search must not resume its keyset. + const foreign = encodeCursor({ v: 1, q: 'events', args: { popular: true }, raw: 'nope' }); + await runLoad('kite', foreign); + + expect(mockListDiscoverable.mock.calls[0][1].cursor).toBeUndefined(); + }); }); diff --git a/apps/web/src/routes/(app)/topics/[slug]/+page.server.ts b/apps/web/src/routes/(app)/topics/[slug]/+page.server.ts index e7f9628..ba7b4c4 100644 --- a/apps/web/src/routes/(app)/topics/[slug]/+page.server.ts +++ b/apps/web/src/routes/(app)/topics/[slug]/+page.server.ts @@ -1,15 +1,16 @@ import { error } from '@sveltejs/kit'; -import { getTopicBySlug } from '$lib/topics'; +import { getTopicBySlug, orQueryFromSlug } from '$lib/topics'; import { flattenEventRecords, getServerClient, listDiscoverableEventsFromContrail } from '$lib/contrail'; +import { nextCursor, rawForQuery } from '$lib/contrail/cursor'; import type { PageServerLoad } from './$types'; const PAGE_SIZE = 20; -export const load: PageServerLoad = async ({ params, platform }) => { +export const load: PageServerLoad = async ({ params, url, platform }) => { const topic = getTopicBySlug(params.slug); if (!topic) error(404, 'Topic not found'); @@ -18,9 +19,9 @@ export const load: PageServerLoad = async ({ params, platform }) => { // Match events whose name/description mention ANY of the topic's hashtag // terms. The discoverable list runs `search` through D1's SQLite FTS5 MATCH, // where an uppercase OR is a real disjunction operator — so this is a true - // "any term" query. (Meili treats OR as a literal token, but this page never - // routes through Meili: see the cursor note below.) - const query = topic.hashtags.map((h) => h.replace(/^#/, '')).join(' OR '); + // "any term" query. Derived SERVER-side from the slug via the shared helper the + // load-more registry also uses, so page 1 and load-more can't drift. + const query = orQueryFromSlug(params.slug) ?? ''; const response = await listDiscoverableEventsFromContrail(client, { search: query, @@ -29,7 +30,9 @@ export const load: PageServerLoad = async ({ params, platform }) => { startsAtMin: new Date().toISOString(), sort: 'startsAt', order: 'asc', - limit: PAGE_SIZE + limit: PAGE_SIZE, + // Deep-link ?cursor= resumes only a 'topic' cursor for this slug; else fresh page 1. + cursor: rawForQuery(url.searchParams.get('cursor'), 'topic', { slug: params.slug }) }); const handles: Record = {}; @@ -41,15 +44,11 @@ export const load: PageServerLoad = async ({ params, platform }) => { topic, events: flattenEventRecords(response?.records ?? []), handles, - // First-batch-only, like the search page's D1 fallback. Self-describing - // cursors (om-7dbs) now stop this page's D1 cursor from being mis-consumed - // as a Meili offset, but they don't make it resumable here: this fetchParams - // contract carries no `pipeline`, so a d1-tagged cursor would still route - // load-more through plain listRecords, dropping the discoverable + - // startsAtMin filters this page relies on. So don't hand back a cursor. Deep - // topic pagination (threading the discoverable pipeline through) is tracked - // separately (om-47ak). - cursor: null, + // Self-describing 'topic' envelope carrying the slug: load-more re-derives + // the same OR-search from the slug SERVER-side and re-runs the identical + // discoverable + startsAtMin query — later pages stay upcoming-only and + // discoverable. + cursor: nextCursor('topic', response?.cursor ?? null, { slug: params.slug }), query }; }; diff --git a/apps/web/src/routes/(app)/topics/[slug]/+page.svelte b/apps/web/src/routes/(app)/topics/[slug]/+page.svelte index 119020c..b8ef977 100644 --- a/apps/web/src/routes/(app)/topics/[slug]/+page.svelte +++ b/apps/web/src/routes/(app)/topics/[slug]/+page.svelte @@ -2,18 +2,6 @@ import EventList from '$lib/components/EventList.svelte'; let { data } = $props(); - - // Query is built server-side and passed through `data` so the two stay in - // sync. Mirrors the search page. The topic page is first-batch-only - // (data.cursor is null), so these params are only a contract for EventList, - // not an active pagination path. - let fetchParams = $derived({ - search: data.query, - profiles: 'true', - sort: 'startsAt', - order: 'asc', - limit: '20' - }); @@ -62,11 +50,6 @@

{:else} - + {/if} diff --git a/apps/web/src/routes/(app)/topics/[slug]/page.server.test.ts b/apps/web/src/routes/(app)/topics/[slug]/page.server.test.ts new file mode 100644 index 0000000..f3c8b9e --- /dev/null +++ b/apps/web/src/routes/(app)/topics/[slug]/page.server.test.ts @@ -0,0 +1,121 @@ +import { afterEach, describe, expect, it, vi } from 'vitest'; + +// The topic page used to return cursor:null (first-batch-only) because load-more +// had no safe way to re-run its discoverable + OR-search + startsAtMin query. +// The envelope closes that gap: the load now emits a self-describing 'topic' +// envelope carrying the slug, and the load-more registry re-derives the SAME +// OR-search from that slug server-side. These pin the page-1 side of that +// continuity plus the deep-link query-match rule. +vi.mock('$lib/contrail', () => ({ + getServerClient: vi.fn(() => ({})), + flattenEventRecords: vi.fn((records: unknown[]) => records), + listDiscoverableEventsFromContrail: vi.fn() +})); + +import { load } from './+page.server'; +import { listDiscoverableEventsFromContrail } from '$lib/contrail'; +import { decodeCursor, encodeCursor } from '$lib/contrail/cursor'; + +const mockListDiscoverable = vi.mocked(listDiscoverableEventsFromContrail); + +type LoadResult = { + topic: { slug: string }; + events: unknown[]; + handles: Record; + cursor: string | null; + query: string; +}; + +function event(slug: string, cursor?: string) { + const url = new URL(`https://atmo.test/topics/${slug}`); + if (cursor) url.searchParams.set('cursor', cursor); + return { params: { slug }, url, platform: { env: {} } } as unknown as Parameters[0]; +} + +const run = async (slug: string, cursor?: string) => + (await load(event(slug, cursor))) as unknown as LoadResult; + +afterEach(() => vi.clearAllMocks()); + +describe('topic page load', () => { + it("builds a 'topic' envelope carrying the slug, deriving the OR-search server-side", async () => { + mockListDiscoverable.mockResolvedValue({ + records: [{ uri: 'at://did:plc:a/community.lexicon.calendar.event/1' }], + profiles: [{ did: 'did:plc:a', handle: 'alice' }], + cursor: 'd1-topic-cursor' + } as unknown as Awaited>); + + const result = await run('technology'); + + const params = mockListDiscoverable.mock.calls[0][1]; + // orQueryFromSlug('technology') — the SAME helper the load-more registry uses. + expect(params.search).toBe('tech OR technology'); + expect(params).toMatchObject({ order: 'asc', limit: 20, profiles: true }); + expect(typeof params.startsAtMin).toBe('string'); + expect(result.query).toBe('tech OR technology'); + // A resumable envelope, not cursor:null. + expect(decodeCursor(result.cursor)).toEqual({ + v: 1, + q: 'topic', + args: { slug: 'technology' }, + raw: 'd1-topic-cursor' + }); + }); + + it('ends cleanly (cursor:null) only on a genuinely last page', async () => { + mockListDiscoverable.mockResolvedValue({ + records: [{ uri: 'at://did:plc:a/community.lexicon.calendar.event/1' }], + profiles: [], + cursor: null + } as unknown as Awaited>); + + const result = await run('technology'); + expect(result.cursor).toBeNull(); + }); + + it('deep-link: resumes a topic envelope by feeding its raw keyset to D1', async () => { + mockListDiscoverable.mockResolvedValue({ + records: [], + profiles: [], + cursor: null + } as unknown as Awaited>); + + const inbound = encodeCursor({ v: 1, q: 'topic', args: { slug: 'technology' }, raw: 'p2keyset' }); + await run('technology', inbound); + + expect(mockListDiscoverable.mock.calls[0][1]).toMatchObject({ cursor: 'p2keyset' }); + }); + + it('deep-link: ignores a foreign-query envelope (fresh page 1)', async () => { + mockListDiscoverable.mockResolvedValue({ + records: [], + profiles: [], + cursor: null + } as unknown as Awaited>); + + const foreign = encodeCursor({ v: 1, q: 'events', args: { popular: true }, raw: 'nope' }); + await run('technology', foreign); + + expect(mockListDiscoverable.mock.calls[0][1].cursor).toBeUndefined(); + }); + + it("deep-link: ignores a topic envelope minted for a DIFFERENT slug (fresh page 1)", async () => { + mockListDiscoverable.mockResolvedValue({ + records: [], + profiles: [], + cursor: null + } as unknown as Awaited>); + + // A 'technology' keyset deep-linked into /topics/ai names the same query but + // indexes a different OR-search result set — must not resume. + const otherSlug = encodeCursor({ v: 1, q: 'topic', args: { slug: 'technology' }, raw: 'nope' }); + await run('ai', otherSlug); + + expect(mockListDiscoverable.mock.calls[0][1].cursor).toBeUndefined(); + }); + + it('404s for an unknown slug before touching D1', async () => { + await expect(run('no-such-topic')).rejects.toThrow(); + expect(mockListDiscoverable).not.toHaveBeenCalled(); + }); +});