diff --git a/docs/docs/pages/config.mdx b/docs/docs/pages/config.mdx index ef2e951..75943ec 100644 --- a/docs/docs/pages/config.mdx +++ b/docs/docs/pages/config.mdx @@ -126,6 +126,10 @@ This would produce paths like `/blog/2024/01/my-post`. When `pathTemplate` is set, it overrides `pathPrefix`. If `pathTemplate` is not set, the default `pathPrefix`/slug behavior is used. +:::warning +When `pathTemplate` is set, automation injection of `` tags in the `` may not work. See [verifying](/verifying) for more information. +::: + ### Ignoring Files Some frameworks use special files like `_index.md` (Zola) for section pages that aren't actual blog posts. Use the `ignore` field to skip these files during publishing: diff --git a/docs/docs/pages/verifying.mdx b/docs/docs/pages/verifying.mdx index 74ea777..2327e64 100644 --- a/docs/docs/pages/verifying.mdx +++ b/docs/docs/pages/verifying.mdx @@ -1,6 +1,6 @@ # Verifying -In order for your posts to show up on indexers you need to make sure your publication and your documents are verified. +In order for your posts to show up on indexers you need to make sure your publication and your documents are verified. :::tip You can learn more about Standard.site verification [here](https://standard.site/) @@ -8,17 +8,51 @@ You can learn more about Standard.site verification [here](https://standard.site ## Publication Verification -As specified by Standard.site, the `site.standard.publication` record is verified by placing the record `https://example.com/.well-known/site.standard.publication`. That record might look something like `at://did:plc:abc123/site.standard.publication/rkey`. Sequoia handles this for you automatically if you designate your public/static folder during the [setup](/setup). When the record is created, the record AT URI is saved in `.well-known/site.standard.publication` of your public folder. Once you deploy your site with this addition, the publication will be verified! +As specified by Standard.site, the `site.standard.publication` record is verified by placing the record `https://example.com/.well-known/site.standard.publication`. +That record might look something like `at://did:plc:abc123/site.standard.publication/rkey`. +Pages may also [aid discovery](https://standard.site/docs/verification/#discovery-hint) with a `` tag in the `` with your publication URI. +Sequoia handles this for you automatically if you designate your public/static folder during the [setup](/setup). +When the record is created, the record AT URI is saved in `.well-known/site.standard.publication` of your public folder and a `` tag added to your posts. +Once you deploy your site with this addition, the publication will be verified! ## Document Verification -Every document or blog post that is published needs a `` tag in the `` of your blog post HTML page. The content of that link tag needs to be the AT URI for the record we just published on your PDS. There are two ways you can handle these: -- `sequoia inject` (recommended) - By running this command after publishing, and after building the site with your SSG, Sequoia will inject the link tags into your finished HTML. This way you don't have to manually edit it or mess with an SSG config to set it up. Just deploy the build folder after you have run `sequoia inject`! -- Manual - After you have run `sequoia publish` the CLI will add in a new `atUri` field to every post's frontmatter. This way you can configure your SSG to read that frontmatter and include it in the build step, similar to how it might include an opengraph image in the meta tags. This approach gives you full control over the HTML files but will take a bit more skill. +Every document or blog post that is published needs a `` tag in the `` of your blog post HTML page. +The content of that link tag needs to be the AT URI for the record we just published on your PDS, and an optional publication URI for enhanced links in Bluesky posts if enabled. +There are two ways you can handle these: + +- `sequoia inject` (recommended) - By running this command after publishing, and after building the site with your SSG, Sequoia will inject the link tags into your finished HTML. + This way you don't have to manually edit it or mess with an SSG config to set it up. + Just deploy the build folder after you have run `sequoia inject`! + +- Manual - After you have run `sequoia publish` the CLI will add in a new `atUri` field to every post's frontmatter. + This way you can configure your SSG to read that frontmatter and include it in the build step, similar to how it might include an opengraph image in the meta tags. + You should also include the publication URI in a separate `` tag. + This approach gives you full control over the HTML files but will take a bit more skill. + + :::code-group + + ```html [Hugo] + + {{ if .Params.atUri }} + + {{ end }} + + ``` + + ```html [Jekyll] + + {%- if page.atUri %} + + {%- endif %} + + ``` + + ::: ## Testing Verification -After your publication and your document records have been published and you site has been deployed, you can test the verification of your records a few ways. +After your publication and your document records have been published and you site has been deployed, you can test the verification of your records a few ways. ### pds.ls @@ -31,4 +65,5 @@ Visit [site-validator.fly.dev](https://site-validator.fly.dev/) and paste in the ## Troubleshooting - Make sure that you are either using `sequoia inject` or manually handling the required `` tags for each post. Read [workflows](/workflows) for a clear order of operations to publish, inject, and deploy. +- - Make sure that the `.well-known` publication record is present in your public/static folder, and that it's populating to your build folder (e.g. `dist`). There are some SSGs that will not automatically include dot files or directories. diff --git a/packages/cli/src/commands/init.ts b/packages/cli/src/commands/init.ts index 781e0dc..d3be0f3 100644 --- a/packages/cli/src/commands/init.ts +++ b/packages/cli/src/commands/init.ts @@ -264,13 +264,14 @@ export const initCommand = command({ s.start("Creating publication..."); try { - publicationUri = await createPublication(agent, { + const publicationRef = await createPublication(agent, { url: siteConfig.siteUrl, name: publicationConfig.name, description: publicationConfig.description || undefined, iconPath: publicationConfig.iconPath || undefined, showInDiscover: publicationConfig.showInDiscover, }); + publicationUri = publicationRef.uri; s.stop(`Publication created: ${publicationUri}`); } catch (error) { s.stop("Failed to create publication"); diff --git a/packages/cli/src/commands/inject.ts b/packages/cli/src/commands/inject.ts index 476e131..19e1684 100644 --- a/packages/cli/src/commands/inject.ts +++ b/packages/cli/src/commands/inject.ts @@ -130,35 +130,28 @@ export const injectCommand = command({ // Read the HTML file let content = await fs.readFile(htmlPath, "utf-8"); - // Check if link tag already exists - const linkTag = ``; - if (content.includes('rel="site.standard.document"')) { - alreadyHasCount++; - continue; - } - - // Find and inject before it - const headCloseIndex = content.indexOf(""); - if (headCloseIndex === -1) { - log.warn(` No found in ${relativePath}, skipping`); - skippedCount++; - continue; - } - - if (dryRun) { - log.message(` Would inject into: ${relativePath}`); - log.message(` ${linkTag}`); - injectedCount++; - continue; + // Inject the tags + let injected = injectLinkTags( + dryRun, + relativePath, + content, + atUri, + config.publicationUri, + ); + switch (injected) { + case Injected.AlreadyPresent: + alreadyHasCount++; + continue; + case Injected.Skipped: + skippedCount++; + continue; + case Injected.Faked: + injectedCount++; + continue; + default: + content = injected; } - // Inject the link tag - const indent = " "; // Standard indentation - content = - content.slice(0, headCloseIndex) + - `${indent}${linkTag}\n${indent}` + - content.slice(headCloseIndex); - await fs.writeFile(htmlPath, content); log.success(` Injected into: ${relativePath}`); injectedCount++; @@ -180,3 +173,65 @@ export const injectCommand = command({ } }, }); + +export enum Injected { + AlreadyPresent = 0, + Skipped, + Faked, +} + +export function injectLinkTags( + dryRun: boolean, + relativePath: string, + content: string, + atUri: string, + publicationUri: string, +): string | Injected { + // Check if link tags already exist + let documentLinkTag: string | undefined = + ``; + let publicationLinkTag: string | undefined = + ``; + if (content.includes('rel="site.standard.document"')) { + documentLinkTag = undefined; + } + if (content.includes('rel="site.standard.publication"')) { + publicationLinkTag = undefined; + } + + if (!documentLinkTag && !publicationLinkTag) { + return Injected.AlreadyPresent; + } + + // Find and inject before it + const headCloseIndex = content.indexOf(""); + if (headCloseIndex === -1) { + log.warn(` No found in ${relativePath}, skipping`); + return Injected.Skipped; + } + + if (dryRun) { + log.message(` Would inject into: ${relativePath}`); + if (documentLinkTag) { + log.message(` ${documentLinkTag}`); + } + if (publicationLinkTag) { + log.message(` ${publicationLinkTag}`); + } + return Injected.Faked; + } + + // Inject the link tags + const indent = " "; // Standard indentation + const after = content.slice(headCloseIndex); + content = content.slice(0, headCloseIndex); + if (documentLinkTag) { + content += `${indent}${documentLinkTag}\n${indent}`; + } + if (publicationLinkTag) { + content += `${indent}${publicationLinkTag}\n${indent}`; + } + content += after; + + return content; +} diff --git a/packages/cli/src/commands/publish.ts b/packages/cli/src/commands/publish.ts index 2c31c84..c2e417d 100644 --- a/packages/cli/src/commands/publish.ts +++ b/packages/cli/src/commands/publish.ts @@ -2,7 +2,13 @@ import * as fs from "node:fs/promises"; import { command, flag } from "cmd-ts"; import { select, spinner, log } from "@clack/prompts"; import * as path from "node:path"; -import { CONFIG_FILENAME, loadConfig, loadState, saveState, findConfig } from "../lib/config"; +import { + CONFIG_FILENAME, + loadConfig, + loadState, + saveState, + findConfig, +} from "../lib/config"; import { loadCredentials, listAllCredentials, @@ -17,7 +23,8 @@ import { resolveImagePath, createBlueskyPost, addBskyPostRefToDocument, - COVER_IMAGE_MAX_SIZE, + COVER_IMAGE_MAX_SIZE, + getPublication, } from "../lib/atproto"; import { scanContentDirectory, @@ -26,7 +33,7 @@ import { resolvePostPath, } from "../lib/markdown"; import type { BlogPost, BlobObject, StrongRef } from "../lib/types"; -import { syncStateFromPDS } from "../lib/sync"; +import { syncPublication, syncStateFromPDS } from "../lib/sync"; import { exitOnCancel } from "../lib/prompts"; export const publishCommand = command({ @@ -155,7 +162,7 @@ export const publishCommand = command({ if ( config.autoSync !== false && - Object.keys(state.posts).length === 0 && + (!state.publication || Object.keys(state.posts).length === 0) && !dryRun ) { // Create agent early for sync (will be reused for publishing) @@ -334,6 +341,15 @@ export const publishCommand = command({ } } + // Make sure publication state is available for enhanced links. + if (!state.publication) { + state.publication = await syncPublication( + agent, + config.publicationUri, + false, + ); + } + // Publish posts let publishedCount = 0; let updatedCount = 0; @@ -347,7 +363,9 @@ export const publishCommand = command({ // Handle cover image upload let coverImage: BlobObject | undefined; if (post.coverImagePath) { - log.info(` Uploading cover image: ${path.basename(post.coverImagePath)}`); + log.info( + ` Uploading cover image: ${path.basename(post.coverImagePath)}`, + ); coverImage = await uploadImage(agent, post.coverImagePath); if (coverImage) { log.info(` Uploaded image blob: ${coverImage.ref.$link}`); @@ -357,7 +375,7 @@ export const publishCommand = command({ } // Track atUri, content for state saving, and bskyPostRef - let atUri: string; + let documentRef: StrongRef; let contentForHash: string; let bskyPostRef: StrongRef | undefined; const relativeFilePath = path.relative(configDir, post.filePath); @@ -366,13 +384,13 @@ export const publishCommand = command({ const existingBskyPostRef = state.posts[relativeFilePath]?.bskyPostRef; if (action === "create") { - atUri = await createDocument(agent, post, config, coverImage); - s.stop(`Created: ${atUri}`); + documentRef = await createDocument(agent, post, config, coverImage); + s.stop(`Created: ${documentRef.uri}`); // Update frontmatter with atUri const updatedContent = updateFrontmatterWithAtUri( post.rawContent, - atUri, + documentRef.uri, ); await fs.writeFile(post.filePath, updatedContent); log.info(` Updated frontmatter in ${path.basename(post.filePath)}`); @@ -381,11 +399,16 @@ export const publishCommand = command({ contentForHash = updatedContent; publishedCount++; } else { - // Validate post. - atUri = post.frontmatter.atUri!; - await updateDocument(agent, post, atUri, config, coverImage); - s.stop(`Updated: ${atUri}`); + const atUri = post.frontmatter.atUri!; + documentRef = await updateDocument( + agent, + post, + atUri, + config, + coverImage, + ); + s.stop(`Updated: ${documentRef.uri}`); // For updates, rawContent already has atUri contentForHash = post.rawContent; @@ -414,12 +437,18 @@ export const publishCommand = command({ description: post.frontmatter.description, bskyPost: post.frontmatter.bskyPost, canonicalUrl, + documentRef, + publicationRef: state.publication, coverImage, publishedAt: post.frontmatter.publishDate, }); // Update document record with bskyPostRef - await addBskyPostRefToDocument(agent, atUri, bskyPostRef); + await addBskyPostRefToDocument( + agent, + documentRef.uri, + bskyPostRef, + ); log.info(` Created Bluesky post: ${bskyPostRef.uri}`); bskyPostCount++; } catch (bskyError) { @@ -437,7 +466,7 @@ export const publishCommand = command({ const contentHash = await getContentHash(contentForHash); state.posts[relativeFilePath] = { contentHash, - atUri, + atUri: documentRef.uri, lastPublished: new Date().toISOString(), slug: post.slug, bskyPostRef, @@ -478,4 +507,3 @@ async function validatePost(post: BlogPost): Promise { return true; } - diff --git a/packages/cli/src/lib/atproto.ts b/packages/cli/src/lib/atproto.ts index db3d973..0400820 100644 --- a/packages/cli/src/lib/atproto.ts +++ b/packages/cli/src/lib/atproto.ts @@ -241,7 +241,7 @@ export async function resolveImagePath( } } - return null; + return undefined; } export async function createDocument( @@ -249,7 +249,7 @@ export async function createDocument( post: BlogPost, config: PublisherConfig, coverImage?: BlobObject, -): Promise { +): Promise { const postPath = resolvePostPath( post, config.pathPrefix, @@ -307,7 +307,10 @@ export async function createDocument( record, }); - return response.data.uri; + return { + cid: response.data.cid, + uri: response.data.uri, + }; } export async function updateDocument( @@ -316,7 +319,7 @@ export async function updateDocument( atUri: string, config: PublisherConfig, coverImage?: BlobObject, -): Promise { +): Promise { // Parse the atUri to get the collection and rkey // Format: at://did:plc:xxx/collection/rkey const uriMatch = atUri.match(/^at:\/\/([^/]+)\/([^/]+)\/(.+)$/); @@ -385,12 +388,17 @@ export async function updateDocument( record.tags = post.frontmatter.tags; } - await agent.com.atproto.repo.putRecord({ + const response = await agent.com.atproto.repo.putRecord({ repo: agent.did!, collection: collection!, rkey: rkey!, record, }); + + return { + cid: response.data.cid, + uri: response.data.uri, + }; } export function parseAtUri( @@ -467,7 +475,7 @@ export async function listDocuments( export async function createPublication( agent: Agent, options: CreatePublicationOptions, -): Promise { +): Promise { let icon: BlobObject | undefined; if (options.iconPath) { @@ -501,7 +509,10 @@ export async function createPublication( record, }); - return response.data.uri; + return { + cid: response.data.cid, + uri: response.data.uri, + }; } export interface GetPublicationResult { @@ -607,6 +618,8 @@ export interface CreateBlueskyPostOptions { description?: string; bskyPost?: string; canonicalUrl: string; + documentRef: StrongRef; + publicationRef: StrongRef; coverImage?: BlobObject; publishedAt: string; // Used as createdAt for the post } @@ -654,6 +667,8 @@ export async function createBlueskyPost( description, bskyPost, canonicalUrl, + documentRef, + publicationRef, coverImage, publishedAt, } = options; @@ -704,6 +719,20 @@ export async function createBlueskyPost( uri: canonicalUrl, title: title.substring(0, 500), // Max 500 chars for title description: (description || "").substring(0, 1000), // Max 1000 chars for description + associatedRefs: [ + { + // site.standard.document + $type: "com.atproto.repo.strongRef", + cid: documentRef.cid, + uri: documentRef.uri, + }, + { + // site.standard.publication + $type: "com.atproto.repo.strongRef", + cid: publicationRef.cid, + uri: publicationRef.uri, + }, + ], }, }; diff --git a/packages/cli/src/lib/sync.ts b/packages/cli/src/lib/sync.ts index 0985684..daa0c9e 100644 --- a/packages/cli/src/lib/sync.ts +++ b/packages/cli/src/lib/sync.ts @@ -1,7 +1,7 @@ import * as fs from "node:fs/promises"; import * as path from "node:path"; import { log } from "@clack/prompts"; -import { listDocuments, type createAgent } from "./atproto"; +import { getPublication, listDocuments, type createAgent } from "./atproto"; import { loadState, saveState } from "./config"; import { scanContentDirectory, @@ -9,7 +9,7 @@ import { updateFrontmatterWithAtUri, resolvePostPath, } from "./markdown"; -import type { PublisherConfig, PublisherState } from "./types"; +import type { PublisherConfig, PublisherState, StrongRef } from "./types"; export interface SyncOptions { updateFrontmatter?: boolean; @@ -81,6 +81,15 @@ export async function syncStateFromPDS( // Load existing state const state = await loadState(configDir); + // Update the publication information for enhanced links. + if (!state.publication) { + state.publication = await syncPublication( + agent, + config.publicationUri, + quiet, + ); + } + // Track changes let matchedCount = 0; let unmatchedCount = 0; @@ -201,3 +210,24 @@ export async function syncStateFromPDS( return { state, matchedCount, unmatchedCount, frontmatterUpdatesApplied }; } + +export async function syncPublication( + agent: Awaited>, + publicationUri: string, + quiet: boolean, +): Promise { + const publicationRef = await getPublication(agent, publicationUri); + if (!publicationRef) { + if (!quiet) { + log.error( + `Publication ${publicationUri} not found. Update your publication record and try again.`, + ); + } + process.exit(1); + } + + return { + cid: publicationRef.cid, + uri: publicationRef.uri, + }; +} diff --git a/packages/cli/src/lib/types.ts b/packages/cli/src/lib/types.ts index fa2c4a6..1618da2 100644 --- a/packages/cli/src/lib/types.ts +++ b/packages/cli/src/lib/types.ts @@ -120,7 +120,10 @@ export interface BlobObject { size: number; } +export interface PublicationState extends StrongRef {} + export interface PublisherState { + publication?: PublicationState; posts: Record; } diff --git a/packages/cli/test/inject.test.ts b/packages/cli/test/inject.test.ts new file mode 100644 index 0000000..93bbc02 --- /dev/null +++ b/packages/cli/test/inject.test.ts @@ -0,0 +1,141 @@ +import { describe, expect, it, spyOn } from "bun:test"; +import { log } from "@clack/prompts"; +import { Injected, injectLinkTags } from "../src/commands/inject"; + +const atUri = "at://did:plc:abc123/app.bsky.feed.post/xyz"; +const publicationUri = "at://did:plc:def456/app.bsky.feed.generator/main"; + +describe("injectLinkTags", () => { + describe("neither tag needs injection", () => { + it("returns AlreadyPresent when both tags already exist", () => { + const content = ` + + +`; + const result = injectLinkTags( + false, + "test.html", + content, + atUri, + publicationUri, + ); + expect(result).toBe(Injected.AlreadyPresent); + }); + }); + + describe("one tag needs injection", () => { + it("injects only documentLinkTag when publicationLinkTag is already present", () => { + const content = ` + +`; + const result = injectLinkTags( + false, + "test.html", + content, + atUri, + publicationUri, + ); + expect(typeof result).toBe("string"); + expect(result as string).toContain( + ``, + ); + expect( + result as string, + ).not.toContain(` + { + const content = ` + +`; + const result = injectLinkTags( + false, + "test.html", + content, + atUri, + publicationUri, + ); + expect(typeof result).toBe("string"); + expect(result as string).toContain( + ``, + ); + }); + }); + + describe("both tags need injection", () => { + it("injects both tags when neither is present", () => { + const content = "\n"; + const result = injectLinkTags( + false, + "test.html", + content, + atUri, + publicationUri, + ); + expect(typeof result).toBe("string"); + expect(result as string).toContain( + ``, + ); + expect(result as string).toContain( + ``, + ); + }); + + it("injects tags before ", () => { + const content = "\n"; + const result = injectLinkTags( + false, + "test.html", + content, + atUri, + publicationUri, + ) as string; + const headCloseIndex = result.indexOf(""); + expect(result.indexOf('rel="site.standard.document"')).toBeLessThan( + headCloseIndex, + ); + expect(result.indexOf('rel="site.standard.publication"')).toBeLessThan( + headCloseIndex, + ); + }); + + it("returns Skipped when no is found", () => { + const warnSpy = spyOn(log, "warn").mockImplementation(() => {}); + const content = "No head tag here"; + const result = injectLinkTags( + false, + "test.html", + content, + atUri, + publicationUri, + ); + expect(result).toBe(Injected.Skipped); + expect(warnSpy).toHaveBeenCalledWith( + " No found in test.html, skipping", + ); + warnSpy.mockRestore(); + }); + + it("returns Faked and does not modify content during dry run", () => { + const messageSpy = spyOn(log, "message").mockImplementation(() => {}); + const content = "\n"; + const result = injectLinkTags( + true, + "test.html", + content, + atUri, + publicationUri, + ); + expect(result).toBe(Injected.Faked); + expect(messageSpy).toHaveBeenCalledWith(" Would inject into: test.html"); + expect(messageSpy).toHaveBeenCalledWith( + ` `, + ); + expect(messageSpy).toHaveBeenCalledWith( + ` `, + ); + messageSpy.mockRestore(); + }); + }); +});