diff --git a/docs/PRD_ADMIN_MODERATION.md b/docs/PRD_ADMIN_MODERATION.md index ca9e309..02a67ba 100644 --- a/docs/PRD_ADMIN_MODERATION.md +++ b/docs/PRD_ADMIN_MODERATION.md @@ -200,6 +200,7 @@ The web currently derives sensitivity from `post.record?.labels`; mobile's `Post - Reverse-chronological, cursor-paginated individual actions; stable ordering by server timestamp and action ID. - Filters: action, subject type/URI, community where publicly disclosable, issuing authority, local versus inherited, and date range. Include actor filtering; public actor identity is approved. - Each item shows action, time, scope, authority, safe subject reference, reason code, and linkage to the action it reverses. +- Decided 2026-09-30 (Q-LABEL-LOG): NSFW label applies and retractions are left out of the public modlog. The admin log keeps them, and the blur on the post is the public signal. - Admin view adds authenticated actor DID, private notes, linked report IDs, and publication/ingestion status when applicable. Public fields are explicitly allowlisted; never serialize private records and strip a few fields afterward. ### Audit contract @@ -462,9 +463,9 @@ All seven endpoints belong to the local milestone. Instance authority and actor |---|---|---| | `moderation/removeContent.json` / `removeContent` | `procedure`; authentication and instance-admin authorization required | Required `subject` strongRef, `expectedVersion`, `idempotencyKey`, and `reason`. Optional `privateNote`; no public free-text field in this release. Return `outcome`, post-operation `state`, and the resulting `action` when an action exists. Removal applies to the subject URI; the strongRef CID is the version inspected and checked before acting. | | `moderation/restoreContent.json` / `restoreContent` | `procedure`; authentication and instance-admin authorization required | Required local `actionId` identifying the removal to retract, `expectedVersion`, `idempotencyKey`, and `reason`; optional `reviewedSubject` strongRef and `privateNote`. Require `reviewedSubject` when a current record exists; allow retraction without one when the record is deleted/unavailable, without making it visible. Return the same mutation-result shape. A restore cannot target an imported source's decision. | -| `moderation/labelContent.json` / `labelContent` | `procedure`; authentication and instance-admin authorization required | Required post `subject` strongRef, `labelValue`, `expectedVersion`, and `idempotencyKey`; optional `reason`, and `privateNote`. The initial supported `labelValue` is `nsfw`, with URI-scoped effect and the CID used for inspection/concurrency. Return `#mutationResult`. This procedure cannot accept `!takedown` or arbitrary label values as a shortcut around the removal workflow. | +| `moderation/labelContent.json` / `labelContent` | `procedure`; authentication and instance-admin authorization required | Required post `subject` strongRef, `labelValue`, `expectedVersion`, and `idempotencyKey`; optional `reason`, and `privateNote`. The initial supported `labelValue` is `nsfw`, with URI-scoped effect and the CID used for inspection/concurrency. Return `#mutationResult`. This procedure cannot accept `!takedown` or arbitrary label values as a shortcut around the removal workflow. Decided 2026-09-30 (Q-LABEL-REASON): `doxing` and `illegal-content` are Remove-only reasons, so `labelContent` and `retractContentLabel` reject them with `UnsupportedReason`. | | `moderation/retractContentLabel.json` / `retractContentLabel` | `procedure`; authentication and instance-admin authorization required | Required local `actionId` for an active label application, `expectedVersion`, and `idempotencyKey`; optional `reason`, `reviewedSubject`, and `privateNote`. Require `reviewedSubject` when the current post exists, as for restore. Derive subject/value/source/scope from the referenced action and retract only that local classification. Return `#mutationResult`; a removal action, imported decision, or author self-label is not a valid target. | -| `moderation/listActions.json` / `listActions` | `query`; public, no personalization or extra fields when authenticated | Optional `limit`, `cursor`, `subject`, `action`, `origin`, `authority`, `community`, `since`, `until`, and, if public actor naming is approved, `actor`. Return required `actions` of public `actionView` objects and optional `cursor`. Hidden subjects cannot be identified through filters, counts, cursors, or error differences. | +| `moderation/listActions.json` / `listActions` | `query`; public, no personalization or extra fields when authenticated | Optional `limit`, `cursor`, `subject`, `action`, `origin`, `authority`, `community`, `since`, `until`, and, if public actor naming is approved, `actor`. Return required `actions` of public `actionView` objects and optional `cursor`. Hidden subjects cannot be identified through filters, counts, cursors, or error differences. Decided 2026-09-30 (Q-LABEL-LOG): `listActions` never returns `label` or `retract-label` actions, under any filter or cursor page, and its `action` filter rejects those two values with `InvalidRequest`; `listAdminActions` keeps them. | | `moderation/listAdminActions.json` / `listAdminActions` | `query`; authentication and instance-admin authorization required | Same pagination/filter conventions, plus optional local `actionId` for exact lookup. Return required `actions` of `adminActionView` objects and optional `cursor`. This is the explicit private history/notes surface; there is no `includePrivate` switch on the public endpoint. | | `moderation/getSubjectState.json` / `getSubjectState` | `query`; authentication and instance-admin authorization required | Required `subject` AT URI. Return required `state` of `subjectState`: concurrency version, independent removal/classification state, source-record availability, optional current strongRef/local removal reference, and local label-action references. A syntactically valid URI in a supported content collection has a state even if never indexed: `recordState: unavailable`, initial `version`, and no removal or classifications unless stored decisions apply. Use `InvalidSubject` for malformed/unsupported subjects, not `SubjectNotFound` for an unindexed one. Supplies preconditions for all four mutations; returns no retained raw content. Public summaries are carried by content/tombstone views rather than exposing admin state tokens. | @@ -496,7 +497,7 @@ Extend `moderation/defs.json` with the following **new** definitions. The existi | `#mutationResult` | `outcome`, `state` (`#subjectState`) | Optional `action` (`#adminActionView`) under the applied/unchanged rules above. Only admin procedures return this private result. | | `#publicationView` | `status` (open `pending`/`published`/`failed`) | Federated milestone only; not authored or published with the local-milestone definitions. Optional `publishedAt`, sanitized error code. Absent when no publication obligation exists; never expose signing keys, endpoint credentials, or raw failure text. | -`action` starts with `remove`, `restore`, `apply-removal`, `retract-removal`, `label`, and `retract-label`. For the last two, `labelValue` identifies `nsfw` and `origin` distinguishes local from inherited activity. Additional observed transitions such as expiry or trust-policy changes need distinct documented values when implemented. Unknown received action/status values are displayable as an unrecognized action, never authorization to mutate state. For source inputs, schema openness does not imply trusting unsupported labels or scopes. +`action` starts with `remove`, `restore`, `apply-removal`, `retract-removal`, `label`, and `retract-label`. For the last two, `labelValue` identifies `nsfw` and `origin` distinguishes local from inherited activity. Decided 2026-09-30 (Q-LABEL-LOG): `label` and `retract-label` actions appear only in the admin log, not in the public modlog. Additional observed transitions such as expiry or trust-policy changes need distinct documented values when implemented. Unknown received action/status values are displayable as an unrecognized action, never authorization to mutate state. For source inputs, schema openness does not imply trusting unsupported labels or scopes. For reasons, define `#reasonType` as a bounded string with fully qualified token references in `knownValues`, and token definitions such as `#reasonSpam`, `#reasonHarassment`, `#reasonDoxing`, `#reasonIllegalContent`, `#reasonRuleViolation`, and `#reasonModeratorDiscretion`, each with a precise description. These subjective classifications benefit from a namespaced extension mechanism. The short codes in section 5 are explanatory names; the new moderation API uses tokens. Do not convert already-published `community.removal.code` values or label `val` values to these tokens. A server may reject unsupported reason tokens on writes with `UnsupportedReason`; accepting a token syntactically does not approve its public disclosure. diff --git a/internal/api/handlers/moderation/get_subject_state.go b/internal/api/handlers/moderation/get_subject_state.go index b1a0847..bd5428f 100644 --- a/internal/api/handlers/moderation/get_subject_state.go +++ b/internal/api/handlers/moderation/get_subject_state.go @@ -38,7 +38,7 @@ func (h *GetSubjectStateHandler) HandleGetSubjectState(w http.ResponseWriter, r view := subjectStateView{ Subject: state.Subject, Version: state.Version, - Moderation: moderationView{State: state.Moderation.State}, + Moderation: newModerationView(state.Moderation), RecordState: state.RecordState, } if state.CurrentSubject != nil { @@ -69,7 +69,33 @@ type subjectStateView struct { } type moderationView struct { - State string `json:"state"` + State string `json:"state"` + ContentLabels []contentLabelView `json:"contentLabels,omitempty"` +} + +type contentLabelView struct { + Value string `json:"value"` + Sources []decisionSourceView `json:"sources,omitempty"` +} + +type decisionSourceView struct { + AuthorityDID string `json:"authorityDid"` + Scope moderation.ScopeView `json:"scope"` +} + +func newModerationView(state moderation.ModerationView) moderationView { + view := moderationView{State: state.State} + for _, label := range state.ContentLabels { + item := contentLabelView{Value: label.Value} + for _, source := range label.Sources { + item.Sources = append(item.Sources, decisionSourceView{ + AuthorityDID: source.AuthorityDID, + Scope: moderation.ScopeView{Kind: source.ScopeKind}, + }) + } + view.ContentLabels = append(view.ContentLabels, item) + } + return view } type strongRefView struct { diff --git a/internal/api/handlers/moderation/get_subject_state_test.go b/internal/api/handlers/moderation/get_subject_state_test.go index d53edf8..57cf32b 100644 --- a/internal/api/handlers/moderation/get_subject_state_test.go +++ b/internal/api/handlers/moderation/get_subject_state_test.go @@ -170,3 +170,43 @@ func TestGetSubjectStateHandlerPassesExactQuerySubject(t *testing.T) { _ = requestGetSubjectState(service, subject, true) assert.Equal(t, []string{subject}, service.subjects) } + +func TestGetSubjectStateHandlerSerializesActiveContentLabels(t *testing.T) { + const subject = "at://did:plc:subject/social.coves.community.postv2/3kabc" + const cid = "bafyreib6tbnql2ux3whnfysbzabthaj2vvck53nimhbi5g5a7jgvgr5eqm" + state := &moderation.SubjectState{ + Subject: subject, Version: "v1", RecordState: moderation.RecordStatePresent, + CurrentSubject: &moderation.StrongRef{URI: subject, CID: cid}, + LocalLabels: []moderation.LocalLabel{{ + Value: moderation.LabelNSFW, Action: moderation.ActionRef{ServiceDID: mutationInstanceDID, ActionID: "label-1"}, + }}, + Moderation: moderation.ModerationView{State: moderation.ModerationStateClear, ContentLabels: []moderation.ContentLabel{{ + Value: moderation.LabelNSFW, + Sources: []moderation.DecisionSource{{AuthorityDID: mutationInstanceDID, ScopeKind: moderation.ScopeInstance}}, + }}}, + } + service := &subjectStateServiceFake{state: state} + response := requestGetSubjectState(service, subject, true) + require.Equalf(t, http.StatusOK, response.Code, "response: %s", response.Body.String()) + var body map[string]any + require.NoError(t, json.Unmarshal(response.Body.Bytes(), &body)) + want := map[string]any{"state": map[string]any{ + "subject": subject, "version": "v1", "recordState": "present", + "currentSubject": map[string]any{"uri": subject, "cid": cid}, + "localLabels": []any{map[string]any{ + "value": "nsfw", "action": map[string]any{"serviceDid": mutationInstanceDID, "actionId": "label-1"}, + }}, + "moderation": map[string]any{"state": "clear", "contentLabels": []any{map[string]any{ + "value": "nsfw", "sources": []any{map[string]any{ + "authorityDid": mutationInstanceDID, "scope": map[string]any{"kind": "instance"}, + }}, + }}}, + }} + assert.Equal(t, want, body) + assert.Equal(t, []string{subject}, service.subjects) + catalog := lexicon.NewBaseCatalog() + require.NoError(t, catalog.LoadDirectory("../../../atproto/lexicon")) + decoded, err := atdata.UnmarshalJSON(response.Body.Bytes()) + require.NoError(t, err) + require.NoError(t, validation.ValidateData(catalog, decoded, "social.coves.moderation.getSubjectState#output", 0)) +} diff --git a/internal/api/handlers/moderation/label_content.go b/internal/api/handlers/moderation/label_content.go new file mode 100644 index 0000000..eb3cc15 --- /dev/null +++ b/internal/api/handlers/moderation/label_content.go @@ -0,0 +1,48 @@ +package moderation + +import ( + "net/http" + + "Coves/internal/api/middleware" + "Coves/internal/api/xrpc" + "Coves/internal/core/moderation" +) + +// LabelContentHandler serves social.coves.moderation.labelContent. +type LabelContentHandler struct { + service moderation.Service +} + +// NewLabelContentHandler builds the labelContent handler. +func NewLabelContentHandler(service moderation.Service) *LabelContentHandler { + return &LabelContentHandler{service: service} +} + +// HandleLabelContent handles POST /xrpc/social.coves.moderation.labelContent. +func (h *LabelContentHandler) HandleLabelContent(w http.ResponseWriter, r *http.Request) { + if !requireMutationPOSTJSON(w, r) { + return + } + var input struct { + Subject strongRefView `json:"subject"` + LabelValue string `json:"labelValue"` + ExpectedVersion string `json:"expectedVersion"` + IdempotencyKey string `json:"idempotencyKey"` + Reason string `json:"reason"` + PrivateNote string `json:"privateNote"` + } + if !decodeMutationRequest(w, r, &input) { + return + } + actorDID := middleware.GetUserDID(r) + if actorDID == "" { + xrpc.WriteError(w, http.StatusUnauthorized, "AuthRequired", "Authentication required") + return + } + result, err := h.service.LabelContent(r.Context(), actorDID, moderation.LabelContentRequest{ + Subject: moderation.StrongRef{URI: input.Subject.URI, CID: input.Subject.CID}, + LabelValue: input.LabelValue, ExpectedVersion: input.ExpectedVersion, + IdempotencyKey: input.IdempotencyKey, Reason: input.Reason, PrivateNote: input.PrivateNote, + }) + writeMutationResult(w, "labelContent", result, err) +} diff --git a/internal/api/handlers/moderation/label_content_test.go b/internal/api/handlers/moderation/label_content_test.go new file mode 100644 index 0000000..5aa8026 --- /dev/null +++ b/internal/api/handlers/moderation/label_content_test.go @@ -0,0 +1,151 @@ +package moderation + +import ( + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "Coves/internal/core/moderation" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +const ( + labelHandlerPath = "/xrpc/social.coves.moderation.labelContent" + labelHandlerURI = "at://did:plc:postauthor/social.coves.community.postv2/3kabc" + labelHandlerBody = `{"subject":{"uri":"` + labelHandlerURI + `","cid":"` + mutationSubjectCID + `"},"labelValue":"nsfw","expectedVersion":"v0","idempotencyKey":"label-key","reason":"` + mutationReason + `","privateNote":"` + mutationNote + `"}` +) + +func labelHandlerResult() *moderation.MutationResult { + result := mutationResult(moderation.ActionLabel, mutationNote) + result.State.Subject = labelHandlerURI + result.State.CurrentSubject.URI = labelHandlerURI + result.State.Moderation = moderation.ModerationView{ + State: moderation.ModerationStateClear, + ContentLabels: []moderation.ContentLabel{{ + Value: moderation.LabelNSFW, + Sources: []moderation.DecisionSource{{AuthorityDID: mutationInstanceDID, ScopeKind: moderation.ScopeInstance}}, + }}, + } + result.State.LocalRemoval = nil + result.State.LocalLabels = []moderation.LocalLabel{{ + Value: moderation.LabelNSFW, Action: moderation.ActionRef{ServiceDID: mutationInstanceDID, ActionID: result.Action.ID}, + }} + result.Action.SubjectURI = labelHandlerURI + result.Action.SubjectCollection = moderation.PostV2Collection + result.Action.LabelValue = moderation.LabelNSFW + return result +} + +func requireLabelMutationResponse(t *testing.T, response *httptest.ResponseRecorder, actionKind string) map[string]any { + t.Helper() + require.Equalf(t, http.StatusOK, response.Code, "response: %s", response.Body.String()) + assert.Equal(t, "application/json", response.Header().Get("Content-Type")) + var body map[string]any + require.NoError(t, json.Unmarshal(response.Body.Bytes(), &body)) + assertMutationLexicon(t, response) + assert.Equal(t, moderation.OutcomeApplied, body["outcome"]) + state, ok := body["state"].(map[string]any) + require.True(t, ok, "state must be an object: %s", response.Body.String()) + view, ok := state["moderation"].(map[string]any) + require.True(t, ok) + assert.Equal(t, map[string]any{"state": moderation.ModerationStateClear, "contentLabels": []any{map[string]any{ + "value": moderation.LabelNSFW, + "sources": []any{map[string]any{"authorityDid": mutationInstanceDID, "scope": map[string]any{"kind": moderation.ScopeInstance}}}, + }}}, view) + labels, ok := state["localLabels"].([]any) + require.True(t, ok) + require.Len(t, labels, 1) + label, ok := labels[0].(map[string]any) + require.True(t, ok) + assert.Equal(t, moderation.LabelNSFW, label["value"]) + ref, ok := label["action"].(map[string]any) + require.True(t, ok) + assert.Equal(t, map[string]any{"serviceDid": mutationInstanceDID, "actionId": "action-1"}, ref) + adminAction, ok := body["action"].(map[string]any) + require.True(t, ok) + action, ok := adminAction["action"].(map[string]any) + require.True(t, ok) + assert.Equal(t, actionKind, action["action"]) + assert.Equal(t, moderation.LabelNSFW, action["labelValue"]) + return action +} + +func TestLabelContentHandlerPassesContextActorAndExactRequest(t *testing.T) { + fake := &mutationServiceFake{result: labelHandlerResult()} + response := httptest.NewRecorder() + NewLabelContentHandler(fake).HandleLabelContent(response, + mutationRequest(http.MethodPost, labelHandlerPath, labelHandlerBody, "application/json")) + require.Equalf(t, http.StatusOK, response.Code, "response: %s", response.Body.String()) + require.NotEmpty(t, response.Body.Bytes(), "successful labelContent must serialize a mutation result") + assert.Equal(t, []string{mutationActorDID}, fake.actors) + assert.Equal(t, []moderation.LabelContentRequest{{ + Subject: moderation.StrongRef{URI: labelHandlerURI, CID: mutationSubjectCID}, + LabelValue: moderation.LabelNSFW, ExpectedVersion: "v0", IdempotencyKey: "label-key", + Reason: mutationReason, PrivateNote: mutationNote, + }}, fake.labelCalls) + _ = requireLabelMutationResponse(t, response, moderation.ActionLabel) +} + +func TestLabelContentHandlerRejectsInvalidHTTPAndMissingActor(t *testing.T) { + for _, test := range []struct { + name, method, body, contentType string + actor bool + status int + code string + }{ + {"GET", http.MethodGet, "", "", true, http.StatusMethodNotAllowed, "MethodNotAllowed"}, + {"missing content type", http.MethodPost, labelHandlerBody, "", true, http.StatusBadRequest, "InvalidRequest"}, + {"text/plain", http.MethodPost, labelHandlerBody, "text/plain", true, http.StatusBadRequest, "InvalidRequest"}, + {"malformed JSON", http.MethodPost, `{"subject":`, "application/json", true, http.StatusBadRequest, "InvalidRequest"}, + {"no actor", http.MethodPost, labelHandlerBody, "application/json", false, http.StatusUnauthorized, "AuthRequired"}, + } { + t.Run(test.name, func(t *testing.T) { + fake := &mutationServiceFake{result: labelHandlerResult()} + var request *http.Request + if test.actor { + request = mutationRequest(test.method, labelHandlerPath, test.body, test.contentType) + } else { + request = httptest.NewRequest(test.method, labelHandlerPath, strings.NewReader(test.body)) + request.Header.Set("Content-Type", test.contentType) + } + response := httptest.NewRecorder() + NewLabelContentHandler(fake).HandleLabelContent(response, request) + require.Equalf(t, test.status, response.Code, "response: %s", response.Body.String()) + assertMutationError(t, response, test.status, test.code) + assert.Empty(t, fake.labelCalls) + }) + } +} + +func TestLabelContentHandlerMapsServiceErrors(t *testing.T) { + for _, test := range []struct { + name string + err error + status int + code string + }{ + {"invalid request", moderation.ErrInvalidRequest, http.StatusBadRequest, "InvalidRequest"}, + {"invalid subject", moderation.ErrInvalidSubject, http.StatusBadRequest, "InvalidSubject"}, + {"subject not found", moderation.ErrSubjectNotFound, http.StatusBadRequest, "SubjectNotFound"}, + {"content changed", moderation.ErrContentChanged, http.StatusBadRequest, "ContentChanged"}, + {"state conflict", moderation.ErrStateConflict, http.StatusBadRequest, "StateConflict"}, + {"idempotency conflict", moderation.ErrIdempotencyConflict, http.StatusBadRequest, "IdempotencyConflict"}, + {"unsupported reason", moderation.ErrUnsupportedReason, http.StatusBadRequest, "UnsupportedReason"}, + {"unsupported label", moderation.ErrUnsupportedLabel, http.StatusBadRequest, "UnsupportedLabel"}, + {"unavailable", moderation.ErrModerationUnavailable, http.StatusServiceUnavailable, "ModerationUnavailable"}, + } { + t.Run(test.name, func(t *testing.T) { + fake := &mutationServiceFake{err: fmt.Errorf("mutation failed: %w", test.err)} + response := httptest.NewRecorder() + NewLabelContentHandler(fake).HandleLabelContent(response, + mutationRequest(http.MethodPost, labelHandlerPath, labelHandlerBody, "application/json")) + require.Equalf(t, test.status, response.Code, "response: %s", response.Body.String()) + assertMutationError(t, response, test.status, test.code) + assert.Len(t, fake.labelCalls, 1) + }) + } +} diff --git a/internal/api/handlers/moderation/remove_content.go b/internal/api/handlers/moderation/remove_content.go index 6a042be..6d6db75 100644 --- a/internal/api/handlers/moderation/remove_content.go +++ b/internal/api/handlers/moderation/remove_content.go @@ -80,7 +80,7 @@ func writeMutationResult(w http.ResponseWriter, operation string, result *modera state := result.State view := subjectStateView{ Subject: state.Subject, Version: state.Version, - Moderation: moderationView{State: state.Moderation.State}, RecordState: state.RecordState, + Moderation: newModerationView(state.Moderation), RecordState: state.RecordState, } if state.CurrentSubject != nil { view.CurrentSubject = &strongRefView{URI: state.CurrentSubject.URI, CID: state.CurrentSubject.CID} @@ -120,6 +120,7 @@ func writeMutationError(w http.ResponseWriter, operation string, err error) { {moderation.ErrStateConflict, "StateConflict"}, {moderation.ErrIdempotencyConflict, "IdempotencyConflict"}, {moderation.ErrUnsupportedReason, "UnsupportedReason"}, + {moderation.ErrUnsupportedLabel, "UnsupportedLabel"}, } { if errors.Is(err, entry.cause) { // Rule errors carry only fixed text and configured limits, such as diff --git a/internal/api/handlers/moderation/remove_content_test.go b/internal/api/handlers/moderation/remove_content_test.go index 263b8fc..2ca7067 100644 --- a/internal/api/handlers/moderation/remove_content_test.go +++ b/internal/api/handlers/moderation/remove_content_test.go @@ -35,6 +35,8 @@ type mutationServiceFake struct { moderation.Service removeCalls []moderation.RemoveContentRequest restoreCalls []moderation.RestoreContentRequest + labelCalls []moderation.LabelContentRequest + retractCalls []moderation.RetractContentLabelRequest actors []string result *moderation.MutationResult err error @@ -52,6 +54,18 @@ func (fake *mutationServiceFake) RestoreContent(_ context.Context, actorDID stri return fake.result, fake.err } +func (fake *mutationServiceFake) LabelContent(_ context.Context, actorDID string, request moderation.LabelContentRequest) (*moderation.MutationResult, error) { + fake.actors = append(fake.actors, actorDID) + fake.labelCalls = append(fake.labelCalls, request) + return fake.result, fake.err +} + +func (fake *mutationServiceFake) RetractContentLabel(_ context.Context, actorDID string, request moderation.RetractContentLabelRequest) (*moderation.MutationResult, error) { + fake.actors = append(fake.actors, actorDID) + fake.retractCalls = append(fake.retractCalls, request) + return fake.result, fake.err +} + func mutationRequest(method, path, body, contentType string) *http.Request { request := httptest.NewRequest(method, path, strings.NewReader(body)) if contentType != "" { @@ -115,6 +129,11 @@ func assertMutationResponse(t *testing.T, response *httptest.ResponseRecorder, r want["action"] = adminAction } assert.Equal(t, want, body, "response must omit null optional fields and expose the exact action projection") + assertMutationLexicon(t, response) +} + +func assertMutationLexicon(t *testing.T, response *httptest.ResponseRecorder) { + t.Helper() catalog := lexicon.NewBaseCatalog() require.NoError(t, catalog.LoadDirectory("../../../atproto/lexicon")) decoded, err := atdata.UnmarshalJSON(response.Body.Bytes()) diff --git a/internal/api/handlers/moderation/retract_content_label.go b/internal/api/handlers/moderation/retract_content_label.go new file mode 100644 index 0000000..3e75405 --- /dev/null +++ b/internal/api/handlers/moderation/retract_content_label.go @@ -0,0 +1,51 @@ +package moderation + +import ( + "net/http" + + "Coves/internal/api/middleware" + "Coves/internal/api/xrpc" + "Coves/internal/core/moderation" +) + +// RetractContentLabelHandler serves social.coves.moderation.retractContentLabel. +type RetractContentLabelHandler struct { + service moderation.Service +} + +// NewRetractContentLabelHandler builds the retractContentLabel handler. +func NewRetractContentLabelHandler(service moderation.Service) *RetractContentLabelHandler { + return &RetractContentLabelHandler{service: service} +} + +// HandleRetractContentLabel handles POST /xrpc/social.coves.moderation.retractContentLabel. +func (h *RetractContentLabelHandler) HandleRetractContentLabel(w http.ResponseWriter, r *http.Request) { + if !requireMutationPOSTJSON(w, r) { + return + } + var input struct { + ActionID string `json:"actionId"` + ReviewedSubject *strongRefView `json:"reviewedSubject"` + ExpectedVersion string `json:"expectedVersion"` + IdempotencyKey string `json:"idempotencyKey"` + Reason string `json:"reason"` + PrivateNote string `json:"privateNote"` + } + if !decodeMutationRequest(w, r, &input) { + return + } + actorDID := middleware.GetUserDID(r) + if actorDID == "" { + xrpc.WriteError(w, http.StatusUnauthorized, "AuthRequired", "Authentication required") + return + } + request := moderation.RetractContentLabelRequest{ + ActionID: input.ActionID, ExpectedVersion: input.ExpectedVersion, + IdempotencyKey: input.IdempotencyKey, Reason: input.Reason, PrivateNote: input.PrivateNote, + } + if input.ReviewedSubject != nil { + request.ReviewedSubject = &moderation.StrongRef{URI: input.ReviewedSubject.URI, CID: input.ReviewedSubject.CID} + } + result, err := h.service.RetractContentLabel(r.Context(), actorDID, request) + writeMutationResult(w, "retractContentLabel", result, err) +} diff --git a/internal/api/handlers/moderation/retract_content_label_test.go b/internal/api/handlers/moderation/retract_content_label_test.go new file mode 100644 index 0000000..1792064 --- /dev/null +++ b/internal/api/handlers/moderation/retract_content_label_test.go @@ -0,0 +1,119 @@ +package moderation + +import ( + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "Coves/internal/core/moderation" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +const ( + retractHandlerPath = "/xrpc/social.coves.moderation.retractContentLabel" + retractHandlerBody = `{"actionId":"previous-label","reviewedSubject":{"uri":"` + labelHandlerURI + `","cid":"` + mutationSubjectCID + `"},"expectedVersion":"v1","idempotencyKey":"retract-key","reason":"` + mutationReason + `","privateNote":"` + mutationNote + `"}` +) + +func retractHandlerResult() *moderation.MutationResult { + result := labelHandlerResult() + result.State.Version = "v2" + result.State.LocalLabels = nil + result.State.Moderation.ContentLabels = nil + result.Action.Action = moderation.ActionRetractLabel + result.Action.ReversesActionID = "previous-label" + return result +} + +func TestRetractContentLabelHandlerPassesContextActorAndExactRequest(t *testing.T) { + for _, test := range []struct { + name, body string + reviewed *moderation.StrongRef + reason string + note string + }{ + {"reviewed subject and note", retractHandlerBody, &moderation.StrongRef{URI: labelHandlerURI, CID: mutationSubjectCID}, mutationReason, mutationNote}, + {"optional fields omitted", `{"actionId":"previous-label","expectedVersion":"v1","idempotencyKey":"retract-key"}`, nil, "", ""}, + } { + t.Run(test.name, func(t *testing.T) { + fake := &mutationServiceFake{result: retractHandlerResult()} + response := httptest.NewRecorder() + NewRetractContentLabelHandler(fake).HandleRetractContentLabel(response, + mutationRequest(http.MethodPost, retractHandlerPath, test.body, "application/json")) + require.Equalf(t, http.StatusOK, response.Code, "response: %s", response.Body.String()) + require.NotEmpty(t, response.Body.Bytes(), "successful retractContentLabel must serialize a mutation result") + assert.Equal(t, []string{mutationActorDID}, fake.actors) + assert.Equal(t, []moderation.RetractContentLabelRequest{{ + ActionID: "previous-label", ReviewedSubject: test.reviewed, ExpectedVersion: "v1", + IdempotencyKey: "retract-key", Reason: test.reason, PrivateNote: test.note, + }}, fake.retractCalls) + require.Equalf(t, http.StatusOK, response.Code, "response: %s", response.Body.String()) + var body map[string]any + require.NoError(t, json.Unmarshal(response.Body.Bytes(), &body)) + assertMutationLexicon(t, response) + assert.Equal(t, moderation.OutcomeApplied, body["outcome"]) + state, ok := body["state"].(map[string]any) + require.True(t, ok) + assert.NotContains(t, state, "localLabels") + assert.Equal(t, map[string]any{"state": moderation.ModerationStateClear}, state["moderation"]) + adminAction, ok := body["action"].(map[string]any) + require.True(t, ok) + action, ok := adminAction["action"].(map[string]any) + require.True(t, ok) + assert.Equal(t, moderation.ActionRetractLabel, action["action"]) + assert.Equal(t, moderation.LabelNSFW, action["labelValue"]) + assert.Equal(t, map[string]any{"serviceDid": mutationInstanceDID, "actionId": "previous-label"}, action["reverses"]) + }) + } +} + +func TestRetractContentLabelHandlerRejectsInvalidHTTPAndMissingActor(t *testing.T) { + for _, test := range []struct { + name, method, body, contentType string + actor bool + status int + code string + }{ + {"GET", http.MethodGet, "", "", true, http.StatusMethodNotAllowed, "MethodNotAllowed"}, + {"missing content type", http.MethodPost, retractHandlerBody, "", true, http.StatusBadRequest, "InvalidRequest"}, + {"text/plain", http.MethodPost, retractHandlerBody, "text/plain", true, http.StatusBadRequest, "InvalidRequest"}, + {"malformed JSON", http.MethodPost, `{"actionId":`, "application/json", true, http.StatusBadRequest, "InvalidRequest"}, + {"no actor", http.MethodPost, retractHandlerBody, "application/json", false, http.StatusUnauthorized, "AuthRequired"}, + } { + t.Run(test.name, func(t *testing.T) { + fake := &mutationServiceFake{result: retractHandlerResult()} + var request *http.Request + if test.actor { + request = mutationRequest(test.method, retractHandlerPath, test.body, test.contentType) + } else { + request = httptest.NewRequest(test.method, retractHandlerPath, strings.NewReader(test.body)) + request.Header.Set("Content-Type", test.contentType) + } + response := httptest.NewRecorder() + NewRetractContentLabelHandler(fake).HandleRetractContentLabel(response, request) + require.Equalf(t, test.status, response.Code, "response: %s", response.Body.String()) + assertMutationError(t, response, test.status, test.code) + assert.Empty(t, fake.retractCalls) + }) + } +} + +func TestRetractContentLabelHandlerMapsServiceErrors(t *testing.T) { + for _, test := range mutationErrors { + if test.err == moderation.ErrInvalidSubject || test.err == moderation.ErrSubjectNotFound { + continue + } + t.Run(test.name, func(t *testing.T) { + fake := &mutationServiceFake{err: fmt.Errorf("mutation failed: %w", test.err)} + response := httptest.NewRecorder() + NewRetractContentLabelHandler(fake).HandleRetractContentLabel(response, + mutationRequest(http.MethodPost, retractHandlerPath, retractHandlerBody, "application/json")) + require.Equalf(t, test.status, response.Code, "response: %s", response.Body.String()) + assertMutationError(t, response, test.status, test.code) + assert.Len(t, fake.retractCalls, 1) + }) + } +} diff --git a/internal/api/routes/moderation.go b/internal/api/routes/moderation.go index a11d232..36cb44d 100644 --- a/internal/api/routes/moderation.go +++ b/internal/api/routes/moderation.go @@ -17,6 +17,8 @@ func RegisterModerationRoutes(r chi.Router, service moderation.Service, adminAut listAdminActions := handler.NewListAdminActionsHandler(service) removeContent := handler.NewRemoveContentHandler(service) restoreContent := handler.NewRestoreContentHandler(service) + labelContent := handler.NewLabelContentHandler(service) + retractContentLabel := handler.NewRetractContentLabelHandler(service) r.Get("/xrpc/social.coves.moderation.listActions", listActions.HandleListActions) r.With(adminAuth.RequireInstanceAdmin).Get( "/xrpc/social.coves.moderation.listAdminActions", listAdminActions.HandleListAdminActions) @@ -26,4 +28,8 @@ func RegisterModerationRoutes(r chi.Router, service moderation.Service, adminAut "/xrpc/social.coves.moderation.removeContent", removeContent.HandleRemoveContent) r.With(adminAuth.RequireInstanceAdmin).Post( "/xrpc/social.coves.moderation.restoreContent", restoreContent.HandleRestoreContent) + r.With(adminAuth.RequireInstanceAdmin).Post( + "/xrpc/social.coves.moderation.labelContent", labelContent.HandleLabelContent) + r.With(adminAuth.RequireInstanceAdmin).Post( + "/xrpc/social.coves.moderation.retractContentLabel", retractContentLabel.HandleRetractContentLabel) } diff --git a/internal/api/routes/moderation_content_label_integration_test.go b/internal/api/routes/moderation_content_label_integration_test.go new file mode 100644 index 0000000..6284f30 --- /dev/null +++ b/internal/api/routes/moderation_content_label_integration_test.go @@ -0,0 +1,483 @@ +//go:build integration + +package routes_test + +import ( + "database/sql" + "encoding/json" + "net/http" + "net/http/httptest" + "net/url" + "sync/atomic" + "testing" + "time" + + "Coves/internal/api/middleware" + "Coves/internal/api/routes" + "Coves/internal/core/communities" + "Coves/internal/core/communityFeeds" + "Coves/internal/core/moderation" + "Coves/internal/core/posts" + "Coves/internal/crypto/credentialcipher/credentialciphertest" + "Coves/internal/db/postgres" + "Coves/internal/validation" + "Coves/tests/fixtures" + "Coves/tests/testkit" + + "github.com/bluesky-social/indigo/atproto/atdata" + "github.com/bluesky-social/indigo/atproto/lexicon" + "github.com/go-chi/chi/v5" + "github.com/stretchr/testify/require" +) + +type contentLabelRouteFixture struct { + db *sql.DB + server *httptest.Server + client *http.Client + instanceDID string + authorDID string + communityDID string + adminDID string + adminToken string + nonAdminToken string + postCID string + pdsRequests *atomic.Int64 + insertPost func(title string) string +} + +func newContentLabelRouteFixture(t *testing.T) contentLabelRouteFixture { + t.Helper() + db := testkit.DB(t) + var pdsRequests atomic.Int64 + pds := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + pdsRequests.Add(1) + http.NotFound(w, nil) + })) + t.Cleanup(pds.Close) + t.Cleanup(func() { require.Zero(t, pdsRequests.Load(), "moderation must never contact the PDS") }) + + postRepo := postgres.NewPostRepository(db) + instanceDID := fixtures.InstanceDID() + moderationService := moderation.NewService( + moderation.NewRepositorySubjectReader(postRepo, postgres.NewCommentRepository(db)), + postgres.NewModerationRepository(db), + moderation.Config{InstanceDID: instanceDID, IdempotencyRetention: 24 * time.Hour, MaxLiveIdempotencyKeys: 1000, CursorSecret: "content-label-acceptance-secret"}, + ) + communityRepo := postgres.NewCommunityRepository(db, credentialciphertest.Fixed()) + communityService := communities.NewCommunityServiceWithPDSFactory( + communityRepo, pds.URL, instanceDID, "", nil, nil, nil, + communities.PrivateHostOptions(true)..., + ) + postService := posts.NewPostService(postRepo, communityService, nil, nil, nil, nil, pds.URL, + posts.WithAdmissionPolicy(posts.NewAllowAllAdmissionPolicyForTests()), + posts.WithSyncAcceptance(postgres.NewAdmissionRepository(db), nil), + ) + feedService := communityFeeds.NewCommunityFeedService( + postgres.NewCommunityFeedRepository(db, "content-label-acceptance-cursor-secret"), communityService, + ) + + authorName := testkit.UniqueIDWithPrefix(t, "labelauthor") + const authorDID = "did:plc:bbbbbbbbbbbbbbbbbbbbbbbb" + fixtures.User(t, db, authorName+".test", authorDID) + communityName := testkit.UniqueIDWithPrefix(t, "labelcommunity") + const communityDID = "did:plc:cccccccccccccccccccccccc" + const ownerDID = "did:plc:dddddddddddddddddddddddd" + fixtures.User(t, db, "owner"+communityName+".test", ownerDID) + _, err := db.ExecContext(t.Context(), ` + INSERT INTO communities (did, name, owner_did, created_by_did, hosted_by_did, handle, pds_url, created_at) + VALUES ($1, $2, $3, $3, $4, $5, $6, NOW()) + `, communityDID, communityName, ownerDID, instanceDID, communityName+".coves.social", pds.URL) + require.NoError(t, err) + const postCID = "bafyreihgdyzzpkkzq2izfnhcmm77ycuacvkuziwbnqxfxtqsz7tmxwhnshi" + insertAcceptedPost := func(title string) string { + t.Helper() + rkey := testkit.TID() + uri := "at://" + authorDID + "/" + moderation.PostV2Collection + "/" + rkey + _, err := db.ExecContext(t.Context(), ` + INSERT INTO posts (uri, cid, rkey, author_did, community_did, title, content, created_at) + VALUES ($1, $2, $3, $4, $5, $6, $7, NOW()) + `, uri, postCID, rkey, authorDID, communityDID, title, "original record body") + require.NoError(t, err) + _, err = db.ExecContext(t.Context(), ` + INSERT INTO community_post_admissions + (community_did, post_uri, status, accepted_cid, evaluated_cid, last_community_rev, last_community_op_rank, created_at, updated_at) + VALUES ($1, $2, 'accepted', $3, $3, '3lqqqqqqqqqq2', 1, NOW(), NOW()) + `, communityDID, uri, postCID) + require.NoError(t, err) + return uri + } + adminDID := fixtures.DID(testkit.UniqueIDWithPrefix(t, "labeladmin")) + const adminToken = "content-label-admin-session" + unsealer := fixtures.NewSessionUnsealer() + oauthStore := fixtures.NewOAuthStore() + unsealer.AddSession(adminToken, adminDID, "content-label-admin") + oauthStore.AddSession(adminDID, "content-label-admin", "test-access-token") + adminAuth := middleware.NewInstanceAdminMiddleware(unsealer, oauthStore, nil, + moderation.NewAllowlistAuthority([]string{adminDID})) + nonAdminDID := fixtures.DID(testkit.UniqueIDWithPrefix(t, "labelother")) + const nonAdminToken = "content-label-nonadmin-session" + unsealer.AddSession(nonAdminToken, nonAdminDID, "content-label-nonadmin") + oauthStore.AddSession(nonAdminDID, "content-label-nonadmin", "non-admin-access-token") + optionalAuth := middleware.NewOAuthAuthMiddleware(unsealer, oauthStore) + mux := chi.NewRouter() + routes.RegisterModerationRoutes(mux, moderationService, adminAuth) + routes.RegisterPostRoutes(mux, postService, nil, nil, optionalAuth, optionalAuth) + routes.RegisterCommunityFeedRoutes(mux, feedService, nil, nil, optionalAuth) + routes.RegisterActorRoutes(mux, postService, nil, nil, nil, nil, optionalAuth) + server := httptest.NewServer(mux) + t.Cleanup(server.Close) + return contentLabelRouteFixture{ + db: db, server: server, client: server.Client(), instanceDID: instanceDID, + authorDID: authorDID, communityDID: communityDID, adminDID: adminDID, + adminToken: adminToken, nonAdminToken: nonAdminToken, postCID: postCID, + pdsRequests: &pdsRequests, insertPost: insertAcceptedPost, + } +} + +func TestModerationContentLabelAcceptance(t *testing.T) { + f := newContentLabelRouteFixture(t) + db, server, client, instanceDID, adminDID, adminToken := f.db, f.server, f.client, f.instanceDID, f.adminDID, f.adminToken + communityDID, postCID, pdsRequests := f.communityDID, f.postCID, f.pdsRequests + postURI := f.insertPost("original labelled post") + controlURI := f.insertPost("unlabelled control post") + postURL := server.URL + "/xrpc/social.coves.community.post.get?" + url.Values{"uris": {postURI}}.Encode() + feedURL := server.URL + "/xrpc/social.coves.communityFeed.getCommunity?" + url.Values{"community": {communityDID}, "sort": {"new"}}.Encode() + + catalog := lexicon.NewBaseCatalog() + require.NoError(t, catalog.LoadDirectory("../../atproto/lexicon")) + validate := func(response subjectStateHTTPResponse, lexiconID string) { + t.Helper() + data, err := atdata.UnmarshalJSON(response.body) + require.NoError(t, err) + require.NoError(t, validation.ValidateData(catalog, data, lexiconID, 0)) + } + getPost := func() (map[string]any, json.RawMessage) { + t.Helper() + response := moderationAcceptanceRequest(t, client, http.MethodGet, postURL, "", nil) + body := moderationAcceptanceBody(t, response) + items := moderationAcceptanceArray(t, body["posts"]) + require.Len(t, items, 1) + item := moderationAcceptanceObject(t, items[0]) + require.Equal(t, postURI, item["uri"]) + require.NotEqual(t, "social.coves.community.post.defs#moderatedPost", item["$type"]) + encoded, err := json.Marshal(item) + require.NoError(t, err) + data, err := atdata.UnmarshalJSON(encoded) + require.NoError(t, err) + require.NoError(t, validation.ValidateData(catalog, data, "social.coves.community.post.defs#postView", 0)) + var raw struct { + Posts []struct { + Record json.RawMessage `json:"record"` + } `json:"posts"` + } + require.NoError(t, json.Unmarshal(response.body, &raw)) + require.Len(t, raw.Posts, 1) + require.NotEmpty(t, raw.Posts[0].Record) + return item, raw.Posts[0].Record + } + labelView := map[string]any{ + "state": "clear", + "contentLabels": []any{map[string]any{ + "value": "nsfw", "sources": []any{map[string]any{ + "authorityDid": instanceDID, "scope": map[string]any{"kind": "instance"}, + }}, + }}, + } + + initialPost, initialRecord := getPost() + require.NotContains(t, initialPost, "moderation") + initialFeed := moderationPostAcceptanceFeedURIs(t, moderationAcceptanceBody(t, + moderationAcceptanceRequest(t, client, http.MethodGet, feedURL, "", nil))) + require.ElementsMatch(t, []string{postURI, controlURI}, initialFeed) + + labelResponse := moderationAcceptanceRequest(t, client, http.MethodPost, + server.URL+"/xrpc/social.coves.moderation.labelContent", adminToken, map[string]any{ + "subject": map[string]string{"uri": postURI, "cid": postCID}, + "labelValue": "nsfw", "expectedVersion": "v0", "idempotencyKey": testkit.UniqueIDWithPrefix(t, "label"), + }) + labelResult := moderationAcceptanceBody(t, labelResponse) + validate(labelResponse, "social.coves.moderation.defs#mutationResult") + require.Equal(t, "applied", labelResult["outcome"]) + labelAction := modlogAction(t, labelResult["action"], true) + require.Equal(t, "label", labelAction["action"]) + require.Equal(t, "nsfw", labelAction["labelValue"]) + labelID := modlogID(t, labelAction) + + labelledPost, labelledRecord := getPost() + require.Equal(t, []byte(initialRecord), []byte(labelledRecord), "label must not rewrite the served record") + require.Equal(t, labelView, labelledPost["moderation"]) + stateResponse := requestSubjectState(t, client, server.URL, postURI, adminToken) + stateBody := moderationAcceptanceBody(t, stateResponse) + validate(stateResponse, "social.coves.moderation.getSubjectState#output") + state := moderationAcceptanceObject(t, stateBody["state"]) + require.Equal(t, []any{map[string]any{ + "value": "nsfw", "action": map[string]any{"serviceDid": instanceDID, "actionId": labelID}, + }}, state["localLabels"]) + require.Equal(t, "nsfw", moderationAcceptanceObject(t, + moderationAcceptanceArray(t, moderationAcceptanceObject(t, state["moderation"])["contentLabels"])[0])["value"]) + labelledFeed := moderationAcceptanceBody(t, moderationAcceptanceRequest(t, client, http.MethodGet, feedURL, "", nil)) + require.Equal(t, initialFeed, moderationPostAcceptanceFeedURIs(t, labelledFeed)) + feedItems := moderationAcceptanceArray(t, labelledFeed["feed"]) + var feedPost map[string]any + for _, entry := range feedItems { + post := moderationAcceptanceObject(t, moderationAcceptanceObject(t, entry)["post"]) + if post["uri"] == postURI { + feedPost = post + } + } + require.NotNil(t, feedPost) + require.Equal(t, labelView, feedPost["moderation"]) + version, ok := state["version"].(string) + require.True(t, ok) + + retractResponse := moderationAcceptanceRequest(t, client, http.MethodPost, + server.URL+"/xrpc/social.coves.moderation.retractContentLabel", adminToken, map[string]any{ + "actionId": labelID, "expectedVersion": version, + "idempotencyKey": testkit.UniqueIDWithPrefix(t, "retract"), + "reviewedSubject": map[string]string{"uri": postURI, "cid": postCID}, + }) + retractResult := moderationAcceptanceBody(t, retractResponse) + validate(retractResponse, "social.coves.moderation.defs#mutationResult") + require.Equal(t, "applied", retractResult["outcome"]) + retractAction := modlogAction(t, retractResult["action"], true) + require.Equal(t, "retract-label", retractAction["action"]) + require.Equal(t, "nsfw", retractAction["labelValue"]) + require.Equal(t, labelID, moderationAcceptanceObject(t, retractAction["reverses"])["actionId"]) + retractedPost, retractedRecord := getPost() + require.Equal(t, []byte(initialRecord), []byte(retractedRecord)) + require.NotContains(t, retractedPost, "moderation") + logResponse := moderationAcceptanceRequest(t, client, http.MethodGet, + server.URL+modlogPublicPath+"?"+url.Values{"subject": {postURI}}.Encode(), "", nil) + _, publicActions := modlogPage(t, logResponse, false) + require.Empty(t, publicActions, "the public log never serves label or retract-label actions") + adminLogResponse := moderationAcceptanceRequest(t, client, http.MethodGet, + server.URL+modlogAdminPath+"?"+url.Values{"subject": {postURI}}.Encode(), adminToken, nil) + _, adminActions := modlogPage(t, adminLogResponse, true) + require.Len(t, adminActions, 2) + actionsByKind := make(map[string]map[string]any) + for _, item := range adminActions { + action := modlogAction(t, item, true) + kind, ok := action["action"].(string) + require.True(t, ok) + actionsByKind[kind] = action + } + require.Len(t, actionsByKind, 2) + require.Contains(t, actionsByKind, "label") + require.Contains(t, actionsByKind, "retract-label") + require.Equal(t, "nsfw", actionsByKind["label"]["labelValue"]) + require.Equal(t, postURI, moderationAcceptanceObject(t, actionsByKind["label"]["subject"])["uri"]) + require.Equal(t, labelID, modlogID(t, actionsByKind["label"])) + require.Equal(t, "nsfw", actionsByKind["retract-label"]["labelValue"]) + require.Equal(t, labelID, moderationAcceptanceObject(t, actionsByKind["retract-label"]["reverses"])["actionId"]) + + // A decision from another authority must retain its own source identity; + // an inactive decision on the same subject must not contribute a source. + for _, decision := range []struct { + authority string + active bool + }{ + {"did:plc:eeeeeeeeeeeeeeeeeeeeeeee", true}, + {"did:plc:ffffffffffffffffffffffff", false}, + } { + actionID := testkit.TID() + _, err := db.ExecContext(t.Context(), ` + INSERT INTO moderation_actions + (id, actor_did, authority_did, scope_kind, subject_uri, subject_collection, + subject_community_did, observed_cid, action, label_value, origin, created_at) + VALUES ($1, $2, $3, 'instance', $4, $5, $6, $7, 'label', 'nsfw', 'inherited', NOW()) + `, actionID, adminDID, decision.authority, postURI, moderation.PostV2Collection, communityDID, postCID) + require.NoError(t, err) + _, err = db.ExecContext(t.Context(), ` + INSERT INTO moderation_decisions + (authority_did, scope_kind, subject_uri, kind, value, active_action_id, active) + VALUES ($1, 'instance', $2, 'label', 'nsfw', $3, $4) + `, decision.authority, postURI, actionID, decision.active) + require.NoError(t, err) + } + foreignView := map[string]any{ + "state": "clear", + "contentLabels": []any{map[string]any{ + "value": "nsfw", "sources": []any{map[string]any{ + "authorityDid": "did:plc:eeeeeeeeeeeeeeeeeeeeeeee", "scope": map[string]any{"kind": "instance"}, + }}, + }}, + } + foreignPost, _ := getPost() + require.Equal(t, foreignView, foreignPost["moderation"]) + foreignFeed := moderationAcceptanceBody(t, moderationAcceptanceRequest(t, client, http.MethodGet, feedURL, "", nil)) + require.Equal(t, initialFeed, moderationPostAcceptanceFeedURIs(t, foreignFeed)) + for _, entry := range moderationAcceptanceArray(t, foreignFeed["feed"]) { + post := moderationAcceptanceObject(t, moderationAcceptanceObject(t, entry)["post"]) + if post["uri"] == postURI { + require.Equal(t, foreignView, post["moderation"]) + } + } + require.Zero(t, pdsRequests.Load(), "apply and retract must make no PDS requests") +} + +const ( + contentLabelRouteLabelPath = "/xrpc/social.coves.moderation.labelContent" + contentLabelRouteRetractPath = "/xrpc/social.coves.moderation.retractContentLabel" +) + +func (f contentLabelRouteFixture) post(t *testing.T, path, token string, body map[string]any) subjectStateHTTPResponse { + t.Helper() + return moderationAcceptanceRequest(t, f.client, http.MethodPost, f.server.URL+path, token, body) +} + +func (f contentLabelRouteFixture) label(t *testing.T, postURI, version, reason string) (string, string) { + t.Helper() + request := map[string]any{ + "subject": map[string]string{"uri": postURI, "cid": f.postCID}, "labelValue": "nsfw", + "expectedVersion": version, "idempotencyKey": testkit.UniqueIDWithPrefix(t, "label"), + } + if reason != "" { + request["reason"] = reason + } + result := moderationAcceptanceBody(t, f.post(t, contentLabelRouteLabelPath, f.adminToken, request)) + require.Equal(t, "applied", result["outcome"]) + labelVersion, ok := moderationAcceptanceObject(t, result["state"])["version"].(string) + require.True(t, ok) + return modlogID(t, modlogAction(t, result["action"], true)), labelVersion +} + +// contentLabelRouteSnapshot is everything a rejected label mutation must leave +// unchanged: the admin subject state and both logs' entries for the subject. +type contentLabelRouteSnapshot struct { + version any + localLabels any + moderation any + publicLogIDs []string + adminLogIDs []string +} + +func (f contentLabelRouteFixture) snapshot(t *testing.T, postURI string) contentLabelRouteSnapshot { + t.Helper() + state := moderationAcceptanceObject(t, moderationAcceptanceBody(t, + requestSubjectState(t, f.client, f.server.URL, postURI, f.adminToken))["state"]) + logIDs := func(path string, admin bool) []string { + response := moderationAcceptanceRequest(t, f.client, http.MethodGet, + f.server.URL+path+"?"+url.Values{"subject": {postURI}}.Encode(), f.adminToken, nil) + _, actions := modlogPage(t, response, admin) + ids := make([]string, 0, len(actions)) + for _, item := range actions { + ids = append(ids, modlogID(t, modlogAction(t, item, admin))) + } + return ids + } + return contentLabelRouteSnapshot{ + version: state["version"], localLabels: state["localLabels"], moderation: state["moderation"], + publicLogIDs: logIDs(modlogPublicPath, false), adminLogIDs: logIDs(modlogAdminPath, true), + } +} + +func TestModerationContentLabelRejectsRemovalOnlyReasons(t *testing.T) { + f := newContentLabelRouteFixture(t) + postURI := f.insertPost("post an admin reviews for a removal-only reason") + removalOnlyReasons := []string{ + "social.coves.moderation.defs#reasonDoxing", + "social.coves.moderation.defs#reasonIllegalContent", + } + + unlabelled := f.snapshot(t, postURI) + require.Equal(t, "v0", unlabelled.version) + require.Empty(t, unlabelled.adminLogIDs) + for _, reason := range removalOnlyReasons { + response := f.post(t, contentLabelRouteLabelPath, f.adminToken, map[string]any{ + "subject": map[string]string{"uri": postURI, "cid": f.postCID}, "labelValue": "nsfw", + "expectedVersion": "v0", "idempotencyKey": testkit.UniqueIDWithPrefix(t, "label"), "reason": reason, + }) + requireXRPCError(t, response, http.StatusBadRequest, "UnsupportedReason") + require.Equal(t, unlabelled, f.snapshot(t, postURI), "a rejected label must not write an action or change state") + } + + labelID, version := f.label(t, postURI, "v0", "social.coves.moderation.defs#reasonSpam") + labelled := f.snapshot(t, postURI) + require.Equal(t, version, labelled.version) + require.Equal(t, []string{labelID}, labelled.adminLogIDs) + for _, reason := range removalOnlyReasons { + response := f.post(t, contentLabelRouteRetractPath, f.adminToken, map[string]any{ + "actionId": labelID, "expectedVersion": version, + "idempotencyKey": testkit.UniqueIDWithPrefix(t, "retract"), + "reviewedSubject": map[string]string{"uri": postURI, "cid": f.postCID}, "reason": reason, + }) + requireXRPCError(t, response, http.StatusBadRequest, "UnsupportedReason") + require.Equal(t, labelled, f.snapshot(t, postURI), "a rejected retraction must not write an action or change state") + } +} + +func TestModerationContentLabelRequiresAdmin(t *testing.T) { + f := newContentLabelRouteFixture(t) + postURI := f.insertPost("post a non-admin tries to label") + + unlabelled := f.snapshot(t, postURI) + response := f.post(t, contentLabelRouteLabelPath, f.nonAdminToken, map[string]any{ + "subject": map[string]string{"uri": postURI, "cid": f.postCID}, "labelValue": "nsfw", + "expectedVersion": "v0", "idempotencyKey": testkit.UniqueIDWithPrefix(t, "label"), + }) + requireXRPCError(t, response, http.StatusForbidden, "Forbidden") + require.Equal(t, unlabelled, f.snapshot(t, postURI)) + + labelID, version := f.label(t, postURI, "v0", "") + labelled := f.snapshot(t, postURI) + response = f.post(t, contentLabelRouteRetractPath, f.nonAdminToken, map[string]any{ + "actionId": labelID, "expectedVersion": version, + "idempotencyKey": testkit.UniqueIDWithPrefix(t, "retract"), + "reviewedSubject": map[string]string{"uri": postURI, "cid": f.postCID}, + }) + requireXRPCError(t, response, http.StatusForbidden, "Forbidden") + require.Equal(t, labelled, f.snapshot(t, postURI)) +} + +func TestModerationContentLabelOnRemovedPostAppearsAfterRestore(t *testing.T) { + f := newContentLabelRouteFixture(t) + postURI := f.insertPost("post removed and then labelled") + postURL := f.server.URL + "/xrpc/social.coves.community.post.get?" + url.Values{"uris": {postURI}}.Encode() + getPost := func() map[string]any { + t.Helper() + items := moderationAcceptanceArray(t, moderationAcceptanceBody(t, + moderationAcceptanceRequest(t, f.client, http.MethodGet, postURL, "", nil))["posts"]) + require.Len(t, items, 1) + item := moderationAcceptanceObject(t, items[0]) + require.Equal(t, postURI, item["uri"]) + return item + } + + removed := moderationAcceptanceBody(t, f.post(t, moderationRouteRemovePath, f.adminToken, map[string]any{ + "subject": map[string]string{"uri": postURI, "cid": f.postCID}, "expectedVersion": "v0", + "idempotencyKey": testkit.UniqueIDWithPrefix(t, "remove"), "reason": "social.coves.moderation.defs#reasonSpam", + })) + require.Equal(t, "applied", removed["outcome"]) + removalID := modlogID(t, modlogAction(t, removed["action"], true)) + removedVersion, ok := moderationAcceptanceObject(t, removed["state"])["version"].(string) + require.True(t, ok) + + labelID, labelledVersion := f.label(t, postURI, removedVersion, "") + labelledRemoved := getPost() + require.Equal(t, "social.coves.community.post.defs#moderatedPost", labelledRemoved["$type"]) + removedView := moderationAcceptanceObject(t, labelledRemoved["moderation"]) + require.Equal(t, "removed", removedView["state"]) + require.NotContains(t, removedView, "contentLabels", "a removed post must not reveal its labels") + + restored := moderationAcceptanceBody(t, f.post(t, modlogRestorePath, f.adminToken, map[string]any{ + "actionId": removalID, "reviewedSubject": map[string]string{"uri": postURI, "cid": f.postCID}, + "expectedVersion": labelledVersion, "idempotencyKey": testkit.UniqueIDWithPrefix(t, "restore"), + "reason": "social.coves.moderation.defs#reasonModeratorDiscretion", + })) + require.Equal(t, "applied", restored["outcome"]) + restoredPost := getPost() + require.NotEqual(t, "social.coves.community.post.defs#moderatedPost", restoredPost["$type"]) + require.Equal(t, map[string]any{ + "state": "clear", + "contentLabels": []any{map[string]any{ + "value": "nsfw", "sources": []any{map[string]any{ + "authorityDid": f.instanceDID, "scope": map[string]any{"kind": "instance"}, + }}, + }}, + }, restoredPost["moderation"]) + state := moderationAcceptanceObject(t, moderationAcceptanceBody(t, + requestSubjectState(t, f.client, f.server.URL, postURI, f.adminToken))["state"]) + require.Equal(t, []any{map[string]any{ + "value": "nsfw", "action": map[string]any{"serviceDid": f.instanceDID, "actionId": labelID}, + }}, state["localLabels"]) +} diff --git a/internal/api/routes/moderation_label_modlog_integration_test.go b/internal/api/routes/moderation_label_modlog_integration_test.go new file mode 100644 index 0000000..ead1a0a --- /dev/null +++ b/internal/api/routes/moderation_label_modlog_integration_test.go @@ -0,0 +1,167 @@ +//go:build integration + +package routes_test + +import ( + "net/http" + "net/url" + "testing" + "time" + + "Coves/internal/core/moderation" + "Coves/tests/fixtures" + "Coves/tests/testkit" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +const ( + modlogLabelPath = "/xrpc/social.coves.moderation.labelContent" + modlogRetractPath = "/xrpc/social.coves.moderation.retractContentLabel" + // modlogRuleViolation is a public reason that labelContent accepts. + modlogRuleViolation = "social.coves.moderation.defs#reasonRuleViolation" +) + +func (f *modlogFixture) labelPost(t *testing.T) moderation.StrongRef { + t.Helper() + rkey := testkit.TID() + subject := moderation.StrongRef{ + URI: "at://" + f.authorDID + "/" + moderation.PostV2Collection + "/" + rkey, + CID: f.postCID, + } + _, err := f.db.ExecContext(t.Context(), ` + INSERT INTO posts (uri, cid, rkey, author_did, community_did, title, content, created_at) + VALUES ($1, $2, $3, $4, $5, 'modlog labelled post', 'indexed body', $6) + `, subject.URI, subject.CID, rkey, f.authorDID, f.communityDID, time.Now()) + require.NoError(t, err) + return subject +} + +func (f *modlogFixture) label(t *testing.T, subject moderation.StrongRef, reason, note string) (string, string) { + t.Helper() + response := moderationAcceptanceRequest(t, f.client, http.MethodPost, f.server.URL+modlogLabelPath, f.tokenA, map[string]any{ + "subject": map[string]string{"uri": subject.URI, "cid": subject.CID}, "labelValue": "nsfw", + "expectedVersion": "v0", "idempotencyKey": testkit.UniqueIDWithPrefix(t, "label"), + "reason": reason, "privateNote": note, + }) + body := moderationAcceptanceBody(t, response) + require.Equal(t, "applied", body["outcome"]) + action := modlogAction(t, body["action"], true) + require.Equal(t, "label", action["action"]) + version, ok := moderationAcceptanceObject(t, body["state"])["version"].(string) + require.True(t, ok) + return modlogID(t, action), version +} + +func (f *modlogFixture) retractLabel(t *testing.T, subject moderation.StrongRef, labelID, version, reason, note string) string { + t.Helper() + response := moderationAcceptanceRequest(t, f.client, http.MethodPost, f.server.URL+modlogRetractPath, f.tokenB, map[string]any{ + "actionId": labelID, "reviewedSubject": map[string]string{"uri": subject.URI, "cid": subject.CID}, + "expectedVersion": version, "idempotencyKey": testkit.UniqueIDWithPrefix(t, "retract"), + "reason": reason, "privateNote": note, + }) + body := moderationAcceptanceBody(t, response) + require.Equal(t, "applied", body["outcome"]) + action := modlogAction(t, body["action"], true) + require.Equal(t, "retract-label", action["action"]) + return modlogID(t, action) +} + +// TestModerationLabelModlogPublicLogOmitsLabelActions pins the 2026-09-30 +// decision: NSFW label applies and retractions are left out of the public +// modlog under every filter and cursor page; the admin log keeps them. +func TestModerationLabelModlogPublicLogOmitsLabelActions(t *testing.T) { + f := newModlogFixture(t) + firstURI, firstCID := f.comment(t) + secondURI, secondCID := f.comment(t) + firstRemove, _ := f.remove(t, firstURI, firstCID, f.tokenA, modlogSpam, "first removal note") + subject := f.labelPost(t) + labelID, version := f.label(t, subject, modlogRuleViolation, "private label review") + retractID := f.retractLabel(t, subject, labelID, version, "", "private retraction review") + secondRemove, _ := f.remove(t, secondURI, secondCID, f.tokenB, modlogSpam, "second removal note") + removals := []string{secondRemove, firstRemove} + + publicIDs := func(query url.Values) []string { + t.Helper() + response := f.list(t, false, "", query) + require.Equal(t, http.StatusOK, response.status, "%s", query.Encode()) + for _, leaked := range []string{subject.URI, labelID, retractID, "private label review", "private retraction review"} { + assert.NotContains(t, string(response.body), leaked, "%s", query.Encode()) + } + _, items := modlogPage(t, response, false) + ids := make([]string, 0, len(items)) + for _, item := range items { + action := modlogAction(t, item, false) + assert.NotContains(t, []any{"label", "retract-label"}, action["action"], "%s", query.Encode()) + ids = append(ids, modlogID(t, action)) + } + return ids + } + window := url.Values{ + "since": {time.Now().Add(-time.Hour).UTC().Format(time.RFC3339Nano)}, + "until": {time.Now().Add(time.Hour).UTC().Format(time.RFC3339Nano)}, + } + for _, test := range []struct { + query url.Values + want []string + }{ + {nil, removals}, + {url.Values{"subject": {subject.URI}}, []string{}}, + {url.Values{"collection": {moderation.PostV2Collection}}, []string{}}, + {url.Values{"community": {f.communityDID}}, removals}, + {url.Values{"actor": {f.adminA}}, []string{firstRemove}}, + {url.Values{"actor": {f.adminB}}, []string{secondRemove}}, + {url.Values{"authority": {fixtures.InstanceDID()}}, removals}, + {url.Values{"origin": {"local"}}, removals}, + {window, removals}, + } { + assert.Equal(t, test.want, publicIDs(test.query), "%s", test.query.Encode()) + } + for _, kind := range []string{"label", "retract-label"} { + requireXRPCError(t, f.list(t, false, "", url.Values{"action": {kind}}), http.StatusBadRequest, "InvalidRequest") + } + + var walked []string + pages := modlogWalk(t, f, url.Values{"limit": {"1"}}) + for _, page := range pages { + for _, leaked := range []string{subject.URI, labelID, retractID} { + assert.NotContains(t, string(page), leaked) + } + _, items := modlogPage(t, subjectStateHTTPResponse{status: http.StatusOK, body: page}, false) + for _, item := range items { + walked = append(walked, modlogID(t, modlogAction(t, item, false))) + } + } + assert.Equal(t, removals, walked) + + want := map[string]struct{ kind, note, actor string }{ + labelID: {"label", "private label review", f.adminA}, + retractID: {"retract-label", "private retraction review", f.adminB}, + } + _, admin := modlogPage(t, f.list(t, true, f.tokenA, url.Values{"subject": {subject.URI}}), true) + require.Len(t, admin, 2) + for _, item := range admin { + entry := moderationAcceptanceObject(t, item) + action := modlogAction(t, item, true) + expected, ok := want[modlogID(t, action)] + require.True(t, ok) + assert.Equal(t, expected.kind, action["action"]) + assert.Equal(t, "nsfw", action["labelValue"]) + assert.Equal(t, subject.URI, moderationAcceptanceObject(t, action["subject"])["uri"]) + assert.Equal(t, expected.actor, entry["actorDid"]) + assert.Equal(t, expected.note, entry["privateNote"]) + if expected.kind == "label" { + assert.Equal(t, modlogRuleViolation, action["reason"]) + } else { + assert.Equal(t, labelID, moderationAcceptanceObject(t, action["reverses"])["actionId"]) + } + } + for kind, id := range map[string]string{"label": labelID, "retract-label": retractID} { + _, filtered := modlogPage(t, f.list(t, true, f.tokenA, url.Values{"action": {kind}}), true) + require.Len(t, filtered, 1, kind) + assert.Equal(t, id, modlogID(t, modlogAction(t, filtered[0], true))) + } + _, everything := modlogPage(t, f.list(t, true, f.tokenA, nil), true) + assert.Len(t, everything, 4) +} diff --git a/internal/api/routes/moderation_modlog_integration_test.go b/internal/api/routes/moderation_modlog_integration_test.go index 70be9bc..7147a96 100644 --- a/internal/api/routes/moderation_modlog_integration_test.go +++ b/internal/api/routes/moderation_modlog_integration_test.go @@ -520,7 +520,7 @@ func TestModerationModlogFilters(t *testing.T) { } for _, query := range []url.Values{ {"actor": {"missing.test"}}, {"authority": {"missing.test"}}, - {"community": {"nonexistent.coves.social"}}, {"action": {"label"}}, + {"community": {"nonexistent.coves.social"}}, {"origin": {"inherited"}}, {"collection": {moderation.PostV2Collection}}, } { response := f.list(t, false, "", query) diff --git a/internal/api/routes/registration_test.go b/internal/api/routes/registration_test.go index 78fae74..ec7d8dd 100644 --- a/internal/api/routes/registration_test.go +++ b/internal/api/routes/registration_test.go @@ -253,6 +253,8 @@ var declaredRoutes = []declaredRoute{ {http.MethodGet, "/xrpc/social.coves.moderation.getSubjectState", authRequired, 0, false}, {http.MethodPost, "/xrpc/social.coves.moderation.removeContent", authRequired, 0, false}, {http.MethodPost, "/xrpc/social.coves.moderation.restoreContent", authRequired, 0, false}, + {http.MethodPost, "/xrpc/social.coves.moderation.labelContent", authRequired, 0, false}, + {http.MethodPost, "/xrpc/social.coves.moderation.retractContentLabel", authRequired, 0, false}, // RegisterCommunitySuggestionRoutes — social.coves.community.suggestion.* {http.MethodGet, "/xrpc/social.coves.community.suggestion.list", authOptional, 0, false}, diff --git a/internal/atproto/jetstream/post_label_consumer_test.go b/internal/atproto/jetstream/post_label_consumer_test.go new file mode 100644 index 0000000..a1b2947 --- /dev/null +++ b/internal/atproto/jetstream/post_label_consumer_test.go @@ -0,0 +1,165 @@ +//go:build integration + +package jetstream + +import ( + "encoding/json" + "testing" + + "Coves/internal/core/moderation" + "Coves/internal/core/posts" + "Coves/tests/fixtures" + "Coves/tests/testkit" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func postLabelSelfLabel() map[string]interface{} { + return map[string]interface{}{ + "$type": "com.atproto.label.defs#selfLabels", + "values": []interface{}{map[string]interface{}{"val": "nsfw"}}, + } +} + +func postLabelAcceptRevision(t *testing.T, f postModerationConsumerFixture, cid string) { + t.Helper() + rkey := testkit.TID() + accepted, err := f.admissions.ApplyAcceptance(t.Context(), posts.ApplyAcceptanceCommand{ + CommunityDID: pv2Community, PostURI: f.uri, + AcceptanceURI: "at://" + pv2Community + "/" + posts.AcceptanceCollection + "/" + rkey, + AcceptanceRkey: rkey, PinnedCID: cid, + Watermark: posts.CommunityWatermark{Rev: testkit.TID()}, + }) + require.NoError(t, err) + require.Equal(t, posts.AdmissionApplied, accepted.Outcome) +} + +func postLabelView(t *testing.T, f postModerationConsumerFixture, viewerDID string) *posts.PostView { + t.Helper() + results, err := f.postService.GetPosts(t.Context(), posts.GetPostsRequest{URIs: []string{f.uri}, ViewerDID: viewerDID}) + require.NoError(t, err) + require.Len(t, results, 1) + require.NotNil(t, results[0].Post, "post must be visible to %q after the expected acceptance", viewerDID) + return results[0].Post +} + +func postLabelRecordJSON(t *testing.T, record interface{}) string { + t.Helper() + encoded, err := json.Marshal(record) + require.NoError(t, err) + return string(encoded) +} + +func requirePostLabelEditedRecord(t *testing.T, view *posts.PostView, edited map[string]interface{}) { + t.Helper() + require.Equal(t, postModerationCIDTwo, view.CID) + var record map[string]interface{} + require.NoError(t, json.Unmarshal([]byte(postLabelRecordJSON(t, view.Record)), &record)) + assert.Equal(t, edited["title"], record["title"]) + assert.Equal(t, edited["content"], record["content"]) + labels, ok := record["labels"].(map[string]interface{}) + require.True(t, ok, "the served record must retain author self-labels") + values, ok := labels["values"].([]interface{}) + require.True(t, ok) + require.Equal(t, []interface{}{map[string]interface{}{"val": "nsfw"}}, values) +} + +func TestModerationLabelPostConsumerEditReplayAndSelfLabels(t *testing.T) { + f := newPostModerationConsumerFixture(t) + // The fixture initially indexes an unlabelled post. An author update adds + // real self-labels, then the community accepts that revision before the + // moderator acts; the second update is the edit under the active decision. + initial := postModerationRecord(postModerationCIDOne) + initial["labels"] = postLabelSelfLabel() + initialUpdate := pv2Event(pv2Author, "update", f.rkey, f.revs[1], postModerationCIDOne, f.createdAt+1_000_000, initial) + require.NoError(t, f.consumer.HandleEvent(t.Context(), initialUpdate)) + postLabelAcceptRevision(t, f, postModerationCIDOne) + initialView := postLabelView(t, f, "") + require.Nil(t, initialView.Moderation) + require.Contains(t, postLabelRecordJSON(t, initialView.Record), `"labels"`, "the fixture must index an author self-label before admin labelling") + beforeApply := postLabelRecordJSON(t, initialView.Record) + actor := fixtures.DID("labelconsumeradmin") + initialSubject := moderation.StrongRef{URI: f.uri, CID: postModerationCIDOne} + label, err := f.moderator.LabelContent(t.Context(), actor, moderation.LabelContentRequest{ + Subject: initialSubject, LabelValue: moderation.LabelNSFW, ExpectedVersion: "v0", IdempotencyKey: "consumer-label", + }) + require.NoError(t, err) + require.NotNil(t, label.Action) + require.Equal(t, moderation.OutcomeApplied, label.Outcome) + labelID := label.Action.ID + assert.Equal(t, beforeApply, postLabelRecordJSON(t, postLabelView(t, f, "").Record), + "applying an admin label must not change the served author record") + + edited := postModerationRecord(postModerationCIDOne) + edited["title"] = "edited author title" + edited["content"] = "new author text with unchanged self-label" + edited["labels"] = postLabelSelfLabel() + update := pv2Event(pv2Author, "update", f.rkey, f.revs[2], postModerationCIDTwo, f.createdAt+2_000_000, edited) + require.NoError(t, f.consumer.HandleEvent(t.Context(), update)) + indexed, err := f.postRepository.GetRawIndexedRow(t.Context(), f.uri) + require.NoError(t, err) + require.Equal(t, postModerationCIDTwo, indexed.CID) + state, err := f.moderator.GetSubjectState(t.Context(), f.uri) + require.NoError(t, err) + require.Len(t, state.LocalLabels, 1) + assert.Equal(t, labelID, state.LocalLabels[0].Action.ActionID, "the decision is URI-scoped across the edit") + assert.Equal(t, "v1", state.Version) + var admission string + require.NoError(t, f.db.QueryRowContext(t.Context(), `SELECT status FROM community_post_admissions WHERE post_uri = $1`, f.uri).Scan(&admission)) + require.Equal(t, "pending_reacceptance", admission, "the author sees the edit before public reacceptance") + authorView := postLabelView(t, f, pv2Author) + assert.Equal(t, &posts.ModerationView{State: "clear", ContentLabels: []posts.ContentLabelView{{ + Value: "nsfw", Sources: []posts.ModerationSourceView{{AuthorityDID: fixtures.InstanceDID(), Scope: posts.ModerationScopeView{Kind: "instance"}}}, + }}}, authorView.Moderation) + requirePostLabelEditedRecord(t, authorView, edited) + postLabelAcceptRevision(t, f, postModerationCIDTwo) + publicView := postLabelView(t, f, "") + assert.Equal(t, &posts.ModerationView{State: "clear", ContentLabels: []posts.ContentLabelView{{ + Value: "nsfw", Sources: []posts.ModerationSourceView{{AuthorityDID: fixtures.InstanceDID(), Scope: posts.ModerationScopeView{Kind: "instance"}}}, + }}}, publicView.Moderation) + requirePostLabelEditedRecord(t, publicView, edited) + + // A duplicate delivery of the exact update must not replace the active + // action, rewrite the indexed record, or manufacture a second label action. + var rowBefore, rowAfter string + require.NoError(t, f.db.QueryRowContext(t.Context(), `SELECT row_to_json(p)::text FROM posts p WHERE uri = $1`, f.uri).Scan(&rowBefore)) + require.NoError(t, f.consumer.HandleEvent(t.Context(), update)) + require.NoError(t, f.db.QueryRowContext(t.Context(), `SELECT row_to_json(p)::text FROM posts p WHERE uri = $1`, f.uri).Scan(&rowAfter)) + assert.Equal(t, rowBefore, rowAfter, "a duplicate update must not rewrite any indexed post fields") + state, err = f.moderator.GetSubjectState(t.Context(), f.uri) + require.NoError(t, err) + require.Equal(t, "v1", state.Version) + require.Len(t, state.LocalLabels, 1) + assert.Equal(t, labelID, state.LocalLabels[0].Action.ActionID) + assert.Equal(t, 1, countRows(t, f.db, `SELECT count(*) FROM moderation_actions WHERE subject_uri = $1 AND action = 'label'`, f.uri)) + assert.Equal(t, postModerationCIDTwo, postLabelView(t, f, "").CID) + beforeRetract := postLabelRecordJSON(t, postLabelView(t, f, "").Record) + + withoutReview, err := f.moderator.RetractContentLabel(t.Context(), actor, moderation.RetractContentLabelRequest{ + ActionID: labelID, ExpectedVersion: "v1", IdempotencyKey: "consumer-no-review", + }) + assert.ErrorIs(t, err, moderation.ErrInvalidRequest) + assert.Nil(t, withoutReview) + withStaleCID, err := f.moderator.RetractContentLabel(t.Context(), actor, moderation.RetractContentLabelRequest{ + ActionID: labelID, ReviewedSubject: &initialSubject, ExpectedVersion: "v1", IdempotencyKey: "consumer-stale-review", + }) + assert.ErrorIs(t, err, moderation.ErrContentChanged) + assert.Nil(t, withStaleCID) + currentSubject := moderation.StrongRef{URI: f.uri, CID: postModerationCIDTwo} + retracted, err := f.moderator.RetractContentLabel(t.Context(), actor, moderation.RetractContentLabelRequest{ + ActionID: labelID, ReviewedSubject: ¤tSubject, ExpectedVersion: "v1", IdempotencyKey: "consumer-retract", + }) + require.NoError(t, err) + require.Equal(t, moderation.OutcomeApplied, retracted.Outcome) + assert.Empty(t, retracted.State.LocalLabels) + view := postLabelView(t, f, "") + assert.Nil(t, view.Moderation, "retraction removes only the admin label") + assert.Equal(t, beforeRetract, postLabelRecordJSON(t, view.Record), + "retracting an admin label must not change the served author record") + requirePostLabelEditedRecord(t, view, edited) + assert.Equal(t, 1, countRows(t, f.db, `SELECT count(*) FROM moderation_actions WHERE subject_uri = $1 AND action = 'label'`, f.uri)) + // newPostModerationConsumerFixture has no PDS URL parameter: its consumer + // and moderation service use only Postgres, so there is no PDS seam to + // redirect to a recording httptest server in this fixture. +} diff --git a/internal/atproto/lexicon/social/coves/moderation/listActions.json b/internal/atproto/lexicon/social/coves/moderation/listActions.json index bb0c1e8..8fcb827 100644 --- a/internal/atproto/lexicon/social/coves/moderation/listActions.json +++ b/internal/atproto/lexicon/social/coves/moderation/listActions.json @@ -38,9 +38,7 @@ "remove", "restore", "apply-removal", - "retract-removal", - "label", - "retract-label" + "retract-removal" ], "description": "Return actions with this moderation transition" }, diff --git a/internal/core/moderation/fake_store_test.go b/internal/core/moderation/fake_store_test.go index 60f65df..68efa3d 100644 --- a/internal/core/moderation/fake_store_test.go +++ b/internal/core/moderation/fake_store_test.go @@ -3,6 +3,7 @@ package moderation_test import ( "context" "fmt" + "sort" "sync" "time" @@ -14,6 +15,17 @@ type inMemoryModerationDecisionKey struct { subjectURI string } +type inMemoryModerationLabelKey struct { + authorityDID string + subjectURI string + value string +} + +type inMemoryModerationLabelDecision struct { + actionID string + active bool +} + type inMemoryModerationIdempotencyKey struct { actorDID string authorityDID string @@ -26,6 +38,7 @@ type inMemoryModerationState struct { versions map[string]int64 actions map[string]moderation.Action activeRemovals map[inMemoryModerationDecisionKey]string + labelDecisions map[inMemoryModerationLabelKey]inMemoryModerationLabelDecision idempotency map[inMemoryModerationIdempotencyKey]moderation.IdempotencyRecord mediaBlocks map[moderation.MediaBlock]bool nextActionID int @@ -38,6 +51,7 @@ func (state inMemoryModerationState) copy() inMemoryModerationState { versions: make(map[string]int64, len(state.versions)), actions: make(map[string]moderation.Action, len(state.actions)), activeRemovals: make(map[inMemoryModerationDecisionKey]string, len(state.activeRemovals)), + labelDecisions: make(map[inMemoryModerationLabelKey]inMemoryModerationLabelDecision, len(state.labelDecisions)), idempotency: make(map[inMemoryModerationIdempotencyKey]moderation.IdempotencyRecord, len(state.idempotency)), mediaBlocks: make(map[moderation.MediaBlock]bool, len(state.mediaBlocks)), nextActionID: state.nextActionID, @@ -59,6 +73,9 @@ func (state inMemoryModerationState) copy() inMemoryModerationState { for key, actionID := range state.activeRemovals { working.activeRemovals[key] = actionID } + for key, decision := range state.labelDecisions { + working.labelDecisions[key] = decision + } for key, record := range state.idempotency { working.idempotency[key] = record } @@ -80,6 +97,10 @@ type inMemoryModerationStore struct { failInsertAction error failSetRemovalDecision error failListActions error + failSetLabelDecision error + // failActiveLabelsAfterLabelDecision fails ActiveLabels reads made after + // the transaction has changed a label decision. + failActiveLabelsAfterLabelDecision error } func newInMemoryModerationStore(now time.Time) *inMemoryModerationStore { @@ -91,12 +112,19 @@ func newInMemoryModerationStore(now time.Time) *inMemoryModerationStore { versions: make(map[string]int64), actions: make(map[string]moderation.Action), activeRemovals: make(map[inMemoryModerationDecisionKey]string), + labelDecisions: make(map[inMemoryModerationLabelKey]inMemoryModerationLabelDecision), idempotency: make(map[inMemoryModerationIdempotencyKey]moderation.IdempotencyRecord), mediaBlocks: make(map[moderation.MediaBlock]bool), }, } } +func (store *inMemoryModerationStore) seedActiveLabel(action moderation.Action) { + store.state.actions[action.ID] = action + store.state.labelDecisions[inMemoryModerationLabelKey{action.AuthorityDID, action.SubjectURI, action.LabelValue}] = + inMemoryModerationLabelDecision{actionID: action.ID, active: true} +} + func (store *inMemoryModerationStore) InTransaction(ctx context.Context, fn func(context.Context, moderation.Transaction) error) error { store.mu.Lock() defer store.mu.Unlock() @@ -118,6 +146,17 @@ func (store *inMemoryModerationStore) SubjectModeration(_ context.Context, autho action := store.state.actions[actionID] state.ActiveRemoval = &action } + var values []string + for key, decision := range store.state.labelDecisions { + if key.authorityDID == authorityDID && key.subjectURI == subjectURI && decision.active { + values = append(values, key.value) + } + } + sort.Strings(values) + for _, value := range values { + key := inMemoryModerationLabelKey{authorityDID, subjectURI, value} + state.ActiveLabels = append(state.ActiveLabels, store.state.actions[store.state.labelDecisions[key].actionID]) + } return state, nil } @@ -128,12 +167,23 @@ func (store *inMemoryModerationStore) ListActions(_ context.Context, query moder if store.failListActions != nil { return nil, store.failListActions } - return append([]moderation.Action(nil), store.listRows...), nil + rows := make([]moderation.Action, 0, len(store.listRows)) + for _, row := range store.listRows { + excluded := false + for _, kind := range moderation.PublicExcludedActions() { + excluded = excluded || (query.ExcludeLabelActions && row.Action == kind) + } + if !excluded { + rows = append(rows, row) + } + } + return rows, nil } type inMemoryModerationTransaction struct { - store *inMemoryModerationStore - state *inMemoryModerationState + store *inMemoryModerationStore + state *inMemoryModerationState + labelDecisionWritten bool } func (*inMemoryModerationTransaction) LockActor(context.Context, string) error { @@ -208,6 +258,51 @@ func (transaction *inMemoryModerationTransaction) ActiveRemoval(_ context.Contex return &action, nil } +func (transaction *inMemoryModerationTransaction) ActiveLabels(_ context.Context, authorityDID, subjectURI string) ([]moderation.Action, error) { + if transaction.labelDecisionWritten && transaction.store.failActiveLabelsAfterLabelDecision != nil { + return nil, transaction.store.failActiveLabelsAfterLabelDecision + } + var values []string + for key, decision := range transaction.state.labelDecisions { + if key.authorityDID == authorityDID && key.subjectURI == subjectURI && decision.active { + values = append(values, key.value) + } + } + sort.Strings(values) + actions := make([]moderation.Action, 0, len(values)) + for _, value := range values { + key := inMemoryModerationLabelKey{authorityDID, subjectURI, value} + actions = append(actions, transaction.state.actions[transaction.state.labelDecisions[key].actionID]) + } + return actions, nil +} + +func (transaction *inMemoryModerationTransaction) SetLabelDecision(_ context.Context, authorityDID, subjectURI, value, actionID string, active bool) error { + transaction.store.writeCalls = append(transaction.store.writeCalls, "SetLabelDecision") + if transaction.store.failSetLabelDecision != nil { + return transaction.store.failSetLabelDecision + } + key := inMemoryModerationLabelKey{authorityDID, subjectURI, value} + decision, exists := transaction.state.labelDecisions[key] + if active { + if _, actionExists := transaction.state.actions[actionID]; !actionExists { + return moderation.ErrDecisionNotFound + } + if exists && decision.active { + return fmt.Errorf("label decision %q on %s is already active", value, subjectURI) + } + transaction.state.labelDecisions[key] = inMemoryModerationLabelDecision{actionID: actionID, active: true} + } else { + if !exists || !decision.active || decision.actionID != actionID { + return fmt.Errorf("no active label decision %q on %s for action %s", value, subjectURI, actionID) + } + decision.active = false + transaction.state.labelDecisions[key] = decision + } + transaction.labelDecisionWritten = true + return nil +} + func (transaction *inMemoryModerationTransaction) InsertAction(_ context.Context, action moderation.Action) (*moderation.Action, error) { transaction.store.writeCalls = append(transaction.store.writeCalls, "InsertAction") if transaction.store.failInsertAction != nil { diff --git a/internal/core/moderation/idempotency_golden_test.go b/internal/core/moderation/idempotency_golden_test.go new file mode 100644 index 0000000..7750c55 --- /dev/null +++ b/internal/core/moderation/idempotency_golden_test.go @@ -0,0 +1,165 @@ +package moderation_test + +import ( + "encoding/json" + "testing" + "time" + + "Coves/internal/core/moderation" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// Idempotency records outlive deploys: a fingerprint or stored result written +// by one build is read back by the next within the retention window. These +// pins fail when the fingerprint tuple or the stored result's JSON field names +// change. Remove and restore fingerprints equal what the code before label +// support produced for the same requests. + +const ( + goldenRemoveFingerprint = "6eed62114f94a7e5f8be6fdac815f87172d5bfe0d6e63bb86167692fdac715a0" + goldenRestoreFingerprint = "e349f3bd469aa767505477b493fa9cb01406cac8e57b21fdd82d9d95829856ef" + goldenLabelFingerprint = "e6482a4e5c96496e95308e900a948e0f7b92b5397fb47a39d5bbab82a510da4d" + goldenRetractLabelFingerprint = "8eb887b44b9c3b17b0760156a007127a88eb8b88c8381ebe89c27c487330a378" +) + +func goldenLabelledMutationResult() moderation.MutationResult { + return moderation.MutationResult{ + Outcome: moderation.OutcomeApplied, + State: moderation.SubjectState{ + Subject: postRulesURI, Version: "v7", + Moderation: moderation.ModerationView{ + State: moderation.ModerationStateRemoved, + ContentLabels: []moderation.ContentLabel{{ + Value: moderation.LabelNSFW, + Sources: []moderation.DecisionSource{{AuthorityDID: removeRulesInstanceDID, ScopeKind: moderation.ScopeInstance}}, + }}, + }, + RecordState: moderation.RecordStatePresent, + CurrentSubject: &moderation.StrongRef{URI: postRulesURI, CID: postRulesCID}, + LocalRemoval: &moderation.ActionRef{ServiceDID: removeRulesInstanceDID, ActionID: "golden-removal"}, + LocalLabels: []moderation.LocalLabel{{ + Value: moderation.LabelNSFW, Action: moderation.ActionRef{ServiceDID: removeRulesInstanceDID, ActionID: "golden-label"}, + }}, + }, + Action: &moderation.Action{ + ID: "golden-label", ActorDID: removeRulesAdminDID, AuthorityDID: removeRulesInstanceDID, + ScopeKind: moderation.ScopeInstance, ScopeCommunityDID: "did:plc:scopecommunity", + SubjectURI: postRulesURI, SubjectCollection: moderation.PostV2Collection, + SubjectCommunityDID: postRulesCommunityDID, ObservedCID: postRulesCID, + Action: moderation.ActionLabel, LabelValue: moderation.LabelNSFW, Reason: removeRulesSpam, + PrivateNote: "golden note", ReversesActionID: "golden-reversed", + ReversedActionReason: "social.coves.moderation.defs#reasonHarassment", + Origin: moderation.OriginLocal, CreatedAt: time.Date(2026, time.September, 24, 12, 0, 0, 0, time.UTC), + }, + } +} + +const goldenLabelledMutationResultJSON = `{ + "Outcome": "applied", + "State": { + "Subject": "at://did:plc:postauthor/social.coves.community.postv2/3kabc", + "Version": "v7", + "Moderation": { + "State": "removed", + "ContentLabels": [{"Value": "nsfw", "Sources": [{"AuthorityDID": "did:web:moderation.test", "ScopeKind": "instance"}]}] + }, + "RecordState": "present", + "CurrentSubject": {"URI": "at://did:plc:postauthor/social.coves.community.postv2/3kabc", "CID": "bafyreipostversion"}, + "LocalRemoval": {"ServiceDID": "did:web:moderation.test", "ActionID": "golden-removal"}, + "LocalLabels": [{"Value": "nsfw", "Action": {"ServiceDID": "did:web:moderation.test", "ActionID": "golden-label"}}] + }, + "Action": { + "ID": "golden-label", + "ActorDID": "did:plc:firstadmin", + "AuthorityDID": "did:web:moderation.test", + "ScopeKind": "instance", + "ScopeCommunityDID": "did:plc:scopecommunity", + "SubjectURI": "at://did:plc:postauthor/social.coves.community.postv2/3kabc", + "SubjectCollection": "social.coves.community.postv2", + "SubjectCommunityDID": "did:plc:postcommunity", + "ObservedCID": "bafyreipostversion", + "Action": "label", + "LabelValue": "nsfw", + "Reason": "social.coves.moderation.defs#reasonSpam", + "PrivateNote": "golden note", + "ReversesActionID": "golden-reversed", + "ReversedActionReason": "social.coves.moderation.defs#reasonHarassment", + "Origin": "local", + "CreatedAt": "2026-09-24T12:00:00Z" + } +}` + +func TestModerationIdempotencyStoredResultJSONIsStable(t *testing.T) { + encoded, err := json.Marshal(goldenLabelledMutationResult()) + require.NoError(t, err) + assert.JSONEq(t, goldenLabelledMutationResultJSON, string(encoded)) + var decoded moderation.MutationResult + require.NoError(t, json.Unmarshal([]byte(goldenLabelledMutationResultJSON), &decoded)) + assert.Equal(t, goldenLabelledMutationResult(), decoded, "a stored result written before a deploy must replay unchanged") +} + +// seedGoldenIdempotencyRecord stores a record under the golden fingerprint with +// a result no mutation could have produced, so a replay is unmistakable. +func seedGoldenIdempotencyRecord(scenario removeRulesScenario, key, fingerprint string) moderation.MutationResult { + stored := goldenLabelledMutationResult() + scenario.store.state.idempotency[inMemoryModerationIdempotencyKey{removeRulesAdminDID, removeRulesInstanceDID, key}] = moderation.IdempotencyRecord{ + ActorDID: removeRulesAdminDID, AuthorityDID: removeRulesInstanceDID, Key: key, + Fingerprint: fingerprint, Result: stored, + CreatedAt: scenario.store.now, ExpiresAt: scenario.store.now.Add(time.Hour), + } + return stored +} + +func TestModerationIdempotencyFingerprintsAreStable(t *testing.T) { + const goldenKey = "golden-key" + for _, test := range []struct { + name string + fingerprint string + scenario func() removeRulesScenario + call func(removeRulesScenario) (*moderation.MutationResult, error) + }{ + {"remove", goldenRemoveFingerprint, newRemoveRulesScenario, func(scenario removeRulesScenario) (*moderation.MutationResult, error) { + return scenario.service.RemoveContent(t.Context(), removeRulesAdminDID, moderation.RemoveContentRequest{ + Subject: moderation.StrongRef{URI: removeRulesURI, CID: removeRulesCID}, + ExpectedVersion: "v0", IdempotencyKey: goldenKey, Reason: removeRulesSpam, PrivateNote: "golden removal note", + }) + }}, + {"restore", goldenRestoreFingerprint, newRemoveRulesScenario, func(scenario removeRulesScenario) (*moderation.MutationResult, error) { + return scenario.service.RestoreContent(t.Context(), removeRulesAdminDID, moderation.RestoreContentRequest{ + ActionID: "golden-removal", ReviewedSubject: &moderation.StrongRef{URI: removeRulesURI, CID: removeRulesCID}, + ExpectedVersion: "v1", IdempotencyKey: goldenKey, Reason: removeRulesSpam, PrivateNote: "golden restore note", + }) + }}, + {"label", goldenLabelFingerprint, func() removeRulesScenario { + scenario, _ := newLabelRulesScenario() + return scenario + }, func(scenario removeRulesScenario) (*moderation.MutationResult, error) { + return scenario.service.LabelContent(t.Context(), removeRulesAdminDID, moderation.LabelContentRequest{ + Subject: moderation.StrongRef{URI: postRulesURI, CID: postRulesCID}, LabelValue: moderation.LabelNSFW, + ExpectedVersion: "v0", IdempotencyKey: goldenKey, Reason: removeRulesSpam, PrivateNote: "golden label note", + }) + }}, + {"retract-label", goldenRetractLabelFingerprint, func() removeRulesScenario { + scenario, _ := newLabelRulesScenario() + return scenario + }, func(scenario removeRulesScenario) (*moderation.MutationResult, error) { + return scenario.service.RetractContentLabel(t.Context(), removeRulesAdminDID, moderation.RetractContentLabelRequest{ + ActionID: "golden-label", ReviewedSubject: &moderation.StrongRef{URI: postRulesURI, CID: postRulesCID}, + ExpectedVersion: "v1", IdempotencyKey: goldenKey, + Reason: "social.coves.moderation.defs#reasonHarassment", PrivateNote: "golden retraction note", + }) + }}, + } { + t.Run(test.name, func(t *testing.T) { + scenario := test.scenario() + stored := seedGoldenIdempotencyRecord(scenario, goldenKey, test.fingerprint) + result, err := assertIdempotencyNoMutation(t, scenario, func() (*moderation.MutationResult, error) { + return test.call(scenario) + }) + require.NoError(t, err, "the request must still match the fingerprint stored by an earlier build") + assert.Equal(t, &stored, result) + }) + } +} diff --git a/internal/core/moderation/interfaces.go b/internal/core/moderation/interfaces.go index f75ded0..fa640ad 100644 --- a/internal/core/moderation/interfaces.go +++ b/internal/core/moderation/interfaces.go @@ -33,6 +33,8 @@ type Service interface { GetSubjectState(ctx context.Context, subject string) (*SubjectState, error) RemoveContent(ctx context.Context, actorDID string, request RemoveContentRequest) (*MutationResult, error) RestoreContent(ctx context.Context, actorDID string, request RestoreContentRequest) (*MutationResult, error) + LabelContent(ctx context.Context, actorDID string, request LabelContentRequest) (*MutationResult, error) + RetractContentLabel(ctx context.Context, actorDID string, request RetractContentLabelRequest) (*MutationResult, error) ListActions(ctx context.Context, params ListActionsParams) (*ActionPage, error) ListAdminActions(ctx context.Context, params ListAdminActionsParams) (*AdminActionPage, error) } diff --git a/internal/core/moderation/label.go b/internal/core/moderation/label.go new file mode 100644 index 0000000..23655b9 --- /dev/null +++ b/internal/core/moderation/label.go @@ -0,0 +1,162 @@ +package moderation + +import ( + "context" + "errors" + "fmt" + "time" + + "github.com/bluesky-social/indigo/atproto/syntax" +) + +func validateLabelRequest(request LabelContentRequest) error { + uri, err := syntax.ParseATURI(request.Subject.URI) + if err != nil || !uri.Authority().IsDID() || uri.RecordKey().String() == "" { + return fmt.Errorf("%w: expected a post record URI with a DID authority", ErrInvalidSubject) + } + if uri.Collection().String() != PostV2Collection && uri.Collection().String() != LegacyPostCollection { + return fmt.Errorf("%w: unsupported subject collection", ErrInvalidSubject) + } + if _, err := syntax.ParseCID(request.Subject.CID); err != nil { + return fmt.Errorf("%w: invalid subject CID", ErrInvalidRequest) + } + if err := validateMutationBaseFields(request.IdempotencyKey, request.ExpectedVersion, request.PrivateNote); err != nil { + return err + } + if !validOpaqueField(request.LabelValue) { + return fmt.Errorf("%w: labelValue must be 1-128 UTF-8 bytes", ErrInvalidRequest) + } + return validateOptionalReason(request.Reason) +} + +func (s *service) labelContent(ctx context.Context, actorDID string, request LabelContentRequest) (*MutationResult, error) { + if err := validateLabelRequest(request); err != nil { + return nil, err + } + now := time.Now() + if s.config.Now != nil { + now = s.config.Now() + } + fingerprint := mutationFingerprint(ActionLabel, "", request.Subject.URI, request.Subject.CID, request.ExpectedVersion, request.Reason, request.LabelValue, request.PrivateNote) + var result *MutationResult + var ruleError error + err := s.store.InTransaction(ctx, func(ctx context.Context, tx Transaction) error { + fail := func(err error) error { + ruleError = err + return err + } + unavailable := func(err error) error { return fmt.Errorf("%w: %w", ErrModerationUnavailable, err) } + if err := tx.LockActor(ctx, actorDID); err != nil { + return unavailable(err) + } + stored, err := tx.LiveIdempotencyRecord(ctx, actorDID, s.config.InstanceDID, request.IdempotencyKey, now) + if err != nil { + return unavailable(err) + } + if stored != nil { + if stored.Fingerprint != fingerprint { + return fail(ErrIdempotencyConflict) + } + copy := stored.Result + result = © + return nil + } + if request.LabelValue != LabelNSFW { + return fail(ErrUnsupportedLabel) + } + count, err := tx.CountLiveIdempotencyKeys(ctx, actorDID, now) + if err != nil { + return unavailable(err) + } + if count >= s.config.MaxLiveIdempotencyKeys { + return fail(fmt.Errorf("%w: live idempotency key limit %d reached", ErrInvalidRequest, s.config.MaxLiveIdempotencyKeys)) + } + version, err := tx.LockSubject(ctx, request.Subject.URI) + if err != nil { + return unavailable(err) + } + if request.ExpectedVersion != versionToken(version) { + return fail(ErrStateConflict) + } + subject, err := readIndexedSubject(ctx, tx, request.Subject.URI) + if errors.Is(err, ErrSubjectNotIndexed) { + return fail(ErrSubjectNotFound) + } + if err != nil { + return unavailable(err) + } + if subject == nil { + return unavailable(errors.New("indexed subject missing")) + } + if subject.AuthorDeleted { + return fail(ErrSubjectNotFound) + } + if subject.CID != request.Subject.CID { + return fail(ErrContentChanged) + } + labels, err := tx.ActiveLabels(ctx, s.config.InstanceDID, request.Subject.URI) + if err != nil { + return unavailable(err) + } + active, err := tx.ActiveRemoval(ctx, s.config.InstanceDID, request.Subject.URI) + if err != nil { + return unavailable(err) + } + current := &StrongRef{URI: subject.URI, CID: subject.CID} + alreadyActive := false + for _, label := range labels { + if label.LabelValue == LabelNSFW { + alreadyActive = true + break + } + } + if alreadyActive { + state, err := newSubjectState(request.Subject.URI, version, RecordStatePresent, current, active, labels, s.config.InstanceDID) + if err != nil { + return err + } + result = &MutationResult{Outcome: OutcomeUnchanged, State: state} + } else { + action, err := tx.InsertAction(ctx, Action{ + ActorDID: actorDID, AuthorityDID: s.config.InstanceDID, + ScopeKind: ScopeInstance, SubjectURI: request.Subject.URI, + SubjectCollection: subject.Collection, SubjectCommunityDID: subject.CommunityDID, + ObservedCID: subject.CID, Action: ActionLabel, LabelValue: LabelNSFW, + Reason: request.Reason, PrivateNote: request.PrivateNote, + Origin: OriginLocal, CreatedAt: now, + }) + if err != nil { + return unavailable(err) + } + if action == nil || action.ID == "" { + return unavailable(errors.New("action insert returned no identifier")) + } + if err := tx.SetLabelDecision(ctx, s.config.InstanceDID, request.Subject.URI, LabelNSFW, action.ID, true); err != nil { + return unavailable(err) + } + if err := tx.SetSubjectVersion(ctx, request.Subject.URI, version+1); err != nil { + return unavailable(err) + } + appliedLabels, err := tx.ActiveLabels(ctx, s.config.InstanceDID, request.Subject.URI) + if err != nil { + return unavailable(err) + } + state, err := newSubjectState(request.Subject.URI, version+1, RecordStatePresent, current, active, appliedLabels, s.config.InstanceDID) + if err != nil { + return err + } + result = &MutationResult{Outcome: OutcomeApplied, State: state, Action: action} + } + return tx.SaveIdempotencyRecord(ctx, IdempotencyRecord{ + ActorDID: actorDID, AuthorityDID: s.config.InstanceDID, Key: request.IdempotencyKey, + Fingerprint: fingerprint, Result: *result, CreatedAt: now, ExpiresAt: now.Add(s.config.IdempotencyRetention), + }) + }) + if err != nil { + if ruleError != nil && errors.Is(err, ruleError) { + return nil, ruleError + } + return nil, fmt.Errorf("%w: %w", ErrModerationUnavailable, err) + } + return result, nil +} diff --git a/internal/core/moderation/label_rules_test.go b/internal/core/moderation/label_rules_test.go new file mode 100644 index 0000000..6cce895 --- /dev/null +++ b/internal/core/moderation/label_rules_test.go @@ -0,0 +1,308 @@ +package moderation_test + +import ( + "errors" + "strings" + "testing" + + "Coves/internal/core/moderation" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func newLabelRulesScenario() (removeRulesScenario, moderation.LabelContentRequest) { + scenario := newPostRulesScenario(postRulesURI, postRulesAuthorDID, nil) + return scenario, moderation.LabelContentRequest{ + Subject: scenario.request.Subject, LabelValue: moderation.LabelNSFW, + ExpectedVersion: "v0", IdempotencyKey: "first-label", + } +} + +func labelRulesNoWrites(t *testing.T, scenario removeRulesScenario, before inMemoryModerationState) { + t.Helper() + assert.Empty(t, scenario.store.writeCalls) + assert.Equal(t, before, scenario.store.state, "rejection must not commit even a version or idempotency key") + assert.Empty(t, scenario.purger.ownerPurges) + assert.Empty(t, scenario.purger.blobPurges) +} + +func labelRulesAction(id, uri string) moderation.Action { + return moderation.Action{ + ID: id, Action: moderation.ActionLabel, LabelValue: moderation.LabelNSFW, + AuthorityDID: removeRulesInstanceDID, ScopeKind: moderation.ScopeInstance, + SubjectURI: uri, SubjectCollection: moderation.PostV2Collection, + SubjectCommunityDID: postRulesCommunityDID, ObservedCID: postRulesCID, + Origin: moderation.OriginLocal, + } +} + +func TestLabelContentRejectsInvalidFieldsWithoutWrites(t *testing.T) { + for _, test := range []struct { + name string + change func(*removeRulesScenario, *moderation.LabelContentRequest) + want error + }{ + {"comment subject", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.Subject.URI = removeRulesURI + request.Subject.CID = removeRulesCID + }, moderation.ErrInvalidSubject}, + {"non-content collection", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.Subject.URI = "at://did:plc:postauthor/social.coves.actor.profile/self" + }, moderation.ErrInvalidSubject}, + {"invalid CID", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.Subject.CID = "not a cid" + }, moderation.ErrInvalidRequest}, + {"empty value", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.LabelValue = "" + }, moderation.ErrInvalidRequest}, + {"129 byte value", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.LabelValue = strings.Repeat("x", 129) + }, moderation.ErrInvalidRequest}, + {"128 byte value is well formed but unsupported", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.LabelValue = strings.Repeat("x", 128) + }, moderation.ErrUnsupportedLabel}, + {"gore", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.LabelValue = "gore" + }, moderation.ErrUnsupportedLabel}, + {"takedown", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.LabelValue = "!takedown" + }, moderation.ErrUnsupportedLabel}, + {"reason over 640 bytes", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.Reason = strings.Repeat("r", 641) + }, moderation.ErrInvalidRequest}, + {"invalid UTF-8 reason", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.Reason = string([]byte{0xff}) + }, moderation.ErrInvalidRequest}, + {"unknown reason", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.Reason = "social.coves.moderation.defs#reasonUnknown" + }, moderation.ErrUnsupportedReason}, + {"doxing reason is only for removal", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.Reason = "social.coves.moderation.defs#reasonDoxing" + }, moderation.ErrUnsupportedReason}, + {"illegal-content reason is only for removal", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.Reason = removeRulesIllegal + }, moderation.ErrUnsupportedReason}, + } { + t.Run(test.name, func(t *testing.T) { + scenario, request := newLabelRulesScenario() + test.change(&scenario, &request) + before := scenario.store.state.copy() + result, err := scenario.service.LabelContent(t.Context(), removeRulesAdminDID, request) + require.ErrorIs(t, err, test.want) + assert.Nil(t, result) + labelRulesNoWrites(t, scenario, before) + }) + } +} + +func TestLabelContentAppliesToBothPostCollectionsAndAcceptsOptionalReasons(t *testing.T) { + for _, reason := range []string{ + "", removeRulesSpam, "social.coves.moderation.defs#reasonHarassment", + "social.coves.moderation.defs#reasonRuleViolation", + "social.coves.moderation.defs#reasonModeratorDiscretion", + } { + for _, post := range []struct{ name, uri, owner string }{ + {"postv2", postRulesURI, postRulesAuthorDID}, + {"legacy post", legacyPostRulesURI, postRulesCommunityDID}, + } { + t.Run(post.name+" reason "+reason, func(t *testing.T) { + scenario := newPostRulesScenario(post.uri, post.owner, nil) + request := moderation.LabelContentRequest{ + Subject: scenario.request.Subject, LabelValue: moderation.LabelNSFW, + ExpectedVersion: "v0", IdempotencyKey: "apply-label", Reason: reason, PrivateNote: "operator note", + } + result, err := scenario.service.LabelContent(t.Context(), removeRulesAdminDID, request) + require.NoError(t, err) + require.NotNil(t, result, "a successful label must return its mutation result") + require.Equal(t, moderation.OutcomeApplied, result.Outcome) + require.NotNil(t, result.Action) + action := result.Action + require.NotEmpty(t, action.ID) + assert.Equal(t, moderation.ActionLabel, action.Action) + assert.Equal(t, moderation.LabelNSFW, action.LabelValue) + assert.Equal(t, postRulesCID, action.ObservedCID) + assert.Equal(t, reason, action.Reason) + assert.Equal(t, request.PrivateNote, action.PrivateNote) + assert.Equal(t, removeRulesAdminDID, action.ActorDID) + assert.Equal(t, removeRulesInstanceDID, action.AuthorityDID) + assert.Equal(t, moderation.ScopeInstance, action.ScopeKind) + assert.Equal(t, moderation.OriginLocal, action.Origin) + assert.Equal(t, post.uri, action.SubjectURI) + assert.Equal(t, postRulesCommunityDID, action.SubjectCommunityDID) + collection := moderation.PostV2Collection + if post.uri == legacyPostRulesURI { + collection = moderation.LegacyPostCollection + } + assert.Equal(t, collection, action.SubjectCollection) + assert.Equal(t, "v1", result.State.Version) + assert.Equal(t, moderation.RecordStatePresent, result.State.RecordState) + assert.Equal(t, moderation.ModerationStateClear, result.State.Moderation.State) + assert.Equal(t, []moderation.LocalLabel{{Value: moderation.LabelNSFW, + Action: moderation.ActionRef{ServiceDID: removeRulesInstanceDID, ActionID: action.ID}}}, result.State.LocalLabels) + assert.Equal(t, []moderation.ContentLabel{{Value: moderation.LabelNSFW, + Sources: []moderation.DecisionSource{{AuthorityDID: removeRulesInstanceDID, ScopeKind: moderation.ScopeInstance}}}}, + result.State.Moderation.ContentLabels) + assert.Equal(t, map[string]moderation.Action{action.ID: *action}, scenario.store.state.actions) + assert.Equal(t, inMemoryModerationLabelDecision{actionID: action.ID, active: true}, + scenario.store.state.labelDecisions[inMemoryModerationLabelKey{removeRulesInstanceDID, post.uri, moderation.LabelNSFW}]) + assert.Equal(t, map[string]int64{post.uri: 1}, scenario.store.state.versions) + assert.Equal(t, []string{"InsertAction", "SetLabelDecision", "SetSubjectVersion", "SaveIdempotencyRecord"}, scenario.store.writeCalls) + assert.Empty(t, scenario.store.state.mediaBlocks) + assert.Empty(t, scenario.purger.ownerPurges) + assert.Empty(t, scenario.purger.blobPurges) + }) + } + } +} + +func TestLabelContentRedundantApplyKeepsActionAndVersion(t *testing.T) { + scenario, request := newLabelRulesScenario() + prior := labelRulesAction("previous-label", postRulesURI) + scenario.store.seedActiveLabel(prior) + scenario.store.state.versions[postRulesURI] = 1 + request.ExpectedVersion = "v1" + request.IdempotencyKey = "different-key" + result, err := scenario.service.LabelContent(t.Context(), removeRulesAdminDID, request) + require.NoError(t, err) + require.NotNil(t, result) + assert.Equal(t, moderation.OutcomeUnchanged, result.Outcome) + assert.Nil(t, result.Action) + assert.Equal(t, "v1", result.State.Version) + assert.Equal(t, []moderation.LocalLabel{{Value: moderation.LabelNSFW, + Action: moderation.ActionRef{ServiceDID: removeRulesInstanceDID, ActionID: prior.ID}}}, result.State.LocalLabels) + assert.Equal(t, []moderation.ContentLabel{{Value: moderation.LabelNSFW, + Sources: []moderation.DecisionSource{{AuthorityDID: removeRulesInstanceDID, ScopeKind: moderation.ScopeInstance}}}}, + result.State.Moderation.ContentLabels) + assert.Equal(t, map[string]moderation.Action{prior.ID: prior}, scenario.store.state.actions) + assert.Equal(t, map[string]int64{postRulesURI: 1}, scenario.store.state.versions) + assert.Equal(t, []string{"SaveIdempotencyRecord"}, scenario.store.writeCalls) +} + +func TestLabelContentIdempotencyReplayAndChangedValue(t *testing.T) { + scenario, request := newLabelRulesScenario() + original, err := scenario.service.LabelContent(t.Context(), removeRulesAdminDID, request) + require.NoError(t, err) + require.NotNil(t, original) + require.NotNil(t, original.Action) + replayed, err := assertIdempotencyNoMutation(t, scenario, func() (*moderation.MutationResult, error) { + return scenario.service.LabelContent(t.Context(), removeRulesAdminDID, request) + }) + require.NoError(t, err) + assert.Equal(t, original, replayed) + changed := request + changed.LabelValue = "gore" + result, err := assertIdempotencyNoMutation(t, scenario, func() (*moderation.MutationResult, error) { + return scenario.service.LabelContent(t.Context(), removeRulesAdminDID, changed) + }) + require.ErrorIs(t, err, moderation.ErrIdempotencyConflict) + assert.Nil(t, result) + assert.Len(t, scenario.store.state.actions, 1) +} + +func TestLabelContentRejectsStaleMissingAndDeletedPostsWithoutWrites(t *testing.T) { + for _, test := range []struct { + name string + change func(*removeRulesScenario, *moderation.LabelContentRequest) + want error + }{ + {"stale version", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.ExpectedVersion = "v2" + }, moderation.ErrStateConflict}, + {"changed CID", func(_ *removeRulesScenario, request *moderation.LabelContentRequest) { + request.Subject.CID = "bafyreidifferentpostcid" + }, moderation.ErrContentChanged}, + {"never indexed", func(scenario *removeRulesScenario, _ *moderation.LabelContentRequest) { + delete(scenario.store.state.indexedPosts, postRulesURI) + scenario.reader.record = nil + scenario.reader.err = moderation.ErrSubjectNotIndexed + }, moderation.ErrSubjectNotFound}, + {"author deleted", func(scenario *removeRulesScenario, _ *moderation.LabelContentRequest) { + post := scenario.store.state.indexedPosts[postRulesURI] + post.AuthorDeleted = true + scenario.store.state.indexedPosts[postRulesURI] = post + scenario.reader.record.Deleted = true + }, moderation.ErrSubjectNotFound}, + } { + t.Run(test.name, func(t *testing.T) { + scenario, request := newLabelRulesScenario() + test.change(&scenario, &request) + before := scenario.store.state.copy() + result, err := scenario.service.LabelContent(t.Context(), removeRulesAdminDID, request) + require.ErrorIs(t, err, test.want) + assert.Nil(t, result) + labelRulesNoWrites(t, scenario, before) + }) + } +} + +func TestLabelContentCanLabelRemovedPresentPost(t *testing.T) { + scenario, request := newLabelRulesScenario() + removal := moderation.Action{ + ID: "existing-removal", Action: moderation.ActionRemove, AuthorityDID: removeRulesInstanceDID, + ScopeKind: moderation.ScopeInstance, SubjectURI: postRulesURI, SubjectCollection: moderation.PostV2Collection, + } + scenario.store.state.actions[removal.ID] = removal + scenario.store.state.activeRemovals[inMemoryModerationDecisionKey{removeRulesInstanceDID, postRulesURI}] = removal.ID + scenario.store.state.versions[postRulesURI] = 1 + request.ExpectedVersion = "v1" + result, err := scenario.service.LabelContent(t.Context(), removeRulesAdminDID, request) + require.NoError(t, err) + require.NotNil(t, result) + require.Equal(t, moderation.OutcomeApplied, result.Outcome) + require.NotNil(t, result.Action) + assert.Equal(t, "v2", result.State.Version) + assert.Equal(t, moderation.ModerationStateRemoved, result.State.Moderation.State) + assert.Equal(t, &moderation.ActionRef{ServiceDID: removeRulesInstanceDID, ActionID: removal.ID}, result.State.LocalRemoval) + assert.Equal(t, []moderation.LocalLabel{{Value: moderation.LabelNSFW, + Action: moderation.ActionRef{ServiceDID: removeRulesInstanceDID, ActionID: result.Action.ID}}}, result.State.LocalLabels) + assert.Equal(t, []moderation.ContentLabel{{Value: moderation.LabelNSFW, + Sources: []moderation.DecisionSource{{AuthorityDID: removeRulesInstanceDID, ScopeKind: moderation.ScopeInstance}}}}, + result.State.Moderation.ContentLabels) + assert.Equal(t, removal.ID, scenario.store.state.activeRemovals[inMemoryModerationDecisionKey{removeRulesInstanceDID, postRulesURI}]) +} + +func TestLabelContentReturnsActiveLabelsOrderedByValue(t *testing.T) { + scenario, request := newLabelRulesScenario() + spoiler := labelRulesAction("existing-spoiler-label", postRulesURI) + spoiler.LabelValue = "spoiler" + scenario.store.seedActiveLabel(spoiler) + scenario.store.state.versions[postRulesURI] = 1 + request.ExpectedVersion = "v1" + result, err := scenario.service.LabelContent(t.Context(), removeRulesAdminDID, request) + require.NoError(t, err) + require.NotNil(t, result) + require.NotNil(t, result.Action) + assert.Equal(t, []moderation.LocalLabel{ + {Value: moderation.LabelNSFW, Action: moderation.ActionRef{ServiceDID: removeRulesInstanceDID, ActionID: result.Action.ID}}, + {Value: "spoiler", Action: moderation.ActionRef{ServiceDID: removeRulesInstanceDID, ActionID: spoiler.ID}}, + }, result.State.LocalLabels, "labels must be ordered by value, as getSubjectState orders them") +} + +func TestLabelContentStoreFailureRollsBackEverything(t *testing.T) { + for _, test := range []struct { + name string + inject func(*inMemoryModerationStore, error) + wantCalls []string + }{ + {"SetLabelDecision failure rolls back the inserted action", func(store *inMemoryModerationStore, err error) { + store.failSetLabelDecision = err + }, []string{"InsertAction", "SetLabelDecision"}}, + {"label read failure rolls back the action, decision and version", func(store *inMemoryModerationStore, err error) { + store.failActiveLabelsAfterLabelDecision = err + }, []string{"InsertAction", "SetLabelDecision", "SetSubjectVersion"}}, + } { + t.Run(test.name, func(t *testing.T) { + scenario, request := newLabelRulesScenario() + storageError := errors.New("injected store failure") + test.inject(scenario.store, storageError) + before := scenario.store.state.copy() + result, err := scenario.service.LabelContent(t.Context(), removeRulesAdminDID, request) + require.ErrorIs(t, err, moderation.ErrModerationUnavailable) + assert.ErrorIs(t, err, storageError) + assert.Nil(t, result) + assert.Equal(t, test.wantCalls, scenario.store.writeCalls) + assert.Equal(t, before, scenario.store.state, "a failed label must persist no action, decision, version or idempotency key") + }) + } +} diff --git a/internal/core/moderation/modlog.go b/internal/core/moderation/modlog.go index cd290e6..97e209c 100644 --- a/internal/core/moderation/modlog.go +++ b/internal/core/moderation/modlog.go @@ -5,6 +5,7 @@ import ( "crypto/sha256" "errors" "fmt" + "slices" "strings" "time" "unicode" @@ -36,6 +37,17 @@ func HiddenActionReasons() []string { return []string{illegalContentReason, doxingReason} } +// PublicExcludedActions returns the action kinds the public log never serves. +// NSFW label applies and retractions are left out of it (user decision, +// 2026-09-30); the admin log keeps them. +func PublicExcludedActions() []string { + return []string{ActionLabel, ActionRetractLabel} +} + +func publicExcludedAction(kind string) bool { + return slices.Contains(PublicExcludedActions(), kind) +} + func hiddenAction(action Action) bool { for _, reason := range HiddenActionReasons() { if action.Reason == reason || action.ReversedActionReason == reason { @@ -201,11 +213,12 @@ func listActionPage[T any](ctx context.Context, s *service, params ListActionsPa if err != nil { return nil, "", fmt.Errorf("%w: list %s actions: %w", ErrModerationUnavailable, visibility, err) } - if query.ExcludeHidden { - for _, row := range rows { - if hiddenAction(row) { - return nil, "", fmt.Errorf("%w: list %s actions: store returned hidden action %s despite ExcludeHidden", ErrModerationUnavailable, visibility, row.ID) - } + for _, row := range rows { + if query.ExcludeHidden && hiddenAction(row) { + return nil, "", fmt.Errorf("%w: list %s actions: store returned hidden action %s despite ExcludeHidden", ErrModerationUnavailable, visibility, row.ID) + } + if query.ExcludeLabelActions && publicExcludedAction(row.Action) { + return nil, "", fmt.Errorf("%w: list %s actions: store returned %s action %s despite ExcludeLabelActions", ErrModerationUnavailable, visibility, row.Action, row.ID) } } pageSize := query.Limit - 1 @@ -226,6 +239,9 @@ func (s *service) actionListQuery(ctx context.Context, params ListActionsParams, if err != nil { return ActionListQuery{}, digest, err } + if !admin && publicExcludedAction(params.Action) { + return ActionListQuery{}, digest, fmt.Errorf("%w: the public log does not serve %s actions", ErrInvalidRequest, params.Action) + } if s.config.CursorSecret == "" { return ActionListQuery{}, digest, fmt.Errorf("%w: cursor secret is not configured", ErrModerationUnavailable) } @@ -288,6 +304,7 @@ func (s *service) actionListQuery(ctx context.Context, params ListActionsParams, query.Before = &cursor.key } query.ExcludeHidden = !admin && (params.Subject != "" || params.Collection != "" || params.Community != "") + query.ExcludeLabelActions = !admin return query, digest, nil } diff --git a/internal/core/moderation/modlog_label_exclusion_test.go b/internal/core/moderation/modlog_label_exclusion_test.go new file mode 100644 index 0000000..e1440e3 --- /dev/null +++ b/internal/core/moderation/modlog_label_exclusion_test.go @@ -0,0 +1,116 @@ +package moderation_test + +import ( + "context" + "fmt" + "testing" + "time" + + "Coves/internal/core/moderation" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// NSFW label applies and retractions are left out of the public modlog +// (user decision, 2026-09-30). The admin log keeps them. + +func TestModlogPublicRejectsLabelActionFilters(t *testing.T) { + for _, action := range []string{moderation.ActionLabel, moderation.ActionRetractLabel} { + t.Run(action+"/public", func(t *testing.T) { + store := newInMemoryModerationStore(time.Now()) + page, err := modlogFilterList(t, modlogFilterService(store, nil, nil), false, moderation.ListActionsParams{Action: action}, "") + require.ErrorIs(t, err, moderation.ErrInvalidRequest) + assert.Nil(t, page) + assert.Empty(t, store.listQueries, "a refused public filter must not reach the store") + }) + t.Run(action+"/admin", func(t *testing.T) { + store := newInMemoryModerationStore(time.Now()) + _, err := modlogFilterList(t, modlogFilterService(store, nil, nil), true, moderation.ListActionsParams{Action: action}, "") + require.NoError(t, err) + require.Len(t, store.listQueries, 1) + assert.Equal(t, action, store.listQueries[0].Action) + assert.False(t, store.listQueries[0].ExcludeLabelActions) + }) + } +} + +func TestModlogExcludeLabelActionsOnPublicQueriesOnly(t *testing.T) { + for _, test := range []struct { + name string + params moderation.ListActionsParams + }{ + {"unfiltered", moderation.ListActionsParams{}}, + {"subject", moderation.ListActionsParams{Subject: modlogTestAction().SubjectURI}}, + {"collection", moderation.ListActionsParams{Collection: moderation.PostV2Collection}}, + {"community", moderation.ListActionsParams{Community: "did:plc:community"}}, + {"actor", moderation.ListActionsParams{Actor: "did:plc:actor"}}, + {"authority", moderation.ListActionsParams{Authority: "did:plc:authority"}}, + {"action", moderation.ListActionsParams{Action: moderation.ActionRemove}}, + {"origin", moderation.ListActionsParams{Origin: moderation.OriginLocal}}, + {"time range", moderation.ListActionsParams{Since: "2026-09-28T12:00:00Z", Until: "2026-09-29T12:00:00Z"}}, + } { + for _, admin := range []bool{false, true} { + t.Run(fmt.Sprintf("%s/admin=%t", test.name, admin), func(t *testing.T) { + store := newInMemoryModerationStore(time.Now()) + store.listRows = modlogCursorRows() + service := modlogFilterService(store, nil, nil) + params := test.params + params.Limit = modlogLimit(1) + page, err := modlogFilterList(t, service, admin, params, "") + require.NoError(t, err) + params.Cursor = modlogPageCursor(t, page) + require.NotEmpty(t, params.Cursor) + _, err = modlogFilterList(t, service, admin, params, "") + require.NoError(t, err) + require.Len(t, store.listQueries, 2) + for index, query := range store.listQueries { + assert.Equal(t, !admin, query.ExcludeLabelActions, "query %d", index) + } + require.NotNil(t, store.listQueries[1].Before, "the second query must be a cursor page") + }) + } + } +} + +// labelLeakingStore returns its rows whatever the query asks, standing in for +// SQL that disagrees with the Go rule. +type labelLeakingStore struct { + *inMemoryModerationStore + rows []moderation.Action +} + +func (store *labelLeakingStore) ListActions(context.Context, moderation.ActionListQuery) ([]moderation.Action, error) { + return append([]moderation.Action(nil), store.rows...), nil +} + +// SQL and Go must agree on the excluded kinds. A label row returned under +// ExcludeLabelActions means they disagree, and the public page fails closed. +func TestModlogListFailsClosedWhenStoreReturnsLabelActionPublicly(t *testing.T) { + for _, kind := range []string{moderation.ActionLabel, moderation.ActionRetractLabel} { + for _, lookAhead := range []bool{false, true} { + t.Run(fmt.Sprintf("%s/look-ahead=%t", kind, lookAhead), func(t *testing.T) { + rows := modlogCursorRows() + index := 0 + if lookAhead { + index = 2 // The unprojected look-ahead row still proves the disagreement. + } + rows[index].Action = kind + rows[index].LabelValue = moderation.LabelNSFW + store := &labelLeakingStore{inMemoryModerationStore: newInMemoryModerationStore(time.Now()), rows: rows} + service := moderation.NewService(&fakeSubjectReader{}, store, + moderation.Config{InstanceDID: "did:web:instance.example", CursorSecret: "secret-one"}) + for _, params := range []moderation.ListActionsParams{ + {Limit: modlogLimit(2)}, + {Limit: modlogLimit(2), Subject: modlogTestAction().SubjectURI}, + } { + page, err := modlogFilterList(t, service, false, params, "") + require.ErrorIs(t, err, moderation.ErrModerationUnavailable) + assert.Nil(t, page) + _, err = modlogFilterList(t, service, true, params, "") + require.NoError(t, err, "the admin log serves label actions") + } + }) + } + } +} diff --git a/internal/core/moderation/mutation_validation.go b/internal/core/moderation/mutation_validation.go index 3630b01..2814c9b 100644 --- a/internal/core/moderation/mutation_validation.go +++ b/internal/core/moderation/mutation_validation.go @@ -16,11 +16,8 @@ func validOpaqueField(value string) bool { } func validateMutationFields(key, expectedVersion, reason, privateNote string) error { - if !validOpaqueField(key) || !validOpaqueField(expectedVersion) { - return fmt.Errorf("%w: idempotencyKey and expectedVersion must be 1-128 UTF-8 bytes", ErrInvalidRequest) - } - if !utf8.ValidString(privateNote) || len(privateNote) > 10000 || uniseg.GraphemeClusterCount(privateNote) > 1000 { - return fmt.Errorf("%w: privateNote exceeds its length limit", ErrInvalidRequest) + if err := validateMutationBaseFields(key, expectedVersion, privateNote); err != nil { + return err } if reason == "" || len(reason) > 640 || !utf8.ValidString(reason) { return fmt.Errorf("%w: reason must be 1-640 UTF-8 bytes", ErrInvalidRequest) @@ -31,6 +28,36 @@ func validateMutationFields(key, expectedVersion, reason, privateNote string) er return nil } +// validateOptionalReason checks the reason of a classification mutation +// (labelContent, retractContentLabel). The reason may be empty. The doxing and +// illegal-content reasons are only for removal: content left readable under +// either would point readers at it. +func validateOptionalReason(reason string) error { + if reason == "" { + return nil + } + if len(reason) > 640 || !utf8.ValidString(reason) { + return fmt.Errorf("%w: reason must be at most 640 UTF-8 bytes", ErrInvalidRequest) + } + if _, ok := removeReasons[reason]; !ok { + return fmt.Errorf("%w: unsupported moderation reason", ErrUnsupportedReason) + } + if reason == illegalContentReason || reason == doxingReason { + return fmt.Errorf("%w: the illegal-content and doxing reasons are only for removal", ErrUnsupportedReason) + } + return nil +} + +func validateMutationBaseFields(key, expectedVersion, privateNote string) error { + if !validOpaqueField(key) || !validOpaqueField(expectedVersion) { + return fmt.Errorf("%w: idempotencyKey and expectedVersion must be 1-128 UTF-8 bytes", ErrInvalidRequest) + } + if !utf8.ValidString(privateNote) || len(privateNote) > 10000 || uniseg.GraphemeClusterCount(privateNote) > 1000 { + return fmt.Errorf("%w: privateNote exceeds its length limit", ErrInvalidRequest) + } + return nil +} + func validContentStrongRef(ref StrongRef) bool { uri, err := syntax.ParseATURI(ref.URI) if err != nil || !uri.Authority().IsDID() || uri.RecordKey().String() == "" { @@ -47,9 +74,9 @@ func validContentStrongRef(ref StrongRef) bool { // A fixed ordered tuple makes request fingerprints unambiguous even if a // caller includes delimiters in opaque fields or private notes. -func mutationFingerprint(operation, actionID, subjectURI, subjectCID, expectedVersion, reason, privateNote string) string { +func mutationFingerprint(operation, actionID, subjectURI, subjectCID, expectedVersion, reason, labelValue, privateNote string) string { payload, _ := json.Marshal([8]string{ - operation, actionID, subjectURI, subjectCID, expectedVersion, reason, "", privateNote, + operation, actionID, subjectURI, subjectCID, expectedVersion, reason, labelValue, privateNote, }) hash := sha256.Sum256(payload) return hex.EncodeToString(hash[:]) diff --git a/internal/core/moderation/remove.go b/internal/core/moderation/remove.go index 8a79fab..ca0655e 100644 --- a/internal/core/moderation/remove.go +++ b/internal/core/moderation/remove.go @@ -14,6 +14,10 @@ const ( doxingReason = "social.coves.moderation.defs#reasonDoxing" ) +// removeReasons is every known moderation reason. removeContent and +// restoreContent require one; labelContent and retractContentLabel accept one +// optionally. Restore, label and retract-label reject the illegal-content and +// doxing reasons on top of this set. var removeReasons = map[string]struct{}{ "social.coves.moderation.defs#reasonSpam": {}, "social.coves.moderation.defs#reasonHarassment": {}, @@ -47,7 +51,7 @@ func (s *service) removeContent(ctx context.Context, actorDID string, request Re if s.config.Now != nil { now = s.config.Now() } - fingerprint := mutationFingerprint(ActionRemove, "", request.Subject.URI, request.Subject.CID, request.ExpectedVersion, request.Reason, request.PrivateNote) + fingerprint := mutationFingerprint(ActionRemove, "", request.Subject.URI, request.Subject.CID, request.ExpectedVersion, request.Reason, "", request.PrivateNote) var result *MutationResult var newlyBlocked []MediaBlock var ruleError error @@ -103,6 +107,10 @@ func (s *service) removeContent(ctx context.Context, actorDID string, request Re if err != nil { return unavailable(err) } + activeLabels, err := tx.ActiveLabels(ctx, s.config.InstanceDID, request.Subject.URI) + if err != nil { + return unavailable(err) + } recordState := RecordStatePresent var current *StrongRef if subject.AuthorDeleted { @@ -111,7 +119,7 @@ func (s *service) removeContent(ctx context.Context, actorDID string, request Re current = &StrongRef{URI: subject.URI, CID: subject.CID} } if active != nil { - state, err := newSubjectState(request.Subject.URI, version, recordState, current, active, s.config.InstanceDID) + state, err := newSubjectState(request.Subject.URI, version, recordState, current, active, activeLabels, s.config.InstanceDID) if err != nil { return err } @@ -142,7 +150,7 @@ func (s *service) removeContent(ctx context.Context, actorDID string, request Re return unavailable(err) } } - state, err := newSubjectState(request.Subject.URI, version+1, recordState, current, action, s.config.InstanceDID) + state, err := newSubjectState(request.Subject.URI, version+1, recordState, current, action, activeLabels, s.config.InstanceDID) if err != nil { return err } diff --git a/internal/core/moderation/restore.go b/internal/core/moderation/restore.go index 96984fa..d043391 100644 --- a/internal/core/moderation/restore.go +++ b/internal/core/moderation/restore.go @@ -38,7 +38,7 @@ func (s *service) restoreContent(ctx context.Context, actorDID string, request R reviewedURI = request.ReviewedSubject.URI reviewedCID = request.ReviewedSubject.CID } - fingerprint := mutationFingerprint(ActionRestore, request.ActionID, reviewedURI, reviewedCID, request.ExpectedVersion, request.Reason, request.PrivateNote) + fingerprint := mutationFingerprint(ActionRestore, request.ActionID, reviewedURI, reviewedCID, request.ExpectedVersion, request.Reason, "", request.PrivateNote) var result *MutationResult var ruleError error err := s.store.InTransaction(ctx, func(ctx context.Context, tx Transaction) error { @@ -148,7 +148,11 @@ func (s *service) restoreContent(ctx context.Context, actorDID string, request R if err := tx.DeactivateMediaBlocks(ctx, removal.ID); err != nil { return unavailable(err) } - state, err := newSubjectState(removal.SubjectURI, version+1, recordState, current, nil, s.config.InstanceDID) + activeLabels, err := tx.ActiveLabels(ctx, s.config.InstanceDID, removal.SubjectURI) + if err != nil { + return unavailable(err) + } + state, err := newSubjectState(removal.SubjectURI, version+1, recordState, current, nil, activeLabels, s.config.InstanceDID) if err != nil { return err } diff --git a/internal/core/moderation/retract_label.go b/internal/core/moderation/retract_label.go new file mode 100644 index 0000000..36fa7f5 --- /dev/null +++ b/internal/core/moderation/retract_label.go @@ -0,0 +1,178 @@ +package moderation + +import ( + "context" + "errors" + "fmt" + "time" +) + +func validateRetractLabelRequest(request RetractContentLabelRequest) error { + if !validOpaqueField(request.ActionID) { + return fmt.Errorf("%w: actionId must be 1-128 UTF-8 bytes", ErrInvalidRequest) + } + if err := validateMutationBaseFields(request.IdempotencyKey, request.ExpectedVersion, request.PrivateNote); err != nil { + return err + } + if err := validateOptionalReason(request.Reason); err != nil { + return err + } + if request.ReviewedSubject != nil && !validContentStrongRef(*request.ReviewedSubject) { + return fmt.Errorf("%w: reviewedSubject must be a content strongRef", ErrInvalidRequest) + } + return nil +} + +func (s *service) retractContentLabel(ctx context.Context, actorDID string, request RetractContentLabelRequest) (*MutationResult, error) { + if err := validateRetractLabelRequest(request); err != nil { + return nil, err + } + now := time.Now() + if s.config.Now != nil { + now = s.config.Now() + } + var reviewedURI, reviewedCID string + if request.ReviewedSubject != nil { + reviewedURI = request.ReviewedSubject.URI + reviewedCID = request.ReviewedSubject.CID + } + fingerprint := mutationFingerprint(ActionRetractLabel, request.ActionID, reviewedURI, reviewedCID, request.ExpectedVersion, request.Reason, "", request.PrivateNote) + var result *MutationResult + var ruleError error + err := s.store.InTransaction(ctx, func(ctx context.Context, tx Transaction) error { + fail := func(err error) error { + ruleError = err + return err + } + unavailable := func(err error) error { return fmt.Errorf("%w: %w", ErrModerationUnavailable, err) } + if err := tx.LockActor(ctx, actorDID); err != nil { + return unavailable(err) + } + stored, err := tx.LiveIdempotencyRecord(ctx, actorDID, s.config.InstanceDID, request.IdempotencyKey, now) + if err != nil { + return unavailable(err) + } + if stored != nil { + if stored.Fingerprint != fingerprint { + return fail(ErrIdempotencyConflict) + } + copy := stored.Result + result = © + return nil + } + count, err := tx.CountLiveIdempotencyKeys(ctx, actorDID, now) + if err != nil { + return unavailable(err) + } + if count >= s.config.MaxLiveIdempotencyKeys { + return fail(fmt.Errorf("%w: live idempotency key limit %d reached", ErrInvalidRequest, s.config.MaxLiveIdempotencyKeys)) + } + label, err := tx.GetAction(ctx, request.ActionID) + if errors.Is(err, ErrDecisionNotFound) { + return fail(ErrDecisionNotFound) + } + if err != nil { + return unavailable(err) + } + if label == nil { + return unavailable(errors.New("action lookup returned no action")) + } + if label.Action != ActionLabel || label.AuthorityDID != s.config.InstanceDID || label.ScopeKind != ScopeInstance || label.Origin != OriginLocal { + return fail(ErrInvalidDecision) + } + version, err := tx.LockSubject(ctx, label.SubjectURI) + if err != nil { + return unavailable(err) + } + activeLabels, err := tx.ActiveLabels(ctx, s.config.InstanceDID, label.SubjectURI) + if err != nil { + return unavailable(err) + } + found := false + for _, activeLabel := range activeLabels { + if activeLabel.ID == label.ID { + found = true + break + } + } + if !found { + return fail(ErrInvalidDecision) + } + if request.ExpectedVersion != versionToken(version) { + return fail(ErrStateConflict) + } + if request.ReviewedSubject != nil && reviewedURI != label.SubjectURI { + return fail(fmt.Errorf("%w: reviewedSubject URI differs from label subject", ErrInvalidRequest)) + } + subject, err := readIndexedSubject(ctx, tx, label.SubjectURI) + if err != nil && !errors.Is(err, ErrSubjectNotIndexed) { + return unavailable(err) + } + recordState := RecordStateUnavailable + var current *StrongRef + var observedCID string + if err == nil { + if subject == nil { + return unavailable(errors.New("indexed subject lookup returned no subject")) + } + observedCID = subject.CID + if subject.AuthorDeleted { + recordState = RecordStateDeleted + } else { + recordState = RecordStatePresent + current = &StrongRef{URI: subject.URI, CID: subject.CID} + if request.ReviewedSubject == nil { + return fail(fmt.Errorf("%w: reviewedSubject is required for present content", ErrInvalidRequest)) + } + if reviewedCID != subject.CID { + return fail(ErrContentChanged) + } + } + } + action, err := tx.InsertAction(ctx, Action{ + ActorDID: actorDID, AuthorityDID: s.config.InstanceDID, + ScopeKind: ScopeInstance, SubjectURI: label.SubjectURI, + SubjectCollection: label.SubjectCollection, SubjectCommunityDID: label.SubjectCommunityDID, + ObservedCID: observedCID, Action: ActionRetractLabel, LabelValue: label.LabelValue, + Reason: request.Reason, PrivateNote: request.PrivateNote, ReversesActionID: label.ID, + ReversedActionReason: label.Reason, + Origin: OriginLocal, CreatedAt: now, + }) + if err != nil { + return unavailable(err) + } + if action == nil || action.ID == "" { + return unavailable(errors.New("action insert returned no identifier")) + } + if err := tx.SetLabelDecision(ctx, s.config.InstanceDID, label.SubjectURI, label.LabelValue, label.ID, false); err != nil { + return unavailable(err) + } + if err := tx.SetSubjectVersion(ctx, label.SubjectURI, version+1); err != nil { + return unavailable(err) + } + removal, err := tx.ActiveRemoval(ctx, s.config.InstanceDID, label.SubjectURI) + if err != nil { + return unavailable(err) + } + remainingLabels, err := tx.ActiveLabels(ctx, s.config.InstanceDID, label.SubjectURI) + if err != nil { + return unavailable(err) + } + state, err := newSubjectState(label.SubjectURI, version+1, recordState, current, removal, remainingLabels, s.config.InstanceDID) + if err != nil { + return err + } + result = &MutationResult{Outcome: OutcomeApplied, State: state, Action: action} + return tx.SaveIdempotencyRecord(ctx, IdempotencyRecord{ + ActorDID: actorDID, AuthorityDID: s.config.InstanceDID, Key: request.IdempotencyKey, + Fingerprint: fingerprint, Result: *result, CreatedAt: now, ExpiresAt: now.Add(s.config.IdempotencyRetention), + }) + }) + if err != nil { + if ruleError != nil && errors.Is(err, ruleError) { + return nil, ruleError + } + return nil, fmt.Errorf("%w: %w", ErrModerationUnavailable, err) + } + return result, nil +} diff --git a/internal/core/moderation/retract_label_rules_test.go b/internal/core/moderation/retract_label_rules_test.go new file mode 100644 index 0000000..737d649 --- /dev/null +++ b/internal/core/moderation/retract_label_rules_test.go @@ -0,0 +1,427 @@ +package moderation_test + +import ( + "context" + "errors" + "strings" + "testing" + + "Coves/internal/core/moderation" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +type retractLabelRulesScenario struct { + removeRulesScenario + label moderation.Action + request moderation.RetractContentLabelRequest +} + +func newRetractLabelRulesScenario(t *testing.T) retractLabelRulesScenario { + t.Helper() + scenario, labelRequest := newLabelRulesScenario() + labelRequest.Reason = removeRulesSpam + labelled, err := scenario.service.LabelContent(t.Context(), removeRulesAdminDID, labelRequest) + require.NoError(t, err) + require.NotNil(t, labelled) + require.NotNil(t, labelled.Action) + scenario.store.writeCalls = nil + return retractLabelRulesScenario{ + removeRulesScenario: scenario, + label: *labelled.Action, + request: moderation.RetractContentLabelRequest{ + ActionID: labelled.Action.ID, ReviewedSubject: &moderation.StrongRef{URI: postRulesURI, CID: postRulesCID}, + ExpectedVersion: labelled.State.Version, IdempotencyKey: "first-retraction", + PrivateNote: "retraction note", + }, + } +} + +func assertRetractLabelRejected(t *testing.T, scenario retractLabelRulesScenario, want error) { + t.Helper() + before := scenario.store.state.copy() + result, err := scenario.service.RetractContentLabel(t.Context(), restoreRulesAdminDID, scenario.request) + require.ErrorIs(t, err, want) + assert.Nil(t, result) + labelRulesNoWrites(t, scenario.removeRulesScenario, before) +} + +func TestRetractContentLabelValidatesRequestWithoutWrites(t *testing.T) { + for _, test := range []struct { + name string + change func(*moderation.RetractContentLabelRequest) + want error + }{ + {"missing action ID", func(request *moderation.RetractContentLabelRequest) { request.ActionID = "" }, moderation.ErrInvalidRequest}, + {"long action ID", func(request *moderation.RetractContentLabelRequest) { request.ActionID = strings.Repeat("a", 129) }, moderation.ErrInvalidRequest}, + {"missing key", func(request *moderation.RetractContentLabelRequest) { request.IdempotencyKey = "" }, moderation.ErrInvalidRequest}, + {"long key", func(request *moderation.RetractContentLabelRequest) { + request.IdempotencyKey = strings.Repeat("k", 129) + }, moderation.ErrInvalidRequest}, + {"missing version", func(request *moderation.RetractContentLabelRequest) { request.ExpectedVersion = "" }, moderation.ErrInvalidRequest}, + {"long version", func(request *moderation.RetractContentLabelRequest) { + request.ExpectedVersion = strings.Repeat("v", 129) + }, moderation.ErrInvalidRequest}, + {"long reason", func(request *moderation.RetractContentLabelRequest) { request.Reason = strings.Repeat("r", 641) }, moderation.ErrInvalidRequest}, + {"invalid UTF-8 reason", func(request *moderation.RetractContentLabelRequest) { request.Reason = string([]byte{0xff}) }, moderation.ErrInvalidRequest}, + {"unknown reason", func(request *moderation.RetractContentLabelRequest) { + request.Reason = "social.coves.moderation.defs#reasonUnknown" + }, moderation.ErrUnsupportedReason}, + {"640 byte unknown reason", func(request *moderation.RetractContentLabelRequest) { + request.Reason = strings.Repeat("r", 640) + }, moderation.ErrUnsupportedReason}, + {"doxing reason is only for removal", func(request *moderation.RetractContentLabelRequest) { + request.Reason = "social.coves.moderation.defs#reasonDoxing" + }, moderation.ErrUnsupportedReason}, + {"illegal-content reason is only for removal", func(request *moderation.RetractContentLabelRequest) { + request.Reason = removeRulesIllegal + }, moderation.ErrUnsupportedReason}, + {"long private note", func(request *moderation.RetractContentLabelRequest) { + request.PrivateNote = strings.Repeat("n", 10001) + }, moderation.ErrInvalidRequest}, + {"malformed reviewed URI", func(request *moderation.RetractContentLabelRequest) { + request.ReviewedSubject.URI = "not an at URI" + }, moderation.ErrInvalidRequest}, + {"malformed reviewed CID", func(request *moderation.RetractContentLabelRequest) { + request.ReviewedSubject.CID = "not a CID" + }, moderation.ErrInvalidRequest}, + } { + t.Run(test.name, func(t *testing.T) { + scenario := newRetractLabelRulesScenario(t) + test.change(&scenario.request) + assertRetractLabelRejected(t, scenario, test.want) + }) + } +} + +func TestRetractContentLabelRejectsUnknownAndInvalidDecisionWithoutWrites(t *testing.T) { + for _, test := range []struct { + name string + change func(*retractLabelRulesScenario) + want error + }{ + {"unknown action", func(scenario *retractLabelRulesScenario) { + scenario.request.ActionID = "missing-action" + }, moderation.ErrDecisionNotFound}, + {"remove action", func(scenario *retractLabelRulesScenario) { + action := scenario.label + action.ID, action.Action = "removal-action", moderation.ActionRemove + scenario.store.state.actions[action.ID] = action + scenario.request.ActionID = action.ID + }, moderation.ErrInvalidDecision}, + {"restore action", func(scenario *retractLabelRulesScenario) { + action := scenario.label + action.ID, action.Action = "restore-action", moderation.ActionRestore + scenario.store.state.actions[action.ID] = action + scenario.request.ActionID = action.ID + }, moderation.ErrInvalidDecision}, + {"retract-label action", func(scenario *retractLabelRulesScenario) { + action := scenario.label + action.ID, action.Action = "other-retraction", moderation.ActionRetractLabel + scenario.store.state.actions[action.ID] = action + scenario.request.ActionID = action.ID + }, moderation.ErrInvalidDecision}, + {"foreign authority label", func(scenario *retractLabelRulesScenario) { + action := scenario.label + action.ID, action.AuthorityDID = "foreign-label", "did:web:foreign.test" + scenario.store.seedActiveLabel(action) + scenario.request.ActionID = action.ID + }, moderation.ErrInvalidDecision}, + {"inherited label", func(scenario *retractLabelRulesScenario) { + action := scenario.label + action.ID, action.Origin = "inherited-label", "inherited" + scenario.store.seedActiveLabel(action) + scenario.request.ActionID = action.ID + }, moderation.ErrInvalidDecision}, + {"previously retracted label", func(scenario *retractLabelRulesScenario) { + key := inMemoryModerationLabelKey{removeRulesInstanceDID, postRulesURI, moderation.LabelNSFW} + scenario.store.state.labelDecisions[key] = inMemoryModerationLabelDecision{actionID: scenario.label.ID, active: false} + scenario.store.state.versions[postRulesURI] = 2 + scenario.request.ExpectedVersion = "v2" + scenario.store.state.actions["prior-retraction"] = moderation.Action{ + ID: "prior-retraction", Action: moderation.ActionRetractLabel, ReversesActionID: scenario.label.ID, + AuthorityDID: removeRulesInstanceDID, SubjectURI: postRulesURI, + } + }, moderation.ErrInvalidDecision}, + } { + t.Run(test.name, func(t *testing.T) { + scenario := newRetractLabelRulesScenario(t) + test.change(&scenario) + assertRetractLabelRejected(t, scenario, test.want) + }) + } +} + +func TestRetractContentLabelRejectsSupersededLabelWithoutWrites(t *testing.T) { + scenario := newRetractLabelRulesScenario(t) + first := scenario.label + key := inMemoryModerationLabelKey{removeRulesInstanceDID, postRulesURI, moderation.LabelNSFW} + scenario.store.state.labelDecisions[key] = inMemoryModerationLabelDecision{actionID: first.ID, active: false} + scenario.store.state.actions["first-retraction"] = moderation.Action{ + ID: "first-retraction", Action: moderation.ActionRetractLabel, ReversesActionID: first.ID, + AuthorityDID: removeRulesInstanceDID, SubjectURI: postRulesURI, + } + scenario.store.state.versions[postRulesURI] = 2 + secondRequest := moderation.LabelContentRequest{ + Subject: scenario.removeRulesScenario.request.Subject, LabelValue: moderation.LabelNSFW, + ExpectedVersion: "v2", IdempotencyKey: "second-apply", + } + second, err := scenario.service.LabelContent(t.Context(), removeRulesAdminDID, secondRequest) + require.NoError(t, err) + require.NotNil(t, second) + require.NotNil(t, second.Action) + require.NotEqual(t, first.ID, second.Action.ID) + scenario.store.writeCalls = nil + scenario.request.ExpectedVersion = second.State.Version + assertRetractLabelRejected(t, scenario, moderation.ErrInvalidDecision) + assert.Equal(t, second.Action.ID, scenario.store.state.labelDecisions[key].actionID) +} + +func TestRetractContentLabelRequiresCurrentReviewAndVersion(t *testing.T) { + for _, test := range []struct { + name string + change func(*retractLabelRulesScenario) + want error + }{ + {"present post requires reviewed subject", func(scenario *retractLabelRulesScenario) { + scenario.request.ReviewedSubject = nil + }, moderation.ErrInvalidRequest}, + {"reviewed URI differs", func(scenario *retractLabelRulesScenario) { + scenario.request.ReviewedSubject.URI = "at://did:plc:postauthor/social.coves.community.postv2/3kother" + }, moderation.ErrInvalidRequest}, + {"indexed CID differs", func(scenario *retractLabelRulesScenario) { + post := scenario.store.state.indexedPosts[postRulesURI] + post.CID = "bafyreinewpostversion" + scenario.store.state.indexedPosts[postRulesURI] = post + scenario.reader.record.CID = post.CID + }, moderation.ErrContentChanged}, + {"stale version", func(scenario *retractLabelRulesScenario) { + scenario.request.ExpectedVersion = "v0" + }, moderation.ErrStateConflict}, + } { + t.Run(test.name, func(t *testing.T) { + scenario := newRetractLabelRulesScenario(t) + test.change(&scenario) + assertRetractLabelRejected(t, scenario, test.want) + }) + } +} + +func TestRetractContentLabelAppliesWithOptionalReason(t *testing.T) { + for _, reason := range []string{ + "", removeRulesSpam, "social.coves.moderation.defs#reasonHarassment", + "social.coves.moderation.defs#reasonRuleViolation", + "social.coves.moderation.defs#reasonModeratorDiscretion", + } { + t.Run("reason "+reason, func(t *testing.T) { + scenario := newRetractLabelRulesScenario(t) + scenario.request.Reason = reason + result, err := scenario.service.RetractContentLabel(t.Context(), restoreRulesAdminDID, scenario.request) + require.NoError(t, err) + require.NotNil(t, result, "retraction must return a mutation result") + require.Equal(t, moderation.OutcomeApplied, result.Outcome) + require.NotNil(t, result.Action) + action := result.Action + require.NotEmpty(t, action.ID) + assert.NotEqual(t, scenario.label.ID, action.ID) + assert.Equal(t, moderation.ActionRetractLabel, action.Action) + assert.Equal(t, moderation.LabelNSFW, action.LabelValue) + assert.Equal(t, scenario.label.ID, action.ReversesActionID) + assert.Equal(t, scenario.label.Reason, action.ReversedActionReason) + assert.Equal(t, reason, action.Reason) + assert.Equal(t, scenario.request.PrivateNote, action.PrivateNote) + assert.Equal(t, postRulesCID, action.ObservedCID) + assert.Equal(t, restoreRulesAdminDID, action.ActorDID) + assert.Equal(t, removeRulesInstanceDID, action.AuthorityDID) + assert.Equal(t, moderation.ScopeInstance, action.ScopeKind) + assert.Equal(t, moderation.OriginLocal, action.Origin) + assert.Equal(t, postRulesURI, action.SubjectURI) + assert.Equal(t, moderation.PostV2Collection, action.SubjectCollection) + assert.Equal(t, postRulesCommunityDID, action.SubjectCommunityDID) + assert.Equal(t, "v2", result.State.Version) + assert.Equal(t, moderation.RecordStatePresent, result.State.RecordState) + assert.Empty(t, result.State.LocalLabels) + assert.Equal(t, moderation.ModerationView{State: moderation.ModerationStateClear}, result.State.Moderation) + require.Len(t, scenario.store.state.actions, 2) + assert.Equal(t, scenario.label, scenario.store.state.actions[scenario.label.ID]) + assert.Equal(t, *action, scenario.store.state.actions[action.ID]) + key := inMemoryModerationLabelKey{removeRulesInstanceDID, postRulesURI, moderation.LabelNSFW} + assert.Equal(t, inMemoryModerationLabelDecision{actionID: scenario.label.ID, active: false}, scenario.store.state.labelDecisions[key]) + require.NoError(t, scenario.store.InTransaction(t.Context(), func(ctx context.Context, transaction moderation.Transaction) error { + labels, err := transaction.ActiveLabels(ctx, removeRulesInstanceDID, postRulesURI) + assert.Empty(t, labels) + return err + })) + assert.Equal(t, int64(2), scenario.store.state.versions[postRulesURI]) + assert.Equal(t, []string{"InsertAction", "SetLabelDecision", "SetSubjectVersion", "SaveIdempotencyRecord"}, scenario.store.writeCalls) + assert.Empty(t, scenario.store.state.mediaBlocks) + assert.Empty(t, scenario.purger.ownerPurges) + assert.Empty(t, scenario.purger.blobPurges) + }) + } +} + +func TestRetractContentLabelAppliesWithoutReviewForDeletedOrUnavailablePost(t *testing.T) { + for _, test := range []struct { + name string + change func(*retractLabelRulesScenario) + want moderation.RecordState + }{ + {"author deleted", func(scenario *retractLabelRulesScenario) { + post := scenario.store.state.indexedPosts[postRulesURI] + post.AuthorDeleted = true + scenario.store.state.indexedPosts[postRulesURI] = post + scenario.reader.record.Deleted = true + }, moderation.RecordStateDeleted}, + {"never indexed with stored label", func(scenario *retractLabelRulesScenario) { + delete(scenario.store.state.indexedPosts, postRulesURI) + scenario.reader.record = nil + scenario.reader.err = moderation.ErrSubjectNotIndexed + }, moderation.RecordStateUnavailable}, + } { + t.Run(test.name, func(t *testing.T) { + var scenario retractLabelRulesScenario + if test.want == moderation.RecordStateUnavailable { + base, _ := newLabelRulesScenario() + scenario.removeRulesScenario = base + scenario.label = labelRulesAction("stored-label", postRulesURI) + scenario.label.Reason = removeRulesSpam + scenario.store.seedActiveLabel(scenario.label) + scenario.store.state.versions[postRulesURI] = 1 + scenario.request = moderation.RetractContentLabelRequest{ + ActionID: scenario.label.ID, ExpectedVersion: "v1", IdempotencyKey: "retract-unindexed", + } + } else { + scenario = newRetractLabelRulesScenario(t) + } + test.change(&scenario) + scenario.request.ReviewedSubject = nil + scenario.store.writeCalls = nil + result, err := scenario.service.RetractContentLabel(t.Context(), restoreRulesAdminDID, scenario.request) + require.NoError(t, err) + require.NotNil(t, result) + require.Equal(t, moderation.OutcomeApplied, result.Outcome) + require.NotNil(t, result.Action) + assert.Equal(t, test.want, result.State.RecordState) + assert.Nil(t, result.State.CurrentSubject) + assert.Empty(t, result.State.LocalLabels) + assert.Equal(t, moderation.ModerationStateClear, result.State.Moderation.State) + assert.Equal(t, "v2", result.State.Version) + assert.False(t, scenario.store.state.labelDecisions[inMemoryModerationLabelKey{removeRulesInstanceDID, postRulesURI, moderation.LabelNSFW}].active) + }) + } +} + +func TestRetractContentLabelPreservesRemoval(t *testing.T) { + scenario := newRetractLabelRulesScenario(t) + removeRequest := scenario.removeRulesScenario.request + removeRequest.ExpectedVersion = "v1" + removeRequest.IdempotencyKey = "remove-after-label" + removed, err := scenario.service.RemoveContent(t.Context(), removeRulesAdminDID, removeRequest) + require.NoError(t, err) + require.NotNil(t, removed) + require.NotNil(t, removed.Action) + scenario.store.writeCalls = nil + scenario.request.ExpectedVersion = removed.State.Version + result, err := scenario.service.RetractContentLabel(t.Context(), restoreRulesAdminDID, scenario.request) + require.NoError(t, err) + require.NotNil(t, result) + require.Equal(t, moderation.OutcomeApplied, result.Outcome) + assert.Equal(t, "v3", result.State.Version) + assert.Equal(t, moderation.ModerationStateRemoved, result.State.Moderation.State) + assert.Equal(t, &moderation.ActionRef{ServiceDID: removeRulesInstanceDID, ActionID: removed.Action.ID}, result.State.LocalRemoval) + assert.Empty(t, result.State.LocalLabels) + assert.Empty(t, result.State.Moderation.ContentLabels) + assert.Equal(t, removed.Action.ID, scenario.store.state.activeRemovals[inMemoryModerationDecisionKey{removeRulesInstanceDID, postRulesURI}]) +} + +func TestRetractContentLabelIdempotencyReplayAndConflicts(t *testing.T) { + scenario := newRetractLabelRulesScenario(t) + original, err := scenario.service.RetractContentLabel(t.Context(), restoreRulesAdminDID, scenario.request) + require.NoError(t, err) + require.NotNil(t, original) + require.NotNil(t, original.Action) + replayed, err := assertIdempotencyNoMutation(t, scenario.removeRulesScenario, func() (*moderation.MutationResult, error) { + return scenario.service.RetractContentLabel(t.Context(), restoreRulesAdminDID, scenario.request) + }) + require.NoError(t, err) + assert.Equal(t, original, replayed) + for _, test := range []struct { + name string + change func(*moderation.RetractContentLabelRequest) + }{ + {"different action ID", func(request *moderation.RetractContentLabelRequest) { request.ActionID = "other-action" }}, + {"different reason", func(request *moderation.RetractContentLabelRequest) { + request.Reason = "social.coves.moderation.defs#reasonHarassment" + }}, + } { + t.Run(test.name, func(t *testing.T) { + changed := scenario.request + test.change(&changed) + result, err := assertIdempotencyNoMutation(t, scenario.removeRulesScenario, func() (*moderation.MutationResult, error) { + return scenario.service.RetractContentLabel(t.Context(), restoreRulesAdminDID, changed) + }) + require.ErrorIs(t, err, moderation.ErrIdempotencyConflict) + assert.Nil(t, result) + }) + } + assert.Len(t, scenario.store.state.actions, 2) +} + +func TestRetractContentLabelStoreFailureRollsBackEverything(t *testing.T) { + for _, test := range []struct { + name string + inject func(*inMemoryModerationStore, error) + wantCalls []string + }{ + {"SetLabelDecision failure rolls back the inserted action", func(store *inMemoryModerationStore, err error) { + store.failSetLabelDecision = err + }, []string{"InsertAction", "SetLabelDecision"}}, + {"label read failure rolls back the action, decision and version", func(store *inMemoryModerationStore, err error) { + store.failActiveLabelsAfterLabelDecision = err + }, []string{"InsertAction", "SetLabelDecision", "SetSubjectVersion"}}, + } { + t.Run(test.name, func(t *testing.T) { + scenario := newRetractLabelRulesScenario(t) + storageError := errors.New("injected store failure") + test.inject(scenario.store, storageError) + before := scenario.store.state.copy() + result, err := scenario.service.RetractContentLabel(t.Context(), restoreRulesAdminDID, scenario.request) + require.ErrorIs(t, err, moderation.ErrModerationUnavailable) + assert.ErrorIs(t, err, storageError) + assert.Nil(t, result) + assert.Equal(t, test.wantCalls, scenario.store.writeCalls) + assert.Equal(t, before, scenario.store.state, "a failed retraction must persist no action, decision change, version or idempotency key") + assert.True(t, scenario.store.state.labelDecisions[inMemoryModerationLabelKey{removeRulesInstanceDID, postRulesURI, moderation.LabelNSFW}].active) + }) + } +} + +// The retraction target check rejects any label action the instance did not +// establish locally at instance scope, even if the stored instance decision +// points at it. +func TestRetractContentLabelRejectsNonLocalInstanceLabelBehindInstanceDecision(t *testing.T) { + for _, test := range []struct { + name string + change func(*moderation.Action) + }{ + {"foreign authority", func(action *moderation.Action) { action.AuthorityDID = "did:web:foreign.test" }}, + {"community scope", func(action *moderation.Action) { + action.ScopeKind, action.ScopeCommunityDID = "community", postRulesCommunityDID + }}, + {"inherited origin", func(action *moderation.Action) { action.Origin = "inherited" }}, + } { + t.Run(test.name, func(t *testing.T) { + scenario := newRetractLabelRulesScenario(t) + action := scenario.label + action.ID = "non-local-label" + test.change(&action) + scenario.store.state.actions[action.ID] = action + key := inMemoryModerationLabelKey{removeRulesInstanceDID, postRulesURI, moderation.LabelNSFW} + scenario.store.state.labelDecisions[key] = inMemoryModerationLabelDecision{actionID: action.ID, active: true} + scenario.request.ActionID = action.ID + assertRetractLabelRejected(t, scenario, moderation.ErrInvalidDecision) + }) + } +} diff --git a/internal/core/moderation/service.go b/internal/core/moderation/service.go index 91481b2..270a236 100644 --- a/internal/core/moderation/service.go +++ b/internal/core/moderation/service.go @@ -27,6 +27,14 @@ func (s *service) RestoreContent(ctx context.Context, actorDID string, request R return s.restoreContent(ctx, actorDID, request) } +func (s *service) LabelContent(ctx context.Context, actorDID string, request LabelContentRequest) (*MutationResult, error) { + return s.labelContent(ctx, actorDID, request) +} + +func (s *service) RetractContentLabel(ctx context.Context, actorDID string, request RetractContentLabelRequest) (*MutationResult, error) { + return s.retractContentLabel(ctx, actorDID, request) +} + func (s *service) GetSubjectState(ctx context.Context, subject string) (*SubjectState, error) { uri, err := syntax.ParseATURI(subject) if err != nil || !uri.Authority().IsDID() || !IsSubjectCollection(uri.Collection().String()) || uri.RecordKey().String() == "" { @@ -57,7 +65,7 @@ func (s *service) GetSubjectState(ctx context.Context, subject string) (*Subject current = &StrongRef{URI: record.URI, CID: record.CID} } } - state, err := newSubjectState(subject, stored.Version, recordState, current, stored.ActiveRemoval, s.config.InstanceDID) + state, err := newSubjectState(subject, stored.Version, recordState, current, stored.ActiveRemoval, stored.ActiveLabels, s.config.InstanceDID) if err != nil { return nil, err } diff --git a/internal/core/moderation/store.go b/internal/core/moderation/store.go index e6c9293..d274574 100644 --- a/internal/core/moderation/store.go +++ b/internal/core/moderation/store.go @@ -7,8 +7,14 @@ import ( // Action kinds, scope kinds and origins recorded on moderation actions. const ( - ActionRemove = "remove" - ActionRestore = "restore" + ActionRemove = "remove" + ActionRestore = "restore" + ActionLabel = "label" + ActionRetractLabel = "retract-label" + + // LabelNSFW is the only content label value the local classification + // procedures accept. + LabelNSFW = "nsfw" ScopeInstance = "instance" @@ -100,6 +106,7 @@ type IdempotencyRecord struct { type SubjectModeration struct { Version int64 ActiveRemoval *Action + ActiveLabels []Action } // Store persists moderation state. Mutations run inside InTransaction. @@ -108,7 +115,8 @@ type Store interface { SubjectModeration(ctx context.Context, authorityDID, subjectURI string) (*SubjectModeration, error) // ListActions returns up to query.Limit actions, newest first, each with // ReversedActionReason populated. Under ExcludeHidden it must omit every - // hidden action; the service fails closed if one comes back. + // hidden action, and under ExcludeLabelActions every action whose kind is + // in PublicExcludedActions; the service fails closed if either comes back. ListActions(ctx context.Context, query ActionListQuery) ([]Action, error) } @@ -129,6 +137,8 @@ type ActionListQuery struct { Since *time.Time Until *time.Time ExcludeHidden bool + // ExcludeLabelActions omits every action kind in PublicExcludedActions. + ExcludeLabelActions bool } // ActionKey is an action log position: rows strictly older than it follow. @@ -156,8 +166,17 @@ type Transaction interface { GetAction(ctx context.Context, actionID string) (*Action, error) // ActiveRemoval returns the active removal action, or nil. ActiveRemoval(ctx context.Context, authorityDID, subjectURI string) (*Action, error) + // ActiveLabels returns active instance-scope label actions, ordered by value. + ActiveLabels(ctx context.Context, authorityDID, subjectURI string) ([]Action, error) InsertAction(ctx context.Context, action Action) (*Action, error) SetRemovalDecision(ctx context.Context, authorityDID, subjectURI, actionID string, active bool) error + // SetLabelDecision changes the instance-scope label decision for + // (authorityDID, subjectURI, value). With active true it activates the + // decision keyed to actionID, inserting it or reactivating an inactive row, + // and fails if the decision is already active. With active false it + // deactivates exactly one active decision whose action is actionID, and + // fails if there is none. + SetLabelDecision(ctx context.Context, authorityDID, subjectURI, value, actionID string, active bool) error SetSubjectVersion(ctx context.Context, subjectURI string, version int64) error InsertMediaBlocks(ctx context.Context, blocks []MediaBlock) error DeactivateMediaBlocks(ctx context.Context, actionID string) error diff --git a/internal/core/moderation/subject_state.go b/internal/core/moderation/subject_state.go index cf974f1..f41c1a9 100644 --- a/internal/core/moderation/subject_state.go +++ b/internal/core/moderation/subject_state.go @@ -4,7 +4,7 @@ import "fmt" // newSubjectState constructs a state with a current strong reference exactly // when the indexed record is present. -func newSubjectState(subject string, version int64, recordState RecordState, current *StrongRef, activeRemoval *Action, authorityDID string) (SubjectState, error) { +func newSubjectState(subject string, version int64, recordState RecordState, current *StrongRef, activeRemoval *Action, activeLabels []Action, authorityDID string) (SubjectState, error) { if (recordState == RecordStatePresent) != (current != nil) || (recordState != RecordStatePresent && recordState != RecordStateDeleted && recordState != RecordStateUnavailable) || version < 0 { return SubjectState{}, fmt.Errorf("%w: inconsistent subject state", ErrModerationUnavailable) @@ -20,5 +20,14 @@ func newSubjectState(subject string, version int64, recordState RecordState, cur state.Moderation.State = ModerationStateRemoved state.LocalRemoval = &ActionRef{ServiceDID: authorityDID, ActionID: activeRemoval.ID} } + for _, label := range activeLabels { + state.LocalLabels = append(state.LocalLabels, LocalLabel{ + Value: label.LabelValue, Action: ActionRef{ServiceDID: authorityDID, ActionID: label.ID}, + }) + state.Moderation.ContentLabels = append(state.Moderation.ContentLabels, ContentLabel{ + Value: label.LabelValue, + Sources: []DecisionSource{{AuthorityDID: authorityDID, ScopeKind: ScopeInstance}}, + }) + } return state, nil } diff --git a/internal/core/moderation/subject_state_labels_test.go b/internal/core/moderation/subject_state_labels_test.go new file mode 100644 index 0000000..4c555a6 --- /dev/null +++ b/internal/core/moderation/subject_state_labels_test.go @@ -0,0 +1,80 @@ +package moderation_test + +import ( + "testing" + + "Coves/internal/core/moderation" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func assertSubjectStateLocalLabel(t *testing.T, state moderation.SubjectState, actionID, removalState string) { + t.Helper() + assert.Equal(t, removalState, state.Moderation.State) + assert.Equal(t, []moderation.LocalLabel{{ + Value: moderation.LabelNSFW, Action: moderation.ActionRef{ServiceDID: removeRulesInstanceDID, ActionID: actionID}, + }}, state.LocalLabels) + assert.Equal(t, []moderation.ContentLabel{{ + Value: moderation.LabelNSFW, + Sources: []moderation.DecisionSource{{AuthorityDID: removeRulesInstanceDID, ScopeKind: moderation.ScopeInstance}}, + }}, state.Moderation.ContentLabels) +} + +func TestSubjectStateListsActiveLabelsAndClearsThemAfterRetract(t *testing.T) { + scenario := newRetractLabelRulesScenario(t) + stored, err := scenario.store.SubjectModeration(t.Context(), removeRulesInstanceDID, postRulesURI) + require.NoError(t, err) + require.NotNil(t, stored) + require.Equal(t, []moderation.Action{scenario.label}, stored.ActiveLabels) + state, err := scenario.service.GetSubjectState(t.Context(), postRulesURI) + require.NoError(t, err) + require.NotNil(t, state) + assert.Equal(t, "v1", state.Version) + assertSubjectStateLocalLabel(t, *state, scenario.label.ID, moderation.ModerationStateClear) + retracted, err := scenario.service.RetractContentLabel(t.Context(), restoreRulesAdminDID, scenario.request) + require.NoError(t, err) + require.NotNil(t, retracted) + state, err = scenario.service.GetSubjectState(t.Context(), postRulesURI) + require.NoError(t, err) + require.NotNil(t, state) + assert.Equal(t, "v2", state.Version) + assert.Empty(t, state.LocalLabels) + assert.Equal(t, moderation.ModerationView{State: moderation.ModerationStateClear}, state.Moderation) +} + +func TestSubjectStateRemovalAndRestorePreserveActiveLabel(t *testing.T) { + scenario := newRetractLabelRulesScenario(t) + request := scenario.removeRulesScenario.request + request.ExpectedVersion = "v1" + request.IdempotencyKey = "remove-labelled-post" + removed, err := scenario.service.RemoveContent(t.Context(), removeRulesAdminDID, request) + require.NoError(t, err) + require.NotNil(t, removed) + require.NotNil(t, removed.Action) + assertSubjectStateLocalLabel(t, removed.State, scenario.label.ID, moderation.ModerationStateRemoved) + assert.Equal(t, &moderation.ActionRef{ServiceDID: removeRulesInstanceDID, ActionID: removed.Action.ID}, removed.State.LocalRemoval) + + redundant := request + redundant.ExpectedVersion = removed.State.Version + redundant.IdempotencyKey = "redundant-labelled-removal" + unchanged, err := scenario.service.RemoveContent(t.Context(), restoreRulesAdminDID, redundant) + require.NoError(t, err) + require.NotNil(t, unchanged) + assert.Equal(t, moderation.OutcomeUnchanged, unchanged.Outcome) + assert.Nil(t, unchanged.Action) + assert.Equal(t, removed.State.Version, unchanged.State.Version) + assertSubjectStateLocalLabel(t, unchanged.State, scenario.label.ID, moderation.ModerationStateRemoved) + assert.Equal(t, removed.State.LocalRemoval, unchanged.State.LocalRemoval) + + restored, err := scenario.service.RestoreContent(t.Context(), restoreRulesAdminDID, moderation.RestoreContentRequest{ + ActionID: removed.Action.ID, ReviewedSubject: &moderation.StrongRef{URI: postRulesURI, CID: postRulesCID}, + ExpectedVersion: removed.State.Version, IdempotencyKey: "restore-labelled-post", Reason: removeRulesSpam, + }) + require.NoError(t, err) + require.NotNil(t, restored) + assert.Equal(t, moderation.OutcomeApplied, restored.Outcome) + assert.Equal(t, "v3", restored.State.Version) + assert.Nil(t, restored.State.LocalRemoval) + assertSubjectStateLocalLabel(t, restored.State, scenario.label.ID, moderation.ModerationStateClear) +} diff --git a/internal/core/moderation/types.go b/internal/core/moderation/types.go index 5b35ca5..5182c23 100644 --- a/internal/core/moderation/types.go +++ b/internal/core/moderation/types.go @@ -56,9 +56,23 @@ type LocalLabel struct { Action ActionRef } -// ModerationView is the effective public removal state of a subject. +// ModerationView is the effective public moderation of a subject: its removal +// state and its active content labels. type ModerationView struct { - State string + State string + ContentLabels []ContentLabel +} + +// ContentLabel is one active classification value with the sources applying it. +type ContentLabel struct { + Value string + Sources []DecisionSource +} + +// DecisionSource attributes a decision to its authority and scope. +type DecisionSource struct { + AuthorityDID string + ScopeKind string } // SubjectState is the versioned moderation and repository state of a subject. @@ -79,8 +93,9 @@ type IndexedRecord struct { Deleted bool } -// MutationResult is the outcome of a removeContent/restoreContent call. Action -// is nil for an unchanged outcome. +// MutationResult is the outcome of a removeContent, restoreContent, +// labelContent or retractContentLabel call. Action is nil for an unchanged +// outcome. type MutationResult struct { Outcome string State SubjectState @@ -96,6 +111,30 @@ type RemoveContentRequest struct { PrivateNote string } +// LabelContentRequest is the caller-supplied part of a labelContent call. +// Reason is optional; the doxing and illegal-content reasons are rejected +// because they are only for removal. +type LabelContentRequest struct { + Subject StrongRef + LabelValue string + ExpectedVersion string + IdempotencyKey string + Reason string + PrivateNote string +} + +// RetractContentLabelRequest is the caller-supplied part of a retractContentLabel call. +// Reason is optional; the doxing and illegal-content reasons are rejected +// because they are only for removal. +type RetractContentLabelRequest struct { + ActionID string + ReviewedSubject *StrongRef + ExpectedVersion string + IdempotencyKey string + Reason string + PrivateNote string +} + // RestoreContentRequest is the caller-supplied part of a restoreContent call. type RestoreContentRequest struct { ActionID string diff --git a/internal/core/posts/post.go b/internal/core/posts/post.go index f22eefe..d28d15a 100644 --- a/internal/core/posts/post.go +++ b/internal/core/posts/post.go @@ -203,22 +203,40 @@ type RemovalSource struct { ScopeKind string } -// ModerationView is a post's public removal state +// ModerationView states served on the wire. +const ( + // ModerationViewStateClear is served with content labels. It is correct only + // on rows already filtered by visiblePostsPredicate, which drops removed posts. + ModerationViewStateClear = "clear" + // ModerationViewStateRemoved is served on a removal tombstone. + ModerationViewStateRemoved = "removed" +) + +// ModerationView is a post's public removal or content-label state // (social.coves.moderation.defs#moderationView). type ModerationView struct { - State string `json:"state"` + State string `json:"state"` + Sources []ModerationSourceView `json:"sources,omitempty"` + ContentLabels []ContentLabelView `json:"contentLabels,omitempty"` +} + +// ContentLabelView is one active moderator classification value +// (social.coves.moderation.defs#contentLabelView). +type ContentLabelView struct { + Value string `json:"value"` Sources []ModerationSourceView `json:"sources,omitempty"` } -// ModerationSourceView attributes a removal (social.coves.moderation.defs#sourceView). +// ModerationSourceView attributes a removal or label (social.coves.moderation.defs#sourceView). type ModerationSourceView struct { AuthorityDID string `json:"authorityDid"` Scope ModerationScopeView `json:"scope"` } -// ModerationScopeView is a removal's scope (social.coves.moderation.defs#scopeView). +// ModerationScopeView is a removal or label's scope (social.coves.moderation.defs#scopeView). type ModerationScopeView struct { - Kind string `json:"kind"` + Kind string `json:"kind"` + CommunityDID string `json:"communityDid,omitempty"` } // ModeratedPost is the content-free social.coves.community.post.defs#moderatedPost @@ -281,7 +299,7 @@ func moderatedResult(post *Post, sources []RemovalSource) *PostResult { } result := &ModeratedPost{ URI: post.URI, AuthorDID: post.AuthorDID, - Moderation: &ModerationView{State: "removed", Sources: viewSources}, + Moderation: &ModerationView{State: ModerationViewStateRemoved, Sources: viewSources}, } if post.CommunityDID != "" && post.CommunityName != "" { result.Community = &CommunityRef{DID: post.CommunityDID, Handle: post.CommunityHandle, Name: post.CommunityName} @@ -386,9 +404,11 @@ type PostView struct { Author *AuthorView `json:"author"` Stats *PostStats `json:"stats,omitempty"` Community *CommunityRef `json:"community"` - RKey string `json:"rkey"` - CID string `json:"cid"` - URI string `json:"uri"` + // Moderation carries active content labels on visible posts; absent when none apply. + Moderation *ModerationView `json:"moderation,omitempty"` + RKey string `json:"rkey"` + CID string `json:"cid"` + URI string `json:"uri"` // Status and AcceptanceURI are the per-community admission context (PRD §6.2), // populated from the visibility join. Both are additive-optional: a public diff --git a/internal/db/postgres/moderation_action_log_integration_test.go b/internal/db/postgres/moderation_action_log_integration_test.go index e5457b7..f1967ad 100644 --- a/internal/db/postgres/moderation_action_log_integration_test.go +++ b/internal/db/postgres/moderation_action_log_integration_test.go @@ -242,3 +242,34 @@ func TestModerationActionLogHiddenReasonsAndFullRowMapping(t *testing.T) { } assert.Equal(t, []string{"h"}, actionLogIDs(listActionLog(t, db, moderation.ActionListQuery{Limit: 20, ActionID: "h", CommunityDID: rows[7].SubjectCommunityDID}))) } + +// TestModerationActionLogExcludesLabelActions pins the SQL half of the public +// label exclusion: ExcludeLabelActions omits label and retract-label rows on +// every page and filter, and leaves every other kind in place. +func TestModerationActionLogExcludesLabelActions(t *testing.T) { + db := testkit.DB(t) + base := time.Date(2026, 9, 30, 12, 0, 0, 0, time.UTC) + removal := actionLogRow("a", base) + label := actionLogRow("b", base.Add(time.Microsecond)) + label.Action, label.LabelValue, label.Reason = moderation.ActionLabel, moderation.LabelNSFW, actionLogRule + retraction := actionLogRow("c", base.Add(2*time.Microsecond)) + retraction.Action, retraction.LabelValue, retraction.Reason = moderation.ActionRetractLabel, moderation.LabelNSFW, "" + retraction.ReversesActionID = label.ID + restore := actionLogRow("d", base.Add(3*time.Microsecond)) + restore.Action, restore.ReversesActionID, restore.Reason = moderation.ActionRestore, removal.ID, actionLogDiscretion + for _, row := range []moderation.Action{removal, label, retraction, restore} { + insertActionLogRow(t, db, row) + } + assert.Equal(t, []string{"d", "c", "b", "a"}, actionLogIDs(listActionLog(t, db, moderation.ActionListQuery{Limit: 20}))) + assert.Equal(t, []string{"d", "a"}, actionLogIDs(listActionLog(t, db, moderation.ActionListQuery{Limit: 20, ExcludeLabelActions: true}))) + assert.Equal(t, []string{"a"}, actionLogIDs(listActionLog(t, db, moderation.ActionListQuery{ + Limit: 20, ExcludeLabelActions: true, Before: &moderation.ActionKey{CreatedAt: restore.CreatedAt, ID: restore.ID}, + }))) + for _, kind := range moderation.PublicExcludedActions() { + assert.Empty(t, listActionLog(t, db, moderation.ActionListQuery{Limit: 20, Action: kind, ExcludeLabelActions: true}), kind) + assert.Len(t, listActionLog(t, db, moderation.ActionListQuery{Limit: 20, Action: kind}), 1, kind) + } + assert.Equal(t, []string{"d", "a"}, actionLogIDs(listActionLog(t, db, moderation.ActionListQuery{ + Limit: 20, ExcludeLabelActions: true, ExcludeHidden: true, SubjectCollection: moderation.CommentCollection, + }))) +} diff --git a/internal/db/postgres/moderation_label_integration_test.go b/internal/db/postgres/moderation_label_integration_test.go new file mode 100644 index 0000000..85fd8f5 --- /dev/null +++ b/internal/db/postgres/moderation_label_integration_test.go @@ -0,0 +1,267 @@ +//go:build integration + +package postgres_test + +import ( + "context" + "database/sql" + "errors" + "testing" + "time" + + "Coves/internal/core/moderation" + "Coves/internal/db/postgres" + "Coves/tests/fixtures" + "Coves/tests/testkit" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func indexedLabelPost(t *testing.T, db *sql.DB) moderation.StrongRef { + t.Helper() + communityDID, authorDID := moderationPostActors(t, db) + return indexedModerationPost(t, db, moderation.PostV2Collection, communityDID, authorDID, "label persistence", "") +} + +func labelPost(t *testing.T, service moderation.Service, actor string, subject moderation.StrongRef, version, key string) *moderation.MutationResult { + t.Helper() + result, err := service.LabelContent(t.Context(), actor, moderation.LabelContentRequest{ + Subject: subject, LabelValue: moderation.LabelNSFW, ExpectedVersion: version, IdempotencyKey: key, + }) + require.NoError(t, err) + require.NotNil(t, result) + require.Equal(t, moderation.OutcomeApplied, result.Outcome) + require.NotNil(t, result.Action) + return result +} + +func requirePersistedLabelState(t *testing.T, state moderation.SubjectState, id, version, status string) { + t.Helper() + assert.Equal(t, version, state.Version) + assert.Equal(t, status, state.Moderation.State) + require.Equal(t, []moderation.LocalLabel{{Value: moderation.LabelNSFW, Action: moderation.ActionRef{ + ServiceDID: fixtures.InstanceDID(), ActionID: id, + }}}, state.LocalLabels) + require.Equal(t, []moderation.ContentLabel{{Value: moderation.LabelNSFW, Sources: []moderation.DecisionSource{{ + AuthorityDID: fixtures.InstanceDID(), ScopeKind: moderation.ScopeInstance, + }}}}, state.Moderation.ContentLabels) +} + +func TestModerationLabelPostgresApplyPersistsOverlayAndReplay(t *testing.T) { + db := testkit.DB(t) + subject := indexedLabelPost(t, db) + service := newPostgresModerationService(db) + actor := fixtures.DID("labelpersistadmin") + var beforeRow, afterRow string + require.NoError(t, db.QueryRowContext(t.Context(), `SELECT row_to_json(p)::text FROM posts p WHERE uri = $1`, subject.URI).Scan(&beforeRow)) + views, err := postgres.NewPostRepository(db).GetViewsByURIs(t.Context(), []string{subject.URI}, "") + require.NoError(t, err) + require.Contains(t, views, subject.URI) + beforeRecord := views[subject.URI].Record + request := moderation.LabelContentRequest{Subject: subject, LabelValue: moderation.LabelNSFW, ExpectedVersion: "v0", IdempotencyKey: "label-postgres-apply"} + first, err := service.LabelContent(t.Context(), actor, request) + require.NoError(t, err) + require.NotNil(t, first) + require.Equal(t, moderation.OutcomeApplied, first.Outcome) + require.NotNil(t, first.Action) + id := first.Action.ID + requirePersistedLabelState(t, first.State, id, "v1", moderation.ModerationStateClear) + var actionCount int + var kind, value, observedCID, origin string + require.NoError(t, db.QueryRowContext(t.Context(), `SELECT action, label_value, observed_cid, origin FROM moderation_actions WHERE id = $1 AND subject_uri = $2`, id, subject.URI).Scan(&kind, &value, &observedCID, &origin)) + assert.Equal(t, moderation.ActionLabel, kind) + assert.Equal(t, moderation.LabelNSFW, value) + assert.Equal(t, subject.CID, observedCID) + assert.Equal(t, moderation.OriginLocal, origin) + assert.Equal(t, 1, countModerationActions(t, db, subject.URI, moderation.ActionLabel)) + var decisionKind, decisionValue, activeAction string + var active bool + require.NoError(t, db.QueryRowContext(t.Context(), `SELECT kind, value, active, active_action_id FROM moderation_decisions WHERE subject_uri = $1`, subject.URI).Scan(&decisionKind, &decisionValue, &active, &activeAction)) + assert.Equal(t, "label", decisionKind) + assert.Equal(t, moderation.LabelNSFW, decisionValue) + assert.True(t, active) + assert.Equal(t, id, activeAction) + var version int64 + require.NoError(t, db.QueryRowContext(t.Context(), `SELECT version FROM moderation_subjects WHERE subject_uri = $1`, subject.URI).Scan(&version)) + assert.EqualValues(t, 1, version) + state, err := service.GetSubjectState(t.Context(), subject.URI) + require.NoError(t, err) + require.NotNil(t, state) + requirePersistedLabelState(t, *state, id, "v1", moderation.ModerationStateClear) + require.NoError(t, db.QueryRowContext(t.Context(), `SELECT row_to_json(p)::text FROM posts p WHERE uri = $1`, subject.URI).Scan(&afterRow)) + assert.Equal(t, beforeRow, afterRow, "the entire indexed post row, including content, labels and CID, must be unchanged") + views, err = postgres.NewPostRepository(db).GetViewsByURIs(t.Context(), []string{subject.URI}, "") + require.NoError(t, err) + require.Contains(t, views, subject.URI) + assert.Equal(t, beforeRecord, views[subject.URI].Record, "the served record must also be unchanged") + + replayed, err := service.LabelContent(t.Context(), actor, request) + require.NoError(t, err) + assert.Equal(t, first, replayed, "Postgres idempotency must round-trip the full result and labels") + require.NoError(t, db.QueryRowContext(t.Context(), `SELECT count(*) FROM moderation_actions WHERE subject_uri = $1`, subject.URI).Scan(&actionCount)) + assert.Equal(t, 1, actionCount) +} + +func TestModerationLabelPostgresRetractAndReactivateSameDecision(t *testing.T) { + db := testkit.DB(t) + subject := indexedLabelPost(t, db) + service := newPostgresModerationService(db) + actor := fixtures.DID("labelcycleadmin") + first := labelPost(t, service, actor, subject, "v0", "cycle-first") + retracted, err := service.RetractContentLabel(t.Context(), actor, moderation.RetractContentLabelRequest{ + ActionID: first.Action.ID, ReviewedSubject: &subject, ExpectedVersion: "v1", IdempotencyKey: "cycle-retract", + }) + require.NoError(t, err) + require.NotNil(t, retracted) + require.Equal(t, moderation.OutcomeApplied, retracted.Outcome) + require.NotNil(t, retracted.Action) + assert.Equal(t, "v2", retracted.State.Version) + assert.Empty(t, retracted.State.LocalLabels) + assert.Empty(t, retracted.State.Moderation.ContentLabels) + var value, reverses string + require.NoError(t, db.QueryRowContext(t.Context(), `SELECT label_value, reverses_action_id FROM moderation_actions WHERE id = $1 AND action = 'retract-label'`, retracted.Action.ID).Scan(&value, &reverses)) + assert.Equal(t, moderation.LabelNSFW, value) + assert.Equal(t, first.Action.ID, reverses) + var active bool + var activeAction string + require.NoError(t, db.QueryRowContext(t.Context(), `SELECT active, active_action_id FROM moderation_decisions WHERE subject_uri = $1 AND kind = 'label' AND value = 'nsfw'`, subject.URI).Scan(&active, &activeAction)) + assert.False(t, active) + assert.Equal(t, first.Action.ID, activeAction) + state, err := service.GetSubjectState(t.Context(), subject.URI) + require.NoError(t, err) + require.NotNil(t, state) + assert.Equal(t, "v2", state.Version) + assert.Empty(t, state.LocalLabels) + assert.Empty(t, state.Moderation.ContentLabels) + second := labelPost(t, service, actor, subject, "v2", "cycle-second") + requirePersistedLabelState(t, second.State, second.Action.ID, "v3", moderation.ModerationStateClear) + var decisions int + require.NoError(t, db.QueryRowContext(t.Context(), `SELECT count(*) FROM moderation_decisions WHERE subject_uri = $1 AND kind = 'label' AND value = 'nsfw'`, subject.URI).Scan(&decisions)) + assert.Equal(t, 1, decisions, "re-apply must reuse the existing decision row") + require.NoError(t, db.QueryRowContext(t.Context(), `SELECT active, active_action_id FROM moderation_decisions WHERE subject_uri = $1 AND kind = 'label' AND value = 'nsfw'`, subject.URI).Scan(&active, &activeAction)) + assert.True(t, active) + assert.Equal(t, second.Action.ID, activeAction) + state, err = service.GetSubjectState(t.Context(), subject.URI) + require.NoError(t, err) + require.NotNil(t, state) + requirePersistedLabelState(t, *state, second.Action.ID, "v3", moderation.ModerationStateClear) + retry, err := service.RetractContentLabel(t.Context(), actor, moderation.RetractContentLabelRequest{ + ActionID: first.Action.ID, ReviewedSubject: &subject, ExpectedVersion: "v3", IdempotencyKey: "cycle-superseded", + }) + assert.ErrorIs(t, err, moderation.ErrInvalidDecision) + assert.Nil(t, retry) + assert.Equal(t, 1, countModerationActions(t, db, subject.URI, moderation.ActionRetractLabel)) +} + +func TestModerationLabelPostgresCoexistsWithRemoval(t *testing.T) { + db := testkit.DB(t) + subject := indexedLabelPost(t, db) + service := newPostgresModerationService(db) + actor := fixtures.DID("labeloverlayadmin") + label := labelPost(t, service, actor, subject, "v0", "overlay-label") + removed, err := service.RemoveContent(t.Context(), actor, moderation.RemoveContentRequest{ + Subject: subject, ExpectedVersion: "v1", IdempotencyKey: "overlay-remove", Reason: moderationConcurrencyReason, + }) + require.NoError(t, err) + require.NotNil(t, removed.Action) + state, err := service.GetSubjectState(t.Context(), subject.URI) + require.NoError(t, err) + require.NotNil(t, state) + requirePersistedLabelState(t, *state, label.Action.ID, "v2", moderation.ModerationStateRemoved) + require.NotNil(t, state.LocalRemoval) + assert.Equal(t, removed.Action.ID, state.LocalRemoval.ActionID) + restored, err := service.RestoreContent(t.Context(), actor, moderation.RestoreContentRequest{ + ActionID: removed.Action.ID, ReviewedSubject: &subject, ExpectedVersion: "v2", + IdempotencyKey: "overlay-restore", Reason: moderationConcurrencyReason, + }) + require.NoError(t, err) + require.NotNil(t, restored) + requirePersistedLabelState(t, restored.State, label.Action.ID, "v3", moderation.ModerationStateClear) + state, err = service.GetSubjectState(t.Context(), subject.URI) + require.NoError(t, err) + require.NotNil(t, state) + requirePersistedLabelState(t, *state, label.Action.ID, "v3", moderation.ModerationStateClear) + assert.Nil(t, state.LocalRemoval) +} + +func TestModerationLabelPostgresConcurrentAppliesRejectStaleVersion(t *testing.T) { + db := testkit.DB(t) + subject := indexedLabelPost(t, db) + service := newPostgresModerationService(db) + // Establish the subject row before either request so the second waiter is + // demonstrably blocked on LockSubject, not on a concurrent INSERT. + _, err := db.ExecContext(t.Context(), `INSERT INTO moderation_subjects (subject_uri, version) VALUES ($1, 0)`, subject.URI) + require.NoError(t, err) + connection, release := holdModerationActionInsert(t, db) + ctx, cancel := context.WithTimeout(t.Context(), 10*time.Second) + defer cancel() + firstAdmin, secondAdmin := fixtures.DID("labelracefirst"), fixtures.DID("labelracesecond") + require.NotEqual(t, firstAdmin, secondAdmin) + results := startConcurrentModerationCalls( + func() (*moderation.MutationResult, error) { + return service.LabelContent(ctx, firstAdmin, moderation.LabelContentRequest{Subject: subject, LabelValue: moderation.LabelNSFW, ExpectedVersion: "v0", IdempotencyKey: "race-first"}) + }, + func() (*moderation.MutationResult, error) { + return service.LabelContent(ctx, secondAdmin, moderation.LabelContentRequest{Subject: subject, LabelValue: moderation.LabelNSFW, ExpectedVersion: "v0", IdempotencyKey: "race-second"}) + }, + ) + waitForSubjectLockContention(t, connection) + release() + var applied, conflicts int + for range 2 { + outcome := <-results + if outcome.err != nil { + assert.ErrorIs(t, outcome.err, moderation.ErrStateConflict) + assert.Nil(t, outcome.result) + if errors.Is(outcome.err, moderation.ErrStateConflict) { + conflicts++ + } + continue + } + require.NotNil(t, outcome.result) + assert.Equal(t, moderation.OutcomeApplied, outcome.result.Outcome) + if outcome.result.Outcome == moderation.OutcomeApplied { + applied++ + } + } + assert.Equal(t, 1, applied) + assert.Equal(t, 1, conflicts) + assert.Equal(t, 1, countModerationActions(t, db, subject.URI, moderation.ActionLabel), "exactly one racing apply") +} + +// Activating a label decision that is already active must fail closed and +// leave the active decision pointing at its original action. +func TestModerationLabelPostgresActivateRejectsAlreadyActiveDecision(t *testing.T) { + db := testkit.DB(t) + subject := indexedLabelPost(t, db) + service := newPostgresModerationService(db) + actor := fixtures.DID("labeldoubleadmin") + first := labelPost(t, service, actor, subject, "v0", "double-activate") + var activateErr error + var secondID string + require.NoError(t, postgres.NewModerationRepository(db).InTransaction(t.Context(), func(ctx context.Context, tx moderation.Transaction) error { + second, err := tx.InsertAction(ctx, moderation.Action{ + ActorDID: actor, AuthorityDID: fixtures.InstanceDID(), ScopeKind: moderation.ScopeInstance, + SubjectURI: subject.URI, SubjectCollection: first.Action.SubjectCollection, + SubjectCommunityDID: first.Action.SubjectCommunityDID, ObservedCID: subject.CID, + Action: moderation.ActionLabel, LabelValue: moderation.LabelNSFW, + Origin: moderation.OriginLocal, CreatedAt: time.Now(), + }) + if err != nil { + return err + } + secondID = second.ID + // Commit whatever the failed activation wrote, so a write it made + // before reporting the error would be visible below. + activateErr = tx.SetLabelDecision(ctx, fixtures.InstanceDID(), subject.URI, moderation.LabelNSFW, second.ID, true) + return nil + })) + require.Error(t, activateErr) + require.NotEmpty(t, secondID) + var active bool + var activeAction string + require.NoError(t, db.QueryRowContext(t.Context(), `SELECT active, active_action_id FROM moderation_decisions WHERE subject_uri = $1 AND kind = 'label' AND value = 'nsfw'`, subject.URI).Scan(&active, &activeAction)) + assert.True(t, active) + assert.Equal(t, first.Action.ID, activeAction, "the already-active decision must keep its original action") +} diff --git a/internal/db/postgres/moderation_label_precedence_integration_test.go b/internal/db/postgres/moderation_label_precedence_integration_test.go new file mode 100644 index 0000000..8f0dd7f --- /dev/null +++ b/internal/db/postgres/moderation_label_precedence_integration_test.go @@ -0,0 +1,142 @@ +//go:build integration + +package postgres_test + +import ( + "encoding/json" + "testing" + + "Coves/internal/core/moderation" + "Coves/internal/core/posts" + "Coves/internal/db/postgres" + "Coves/tests/fixtures" + "Coves/tests/testkit" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func labelPrecedencePost(t *testing.T, service posts.Service, subject moderation.StrongRef) *posts.PostResult { + t.Helper() + results, err := service.GetPosts(t.Context(), posts.GetPostsRequest{URIs: []string{subject.URI}}) + require.NoError(t, err) + require.Len(t, results, 1) + return results[0] +} + +func labelPrecedenceJSON(t *testing.T, value any) []byte { + t.Helper() + encoded, err := json.Marshal(value) + require.NoError(t, err) + return encoded +} + +func requireLabelPrecedenceTombstone(t *testing.T, result *posts.PostResult, subject moderation.StrongRef) { + t.Helper() + require.NotNil(t, result.Moderated, "instance removal must take precedence over the active label") + assert.Nil(t, result.Post) + assert.Equal(t, subject.URI, result.Moderated.URI) + require.NotNil(t, result.Moderated.Moderation) + assert.Equal(t, moderation.ModerationStateRemoved, result.Moderated.Moderation.State) + assert.Empty(t, result.Moderated.Moderation.ContentLabels) + encoded := labelPrecedenceJSON(t, result) + assert.Contains(t, string(encoded), `"$type":"social.coves.community.post.defs#moderatedPost"`) + for _, field := range []string{`"contentLabels"`, `"record"`, `"title"`, `"content"`} { + assert.NotContains(t, string(encoded), field, "the tombstone must not leak classification or author content") + } +} + +func TestModerationLabelPostGetRemovalPrecedenceAndRestore(t *testing.T) { + for _, scenario := range []struct { + name string + retractOnRemoval bool + }{ + {name: "label survives removal and restore"}, + {name: "retract while removed does not restore post", retractOnRemoval: true}, + } { + t.Run(scenario.name, func(t *testing.T) { + db := testkit.DB(t) + communityDID, authorDID := moderationPostTombstoneActors(t, db) + subject := indexedModerationPost(t, db, moderation.PostV2Collection, communityDID, authorDID, "private label precedence content", "") + service := newPostgresModerationService(db) + postService := moderationPostGetService(db, postgres.NewPostRepository(db)) + before := labelPrecedencePost(t, postService, subject) + require.NotNil(t, before.Post, "the accepted post must be visible before mutation") + label := labelPost(t, service, fixtures.DID("labelprecedenceadmin"), subject, "v0", "precedence-label") + // removeIndexedModerationPost is pinned to v0; the label has advanced + // this subject to v1, so remove with its actual expected version. + removed, err := service.RemoveContent(t.Context(), fixtures.DID("postremovaladmin"), moderation.RemoveContentRequest{ + Subject: subject, ExpectedVersion: "v1", IdempotencyKey: "precedence-remove", + Reason: "social.coves.moderation.defs#reasonSpam", + }) + require.NoError(t, err) + require.NotNil(t, removed.Action) + requireLabelPrecedenceTombstone(t, labelPrecedencePost(t, postService, subject), subject) + state, err := service.GetSubjectState(t.Context(), subject.URI) + require.NoError(t, err) + require.NotNil(t, state) + requirePersistedLabelState(t, *state, label.Action.ID, "v2", moderation.ModerationStateRemoved) + require.NotNil(t, state.LocalRemoval) + assert.Equal(t, removed.Action.ID, state.LocalRemoval.ActionID) + + restorable := *removed + if scenario.retractOnRemoval { + retracted, err := service.RetractContentLabel(t.Context(), fixtures.DID("labelprecedenceadmin"), moderation.RetractContentLabelRequest{ + ActionID: label.Action.ID, ReviewedSubject: &subject, ExpectedVersion: "v2", IdempotencyKey: "precedence-retract", + }) + require.NoError(t, err) + require.Equal(t, moderation.OutcomeApplied, retracted.Outcome) + assert.Equal(t, moderation.ModerationStateRemoved, retracted.State.Moderation.State) + assert.Empty(t, retracted.State.LocalLabels) + requireLabelPrecedenceTombstone(t, labelPrecedencePost(t, postService, subject), subject) + state, err = service.GetSubjectState(t.Context(), subject.URI) + require.NoError(t, err) + assert.Equal(t, moderation.ModerationStateRemoved, state.Moderation.State) + assert.Empty(t, state.LocalLabels) + // The existing restore helper reads the version from its result. + restorable.State.Version = retracted.State.Version + } + restoreModerationPost(t, service, subject, &restorable) + visible := labelPrecedencePost(t, postService, subject) + require.NotNil(t, visible.Post, "restored accepted post must serve a normal postView") + assert.Nil(t, visible.Moderated) + assert.Equal(t, subject.URI, visible.Post.URI) + encoded := labelPrecedenceJSON(t, visible.Post) + if scenario.retractOnRemoval { + assert.Nil(t, visible.Post.Moderation) + assert.NotContains(t, string(encoded), `"moderation"`) + } else { + assert.Equal(t, labelViewExpected(posts.ModerationSourceView{ + AuthorityDID: fixtures.InstanceDID(), Scope: posts.ModerationScopeView{Kind: moderation.ScopeInstance}, + }), visible.Post.Moderation) + assert.Contains(t, string(encoded), `"contentLabels"`) + } + }) + } +} + +func TestModerationLabelPostGetAuthorDeletionAllowsRetractWithoutReview(t *testing.T) { + db := testkit.DB(t) + communityDID, authorDID := moderationPostTombstoneActors(t, db) + subject := indexedModerationPost(t, db, moderation.PostV2Collection, communityDID, authorDID, "author deleted label", "") + service := newPostgresModerationService(db) + postService := moderationPostGetService(db, postgres.NewPostRepository(db)) + require.NotNil(t, labelPrecedencePost(t, postService, subject).Post) + label := labelPost(t, service, fixtures.DID("labeldeletedadmin"), subject, "v0", "deleted-label") + _, err := db.ExecContext(t.Context(), `UPDATE posts SET deleted_at = NOW() WHERE uri = $1`, subject.URI) + require.NoError(t, err) + deletedBefore := labelPrecedencePost(t, postService, subject) + require.NotNil(t, deletedBefore.NotFound, "author-deleted post is unavailable before retraction") + retracted, err := service.RetractContentLabel(t.Context(), fixtures.DID("labeldeletedadmin"), moderation.RetractContentLabelRequest{ + ActionID: label.Action.ID, ExpectedVersion: "v1", IdempotencyKey: "deleted-retract", + }) + require.NoError(t, err) + require.NotNil(t, retracted) + assert.Equal(t, moderation.OutcomeApplied, retracted.Outcome) + assert.Equal(t, moderation.RecordStateDeleted, retracted.State.RecordState) + assert.Empty(t, retracted.State.LocalLabels) + deletedAfter := labelPrecedencePost(t, postService, subject) + require.NotNil(t, deletedAfter.NotFound) + assert.Nil(t, deletedAfter.Post) + assert.Nil(t, deletedAfter.Moderated) +} diff --git a/internal/db/postgres/moderation_label_views_integration_test.go b/internal/db/postgres/moderation_label_views_integration_test.go new file mode 100644 index 0000000..e101e9b --- /dev/null +++ b/internal/db/postgres/moderation_label_views_integration_test.go @@ -0,0 +1,380 @@ +//go:build integration + +package postgres_test + +import ( + "encoding/json" + "testing" + "time" + + "Coves/internal/core/communityFeeds" + "Coves/internal/core/discover" + "Coves/internal/core/moderation" + "Coves/internal/core/posts" + "Coves/internal/core/timeline" + "Coves/internal/db/postgres" + "Coves/tests/fixtures" + "Coves/tests/testkit" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func labelViewURIs(views []*posts.PostView) []string { + uris := make([]string, 0, len(views)) + for _, view := range views { + uris = append(uris, view.URI) + } + return uris +} + +func labelViewByURI(t *testing.T, views []*posts.PostView, uri string) *posts.PostView { + t.Helper() + for _, view := range views { + if view.URI == uri { + return view + } + } + t.Fatalf("post %s absent from read path", uri) + return nil +} + +func labelViewJSON(t *testing.T, value any) []byte { + t.Helper() + encoded, err := json.Marshal(value) + require.NoError(t, err) + return encoded +} + +// labelViewPages is one paged walk of a read path: the URIs in served order and +// the cursor returned after each page. +type labelViewPages struct { + uris []string + cursors []string +} + +func walkLabelViewPages(t *testing.T, page func(*testing.T, *string) ([]*posts.PostView, *string)) labelViewPages { + t.Helper() + var walked labelViewPages + var cursor *string + for range 10 { + views, next := page(t, cursor) + walked.uris = append(walked.uris, labelViewURIs(views)...) + if next == nil { + return walked + } + walked.cursors = append(walked.cursors, *next) + cursor = next + } + t.Fatalf("paged walk did not end after 10 pages: %v", walked.uris) + return walked +} + +func labelViewExpected(sources ...posts.ModerationSourceView) *posts.ModerationView { + return &posts.ModerationView{State: moderation.ModerationStateClear, ContentLabels: []posts.ContentLabelView{{ + Value: moderation.LabelNSFW, Sources: sources, + }}} +} + +func TestModerationLabelServedPostViews(t *testing.T) { + db := testkit.DB(t) + communityDID, authorDID := moderationPostActors(t, db) + viewerName := testkit.UniqueIDWithPrefix(t, "labelviewer") + viewerDID := fixtures.DID(viewerName) + fixtures.User(t, db, viewerName+".test", viewerDID) + _, err := db.ExecContext(t.Context(), `INSERT INTO community_subscriptions (user_did, community_did, subscribed_at) VALUES ($1, $2, NOW())`, viewerDID, communityDID) + require.NoError(t, err) + // Large score gaps keep the Hot order stable across snapshot reuse; all + // three posts are accepted and old enough to contribute to Hot history. + // Distinct creation times make every ordering, not only tie-breaks, count. + controlFirst := indexedModerationPost(t, db, moderation.PostV2Collection, communityDID, authorDID, "label view search control first", "") + labelled := indexedModerationPost(t, db, moderation.PostV2Collection, communityDID, authorDID, "label view search target", "") + controlLast := indexedModerationPost(t, db, moderation.PostV2Collection, communityDID, authorDID, "label view search control last", "") + for _, entry := range []struct { + uri string + score int + age time.Duration + }{{controlFirst.URI, 900, 4 * time.Hour}, {labelled.URI, 500, 5 * time.Hour}, {controlLast.URI, 100, 6 * time.Hour}} { + _, err := db.ExecContext(t.Context(), `UPDATE posts SET score = $2, upvote_count = $2, created_at = $3 WHERE uri = $1`, entry.uri, entry.score, time.Now().Add(-entry.age).Truncate(time.Second)) + require.NoError(t, err) + } + _, err = db.ExecContext(t.Context(), `UPDATE posts SET content_labels = '{"values":[{"val":"spoiler"}]}'::jsonb WHERE uri = $1`, labelled.URI) + require.NoError(t, err) + postRepo := postgres.NewPostRepository(db) + feedRepo := postgres.NewCommunityFeedRepository(db, "label-views-feed-secret") + timelineRepo := postgres.NewTimelineRepository(db, "label-views-timeline-secret") + discoverRepo := postgres.NewDiscoverRepository(db, "label-views-discover-secret") + service := newPostgresModerationService(db) + allURIs := []string{controlFirst.URI, labelled.URI, controlLast.URI} + reads := []struct { + name string + read func(*testing.T) []*posts.PostView + }{ + {"post.get / GetViewsByURIs", func(t *testing.T) []*posts.PostView { + found, err := postRepo.GetViewsByURIs(t.Context(), allURIs, "") + require.NoError(t, err) + views := make([]*posts.PostView, 0, len(allURIs)) + for _, uri := range allURIs { + require.Contains(t, found, uri) + views = append(views, found[uri]) + } + return views + }}, + {"actor.getPosts / GetByAuthor", func(t *testing.T) []*posts.PostView { + views, _, err := postRepo.GetByAuthor(t.Context(), posts.GetAuthorPostsRequest{ActorDID: authorDID, Limit: 50}) + require.NoError(t, err) + return views + }}, + {"community feed", func(t *testing.T) []*posts.PostView { + feed, _, err := feedRepo.GetCommunityFeed(t.Context(), communityFeeds.GetCommunityFeedRequest{Community: communityDID, Sort: "new", Timeframe: "all", Limit: 50}) + require.NoError(t, err) + views := make([]*posts.PostView, 0, len(feed)) + for _, entry := range feed { + views = append(views, entry.Post) + } + return views + }}, + {"searchPosts", func(t *testing.T) []*posts.PostView { + feed, _, err := feedRepo.SearchPosts(t.Context(), communityFeeds.SearchPostsRequest{Query: "label view search", Community: communityDID, Sort: "relevance", Timeframe: "all", Limit: 50}) + require.NoError(t, err) + views := make([]*posts.PostView, 0, len(feed)) + for _, entry := range feed { + views = append(views, entry.Post) + } + return views + }}, + {"timeline", func(t *testing.T) []*posts.PostView { + feed, _, err := timelineRepo.GetTimeline(t.Context(), timeline.GetTimelineRequest{UserDID: viewerDID, Sort: "new", Limit: 50}) + require.NoError(t, err) + views := make([]*posts.PostView, 0, len(feed)) + for _, entry := range feed { + views = append(views, entry.Post) + } + return views + }}, + {"Discover new", func(t *testing.T) []*posts.PostView { + feed, _, err := discoverRepo.GetDiscover(t.Context(), discover.GetDiscoverRequest{Sort: "new", Timeframe: "all", Limit: 50}) + require.NoError(t, err) + views := make([]*posts.PostView, 0, len(feed)) + for _, entry := range feed { + views = append(views, entry.Post) + } + return views + }}, + {"getComments / VisibleHeaderView", func(t *testing.T) []*posts.PostView { + views := make([]*posts.PostView, 0, len(allURIs)) + for _, uri := range allURIs { + view, err := postRepo.VisibleHeaderView(t.Context(), uri, "") + require.NoError(t, err) + require.NotNil(t, view) + views = append(views, view) + } + return views + }}, + } + // Walk each paged path one post per page, so a label-driven change to rank + // or cursor building moves a post across a page boundary. + pagedReads := []struct { + name string + // stableCursor is false for hot sort, whose cursor embeds the query time. + stableCursor bool + page func(*testing.T, *string) ([]*posts.PostView, *string) + }{ + {"community feed hot", false, func(t *testing.T, cursor *string) ([]*posts.PostView, *string) { + feed, next, err := feedRepo.GetCommunityFeed(t.Context(), communityFeeds.GetCommunityFeedRequest{Community: communityDID, Sort: "hot", Timeframe: "all", Limit: 1, Cursor: cursor}) + require.NoError(t, err) + views := make([]*posts.PostView, 0, len(feed)) + for _, entry := range feed { + views = append(views, entry.Post) + } + return views, next + }}, + {"searchPosts relevance", true, func(t *testing.T, cursor *string) ([]*posts.PostView, *string) { + feed, next, err := feedRepo.SearchPosts(t.Context(), communityFeeds.SearchPostsRequest{Query: "label view search", Community: communityDID, Sort: "relevance", Timeframe: "all", Limit: 1, Cursor: cursor}) + require.NoError(t, err) + views := make([]*posts.PostView, 0, len(feed)) + for _, entry := range feed { + views = append(views, entry.Post) + } + return views, next + }}, + {"actor.getPosts", true, func(t *testing.T, cursor *string) ([]*posts.PostView, *string) { + views, next, err := postRepo.GetByAuthor(t.Context(), posts.GetAuthorPostsRequest{ActorDID: authorDID, Limit: 1, Cursor: cursor}) + require.NoError(t, err) + return views, next + }}, + } + baselinePages := make(map[string]labelViewPages, len(pagedReads)) + for _, read := range pagedReads { + walked := walkLabelViewPages(t, read.page) + require.ElementsMatch(t, allURIs, walked.uris, "%s must page every accepted post once", read.name) + require.GreaterOrEqual(t, len(walked.cursors), 2, "%s must be compared across several pages", read.name) + baselinePages[read.name] = walked + } + require.Equal(t, []string{controlFirst.URI, labelled.URI, controlLast.URI}, baselinePages["community feed hot"].uris, "Hot must rank the labelled post between the controls") + checkPages := func(stage string) { + t.Helper() + for _, read := range pagedReads { + t.Run(stage+"/paged "+read.name, func(t *testing.T) { + walked := walkLabelViewPages(t, read.page) + assert.Equal(t, baselinePages[read.name].uris, walked.uris, "label must not move a post across pages") + if read.stableCursor { + assert.Equal(t, baselinePages[read.name].cursors, walked.cursors, "label must not change cursors") + } else { + assert.Len(t, walked.cursors, len(baselinePages[read.name].cursors)) + } + }) + } + } + + baselineOrder := make(map[string][]string, len(reads)) + baselineRecord := make(map[string][]byte, len(reads)) + for _, read := range reads { + // The after-apply and after-retract checks compare against this baseline. + if !t.Run("before/"+read.name, func(t *testing.T) { + views := read.read(t) + baselineOrder[read.name] = labelViewURIs(views) + require.Len(t, views, 3, "all three accepted posts must be visible before moderation") + require.ElementsMatch(t, allURIs, baselineOrder[read.name]) + baselineRecord[read.name] = labelViewJSON(t, labelViewByURI(t, views, labelled.URI).Record) + require.Contains(t, string(baselineRecord[read.name]), `"labels"`, "the author self-label must be present so an overwritten record.labels cannot pass") + for _, view := range views { + require.Nil(t, view.Moderation, "no decisions exist before apply") + require.NotContains(t, string(labelViewJSON(t, view)), `"moderation"`) + } + }) { + t.Fatalf("baseline read %s failed", read.name) + } + } + + readHot := func(t *testing.T, limit int, cursor *string) ([]*posts.PostView, *string) { + t.Helper() + feed, next, err := discoverRepo.GetDiscover(t.Context(), discover.GetDiscoverRequest{Sort: "hot", Limit: limit, Cursor: cursor}) + require.NoError(t, err) + views := make([]*posts.PostView, 0, len(feed)) + for _, entry := range feed { + views = append(views, entry.Post) + } + return views, next + } + first, preLabelCursor := readHot(t, 1, nil) + require.Equal(t, []string{controlFirst.URI}, labelViewURIs(first), "Hot must page the unlabelled high-ranked control first") + require.NotNil(t, preLabelCursor) + hotRefreshBaseline, _ := readHot(t, 2, nil) + hotContinuationBaseline, _ := readHot(t, 2, preLabelCursor) + require.Equal(t, []string{controlFirst.URI, labelled.URI}, labelViewURIs(hotRefreshBaseline)) + require.Equal(t, []string{labelled.URI, controlLast.URI}, labelViewURIs(hotContinuationBaseline)) + for _, view := range append(hotRefreshBaseline, hotContinuationBaseline...) { + require.Nil(t, view.Moderation) + } + hotRecord := labelViewJSON(t, labelViewByURI(t, hotContinuationBaseline, labelled.URI).Record) + + label := labelPost(t, service, fixtures.DID("labelviewsadmin"), labelled, "v0", "views-apply") + want := labelViewExpected(posts.ModerationSourceView{AuthorityDID: fixtures.InstanceDID(), Scope: posts.ModerationScopeView{Kind: moderation.ScopeInstance}}) + for _, read := range reads { + t.Run("after apply/"+read.name, func(t *testing.T) { + views := read.read(t) + assert.Equal(t, baselineOrder[read.name], labelViewURIs(views), "label must not change ordering or membership") + target := labelViewByURI(t, views, labelled.URI) + assert.Equal(t, baselineRecord[read.name], labelViewJSON(t, target.Record), "record.labels must be served verbatim") + assert.Equal(t, want, target.Moderation) + assert.Contains(t, string(labelViewJSON(t, target)), `"moderation":{"state":"clear","contentLabels":[{"value":"nsfw","sources":[{"authorityDid":"`+fixtures.InstanceDID()+`","scope":{"kind":"instance"}}]}]}`) + for _, control := range []string{controlFirst.URI, controlLast.URI} { + view := labelViewByURI(t, views, control) + assert.Nil(t, view.Moderation) + assert.NotContains(t, string(labelViewJSON(t, view)), `"moderation"`) + } + }) + } + checkHot := func(stage string, expected *posts.ModerationView) { + t.Helper() + for _, page := range []struct { + name string + cursor *string + want []string + }{ + {"refreshed first page", nil, labelViewURIs(hotRefreshBaseline)}, + {"continuation from pre-label cursor", preLabelCursor, labelViewURIs(hotContinuationBaseline)}, + } { + t.Run(stage+"/Discover hot/"+page.name, func(t *testing.T) { + views, _ := readHot(t, 2, page.cursor) + assert.Equal(t, page.want, labelViewURIs(views), "reused snapshot must keep page order and membership") + target := labelViewByURI(t, views, labelled.URI) + assert.Equal(t, hotRecord, labelViewJSON(t, target.Record)) + assert.Equal(t, expected, target.Moderation) + for _, view := range views { + if view.URI != labelled.URI { + assert.Nil(t, view.Moderation) + } + } + }) + } + } + checkHot("after apply", want) + checkPages("after apply") + _, err = service.RetractContentLabel(t.Context(), fixtures.DID("labelviewsadmin"), moderation.RetractContentLabelRequest{ + ActionID: label.Action.ID, ReviewedSubject: &labelled, ExpectedVersion: "v1", IdempotencyKey: "views-retract", + }) + require.NoError(t, err) + for _, read := range reads { + t.Run("after retract/"+read.name, func(t *testing.T) { + views := read.read(t) + assert.Equal(t, baselineOrder[read.name], labelViewURIs(views)) + assert.Equal(t, baselineRecord[read.name], labelViewJSON(t, labelViewByURI(t, views, labelled.URI).Record)) + for _, view := range views { + assert.Nil(t, view.Moderation) + assert.NotContains(t, string(labelViewJSON(t, view)), `"moderation"`) + } + }) + } + checkHot("after retract", nil) + checkPages("after retract") +} + +func TestModerationLabelDecisionSourcesInServedViews(t *testing.T) { + db := testkit.DB(t) + communityDID, authorDID := moderationPostActors(t, db) + local := indexedModerationPost(t, db, moderation.PostV2Collection, communityDID, authorDID, "multi-source", "") + inactive := indexedModerationPost(t, db, moderation.PostV2Collection, communityDID, authorDID, "inactive label", "") + foreign := indexedModerationPost(t, db, moderation.PostV2Collection, communityDID, authorDID, "foreign label", "") + repo := postgres.NewPostRepository(db) + for _, uri := range []string{local.URI, inactive.URI, foreign.URI} { + views, err := repo.GetViewsByURIs(t.Context(), []string{uri}, "") + require.NoError(t, err) + require.Contains(t, views, uri) + require.Nil(t, views[uri].Moderation, "raw fixtures have not been inserted yet") + } + labelPost(t, newPostgresModerationService(db), fixtures.DID("labelsourcesadmin"), local, "v0", "sources-local") + foreignAuthority := fixtures.DID("foreignlabels") + insertDecision := func(subject moderation.StrongRef, authority, scopeKind, scopeCommunity string, active bool) { + t.Helper() + id := testkit.TID() + _, err := db.ExecContext(t.Context(), ` + INSERT INTO moderation_actions + (id, actor_did, authority_did, scope_kind, scope_community_did, subject_uri, subject_collection, + subject_community_did, observed_cid, action, label_value, origin, created_at) + VALUES ($1, $2, $2, $3, NULLIF($4, ''), $5, $6, $7, $8, 'label', 'nsfw', 'inherited', NOW()) + `, id, authority, scopeKind, scopeCommunity, subject.URI, moderation.PostV2Collection, communityDID, subject.CID) + require.NoError(t, err) + _, err = db.ExecContext(t.Context(), ` + INSERT INTO moderation_decisions + (authority_did, scope_kind, scope_community_did, subject_uri, kind, value, active_action_id, active) + VALUES ($1, $2, NULLIF($3, ''), $4, 'label', 'nsfw', $5, $6) + `, authority, scopeKind, scopeCommunity, subject.URI, id, active) + require.NoError(t, err) + } + insertDecision(inactive, foreignAuthority, moderation.ScopeInstance, "", false) + insertDecision(foreign, foreignAuthority, "community", communityDID, true) + insertDecision(local, foreignAuthority, moderation.ScopeInstance, "", true) + views, err := repo.GetViewsByURIs(t.Context(), []string{inactive.URI, foreign.URI, local.URI}, "") + require.NoError(t, err) + require.Len(t, views, 3) + assert.Nil(t, views[inactive.URI].Moderation, "an inactive decision must not produce a label") + assert.Equal(t, labelViewExpected(posts.ModerationSourceView{ + AuthorityDID: foreignAuthority, Scope: posts.ModerationScopeView{Kind: "community", CommunityDID: communityDID}, + }), views[foreign.URI].Moderation, "a foreign decision must retain its own authority and community scope") + assert.Equal(t, labelViewExpected( + posts.ModerationSourceView{AuthorityDID: foreignAuthority, Scope: posts.ModerationScopeView{Kind: moderation.ScopeInstance}}, + posts.ModerationSourceView{AuthorityDID: fixtures.InstanceDID(), Scope: posts.ModerationScopeView{Kind: moderation.ScopeInstance}}, + ), views[local.URI].Moderation, "the same value from two authorities must aggregate into one label with sorted sources") +} diff --git a/internal/db/postgres/moderation_repo.go b/internal/db/postgres/moderation_repo.go index 7463470..a400f1f 100644 --- a/internal/db/postgres/moderation_repo.go +++ b/internal/db/postgres/moderation_repo.go @@ -48,8 +48,13 @@ func (r *ModerationRepository) InTransaction(ctx context.Context, fn func(ctx co // SubjectModeration reads the stored moderation state of a subject. func (r *ModerationRepository) SubjectModeration(ctx context.Context, authorityDID, subjectURI string) (*moderation.SubjectModeration, error) { + tx, err := r.db.BeginTx(ctx, &sql.TxOptions{Isolation: sql.LevelRepeatableRead, ReadOnly: true}) + if err != nil { + return nil, fmt.Errorf("begin subject moderation read: %w", err) + } + defer tx.Rollback() var version int64 - row := r.db.QueryRowContext(ctx, ` + row := tx.QueryRowContext(ctx, ` SELECT COALESCE(s.version, 0), a.id, a.actor_did, a.authority_did, a.scope_kind, a.scope_community_did, a.subject_uri, a.subject_collection, a.subject_community_did, a.observed_cid, @@ -64,9 +69,18 @@ func (r *ModerationRepository) SubjectModeration(ctx context.Context, authorityD `, authorityDID, subjectURI) action, err := scanModerationAction(row, &version, nil) if err != nil { - return nil, err + return nil, fmt.Errorf("read subject moderation: %w", err) } - return &moderation.SubjectModeration{Version: version, ActiveRemoval: action}, nil + // A locking read here would fail with a serialization error, or wait, when + // a concurrent retraction updates a label decision after the snapshot. + labels, err := activeLabels(ctx, tx, authorityDID, subjectURI, false) + if err != nil { + return nil, fmt.Errorf("read subject moderation labels: %w", err) + } + if err := tx.Commit(); err != nil { + return nil, fmt.Errorf("commit subject moderation read: %w", err) + } + return &moderation.SubjectModeration{Version: version, ActiveRemoval: action, ActiveLabels: labels}, nil } type moderationTransaction struct { @@ -303,6 +317,45 @@ func (t *moderationTransaction) ActiveRemoval(ctx context.Context, authorityDID, return action, err } +func (t *moderationTransaction) ActiveLabels(ctx context.Context, authorityDID, subjectURI string) ([]moderation.Action, error) { + return activeLabels(ctx, t.tx, authorityDID, subjectURI, true) +} + +// activeLabels reads the instance's active labels on a subject, ordered by +// value. Mutations lock the decision rows; snapshot reads must not. +func activeLabels(ctx context.Context, tx *sql.Tx, authorityDID, subjectURI string, lockDecisions bool) ([]moderation.Action, error) { + query := ` + SELECT ` + moderationActionColumns + ` FROM moderation_actions a + JOIN moderation_decisions d ON d.active_action_id = a.id + WHERE d.kind = 'label' AND d.active AND d.authority_did = $1 + AND d.scope_kind = 'instance' AND d.subject_uri = $2 + ORDER BY d.value` + if lockDecisions { + query += ` + FOR SHARE OF d` + } + rows, err := tx.QueryContext(ctx, query, authorityDID, subjectURI) + if err != nil { + return nil, fmt.Errorf("query active moderation labels: %w", err) + } + defer rows.Close() + var actions []moderation.Action + for rows.Next() { + action, err := scanModerationAction(rows, nil, nil) + if err != nil { + return nil, fmt.Errorf("scan active moderation label: %w", err) + } + if action == nil { + return nil, errors.New("active moderation label has no action") + } + actions = append(actions, *action) + } + if err := rows.Err(); err != nil { + return nil, fmt.Errorf("read active moderation labels: %w", err) + } + return actions, nil +} + func (t *moderationTransaction) InsertAction(ctx context.Context, action moderation.Action) (*moderation.Action, error) { action.ID = moderationActionClock.Next().String() // Postgres timestamps have microsecond precision. Return the same instant @@ -354,6 +407,48 @@ func (t *moderationTransaction) SetRemovalDecision(ctx context.Context, authorit return err } +func (t *moderationTransaction) SetLabelDecision(ctx context.Context, authorityDID, subjectURI, value, actionID string, active bool) error { + if !active { + result, err := t.tx.ExecContext(ctx, ` + UPDATE moderation_decisions SET active = FALSE + WHERE authority_did = $1 AND subject_uri = $2 AND scope_kind = 'instance' + AND kind = 'label' AND value = $3 AND active_action_id = $4 AND active + `, authorityDID, subjectURI, value, actionID) + if err != nil { + return fmt.Errorf("deactivate moderation label: %w", err) + } + count, err := result.RowsAffected() + if err != nil { + return fmt.Errorf("count deactivated moderation labels: %w", err) + } + if count != 1 { + return fmt.Errorf("active moderation label decision not found for retraction: updated %d rows", count) + } + return nil + } + // An already-active decision is never overwritten: the redundant apply + // path returns unchanged without calling here, so a conflict is a bug. + result, err := t.tx.ExecContext(ctx, ` + INSERT INTO moderation_decisions + (authority_did, scope_kind, scope_community_did, subject_uri, kind, value, active_action_id, active) + VALUES ($1, 'instance', NULL, $2, 'label', $3, $4, TRUE) + ON CONFLICT ON CONSTRAINT moderation_decisions_key DO UPDATE SET + active_action_id = EXCLUDED.active_action_id, active = TRUE + WHERE NOT moderation_decisions.active + `, authorityDID, subjectURI, value, actionID) + if err != nil { + return fmt.Errorf("activate moderation label: %w", err) + } + count, err := result.RowsAffected() + if err != nil { + return fmt.Errorf("count activated moderation labels: %w", err) + } + if count != 1 { + return fmt.Errorf("moderation label decision already active: updated %d rows", count) + } + return nil +} + func (t *moderationTransaction) SetSubjectVersion(ctx context.Context, subjectURI string, version int64) error { result, err := t.tx.ExecContext(ctx, ` UPDATE moderation_subjects SET version = $2, updated_at = NOW() WHERE subject_uri = $1 @@ -481,11 +576,13 @@ func (r *ModerationRepository) ListActions(ctx context.Context, query moderation AND (NOT $14::boolean OR NOT ( COALESCE(a.reason = ANY($15::text[]), FALSE) OR COALESCE(reversed.reason = ANY($15::text[]), FALSE))) + AND (NOT $16::boolean OR a.action <> ALL($17::text[])) ORDER BY a.created_at DESC, a.id DESC LIMIT $1 `, query.Limit, beforeCreatedAt, beforeID, query.SubjectURI, query.SubjectCollection, query.Action, query.Origin, query.AuthorityDID, query.ActorDID, query.CommunityDID, - query.ActionID, query.Since, query.Until, query.ExcludeHidden, pq.Array(moderation.HiddenActionReasons())) + query.ActionID, query.Since, query.Until, query.ExcludeHidden, pq.Array(moderation.HiddenActionReasons()), + query.ExcludeLabelActions, pq.Array(moderation.PublicExcludedActions())) if err != nil { return nil, fmt.Errorf("list moderation actions: %w", err) } diff --git a/internal/db/postgres/moderation_subject_snapshot_integration_test.go b/internal/db/postgres/moderation_subject_snapshot_integration_test.go new file mode 100644 index 0000000..cb869f9 --- /dev/null +++ b/internal/db/postgres/moderation_subject_snapshot_integration_test.go @@ -0,0 +1,92 @@ +//go:build integration + +package postgres_test + +import ( + "testing" + "time" + + "Coves/internal/core/moderation" + "Coves/tests/fixtures" + "Coves/tests/testkit" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +type subjectStateOutcome struct { + state *moderation.SubjectState + err error +} + +// getSubjectState reads one consistent snapshot. A retraction that holds the +// label decision row and commits while the read is in flight must neither +// block the read nor fail it with a serialization error: the read returns the +// state as of its snapshot. +func TestModerationSubjectStateReadSurvivesConcurrentLabelRetraction(t *testing.T) { + db := testkit.DB(t) + subject := indexedLabelPost(t, db) + service := newPostgresModerationService(db) + label := labelPost(t, service, fixtures.DID("snapshotlabeladmin"), subject, "v0", "snapshot-label") + + // Hold the writes a retraction makes, uncommitted, on a second connection. + retraction, err := db.BeginTx(t.Context(), nil) + require.NoError(t, err) + t.Cleanup(func() { _ = retraction.Rollback() }) + result, err := retraction.ExecContext(t.Context(), ` + UPDATE moderation_decisions SET active = FALSE + WHERE subject_uri = $1 AND kind = 'label' AND active_action_id = $2 AND active + `, subject.URI, label.Action.ID) + require.NoError(t, err) + updated, err := result.RowsAffected() + require.NoError(t, err) + require.EqualValues(t, 1, updated) + _, err = retraction.ExecContext(t.Context(), `UPDATE moderation_subjects SET version = 2 WHERE subject_uri = $1`, subject.URI) + require.NoError(t, err) + + outcomes := make(chan subjectStateOutcome, 1) + go func() { + state, err := service.GetSubjectState(t.Context(), subject.URI) + outcomes <- subjectStateOutcome{state: state, err: err} + }() + + // Let the read either finish or reach a lock wait on the held decision row + // before the retraction commits, so a locking label read sees a row + // updated after its snapshot. + var outcome *subjectStateOutcome + testkit.WaitFor(t, 5*time.Second, func() (bool, error) { + select { + case finished := <-outcomes: + outcome = &finished + return true, nil + default: + } + var waiting int + err := db.QueryRowContext(t.Context(), ` + SELECT count(*) FROM pg_stat_activity + WHERE datname = current_database() AND pid <> pg_backend_pid() + AND wait_event_type = 'Lock' AND query LIKE '%moderation_decisions%' + `).Scan(&waiting) + return waiting == 1, err + }, testkit.WithDescription("subject state read finished or waiting on the held label decision row")) + require.NoError(t, retraction.Commit()) + if outcome == nil { + select { + case finished := <-outcomes: + outcome = &finished + case <-time.After(5 * time.Second): + t.Fatal("subject state read did not finish after the retraction committed") + } + } + + require.NoError(t, outcome.err) + require.NotNil(t, outcome.state) + requirePersistedLabelState(t, *outcome.state, label.Action.ID, "v1", moderation.ModerationStateClear) + + after, err := service.GetSubjectState(t.Context(), subject.URI) + require.NoError(t, err) + require.NotNil(t, after) + assert.Equal(t, "v2", after.Version) + assert.Empty(t, after.LocalLabels) + assert.Empty(t, after.Moderation.ContentLabels) +} diff --git a/internal/db/postgres/post_repo.go b/internal/db/postgres/post_repo.go index 96e20e5..17c7bca 100644 --- a/internal/db/postgres/post_repo.go +++ b/internal/db/postgres/post_repo.go @@ -56,7 +56,21 @@ const postViewSelectColumns = ` p.title, p.content, p.content_facets, p.embed, p.content_labels, p.created_at, p.edited_at, p.indexed_at, p.upvote_count + p.bridged_upvote_count AS upvote_count, p.downvote_count + p.bridged_downvote_count AS downvote_count, p.score, p.comment_count, - a.status AS admission_status, a.acceptance_uri AS admission_acceptance_uri` + a.status AS admission_status, a.acceptance_uri AS admission_acceptance_uri, + (SELECT jsonb_agg(jsonb_build_object('value', labels.value, 'sources', labels.sources) ORDER BY labels.value) + FROM ( + SELECT d.value, + jsonb_agg(jsonb_build_object( + 'authorityDid', d.authority_did, + 'scope', jsonb_build_object('kind', d.scope_kind) || + CASE WHEN d.scope_community_did IS NOT NULL + THEN jsonb_build_object('communityDid', d.scope_community_did) + ELSE '{}'::jsonb END + ) ORDER BY d.authority_did, d.scope_kind, d.scope_community_did) AS sources + FROM moderation_decisions d + WHERE d.subject_uri = p.uri AND d.kind = 'label' AND d.active + GROUP BY d.value + ) labels) AS active_content_labels` // NewPostRepository creates a new PostgreSQL post repository. // @@ -666,6 +680,7 @@ func scanPostView(rows *sql.Rows, extraDest ...interface{}) (*posts.PostView, er communityOrigin sql.NullString admissionStatus sql.NullString acceptanceURI sql.NullString + activeLabelsJSON sql.NullString ) dest := []interface{}{ @@ -675,7 +690,7 @@ func scanPostView(rows *sql.Rows, extraDest ...interface{}) (*posts.PostView, er &title, &content, &facets, &embed, &labelsJSON, &postView.CreatedAt, &editedAt, &postView.IndexedAt, &postView.UpvoteCount, &postView.DownvoteCount, &postView.Score, &postView.CommentCount, - &admissionStatus, &acceptanceURI, + &admissionStatus, &acceptanceURI, &activeLabelsJSON, } dest = append(dest, extraDest...) @@ -734,6 +749,14 @@ func scanPostView(rows *sql.Rows, extraDest ...interface{}) (*posts.PostView, er if acceptanceURI.Valid { postView.AcceptanceURI = acceptanceURI.String } + if activeLabelsJSON.Valid { + var labels []posts.ContentLabelView + if err := json.Unmarshal([]byte(activeLabelsJSON.String), &labels); err != nil { + return nil, fmt.Errorf("decode active post labels for %s: %w", postView.URI, err) + } + // Clear holds only because every caller filters with visiblePostsPredicate. + postView.Moderation = &posts.ModerationView{State: posts.ModerationViewStateClear, ContentLabels: labels} + } // Parse facets JSON into local variable (will be added to record below) // Log errors but continue - malformed optional fields shouldn't break the response diff --git a/tests/e2e/moderation_contract_test.go b/tests/e2e/moderation_contract_test.go index 37db2ed..10fe2dc 100644 --- a/tests/e2e/moderation_contract_test.go +++ b/tests/e2e/moderation_contract_test.go @@ -9,6 +9,7 @@ import ( "fmt" "net/http" "net/url" + "reflect" "strings" "testing" @@ -20,10 +21,188 @@ import ( const ( removeContentMethod = "social.coves.moderation.removeContent" restoreContentMethod = "social.coves.moderation.restoreContent" + labelContentMethod = "social.coves.moderation.labelContent" + retractLabelMethod = "social.coves.moderation.retractContentLabel" listActionsMethod = "social.coves.moderation.listActions" listAdminActionsMethod = "social.coves.moderation.listAdminActions" ) +// TestModerationContentLabelContract verifies that an instance label is served +// on a pipeline-indexed post without changing the author's PDS record. +func TestModerationContentLabelContract(t *testing.T) { + p := newPipeline(t) + author := p.IndexedAccount(t, "mcl") + community := indexedCommunity(t, p, "mcl", author.DID) + post := indexedPost(t, p, community, author, "label contract "+testkit.UniqueID(t)) + outsider := p.IndexedAccount(t, "mclout") + admin := testkit.ModerationAdmin(t, 1) + + readPost := func() (map[string]any, error) { + var response struct { + Posts []map[string]any `json:"posts"` + } + err := p.AppView.Query(context.Background(), "social.coves.community.post.get", + url.Values{"uris": {post.URI}}, &response) + if err != nil { + return nil, err + } + if len(response.Posts) != 1 { + return nil, fmt.Errorf("post.get returned %d members for one URI", len(response.Posts)) + } + return response.Posts[0], nil + } + readFeedPost := func() (map[string]any, error) { + var response struct { + Feed []struct { + Post map[string]any `json:"post"` + } `json:"feed"` + } + err := p.AppView.Query(context.Background(), "social.coves.communityFeed.getCommunity", + url.Values{"community": {community.DID}, "sort": {"new"}, "limit": {"50"}}, &response) + if err != nil { + return nil, err + } + for _, item := range response.Feed { + if item.Post["uri"] == post.URI { + return item.Post, nil + } + } + return nil, nil + } + readPDSRecord := func() (testkit.RecordValue, error) { + var record testkit.RecordValue + err := author.XRPC().Query(context.Background(), "com.atproto.repo.getRecord", + url.Values{"repo": {author.DID}, "collection": {postV2Collection}, "rkey": {post.URI[strings.LastIndex(post.URI, "/")+1:]}}, &record) + return record, err + } + + p.Await(t, "the accepted post to serve through post.get", func() (bool, error) { + view, err := readPost() + if err != nil { + return false, err + } + return view["uri"] == post.URI && view["cid"] == post.CID && view["record"] != nil, nil + }, withReadCadence()) + require.Contains(t, communityFeedURIs(t, p, community.DID), post.URI) + before, err := readPDSRecord() + require.NoError(t, err) + require.Equal(t, post.URI, before.URI) + require.Equal(t, post.CID, before.CID) + require.NotNil(t, before.Value) + initialView, err := readPost() + require.NoError(t, err) + require.NotContains(t, initialView, "moderation") + + stateToken := admin.ServiceAuth(t, communityInstanceDID, subjectStateMethod) + readState := func() (moderationSubjectStateResponse, error) { + var response moderationSubjectStateResponse + err := p.AppView.As(stateToken).Query(context.Background(), subjectStateMethod, + url.Values{"subject": {post.URI}}, &response) + return response, err + } + state, err := readState() + require.NoError(t, err) + require.Equal(t, post.CID, state.State.CurrentSubject.CID) + input := map[string]any{ + "subject": map[string]any{"uri": post.URI, "cid": post.CID}, + "labelValue": "nsfw", + "expectedVersion": state.State.Version, + "idempotencyKey": testkit.UniqueID(t), + } + err = p.AppView.As(outsider.ServiceAuth(t, communityInstanceDID, labelContentMethod)).Procedure( + t.Context(), labelContentMethod, input, nil) + requireXRPCRefusal(t, err, http.StatusForbidden, "Forbidden", "a non-admin label request") + stillClear, err := readPost() + require.NoError(t, err) + require.Equal(t, post.URI, stillClear["uri"]) + require.NotContains(t, stillClear, "moderation") + + var applied struct { + Outcome string `json:"outcome"` + Action struct { + Action struct { + Action string `json:"action"` + LabelValue string `json:"labelValue"` + Ref struct { + ActionID string `json:"actionId"` + } `json:"ref"` + } `json:"action"` + } `json:"action"` + } + err = p.AppView.As(admin.ServiceAuth(t, communityInstanceDID, labelContentMethod)).Procedure( + t.Context(), labelContentMethod, input, &applied) + require.NoError(t, err) + require.Equal(t, "applied", applied.Outcome) + require.Equal(t, "label", applied.Action.Action.Action) + require.Equal(t, "nsfw", applied.Action.Action.LabelValue) + labelID := applied.Action.Action.Ref.ActionID + require.NotEmpty(t, labelID) + + wantModeration := map[string]any{ + "state": "clear", + "contentLabels": []any{map[string]any{ + "value": "nsfw", "sources": []any{map[string]any{ + "authorityDid": communityInstanceDID, "scope": map[string]any{"kind": "instance"}, + }}, + }}, + } + p.Await(t, "post.get to serve the instance NSFW label", func() (bool, error) { + view, err := readPost() + if err != nil { + return false, err + } + return view["uri"] == post.URI && view["record"] != nil && + reflect.DeepEqual(view["moderation"], wantModeration), nil + }, withReadCadence()) + p.FreshReadQuota(t, "labelled-community-feed") + p.Await(t, "the accepted community feed post to carry the same label", func() (bool, error) { + view, err := readFeedPost() + if err != nil { + return false, err + } + return view != nil && reflect.DeepEqual(view["moderation"], wantModeration), nil + }, withReadCadence()) + require.Contains(t, communityFeedURIs(t, p, community.DID), post.URI) + + state, err = readState() + require.NoError(t, err) + require.NotEqual(t, input["expectedVersion"], state.State.Version) + var retracted struct { + Outcome string `json:"outcome"` + Action struct { + Action struct { + Action string `json:"action"` + } `json:"action"` + } `json:"action"` + } + err = p.AppView.As(admin.ServiceAuth(t, communityInstanceDID, retractLabelMethod)).Procedure( + t.Context(), retractLabelMethod, map[string]any{ + "actionId": labelID, + "expectedVersion": state.State.Version, + "idempotencyKey": testkit.UniqueID(t), + "reviewedSubject": map[string]any{"uri": post.URI, "cid": post.CID}, + }, &retracted) + require.NoError(t, err) + require.Equal(t, "applied", retracted.Outcome) + require.Equal(t, "retract-label", retracted.Action.Action.Action) + + p.FreshReadQuota(t, "retracted-post-label") + p.Holds(t, "post.get to keep serving the post without moderation after retraction", func() (bool, error) { + view, err := readPost() + if err != nil { + return false, err + } + _, labelled := view["moderation"] + return view["uri"] == post.URI && view["cid"] == post.CID && view["record"] != nil && + view["$type"] == nil && !labelled, nil + }) + after, err := readPDSRecord() + require.NoError(t, err) + require.Equal(t, before.URI, after.URI) + require.Equal(t, before.CID, after.CID, "moderation must not rewrite the author's PDS record") + require.Equal(t, before.Value, after.Value, "moderation must not change the PDS value, including labels") +} + // TestModerationCommentRemovalContract crosses the real PDS → consumer → AppView // boundary twice: once for the comment and again for the author's edit while the // moderation overlay is active. The edited CID is the delivery barrier for the diff --git a/tests/lexicon_moderation_test.go b/tests/lexicon_moderation_test.go index af872be..00331b9 100644 --- a/tests/lexicon_moderation_test.go +++ b/tests/lexicon_moderation_test.go @@ -673,29 +673,37 @@ func TestLexiconModerationProcedureInputs(t *testing.T) { } } -func listActionParameterExpectations(includeActionID bool) map[string]moderationPropertyExpectation { +// listActionParameterExpectations returns the public or admin action-log +// filters. They differ only in actionId and in the action filter: the public +// log never serves label or retract-label actions (2026-09-30), so its action +// filter does not list them. +func listActionParameterExpectations(admin bool) map[string]moderationPropertyExpectation { + actionKinds := []string{"remove", "restore", "apply-removal", "retract-removal"} + if admin { + actionKinds = append(actionKinds, "label", "retract-label") + } properties := map[string]moderationPropertyExpectation{ "limit": {schemaType: "integer", minimum: 1, maximum: 100, defaultValue: float64(50), hasDefault: true}, "cursor": {schemaType: "string", maxLength: 2048}, "subject": {schemaType: "string", format: "at-uri"}, "collection": {schemaType: "string", format: "nsid"}, - "action": {schemaType: "string", maxLength: 64, - knownValues: []string{"remove", "restore", "apply-removal", "retract-removal", "label", "retract-label"}}, - "origin": {schemaType: "string", maxLength: 64, knownValues: []string{"local", "inherited"}}, - "authority": {schemaType: "string", format: "at-identifier"}, - "actor": {schemaType: "string", format: "at-identifier"}, - "community": {schemaType: "string", minLength: 1, maxLength: 320}, - "since": {schemaType: "string", format: "datetime"}, - "until": {schemaType: "string", format: "datetime"}, - } - if includeActionID { + "action": {schemaType: "string", maxLength: 64, knownValues: actionKinds}, + "origin": {schemaType: "string", maxLength: 64, knownValues: []string{"local", "inherited"}}, + "authority": {schemaType: "string", format: "at-identifier"}, + "actor": {schemaType: "string", format: "at-identifier"}, + "community": {schemaType: "string", minLength: 1, maxLength: 320}, + "since": {schemaType: "string", format: "datetime"}, + "until": {schemaType: "string", format: "datetime"}, + } + if admin { properties["actionId"] = moderationPropertyExpectation{schemaType: "string", minLength: 1, maxLength: 128} } return properties } -// TestLexiconModerationQueryParameters pins public/admin filter parity and the -// required getSubjectState subject, preventing hidden filters or inconsistent +// TestLexiconModerationQueryParameters pins public/admin filter parity (apart +// from the admin-only actionId filter and label action kinds) and the required +// getSubjectState subject, preventing hidden filters or inconsistent // pagination bounds from changing what callers can enumerate. func TestLexiconModerationQueryParameters(t *testing.T) { queryProperties := map[string]map[string]moderationPropertyExpectation{