diff --git a/docsite/docs/configuration/transforms.mdx b/docsite/docs/configuration/transforms.mdx deleted file mode 100644 index 67336461..00000000 --- a/docsite/docs/configuration/transforms.mdx +++ /dev/null @@ -1,523 +0,0 @@ ---- -sidebar_position: 4 -title: Scrobble Modification -toc_max_heading_level: 4 ---- - -Multi-scrobbler configs support the ability to modify scrobble data in an automated fashion by matching and replacing strings in **title, artists, and album** at many different times in multi-scrobbler's lifecycle. - -### Why? - -You may need to "clean up" data from a Source or before sending to a scrobble Client due to any number of reasons: - -* ID3 tags in your music collection are dirty or have repeating garbage IE `[YourMusicSource.com] My Artist - My Title` -* A Source's service often incorrectly adds data to some field IE `My Artist - My Title (Album Version)` when the title should just be `My Title` -* An Artist you listen to often is spelled different between a Source and a Client which causes duplicate scrobbles - -In any scenario where a repeating pattern can be found in the data it would be nice to be able to fix it before the data gets downstream or to help prevent duplicate scrobbling. Multi-scrobbler can help you do this. - -## Overview - -### Journey of a Scrobble - -First, let's recap the lifecycle of a scrobble in multi-scrobbler: - -**Sources** are the beginning of the journey for a **Play** (song you've listened to long enough to be scrobblable) - -* A Source finds a new valid **Play** -* The Source **compares** this new Play to all the other Plays it has already seen, if the Play is unique (title/artist/album/listened datetime) then... -* The Source **discovers** the Play, adds it to Plays it has seen already, and broadcasts the Play should be scrobbled to all Clients - -Scrobble **Clients** listen for discovered Plays from Sources, then... - -* A Client receives a **Play** from a Source -* The Client **compares** this Play to all the other scrobbles it has already seen, if the Play is unique (title/artist/album/listened datetime) then... -* The Client **scrobbles** the Play downstream to the scrobble service and adds it as a Scrobble it has seen already - -### Lifecyle Hooks - -You'll notice there is a pattern above that looks like this: - -* **Before** data is compared -* Data is **compared** -* **After** data is compared - -These points, during both Source and Client processes, are when you can hook into the scrobble lifecycle and modify it. - -#### TLDR - -In more concrete terms this is the structure of hooks within a configuration (can be used in any **Source** or **Client**): - -```json5 title="lastfm.json" {10-14} -[ - { - "name": "myLastFm", - "enable": true, - "configureAs": "source", - "data": { - // ... - }, - "options": { - "playTransform": { - "preCompare": {/* ... */}, - "compare": {/* ... */}, - "postCompare": {/* ... */} - } - } - } -] -``` - -##### Hook - -For **Sources**: - -* `preCompare` - modify Play data immediately when received -* `compare` - temporarily modify Play data when it is being compared to see if Play was already discovered -* `postCompare` - modify Play data before sending to scrobble **Clients** - -For **Clients**: - -* `preCompare` - modify Play data immediately when received -* `compare` - temporarily modify Play data when it is being compared to see if it was already scrobbled -* `postCompare` - modify Play data before scrobbling it to downstream service and adding to already seen scrobbles - -:::tip - -Keep in mind that modifying Scrobble/Play data earlier in the lifecycle will affect that data at all times later in the lifecycle. - -For example, to modify the track so it's the same anywhere it is processed in multi-scrobbler you only need to modify it in the **Source's** `preCompare` hook because all later processes will receive the data with the modified track. - -::: - -### Modification Parts - - -Each [**hook**](#hook) (`preCompare` etc...) is an object that specifies what part of the **Play** to modify: - -```json5 -{ - "title": [/* ... */], - "artists": [/* ... */], - "album": [/* ... */] -} -``` - -##### Expression - -and then a **list** what pattern/replacements (expressions) to use for the modification by using either simple strings or `search-replace` objects: - -```json5 -[ - "badTerm", // remove all instances of 'badTerm' - { - "search": "anotherBadTerm", // and also match all instances of 'anotherBadTerm' - "replace": "goodTerm" // replace with the string 'goodTerm' - } -] -``` - -Putting it all together: - -```json5 title="lastfm.json" -[ - { - "name": "myLastFm", - "enable": true, - "configureAs": "source", - "data": { - // ... - }, - "options": { - "playTransform": { - "preCompare": { - "title": [ - "badTerm", - { - "search": "badTerm", - "replace": "goodTerm" - } - ] - }, - } - } - } -] -``` - -:::note - -If the value of the field (title, an artist, album) is an empty string after transforming then the field is **removed.** - -::: - -:::tip - -Modifications can also be applied to **all Sources** or **all Clients** when using the [AIO Config](./configuration.mdx?configType=aio#configuration-types) `config.json` by setting `playTransform` in `sourceDefaults` or `clientDefaults`: - -
- - Example -```json5 title="config.json" -{ - "sourceDefaults": { // will apply playTransform to all sources - "playTransform": { - "preCompare": { - "title": [ - "(Album Version)" - ] - } - } - }, - "sources": [/* ... */], - "clients": [/* ... */] -} -``` -
- -::: - -#### Compare Hook - -The `compare` [hook](#hook) is slightly different than `preCompare` and `postCompare`. It consists of an object where you define which side(s) of the comparison should be modified. It also **does not modify downstream data!** Instead, the modifications are made only for use in the comparison. - -```json5 title="lastfm.json" -[ - { - "name": "myLastFm", - // ... - "options": { - "playTransform": { - "compare": { - "candidate": {/* ... */}, // modify the "new" Play being compared - "existing": {/* ... */}, // modify all "existing" Play/Scrobbles the new Play is being compared against - }, - } - } - } -] -``` - -#### Regular Expressions - -In addition to plain strings [expressions](#expression) that are matched and removed you can also use Regular Expressions. Write your regex like you normally would, but as a string, and it'll automatically be parsed: - -```json5 -[ - "/^\(\w+.com)/i", // matches any string that starts with '(YourMusic.com)' and removes it - { - "search": "/^\(\w+.com)/i", // matches any string that starts with '(YourMusic.com)' - "replace": "[MySite.com]" // replace with the string '[MySite.com]' - } -] -``` - -The `replace` property uses javascript's [`replace()` function and so can use any special string characters.](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace#specifying_a_string_as_the_replacement) - -### Conditional Modification - -#### "When" Condition - -Top-level hooks **and** individual rules also support a `when` key for testing **if they should be run.** - -The `when` key is similar to a normal [modification](#modification-parts) except: - -* the keys accept a single string instead of an array -* the `when` key data is an array instead of a single object - -All parts of an individual `when` clause must test true to "pass" but if **any** `when` clauses pass the hook/rule is processed. Example `when` data: - -```json5 -{ - "when": [ - { - "artist": "Elephant Gym", // both of these must match the Play object (AND) - "album": "Dreams" // both of these must match the Play object (AND) - }, - // OR - { - "title": "/(Remastered)$/", // both of these must match the Play object (AND) - "album": "Various Artists" // both of these must match the Play object (AND) - } - ] -} -``` - -More succinctly: - -* All parts (`artist` `album` `title`) of a `when` are `AND` conditions -* All part-objects in the `when` array are `OR` conditions - -
- -Example of top-level hook with when condition - -```json5 -{ - // IF the artist is Elephant Gym - // THEN Run preCompare hook ELSE skip this hook - // - // Run search-replace on album - // Run regex title remove - "sourceDefaults": { - "playTransform": { - "preCompare": { - "when": [ - { - "artist": "/Elephant Gym/" - } - ], - "album": [ - { - "search": "Dreams", - "replace": "夢境" - } - ], - "title": ["/\s\-\s滾石40\s滾石撞樂隊\s40團拚經典(.+)$/i"] - }, - } - } -} -``` - -
- -
- -Example of individual rule with when condition - -```json5 -{ - // Always run preCompare - // - // On search-replace in title... - // IF artist matches "Elephant Gym" - // THEN Run regex search-replace ELSE skip this rule - // - // Run live|remastered regex remove - "sourceDefaults": { - "playTransform": { - "preCompare": { - "title": [ - { - "search": "/\\s\\-\\s滾石40\\s滾石撞樂隊\\s40團拚經典(.+)$/i", - "replace": "", - "when": [ - { - "artist": "/Elephant Gym/" - } - ] - }, - "/(\\s\\-\\s|\\s)(feat\\.(.+)|live|remastered(.+))$/i" - ], - } - } - } -} -``` - -
- -#### Top-level Hook array - -Top-level hooks can also be an array of hooks. This makes creating multiple scenarios for top-level `when`-gated hooks easier. All hooks in the array will be run (assuming their `when`'s pass, if they exist) and their **input will be the Play object output of the previous hook in the array.** - -
- -Example - -```json5 -{ - "sourceDefaults": { - "playTransform": { - "preCompare": [ - // first lifecycle hook of preCompare to run - { - "title": [ - { - "search": "something", - "replace": "else unique" - } - ] - }, - // second lifecycle hook of preCompare to run - { - "title": [ - { - "search": "else unique", - "replace": "very demure" - } - ] - }, - ] - } - } -} -``` - -
- -### Logging - -MS can log the output of hook transformations if/when they occur. In the `playTransform` object of a Source/Client config use `log`: - -* `"log": true` => Output original play + final transformed output of last hook in the array -* `"log": "all"` => Output original play + final transformed output of **each** hook in the array - -```json5 -{ - "name": "myThing", - "data": {/*...*/}, - "options": { - "playTransform": { - "preCompare": {/*...*/}, - "log": true - } - } -} -``` - -## Examples - -### Remove phrase from Title in all new Plays - -Removes the phrase `(Album Version)` from the Title of a Play - - -
- - Example -```json5 title="config.json" -{ - "sourceDefaults": { - "playTransform": { - "preCompare": { - "title": [ - "(Album Version)" - ] - } - } - } -} - -``` -
- -### Remove all parenthesized content from the end of a title - -
- - Example -```json5 title="lastfm.json" -[ - { - "name": "myLastFm", - // ... - "options": { - "playTransform": { - "compare": { - "candidate": { - "title": [ - "/(\(.+\))\s*$/" - ] - }, - "existing": { - "title": [ - "/(\(.+\))\s*$/" - ] - }, - }, - } - } - } -] -``` -
- -### Rename misspelled artist in all new Plays - -
- - Example -```json5 title="config.json" -{ - "sourceDefaults": { - "playTransform": { - "preCompare": { - "artists": [ - { - "search": "Boz Skaggs", - "replace": "Boz Scaggs" - } - ] - } - } - } -} -``` -
- -### Remove "Various Artists" albums in all new Plays - -
- - Example -```json5 title="config.json" -{ - "sourceDefaults": { - "playTransform": { - "preCompare": { - "album": [ - { - "search": "Various Artists", - "replace": "" - } - ] - } - } - } -} -``` -
- -### Extract primary Artist from delimited, multi-Artist string - -
- - When the Artist string is actually a multi-artist, delimited string, this search-and-replace will replace the string with just the first artist found. - - Ex - - ``` - My Artist One / My Artist Two / Another Guy - My Artist One - ``` - - Artists are delimited with a spaced forward slash (`/`) in the regex below. Replace the contents of the `delim` capture group with the delimiter for your use case. Some more common scenarios: - - * `(?\\/)` No spaces between slash IE `My Artist One/My Artist Two/Another Guy` - * `(?\\s*\\\\\s*)` Backslash instead of forward slash IE `My Artist One \ My Artist Two \ Another Guy` - * `(?,)` Comma IE `My Artist One, My Artist Two, Another Guy` - -
- - Example - ```json5 title="config.json" - { - "sourceDefaults": { - "playTransform": { - "preCompare": { - "artists": [ - { - "search": "(.*?)(?\\s*\\/\\s*)(.*$)", - "replace": "$1" - } - ] - } - } - } - } - ``` -
- -
\ No newline at end of file diff --git a/docsite/docs/configuration/transforms/_category_.json b/docsite/docs/configuration/transforms/_category_.json new file mode 100644 index 00000000..f84f8c7f --- /dev/null +++ b/docsite/docs/configuration/transforms/_category_.json @@ -0,0 +1,7 @@ +{ + "label": "Enhance Scrobbles", + "link": { + "type": "doc", + "id": "configuration/transforms/transforms" + } +} diff --git a/docsite/docs/configuration/transforms/native.mdx b/docsite/docs/configuration/transforms/native.mdx new file mode 100644 index 00000000..650c7fa0 --- /dev/null +++ b/docsite/docs/configuration/transforms/native.mdx @@ -0,0 +1,174 @@ +--- +title: Native Stage +toc_min_heading_level: 2 +toc_max_heading_level: 5 +--- + +The **Native** [Stage](/configuration/transforms#stage) uses [built-in heuristics](https://github.com/FoxxMD/multi-scrobbler/blob/master/src/backend/tests/plays/playParsing.test.ts) to try to extract Artists from Play artist/track data. + +This Stage is most useful for Sources that report limited data such as: + +* [Subsonic](/configuration/sources/subsonic) - Reports Artists as a single string + +A non-exhaustive list of heuristics: + +* Splits artists in artist string using common delimiters EX `Foo Artist, Bar Guy, Baz Band - My Cool Song` + * Does not split artists with `&` in name, if other delimiters are present + * Does not split artist name when only one delimiter is present +* Splits artists on common joiner phrases (ft. feat. vs. etc...) + * Extracts artists from Play title using joiner phrases EX `My Cool Song (feat. SomeGuy)` + + +## Configuration + +Available properties for [Stage Configuration](/configuration/transforms#configuring-stages): + + +* `delimitersExtra` - A list of string characters that should be considered delimiters for artists **in addition to** multi-scrobbler's default list (`, / \ `) +* `delimiters` - A list of string characters that should be considered delimiters for artists. + * **Replaces** all delimiters (MS not use any defaults, only what you give it) +* `artistsIgnore` - a list of strings and/or regular expressions. Any monolothic artist string that matches from the list _will not be modified._ +* `artistsParseFrom` a list of the properties that should be used to try to extract artists. Can be `artists` `title` or both. Defaults to both when not provided in options ( `["artists", "title"]` ) + * When `artists` is present it tries to extract additional artists from artist strings + * When `title` is present it tries to extract artists from ft. feat. vs. etc... found in the track title + * Importantly, if `artists` is _not_ present in `artistsParseFrom` then _no artists_ are used at all (only those from `title`, if present) +* `artistsParseMonolithicOnly` - boolean value, defaults to `true`. When `true` native tranformer will only attempt to extract artists if the scrobble data has _only one string for artist_ (pre-transform) + * This means that, for Sources like Spotify/Jellyfin/etc. that provide proper lists of artists in their data _and the list has more than one string_, it will not try to extract artists from their strings + +
+ +Example of the behavior for `artistsParseFrom` + +``` +The Foos, The Bars - My Cool Track (ft. Frank) +``` + +Config => extracted artists + +``` +"artistsParseFrom": ["artists", "title"] => The Foos, The Bars, Frank +"artistsParseFrom": ["artists"] => The Foos, The Bars +"artistsParseFrom": ["title"] => Frank +"artistsParseFrom": [] => +``` + +
+ +### Rules + +Each [Rule](/configuration/transforms#stage-rules) should be either a boolean, specifying if the transformed data should be used for this field, or a [`when` condition.](/configuration/transforms#conditional-moditication): + +```json5 +{ + "type": "native", + // ... + "title": false, // will not apply any changes to Play title + "artists": { + "when": {/* ... */}, // will only apply changes to Play artists if "when" is satisfied + /* ... */ + }, + "album": true // will always apply changes to Play album +} +``` + +If a rule is not present then multi-scrobbler defaults it to `true`. + +## Examples + +### Parse only artists string using a custom delimiter + +
+ +Example + +Your [AIO Config](/configuration?configType=aio#configuration-types): + +```json5 title="config.json" +{ + // ... + "transformers": [ + { + "type": "native", + "name": "MyNativeTransformer", + "defaults": { + // default delimiters when this Stage is used in a hook + "delimiters": [ + "•" + ], + // default delimiters when this Stage is used in a hook + "artistsParseFrom": ["artists"] + } + } + ] +} +``` + +In a [Subsonic](/configuration/sources/subsonic) [File Config](/configuration?configType=file#configuration-types): + +```json5 title="subsonic.json" +[ + { + "name": "MySubsonic", + "data": { /* ... */}, + "options": { + "playTransform": { + "preCompare": [ + { + "type": "native", + "name": "MyNativeTransformer" + } + ] + } + } + } +] +``` +
+ +### Don't parse specific artists + +
+ +Example + +Your [AIO Config](/configuration?configType=aio#configuration-types): + +```json5 title="config.json" +{ + // ... + "transformers": [ + { + "type": "native", + "name": "MyNativeTransformer", + "defaults": { + "artistsIgnore": [ + "Crosby, Stills, Nash & Young", + "Polo & Pan" + ] + } + } + ] +} +``` + +In a [Subsonic](/configuration/sources/subsonic) [File Config](/configuration?configType=file#configuration-types): + +```json5 title="subsonic.json" +[ + { + "name": "MySubsonic", + "data": { /* ... */}, + "options": { + "playTransform": { + "preCompare": [ + { + "type": "native", + "name": "MyNativeTransformer" + } + ] + } + } + } +] +``` +
\ No newline at end of file diff --git a/docsite/docs/configuration/transforms/transforms.mdx b/docsite/docs/configuration/transforms/transforms.mdx new file mode 100644 index 00000000..f7cbae79 --- /dev/null +++ b/docsite/docs/configuration/transforms/transforms.mdx @@ -0,0 +1,475 @@ +--- +sidebar_position: 4 +title: Enhancing Scrobbles +toc_max_heading_level: 5 +--- + +Multi-scrobbler configs support the ability to enhance scrobble data in an automated fashion by matching and replacing strings in **title, artists, and album** at many different times in multi-scrobbler's lifecycle. + +
+ +Why Would I Do This? + +You may need to "clean up" data from a Source or before sending to a scrobble Client due to any number of reasons: + +* ID3 tags in your music collection are dirty or have repeating garbage IE `[YourMusicSource.com] My Artist - My Title` +* A Source's service often incorrectly adds data to some field IE `My Artist - My Title (Album Version)` when the title should just be `My Title` +* An Artist you listen to often is spelled different between a Source and a Client which causes duplicate scrobbles + +In any scenario where a repeating pattern can be found in the data it would be nice to be able to fix it before the data gets downstream or to help prevent duplicate scrobbling. Multi-scrobbler can help you do this. + +
+ +## Journey of a Scrobble + +First, let's recap the lifecycle of a scrobble in multi-scrobbler: + +**Sources** are the beginning of the journey for a **Play** (song you've listened to long enough to be scrobblable) + +* A Source finds a new valid **Play** +* The Source **compares** this new Play to all the other Plays it has already seen, if the Play is unique (title/artist/album/listened datetime) then... +* The Source **discovers** the Play, adds it to Plays it has seen already, and broadcasts the Play should be scrobbled to all Clients + +Scrobble **Clients** listen for discovered Plays from Sources, then... + +* A Client receives a **Play** from a Source +* The Client **compares** this Play to all the other scrobbles it has already seen, if the Play is unique (title/artist/album/listened datetime) then... +* The Client **scrobbles** the Play downstream to the scrobble service and adds it as a Scrobble it has seen already + +## Lifecyle Hooks + +You'll notice there is a pattern above that looks like this: + +* **Before** data is compared +* Data is **compared** +* **After** data is compared + +These points, during both Source and Client processes, are when you can hook into the scrobble lifecycle and modify it. + +##### TLDR + +In more concrete terms this is the structure of hooks within a configuration (can be used in any **Source** or **Client**): + +```json5 title="lastfm.json" {10-14} +[ + { + "name": "myLastFm", + "enable": true, + "configureAs": "source", + "data": { + // ... + }, + "options": { + "playTransform": { + "preCompare": [/* ... */], + "compare": [/* ... */], + "postCompare": [/* ... */] + } + } + } +] +``` + +### Hook + +For **Sources**: + +* `preCompare` - modify Play data immediately when received +* `compare` - temporarily modify Play data when it is being compared to see if Play was already discovered +* `postCompare` - modify Play data before sending to scrobble **Clients** + +For **Clients**: + +* `preCompare` - modify Play data immediately when received +* `compare` - temporarily modify Play data when it is being compared to see if it was already scrobbled +* `postCompare` - modify Play data before scrobbling it to downstream service and adding to already seen scrobbles + +:::tip + +Keep in mind that modifying Scrobble/Play data earlier in the lifecycle will affect that data at all times later in the lifecycle (except when using the **compare** hook). + +For example, to modify the track so it's the same anywhere it is processed in multi-scrobbler you only need to modify it in the **Source's** `preCompare` hook because all later processes will receive the data with the modified track. + +::: + +
+ +Using `compare` hook + +The `compare` [hook](#hook) is slightly different than `preCompare` and `postCompare`. It consists of an object where you define which side(s) of the comparison should be modified. It also **does not modify downstream data!** Instead, the modifications are made only for use in the comparison. + +```json5 title="lastfm.json" +[ + { + "name": "myLastFm", + // ... + "options": { + "playTransform": { + "compare": [ + { + "candidate": {/* ... */}, // modify the "new" Play being compared + "existing": {/* ... */}, // modify all "existing" Play/Scrobbles the new Play is being compared against + } + ], + } + } + } +] +``` + +
+ +## Modification Stage {#stage} + +Each [**hook**](#hook) is made up of one or more **Stages**. A Stage is a self-contained, unique way of enhancing or modifying the Play data. Some examples of a Stage: + +* The [User](/configuration/transforms/user) Stage allows a user to define search-and-replace terms for Artist/Title/Album +* The [Native](/configuration/transforms/native) Stage uses MS's built-in heuristics to extract Artists from a single Artist string +* The Musicbrainz Stage tries to match Play data with the Musicbrainz database and to standardize the Artist/Title/Album data + +Each Stage in a Hook receives Play data from the previous Stage. + +Within a hook, each Stage minimally consists of a `type` to identify what Stage it is along with any other data specific to that stage: + +```json5 +{ + "type": "native" + // optional, stage specific data here... +} +``` + +### Configuring Stages {#configuring-stages} + +Stages may be globally configured using [AIO Config](/configuration?configType=aio#configuration-types) `config.json` file in the top-level `transformers` block. + +Each Stage consists of: + +* `type` the type of Stage +* `name` a unique name for the Stage, to be (potentially) used with hooks +* `defaults` - An object defining default configuration for this stage, when used in a Hook. +* `data` - An object containing any data required to initially configure the stage itself (Example: API URL, username, password, etc...) + +
+ +Example + +Your [AIO Config](/configuration?configType=aio#configuration-types): + +```json5 title="config.json" +{ + // ... + "transformers": [ + { + "type": "native", + "name": "MyNativeTransformer", + "defaults": { + // default delimiters when this Stage is used in a hook + "delimiters": [ + "•" + ], + // default delimiters when this Stage is used in a hook + "artistsParseFrom": ["artists"] + } + } + ] +} +``` + +In a [Subsonic](/configuration/sources/subsonic) [File Config](/configuration?configType=file#configuration-types): + +```json5 title="subsonic.json" +[ + { + "name": "MySubsonic", + "data": { /* ... */}, + "options": { + "playTransform": { + "preCompare": [ + { + "type": "native" + // when "name" is not defined, uses first found "native" transformer + } + ] + } + } + } +] +``` + + +
+ +Multiple stages of the same type may be configured, allowing you to define several sets of default behavior. + +
+ +Example + +Your [AIO Config](/configuration?configType=aio#configuration-types): + +```json5 title="config.json" +{ + // ... + "transformers": [ + { + "type": "native", + "name": "DotTransformer", + "defaults": { + "delimiters": [ + "•" + ], + "artistsParseFrom": ["artists"] + } + }, + { + "type": "native", + "name": "TitleOnly", + "defaults": { + // extracts and uses *only* artists found in title string + "artistsParseFrom": ["title"] + } + } + ] +} +``` + +In a [Subsonic](/configuration/sources/subsonic) [File Config](/configuration?configType=file#configuration-types): + +```json5 title="subsonic.json" +[ + { + "name": "MySubsonic", + "data": { /* ... */}, + "options": { + "playTransform": { + "preCompare": [ + { + "type": "native" + "name": "DotTransformer" + } + ] + } + } + } +] +``` + +In a [VLC](/configuration/sources/vlc) [File Config](/configuration?configType=file#configuration-types): + +```json5 title="vlc.json" +[ + { + "name": "MyVLC", + "data": { /* ... */}, + "options": { + "playTransform": { + "preCompare": [ + { + "type": "native" + "name": "TitleOnly" + } + ] + } + } + } +] +``` + +
+ +#### Overriding Configuration + +The default configuration you set for your Stage may be overridden in any usage of the Stage within a Hook. + +
+ +Example + +Your [AIO Config](/configuration?configType=aio#configuration-types): + +```json5 title="config.json" +{ + // ... + "transformers": [ + { + "type": "native", + "name": "MyNativeTransformer", + "defaults": { + // default delimiters when this Stage is used in a hook + "delimiters": [ + "•" + ], + // default delimiters when this Stage is used in a hook + "artistsParseFrom": ["artists"] + } + } + ] +} +``` + +In a [Subsonic](/configuration/sources/subsonic) [File Config](/configuration?configType=file#configuration-types): + +```json5 title="subsonic.json" +[ + { + "name": "MySubsonic", + "data": { /* ... */}, + "options": { + "playTransform": { + "preCompare": [ + { + "type": "native" + "name": "MyNativeTransformer", + // overrides property from "defaults" + "artistsParseFrom": ["artists", "title"] + } + ] + } + } + } +] +``` + +
+ + +### Rules for Play Data {#stage-rules} + +Each [Stage](#stage) may specify whether it should apply the resulting transformation to different parts of the Play data by specifying `title`, `artists` and/or `album` in the Stage object. + +```json5 +{ + "type": "native", + // ... + "title": false, // will not apply any changes to Play title + "artists": { + "when": {/* ... */}, // will only apply changes to Play artists if "when" is satisfied + /* ... */ + }, + "album": true // will always apply changes to Play album +} +``` + +The actual value of each property may be different for each Stage. Check the docs for the Stage you want to use to see its usage of `title`, `artists`, and `album`. + +Generically, though, each property may be some value **or** an object combining a [`when` condition](#conditional-modification) and that value. + +If none of the properties are specified in the stage then it's assumed all transformed data should be used. + +:::note + +Specifying these Rules is **not** the same as [configuring the Stage](#configuring-stages). Rules only determine if the *result* of the transformation should be used (replace) the existing Play Data. + +::: + +## Conditional Modification + +[Stages](#stage) within a [Hook](#hook), and [Rules](#stage-rules) within each Stage, support a `when` object for testing **if they should be run.** + +The `when` object may have propertes for `artist`, `title` and/or `album`. Each property may be a string or regular expression. The value of the property is used to match the **pre-transformation** values from Play data. + +All parts of an individual `when` clause must test true to "pass" but if **any** `when` clauses pass the Stage/Rule is processed. + +```json5 +{ + "when": + { + "artist": "Elephant Gym", // Play must have an artist matching "Elephant Gym" (AND) + "album": "Dreams" // Play object must have an album matching "Dreams" (AND) + } +} +``` + +```json5 +{ + "when": [ + { + "artist": "Elephant Gym", // Play must have an artist matching "Elephant Gym" (AND) + "album": "Dreams" // Play object must have an album matching "Dreams" (AND) + }, + // OR + { + "title": "/(Remastered)$/", // Play title must match regular expression (AND) + "album": "Various Artists" // Play album must match "Various Artists" (AND) + } + ] +} +``` + +More succinctly: + +* All parts (`artist` `album` `title`) of a `when` are `AND` conditions +* All part-objects in the `when` array are `OR` conditions + +
+ +Example of Stage with `when` condition + +```json5 +{ + // IF the artist is Elephant Gym + // THEN Run native stage + "playTransform": { + "preCompare": [ + { + "type": "native", + "when": [ + { + "artist": "/Elephant Gym/" + } + ] + } + ], + } +} +``` + +
+ +
+ +Example of individual rule with when condition + +```json5 +{ + // Always run native Stage + // + // IF artist matches "Elephant Gym" + // THEN use result of native stage for "artists" of Play data + "playTransform": { + "preCompare": { + "type": "native", + "artists": + { + "when": [ + { + "artist": "/Elephant Gym/" + } + ] + }, + } + } +} +``` + +
+ +## Logging + +MS can log the output of Stage transformations if/when they occur. In the `playTransform` object of a Source/Client config use `log`: + +* `"log": true` => Output original play + final transformed output of last Stage in the array +* `"log": "all"` => Output original play + final transformed output of **each** Stage in the array + +```json5 +{ + "name": "myThing", + "data": {/*...*/}, + "options": { + "playTransform": { + "preCompare": {/*...*/}, + "log": true + } + } +} +``` \ No newline at end of file diff --git a/docsite/docs/configuration/transforms/user.mdx b/docsite/docs/configuration/transforms/user.mdx new file mode 100644 index 00000000..7492f8a7 --- /dev/null +++ b/docsite/docs/configuration/transforms/user.mdx @@ -0,0 +1,275 @@ +--- +title: User Stage +toc_min_heading_level: 2 +toc_max_heading_level: 5 +--- + +The **User** [Stage](/configuration/transforms#stage) uses [**search-and-replace** expressions](#search-and-replace-expression), provided by you, to modify/replace parts of Play data. + +This Stage is most useful for correcting individual instances of bad data, or patterns, in your known data. Example scenarios: + +* Removing `(Album Version)` from all Titles +* Removing `Various Artists` from Artist data +* Correcting spelling mistakes for individual Artist names + +The user [Stage `type`](/configuration/transforms#stage) is `user`. + +## Configuration + +All [Stage Configuration](/configuration/transforms#configuring-stage) is done using [**search-and-replace** expressions](#search-and-replace-expression) inside individual [rules](#rules). Default configuration for all Rules can still be done using Stage `defaults`. + +
+ +Example + +```json5 title="config.json" +{ + // ... + "transformers": [ + { + "type": "user", + "name": "Normalizer", + "defaults": { + "title": [ + "(Album Version)" + ], + "artists": [ + "Various Artists" + ] + } + } + ] +} +``` + +
+ +### Search-And-Replace Expression + +A Search-And-Replace Expression can be a plain string that matches a literal, then removes it: + +``` +Expression: "badTerm" + +"this is badTerm cool string" => "this is a cool string" +``` + +or a regular expression that matches and removes the match: + +``` +Expression: "/bad\w+/i" + +"this is badSomething cool string" => "this is a cool string" +``` + +Or it may be an object that specifies what to match (using either plain string or regular expression) and what to replace it with: + +```json5 +{ + "search": "anotherBadTerm", // match all instances of 'anotherBadTerm' + "replace": "goodTerm" // replace with the string 'goodTerm' +} +``` + +``` +"this is anotherBadTerm cool string" => "this is goodTerm cool string" +``` + +```json5 +{ + "search": "/^\(\w+.com)/i", // matches any string that starts with EX '(YourMusic.com)' + "replace": "[MySite.com]" // replace with the string '[MySite.com]' +} +``` + +``` +"(Foo.com) this is a cool string" => "[MySite.com] this is a cool string" +``` + +The `replace` property uses javascript's [`replace()` function and so can use any special string characters.](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace#specifying_a_string_as_the_replacement) + +### Rules + +Each [Rule](/configuration/transforms#stage-rules) must be an array of [search-and-replace expressions](#search-and-replace-expressions): + +```json5 title="lastfm.json" +[ + { + "name": "myLastFm", + "configureAs": "source", + "data": { + // ... + }, + "options": { + "playTransform": { + "type": "user", + "preCompare": [ + { + "title": [ + // removes "badTerm" from title + "badTerm", + { + // removes "fooTerm" from title and replaces it with "barTerm" + "search": "fooTerm", + "replace": "barTerm" + } + ] + } + ], + } + } + } +] +``` + +:::note + +If the value of the field (title, an artist, album) is an empty string after transforming then the field is **removed.** + +::: + + +## Examples + +### Usage with `when` condition + +Using `when` for [Conditional Modification](/configuration/transforms#conditional-modification) +
+ +Example + +```json5 +// On search-replace in title... +// IF artist matches "Elephant Gym" +// THEN Run regex search-replace ELSE skip this rule +// +// Run live|remastered regex remove on title +{ + "title": [ + { + "search": "/\\s\\-\\s滾石40\\s滾石撞樂隊\\s40團拚經典(.+)$/i", + "replace": "", + "when": [ + { + "artist": "/Elephant Gym/" + } + ] + }, + "/(\\s\\-\\s|\\s)(feat\\.(.+)|live|remastered(.+))$/i" + ], +} +``` + +
+ +### Remove phrase from Title in all new Plays + +Removes the phrase `(Album Version)` from the Title of a Play + + +
+ +Example + +```json5 +{ + "title": [ + "(Album Version)" + ] +} + +``` +
+ +### Remove all parenthesized content from the end of a title + +
+ + Example +```json5 +{ +"compare": { + "candidate": { + "title": [ + "/(\(.+\))\s*$/" + ] + }, + "existing": { + "title": [ + "/(\(.+\))\s*$/" + ] + }, + }, +} +``` +
+ +### Rename misspelled artist in all new Plays + +
+ + Example +```json5 +{ + "artists": [ + { + "search": "Boz Skaggs", + "replace": "Boz Scaggs" + } + ] +} +``` +
+ +### Remove "Various Artists" albums in all new Plays + +
+ + Example +```json5 +{ + "album": [ + { + "search": "Various Artists", + "replace": "" + } + ] +} +``` +
+ +### Extract primary Artist from delimited, multi-Artist string + +
+ + When the Artist string is actually a multi-artist, delimited string, this search-and-replace will replace the string with just the first artist found. + + Ex + + ``` + My Artist One / My Artist Two / Another Guy + My Artist One + ``` + + Artists are delimited with a spaced forward slash (`/`) in the regex below. Replace the contents of the `delim` capture group with the delimiter for your use case. Some more common scenarios: + + * `(?\\/)` No spaces between slash IE `My Artist One/My Artist Two/Another Guy` + * `(?\\s*\\\\\s*)` Backslash instead of forward slash IE `My Artist One \ My Artist Two \ Another Guy` + * `(?,)` Comma IE `My Artist One, My Artist Two, Another Guy` + +
+ + Example +```json +{ + "artists": [ + { + "search": "(.*?)(?\\s*\\/\\s*)(.*$)", + "replace": "$1" + } + ] +} +``` +
+ +
\ No newline at end of file