Thomas Lawson Structured Catalog — Plan #
Current State #
The scraper (v2) uses the WP REST API (/wp-json/wp/v2/pages/{id}) to fetch
rendered Elementor page content rather than scraping raw HTML. It handles two
Elementor caption patterns:
- Pattern A (1977-1979):
<h5><i>Title</i>, YYYY<br>medium<br>dimensions</h5> - Pattern B (all other pages):
<h5>TITLE</h5>+<h6>YEAR</h6>as separate widgets
Results so far:
- 213 artworks, 13 exhibitions — all pass schema validation
- 77% have year, but only 7% have medium/dimensions (only the 1977-1979 page embeds that data in the HTML)
- 0% have description or collection info
Full WP page inventory (51 pages):
- 11 In the Studio pages (all scraped)
- 8 Beyond the Studio pages (all scraped)
- 13 Art in a Broader Context / exhibition pages (all scraped)
- 7 Bookshelf pages (writings — NOT yet scraped)
- 1 Notes page, 1 About, 1 Contact, 1 Home, etc.
Schema Alignment with Schema.org VisualArtwork #
Current schema maps well to Schema.org but should add a few fields for completeness:
| Our field | Schema.org equivalent | Status |
|---|---|---|
title |
name |
done |
year |
dateCreated |
done |
medium |
artMedium |
done |
dimensions |
width / height / depth |
done |
type |
artform |
done |
images |
image |
done |
| (missing) | artworkSurface |
add |
| (missing) | artEdition |
add |
| (missing) | provenance |
add |
collection |
(custom) | done |
description |
description |
done |
Plan — 4 Steps #
Step 1: Enhance the JSON schema #
Add fields to artwork.schema.json:
artworkSurface— canvas, paper, linen, board, etc. (nullable string)artEdition— edition info for prints/multiples (nullable string)provenance— array of{ owner, acquired, notes }objectswritings— array of IDs linking to a newwriting.schema.json- Split
mediumintoartMedium(materials) + keepmediumas full freeform string
Add a new writing.schema.json for Bookshelf entries:
id,title,slug,year,publication,url,type(essay/review/interview/anthology),sourceUrl
Update catalog.schema.json to include writings array.
Step 2: Scrape Bookshelf pages #
7 Bookshelf pages to scrape for writing/publication records:
/bookshelf/(index)/bookshelf_afterall/(Afterall journal)/bookshelf_artforum/(Artforum)/bookshelf_eastofborneo/(East of Borneo)/bookshelf_writingsabouttl/(writings about TL by others)/bookshelf-anthologies/(anthologies)/bookshelf-reallife/(REALLIFE Magazine)/elementor-1796/(interviews)/elementor-395/(Considering Other Artists)
These are text-heavy pages with titles, publication names, and years. Parse them
into structured writing records.
Step 3: Enrich artwork data via WP media API + filenames #
The WP media library (/wp-json/wp/v2/media) contains metadata:
- EXIF data (photographer credit, copyright)
- Captions on some images
- Filenames encode dimensions: e.g.
Thomas-Lawson-Flying-brick-2019-18-x-24-in-45.72-x-60.96-cm
Strategy:
- Fetch all media items (paginated, ~200+ items)
- Parse filenames for embedded dimensions (regex:
(\d+)-x-(\d+)-in) - Match media items to artworks by image URL
- Fill in dimensions from filename where not already set from Elementor captions
This could raise medium/dimensions coverage from 7% to ~30-40%.
Step 4: Manual enrichment layer #
Create a data/overrides.json file for manual corrections and additions that
can't be scraped:
- Medium/materials for works where the site doesn't list them
- Collection/owner info
- Exhibition history links
- Description text
- Series assignments beyond auto-detection
The scraper merges overrides.json on top of scraped data, so manual edits
persist across re-scrapes.
Files to create/modify #
| File | Action |
|---|---|
schema/artwork.schema.json |
Update — add artworkSurface, artEdition, provenance |
schema/writing.schema.json |
New — structured writing/publication record |
schema/catalog.schema.json |
Update — add writings array |
scrape.mjs |
Update — add bookshelf scraping, media API enrichment, overrides merge |
validate.mjs |
Update — validate writings, show enrichment stats |
data/overrides.json |
New — manual corrections layer (starts empty) |