diff --git a/docs/internal/specs/14_project_compilation_and_dependencies.md b/docs/internal/specs/14_project_compilation_and_dependencies.md index 06d8214..72514b3 100644 --- a/docs/internal/specs/14_project_compilation_and_dependencies.md +++ b/docs/internal/specs/14_project_compilation_and_dependencies.md @@ -103,6 +103,48 @@ The second layer should load selected Hex and path dependency source modules and compile them through the same pipeline as project modules. This is preferred when the package code uses the supported language subset. +Hex loading should follow the public Hex repository API used by Gleam: + +- package metadata: `GET https://repo.hex.pm/packages/{name}` +- package tarball: `GET https://repo.hex.pm/tarballs/{name}-{version}.tar` +- optional full version index: `GET https://repo.hex.pm/versions` + +Repository metadata responses are gzip-compressed signed protobuf payloads and +must be verified before use. Tarballs should be checksum-verified from the lock +metadata, then cached by checksum under: + +```text +$HOME/.regulus/store/hex/tarballs/{outer-checksum}.tar +``` + +The global registry cache is immutable and shared across projects. Project +builds should extract packages into disposable project-local build directories: + +```text +{project}/build/packages/packages.toml +{project}/build/packages/{package-name}/ +``` + +The outer tarball contains `contents.tar.gz`, which holds source files, +`gleam.toml`, and package metadata. The compiler should extract that inner +archive into the project package directory. A small stamp file in each extracted +package should record name, version, checksum, source, and cache schema version. +If the stamp does not match, the package directory should be replaced. + +Path dependencies should be loaded from their declared path without network +access. The build should make clear progress output when resolving, downloading, +using cached packages, and extracting sources: + +```text +Resolving dependencies +Downloading gleam_stdlib 0.50.0 +Using cached gleam_erlang 1.3.0 +Extracting gleam_stdlib 0.50.0 +``` + +Quiet output should stay stable. `--verbose` may print source URLs, checksums, +cache paths, package build paths, and source paths. + ### Unsupported dependencies Unsupported dependency members, modules, syntax, runtime primitives, or ABI diff --git a/docs/internal/tasks/14_project_compilation_and_dependencies.md b/docs/internal/tasks/14_project_compilation_and_dependencies.md index be10865..67d4316 100644 --- a/docs/internal/tasks/14_project_compilation_and_dependencies.md +++ b/docs/internal/tasks/14_project_compilation_and_dependencies.md @@ -87,6 +87,21 @@ The goal here is to make linked project names deterministic and collision-free. ### Dependency source loading - [ ] Define selected Hex and path dependency source loading rules. +- [ ] Fetch Hex package metadata from `repo.hex.pm/packages/{name}`. +- [ ] Download Hex tarballs from + `repo.hex.pm/tarballs/{name}-{version}.tar`. +- [ ] Verify signed repository metadata and package tarball checksums. +- [ ] Cache downloaded tarballs by checksum under `$HOME/.regulus/store/hex/tarballs/` + - we should use `dirs = "6.0.0"` to get the home dir +- [ ] Extract `contents.tar.gz` into `{project}/build/packages/{name}/`. +- [ ] Write package extraction stamps with name, version, checksum, source, and + cache schema version. +- [ ] Replace stale extracted package directories when stamps do not match. +- [ ] Load path dependency source without network access. +- [ ] Add CLI progress output: `Resolving dependencies`, `Downloading ...`, + `Using cached ...`, and `Extracting ...`. +- [ ] Keep quiet output deterministic and put URLs, checksums, cache paths, and + source paths behind `--verbose`. - [ ] Add a dependency source loader for selected packages and paths. - [ ] Compile one package module from source through the normal pipeline. - [ ] Link compiled dependency module calls without treating them as host