From 5ea2fc35e9ff2ae38adc4b9efae882b7e21fe6df Mon Sep 17 00:00:00 2001 From: Tsiry Sandratraina Date: Wed, 19 Aug 2026 09:46:50 +0300 Subject: [PATCH] =?UTF-8?q?ci:=20a=20real=20GitHub=20Actions=20runner=20?= =?UTF-8?q?=E2=80=94=20uses:=20steps=20execute,=20not=20translate?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Any JavaScript or composite action now runs. At plan time the action's action.yml is fetched (host-side, cached, fetcher injectable for offline tests) to learn what the action is; at run time the guest clones it at its ref and executes it under the genuine protocol: INPUT_* from with: and defaults (dashes preserved, as the real runner sets them), GITHUB_ENV / GITHUB_PATH / GITHUB_OUTPUT / GITHUB_STATE command files — heredoc spelling included — whose effects persist into every later step through a durable env file each step sources, run: steps included, because that is GitHub's semantics. steps..outputs resolve in the guest at run time, where the outputs live. A node runtime and the tool baseline actions assume (unzip, xz — GitHub's runners preship them) provision once per job. Container actions and pre/post hooks are refused visibly; composite if: conditions are not evaluated (stated limits, not silent ones). Three lessons the live run taught, each now a test: - ${{ github.token }} in an env value must never leave a literal or an empty-but-present bearer behind — both were 401s. Values get strict interpolation (unknown expressions vanish; setup-bun's token default is a && || expression), empties resolve to unset, and an injected --secret GITHUB_TOKEN backs github.token when authentication is wanted. - Interpolate's keep-visible fallback re-emitted the marker it scanned for — an infinite loop; replacements now go through a cursor that never rescans output. - @actions/tool-cache shells out to unzip, which slim images lack. Verified on arm64/macOS: the actual oven-sh/setup-bun@v2 executes end to end — bun downloaded and installed by the action's own JavaScript, GITHUB_PATH written, and the next plain run: step finds bun on PATH. Unit tests cover ref parsing, JS and composite translation, input layering, expression semantics in value vs command position, protocol wrapping, and every refusal reason. e2e gains the example, continue- on-error until x86_64 rules. --- .github/workflows/e2e-ci.yml | 24 +++++++ CHANGELOG.md | 9 +++ ci/README.md | 20 +++++- ci/foreign.go | 5 ++ ci/main.go | 1 + ci/platforms/actions/actions.go | 64 +++++++++++++++---- ci/platforms/actions/actions_test.go | 25 ++++++++ ci/platforms/actions/protocol.go | 14 +++- ci/platforms/github.go | 3 +- ci/platforms/platforms.go | 3 + .../.github/workflows/ci.yml | 15 +++++ examples/ci-github-actions/README.md | 23 +++++++ examples/ci-github-actions/hello.txt | 1 + 13 files changed, 189 insertions(+), 18 deletions(-) create mode 100644 examples/ci-github-actions/.github/workflows/ci.yml create mode 100644 examples/ci-github-actions/README.md create mode 100644 examples/ci-github-actions/hello.txt diff --git a/.github/workflows/e2e-ci.yml b/.github/workflows/e2e-ci.yml index 6027a4e..f616123 100644 --- a/.github/workflows/e2e-ci.yml +++ b/.github/workflows/e2e-ci.yml @@ -217,6 +217,30 @@ jobs: done test -z "$failed" || { echo "FAILED:$failed"; exit 1; } + # The real actions runner: this example executes the actual + # oven-sh/setup-bun@v2 JavaScript action under the Actions protocol, + # and the following run: step's `bun --version` only works if the + # action's GITHUB_PATH write survived into the next step. + - name: e2e — real GitHub Actions (setup-bun executes for real) + continue-on-error: true # flip to false after the first green run on x86_64 + run: | + set -eux + d=$(mktemp -d) + cp -r "examples/ci-github-actions/." "$d" + git -C "$d" init -q + git -C "$d" add -A + git -C "$d" -c user.email=ci@e2e -c user.name=e2e commit -qm init + set +e + timeout 900 ./target/release/bsdkrun ci run --plain -w "$d" \ + --secret GITHUB_TOKEN="${GITHUB_TOKEN}" 2>&1 | tee /tmp/ci-gha.log + rc=${PIPESTATUS[0]} + set -e + grep -q "Provision actions runtime (node)" /tmp/ci-gha.log + grep -q "github-actions-example-ok" /tmp/ci-gha.log + test "$rc" -eq 0 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + # No CI config at all: the runner must detect the project (a bare Go # module here) and generate + run a workflow for it. Asserts the # detection announcement and the test-before-build execution. diff --git a/CHANGELOG.md b/CHANGELOG.md index c459e0b..96b2655 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,15 @@ shipped alongside it. ### Added +- **GitHub Actions `uses:` steps execute for real.** A genuine actions + runner for JavaScript and composite actions: action.yml fetched and + parsed at plan time, the action cloned into the guest at its ref, a node + runtime provisioned once per job, and the real Actions protocol + throughout — `INPUT_*` with expression-aware defaults, GITHUB_ENV / + GITHUB_PATH / GITHUB_OUTPUT command files persisting across steps, and + run-time resolution of step outputs. Container actions and pre/post + hooks are refused visibly. Verified by running the actual + oven-sh/setup-bun@v2 end to end. - **Project detection: CI with no config at all.** When a repository has no recognizable CI configuration (or `--detect` forces it), `bsdkrun ci` detects the project — go, rust, nodejs, bun, deno, python, ruby, php, diff --git a/ci/README.md b/ci/README.md index 8137018..0721d97 100644 --- a/ci/README.md +++ b/ci/README.md @@ -137,9 +137,23 @@ platform's identity env (`GITHUB_SHA`, `CI_PROJECT_DIR`, `DRONE_COMMIT_SHA`, …) pointed at the runner's own workspace. Secrets, the TUI, tracing and the log stream all work the same as for native workflows. -What deliberately does not translate is announced rather than faked: -`uses:` actions (except `actions/checkout`, which the clone genuinely -covers), plugins, orbs, human gates and cross-pipeline triggers become +**GitHub Actions `uses:` steps run for real.** Any JavaScript or composite +action executes: the runner fetches the action's `action.yml` (cached +host-side) to learn what it is, clones it into the guest at its ref, +provisions a node runtime once per job, and executes it under the genuine +Actions protocol — `INPUT_*` from `with:` (defaults included, expression +defaults evaluated where the subset allows and dropped rather than +mistranslated otherwise), and `GITHUB_ENV` / `GITHUB_PATH` / +`GITHUB_OUTPUT` command files whose effects persist into every later step, +`run:` steps included. `${{ steps..outputs.* }}` resolves in the guest, +where the outputs live. Inject `--secret GITHUB_TOKEN` to authenticate +actions that call the GitHub API. Container actions are refused visibly (a +microVM runs no Docker daemon), as are `pre`/`post` hooks — stated limits. +See [`examples/ci-github-actions`](../examples/ci-github-actions), which +runs the actual `oven-sh/setup-bun@v2`. + +What deliberately does not translate elsewhere is announced rather than +faked: plugins, orbs, human gates and cross-pipeline triggers become visible skipped steps in the timeline; matrix strategies run once and say so; **jobs that ask for windows or macos are skipped** — a Linux microVM cannot become another OS, and a green checkmark on a lie helps nobody. diff --git a/ci/foreign.go b/ci/foreign.go index 7d42472..e8c1c00 100644 --- a/ci/foreign.go +++ b/ci/foreign.go @@ -24,9 +24,14 @@ func platformRepo(repo *repoInfo) platforms.Repo { DefaultBranch: repo.DefaultBranch, Name: repo.Name, Workspace: workspaceDir, + Token: ghToken, } } +// ghToken is the operator's GITHUB_TOKEN secret, set by cmdRun before any +// plan is built — what real actions authenticate with. +var ghToken string + // foreignPlans loads the platform's jobs and turns the runnable ones into // plans. Skipped jobs (non-Linux) and name filtering are announced on // stderr, where they stay visible in every output mode. diff --git a/ci/main.go b/ci/main.go index e4017cb..c96735f 100644 --- a/ci/main.go +++ b/ci/main.go @@ -322,6 +322,7 @@ func cmdRun(args []string) error { if err != nil { return err } + ghToken = secrets["GITHUB_TOKEN"] opts := runOpts{ Cpus: *cpus, Mem: *mem, diff --git a/ci/platforms/actions/actions.go b/ci/platforms/actions/actions.go index 6b016bf..dc876eb 100644 --- a/ci/platforms/actions/actions.go +++ b/ci/platforms/actions/actions.go @@ -40,6 +40,9 @@ type Repo struct { Branch string Name string Workspace string + // Token backs `${{ github.token }}` when the operator injected a + // GITHUB_TOKEN secret; empty means unauthenticated. + Token string } // Metadata is the parsed action.yml — the parts execution needs. @@ -165,12 +168,20 @@ func translate(uses, stepID string, with, env map[string]string, repo Repo, dept func resolveInputs(meta *Metadata, with map[string]string, repo Repo) map[string]string { out := map[string]string{} for name, in := range meta.Inputs { - if in.Default != "" { - out[name] = Interpolate(in.Default, nil, repo) + if v := InterpolateValue(in.Default, nil, repo); v != "" { + out[name] = v } } for k, v := range with { - out[k] = Interpolate(v, nil, repo) + out[k] = InterpolateValue(v, nil, repo) + } + // An input that resolved to empty is *unset*, not empty-but-present: + // actions feed inputs straight into auth headers, and an empty bearer + // token is a 401, not an unauthenticated request. + for k, v := range out { + if v == "" { + delete(out, k) + } } return out } @@ -223,7 +234,7 @@ func compositeSteps(uses string, ref Ref, meta *Metadata, stepID string, inputs, stepEnv[k] = v } for k, v := range cs.Env { - stepEnv[k] = Interpolate(v, inputs, repo) + stepEnv[k] = InterpolateValue(v, inputs, repo) } switch { case cs.Uses != "": @@ -283,18 +294,40 @@ func cloneSnippet(ref Ref) string { // Interpolate substitutes the expression lookups real workflows lean on. // `steps..outputs.` resolves in the guest at run time — that is // where the output files live — via command substitution. +// InterpolateValue is Interpolate with *value* semantics: an expression the +// subset cannot evaluate becomes empty rather than a visible marker. The +// marker is right for shell text a human reads; an env value is fed +// straight to code — setup-bun's token default is a && || expression, and +// the marker-as-bearer-token was a 401. Empty is also what the expression +// usually means here: those defaults guard on being on real github.com. +func InterpolateValue(s string, inputs map[string]string, repo Repo) string { + return interpolate(s, inputs, repo, true) +} + func Interpolate(s string, inputs map[string]string, repo Repo) string { + return interpolate(s, inputs, repo, false) +} + +func interpolate(s string, inputs map[string]string, repo Repo, strict bool) string { + // Replacements go to the builder and are never rescanned — the unknown- + // expression fallback re-emits `${{ ... }}` verbatim, which would spin + // an in-place substitution forever. + var b strings.Builder for { start := strings.Index(s, "${{") if start < 0 { - return s + b.WriteString(s) + return b.String() } end := strings.Index(s[start:], "}}") if end < 0 { - return s + b.WriteString(s) + return b.String() } + b.WriteString(s[:start]) expr := strings.TrimSpace(s[start+3 : start+end]) - s = s[:start] + evalExpr(expr, inputs, repo) + s[start+end+2:] + b.WriteString(evalExpr(expr, inputs, repo, strict)) + s = s[start+end+2:] } } @@ -304,12 +337,12 @@ func interpolateMap(m, inputs map[string]string, repo Repo) map[string]string { } out := map[string]string{} for k, v := range m { - out[k] = Interpolate(v, inputs, repo) + out[k] = InterpolateValue(v, inputs, repo) } return out } -func evalExpr(expr string, inputs map[string]string, repo Repo) string { +func evalExpr(expr string, inputs map[string]string, repo Repo, strict bool) string { switch { case strings.HasPrefix(expr, "inputs."): return inputs[strings.TrimPrefix(expr, "inputs.")] @@ -330,7 +363,11 @@ func evalExpr(expr string, inputs map[string]string, repo Repo) string { case expr == "github.repository": return repo.Name case expr == "github.token", expr == "secrets.GITHUB_TOKEN": - return "${GITHUB_TOKEN}" + // The operator's injected GITHUB_TOKEN when present, else empty — + // and empty resolves to *unset* in inputs (see resolveInputs): a + // literal ${GITHUB_TOKEN} and an empty-but-present bearer were each + // a 401 lesson on the way here. + return repo.Token case expr == "runner.os": return "Linux" case expr == "runner.temp": @@ -340,8 +377,11 @@ func evalExpr(expr string, inputs map[string]string, repo Repo) string { case strings.HasPrefix(expr, "github.action_path"): return "${GITHUB_ACTION_PATH}" } - // Anything else (functions, operators) is beyond the subset; keep it - // visible in the command rather than silently emptying it. + // Anything else (functions, operators) is beyond the subset. In value + // position it must become empty; in command text it stays visible. + if strict { + return "" + } return "${{ " + expr + " }}" } diff --git a/ci/platforms/actions/actions_test.go b/ci/platforms/actions/actions_test.go index 6fe443c..6cb029b 100644 --- a/ci/platforms/actions/actions_test.go +++ b/ci/platforms/actions/actions_test.go @@ -171,3 +171,28 @@ func TestWrapStepCarriesTheProtocol(t *testing.T) { } } } + +func TestExpressionDefaultsResolveEmptyInValues(t *testing.T) { + // setup-bun's real token default: an operator expression the subset + // cannot evaluate. In value position it must vanish — the visible + // marker became a bearer token and a 401 once. + fake(t, map[string]string{ + "a/tok": ` +runs: {using: node20, main: index.js} +inputs: + token: + default: ${{ github.server_url == 'https://github.com' && github.token || '' }} + version: {default: latest} +`, + }) + steps, _, err := Translate("a/tok@v1", "s", nil, nil, testRepo) + if err != nil { + t.Fatal(err) + } + if _, present := steps[0].Env["INPUT_TOKEN"]; present { + t.Fatalf("unevaluable default must be unset, got %v", steps[0].Env) + } + if steps[0].Env["INPUT_VERSION"] != "latest" { + t.Fatalf("plain defaults must survive: %v", steps[0].Env) + } +} diff --git a/ci/platforms/actions/protocol.go b/ci/platforms/actions/protocol.go index bb99f23..0657b1b 100644 --- a/ci/platforms/actions/protocol.go +++ b/ci/platforms/actions/protocol.go @@ -104,7 +104,17 @@ func WrapStep(stepID, body string) string { func NodeProvisionStep() Step { return Step{ Name: "Provision actions runtime (node)", - Command: `command -v node >/dev/null 2>&1 && { echo "node $(node --version) already present"; exit 0; } + Command: `# The tool baseline JS actions assume: GitHub's runners preship +# unzip/tar/xz, and @actions/tool-cache shells out to them. +if command -v apt-get >/dev/null 2>&1; then + command -v unzip >/dev/null 2>&1 || { + apt-get -o Acquire::Check-Valid-Until=false update -qq + apt-get install -y -qq --no-install-recommends unzip xz-utils + } +elif command -v apk >/dev/null 2>&1; then + command -v unzip >/dev/null 2>&1 || apk add --no-cache unzip xz tar +fi +command -v node >/dev/null 2>&1 && { echo "node $(node --version) already present"; exit 0; } if command -v apk >/dev/null 2>&1; then # Official tarballs are glibc; on musl the distro package is the one # that actually runs. @@ -113,7 +123,7 @@ if command -v apk >/dev/null 2>&1; then exit 0 fi command -v curl >/dev/null 2>&1 || { - apt-get update -qq && apt-get install -y -qq --no-install-recommends curl ca-certificates xz-utils + apt-get -o Acquire::Check-Valid-Until=false update -qq && apt-get install -y -qq --no-install-recommends curl ca-certificates xz-utils } case "$(uname -m)" in x86_64) a=x64 ;; aarch64|arm64) a=arm64 ;; *) echo "unsupported arch"; exit 1 ;; esac v=$(curl -fsSL https://nodejs.org/dist/latest-v24.x/ | grep -oE 'node-v24[0-9.]*-linux-'"$a"'\.tar\.xz' | head -1) diff --git a/ci/platforms/github.go b/ci/platforms/github.go index 5f204b6..8af37e3 100644 --- a/ci/platforms/github.go +++ b/ci/platforms/github.go @@ -202,7 +202,8 @@ func ghStepToSteps(i int, s ghStep, repo Repo) ([]Step, bool) { stepID = fmt.Sprintf("step-%d", i) } aRepo := actions.Repo{ - Sha: repo.Sha, Branch: repo.branch(), Name: repo.Name, Workspace: repo.Workspace, + Sha: repo.Sha, Branch: repo.branch(), Name: repo.Name, + Workspace: repo.Workspace, Token: repo.Token, } with := map[string]string{} for k, v := range s.With { diff --git a/ci/platforms/platforms.go b/ci/platforms/platforms.go index 1f6cf94..86f9130 100644 --- a/ci/platforms/platforms.go +++ b/ci/platforms/platforms.go @@ -59,6 +59,9 @@ type Repo struct { // Workspace is the in-guest path the clone lands at; every platform's // workspace variable must point there or scripts cd into nothing. Workspace string + // Token is the operator's GITHUB_TOKEN secret, when injected — what + // `${{ github.token }}` resolves to for real actions. + Token string } func (r Repo) branch() string { diff --git a/examples/ci-github-actions/.github/workflows/ci.yml b/examples/ci-github-actions/.github/workflows/ci.yml new file mode 100644 index 0000000..4813040 --- /dev/null +++ b/examples/ci-github-actions/.github/workflows/ci.yml @@ -0,0 +1,15 @@ +name: CI +on: [push] +jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + - name: test + run: | + bun --version + test -f hello.txt + echo "github-actions-example-ok" diff --git a/examples/ci-github-actions/README.md b/examples/ci-github-actions/README.md new file mode 100644 index 0000000..ff04f90 --- /dev/null +++ b/examples/ci-github-actions/README.md @@ -0,0 +1,23 @@ +# ci-github-actions — real `uses:` actions, run locally by `bsdkrun ci` + +This workflow runs **the actual `oven-sh/setup-bun@v2` action** — not a +translation of it. `bsdkrun ci` fetches the action's `action.yml` to learn +what it is, clones it into the guest at its ref, provisions a node runtime, +and executes it under the genuine Actions protocol: `INPUT_*` from `with:`, +and `GITHUB_ENV`/`GITHUB_PATH`/`GITHUB_OUTPUT` command files whose effects +persist into every later step — which is why the plain `run:` step's +`bun --version` works: setup-bun wrote bun's location to `GITHUB_PATH`. + +JavaScript and composite actions run for real. Container actions are +refused visibly (a microVM runs no Docker daemon), as are `pre`/`post` +hooks — stated limits, not silent ones. + +CI runs the repository's **HEAD commit**, so the example needs its own git +repository: + +```sh +cp -r examples/ci-github-actions /tmp/ci-github-actions +cd /tmp/ci-github-actions +git init -q && git add -A && git commit -qm init +bsdkrun ci run +``` diff --git a/examples/ci-github-actions/hello.txt b/examples/ci-github-actions/hello.txt new file mode 100644 index 0000000..7644575 --- /dev/null +++ b/examples/ci-github-actions/hello.txt @@ -0,0 +1 @@ +hello from the GitHub Actions example -- 2.51.2