21 Commits

Author SHA1 Message Date
k3nny 46a1cf3c08 feat: add go lint
ci / vet, staticcheck, test, build (push) Successful in 2m3s
release / Build and publish release (push) Successful in 1m9s
2026-06-11 23:56:09 +02:00
k3nny 18c8fc82c9 feat(linter): add GL032 variable reference validation in rules:if:
release / Build and publish release (push) Successful in 1m11s
Warn when a $VAR or ${VAR} reference in a rules:if: expression is not
declared in pipeline variables:, the job's own variables:, or any
workflow:rules:variables: block. Predefined GitLab CI namespaces (CI_*,
GITLAB_*, FF_*, RUNNER_*, TRIGGER_*, CHAT_*) are always exempt.

Each undeclared variable is reported at most once per job. The finding
is a WARNING (not an error) because variables may also be set in GitLab
CI/CD project settings, which are invisible to glint at lint time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 23:13:25 +02:00
k3nny f48bf02152 feat(linter): add structured rule IDs (GL001–GL031)
Every Finding now carries a stable Rule string field with a GL### code.
The ID appears in output between the source location and the message:

  [ERROR] job "deploy" (ci.yml:14) GL003: missing required field 'script'
  [WARNING] (ci.yml) GL001: no stages defined

Rules:
  GL001 no-stages          GL002 workflow-when       GL003 missing-script
  GL004 unknown-stage      GL005 only-rules-conflict GL006 except-rules-conflict
  GL007 deprecated-only    GL008 invalid-when        GL009 delayed-no-start-in
  GL010 start-in-no-delayed GL011 invalid-parallel   GL012 invalid-retry
  GL013 invalid-retry-when GL014 invalid-allow-failure GL015 invalid-interruptible
  GL016 trigger-with-script GL017 invalid-trigger    GL018 invalid-coverage
  GL019 invalid-release    GL020 invalid-environment GL021 invalid-artifacts
  GL022 pages-public       GL023 invalid-cache       GL024 invalid-rules-when
  GL025 invalid-image      GL026 invalid-inherit     GL027 needs-unknown
  GL028 needs-stage-order  GL029 needs-cycle         GL030 unknown-dependency
  GL031 dependency-stage

Changes:
- internal/linter/rules.go: new file with all 31 constants + doc comments
- linter.Finding: add Rule string field; String() inserts it before the
  message colon when non-empty; format unchanged when Rule == ""
- All Finding{} literals in linter.go, keywords.go, needs.go,
  dependencies.go updated with the correct Rule: constant
- README.md lint rules table: new ID column added to all four sections
- CHANGELOG.md: entry in [Unreleased]

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 22:56:24 +02:00
k3nny 6d0aefca5b docs(docs): update ROADMAP with post-v0.2.0 shipped items
Mark as done:
- include: remote: URL fetching
- workflow:rules:variables: propagation
- Expression evaluator: multi-line, \${VAR}, regex flags, variable regex
  RHS, bare true/false/integer literals
- File and line numbers on findings
- needs: optional: true downgraded to warning
- extends: missing script downgraded to warning
- glint graph includes jobs-per-file

Add new remaining items:
- rules:if: static reachability analysis (future)
- Findings quality section broken out from lint coverage

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 22:38:45 +02:00
k3nny de6a526560 feat(linter): propagate workflow:rules:variables: into job rule evaluation
workflow:rules: can define variables: on matching rules (GitLab CI 15.0+).
These variables are now injected into the evaluation context before job
rules:if: expressions are evaluated, making patterns like:

  workflow:
    rules:
      - if: '$CI_COMMIT_BRANCH == "main"'
        variables:
          DEPLOY_TARGET: production
  deploy:
    rules:
      - if: '$DEPLOY_TARGET == "production"'

work correctly with glint check --branch main.

Changes:
- model.Rule: add Variables map[string]any field (yaml:"variables")
- cicontext.Context: add pinned map tracking which vars must not be
  overwritten; New() pins all shortcut and --var variables; add
  Inject(key, value) which writes only when key is not pinned
- cicontext.ExtractStringVars: shared helper that converts map[string]any
  variable blocks (plain string or {value:...} form) to map[string]string
- cicontext.EvalWorkflow: returns (bool, map[string]string) — the vars of
  the matching workflow rule alongside the runs/no-runs result
- cmd/glint/main.go: enrichContext() injects pipeline-level variable
  defaults then workflow-rule variables before printContext; applied in
  both cmdCheck and cmdGraph

Injection priority (highest wins):
  --var CLI overrides > --branch/--tag/--source shortcuts
  > workflow-rule variables > pipeline variables: defaults

Adds 15 unit tests (TestEvalWorkflow, TestContextInject,
TestExtractStringVars, TestWorkflowVarsJobEval) and a testdata fixture
(workflow_vars.yml) validated across four branch contexts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 22:35:55 +02:00
k3nny a0e2582cf1 fix(linter): support bare true/false and integer literals in rules:if:
release / Build and publish release (push) Successful in 1m15s
GitLab CI expressions allow unquoted true, false, and integers as
comparison operands (all treated as their string representations):

  $GATEWAY_ENABLED == true    (equivalent to == "true")
  $FEATURE_FLAG == false      (equivalent to == "false")
  $PARALLEL == 4              (equivalent to == "4")
  $ENABLED == 1 / == 0

Previously these fell through to permissive true because parseValue
only recognised $VAR, "${VAR}", quoted strings, and null. Added:
  - true/false keyword branch → returns "true"/"false"
  - integer literal branch (digits only) → returns decimal string

All three new forms are correctly excluded from longer identifier
prefixes (identByte boundary check). Adds 8 new unit tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 22:20:59 +02:00
k3nny e931b9d1c9 fix(linter): improve rules:if: expression evaluator
Four correctness fixes to the GitLab CI expression parser in
internal/cicontext/eval.go:

- Multi-line: \n and \r are now treated as whitespace in skipWS so
  block-scalar or folded-scalar if: values with || / && on continuation
  lines evaluate correctly instead of falling back to permissive true.
- ${VAR} curly-brace variable syntax now supported in parseValue.
- Regex flags (/pattern/i, /pattern/m, /pattern/s) are now consumed and
  translated to Go (?i)/(?m)/(?s) prefixes via applyRegexFlags.
- Variable on RHS of =~ / !~: when the right operand is $VAR, the
  variable's value is interpreted as a /regex/[flags] string via
  extractRegexFromString; non-regex values fall back to permissive true.

Adds 16 new unit tests covering all four cases and a testdata fixture
(rules_if_expr.yml) exercising multi-line, ${VAR}, and /pattern/i in a
real pipeline with context flags.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 22:08:08 +02:00
k3nny b21a7d60dc feat(resolver,graph): fetch and resolve include: remote: HTTPS URLs
release / Build and publish release (push) Successful in 1m14s
Remote includes (include: remote: https://...) were previously skipped
silently in the resolver and rendered as unexpanded leaf nodes in the
graph.

Changes:
- fetcher.FetchURL: new shared unauthenticated HTTP GET helper
- resolver: resolveRemoteInclude fetches the URL, parses YAML, sets job
  origin to the URL string, recursively resolves sub-includes, and emits
  a warning on failure (lint continues on the rest of the pipeline)
- graph: recurseRemote fetches the URL, captures direct job names, and
  recurses into sub-includes so remote nodes expand like local ones

Adds testdata/includes_remote.yml fixture.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 21:42:15 +02:00
k3nny d34c39927d fix(linter): downgrade needs optional:true missing-job to warning
release / Build and publish release (push) Successful in 1m12s
parseNeedJobNames is replaced by parseNeedEntries which preserves the
optional flag from each needs: entry. When a referenced job does not
exist and optional:true is set, the finding is now WARNING instead of
ERROR, matching GitLab CI runtime behavior (the dependency is silently
skipped when the job is absent from a conditional include).

Optional missing deps are also excluded from the cycle-detection graph
since there is no real dependency edge to trace.

Adds a fixture case in testdata/needs.yml to prevent regression.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 21:27:16 +02:00
k3nny a303f63a5e feat(linter): add file/line to findings; downgrade extends missing-script to warning
Every finding now carries the source file and exact line number of the job
key in its YAML file. Format: [ERROR] job "name" (file.yml:12): message.

Pipeline-level findings (workflow rules, no stages) reference p.SourceFile.
Cross-file include jobs (local, project, component) carry the include source
as their File, set via Pipeline.SetJobOrigin after each ParseBytes call in
the resolver.

Line numbers come from the yaml.Node key node (exact job-name line) in a
new document-level first pass in ParseBytes, replacing the previous
map[string]yaml.Node approach which only gave value-node lines.

Also: jobs that declare extends: but have no script after resolution now
emit WARNING instead of ERROR. The script may come from a base in a remote
include that was not fetched (no token, offline), making the error a false
positive in common project setups.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 21:24:18 +02:00
k3nny a962c996c1 feat(graph): show jobs per file in include dependency graph
Each node in 'glint graph includes' now lists the jobs defined directly
in that file. Jobs appear as rounded Mermaid nodes with a distinct
light-purple style, connected with dashed arrows (-.->). This visual
distinction separates ownership (file -.-> job) from the include
hierarchy (file --> included-file).

The root file's jobs are collected by re-parsing it without include
resolution; local and fetched project/component nodes populate their
job list in the existing recurse* methods.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 21:03:50 +02:00
k3nny e5f926b55f docs: 📝 add logo
release / Build and publish release (push) Successful in 1m47s
2026-06-11 20:40:39 +02:00
k3nny 8c3ce050f5 Merge pull request 'fix(model): handle YAML map forms that caused unmarshall errors' (#3) from fix/unmarshall_errors into main
Reviewed-on: #3
2026-06-11 20:31:30 +02:00
k3nny c4ab64391d fix(model): handle YAML map forms that caused unmarshall errors
Variables with value/description/options sub-keys, default.image in map
form, default.before_script / default.after_script as block scalars, and
rules.changes / rules.exists in {paths, compare_to} map form all caused
"yaml: cannot unmarshal !!map into string" because the struct fields were
typed too narrowly.

Changed types in model.Pipeline, model.DefaultConfig, and model.Rule to
accept any to match GitLab CI spec flexibility (13.7+ variable declarations,
15.3+ rules.changes map form, image map form in default block).

Adds testdata/script_multiline.yml covering all these patterns.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 20:25:53 +02:00
k3nny 4cc50afb5f fix(build): replace actions/checkout@v4 with plain git clone
release / Build and publish release (push) Successful in 1m3s
actions/checkout@v4 requires Node.js which is absent in golang:alpine.
Clone the repo directly with git using an oauth2 token in the URL,
removing the Node.js dependency entirely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 01:04:27 +02:00
k3nny dfbafd8ed3 chore(build): use golang:1.26-alpine container instead of ubuntu-latest
release / Build and publish release (push) Failing after 41s
Switch from the full ubuntu runner image to golang:1.26-alpine via the
container: directive. Go is pre-installed so actions/setup-go is dropped;
curl, git, and jq are added with apk in the first step.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 01:00:30 +02:00
k3nny 58cfbb4a57 fix(build): replace upload-artifact@v4 with direct Gitea API upload
upload-artifact@v4 uses a backend API incompatible with Gitea (GHES).
Collapse to a single job that builds both targets sequentially and uploads
directly to a Gitea release via the REST API, removing the need for
artifact passing between jobs entirely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 00:59:09 +02:00
k3nny e7b51929f8 chore(build): add Gitea Actions release pipeline
release / Build (${{ matrix.goos }}/${{ matrix.goarch }}) (amd64, linux, -linux-amd64) (push) Failing after 11m48s
release / Build (${{ matrix.goos }}/${{ matrix.goarch }}) (amd64, windows, .exe) (push) Failing after 5m16s
release / Publish release (push) Has been skipped
Builds Linux x64 and Windows x64 binaries on tag pushes (v*) and
publishes them as assets on a Gitea release via the REST API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 00:38:21 +02:00
k3nny 88f20165db feat(cli)!: subcommand CLI, graph tree mode, local include resolution
BREAKING CHANGES:
- `glint <file>` removed; use `glint check <file>`
- `--graph <mode>` removed; use `glint graph [mode]`
- `--graph-out` renamed to `--out` on `glint graph`

feat(cli): ruff-style subcommands — `glint check` and `glint graph [mode]`
feat(graph): `glint graph tree` — terminal job tree with context annotations
feat(graph): context flags (--branch/--tag/--source/--var) on `glint graph`
feat(resolver): recursive local include resolution from disk
fix(resolver): extends unknown base emits warning instead of fatal error
fix(model): script/before_script/after_script accept block scalar string form
test(linter): Samba project CI fixtures as integration tests
chore(build): fix .gitignore to not exclude cmd/glint/ directory
docs: update CHANGELOG, README, ROADMAP for v0.2.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 00:27:28 +02:00
k3nny aca37de2b1 fix(ci): fix build 2026-06-10 23:02:06 +02:00
k3nny 51b3e1f297 fix(project): rename tool to glint 2026-06-10 22:40:42 +02:00
53 changed files with 3900 additions and 390 deletions
+40
View File
@@ -0,0 +1,40 @@
name: ci
on:
push:
branches:
- '**'
pull_request:
jobs:
ci:
name: vet, staticcheck, test, build
runs-on: ubuntu-latest
container:
image: golang:1.26-alpine
steps:
- name: Install tools
run: apk add --no-cache git
- name: Checkout
env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
SERVER_URL: ${{ github.server_url }}
REPO: ${{ github.repository }}
REF: ${{ github.ref_name }}
run: |
git clone --depth 1 --branch "$REF" \
"$(echo "$SERVER_URL" | sed "s|https://|https://oauth2:${TOKEN}@|")/${REPO}.git" .
- name: vet
run: go vet ./...
- name: staticcheck
run: go tool staticcheck ./...
- name: test
run: go test ./...
- name: build
run: go build ./cmd/glint/...
+70
View File
@@ -0,0 +1,70 @@
name: release
on:
push:
tags:
- 'v*'
jobs:
release:
name: Build and publish release
runs-on: ubuntu-latest
container:
image: golang:1.26-alpine
steps:
- name: Install tools
run: apk add --no-cache curl git jq
- name: Checkout
env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
SERVER_URL: ${{ github.server_url }}
REPO: ${{ github.repository }}
REF: ${{ github.ref_name }}
run: |
git clone --depth 1 --branch "$REF" \
"$(echo "$SERVER_URL" | sed "s|https://|https://oauth2:${TOKEN}@|")/${REPO}.git" .
- name: Build Linux (amd64)
env:
GOOS: linux
GOARCH: amd64
CGO_ENABLED: "0"
run: |
go build -trimpath -ldflags="-s -w" \
-o glint-${{ github.ref_name }}-linux-amd64 \
./cmd/glint/...
- name: Build Windows (amd64)
env:
GOOS: windows
GOARCH: amd64
CGO_ENABLED: "0"
run: |
go build -trimpath -ldflags="-s -w" \
-o glint-${{ github.ref_name }}.exe \
./cmd/glint/...
- name: Create release and upload assets
env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ github.ref_name }}
API_URL: ${{ github.api_url }}
REPO: ${{ github.repository }}
run: |
release_id=$(curl -sf -X POST \
-H "Authorization: token $TOKEN" \
-H "Content-Type: application/json" \
"$API_URL/repos/$REPO/releases" \
-d "{\"tag_name\":\"$TAG\",\"name\":\"$TAG\",\"draft\":false,\"prerelease\":false}" \
| jq -r .id)
for file in glint-${{ github.ref_name }}-linux-amd64 glint-${{ github.ref_name }}.exe; do
curl -sf -X POST \
-H "Authorization: token $TOKEN" \
-H "Content-Type: application/octet-stream" \
"$API_URL/repos/$REPO/releases/$release_id/assets?name=$file" \
--data-binary "@$file"
echo "uploaded: $file"
done
+3 -3
View File
@@ -1,11 +1,11 @@
# Compiled binaries # Compiled binaries
gitlab-sim /glint
gitlab-sim.exe /glint.exe
dist/ dist/
bin/ bin/
# Graph output directory # Graph output directory
gitlab-sim-out/ glint-out/
# Go test & coverage artifacts # Go test & coverage artifacts
*.test *.test
+71 -2
View File
@@ -5,6 +5,75 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
This project uses [Semantic Versioning](https://semver.org). This project uses [Semantic Versioning](https://semver.org).
## [Unreleased]
### Added
- **Variable reference validation (GL032)** — glint now warns when a `rules:if:` expression references a variable (`$VAR` or `${VAR}`) that is not declared anywhere in the pipeline YAML: pipeline-level `variables:`, the job's own `variables:`, or any `workflow:rules:variables:` block. Predefined GitLab CI variable namespaces (`CI_*`, `GITLAB_*`, `FF_*`, `RUNNER_*`, `TRIGGER_*`, `CHAT_*`) are exempt. Because variables can also be set in GitLab CI/CD project settings (invisible to glint), the finding is a `[WARNING]` rather than an error. Each undeclared variable is reported at most once per job to keep the output concise.
- **Included-file variables now visible to all lint rules** — `variables:` blocks declared in included files (local, remote, project, and component includes) are now merged into the pipeline's variable namespace before linting. This eliminates false-positive GL032 warnings for variables declared in shared CI templates. Root-pipeline variables take precedence over included-file variables when the same key appears in both (matching GitLab's own override behaviour).
- **Structured rule IDs** — every finding now carries a stable `GL###` identifier (e.g. `GL003`) that appears in the output between the location and the message: `[ERROR] job "deploy" (file.yml:14) GL003: missing required field 'script'`. IDs are assigned per check function across 31 rules (GL001GL031) and are stable across versions. The `linter.Finding` struct exposes the ID as a `Rule string` field for programmatic consumers. The README lint rules table is updated with ID columns.
- **`workflow:rules:variables:` now propagate to job rule evaluation** — when a `workflow:rules:` entry matches, any `variables:` it defines are injected into the evaluation context so job `rules:if:` expressions can reference them. Pipeline-level `variables:` are also available as defaults (lower priority). Variable priority order, highest first: `--var` CLI overrides → `--branch`/`--tag`/`--source` shortcuts → workflow-rule variables → pipeline-level variable defaults. This means `$DEPLOY_TARGET == "production"` in a job rule correctly evaluates when a workflow rule sets `DEPLOY_TARGET: production` for the matching branch. The `glint graph tree` command benefits from the same enrichment.
- **`rules:if:` expression evaluator improvements** — six correctness fixes to the GitLab CI expression parser:
- **Multi-line expressions** — newlines (`\n`, `\r`) are now treated as whitespace between tokens, so block-scalar `if:` values (e.g. `if: | ...`) and folded YAML scalars with `||`/`&&` on a continuation line are parsed correctly instead of falling back to permissive `true`.
- **`${VAR}` curly-brace variable syntax** — `${CI_COMMIT_BRANCH}` is now equivalent to `$CI_COMMIT_BRANCH` everywhere a value is expected.
- **Regex flags** — `/pattern/i`, `/pattern/m`, `/pattern/s` are now honoured; the `i` flag (case-insensitive) is translated to Go's `(?i)` prefix before compiling. Unknown flags are silently ignored.
- **Variable as regex RHS** — `$BRANCH =~ $PATTERN` where `$PATTERN` holds a `/regex/[flags]` string is now evaluated by extracting and compiling the pattern from the variable's value; if the value is empty or does not look like a regex literal the expression falls back to permissive `true`.
- **`true` / `false` keywords** — bare `true` and `false` (without quotes) are now recognised as the string values `"true"` and `"false"`, matching GitLab CI's own behaviour. `$GATEWAY_ENABLED == true` and `$FEATURE_FLAG == false` now evaluate correctly.
- **Integer literals** — bare integers (e.g. `$PARALLEL == 4`, `$ENABLED == 1`, `$DISABLED == 0`) are now parsed as their decimal string representations and compared accordingly.
- **File and line numbers on findings** — every finding now includes the source file and line where the job is defined, e.g. `[ERROR] job "deploy" (src/deploy.yml:14): …`. For jobs that come from local or fetched includes the file reflects the include source. Pipeline-level findings (workflow rules, missing stages) reference the root pipeline file.
- **`glint graph includes` shows jobs per file** — each node in the Mermaid include dependency graph now shows the jobs defined directly in that file. Jobs are rendered as rounded nodes (`(name)`) in a distinct light-purple style, connected with dashed arrows (`-.->`) to distinguish ownership from the include hierarchy (solid `-->` arrows). The root pipeline file always shows its direct jobs; local and fetched project/component nodes show theirs when the file can be read.
### Fixed
- **`include: remote:` URL includes are now fetched and merged** — glint fetches plain HTTPS URLs in `include: remote:` entries (no authentication), parses the resulting YAML, merges its jobs into the pipeline, and recursively resolves any sub-includes the remote file itself declares. Unreachable or unparseable URLs emit a `[WARNING]` and lint continues on the rest of the pipeline. The `glint graph includes` command now expands remote nodes with their jobs and sub-include tree, matching the behaviour of local and project includes.
- **`needs: optional: true` downgraded to warning** — a `needs:` entry that carries `optional: true` and references a job not present in the pipeline now emits `[WARNING]` instead of `[ERROR]`. GitLab CI silently skips such dependencies at runtime (the job is absent when its include was not triggered), so the finding was a false positive. Non-optional missing needs remain errors. Optional missing deps are also excluded from the cycle-detection graph.
- **`extends:` jobs with missing script downgraded to warning** — a job that declares `extends:` but has no `script` after resolution now emits `[WARNING]` instead of `[ERROR]`. The script may legitimately come from a base job in a remote include that could not be fetched at lint time (e.g. no token configured).
- **Variable map form now parses correctly** — `variables:` entries that use the extended `{value, description, options}` form (GitLab CI 13.7+) no longer cause `yaml: cannot unmarshal !!map into string`. Both `Pipeline.Variables` and per-job `Variables` now accept either plain strings or map-form declarations.
- **`default.image` map form now parses correctly** — `default: image: {name: ..., pull_policy: ...}` used to cause `yaml: cannot unmarshal !!map into string`; `DefaultConfig.Image` is now typed as `any` to match `Job.Image`.
- **`default.before_script` / `default.after_script` now accept both list and scalar forms** — previously `DefaultConfig.BeforeScript` and `DefaultConfig.AfterScript` were `[]string`, causing a parse error when the field was written as a block scalar string. They are now typed as `any` to match the corresponding `Job` fields.
- **`rules.changes` / `rules.exists` map form now parses correctly** — extended `changes: {paths: [...], compare_to: "..."}` syntax (GitLab CI 15.3+) used to cause `yaml: cannot unmarshal !!map into []string`.
## [0.2.0] - 2026-06-11
### Added
- **Subcommand CLI** — reworked interface inspired by [ruff](https://docs.astral.sh/ruff/):
- `glint check <file>` — lint a pipeline (replaces bare `glint <file>`)
- `glint graph [mode] <file>` — visualise the pipeline (replaces `--graph` flag)
- Graph modes: no-arg (tree + includes), `tree`, `includes`, `pipeline`, `all`
- Per-command `--help` with ruff-style layout: `Arguments:`, `Options:` (flag declaration on its own line, description below), `[env: ...]` / `[default: ...]` / `[possible values: ...]` metadata, `Examples:` section
- **`glint graph tree`** — jobs displayed as a terminal directory tree grouped by stage (like the `tree` command); job-type annotations (`[manual]`, `[delayed]`, `[trigger]`) when no context is set; evaluated-state annotations (`[skipped]`, `[manual]`) when a context is provided via `--branch` / `--tag` / `--source`
- **Context flags on `glint graph`** — `--branch`, `--tag`, `--source`, `--var` are now available on `glint graph` as well as `glint check`
- **Local include resolution** — `include: local:` entries are now read from disk, recursively resolved, and merged into the pipeline before linting; enables cross-file `extends:` and `needs:` validation for multi-file pipelines
- **Cross-platform release builds** — two Taskfile tasks for tagged release binaries:
- `task build-windows` — cross-compiles for Windows x64; output: `glint-<tag>.exe`
- `task build-linux` — cross-compiles for Linux x64; output: `glint-<tag>-linux-amd64`
- Both tasks require an exact git tag on the current commit
### Fixed
- **`extends:` unknown base no longer fatal** — when a base job referenced by `extends:` does not exist, glint now emits a resolver warning and skips extends resolution for that job rather than aborting with exit code 2; linting continues on the job's own fields
- **`script: |` (block scalar) support** — jobs using a multiline block scalar for `script:`, `before_script:`, or `after_script:` are now parsed correctly; previously caused false-positive "missing script" errors
### Changed
- **`glint <file>` removed** — use `glint check <file>`
- **`--graph <mode>` removed** — replaced by `glint graph [mode]`
- **`--graph-out` renamed to `--out`** — now a flag on `glint graph` (`glint graph pipeline --out <dir>`)
## [0.1.0] - 2026-06-07 ## [0.1.0] - 2026-06-07
### Added ### Added
@@ -24,7 +93,7 @@ This project uses [Semantic Versioning](https://semver.org).
- **Graph output (`--graph`)** — visualises the pipeline instead of running lint rules: - **Graph output (`--graph`)** — visualises the pipeline instead of running lint rules:
- `--graph includes` — [Mermaid](https://mermaid.js.org) flowchart of include dependencies written to stdout; one node per `include:` entry (project, component, local, remote, template), colour-coded by type; pipe to a `.mmd` file or paste into [mermaid.live](https://mermaid.live) - `--graph includes` — [Mermaid](https://mermaid.js.org) flowchart of include dependencies written to stdout; one node per `include:` entry (project, component, local, remote, template), colour-coded by type; pipe to a `.mmd` file or paste into [mermaid.live](https://mermaid.live)
- `--graph pipeline` — GitLab CI-style SVG/PNG pipeline graph written to a timestamped file in `--graph-out` (default: `gitlab-sim-out/`); jobs rendered as white chip cards with a coloured status indicator (blue: regular, orange: manual, purple: trigger, amber: delayed); DAG mode draws job-to-job Bézier arrows when any job has `needs:`, classic mode draws L-shaped connectors between stage columns; converted to PNG automatically when `rsvg-convert`, `inkscape`, or `magick` is available - `--graph pipeline` — GitLab CI-style SVG/PNG pipeline graph written to a timestamped file in `--graph-out` (default: `glint-out/`); jobs rendered as white chip cards with a coloured status indicator (blue: regular, orange: manual, purple: trigger, amber: delayed); DAG mode draws job-to-job Bézier arrows when any job has `needs:`, classic mode draws L-shaped connectors between stage columns; converted to PNG automatically when `rsvg-convert`, `inkscape`, or `magick` is available
- `--graph all` — include Mermaid to stdout, pipeline file path to stderr - `--graph all` — include Mermaid to stdout, pipeline file path to stderr
- New `internal/graph` package (`includes.go`, `pipeline.go`, `render.go`); no new external dependencies - New `internal/graph` package (`includes.go`, `pipeline.go`, `render.go`); no new external dependencies
@@ -74,7 +143,7 @@ This project uses [Semantic Versioning](https://semver.org).
- `only`/`rules` or `except`/`rules` used together (error) - `only`/`rules` or `except`/`rules` used together (error)
- No `stages` block defined (warning) - No `stages` block defined (warning)
- Deprecated `only`/`except` usage (warning) - Deprecated `only`/`except` usage (warning)
- **CLI** — `gitlab-sim <file>` exits 0 on clean pipelines, 1 on errors; prints findings with severity, job name, and message - **CLI** — `glint <file>` exits 0 on clean pipelines, 1 on errors; prints findings with severity, job name, and message
- **YAML parser** — two-pass parse: reserved top-level keys (`stages`, `variables`, `default`, `include`, `workflow`) are decoded into typed structs; remaining keys are treated as job definitions - **YAML parser** — two-pass parse: reserved top-level keys (`stages`, `variables`, `default`, `include`, `workflow`) are decoded into typed structs; remaining keys are treated as job definitions
- **Taskfile** — `build`, `test`, `lint-go`, `validate`, `ci`, `clean` tasks via [Task](https://taskfile.dev) - **Taskfile** — `build`, `test`, `lint-go`, `validate`, `ci`, `clean` tasks via [Task](https://taskfile.dev)
- **Testdata fixtures** — `valid.yml`, `invalid.yml`, `extends.yml`, `needs.yml`, `needs_cycle.yml` - **Testdata fixtures** — `valid.yml`, `invalid.yml`, `extends.yml`, `needs.yml`, `needs_cycle.yml`
+9 -3
View File
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Project Overview ## Project Overview
`gitlab-sim` is a CLI tool that validates and lints `.gitlab-ci.yml` pipelines locally, without a GitLab server. It resolves `extends:` inheritance, fetches remote `include: project:` templates and `include: component:` catalog entries, then runs a set of lint rules over the fully-merged pipeline. `glint` is a CLI tool that validates and lints `.gitlab-ci.yml` pipelines locally, without a GitLab server. It resolves `extends:` inheritance, fetches remote `include: project:` templates and `include: component:` catalog entries, then runs a set of lint rules over the fully-merged pipeline.
**Goals:** catch misconfigured pipelines early in a developer's workflow, before pushing to GitLab. Eventual goal: produce a visual graph of the pipeline DAG. **Goals:** catch misconfigured pipelines early in a developer's workflow, before pushing to GitLab. Eventual goal: produce a visual graph of the pipeline DAG.
@@ -34,7 +34,7 @@ This order is intentional and must be preserved: includes must be merged before
| `internal/resolver` | `extends:` resolution and `include:` merging | | `internal/resolver` | `extends:` resolution and `include:` merging |
| `internal/fetcher` | GitLab REST API client (token auth, file fetch) | | `internal/fetcher` | GitLab REST API client (token auth, file fetch) |
| `internal/graph` | Mermaid graph generators (include dependencies, pipeline jobs) | | `internal/graph` | Mermaid graph generators (include dependencies, pipeline jobs) |
| `cmd/gitlab-sim` | CLI entrypoint, flag parsing, output formatting | | `cmd/glint` | CLI entrypoint, flag parsing, output formatting |
### Two-pass YAML parser (`internal/model/parser.go`) ### Two-pass YAML parser (`internal/model/parser.go`)
@@ -138,7 +138,7 @@ All commit messages must follow this format:
- `resolver`: extends/include resolution (`internal/resolver/`) - `resolver`: extends/include resolution (`internal/resolver/`)
- `fetcher`: GitLab API client (`internal/fetcher/`) - `fetcher`: GitLab API client (`internal/fetcher/`)
- `graph`: Mermaid graph generators (`internal/graph/`) - `graph`: Mermaid graph generators (`internal/graph/`)
- `cli`: CLI entrypoint (`cmd/gitlab-sim/`) - `cli`: CLI entrypoint (`cmd/glint/`)
- `testdata`: fixture files only - `testdata`: fixture files only
- `docs`: README, CHANGELOG, or other documentation - `docs`: README, CHANGELOG, or other documentation
- `build`: Taskfile, go.mod, go.sum, CI config - `build`: Taskfile, go.mod, go.sum, CI config
@@ -185,3 +185,9 @@ Every fixture in `testdata/` is run by `task validate`. Files whose expected beh
Token resolution order (first non-empty wins): `GITLAB_TOKEN``CI_JOB_TOKEN``GITLAB_PRIVATE_TOKEN`. Token resolution order (first non-empty wins): `GITLAB_TOKEN``CI_JOB_TOKEN``GITLAB_PRIVATE_TOKEN`.
URL resolution order: `--gitlab-url` flag → `CI_SERVER_URL``GITLAB_URL``https://gitlab.com`. URL resolution order: `--gitlab-url` flag → `CI_SERVER_URL``GITLAB_URL``https://gitlab.com`.
---
## README and CHANGELOG
On each modification, complete README.md and CHANGELOG.md with the changes made.
+118 -82
View File
@@ -1,7 +1,13 @@
# gitlab-sim <p align="center">
<img src="assets/glint-logo.png" alt="glint logo" width="220" />
</p>
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE) <h1 align="center">glint</h1>
[![Release](https://img.shields.io/badge/release-v0.1.0-blue.svg)](CHANGELOG.md)
<p align="center">
<a href="LICENSE"><img src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" alt="License"></a>
<a href="CHANGELOG.md"><img src="https://img.shields.io/badge/release-v0.2.0-blue.svg" alt="Release"></a>
</p>
> **Disclaimer:** This tool was built through iterative AI-assisted development with [Claude](https://claude.ai). It is experimental, incomplete, and not intended for production use. Coverage of GitLab CI keywords is best-effort and may lag behind GitLab's evolving spec. Use it at your own discretion — no correctness guarantees are made. Contributions and bug reports are welcome. > **Disclaimer:** This tool was built through iterative AI-assisted development with [Claude](https://claude.ai). It is experimental, incomplete, and not intended for production use. Coverage of GitLab CI keywords is best-effort and may lag behind GitLab's evolving spec. Use it at your own discretion — no correctness guarantees are made. Contributions and bug reports are welcome.
@@ -18,8 +24,12 @@ A local tool to validate and lint `.gitlab-ci.yml` pipelines without needing a G
- **Remote project includes** — fetches `include: project:` templates from the GitLab API so extends/needs can be validated against the full merged pipeline - **Remote project includes** — fetches `include: project:` templates from the GitLab API so extends/needs can be validated against the full merged pipeline
- **CI/CD catalog components** — resolves `include: component:` references from the GitLab CI/CD Catalog; public components work without a token - **CI/CD catalog components** — resolves `include: component:` references from the GitLab CI/CD Catalog; public components work without a token
- **Deprecation warnings** — flags `only`/`except` usage in favour of `rules` - **Deprecation warnings** — flags `only`/`except` usage in favour of `rules`
- **Graph output** — emits Mermaid diagrams for the include dependency tree and the pipeline jobs layout (DAG or classic stage ordering) - **Local include resolution** — `include: local:` entries are read from disk and recursively merged before linting, so multi-file pipelines are fully validated
- **Context simulation** — pass `--branch`, `--tag`, or `--source` to see which jobs would be active, manual, or skipped for a specific branch push, tag, or pipeline event; evaluates `rules:if:` expressions and `only`/`except` filters - **Extended variable declarations** — `variables:` entries may use the `{value, description, options}` map form (GitLab CI 13.7+); `default.image` accepts both string and map form; `rules.changes`/`rules.exists` accept both list and `{paths, compare_to}` map form
- **Graph output** — `glint graph` prints a job tree (stages → jobs) to the terminal; `glint graph includes` emits a Mermaid include dependency diagram; `glint graph pipeline` renders a GitLab CI-style PNG/SVG
- **Context simulation** — pass `--branch`, `--tag`, or `--source` to `glint check` or `glint graph` to see which jobs would be active, manual, or skipped for a specific pipeline event; evaluates `rules:if:` expressions and `only`/`except` filters
See [ROADMAP.md](ROADMAP.md) for planned improvements.
## Requirements ## Requirements
@@ -29,9 +39,9 @@ A local tool to validate and lint `.gitlab-ci.yml` pipelines without needing a G
## Installation ## Installation
```bash ```bash
git clone https://git.k3nny.fr/gitlab-sim git clone https://git.k3nny.fr/glint
cd gitlab-sim cd glint
go build -o gitlab-sim ./cmd/gitlab-sim/... go build -o glint ./cmd/glint/...
``` ```
Or with Task: Or with Task:
@@ -42,8 +52,20 @@ task build
## Usage ## Usage
```
glint [OPTIONS] <COMMAND>
Commands:
check Lint a pipeline file — exits 0 (clean) or 1 (errors found)
graph Visualise the pipeline as a job tree or Mermaid graph
```
Run `glint <command> --help` for command-specific options and examples.
### `glint check`
```bash ```bash
gitlab-sim [options] <pipeline.yml> glint check .gitlab-ci.yml
``` ```
Exits `0` when no errors are found, `1` when at least one error is reported. Exits `0` when no errors are found, `1` when at least one error is reported.
@@ -51,20 +73,20 @@ Exits `0` when no errors are found, `1` when at least one error is reported.
### Remote project includes ### Remote project includes
Pipelines that include templates from other GitLab projects are supported. Pipelines that include templates from other GitLab projects are supported.
Provide a token so `gitlab-sim` can fetch them: Provide a token so `glint` can fetch them:
```bash ```bash
# personal access token (read_api scope) # personal access token (read_api scope)
GITLAB_TOKEN=glpat-xxxx gitlab-sim .gitlab-ci.yml GITLAB_TOKEN=glpat-xxxx glint check .gitlab-ci.yml
# CI/CD job token (when running inside a pipeline) # CI/CD job token (when running inside a pipeline)
CI_JOB_TOKEN=$CI_JOB_TOKEN gitlab-sim .gitlab-ci.yml CI_JOB_TOKEN=$CI_JOB_TOKEN glint check .gitlab-ci.yml
# self-hosted GitLab # self-hosted GitLab
GITLAB_TOKEN=glpat-xxxx GITLAB_URL=https://gitlab.example.com gitlab-sim .gitlab-ci.yml GITLAB_TOKEN=glpat-xxxx GITLAB_URL=https://gitlab.example.com glint check .gitlab-ci.yml
# or via flags # or via flags
gitlab-sim --token glpat-xxxx --gitlab-url https://gitlab.example.com .gitlab-ci.yml glint check --token glpat-xxxx --gitlab-url https://gitlab.example.com .gitlab-ci.yml
``` ```
**Project includes** require a token; without one they are skipped with a **Project includes** require a token; without one they are skipped with a
@@ -101,29 +123,39 @@ The component file is looked up in order:
Component input parameters (`with:`) are not validated — they are resolved by Component input parameters (`with:`) are not validated — they are resolved by
GitLab at runtime. Jobs in fetched components may use `$[[ inputs.xxx ]]` GitLab at runtime. Jobs in fetched components may use `$[[ inputs.xxx ]]`
placeholders in fields like `stage`; `gitlab-sim` skips those fields rather placeholders in fields like `stage`; `glint` skips those fields rather
than producing false positive errors. than producing false positive errors.
### Graph output ### `glint graph`
Pass `--graph` to visualise the pipeline instead of running lint rules. Visualise the pipeline. Without a mode word, prints a job tree and the include
dependency graph separated by `---`.
```bash ```bash
# Include dependency graph (which files include which) → Mermaid to stdout # Default: job tree + include dependency graph
gitlab-sim --graph includes .gitlab-ci.yml > includes.mmd glint graph .gitlab-ci.yml
# GitLab-like pipeline layout → PNG (or SVG fallback) written to --graph-out dir # Job tree only (stages → jobs, like the tree command)
gitlab-sim --graph pipeline .gitlab-ci.yml glint graph tree .gitlab-ci.yml
# prints the output file path, e.g.: gitlab-sim-out/pipeline-20260607-143022.png
# Both at once: Mermaid to stdout + pipeline file path to stderr # Include dependency graph → Mermaid flowchart to stdout
gitlab-sim --graph all .gitlab-ci.yml > includes.mmd glint graph includes .gitlab-ci.yml > includes.mmd
# Custom output directory # GitLab-like pipeline layout → PNG (or SVG fallback) written to --out dir
gitlab-sim --graph pipeline --graph-out /tmp/graphs .gitlab-ci.yml glint graph pipeline .gitlab-ci.yml
# prints the output file path, e.g.: glint-out/pipeline-20260607-143022.png
# Mermaid to stdout + pipeline file path to stderr
glint graph all .gitlab-ci.yml > includes.mmd
# Custom output directory (pipeline mode)
glint graph pipeline --out /tmp/graphs .gitlab-ci.yml
``` ```
**Include graph** (`--graph includes`) — [Mermaid](https://mermaid.js.org) flowchart written to stdout. **Job tree** (`graph tree`) — stages as branches, jobs as leaves. Jobs with
`when: manual`, `when: delayed`, or `trigger:` are annotated in brackets.
**Include graph** (`graph includes`) — [Mermaid](https://mermaid.js.org) flowchart written to stdout.
Pipe to a `.mmd` file or paste into [mermaid.live](https://mermaid.live). Pipe to a `.mmd` file or paste into [mermaid.live](https://mermaid.live).
One node per include entry, colour-coded by type: One node per include entry, colour-coded by type:
- Orange (bold): the main pipeline file - Orange (bold): the main pipeline file
@@ -133,8 +165,8 @@ One node per include entry, colour-coded by type:
- Grey: `remote:` URL includes - Grey: `remote:` URL includes
- Light orange: GitLab-provided `template:` includes - Light orange: GitLab-provided `template:` includes
**Pipeline graph** (`--graph pipeline`) — GitLab CI-style SVG rendered to a timestamped file **Pipeline graph** (`graph pipeline`) — GitLab CI-style SVG rendered to a timestamped file
in the `--graph-out` directory (default: `gitlab-sim-out/`). Converted to PNG automatically in the `--out` directory (default: `glint-out/`). Converted to PNG automatically
when `rsvg-convert`, `inkscape`, or `magick` is available; falls back to SVG otherwise. when `rsvg-convert`, `inkscape`, or `magick` is available; falls back to SVG otherwise.
Jobs are colour-coded by type: Jobs are colour-coded by type:
- Blue (`#1f75cb`): regular jobs - Blue (`#1f75cb`): regular jobs
@@ -147,21 +179,22 @@ Classic mode draws L-shaped or straight connectors between stage columns otherwi
### Context simulation ### Context simulation
Pass `--branch`, `--tag`, or `--source` to see which jobs would run for a given Pass `--branch`, `--tag`, or `--source` to `glint check` to see which jobs
pipeline event. The pipeline is still fully linted; context output is printed first. would run for a given pipeline event. The pipeline is still fully linted;
context output is printed first.
```bash ```bash
# What runs on a push to develop? # What runs on a push to develop?
gitlab-sim --branch develop .gitlab-ci.yml glint check --branch develop .gitlab-ci.yml
# What runs when a v1.2.0 tag is pushed? # What runs when a v1.2.0 tag is pushed?
gitlab-sim --tag v1.2.0 .gitlab-ci.yml glint check --tag v1.2.0 .gitlab-ci.yml
# Merge request pipeline # Merge request pipeline
gitlab-sim --source merge_request_event .gitlab-ci.yml glint check --source merge_request_event .gitlab-ci.yml
# Arbitrary variable overrides (repeatable) # Arbitrary variable overrides (repeatable)
gitlab-sim --branch main --var DEPLOY_ENV=production .gitlab-ci.yml glint check --branch main --var DEPLOY_ENV=production .gitlab-ci.yml
``` ```
**Evaluated:** **Evaluated:**
@@ -212,63 +245,64 @@ OK: .gitlab-ci.yml — no issues found (5 jobs, 3 stages)
## Lint rules ## Lint rules
Every finding includes a stable rule ID (e.g. `GL003`) that can be used to filter output or reference a specific check in documentation.
### Pipeline-level ### Pipeline-level
| Severity | Rule | | ID | Severity | Rule |
|----------|------| |----|----------|------|
| ERROR | `workflow.rules[*].when` is not `always` or `never` | | GL002 | ERROR | `workflow.rules[*].when` is not `always` or `never` |
| WARNING | No `stages` defined (GitLab falls back to default stages) | | GL001 | WARNING | No `stages` defined (GitLab falls back to default stages) |
### Job-level — structure ### Job-level — structure
| Severity | Rule | | ID | Severity | Rule |
|----------|------| |----|----------|------|
| ERROR | Job is missing required `script` (or `run`) — non-trigger, non-template jobs | | GL003 | ERROR | Job is missing required `script` (or `run`) — non-trigger, non-template jobs |
| ERROR | Job references a `stage` not declared in `stages` | | GL004 | ERROR | Job references a `stage` not declared in `stages` |
| ERROR | `only` and `rules` used together on the same job | | GL005 | ERROR | `only` and `rules` used together on the same job |
| ERROR | `except` and `rules` used together on the same job | | GL006 | ERROR | `except` and `rules` used together on the same job |
| WARNING | `only`/`except` used (deprecated, prefer `rules`) | | GL007 | WARNING | `only`/`except` used (deprecated, prefer `rules`) |
### Job-level — keyword constraints ### Job-level — keyword constraints
| Severity | Rule | | ID | Severity | Rule |
|----------|------| |----|----------|------|
| ERROR | `when` is not one of `on_success`, `on_failure`, `always`, `manual`, `delayed`, `never` | | GL008 | ERROR | `when` is not one of `on_success`, `on_failure`, `always`, `manual`, `delayed`, `never` |
| ERROR | `when: delayed` without `start_in` | | GL009 | ERROR | `when: delayed` without `start_in` |
| ERROR | `start_in` set but `when` is not `delayed` | | GL010 | ERROR | `start_in` set but `when` is not `delayed` |
| ERROR | `parallel` integer not in range 2200 | | GL011 | ERROR | `parallel` integer not in range 2200, or map form missing `matrix` key |
| ERROR | `parallel` map form missing `matrix` key | | GL012 | ERROR | `retry` integer not in range 02, or `retry.max` out of range |
| ERROR | `retry` integer not in range 02 | | GL013 | ERROR | `retry.when` contains an unrecognised failure type |
| ERROR | `retry.max` not in range 02 | | GL014 | ERROR | `allow_failure` is not a boolean or a map with `exit_codes` |
| ERROR | `retry.when` contains an invalid failure type | | GL015 | ERROR | `interruptible` is not a boolean |
| ERROR | `allow_failure` is not a boolean or a map with `exit_codes` | | GL016 | ERROR | `trigger` job also has `script` |
| ERROR | `interruptible` is not a boolean | | GL017 | ERROR | `trigger` map missing `project` or `include` |
| ERROR | `trigger` job also has `script` | | GL018 | ERROR | `coverage` is not a regex pattern wrapped in `/` |
| ERROR | `trigger` map missing `project` or `include` | | GL019 | ERROR | `release` missing required `tag_name`, or is not a map |
| ERROR | `coverage` is not a regex pattern wrapped in `/` | | GL020 | ERROR | `environment.url` set without `environment.name`, or invalid `environment.action` |
| ERROR | `release` missing required `tag_name` | | GL021 | ERROR | `artifacts.when` invalid, or `artifacts.expose_as` set without `artifacts.paths` |
| ERROR | `environment.url` set without `environment.name` | | GL022 | WARNING | `pages` job `artifacts.paths` does not include `public` |
| ERROR | `environment.action` is not one of `start`, `stop`, `prepare`, `verify`, `access` | | GL023 | ERROR | `cache.when` or `cache.policy` has an invalid value |
| ERROR | `artifacts.when` is not `on_success`, `on_failure`, or `always` | | GL024 | ERROR | `rules[*].when` is not one of the valid `when` values |
| ERROR | `artifacts.expose_as` set without `artifacts.paths` | | GL025 | ERROR | `image` map form missing `name` key |
| ERROR | `cache.when` is not `on_success`, `on_failure`, or `always` | | GL026 | ERROR | `inherit.default` / `inherit.variables` is not a boolean or list |
| ERROR | `cache.policy` is not `pull`, `push`, or `pull-push` |
| ERROR | `rules[*].when` is not one of the valid `when` values |
| ERROR | `image` map form missing `name` key |
| ERROR | `inherit.default` / `inherit.variables` is not a boolean or list |
| WARNING | `pages` job `artifacts.paths` does not include `public` |
### Cross-job graph ### Cross-job graph
| Severity | Rule | | ID | Severity | Rule |
|----------|------| |----|----------|------|
| ERROR | `needs:` references a job that does not exist | | GL027 | ERROR/WARNING | `needs:` references a job that does not exist (WARNING when `optional: true`) |
| ERROR | `needs:` references a job in a later stage | | GL028 | ERROR | `needs:` references a job in a later stage |
| ERROR | Circular dependency detected in `needs:` graph | | GL029 | ERROR | Circular dependency detected in `needs:` graph |
| ERROR | `dependencies:` references a job that does not exist | | GL030 | ERROR | `dependencies:` references a job that does not exist |
| ERROR | `dependencies:` references a job in the same or a later stage | | GL031 | ERROR | `dependencies:` references a job in the same or a later stage |
| ERROR | `extends:` references an unknown job |
| ERROR | Cycle detected in `extends:` graph | ### Expression validation
| ID | Severity | Rule |
|----|----------|------|
| GL032 | WARNING | `rules:if:` references `$VAR` not declared in `variables:` (pipeline, job, or `workflow:rules:variables:`) — may be a false positive for variables set in GitLab CI/CD project settings |
### Hidden jobs (templates) ### Hidden jobs (templates)
@@ -285,6 +319,8 @@ task test # run Go unit tests
task lint-go # run go vet task lint-go # run go vet
task validate # run the binary against all testdata fixtures task validate # run the binary against all testdata fixtures
task ci # full check: vet → test → build → validate task ci # full check: vet → test → build → validate
task build-windows # cross-compile for Windows x64 (requires a tagged commit → glint-<tag>.exe)
task build-linux # cross-compile for Linux x64 (requires a tagged commit → glint-<tag>-linux-amd64)
task clean # remove build artifacts task clean # remove build artifacts
``` ```
@@ -292,7 +328,7 @@ task clean # remove build artifacts
``` ```
. .
├── cmd/gitlab-sim/ # CLI entrypoint ├── cmd/glint/ # CLI entrypoint
├── internal/ ├── internal/
│ ├── cicontext/ # CI variable context, rules:if: evaluator, job reachability │ ├── cicontext/ # CI variable context, rules:if: evaluator, job reachability
│ ├── fetcher/ # GitLab API client (project include fetching) │ ├── fetcher/ # GitLab API client (project include fetching)
+47 -54
View File
@@ -1,61 +1,39 @@
# Roadmap # Roadmap
This document tracks planned improvements to `gitlab-sim`. Items are grouped by theme, roughly in priority order within each group. Nothing here is a commitment — the tool is experimental and the list will shift as real usage surfaces better priorities. This document tracks planned improvements to `glint`. Items are grouped by theme, roughly in priority order within each group. Nothing here is a commitment — the tool is experimental and the list will shift as real usage surfaces better priorities.
--- ---
## Context-aware validation ## Context-aware validation — ✓ single-context shipped in v0.2.0; expression evaluator hardened post-v0.2.0
Pipelines in Git Flow, Trunk-Based Development, or any branching strategy are rarely uniform: jobs activate or skip based on `$CI_COMMIT_BRANCH`, `$CI_COMMIT_TAG`, `$CI_PIPELINE_SOURCE`, and similar runtime variables. Today `gitlab-sim` validates structure but cannot tell which jobs are actually reachable for a given context. Single-context simulation is fully implemented. Pass `--branch`, `--tag`, `--source`, or `--var` to either `glint check` or `glint graph`; jobs are evaluated and shown as active / manual / skipped.
The plan is to make the execution context injectable so the linter can evaluate `rules:if:` / `only` / `except` conditions and report per-context reachability.
**CLI surface**
```bash ```bash
# Push to develop # shipped: single-context simulation
gitlab-sim --branch develop .gitlab-ci.yml glint check --branch develop .gitlab-ci.yml
glint check --tag v1.2.0 .gitlab-ci.yml
# Tag push (v1.2.0) — sets CI_COMMIT_TAG and clears CI_COMMIT_BRANCH glint check --source merge_request_event --var CI_MERGE_REQUEST_TARGET_BRANCH_NAME=main .gitlab-ci.yml
gitlab-sim --tag v1.2.0 .gitlab-ci.yml glint graph tree --branch main .gitlab-ci.yml # tree annotated with [skipped] / [manual]
# Merge request pipeline
gitlab-sim --source merge_request_event \
--var CI_MERGE_REQUEST_TARGET_BRANCH_NAME=main \
.gitlab-ci.yml
# Explicit variable overrides for anything not covered by the shortcuts
gitlab-sim --var CI_COMMIT_BRANCH=feat/my-feature \
--var CI_ENVIRONMENT_NAME=staging \
.gitlab-ci.yml
# Simulate multiple contexts in one run (print per-context job tables)
gitlab-sim --context branch=main \
--context branch=develop \
--context tag=v1.0.0 \
.gitlab-ci.yml
``` ```
**What context injection enables** **Shipped post-v0.2.0 (unreleased)**
- Each job is resolved to **active** / **manual** / **skipped** for the given context - **`workflow:rules:variables:` propagation** — variables defined on the matching `workflow:rules:` entry are injected into the evaluation context before job `rules:if:` expressions are evaluated. Pipeline-level `variables:` defaults are also available. Priority chain (highest wins): `--var` > shortcuts > workflow-rule vars > pipeline defaults.
- Warn when the entire pipeline would produce zero runnable jobs (common mistake when a `workflow:rules:` block is too restrictive) - **Expression evaluator: multi-line expressions** — newlines in block-scalar and folded YAML `if:` values are now treated as whitespace; `||` / `&&` on a continuation line evaluate correctly.
- Lint only the active job subset — skip `needs:` / `dependencies:` cross-checks for jobs that never co-execute in that context - **Expression evaluator: `${VAR}` curly-brace syntax**`${CI_COMMIT_BRANCH}` is equivalent to `$CI_COMMIT_BRANCH` everywhere.
- `--context` multi-simulation: print a table showing which jobs activate per context, making it easy to audit Git Flow rules across branches and tags at once - **Expression evaluator: regex flags**`/pattern/i`, `/pattern/m`, `/pattern/s` are now supported; `i` maps to `(?i)` in Go's regexp.
-**Expression evaluator: variable as regex RHS**`$BRANCH =~ $PATTERN` where `$PATTERN` holds a `/regex/` string is evaluated correctly.
-**Expression evaluator: bare `true` / `false` keywords** — treated as the strings `"true"` / `"false"` matching GitLab CI's own behaviour; `$GATEWAY_ENABLED == true` now evaluates correctly.
-**Expression evaluator: integer literals**`$COUNT == 4`, `$ENABLED == 1`, `$DISABLED == 0` compare as decimal strings.
**Expression evaluator scope** **Remaining work**
GitLab's `rules:if:` expression language will be implemented incrementally: - **Multi-context simulation** — run multiple contexts in one invocation and print a comparison table:
```bash
| Priority | Operators / features | glint check --context branch=main --context branch=develop --context tag=v1.0.0 .gitlab-ci.yml
|----------|----------------------| ```
| 1 (MVP) | `==`, `!=`, `null` check, `&&`, `\|\|`, `!`, parentheses | - **Context-scoped linting** — skip `needs:`/`dependencies:` cross-checks for jobs that are statically unreachable in the given context
| 2 | Regex match `=~` / `!~` with `/pattern/` literals | - **`rules:changes:` evaluation** — path glob evaluation against the local git tree
| 3 | `$CI_COMMIT_BRANCH =~ /^feat\//`, anchored patterns |
| 4 | `only: branches / tags / merge_requests` shorthand mapping |
| 5 | `changes:` path glob evaluation against a real or mock file tree |
Predefined variables populated automatically from `--branch` / `--tag` / `--source` shortcuts: `CI_COMMIT_BRANCH`, `CI_COMMIT_TAG`, `CI_COMMIT_REF_NAME`, `CI_COMMIT_REF_SLUG`, `CI_PIPELINE_SOURCE`, `CI_DEFAULT_BRANCH` (defaulting to `main`).
--- ---
@@ -78,8 +56,8 @@ The current rule set covers the most common sources of broken pipelines. These a
## Include resolution ## Include resolution
- **`include: local:`** full resolution — parse and merge locally-referenced YAML files the same way remote project includes are handled; enables cross-file `extends:` and `needs:` validation for monorepo setups - ~~**`include: local:`** full resolution~~✓ shipped in v0.2.0; local files are read from disk, recursively resolved, and merged before linting
- **`include: remote:`** (URL) — fetch and merge plain HTTP/HTTPS URLs (no auth required) - ~~**`include: remote:`** (URL)~~✓ shipped post-v0.2.0; plain HTTPS URLs are fetched (unauthenticated), parsed, and merged; sub-includes are resolved recursively; unreachable URLs emit `[WARNING]` and linting continues
- **Recursive include depth limit** — guard against include cycles across files - **Recursive include depth limit** — guard against include cycles across files
- **Offline mode / cache** — persist fetched remote templates to a local cache directory; `--offline` flag to skip network calls and use only cached copies - **Offline mode / cache** — persist fetched remote templates to a local cache directory; `--offline` flag to skip network calls and use only cached copies
- **`include: inputs:`** — substitute CI component input values into fetched templates before merging, so component-scoped jobs get their correct `stage:` and keyword values - **`include: inputs:`** — substitute CI component input values into fetched templates before merging, so component-scoped jobs get their correct `stage:` and keyword values
@@ -99,8 +77,10 @@ Right now the only output is plain-text findings. Structured output enables inte
## Pipeline graph improvements ## Pipeline graph improvements
The SVG renderer covers the basic layout. These would bring it closer to GitLab's full interactive view. The SVG renderer and terminal tree cover the basic layout. These would bring it closer to GitLab's full interactive view.
- ~~**Terminal job tree**~~ — ✓ shipped in v0.2.0 as `glint graph tree`; stages as branches, jobs as leaves, context-aware annotations
- ~~**`glint graph includes` shows jobs per file**~~ — ✓ shipped post-v0.2.0; each include node shows the jobs it defines as dashed-arrow rounded nodes in a distinct style
- **Multi-job connector accuracy** — draw one connector per job pair rather than one per stage pair in classic mode, so pipelines with uneven columns look correct - **Multi-job connector accuracy** — draw one connector per job pair rather than one per stage pair in classic mode, so pipelines with uneven columns look correct
- **Job tooltip / detail panel** — embed a hidden `<title>` and `<desc>` per chip so SVG viewers show `stage`, `when`, `image`, and `needs` on hover - **Job tooltip / detail panel** — embed a hidden `<title>` and `<desc>` per chip so SVG viewers show `stage`, `when`, `image`, and `needs` on hover
- **`when: on_failure` visual distinction** — dashed border or distinct icon for failure-path jobs - **`when: on_failure` visual distinction** — dashed border or distinct icon for failure-path jobs
@@ -110,24 +90,36 @@ The SVG renderer covers the basic layout. These would bring it closer to GitLab'
--- ---
## Findings quality — ✓ file and line numbers shipped post-v0.2.0
~~**File and line numbers on findings**~~ — ✓ shipped post-v0.2.0; every `[ERROR]` / `[WARNING]` now includes the source file and exact line of the job key (e.g. `job "deploy" (src/deploy.yml:14): …`). Works across local includes, remote project templates, and fetched component templates.
**Remaining improvements**
- ~~**`needs: optional: true` false-positive errors**~~ — ✓ shipped post-v0.2.0; optional missing needs are downgraded to `[WARNING]`
- ~~**`extends:` jobs with missing script false errors**~~ — ✓ shipped post-v0.2.0; jobs using `extends:` that have no `script` after resolution emit `[WARNING]` (the script may come from an unfetchable remote base)
- **`rules:if:` static reachability** — report when a job's entire `rules:` block can never evaluate to `when: on_success` given the declared pipeline variables (pure static, no context required)
---
## CI / editor integration ## CI / editor integration
- **GitLab CI template** — a `.gitlab-ci.yml` snippet that runs `gitlab-sim` as a pipeline-validation job before the real pipeline executes; publishable to the GitLab CI/CD Catalog - **GitLab CI template** — a `.gitlab-ci.yml` snippet that runs `glint` as a pipeline-validation job before the real pipeline executes; publishable to the GitLab CI/CD Catalog
- **GitHub Actions action** — `uses: k3nny/gitlab-sim@v1` wrapper for repositories that mirror or manage GitLab pipelines from GitHub - **GitHub Actions action** — `uses: k3nny/glint@v1` wrapper for repositories that mirror or manage GitLab pipelines from GitHub
- **Pre-commit hook** — entry for [pre-commit](https://pre-commit.com) so `gitlab-sim` runs automatically on `git commit` when `.gitlab-ci.yml` changes - **Pre-commit hook** — entry for [pre-commit](https://pre-commit.com) so `glint` runs automatically on `git commit` when `.gitlab-ci.yml` changes
- **LSP server** — `gitlab-sim lsp` mode exposing diagnostics over the Language Server Protocol; enables inline squiggles in VS Code, JetBrains, Neovim, etc. without a dedicated extension - **LSP server** — `glint lsp` mode exposing diagnostics over the Language Server Protocol; enables inline squiggles in VS Code, JetBrains, Neovim, etc. without a dedicated extension
- **VS Code extension** — thin wrapper around the LSP server with syntax highlighting for `.gitlab-ci.yml` - **VS Code extension** — thin wrapper around the LSP server with syntax highlighting for `.gitlab-ci.yml`
--- ---
## Configuration ## Configuration
- **`.gitlab-sim.yml` config file** — project-level configuration for: - **`.glint.yml` config file** — project-level configuration for:
- Rule suppression by rule ID (e.g. `ignore: [no-only, missing-stages]`) - Rule suppression by rule ID (e.g. `ignore: [no-only, missing-stages]`)
- Severity overrides (demote specific errors to warnings) - Severity overrides (demote specific errors to warnings)
- Custom `stages` allowlist for projects that use a non-standard default set - Custom `stages` allowlist for projects that use a non-standard default set
- Token and URL defaults so flags are not needed in every invocation - Token and URL defaults so flags are not needed in every invocation
- **Inline suppression comments** — `# gitlab-sim: ignore next-line <rule-id>` in the pipeline YAML - **Inline suppression comments** — `# glint: ignore next-line <rule-id>` in the pipeline YAML
--- ---
@@ -136,5 +128,6 @@ The SVG renderer covers the basic layout. These would bring it closer to GitLab'
- **Structured rule IDs** — assign a stable short ID to every rule (e.g. `GS001`) so suppression, documentation, and SARIF output are stable across versions - **Structured rule IDs** — assign a stable short ID to every rule (e.g. `GS001`) so suppression, documentation, and SARIF output are stable across versions
- **`--explain <rule-id>`** — print the rule description, rationale, and an example fix - **`--explain <rule-id>`** — print the rule description, rationale, and an example fix
- ~~**Semantic versioning and first release**~~ — shipped as `v0.1.0` (2026-06-07) - ~~**Semantic versioning and first release**~~ — shipped as `v0.1.0` (2026-06-07)
- ~~**Subcommand CLI**~~ — shipped as `v0.2.0` (2026-06-11); `glint check` / `glint graph [mode]` with ruff-style `--help`
- **Changelog automation** — generate release notes from Conventional Commits via `git-cliff` or similar - **Changelog automation** — generate release notes from Conventional Commits via `git-cliff` or similar
- **Fuzz testing** — add a `go test -fuzz` target for the YAML parser to harden it against malformed input - **Fuzz testing** — add a `go test -fuzz` target for the YAML parser to harden it against malformed input
+85 -20
View File
@@ -1,7 +1,7 @@
version: "3" version: "3"
vars: vars:
BINARY: gitlab-sim BINARY: glint
GO: /usr/local/go/bin/go GO: /usr/local/go/bin/go
tasks: tasks:
@@ -10,9 +10,9 @@ tasks:
cmd: task --list cmd: task --list
build: build:
desc: Build the gitlab-sim binary desc: Build the glint binary
cmds: cmds:
- "{{.GO}} build -o {{.BINARY}} ./cmd/gitlab-sim/..." - "{{.GO}} build -o {{.BINARY}} ./cmd/glint/..."
sources: sources:
- "**/*.go" - "**/*.go"
- go.mod - go.mod
@@ -24,50 +24,115 @@ tasks:
cmd: "{{.GO}} test ./..." cmd: "{{.GO}} test ./..."
validate: validate:
desc: Run gitlab-sim against all testdata fixtures desc: Run glint against all testdata fixtures
deps: [build] deps: [build]
cmds: cmds:
- cmd: ./{{.BINARY}} testdata/valid.yml - cmd: ./{{.BINARY}} check testdata/valid.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} testdata/extends.yml - cmd: ./{{.BINARY}} check testdata/extends.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} testdata/keywords_valid.yml - cmd: ./{{.BINARY}} check testdata/keywords_valid.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} testdata/invalid.yml - cmd: ./{{.BINARY}} check testdata/invalid.yml
ignore_error: true ignore_error: true
- cmd: ./{{.BINARY}} testdata/needs.yml - cmd: ./{{.BINARY}} check testdata/needs.yml
ignore_error: true ignore_error: true
- cmd: ./{{.BINARY}} testdata/needs_cycle.yml - cmd: ./{{.BINARY}} check testdata/needs_cycle.yml
ignore_error: true ignore_error: true
- cmd: ./{{.BINARY}} testdata/keywords_invalid.yml - cmd: ./{{.BINARY}} check testdata/keywords_invalid.yml
ignore_error: true ignore_error: true
- cmd: ./{{.BINARY}} testdata/includes_project.yml - cmd: ./{{.BINARY}} check testdata/includes_remote.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} testdata/includes_component.yml - cmd: ./{{.BINARY}} check testdata/includes_project.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} testdata/context_rules.yml - cmd: ./{{.BINARY}} check testdata/includes_component.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} --branch main testdata/context_rules.yml - cmd: ./{{.BINARY}} check testdata/script_multiline.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} --branch develop testdata/context_rules.yml - cmd: ./{{.BINARY}} check testdata/context_rules.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} --branch feat/my-feature testdata/context_rules.yml - cmd: ./{{.BINARY}} check --branch main testdata/context_rules.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} --tag v1.0.0 testdata/context_rules.yml - cmd: ./{{.BINARY}} check --branch develop testdata/context_rules.yml
ignore_error: false
- cmd: ./{{.BINARY}} check --branch feat/my-feature testdata/context_rules.yml
ignore_error: false
- cmd: ./{{.BINARY}} check --tag v1.0.0 testdata/context_rules.yml
ignore_error: false
- cmd: ./{{.BINARY}} check testdata/rules_if_expr.yml
ignore_error: false
- cmd: ./{{.BINARY}} check --branch main testdata/rules_if_expr.yml
ignore_error: false
- cmd: ./{{.BINARY}} check --branch feat/x testdata/rules_if_expr.yml
ignore_error: false
- cmd: ./{{.BINARY}} check testdata/workflow_vars.yml
ignore_error: false
- cmd: ./{{.BINARY}} check --branch main testdata/workflow_vars.yml
ignore_error: false
- cmd: ./{{.BINARY}} check --branch develop testdata/workflow_vars.yml
ignore_error: false
- cmd: ./{{.BINARY}} check --branch feat/x testdata/workflow_vars.yml
ignore_error: false
- cmd: ./{{.BINARY}} check testdata/variable_refs.yml
ignore_error: false
- cmd: ./{{.BINARY}} check testdata/variable_refs_included.yml
ignore_error: false
- cmd: ./{{.BINARY}} check testdata/samba/.gitlab-ci.yml
ignore_error: false
- cmd: ./{{.BINARY}} check testdata/samba/.gitlab-ci-coverage.yml
ignore_error: false
- cmd: ./{{.BINARY}} check testdata/samba/.gitlab-ci-private.yml
ignore_error: false ignore_error: false
lint-go: lint-go:
desc: Run go vet on all packages desc: Run go vet on all packages
cmd: "{{.GO}} vet ./..." cmd: "{{.GO}} vet ./..."
lint-static:
desc: Run staticcheck on all packages
cmd: "{{.GO}} tool staticcheck ./..."
ci: ci:
desc: Full CI check — vet, test, build, validate desc: Full CI check — vet, staticcheck, test, build, validate
cmds: cmds:
- task: lint-go - task: lint-go
- task: lint-static
- task: test - task: test
- task: build - task: build
- task: validate - task: validate
build-windows:
desc: Build the glint binary for Windows x64 (requires a tagged commit)
vars:
TAG:
sh: git describe --tags --exact-match
preconditions:
- sh: git describe --tags --exact-match
msg: "Current commit is not tagged — Windows build requires a git tag"
cmds:
- "GOOS=windows GOARCH=amd64 {{.GO}} build -o {{.BINARY}}-{{.TAG}}.exe ./cmd/glint/..."
sources:
- "**/*.go"
- go.mod
generates:
- "{{.BINARY}}-{{.TAG}}.exe"
build-linux:
desc: Build the glint binary for Linux x64 (requires a tagged commit)
vars:
TAG:
sh: git describe --tags --exact-match
preconditions:
- sh: git describe --tags --exact-match
msg: "Current commit is not tagged — Linux build requires a git tag"
cmds:
- "GOOS=linux GOARCH=amd64 {{.GO}} build -o {{.BINARY}}-{{.TAG}}-linux-amd64 ./cmd/glint/..."
sources:
- "**/*.go"
- go.mod
generates:
- "{{.BINARY}}-{{.TAG}}-linux-amd64"
clean: clean:
desc: Remove build artifacts desc: Remove build artifacts
cmd: rm -f {{.BINARY}} cmd: rm -f {{.BINARY}} {{.BINARY}}-*.exe {{.BINARY}}-*-linux-amd64
Binary file not shown.

After

Width:  |  Height:  |  Size: 106 KiB

+351
View File
@@ -0,0 +1,351 @@
package main
import (
"flag"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"git.k3nny.fr/glint/internal/cicontext"
"git.k3nny.fr/glint/internal/fetcher"
"git.k3nny.fr/glint/internal/graph"
"git.k3nny.fr/glint/internal/linter"
"git.k3nny.fr/glint/internal/model"
"git.k3nny.fr/glint/internal/resolver"
)
const globalUsage = `glint: Lint and visualise GitLab CI pipelines locally.
Usage: glint [OPTIONS] <COMMAND>
Commands:
check Lint a pipeline file — exits 0 (clean) or 1 (errors found)
graph Visualise the pipeline as a job tree or Mermaid graph
Options:
-h, --help Print help
For help with a specific command, see: ` + "`glint <command> --help`" + `.
`
func main() {
if len(os.Args) < 2 {
fmt.Fprint(os.Stderr, globalUsage)
os.Exit(2)
}
switch os.Args[1] {
case "check":
cmdCheck(os.Args[2:])
case "graph":
cmdGraph(os.Args[2:])
case "-h", "--help", "help":
fmt.Fprint(os.Stderr, globalUsage)
default:
fmt.Fprintf(os.Stderr, "glint: unknown command %q\n\n%s", os.Args[1], globalUsage)
os.Exit(2)
}
}
// multiFlag allows a flag to be specified multiple times.
type multiFlag []string
func (f *multiFlag) String() string { return strings.Join(*f, ", ") }
func (f *multiFlag) Set(v string) error {
*f = append(*f, v)
return nil
}
func cmdCheck(args []string) {
fs := flag.NewFlagSet("glint check", flag.ExitOnError)
token := fs.String("token", "", "GitLab personal access token (overrides GITLAB_TOKEN)")
gitlabURL := fs.String("gitlab-url", "", "GitLab instance URL (overrides CI_SERVER_URL / GITLAB_URL)")
branch := fs.String("branch", "", "simulate a branch push (sets CI_COMMIT_BRANCH, …)")
tag := fs.String("tag", "", "simulate a tag push (sets CI_COMMIT_TAG, …)")
source := fs.String("source", "", "set CI_PIPELINE_SOURCE")
var vars multiFlag
fs.Var(&vars, "var", "set a CI variable as KEY=VALUE; repeatable")
fs.Usage = func() {
fmt.Fprint(os.Stderr, `Lint a GitLab CI pipeline file.
Resolves local includes and extends chains, then runs all lint rules.
Exits 0 when no errors are found, 1 when at least one error is reported.
Usage: glint check [OPTIONS] <PIPELINE>
Arguments:
<PIPELINE> Path to the .gitlab-ci.yml file to lint
Options:
--token <TOKEN>
GitLab personal access token. Required to fetch project: includes;
component: includes are attempted unauthenticated.
[env: GITLAB_TOKEN | CI_JOB_TOKEN | GITLAB_PRIVATE_TOKEN]
--gitlab-url <URL>
GitLab instance URL.
[env: CI_SERVER_URL | GITLAB_URL] [default: https://gitlab.com]
--branch <NAME>
Simulate a branch push. Populates: CI_COMMIT_BRANCH,
CI_COMMIT_REF_NAME, CI_COMMIT_REF_SLUG, CI_PIPELINE_SOURCE=push.
--tag <NAME>
Simulate a tag push. Populates: CI_COMMIT_TAG, CI_COMMIT_REF_NAME,
CI_COMMIT_REF_SLUG, CI_PIPELINE_SOURCE=push. Clears CI_COMMIT_BRANCH.
--source <EVENT>
Override CI_PIPELINE_SOURCE.
[possible values: push, merge_request_event, schedule, web, api]
--var <KEY=VALUE>
Set or override a CI variable. Takes precedence over --branch, --tag,
and --source. Repeatable.
-h, --help
Print help
Examples:
glint check .gitlab-ci.yml
glint check --branch main .gitlab-ci.yml
GITLAB_TOKEN=glpat-xxxx glint check .gitlab-ci.yml
glint check --token glpat-xxxx --gitlab-url https://gitlab.example.com .gitlab-ci.yml
glint check --branch main --var DEPLOY_ENV=production .gitlab-ci.yml
`)
}
_ = fs.Parse(args)
if fs.NArg() != 1 {
fs.Usage()
os.Exit(2)
}
path := fs.Arg(0)
cfg := fetcher.AutoConfig().WithOverrides(*gitlabURL, *token)
p, err := model.Parse(path)
if err != nil {
fmt.Fprintf(os.Stderr, "error: %v\n", err)
os.Exit(2)
}
rootDir := filepath.Dir(filepath.Clean(path))
warnings, _ := resolver.ResolveIncludes(p, cfg, rootDir)
for _, w := range warnings {
fmt.Fprintf(os.Stderr, "[WARNING] include %s\n", w)
}
extWarnings, err := resolver.Resolve(p)
if err != nil {
fmt.Fprintf(os.Stderr, "error: resolving extends: %v\n", err)
os.Exit(2)
}
for _, w := range extWarnings {
fmt.Fprintf(os.Stderr, "[WARNING] job %q extends unknown job %q; extends chain skipped\n", w.Job, w.Base)
}
ctx := cicontext.New(*branch, *tag, *source, vars)
if !ctx.IsEmpty() {
enrichContext(ctx, p)
printContext(p, ctx)
}
findings := linter.Lint(p)
hasErrors := false
for _, f := range findings {
fmt.Println(f)
if f.Severity == linter.Error {
hasErrors = true
}
}
if len(findings) == 0 {
fmt.Printf("OK: %s — no issues found (%d job(s), %d stage(s))\n", path, len(p.Jobs), len(p.Stages))
} else {
errCount := 0
for _, f := range findings {
if f.Severity == linter.Error {
errCount++
}
}
fmt.Printf("%d finding(s): %d error(s)\n", len(findings), errCount)
}
if hasErrors {
os.Exit(1)
}
}
var knownGraphModes = map[string]bool{
"tree": true, "includes": true, "pipeline": true, "all": true,
}
func cmdGraph(args []string) {
// Optional mode word must come before any flags or the file path.
mode := "default"
if len(args) > 0 && !strings.HasPrefix(args[0], "-") && knownGraphModes[args[0]] {
mode = args[0]
args = args[1:]
}
fs := flag.NewFlagSet("glint graph", flag.ExitOnError)
token := fs.String("token", "", "GitLab personal access token (overrides GITLAB_TOKEN)")
gitlabURL := fs.String("gitlab-url", "", "GitLab instance URL (overrides CI_SERVER_URL / GITLAB_URL)")
out := fs.String("out", "glint-out", "output directory for Mermaid graph files (pipeline mode)")
fs.Usage = func() {
fmt.Fprint(os.Stderr, `Visualise the pipeline as a job tree and/or Mermaid graph.
Usage: glint graph [MODE] [OPTIONS] <PIPELINE>
Arguments:
[MODE] Graph mode; must appear before options [default: tree+includes]
[possible values: tree, includes, pipeline, all]
<PIPELINE> Path to the .gitlab-ci.yml file
Options:
--out <DIR>
Output directory for rendered graph files.
Used by the pipeline and all modes only. [default: glint-out]
--token <TOKEN>
GitLab personal access token. Used to fetch remote project: includes
when building the include dependency graph.
[env: GITLAB_TOKEN | CI_JOB_TOKEN | GITLAB_PRIVATE_TOKEN]
--gitlab-url <URL>
GitLab instance URL.
[env: CI_SERVER_URL | GITLAB_URL] [default: https://gitlab.com]
--branch <NAME>
Simulate a branch push. Jobs in tree output are annotated with their
evaluated state ([skipped] or [manual]; no tag means active).
Populates: CI_COMMIT_BRANCH, CI_COMMIT_REF_NAME, CI_COMMIT_REF_SLUG,
CI_PIPELINE_SOURCE=push.
--tag <NAME>
Simulate a tag push. Populates: CI_COMMIT_TAG, CI_COMMIT_REF_NAME,
CI_COMMIT_REF_SLUG, CI_PIPELINE_SOURCE=push. Clears CI_COMMIT_BRANCH.
--source <EVENT>
Override CI_PIPELINE_SOURCE.
[possible values: push, merge_request_event, schedule, web, api]
--var <KEY=VALUE>
Set or override a CI variable. Repeatable.
-h, --help
Print help
Examples:
glint graph .gitlab-ci.yml
glint graph tree .gitlab-ci.yml
glint graph tree --branch main .gitlab-ci.yml
glint graph includes .gitlab-ci.yml > includes.mmd
glint graph pipeline .gitlab-ci.yml
glint graph pipeline --out /tmp/graphs .gitlab-ci.yml
glint graph all .gitlab-ci.yml > includes.mmd
`)
}
branch := fs.String("branch", "", "simulate a branch push (sets CI_COMMIT_BRANCH, …)")
tag := fs.String("tag", "", "simulate a tag push (sets CI_COMMIT_TAG, …)")
source := fs.String("source", "", "set CI_PIPELINE_SOURCE")
var vars multiFlag
fs.Var(&vars, "var", "set a CI variable as KEY=VALUE; repeatable")
_ = fs.Parse(args)
if fs.NArg() != 1 {
fs.Usage()
os.Exit(2)
}
path := fs.Arg(0)
cfg := fetcher.AutoConfig().WithOverrides(*gitlabURL, *token)
p, err := model.Parse(path)
if err != nil {
fmt.Fprintf(os.Stderr, "error: %v\n", err)
os.Exit(2)
}
rootDir := filepath.Dir(filepath.Clean(path))
resolver.ResolveIncludes(p, cfg, rootDir) //nolint:errcheck
resolver.Resolve(p) //nolint:errcheck
ctx := cicontext.New(*branch, *tag, *source, vars)
if !ctx.IsEmpty() {
enrichContext(ctx, p)
}
switch mode {
case "default":
fmt.Print(graph.Tree(p, ctx))
fmt.Println("---")
fmt.Print(graph.Includes(path, p.Include, cfg))
case "tree":
fmt.Print(graph.Tree(p, ctx))
case "includes":
fmt.Print(graph.Includes(path, p.Include, cfg))
case "pipeline":
outPath, err := graph.RenderPipeline(p, *out)
if err != nil {
fmt.Fprintf(os.Stderr, "error: rendering pipeline graph: %v\n", err)
os.Exit(2)
}
fmt.Println(outPath)
case "all":
fmt.Print(graph.Includes(path, p.Include, cfg))
outPath, err := graph.RenderPipeline(p, *out)
if err != nil {
fmt.Fprintf(os.Stderr, "error: rendering pipeline graph: %v\n", err)
os.Exit(2)
}
fmt.Fprintln(os.Stderr, outPath)
}
}
// enrichContext injects pipeline-level variable defaults and then
// workflow-rule-generated variables into ctx before job evaluation.
// Injection respects pinned variables (--branch/--tag/--source/--var always win).
func enrichContext(ctx *cicontext.Context, p *model.Pipeline) {
// Pipeline variables: injected as defaults (lowest priority).
for k, v := range cicontext.ExtractStringVars(p.Variables) {
ctx.Inject(k, v)
}
// Workflow rules: evaluate to find which rule matches, then inject its variables.
runs, ruleVars := cicontext.EvalWorkflow(p, ctx)
if !runs {
fmt.Fprintln(os.Stderr, "[WARNING] workflow:rules: pipeline would not start for this context")
}
for k, v := range ruleVars {
ctx.Inject(k, v)
}
}
func printContext(p *model.Pipeline, ctx *cicontext.Context) {
fmt.Printf("Context: %s\n\n", ctx.Summary())
byState := map[cicontext.JobState][]string{}
for name, job := range p.Jobs {
if strings.HasPrefix(name, ".") {
continue
}
state := cicontext.EvalJob(job, ctx)
byState[state] = append(byState[state], name)
}
for _, jobs := range byState {
sort.Strings(jobs)
}
printJobGroup("Active ", byState[cicontext.JobActive])
printJobGroup("Manual ", byState[cicontext.JobManual])
printJobGroup("Skipped", byState[cicontext.JobSkipped])
fmt.Println()
}
func printJobGroup(label string, jobs []string) {
if len(jobs) == 0 {
return
}
fmt.Printf("%s (%d): %s\n", label, len(jobs), strings.Join(jobs, ", "))
}
+12 -2
View File
@@ -1,5 +1,15 @@
module git.k3nny.fr/gitlab-sim module git.k3nny.fr/glint
go 1.26.4 go 1.26.4
require gopkg.in/yaml.v3 v3.0.1 // indirect require (
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c // indirect
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678 // indirect
golang.org/x/mod v0.31.0 // indirect
golang.org/x/sync v0.19.0 // indirect
golang.org/x/tools v0.40.1-0.20260108161641-ca281cf95054 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
honnef.co/go/tools v0.7.0 // indirect
)
tool honnef.co/go/tools/cmd/staticcheck
+13
View File
@@ -1,3 +1,16 @@
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c h1:pxW6RcqyfI9/kWtOwnv/G+AzdKuy2ZrqINhenH4HyNs=
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa h1:FRnLl4eNAQl8hwxVVC17teOw8kdjVDVAiFMtgUdTSRQ=
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678 h1:1P7xPZEwZMoBoz0Yze5Nx2/4pxj6nw9ZqHWXqP0iRgQ=
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=
golang.org/x/mod v0.31.0 h1:HaW9xtz0+kOcWKwli0ZXy79Ix+UW/vOfmWI5QVd2tgI=
golang.org/x/mod v0.31.0/go.mod h1:43JraMp9cGx1Rx3AqioxrbrhNsLl2l/iNAvuBkrezpg=
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/tools v0.40.1-0.20260108161641-ca281cf95054 h1:CHVDrNHx9ZoOrNN9kKWYIbT5Rj+WF2rlwPkhbQQ5V4U=
golang.org/x/tools v0.40.1-0.20260108161641-ca281cf95054/go.mod h1:Ik/tzLRlbscWpqqMRjyWYDisX8bG13FrdXp3o4Sr9lc=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.7.0 h1:w6WUp1VbkqPEgLz4rkBzH/CSU6HkoqNLp6GstyTx3lU=
honnef.co/go/tools v0.7.0/go.mod h1:pm29oPxeP3P82ISxZDgIYeOaf9ta6Pi0EWvCFoLG2vc=
+58 -11
View File
@@ -7,6 +7,7 @@ import "strings"
// Variables are keyed by their name without the leading $. // Variables are keyed by their name without the leading $.
type Context struct { type Context struct {
Vars map[string]string Vars map[string]string
pinned map[string]bool // vars set via --var or shortcuts; never overwritten by Inject
} }
// New builds a Context from high-level shortcut values and optional KEY=VALUE // New builds a Context from high-level shortcut values and optional KEY=VALUE
@@ -17,46 +18,55 @@ type Context struct {
// preserving the existing linting behaviour when no context flags are given. // preserving the existing linting behaviour when no context flags are given.
// //
// Override priority (highest wins): extraVars > branch/tag/source shortcuts. // Override priority (highest wins): extraVars > branch/tag/source shortcuts.
// Both shortcut-derived and extraVar variables are pinned — they will not be
// overwritten by Inject (used for pipeline-level and workflow-rule variables).
func New(branch, tag, source string, extraVars []string) *Context { func New(branch, tag, source string, extraVars []string) *Context {
if branch == "" && tag == "" && source == "" && len(extraVars) == 0 { if branch == "" && tag == "" && source == "" && len(extraVars) == 0 {
return &Context{} return &Context{}
} }
vars := make(map[string]string) vars := make(map[string]string)
pinned := make(map[string]bool)
pin := func(k, v string) {
vars[k] = v
pinned[k] = true
}
if branch != "" { if branch != "" {
vars["CI_COMMIT_BRANCH"] = branch pin("CI_COMMIT_BRANCH", branch)
vars["CI_COMMIT_REF_NAME"] = branch pin("CI_COMMIT_REF_NAME", branch)
vars["CI_COMMIT_REF_SLUG"] = slugify(branch) pin("CI_COMMIT_REF_SLUG", slugify(branch))
if source == "" { if source == "" {
source = "push" source = "push"
} }
} }
if tag != "" { if tag != "" {
vars["CI_COMMIT_TAG"] = tag pin("CI_COMMIT_TAG", tag)
vars["CI_COMMIT_REF_NAME"] = tag pin("CI_COMMIT_REF_NAME", tag)
vars["CI_COMMIT_REF_SLUG"] = slugify(tag) pin("CI_COMMIT_REF_SLUG", slugify(tag))
delete(vars, "CI_COMMIT_BRANCH") // tag pushes have no branch variable delete(vars, "CI_COMMIT_BRANCH")
delete(pinned, "CI_COMMIT_BRANCH")
if source == "" { if source == "" {
source = "push" source = "push"
} }
} }
if source != "" { if source != "" {
vars["CI_PIPELINE_SOURCE"] = source pin("CI_PIPELINE_SOURCE", source)
} }
if _, ok := vars["CI_DEFAULT_BRANCH"]; !ok { if _, ok := vars["CI_DEFAULT_BRANCH"]; !ok {
vars["CI_DEFAULT_BRANCH"] = "main" vars["CI_DEFAULT_BRANCH"] = "main"
} }
// KEY=VALUE overrides win over shortcuts. // KEY=VALUE overrides win over shortcuts and everything else.
for _, kv := range extraVars { for _, kv := range extraVars {
k, v, ok := strings.Cut(kv, "=") k, v, ok := strings.Cut(kv, "=")
if ok { if ok {
vars[k] = v pin(k, v)
} }
} }
return &Context{Vars: vars} return &Context{Vars: vars, pinned: pinned}
} }
// IsEmpty reports whether no variables have been set (no context flags given). // IsEmpty reports whether no variables have been set (no context flags given).
@@ -73,6 +83,21 @@ func (c *Context) Get(key string) string {
return c.Vars[key] return c.Vars[key]
} }
// Inject sets key=value only if key is not already pinned (i.e. not set via
// --branch / --tag / --source / --var). Used to inject pipeline-level variable
// defaults and workflow-rule variables without overriding explicit user input.
// Calling Inject in order from lowest-priority to highest-priority source
// ensures later calls win over earlier ones.
func (c *Context) Inject(key, value string) {
if c.pinned[key] {
return
}
if c.Vars == nil {
c.Vars = make(map[string]string)
}
c.Vars[key] = value
}
// Summary returns a short human-readable description of the context for CLI output. // Summary returns a short human-readable description of the context for CLI output.
func (c *Context) Summary() string { func (c *Context) Summary() string {
if c.IsEmpty() { if c.IsEmpty() {
@@ -90,6 +115,28 @@ func (c *Context) Summary() string {
return strings.Join(parts, ", ") return strings.Join(parts, ", ")
} }
// ExtractStringVars converts a map[string]any variable block (as used by
// Pipeline.Variables and Rule.Variables) to a flat map[string]string.
// Plain string values are used directly. Extended {value: "..."} map form
// uses the "value" key. Other forms are skipped.
func ExtractStringVars(m map[string]any) map[string]string {
if len(m) == 0 {
return nil
}
out := make(map[string]string, len(m))
for k, v := range m {
switch val := v.(type) {
case string:
out[k] = val
case map[string]any:
if s, ok := val["value"].(string); ok {
out[k] = s
}
}
}
return out
}
// slugify converts a ref name to its GitLab slug form: // slugify converts a ref name to its GitLab slug form:
// lowercased, non-alphanumeric characters replaced with '-', leading/trailing '-' removed. // lowercased, non-alphanumeric characters replaced with '-', leading/trailing '-' removed.
func slugify(s string) string { func slugify(s string) string {
+145 -17
View File
@@ -9,12 +9,15 @@ import (
// variable resolver. // variable resolver.
// //
// Supported: // Supported:
// - Variable references: $VAR_NAME // - Variable references: $VAR_NAME or ${VAR_NAME}
// - String literals: "value" or 'value' // - String literals: "value" or 'value'
// - Null keyword: null // - Null keyword: null
// - Comparison: == != =~ !~ // - Comparison: == != =~ !~
// - Boolean: && || ! // - Boolean: && || !
// - Grouping: ( ) // - Grouping: ( )
// - Regex flags: /pattern/i (case-insensitive), /pattern/m, /pattern/s
// - Multi-line: newlines between tokens are treated as whitespace
// - Variable regex RHS: $VAR =~ $PATTERN when $PATTERN holds a /regex/ string
// //
// Regex patterns use Go's regexp syntax, which covers the common RE2 subset // Regex patterns use Go's regexp syntax, which covers the common RE2 subset
// used by GitLab CI. Unsupported or unparseable expressions fall back to true // used by GitLab CI. Unsupported or unparseable expressions fall back to true
@@ -56,8 +59,13 @@ func (p *exprParser) consume(tok string) bool {
} }
func (p *exprParser) skipWS() { func (p *exprParser) skipWS() {
for p.pos < len(p.s) && (p.s[p.pos] == ' ' || p.s[p.pos] == '\t') { for p.pos < len(p.s) {
b := p.s[p.pos]
if b == ' ' || b == '\t' || b == '\n' || b == '\r' {
p.pos++ p.pos++
continue
}
break
} }
} }
@@ -67,11 +75,11 @@ func (p *exprParser) skipWS() {
// and_expr → not_expr ( '&&' not_expr )* // and_expr → not_expr ( '&&' not_expr )*
// not_expr → '!' not_expr | primary // not_expr → '!' not_expr | primary
// primary → '(' or_expr ')' | comparison // primary → '(' or_expr ')' | comparison
// comparison → value ( op value | regex_op regex )? | value // comparison → value ( op value | regex_op regex_rhs )?
// value → '$' ident | '"' … '"' | "'" … "'" | 'null' // value → '$' '{' ident '}' | '$' ident | '"' … '"' | "'" … "'" | 'null'
// op → '==' | '!=' // op → '==' | '!='
// regex_op → '=~' | '!~' // regex_op → '=~' | '!~'
// regex '/' … '/' // regex_rhs → '/' … '/' flags? | '$' ident (where ident value is '/…/flags')
func (p *exprParser) parseOr() (bool, bool) { func (p *exprParser) parseOr() (bool, bool) {
left, ok := p.parseAnd() left, ok := p.parseAnd()
@@ -165,8 +173,11 @@ func (p *exprParser) parseComparison() (bool, bool) {
case p.consume("=~"): case p.consume("=~"):
p.skipWS() p.skipWS()
pat, ok := p.parseRegexLiteral() pat, patOk, permissive := p.parseRegexRHS()
if !ok { if permissive {
return true, true
}
if !patOk {
return false, false return false, false
} }
re, err := regexp.Compile(pat) re, err := regexp.Compile(pat)
@@ -177,8 +188,11 @@ func (p *exprParser) parseComparison() (bool, bool) {
case p.consume("!~"): case p.consume("!~"):
p.skipWS() p.skipWS()
pat, ok := p.parseRegexLiteral() pat, patOk, permissive := p.parseRegexRHS()
if !ok { if permissive {
return true, true
}
if !patOk {
return false, false return false, false
} }
re, err := regexp.Compile(pat) re, err := regexp.Compile(pat)
@@ -192,13 +206,49 @@ func (p *exprParser) parseComparison() (bool, bool) {
return leftStr != "", true return leftStr != "", true
} }
// parseValue reads $VAR, "string", 'string', or null. // parseRegexRHS parses the right-hand side of =~ / !~ operators.
// null and undefined variables both produce an empty string. // Returns (pattern, ok, permissive):
// - /regex/flags literal → (pattern, true, false)
// - $VAR whose value is /regex/flags → (pattern, true, false)
// - $VAR whose value is empty or not a /regex/ → ("", false, true) — caller uses permissive true
// - parse error → ("", false, false)
func (p *exprParser) parseRegexRHS() (pat string, ok bool, permissive bool) {
if p.peek() == '/' {
pat, ok = p.parseRegexLiteral()
return pat, ok, false
}
if p.peek() == '$' {
varVal, varOk := p.parseValue()
if !varOk {
return "", false, false
}
pat, ok = extractRegexFromString(varVal)
if !ok {
return "", false, true // variable is not a /regex/ value → permissive
}
return pat, true, false
}
return "", false, false
}
// parseValue reads $VAR, ${VAR}, "string", 'string', null, true, false, or an
// integer literal. null and undefined variables both produce an empty string.
// true/false and integers produce their string representations (GitLab CI
// compares all values as strings).
func (p *exprParser) parseValue() (string, bool) { func (p *exprParser) parseValue() (string, bool) {
p.skipWS() p.skipWS()
if p.peek() == '$' { if p.peek() == '$' {
p.pos++ // consume '$' p.pos++ // consume '$'
if p.peek() == '{' {
p.pos++ // consume '{'
name := p.parseIdent()
if name == "" || p.peek() != '}' {
return "", false
}
p.pos++ // consume '}'
return p.vars(name), true
}
name := p.parseIdent() name := p.parseIdent()
if name == "" { if name == "" {
return "", false return "", false
@@ -206,12 +256,18 @@ func (p *exprParser) parseValue() (string, bool) {
return p.vars(name), true return p.vars(name), true
} }
// null keyword — must not be a prefix of a longer identifier. // Keywords and string literals must not be prefixes of longer identifiers.
if p.startsWith("null") { for _, kw := range []struct{ tok, val string }{
end := p.pos + 4 {"null", ""},
{"true", "true"},
{"false", "false"},
} {
if p.startsWith(kw.tok) {
end := p.pos + len(kw.tok)
if end >= len(p.s) || !isIdentByte(p.s[end]) { if end >= len(p.s) || !isIdentByte(p.s[end]) {
p.pos += 4 p.pos += len(kw.tok)
return "", true // null → empty string return kw.val, true
}
} }
} }
@@ -219,6 +275,15 @@ func (p *exprParser) parseValue() (string, bool) {
return p.parseStringLiteral() return p.parseStringLiteral()
} }
// Integer literal — returned as its decimal string for string comparison.
if p.peek() >= '0' && p.peek() <= '9' {
start := p.pos
for p.pos < len(p.s) && p.s[p.pos] >= '0' && p.s[p.pos] <= '9' {
p.pos++
}
return p.s[start:p.pos], true
}
return "", false return "", false
} }
@@ -261,7 +326,8 @@ func (p *exprParser) parseRegexLiteral() (string, bool) {
b := p.s[p.pos] b := p.s[p.pos]
if b == '/' { if b == '/' {
p.pos++ // consume closing '/' p.pos++ // consume closing '/'
return sb.String(), true flags := p.parseRegexFlags()
return applyRegexFlags(flags, sb.String()), true
} }
if b == '\\' && p.pos+1 < len(p.s) { if b == '\\' && p.pos+1 < len(p.s) {
p.pos++ p.pos++
@@ -275,6 +341,68 @@ func (p *exprParser) parseRegexLiteral() (string, bool) {
return "", false // unterminated regex return "", false // unterminated regex
} }
// parseRegexFlags reads zero or more regex flag letters (i, m, s) after the
// closing '/'. Unknown letters are consumed but ignored.
func (p *exprParser) parseRegexFlags() string {
start := p.pos
for p.pos < len(p.s) && isIdentByte(p.s[p.pos]) {
p.pos++
}
return p.s[start:p.pos]
}
// applyRegexFlags prepends Go regexp flag groups to pattern (e.g. (?i) for 'i').
// Unknown flags are silently ignored.
func applyRegexFlags(flags, pattern string) string {
if flags == "" {
return pattern
}
var prefix strings.Builder
for _, f := range flags {
switch f {
case 'i':
prefix.WriteString("(?i)")
case 'm':
prefix.WriteString("(?m)")
case 's':
prefix.WriteString("(?s)")
}
}
return prefix.String() + pattern
}
// extractRegexFromString parses a /pattern/flags string (typically from a CI
// variable) and returns a Go regexp pattern with flags applied.
func extractRegexFromString(s string) (string, bool) {
s = strings.TrimSpace(s)
if len(s) == 0 || s[0] != '/' {
return "", false
}
var sb strings.Builder
i := 1
for i < len(s) {
b := s[i]
if b == '/' {
i++ // past closing '/'
var flags strings.Builder
for i < len(s) && isIdentByte(s[i]) {
flags.WriteByte(s[i])
i++
}
return applyRegexFlags(flags.String(), sb.String()), true
}
if b == '\\' && i+1 < len(s) {
i++
sb.WriteByte('\\')
sb.WriteByte(s[i])
} else {
sb.WriteByte(b)
}
i++
}
return "", false // unterminated
}
func isIdentByte(b byte) bool { func isIdentByte(b byte) bool {
return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || (b >= '0' && b <= '9') || b == '_' return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || (b >= '0' && b <= '9') || b == '_'
} }
+48
View File
@@ -9,6 +9,10 @@ func TestEvalIf(t *testing.T) {
"CI_COMMIT_TAG": "", "CI_COMMIT_TAG": "",
"CI_PIPELINE_SOURCE": "push", "CI_PIPELINE_SOURCE": "push",
"DEPLOY_ENV": "staging", "DEPLOY_ENV": "staging",
"BRANCH_PATTERN": "/^dev/",
"BRANCH_PATTERN_CI": "/^DEV/i",
"EMPTY_PATTERN": "",
"PLAIN_PATTERN": "develop",
} }
return m[key] return m[key]
} }
@@ -69,6 +73,50 @@ func TestEvalIf(t *testing.T) {
{"extra spaces", ` $CI_COMMIT_BRANCH == "develop" `, true}, {"extra spaces", ` $CI_COMMIT_BRANCH == "develop" `, true},
{"tabs", "$CI_COMMIT_BRANCH\t==\t\"develop\"", true}, {"tabs", "$CI_COMMIT_BRANCH\t==\t\"develop\"", true},
// ── Multi-line expressions (newlines between tokens) ──────────────────
{"multiline or true", "$CI_COMMIT_BRANCH == \"develop\" ||\n$CI_COMMIT_TAG != null", true},
{"multiline or false", "$CI_COMMIT_BRANCH == \"main\" ||\n$CI_COMMIT_TAG != null", false},
{"multiline and true", "$CI_COMMIT_BRANCH == \"develop\" &&\n$CI_PIPELINE_SOURCE == \"push\"", true},
{"multiline and false", "$CI_COMMIT_BRANCH == \"main\" &&\n$CI_PIPELINE_SOURCE == \"push\"", false},
{"multiline with crlf", "$CI_COMMIT_BRANCH == \"develop\" ||\r\n$CI_COMMIT_TAG != null", true},
// ── ${VAR} curly-brace syntax ─────────────────────────────────────────
{"curly var eq match", `${CI_COMMIT_BRANCH} == "develop"`, true},
{"curly var eq no match", `${CI_COMMIT_BRANCH} == "main"`, false},
{"curly var truthiness", `${CI_COMMIT_BRANCH}`, true},
{"curly var falsy", `${CI_COMMIT_TAG}`, false},
{"curly var neq null", `${CI_COMMIT_BRANCH} != null`, true},
{"curly mixed", `${CI_COMMIT_BRANCH} == "develop" && $CI_PIPELINE_SOURCE == "push"`, true},
// ── Regex flags (/pattern/i etc.) ─────────────────────────────────────
{"regex flag i match", `$CI_COMMIT_BRANCH =~ /^DEV/i`, true},
{"regex flag i no match", `$CI_COMMIT_BRANCH =~ /^MAIN/i`, false},
{"regex flag i not match", `$CI_COMMIT_BRANCH !~ /^MAIN/i`, true},
{"regex no flag case sensitive", `$CI_COMMIT_BRANCH =~ /^DEV/`, false},
{"regex flag i version tag", `$CI_PIPELINE_SOURCE =~ /^PUSH$/i`, true},
// ── Variable on right side of =~ ──────────────────────────────────────
{"var regex rhs match", `$CI_COMMIT_BRANCH =~ $BRANCH_PATTERN`, true},
{"var regex rhs no match", `$CI_PIPELINE_SOURCE =~ $BRANCH_PATTERN`, false},
{"var regex rhs ci flag match", `$CI_COMMIT_BRANCH =~ $BRANCH_PATTERN_CI`, true},
{"var regex rhs empty permissive", `$CI_COMMIT_BRANCH =~ $EMPTY_PATTERN`, true},
{"var regex rhs plain permissive", `$CI_COMMIT_BRANCH =~ $PLAIN_PATTERN`, true},
{"var regex rhs not match", `$CI_COMMIT_BRANCH !~ $BRANCH_PATTERN`, false},
// ── Bare true/false keywords ─────────────────────────────────────────
// GitLab CI treats true/false as the string values "true"/"false".
{"bare true match", `$CI_PIPELINE_SOURCE == true`, false}, // "push" != "true"
{"bare false match", `$CI_COMMIT_TAG == false`, false}, // "" != "false"
{"bare true var set to true", `$DEPLOY_ENV == true`, false}, // "staging" != "true"
{"bare false neq", `$CI_COMMIT_BRANCH != false`, true}, // "develop" != "false"
{"bare true in compound", `$CI_COMMIT_BRANCH != null && $CI_COMMIT_TAG == false`, false},
// ── Integer literals ──────────────────────────────────────────────────
// Compared as decimal strings (GitLab CI converts integers to strings).
{"int eq match", `$CI_PIPELINE_SOURCE != 0`, true}, // "push" != "0"
{"int eq no match", `$CI_COMMIT_TAG == 0`, false}, // "" != "0"
{"int in compound", `$CI_COMMIT_BRANCH != null && $CI_COMMIT_BRANCH != 0`, true},
// ── Permissive fallback ─────────────────────────────────────────────── // ── Permissive fallback ───────────────────────────────────────────────
{"unparseable returns true", `this is not valid syntax %%%`, true}, {"unparseable returns true", `this is not valid syntax %%%`, true},
{"empty expr returns true", ``, true}, {"empty expr returns true", ``, true},
+13 -9
View File
@@ -4,7 +4,7 @@ import (
"regexp" "regexp"
"strings" "strings"
"git.k3nny.fr/gitlab-sim/internal/model" "git.k3nny.fr/glint/internal/model"
) )
// JobState describes whether a job would be included in a pipeline run for the // JobState describes whether a job would be included in a pipeline run for the
@@ -28,13 +28,17 @@ func (s JobState) String() string {
} }
} }
// EvalWorkflow returns false when the pipeline's workflow:rules block would // EvalWorkflow evaluates the pipeline's workflow:rules block against ctx.
// prevent any pipeline from starting in the given context. // Returns (runs, ruleVars):
// Returns true when ctx is empty, when there is no workflow block, or when no // - runs=false means the pipeline would not start for this context.
// rule is configured. // - ruleVars holds any variables: defined on the matching rule; inject these
func EvalWorkflow(p *model.Pipeline, ctx *Context) bool { // into the context so job rules can reference them.
//
// Returns (true, nil) when ctx is empty, when there is no workflow block, or
// when no rules are configured.
func EvalWorkflow(p *model.Pipeline, ctx *Context) (bool, map[string]string) {
if ctx.IsEmpty() || p.Workflow == nil || len(p.Workflow.Rules) == 0 { if ctx.IsEmpty() || p.Workflow == nil || len(p.Workflow.Rules) == 0 {
return true return true, nil
} }
vars := ctx.Get vars := ctx.Get
for _, rule := range p.Workflow.Rules { for _, rule := range p.Workflow.Rules {
@@ -45,9 +49,9 @@ func EvalWorkflow(p *model.Pipeline, ctx *Context) bool {
if when == "" { if when == "" {
when = "always" when = "always"
} }
return when != "never" return when != "never", ExtractStringVars(rule.Variables)
} }
return false // no rule matched → pipeline does not run return false, nil // no rule matched → pipeline does not run
} }
// EvalJob returns the effective JobState for job in the given context. // EvalJob returns the effective JobState for job in the given context.
+243
View File
@@ -0,0 +1,243 @@
package cicontext
import (
"testing"
"git.k3nny.fr/glint/internal/model"
)
func TestEvalWorkflow(t *testing.T) {
makePipeline := func(rules []model.Rule) *model.Pipeline {
return &model.Pipeline{Workflow: &model.Workflow{Rules: rules}}
}
tests := []struct {
name string
rules []model.Rule
branch string
wantRuns bool
wantVars map[string]string
}{
{
name: "no workflow block",
rules: nil,
branch: "main",
wantRuns: true,
wantVars: nil,
},
{
name: "matching rule runs always",
rules: []model.Rule{
{If: `$CI_COMMIT_BRANCH == "main"`, When: "always"},
},
branch: "main",
wantRuns: true,
},
{
name: "matching rule when never",
rules: []model.Rule{
{If: `$CI_COMMIT_BRANCH == "main"`, When: "never"},
},
branch: "main",
wantRuns: false,
},
{
name: "no rule matched",
rules: []model.Rule{
{If: `$CI_COMMIT_BRANCH == "main"`},
},
branch: "develop",
wantRuns: false,
},
{
name: "matching rule with variables",
rules: []model.Rule{
{
If: `$CI_COMMIT_BRANCH == "main"`,
When: "always",
Variables: map[string]any{
"DEPLOY_TARGET": "production",
"ENVIRONMENT": "prod",
},
},
{When: "always"},
},
branch: "main",
wantRuns: true,
wantVars: map[string]string{
"DEPLOY_TARGET": "production",
"ENVIRONMENT": "prod",
},
},
{
name: "fallback rule with different variables",
rules: []model.Rule{
{If: `$CI_COMMIT_BRANCH == "main"`, Variables: map[string]any{"DEPLOY_TARGET": "production"}},
{When: "always", Variables: map[string]any{"DEPLOY_TARGET": "staging"}},
},
branch: "develop",
wantRuns: true,
wantVars: map[string]string{"DEPLOY_TARGET": "staging"},
},
{
name: "empty context always runs",
rules: []model.Rule{
{If: `$CI_COMMIT_BRANCH == "main"`, When: "never"},
},
branch: "", // no context
wantRuns: true,
wantVars: nil,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
var p *model.Pipeline
if tc.rules == nil {
p = &model.Pipeline{}
} else {
p = makePipeline(tc.rules)
}
ctx := New(tc.branch, "", "", nil)
runs, vars := EvalWorkflow(p, ctx)
if runs != tc.wantRuns {
t.Errorf("EvalWorkflow runs = %v, want %v", runs, tc.wantRuns)
}
for k, want := range tc.wantVars {
if got := vars[k]; got != want {
t.Errorf("EvalWorkflow vars[%q] = %q, want %q", k, got, want)
}
}
if len(vars) != len(tc.wantVars) {
t.Errorf("EvalWorkflow returned %d vars, want %d; got %v", len(vars), len(tc.wantVars), vars)
}
})
}
}
func TestContextInject(t *testing.T) {
t.Run("inject does not overwrite pinned var", func(t *testing.T) {
ctx := New("main", "", "", []string{"DEPLOY_TARGET=override"})
ctx.Inject("DEPLOY_TARGET", "workflow-value")
if got := ctx.Get("DEPLOY_TARGET"); got != "override" {
t.Errorf("Inject overwrote pinned var: got %q, want %q", got, "override")
}
})
t.Run("inject does not overwrite shortcut var", func(t *testing.T) {
ctx := New("main", "", "", nil)
ctx.Inject("CI_COMMIT_BRANCH", "other")
if got := ctx.Get("CI_COMMIT_BRANCH"); got != "main" {
t.Errorf("Inject overwrote shortcut var: got %q, want %q", got, "main")
}
})
t.Run("inject sets new variable", func(t *testing.T) {
ctx := New("main", "", "", nil)
ctx.Inject("DEPLOY_TARGET", "production")
if got := ctx.Get("DEPLOY_TARGET"); got != "production" {
t.Errorf("Inject did not set variable: got %q", got)
}
})
t.Run("inject later call overrides earlier call", func(t *testing.T) {
ctx := New("main", "", "", nil)
ctx.Inject("DEPLOY_TARGET", "pipeline-default")
ctx.Inject("DEPLOY_TARGET", "workflow-override")
if got := ctx.Get("DEPLOY_TARGET"); got != "workflow-override" {
t.Errorf("second Inject did not win: got %q, want %q", got, "workflow-override")
}
})
}
func TestExtractStringVars(t *testing.T) {
tests := []struct {
name string
in map[string]any
want map[string]string
}{
{
name: "plain strings",
in: map[string]any{"A": "hello", "B": "world"},
want: map[string]string{"A": "hello", "B": "world"},
},
{
name: "extended value form",
in: map[string]any{
"KEY": map[string]any{"value": "extended", "description": "some desc"},
},
want: map[string]string{"KEY": "extended"},
},
{
name: "mixed forms",
in: map[string]any{
"PLAIN": "str",
"COMPLEX": map[string]any{"value": "val"},
},
want: map[string]string{"PLAIN": "str", "COMPLEX": "val"},
},
{
name: "nil map",
in: nil,
want: nil,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got := ExtractStringVars(tc.in)
for k, want := range tc.want {
if got[k] != want {
t.Errorf("ExtractStringVars[%q] = %q, want %q", k, got[k], want)
}
}
if len(got) != len(tc.want) {
t.Errorf("ExtractStringVars returned %d entries, want %d", len(got), len(tc.want))
}
})
}
}
// TestWorkflowVarsJobEval verifies the end-to-end flow: workflow rule injects
// DEPLOY_TARGET, which is then used in a job's rules:if: expression.
func TestWorkflowVarsJobEval(t *testing.T) {
rules := []model.Rule{
{If: `$CI_COMMIT_BRANCH == "main"`, Variables: map[string]any{"DEPLOY_TARGET": "production"}},
{When: "always", Variables: map[string]any{"DEPLOY_TARGET": "staging"}},
}
p := &model.Pipeline{
Workflow: &model.Workflow{Rules: rules},
Jobs: map[string]model.Job{
"deploy-prod": {
Rules: []model.Rule{
{If: `$DEPLOY_TARGET == "production"`, When: "on_success"},
{When: "never"},
},
},
"deploy-staging": {
Rules: []model.Rule{
{If: `$DEPLOY_TARGET == "staging"`, When: "on_success"},
{When: "never"},
},
},
},
}
for _, tc := range []struct {
branch string
wantProd JobState
wantStaging JobState
}{
{"main", JobActive, JobSkipped},
{"develop", JobSkipped, JobActive},
} {
ctx := New(tc.branch, "", "", nil)
_, ruleVars := EvalWorkflow(p, ctx)
for k, v := range ruleVars {
ctx.Inject(k, v)
}
if got := EvalJob(p.Jobs["deploy-prod"], ctx); got != tc.wantProd {
t.Errorf("branch=%q deploy-prod = %v, want %v", tc.branch, got, tc.wantProd)
}
if got := EvalJob(p.Jobs["deploy-staging"], ctx); got != tc.wantStaging {
t.Errorf("branch=%q deploy-staging = %v, want %v", tc.branch, got, tc.wantStaging)
}
}
}
+18
View File
@@ -142,6 +142,24 @@ func (cfg GitLabConfig) FetchFile(project, filePath, ref string) ([]byte, error)
return body, nil return body, nil
} }
// FetchURL downloads the content at a plain HTTPS URL without authentication.
// Used for include: remote: entries which are public by definition.
func FetchURL(rawURL string) ([]byte, error) {
resp, err := http.Get(rawURL) //nolint:noctx
if err != nil {
return nil, fmt.Errorf("GET %s: %w", rawURL, err)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("reading body: %w", err)
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("GET %s: status %d", rawURL, resp.StatusCode)
}
return body, nil
}
func firstNonEmpty(values ...string) string { func firstNonEmpty(values ...string) string {
for _, v := range values { for _, v := range values {
if v != "" { if v != "" {
+277 -64
View File
@@ -2,12 +2,226 @@ package graph
import ( import (
"fmt" "fmt"
"os"
"path/filepath"
"sort"
"strings" "strings"
"git.k3nny.fr/glint/internal/fetcher"
"git.k3nny.fr/glint/internal/model"
) )
// Includes returns a Mermaid flowchart showing include file dependencies. // Includes returns a Mermaid flowchart of the full include dependency tree.
// sourcePath is the path to the main pipeline file; rawIncludes is Pipeline.Include. // Each node shows the jobs defined directly in that file, connected with
func Includes(sourcePath string, rawIncludes []any) string { // dashed arrows (solid arrows represent the include hierarchy itself).
// It recurses into project:, component:, and local: includes to expose
// transitive dependencies. Includes that cannot be fetched are shown but not expanded.
func Includes(sourcePath string, rawIncludes []any, cfg fetcher.GitLabConfig) string {
b := &treeBuilder{
visited: map[string]bool{},
cfg: cfg,
baseDir: filepath.Dir(sourcePath),
}
root := &treeNode{
id: "root",
label: mermaidLabel(sourcePath),
class: "main",
jobs: directJobs(sourcePath),
}
b.buildChildren(root, rawIncludes)
return renderTree(root)
}
type treeNode struct {
id string
label string
class string
children []*treeNode
jobs []string // job names defined directly in this file
}
// treeBuilder accumulates state while recursively traversing include entries.
type treeBuilder struct {
counter int
visited map[string]bool // prevents infinite loops on circular includes
cfg fetcher.GitLabConfig
baseDir string // directory of the current pipeline file; changes for local includes
}
func (b *treeBuilder) nextID() string {
b.counter++
return fmt.Sprintf("inc%d", b.counter)
}
func (b *treeBuilder) buildChildren(parent *treeNode, rawIncludes []any) {
for _, entry := range rawIncludes {
parent.children = append(parent.children, b.parseEntry(entry)...)
}
}
func (b *treeBuilder) parseEntry(entry any) []*treeNode {
switch v := entry.(type) {
case string:
node := &treeNode{id: b.nextID(), label: "local: " + mermaidLabel(v), class: "local"}
b.recurseLocal(node, v)
return []*treeNode{node}
case map[string]any:
return b.parseMap(v)
}
return nil
}
func (b *treeBuilder) parseMap(m map[string]any) []*treeNode {
if comp, ok := m["component"].(string); ok {
node := &treeNode{
id: b.nextID(),
label: "component:<br>" + mermaidLabel(comp),
class: "component",
}
b.recurseComponent(node, comp)
return []*treeNode{node}
}
if proj, ok := m["project"].(string); ok {
ref, _ := m["ref"].(string)
if ref == "" {
ref = "HEAD"
}
files := includeFileList(m["file"])
if len(files) == 0 {
return []*treeNode{{
id: b.nextID(),
label: fmt.Sprintf("project: %s @ %s", mermaidLabel(proj), ref),
class: "project",
}}
}
var nodes []*treeNode
for _, f := range files {
node := &treeNode{
id: b.nextID(),
label: fmt.Sprintf("project: %s<br>%s @ %s", mermaidLabel(proj), mermaidLabel(f), ref),
class: "project",
}
b.recurseProject(node, proj, f, ref)
nodes = append(nodes, node)
}
return nodes
}
if local, ok := m["local"].(string); ok {
node := &treeNode{id: b.nextID(), label: "local: " + mermaidLabel(local), class: "local"}
b.recurseLocal(node, local)
return []*treeNode{node}
}
if remote, ok := m["remote"].(string); ok {
node := &treeNode{id: b.nextID(), label: "remote:<br>" + mermaidLabel(remote), class: "remote"}
b.recurseRemote(node, remote)
return []*treeNode{node}
}
if tmpl, ok := m["template"].(string); ok {
return []*treeNode{{id: b.nextID(), label: "template:<br>" + mermaidLabel(tmpl), class: "template"}}
}
return nil
}
func (b *treeBuilder) recurseLocal(node *treeNode, path string) {
absPath := filepath.Join(b.baseDir, path)
key := "local:" + absPath
if b.visited[key] {
return
}
b.visited[key] = true
data, err := os.ReadFile(absPath)
if err != nil {
return
}
p, err := model.ParseBytes(data)
if err != nil {
return
}
node.jobs = jobNames(p)
if len(p.Include) == 0 {
return
}
orig := b.baseDir
b.baseDir = filepath.Dir(absPath)
b.buildChildren(node, p.Include)
b.baseDir = orig
}
func (b *treeBuilder) recurseProject(node *treeNode, project, filePath, ref string) {
key := fmt.Sprintf("project:%s:%s@%s", project, filePath, ref)
if b.visited[key] || !b.cfg.HasToken() || filePath == "" {
return
}
b.visited[key] = true
data, err := b.cfg.FetchFile(project, filePath, ref)
if err != nil {
return
}
p, err := model.ParseBytes(data)
if err != nil {
return
}
node.jobs = jobNames(p)
if len(p.Include) == 0 {
return
}
b.buildChildren(node, p.Include)
}
func (b *treeBuilder) recurseRemote(node *treeNode, rawURL string) {
key := "remote:" + rawURL
if b.visited[key] {
return
}
b.visited[key] = true
data, err := fetcher.FetchURL(rawURL)
if err != nil {
return
}
p, err := model.ParseBytes(data)
if err != nil {
return
}
node.jobs = jobNames(p)
if len(p.Include) == 0 {
return
}
b.buildChildren(node, p.Include)
}
func (b *treeBuilder) recurseComponent(node *treeNode, ref string) {
if strings.ContainsRune(ref, '$') {
return
}
key := "component:" + ref
if b.visited[key] {
return
}
b.visited[key] = true
host, project, component, version, err := parseComponentRef(ref)
if err != nil {
return
}
data, err := fetchComponentFile(b.cfg.ForHost(host), project, component, version)
if err != nil {
return
}
p, err := model.ParseBytes(data)
if err != nil {
return
}
node.jobs = jobNames(p)
if len(p.Include) == 0 {
return
}
b.buildChildren(node, p.Include)
}
func renderTree(root *treeNode) string {
var sb strings.Builder var sb strings.Builder
w := func(s string) { sb.WriteString(s + "\n") } w := func(s string) { sb.WriteString(s + "\n") }
wf := func(f string, a ...any) { fmt.Fprintf(&sb, f+"\n", a...) } wf := func(f string, a ...any) { fmt.Fprintf(&sb, f+"\n", a...) }
@@ -22,85 +236,84 @@ func Includes(sourcePath string, rawIncludes []any) string {
w(" classDef local fill:#428fdc,stroke:#1068bf,color:#fff") w(" classDef local fill:#428fdc,stroke:#1068bf,color:#fff")
w(" classDef remote fill:#868686,stroke:#686868,color:#fff") w(" classDef remote fill:#868686,stroke:#686868,color:#fff")
w(" classDef template fill:#fc6d26,stroke:#e56b1f,color:#fff") w(" classDef template fill:#fc6d26,stroke:#e56b1f,color:#fff")
w("") w(" classDef job fill:#f5f5ff,stroke:#7175a0,color:#333")
wf(" root[\"%s\"]:::main", mermaidLabel(sourcePath))
if len(rawIncludes) == 0 {
return sb.String()
}
w("") w("")
counter := 0 jobCounter := 0
for _, entry := range rawIncludes { var emit func(n *treeNode)
for _, n := range parseIncludeEntry(entry, &counter) { emit = func(n *treeNode) {
wf(" %s[\"%s\"]:::%s", n.id, n.label, n.class) wf(" %s[\"%s\"]:::%s", n.id, n.label, n.class)
wf(" root --> %s", n.id) for _, jobName := range n.jobs {
jobCounter++
jobID := fmt.Sprintf("job%d", jobCounter)
wf(" %s(\"%s\"):::job", jobID, mermaidLabel(jobName))
wf(" %s -.-> %s", n.id, jobID)
}
for _, child := range n.children {
emit(child)
wf(" %s --> %s", n.id, child.id)
} }
} }
emit(root)
return sb.String() return sb.String()
} }
type incNode struct { // directJobs parses a single pipeline file (without include resolution) and
id, label, class string // returns the sorted list of job names defined directly in it.
func directJobs(path string) []string {
p, err := model.Parse(path)
if err != nil {
return nil
}
return jobNames(p)
} }
func parseIncludeEntry(entry any, counter *int) []incNode { // jobNames returns a sorted slice of all job names defined in p.
newID := func() string { func jobNames(p *model.Pipeline) []string {
*counter++ if len(p.Jobs) == 0 {
return fmt.Sprintf("inc%d", *counter)
}
switch v := entry.(type) {
case string:
return []incNode{{id: newID(), label: mermaidLabel(v), class: "local"}}
case map[string]any:
return parseIncludeMap(v, newID)
}
return nil return nil
}
names := make([]string, 0, len(p.Jobs))
for name := range p.Jobs {
names = append(names, name)
}
sort.Strings(names)
return names
} }
func parseIncludeMap(m map[string]any, newID func() string) []incNode { // parseComponentRef parses a CI/CD component reference of the form
if comp, ok := m["component"].(string); ok { // <host>/<project-path>/<component-name>@<version>.
return []incNode{{ func parseComponentRef(ref string) (host, project, component, version string, err error) {
id: newID(), atIdx := strings.LastIndex(ref, "@")
label: "component:<br>" + mermaidLabel(comp), if atIdx < 0 || atIdx == len(ref)-1 {
class: "component", err = fmt.Errorf("component reference %q must include a version", ref)
}} return
} }
if proj, ok := m["project"].(string); ok { version = ref[atIdx+1:]
ref, _ := m["ref"].(string) path := ref[:atIdx]
if ref == "" { parts := strings.Split(path, "/")
ref = "HEAD" if len(parts) < 3 {
err = fmt.Errorf("component reference %q must be <host>/<project>/<component>@<version>", ref)
return
} }
files := includeFileList(m["file"]) host = parts[0]
if len(files) == 0 { component = parts[len(parts)-1]
return []incNode{{ project = strings.Join(parts[1:len(parts)-1], "/")
id: newID(), return
label: fmt.Sprintf("project: %s @ %s", mermaidLabel(proj), ref), }
class: "project",
}} // fetchComponentFile fetches a component's template YAML, trying the single-file
// layout first and the directory layout as fallback.
func fetchComponentFile(cfg fetcher.GitLabConfig, project, component, version string) ([]byte, error) {
if data, err := cfg.FetchFile(project, "templates/"+component+".yml", version); err == nil {
return data, nil
} }
var nodes []incNode data, err := cfg.FetchFile(project, "templates/"+component+"/template.yml", version)
for _, f := range files { if err != nil {
nodes = append(nodes, incNode{ return nil, fmt.Errorf("component %s/%s@%s not found", project, component, version)
id: newID(),
label: fmt.Sprintf("project: %s<br>%s @ %s", mermaidLabel(proj), mermaidLabel(f), ref),
class: "project",
})
} }
return nodes return data, nil
}
if local, ok := m["local"].(string); ok {
return []incNode{{id: newID(), label: "local: " + mermaidLabel(local), class: "local"}}
}
if remote, ok := m["remote"].(string); ok {
return []incNode{{id: newID(), label: "remote:<br>" + mermaidLabel(remote), class: "remote"}}
}
if tmpl, ok := m["template"].(string); ok {
return []incNode{{id: newID(), label: "template:<br>" + mermaidLabel(tmpl), class: "template"}}
}
return nil
} }
func includeFileList(v any) []string { func includeFileList(v any) []string {
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"sort" "sort"
"strings" "strings"
"git.k3nny.fr/gitlab-sim/internal/model" "git.k3nny.fr/glint/internal/model"
) )
// Pipeline returns a Mermaid flowchart of pipeline jobs grouped by stage. // Pipeline returns a Mermaid flowchart of pipeline jobs grouped by stage.
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"strings" "strings"
"time" "time"
"git.k3nny.fr/gitlab-sim/internal/model" "git.k3nny.fr/glint/internal/model"
) )
// Layout constants (pixels) tuned to resemble GitLab's full pipeline graph view. // Layout constants (pixels) tuned to resemble GitLab's full pipeline graph view.
+113
View File
@@ -0,0 +1,113 @@
package graph
import (
"sort"
"strings"
"git.k3nny.fr/glint/internal/cicontext"
"git.k3nny.fr/glint/internal/model"
)
// Tree returns a tree-command-style text representation of pipeline jobs
// grouped by stage.
//
// Hidden template jobs (names starting with ".") are excluded.
// When ctx is non-nil and non-empty, each job is annotated with its evaluated
// state ([skipped] or [manual]); active jobs carry no annotation.
// Without a context, job-type annotations ([manual], [delayed], [trigger]) are
// shown instead.
func Tree(p *model.Pipeline, ctx *cicontext.Context) string {
var sb strings.Builder
// Collect visible jobs.
var visible []string
for name := range p.Jobs {
if !strings.HasPrefix(name, ".") {
visible = append(visible, name)
}
}
sort.Strings(visible)
// Group by stage.
byStage := make(map[string][]string)
for _, name := range visible {
stage := p.Jobs[name].Stage
if stage == "" {
stage = "test"
}
byStage[stage] = append(byStage[stage], name)
}
// Build ordered stage list: declared first, then extras.
seen := make(map[string]bool)
var stages []string
for _, s := range p.Stages {
if len(byStage[s]) > 0 && !seen[s] {
stages = append(stages, s)
seen[s] = true
}
}
for _, name := range visible {
stage := p.Jobs[name].Stage
if stage == "" {
stage = "test"
}
if !seen[stage] {
stages = append(stages, stage)
seen[stage] = true
}
}
sb.WriteString("pipeline\n")
for si, stage := range stages {
lastStage := si == len(stages)-1
stagePrefix, childPrefix := branchChars(lastStage)
sb.WriteString(stagePrefix + stage + "\n")
jobs := byStage[stage]
sort.Strings(jobs)
for ji, name := range jobs {
lastJob := ji == len(jobs)-1
jobBranch, _ := branchChars(lastJob)
sb.WriteString(childPrefix + jobBranch + jobLabel(p.Jobs[name], name, ctx) + "\n")
}
}
return sb.String()
}
func branchChars(last bool) (branch, continuation string) {
if last {
return "└── ", " "
}
return "├── ", "│ "
}
func jobLabel(job model.Job, name string, ctx *cicontext.Context) string {
if ctx != nil && !ctx.IsEmpty() {
switch cicontext.EvalJob(job, ctx) {
case cicontext.JobSkipped:
return name + " [skipped]"
case cicontext.JobManual:
return name + " [manual]"
}
return name
}
// No context: annotate by job type.
var tags []string
if job.When == "manual" {
tags = append(tags, "manual")
}
if job.When == "delayed" {
tags = append(tags, "delayed")
}
if job.Trigger != nil {
tags = append(tags, "trigger")
}
if len(tags) == 0 {
return name
}
return name + " [" + strings.Join(tags, ", ") + "]"
}
+7 -1
View File
@@ -3,7 +3,7 @@ package linter
import ( import (
"fmt" "fmt"
"git.k3nny.fr/gitlab-sim/internal/model" "git.k3nny.fr/glint/internal/model"
) )
func checkDependencies(p *model.Pipeline) []Finding { func checkDependencies(p *model.Pipeline) []Finding {
@@ -23,7 +23,10 @@ func checkDependencies(p *model.Pipeline) []Finding {
if !exists { if !exists {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleUnknownDependency,
Job: name, Job: name,
File: job.File,
Line: job.Line,
Message: fmt.Sprintf("'dependencies' references unknown job %q", dep), Message: fmt.Sprintf("'dependencies' references unknown job %q", dep),
}) })
continue continue
@@ -33,7 +36,10 @@ func checkDependencies(p *model.Pipeline) []Finding {
if depHasStage && depIdx >= jobStageIdx { if depHasStage && depIdx >= jobStageIdx {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleDependencyStage,
Job: name, Job: name,
File: job.File,
Line: job.Line,
Message: fmt.Sprintf("'dependencies' job %q must be in an earlier stage (in %q, current job is in %q)", dep, depJob.Stage, job.Stage), Message: fmt.Sprintf("'dependencies' job %q must be in an earlier stage (in %q, current job is in %q)", dep, depJob.Stage, job.Stage),
}) })
} }
+33 -5
View File
@@ -5,7 +5,7 @@ import (
"regexp" "regexp"
"strings" "strings"
"git.k3nny.fr/gitlab-sim/internal/model" "git.k3nny.fr/glint/internal/model"
) )
var validJobWhen = map[string]bool{ var validJobWhen = map[string]bool{
@@ -105,6 +105,7 @@ func checkWhen(name string, job model.Job) []Finding {
if job.When != "" && !validJobWhen[job.When] { if job.When != "" && !validJobWhen[job.When] {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidWhen,
Job: name, Job: name,
Message: fmt.Sprintf("'when' has invalid value %q; valid: on_success, on_failure, always, manual, delayed, never", job.When), Message: fmt.Sprintf("'when' has invalid value %q; valid: on_success, on_failure, always, manual, delayed, never", job.When),
}) })
@@ -112,6 +113,7 @@ func checkWhen(name string, job model.Job) []Finding {
if job.When == "delayed" && job.StartIn == "" { if job.When == "delayed" && job.StartIn == "" {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleDelayedNoStartIn,
Job: name, Job: name,
Message: "'when: delayed' requires 'start_in' (e.g. 'start_in: 30 minutes')", Message: "'when: delayed' requires 'start_in' (e.g. 'start_in: 30 minutes')",
}) })
@@ -119,6 +121,7 @@ func checkWhen(name string, job model.Job) []Finding {
if job.When != "delayed" && job.StartIn != "" { if job.When != "delayed" && job.StartIn != "" {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleStartInNoDelayed,
Job: name, Job: name,
Message: "'start_in' is only valid when 'when: delayed'", Message: "'start_in' is only valid when 'when: delayed'",
}) })
@@ -135,6 +138,7 @@ func checkParallel(name string, job model.Job) []Finding {
if v < 2 || v > 200 { if v < 2 || v > 200 {
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidParallel,
Job: name, Job: name,
Message: fmt.Sprintf("'parallel' must be between 2 and 200, got %d", v), Message: fmt.Sprintf("'parallel' must be between 2 and 200, got %d", v),
}} }}
@@ -143,6 +147,7 @@ func checkParallel(name string, job model.Job) []Finding {
if _, ok := v["matrix"]; !ok { if _, ok := v["matrix"]; !ok {
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidParallel,
Job: name, Job: name,
Message: "'parallel' map form must have a 'matrix' key", Message: "'parallel' map form must have a 'matrix' key",
}} }}
@@ -150,6 +155,7 @@ func checkParallel(name string, job model.Job) []Finding {
default: default:
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidParallel,
Job: name, Job: name,
Message: "'parallel' must be an integer (2200) or a map with 'matrix'", Message: "'parallel' must be an integer (2200) or a map with 'matrix'",
}} }}
@@ -166,6 +172,7 @@ func checkRetry(name string, job model.Job) []Finding {
if v < 0 || v > 2 { if v < 0 || v > 2 {
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidRetry,
Job: name, Job: name,
Message: fmt.Sprintf("'retry' must be 0, 1, or 2; got %d", v), Message: fmt.Sprintf("'retry' must be 0, 1, or 2; got %d", v),
}} }}
@@ -176,6 +183,7 @@ func checkRetry(name string, job model.Job) []Finding {
if n, ok := maxVal.(int); ok && (n < 0 || n > 2) { if n, ok := maxVal.(int); ok && (n < 0 || n > 2) {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidRetry,
Job: name, Job: name,
Message: fmt.Sprintf("'retry.max' must be 0, 1, or 2; got %d", n), Message: fmt.Sprintf("'retry.max' must be 0, 1, or 2; got %d", n),
}) })
@@ -188,6 +196,7 @@ func checkRetry(name string, job model.Job) []Finding {
default: default:
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidRetry,
Job: name, Job: name,
Message: "'retry' must be an integer (02) or a map with 'max'/'when'", Message: "'retry' must be an integer (02) or a map with 'max'/'when'",
}} }}
@@ -201,6 +210,7 @@ func validateRetryWhen(name string, val any) []Finding {
if !validRetryWhen[s] { if !validRetryWhen[s] {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidRetryWhen,
Job: name, Job: name,
Message: fmt.Sprintf("'retry.when' has invalid value %q", s), Message: fmt.Sprintf("'retry.when' has invalid value %q", s),
}) })
@@ -230,6 +240,7 @@ func checkAllowFailure(name string, job model.Job) []Finding {
if _, ok := v["exit_codes"]; !ok { if _, ok := v["exit_codes"]; !ok {
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidAllowFailure,
Job: name, Job: name,
Message: "'allow_failure' map form must contain 'exit_codes'", Message: "'allow_failure' map form must contain 'exit_codes'",
}} }}
@@ -238,6 +249,7 @@ func checkAllowFailure(name string, job model.Job) []Finding {
_ = v _ = v
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidAllowFailure,
Job: name, Job: name,
Message: "'allow_failure' must be a boolean or a map with 'exit_codes'", Message: "'allow_failure' must be a boolean or a map with 'exit_codes'",
}} }}
@@ -252,6 +264,7 @@ func checkInterruptible(name string, job model.Job) []Finding {
if _, ok := job.Interruptible.(bool); !ok { if _, ok := job.Interruptible.(bool); !ok {
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidInterruptible,
Job: name, Job: name,
Message: "'interruptible' must be a boolean", Message: "'interruptible' must be a boolean",
}} }}
@@ -264,9 +277,10 @@ func checkTrigger(name string, job model.Job) []Finding {
return nil return nil
} }
var findings []Finding var findings []Finding
if len(job.Script) > 0 { if scriptNonEmpty(job.Script) {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleTriggerWithScript,
Job: name, Job: name,
Message: "jobs with 'trigger' cannot use 'script'", Message: "jobs with 'trigger' cannot use 'script'",
}) })
@@ -277,6 +291,7 @@ func checkTrigger(name string, job model.Job) []Finding {
if !hasProject && !hasInclude { if !hasProject && !hasInclude {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidTrigger,
Job: name, Job: name,
Message: "'trigger' map must specify 'project' or 'include'", Message: "'trigger' map must specify 'project' or 'include'",
}) })
@@ -294,6 +309,7 @@ func checkCoverage(name string, job model.Job) []Finding {
if !coveragePattern.MatchString(job.Coverage) { if !coveragePattern.MatchString(job.Coverage) {
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidCoverage,
Job: name, Job: name,
Message: fmt.Sprintf("'coverage' must be a regex pattern wrapped in '/' (e.g. '/\\d+\\.?\\d*%%/'), got %q", job.Coverage), Message: fmt.Sprintf("'coverage' must be a regex pattern wrapped in '/' (e.g. '/\\d+\\.?\\d*%%/'), got %q", job.Coverage),
}} }}
@@ -309,6 +325,7 @@ func checkRelease(name string, job model.Job) []Finding {
if !ok { if !ok {
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidRelease,
Job: name, Job: name,
Message: "'release' must be a map", Message: "'release' must be a map",
}} }}
@@ -317,6 +334,7 @@ func checkRelease(name string, job model.Job) []Finding {
if !exists || tagName == "" || tagName == nil { if !exists || tagName == "" || tagName == nil {
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidRelease,
Job: name, Job: name,
Message: "'release' requires 'tag_name'", Message: "'release' requires 'tag_name'",
}} }}
@@ -334,11 +352,12 @@ func checkEnvironment(name string, job model.Job) []Finding {
return nil return nil
} }
var findings []Finding var findings []Finding
envName, _ := m["name"] envName := m["name"]
_, hasURL := m["url"] _, hasURL := m["url"]
if (envName == nil || envName == "") && hasURL { if (envName == nil || envName == "") && hasURL {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidEnvironment,
Job: name, Job: name,
Message: "'environment.url' requires 'environment.name' to be set", Message: "'environment.url' requires 'environment.name' to be set",
}) })
@@ -346,6 +365,7 @@ func checkEnvironment(name string, job model.Job) []Finding {
if action, ok := m["action"].(string); ok && !validEnvironmentAction[action] { if action, ok := m["action"].(string); ok && !validEnvironmentAction[action] {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidEnvironment,
Job: name, Job: name,
Message: fmt.Sprintf("'environment.action' has invalid value %q; valid: start, stop, prepare, verify, access", action), Message: fmt.Sprintf("'environment.action' has invalid value %q; valid: start, stop, prepare, verify, access", action),
}) })
@@ -365,15 +385,17 @@ func checkArtifacts(name string, job model.Job) []Finding {
if w, ok := m["when"].(string); ok && !validArtifactsWhen[w] { if w, ok := m["when"].(string); ok && !validArtifactsWhen[w] {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidArtifacts,
Job: name, Job: name,
Message: fmt.Sprintf("'artifacts.when' has invalid value %q; valid: on_success, on_failure, always", w), Message: fmt.Sprintf("'artifacts.when' has invalid value %q; valid: on_success, on_failure, always", w),
}) })
} }
if _, hasExposeAs := m["expose_as"]; hasExposeAs { if _, hasExposeAs := m["expose_as"]; hasExposeAs {
paths, _ := m["paths"] paths := m["paths"]
if paths == nil { if paths == nil {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidArtifacts,
Job: name, Job: name,
Message: "'artifacts.expose_as' requires 'artifacts.paths'", Message: "'artifacts.expose_as' requires 'artifacts.paths'",
}) })
@@ -392,6 +414,7 @@ func checkArtifacts(name string, job model.Job) []Finding {
if !found { if !found {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Warning, Severity: Warning,
Rule: RulePagesPublic,
Job: name, Job: name,
Message: "the 'pages' job should include 'public' in 'artifacts.paths' for GitLab Pages to deploy", Message: "the 'pages' job should include 'public' in 'artifacts.paths' for GitLab Pages to deploy",
}) })
@@ -421,6 +444,7 @@ func checkCache(name string, job model.Job) []Finding {
if w, ok := m["when"].(string); ok && !validCacheWhen[w] { if w, ok := m["when"].(string); ok && !validCacheWhen[w] {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidCache,
Job: name, Job: name,
Message: fmt.Sprintf("'cache.when' has invalid value %q; valid: on_success, on_failure, always", w), Message: fmt.Sprintf("'cache.when' has invalid value %q; valid: on_success, on_failure, always", w),
}) })
@@ -428,6 +452,7 @@ func checkCache(name string, job model.Job) []Finding {
if p, ok := m["policy"].(string); ok && !validCachePolicy[p] { if p, ok := m["policy"].(string); ok && !validCachePolicy[p] {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidCache,
Job: name, Job: name,
Message: fmt.Sprintf("'cache.policy' has invalid value %q; valid: pull, push, pull-push", p), Message: fmt.Sprintf("'cache.policy' has invalid value %q; valid: pull, push, pull-push", p),
}) })
@@ -442,6 +467,7 @@ func checkRules(name string, job model.Job) []Finding {
if rule.When != "" && !validRuleWhen[rule.When] { if rule.When != "" && !validRuleWhen[rule.When] {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidRulesWhen,
Job: name, Job: name,
Message: fmt.Sprintf("rules[%d].when has invalid value %q; valid: on_success, on_failure, always, manual, delayed, never", i, rule.When), Message: fmt.Sprintf("rules[%d].when has invalid value %q; valid: on_success, on_failure, always, manual, delayed, never", i, rule.When),
}) })
@@ -458,10 +484,11 @@ func checkImage(name string, job model.Job) []Finding {
if !ok { if !ok {
return nil // String form is valid. return nil // String form is valid.
} }
imgName, _ := m["name"] imgName := m["name"]
if imgName == nil || imgName == "" { if imgName == nil || imgName == "" {
return []Finding{{ return []Finding{{
Severity: Error, Severity: Error,
Rule: RuleInvalidImage,
Job: name, Job: name,
Message: "'image' map form requires a 'name' key", Message: "'image' map form requires a 'name' key",
}} }}
@@ -489,6 +516,7 @@ func checkInherit(name string, job model.Job) []Finding {
default: default:
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleInvalidInherit,
Job: name, Job: name,
Message: fmt.Sprintf("'inherit.%s' must be a boolean or a list of names", key), Message: fmt.Sprintf("'inherit.%s' must be a boolean or a list of names", key),
}) })
+57 -4
View File
@@ -4,7 +4,7 @@ import (
"fmt" "fmt"
"strings" "strings"
"git.k3nny.fr/gitlab-sim/internal/model" "git.k3nny.fr/glint/internal/model"
) )
type Severity string type Severity string
@@ -16,13 +16,31 @@ const (
type Finding struct { type Finding struct {
Severity Severity Severity Severity
Rule string // stable rule ID, e.g. "GL003" — see rules.go
Job string // empty for pipeline-level findings Job string // empty for pipeline-level findings
File string // source file where the finding originates
Line int // line number in File (0 = unknown)
Message string Message string
} }
func (f Finding) String() string { func (f Finding) String() string {
loc := ""
if f.File != "" {
if f.Line > 0 {
loc = fmt.Sprintf(" (%s:%d)", f.File, f.Line)
} else {
loc = fmt.Sprintf(" (%s)", f.File)
}
}
ruleStr := ""
if f.Rule != "" {
ruleStr = " " + f.Rule
}
if f.Job != "" { if f.Job != "" {
return fmt.Sprintf("[%s] job %q: %s", f.Severity, f.Job, f.Message) return fmt.Sprintf("[%s] job %q%s%s: %s", f.Severity, f.Job, loc, ruleStr, f.Message)
}
if loc != "" || ruleStr != "" {
return fmt.Sprintf("[%s]%s%s: %s", f.Severity, loc, ruleStr, f.Message)
} }
return fmt.Sprintf("[%s] %s", f.Severity, f.Message) return fmt.Sprintf("[%s] %s", f.Severity, f.Message)
} }
@@ -35,6 +53,7 @@ func Lint(p *model.Pipeline) []Finding {
findings = append(findings, checkJobs(p)...) findings = append(findings, checkJobs(p)...)
findings = append(findings, checkNeeds(p)...) findings = append(findings, checkNeeds(p)...)
findings = append(findings, checkDependencies(p)...) findings = append(findings, checkDependencies(p)...)
findings = append(findings, checkVariableRefs(p)...)
return findings return findings
} }
@@ -43,6 +62,8 @@ func checkStages(p *model.Pipeline) []Finding {
if len(p.Stages) == 0 { if len(p.Stages) == 0 {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Warning, Severity: Warning,
Rule: RuleNoStages,
File: p.SourceFile,
Message: "no stages defined; GitLab will use default stages (build, test, deploy)", Message: "no stages defined; GitLab will use default stages (build, test, deploy)",
}) })
} }
@@ -58,6 +79,8 @@ func checkWorkflow(p *model.Pipeline) []Finding {
if rule.When != "" && !validWorkflowRuleWhen[rule.When] { if rule.When != "" && !validWorkflowRuleWhen[rule.When] {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleWorkflowWhen,
File: p.SourceFile,
Message: fmt.Sprintf("workflow.rules[%d].when has invalid value %q; valid: always, never", i, rule.When), Message: fmt.Sprintf("workflow.rules[%d].when has invalid value %q; valid: always, never", i, rule.When),
}) })
} }
@@ -88,10 +111,17 @@ func checkJob(name string, job model.Job, stageSet map[string]bool) []Finding {
// After extends resolution, a job with no script/run is an error. // After extends resolution, a job with no script/run is an error.
// Exceptions: trigger jobs, pages jobs (use pages: keyword), and template jobs. // Exceptions: trigger jobs, pages jobs (use pages: keyword), and template jobs.
hasScript := len(job.Script) > 0 || job.Run != nil // When the job has extends:, the script may come from a base that couldn't be
// fetched (e.g. a remote include without a token), so downgrade to warning.
hasScript := scriptNonEmpty(job.Script) || job.Run != nil
if !isTemplate && !isTrigger && job.Pages == nil && !hasScript { if !isTemplate && !isTrigger && job.Pages == nil && !hasScript {
sev := Error
if job.Extends != nil {
sev = Warning
}
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: sev,
Rule: RuleMissingScript,
Job: name, Job: name,
Message: "missing required field 'script' (or 'run')", Message: "missing required field 'script' (or 'run')",
}) })
@@ -103,6 +133,7 @@ func checkJob(name string, job model.Job, stageSet map[string]bool) []Finding {
if job.Stage != "" && !strings.Contains(job.Stage, "$[[") && len(stageSet) > 0 && !stageSet[job.Stage] { if job.Stage != "" && !strings.Contains(job.Stage, "$[[") && len(stageSet) > 0 && !stageSet[job.Stage] {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleUnknownStage,
Job: name, Job: name,
Message: fmt.Sprintf("stage %q is not defined in 'stages'", job.Stage), Message: fmt.Sprintf("stage %q is not defined in 'stages'", job.Stage),
}) })
@@ -112,6 +143,7 @@ func checkJob(name string, job model.Job, stageSet map[string]bool) []Finding {
if job.Only != nil && len(job.Rules) > 0 { if job.Only != nil && len(job.Rules) > 0 {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleOnlyRulesConflict,
Job: name, Job: name,
Message: "'only' and 'rules' cannot be used together", Message: "'only' and 'rules' cannot be used together",
}) })
@@ -121,6 +153,7 @@ func checkJob(name string, job model.Job, stageSet map[string]bool) []Finding {
if job.Except != nil && len(job.Rules) > 0 { if job.Except != nil && len(job.Rules) > 0 {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleExceptRulesConflict,
Job: name, Job: name,
Message: "'except' and 'rules' cannot be used together", Message: "'except' and 'rules' cannot be used together",
}) })
@@ -130,6 +163,7 @@ func checkJob(name string, job model.Job, stageSet map[string]bool) []Finding {
if job.Only != nil || job.Except != nil { if job.Only != nil || job.Except != nil {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Warning, Severity: Warning,
Rule: RuleDeprecatedOnly,
Job: name, Job: name,
Message: "'only'/'except' are deprecated; prefer 'rules'", Message: "'only'/'except' are deprecated; prefer 'rules'",
}) })
@@ -137,5 +171,24 @@ func checkJob(name string, job model.Job, stageSet map[string]bool) []Finding {
findings = append(findings, checkJobKeywords(name, job)...) findings = append(findings, checkJobKeywords(name, job)...)
// Attach source location to every job-scoped finding collected above.
for i := range findings {
if findings[i].Job != "" && findings[i].File == "" {
findings[i].File = job.File
findings[i].Line = job.Line
}
}
return findings return findings
} }
// scriptNonEmpty reports whether a script/before_script/after_script field
// (which may be a []any list or a plain string) is non-empty.
func scriptNonEmpty(v any) bool {
switch s := v.(type) {
case []any:
return len(s) > 0
case string:
return s != ""
}
return false
}
+46 -20
View File
@@ -3,9 +3,15 @@ package linter
import ( import (
"fmt" "fmt"
"git.k3nny.fr/gitlab-sim/internal/model" "git.k3nny.fr/glint/internal/model"
) )
// needEntry is a parsed element from a job's needs: list.
type needEntry struct {
job string
optional bool // true when the needs entry carries optional: true
}
func checkNeeds(p *model.Pipeline) []Finding { func checkNeeds(p *model.Pipeline) []Finding {
var findings []Finding var findings []Finding
@@ -15,7 +21,7 @@ func checkNeeds(p *model.Pipeline) []Finding {
stageIndex[s] = i stageIndex[s] = i
} }
// needsGraph maps each job to the list of jobs it depends on. // needsGraph maps each job to the jobs it depends on (existing jobs only).
// Used for cycle detection after individual checks. // Used for cycle detection after individual checks.
needsGraph := make(map[string][]string) needsGraph := make(map[string][]string)
@@ -24,32 +30,47 @@ func checkNeeds(p *model.Pipeline) []Finding {
continue continue
} }
neededNames := parseNeedJobNames(job.Needs) entries := parseNeedEntries(job.Needs)
needsGraph[name] = neededNames
jobStageIdx, jobHasStage := stageIndex[job.Stage] jobStageIdx, jobHasStage := stageIndex[job.Stage]
for _, needed := range neededNames { for _, entry := range entries {
neededJob, exists := p.Jobs[needed] neededJob, exists := p.Jobs[entry.job]
if !exists { if !exists {
// optional: true means GitLab CI will silently skip the
// dependency when the job is absent (e.g. from a conditional
// include). Downgrade to warning so users are informed without
// failing the lint.
sev := Error
if entry.optional {
sev = Warning
}
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: sev,
Rule: RuleNeedsUnknown,
Job: name, Job: name,
Message: fmt.Sprintf("needs unknown job %q", needed), File: job.File,
Line: job.Line,
Message: fmt.Sprintf("needs unknown job %q", entry.job),
}) })
continue continue
} }
// Add to the cycle-detection graph only when the dep exists.
needsGraph[name] = append(needsGraph[name], entry.job)
// A job cannot need a job in a later stage. // A job cannot need a job in a later stage.
if len(p.Stages) > 0 && jobHasStage && neededJob.Stage != "" { if len(p.Stages) > 0 && jobHasStage && neededJob.Stage != "" {
neededStageIdx, neededHasStage := stageIndex[neededJob.Stage] neededStageIdx, neededHasStage := stageIndex[neededJob.Stage]
if neededHasStage && neededStageIdx > jobStageIdx { if neededHasStage && neededStageIdx > jobStageIdx {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleNeedsStageOrder,
Job: name, Job: name,
File: job.File,
Line: job.Line,
Message: fmt.Sprintf( Message: fmt.Sprintf(
"needs %q which is in a later stage (%q after %q)", "needs %q which is in a later stage (%q after %q)",
needed, neededJob.Stage, job.Stage, entry.job, neededJob.Stage, job.Stage,
), ),
}) })
} }
@@ -57,32 +78,33 @@ func checkNeeds(p *model.Pipeline) []Finding {
} }
} }
findings = append(findings, detectNeedsCycles(needsGraph)...) findings = append(findings, detectNeedsCycles(needsGraph, p.Jobs)...)
return findings return findings
} }
// parseNeedJobNames extracts job names from a needs: list. // parseNeedEntries extracts needs entries from a needs: list, preserving the
// Each element is either a plain string or a map with a "job" key. // optional flag. Each element is a plain string (job name) or a map with a
// Cross-pipeline needs (maps with a "pipeline" key) are skipped. // "job" key. Cross-pipeline needs (maps with a "pipeline" key) are skipped.
func parseNeedJobNames(needs []any) []string { func parseNeedEntries(needs []any) []needEntry {
var names []string var entries []needEntry
for _, n := range needs { for _, n := range needs {
switch v := n.(type) { switch v := n.(type) {
case string: case string:
names = append(names, v) entries = append(entries, needEntry{job: v})
case map[string]any: case map[string]any:
if _, crossPipeline := v["pipeline"]; crossPipeline { if _, crossPipeline := v["pipeline"]; crossPipeline {
continue continue
} }
if job, ok := v["job"].(string); ok { if job, ok := v["job"].(string); ok {
names = append(names, job) optional, _ := v["optional"].(bool)
entries = append(entries, needEntry{job: job, optional: optional})
} }
} }
} }
return names return entries
} }
func detectNeedsCycles(graph map[string][]string) []Finding { func detectNeedsCycles(graph map[string][]string, jobs map[string]model.Job) []Finding {
const ( const (
unvisited = 0 unvisited = 0
visiting = 1 visiting = 1
@@ -101,9 +123,13 @@ func detectNeedsCycles(graph map[string][]string) []Finding {
case visiting: case visiting:
if !reported[name] { if !reported[name] {
reported[name] = true reported[name] = true
j := jobs[name]
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Rule: RuleNeedsCycle,
Job: name, Job: name,
File: j.File,
Line: j.Line,
Message: fmt.Sprintf("circular dependency in needs: %v → %s", path, name), Message: fmt.Sprintf("circular dependency in needs: %v → %s", path, name),
}) })
} }
+117
View File
@@ -0,0 +1,117 @@
package linter
// Rule ID constants. Each ID is stable across versions and uniquely identifies
// one lint check. Use these when filtering output, writing suppression rules,
// or referencing a check in documentation.
//
// Prefix GL = Glint / GitLab CI lint.
// Numbering is sequential by category; gaps may appear as rules are added.
const (
// ── Pipeline-level ──────────────────────────────────────────────────────
// GL001: no stages: block defined; GitLab falls back to default stages.
RuleNoStages = "GL001"
// GL002: workflow.rules[n].when has an invalid value (only always/never allowed).
RuleWorkflowWhen = "GL002"
// ── Job structure ────────────────────────────────────────────────────────
// GL003: job is missing a required script: (or run:) field.
RuleMissingScript = "GL003"
// GL004: job references a stage not declared in stages:.
RuleUnknownStage = "GL004"
// GL005: only: and rules: used together on the same job.
RuleOnlyRulesConflict = "GL005"
// GL006: except: and rules: used together on the same job.
RuleExceptRulesConflict = "GL006"
// GL007: only:/except: used (deprecated; prefer rules:).
RuleDeprecatedOnly = "GL007"
// ── Keyword constraints ──────────────────────────────────────────────────
// GL008: when: has an invalid value.
RuleInvalidWhen = "GL008"
// GL009: when: delayed without start_in:.
RuleDelayedNoStartIn = "GL009"
// GL010: start_in: set when when: is not delayed.
RuleStartInNoDelayed = "GL010"
// GL011: parallel: value is invalid (integer out of range or map missing matrix:).
RuleInvalidParallel = "GL011"
// GL012: retry: integer is out of range 02, or retry: is neither int nor map.
RuleInvalidRetry = "GL012"
// GL013: retry.when: contains an unrecognised failure type.
RuleInvalidRetryWhen = "GL013"
// GL014: allow_failure: is not a boolean or a map with exit_codes:.
RuleInvalidAllowFailure = "GL014"
// GL015: interruptible: is not a boolean.
RuleInvalidInterruptible = "GL015"
// GL016: trigger: job also defines script: (mutually exclusive).
RuleTriggerWithScript = "GL016"
// GL017: trigger: map does not specify project: or include:.
RuleInvalidTrigger = "GL017"
// GL018: coverage: is not a regex pattern wrapped in /.
RuleInvalidCoverage = "GL018"
// GL019: release: is missing required tag_name:, or is not a map.
RuleInvalidRelease = "GL019"
// GL020: environment: has an invalid url/action configuration.
RuleInvalidEnvironment = "GL020"
// GL021: artifacts: has an invalid when/expose_as configuration.
RuleInvalidArtifacts = "GL021"
// GL022: pages job artifacts.paths does not include public/.
RulePagesPublic = "GL022"
// GL023: cache: has an invalid when/policy value.
RuleInvalidCache = "GL023"
// GL024: rules[n].when has an invalid value.
RuleInvalidRulesWhen = "GL024"
// GL025: image: map form is missing a name: key.
RuleInvalidImage = "GL025"
// GL026: inherit.default or inherit.variables is not a boolean or list.
RuleInvalidInherit = "GL026"
// ── Cross-job graph ──────────────────────────────────────────────────────
// GL027: needs: references a job that does not exist in the pipeline.
RuleNeedsUnknown = "GL027"
// GL028: needs: references a job in a later stage than the current job.
RuleNeedsStageOrder = "GL028"
// GL029: circular dependency detected in the needs: graph.
RuleNeedsCycle = "GL029"
// GL030: dependencies: references a job that does not exist.
RuleUnknownDependency = "GL030"
// GL031: dependencies: references a job in the same or a later stage.
RuleDependencyStage = "GL031"
// ── Expression validation ────────────────────────────────────────────────
// GL032: rules:if: references a variable not declared in pipeline variables:,
// the job's own variables:, or any workflow:rules:variables: block.
// May be a false positive for variables set in GitLab CI/CD project settings.
RuleUndeclaredVariable = "GL032"
)
+89
View File
@@ -0,0 +1,89 @@
package linter_test
import (
"path/filepath"
"testing"
"git.k3nny.fr/glint/internal/fetcher"
"git.k3nny.fr/glint/internal/linter"
"git.k3nny.fr/glint/internal/model"
"git.k3nny.fr/glint/internal/resolver"
)
// TestSambaCI verifies that the Samba project's .gitlab-ci.yml (a real-world
// pipeline that is valid on GitLab) produces no Error findings.
// These files exercise local include resolution and multi-level extends chains.
func TestSambaCI(t *testing.T) {
entryPoint := "../../testdata/samba/.gitlab-ci.yml"
p, err := model.Parse(entryPoint)
if err != nil {
t.Fatalf("Parse: %v", err)
}
rootDir := filepath.Dir(filepath.Clean(entryPoint))
incWarnings, _ := resolver.ResolveIncludes(p, fetcher.GitLabConfig{}, rootDir)
for _, w := range incWarnings {
t.Logf("include warning: %s", w)
}
extWarnings, err := resolver.Resolve(p)
if err != nil {
t.Fatalf("Resolve: %v", err)
}
for _, w := range extWarnings {
t.Logf("extends warning: job %q extends unknown %q", w.Job, w.Base)
}
findings := linter.Lint(p)
for _, f := range findings {
if f.Severity == linter.Error {
t.Errorf("unexpected error finding on valid Samba CI: %s", f)
}
}
}
// TestSambaCIEntryFiles verifies all of the Samba entry-point files
// (files that can each act as the top-level CI file) lint without errors.
func TestSambaCIEntryFiles(t *testing.T) {
entryPoints := []struct {
name string
path string
}{
{"default", "../../testdata/samba/.gitlab-ci.yml"},
{"coverage", "../../testdata/samba/.gitlab-ci-coverage.yml"},
{"private", "../../testdata/samba/.gitlab-ci-private.yml"},
}
for _, tc := range entryPoints {
tc := tc
t.Run(tc.name, func(t *testing.T) {
p, err := model.Parse(tc.path)
if err != nil {
t.Fatalf("Parse: %v", err)
}
rootDir := filepath.Dir(filepath.Clean(tc.path))
incWarnings, _ := resolver.ResolveIncludes(p, fetcher.GitLabConfig{}, rootDir)
for _, w := range incWarnings {
t.Logf("include warning: %s", w)
}
extWarnings, err := resolver.Resolve(p)
if err != nil {
t.Fatalf("Resolve: %v", err)
}
for _, w := range extWarnings {
t.Logf("extends warning: job %q extends unknown %q", w.Job, w.Base)
}
findings := linter.Lint(p)
for _, f := range findings {
if f.Severity == linter.Error {
t.Errorf("unexpected error finding: %s", f)
}
}
})
}
}
+158
View File
@@ -0,0 +1,158 @@
package linter
import (
"fmt"
"strings"
"git.k3nny.fr/glint/internal/model"
)
// predefinedVarPrefixes lists GitLab-maintained variable namespaces that are
// always available without an explicit declaration in variables: blocks.
var predefinedVarPrefixes = []string{
"CI_", // most predefined CI variables (CI_COMMIT_BRANCH, CI_JOB_ID, …)
"GITLAB_", // user/project metadata (GITLAB_USER_ID, GITLAB_FEATURES, …)
"FF_", // GitLab feature flags
"RUNNER_", // runner-level variables
"TRIGGER_", // trigger token variables passed from upstream pipelines
"CHAT_", // ChatOps variables
}
func isPredefinedVar(name string) bool {
for _, prefix := range predefinedVarPrefixes {
if strings.HasPrefix(name, prefix) {
return true
}
}
return false
}
// extractIfVars returns every variable name referenced in a rules:if: expression.
// String literals are skipped so that dollar signs inside quoted values are
// not mistaken for variable references.
func extractIfVars(expr string) []string {
var names []string
i := 0
for i < len(expr) {
switch expr[i] {
case '"', '\'':
quote := expr[i]
i++
for i < len(expr) && expr[i] != quote {
if expr[i] == '\\' {
i++ // skip escaped character
}
i++
}
if i < len(expr) {
i++ // consume closing quote
}
case '$':
i++ // consume '$'
if i < len(expr) && expr[i] == '{' {
i++ // consume '{'
start := i
for i < len(expr) && isVarNameByte(expr[i]) {
i++
}
if i < len(expr) && expr[i] == '}' && i > start {
names = append(names, expr[start:i])
i++ // consume '}'
}
} else {
start := i
for i < len(expr) && isVarNameByte(expr[i]) {
i++
}
if i > start {
names = append(names, expr[start:i])
}
}
default:
i++
}
}
return names
}
func isVarNameByte(b byte) bool {
return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || (b >= '0' && b <= '9') || b == '_'
}
// checkVariableRefs warns when a rules:if: expression references a variable that
// is not declared in pipeline variables:, the job's own variables:, or any
// workflow:rules:variables: block. Predefined GitLab CI variables (CI_*, GITLAB_*,
// …) are always exempt. Variables set in GitLab CI/CD project settings are
// invisible to glint, so the finding is a WARNING rather than an error.
func checkVariableRefs(p *model.Pipeline) []Finding {
pipelineVars := make(map[string]bool, len(p.Variables))
for k := range p.Variables {
pipelineVars[k] = true
}
// Union of all variables any workflow rule might inject into the context.
workflowRuleVars := make(map[string]bool)
if p.Workflow != nil {
for _, rule := range p.Workflow.Rules {
for k := range rule.Variables {
workflowRuleVars[k] = true
}
}
}
var findings []Finding
// Check workflow rules:if: expressions.
if p.Workflow != nil {
seen := make(map[string]bool)
for i, rule := range p.Workflow.Rules {
if rule.If == "" {
continue
}
for _, varName := range extractIfVars(rule.If) {
if isPredefinedVar(varName) || pipelineVars[varName] || seen[varName] {
continue
}
seen[varName] = true
findings = append(findings, Finding{
Severity: Warning,
Rule: RuleUndeclaredVariable,
File: p.SourceFile,
Message: fmt.Sprintf("workflow.rules[%d].if: $%s is not declared in pipeline variables:", i, varName),
})
}
}
}
// Check each job's rules:if: expressions.
for name, job := range p.Jobs {
jobVars := make(map[string]bool, len(job.Variables))
for k := range job.Variables {
jobVars[k] = true
}
// Deduplicate per (job, varName): report each undeclared variable once per job.
seen := make(map[string]bool)
for i, rule := range job.Rules {
if rule.If == "" {
continue
}
for _, varName := range extractIfVars(rule.If) {
if isPredefinedVar(varName) || pipelineVars[varName] || jobVars[varName] || workflowRuleVars[varName] || seen[varName] {
continue
}
seen[varName] = true
findings = append(findings, Finding{
Severity: Warning,
Rule: RuleUndeclaredVariable,
Job: name,
File: job.File,
Line: job.Line,
Message: fmt.Sprintf("rules[%d].if: $%s is not declared in pipeline or job variables:", i, varName),
})
}
}
}
return findings
}
+214
View File
@@ -0,0 +1,214 @@
package linter
import (
"testing"
"git.k3nny.fr/glint/internal/model"
)
func TestExtractIfVars(t *testing.T) {
cases := []struct {
name string
expr string
want []string
}{
{
name: "simple variable",
expr: `$MY_VAR == "value"`,
want: []string{"MY_VAR"},
},
{
name: "curly brace syntax",
expr: `${MY_VAR} != null`,
want: []string{"MY_VAR"},
},
{
name: "multiple variables",
expr: `$BRANCH == "main" && $DEPLOY_ENV == "prod"`,
want: []string{"BRANCH", "DEPLOY_ENV"},
},
{
name: "dollar sign inside string literal is skipped",
expr: `$REAL_VAR == "$not_a_var"`,
want: []string{"REAL_VAR"},
},
{
name: "no variables",
expr: `"main" == "main"`,
want: nil,
},
{
name: "regex rhs variable",
expr: `$BRANCH =~ $PATTERN`,
want: []string{"BRANCH", "PATTERN"},
},
{
name: "multiline expression",
expr: "$BRANCH == \"main\" ||\n$BRANCH == \"develop\"",
want: []string{"BRANCH", "BRANCH"},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := extractIfVars(tc.expr)
if len(got) != len(tc.want) {
t.Fatalf("extractIfVars(%q) = %v, want %v", tc.expr, got, tc.want)
}
for i, v := range got {
if v != tc.want[i] {
t.Errorf("[%d] got %q, want %q", i, v, tc.want[i])
}
}
})
}
}
func TestIsPredefinedVar(t *testing.T) {
cases := []struct {
input string
want bool
}{
{"CI_COMMIT_BRANCH", true},
{"CI_JOB_TOKEN", true},
{"GITLAB_USER_ID", true},
{"GITLAB_FEATURES", true},
{"FF_SOME_FLAG", true},
{"RUNNER_ID", true},
{"TRIGGER_PAYLOAD", true},
{"CHAT_INPUT", true},
{"MY_CUSTOM_VAR", false},
{"DEPLOY_ENV", false},
{"FEATURE_ENABLED", false},
}
for _, tc := range cases {
t.Run(tc.input, func(t *testing.T) {
if got := isPredefinedVar(tc.input); got != tc.want {
t.Errorf("isPredefinedVar(%q) = %v, want %v", tc.input, got, tc.want)
}
})
}
}
func TestCheckVariableRefs(t *testing.T) {
cases := []struct {
name string
pipeline *model.Pipeline
wantWarnings int
}{
{
name: "declared pipeline variable — no warning",
pipeline: &model.Pipeline{
Variables: map[string]any{"MY_VAR": "value"},
Jobs: map[string]model.Job{
"job-a": {Rules: []model.Rule{{If: `$MY_VAR == "value"`}}},
},
},
wantWarnings: 0,
},
{
name: "predefined CI variable — no warning",
pipeline: &model.Pipeline{
Jobs: map[string]model.Job{
"job-a": {Rules: []model.Rule{{If: `$CI_COMMIT_BRANCH == "main"`}}},
},
},
wantWarnings: 0,
},
{
name: "undeclared variable — one warning",
pipeline: &model.Pipeline{
Jobs: map[string]model.Job{
"job-a": {Rules: []model.Rule{{If: `$UNDEFINED_VAR == "yes"`}}},
},
},
wantWarnings: 1,
},
{
name: "same undeclared var in multiple rules — one warning per job",
pipeline: &model.Pipeline{
Jobs: map[string]model.Job{
"job-a": {
Rules: []model.Rule{
{If: `$UNDEFINED_VAR == "yes"`},
{If: `$UNDEFINED_VAR == "no"`},
},
},
},
},
wantWarnings: 1,
},
{
name: "job-level variable — no warning",
pipeline: &model.Pipeline{
Jobs: map[string]model.Job{
"job-a": {
Variables: map[string]any{"LOCAL_VAR": "value"},
Rules: []model.Rule{{If: `$LOCAL_VAR == "value"`}},
},
},
},
wantWarnings: 0,
},
{
name: "workflow rule variable available to job rules — no warning",
pipeline: &model.Pipeline{
Workflow: &model.Workflow{
Rules: []model.Rule{
{
If: `$CI_COMMIT_BRANCH == "main"`,
Variables: map[string]any{"DEPLOY_ENV": "production"},
},
},
},
Jobs: map[string]model.Job{
"job-a": {Rules: []model.Rule{{If: `$DEPLOY_ENV == "production"`}}},
},
},
wantWarnings: 0,
},
{
name: "undeclared variable in workflow rules:if",
pipeline: &model.Pipeline{
Workflow: &model.Workflow{
Rules: []model.Rule{{If: `$UNDECLARED == "main"`}},
},
Jobs: map[string]model.Job{},
},
wantWarnings: 1,
},
{
name: "two jobs each with a different undeclared variable — two warnings",
pipeline: &model.Pipeline{
Jobs: map[string]model.Job{
"job-a": {Rules: []model.Rule{{If: `$UNDEF_A == "x"`}}},
"job-b": {Rules: []model.Rule{{If: `$UNDEF_B == "y"`}}},
},
},
wantWarnings: 2,
},
{
name: "no rules — no warnings",
pipeline: &model.Pipeline{
Jobs: map[string]model.Job{
"job-a": {Script: "echo ok"},
},
},
wantWarnings: 0,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
findings := checkVariableRefs(tc.pipeline)
count := 0
for _, f := range findings {
if f.Severity == Warning && f.Rule == RuleUndeclaredVariable {
count++
}
}
if count != tc.wantWarnings {
t.Errorf("got %d GL032 warnings, want %d; findings: %v", count, tc.wantWarnings, findings)
}
})
}
}
+30 -7
View File
@@ -13,14 +13,22 @@ func Parse(path string) (*Pipeline, error) {
if err != nil { if err != nil {
return nil, fmt.Errorf("reading file: %w", err) return nil, fmt.Errorf("reading file: %w", err)
} }
return ParseBytes(data) p, err := ParseBytes(data)
if err != nil {
return nil, err
}
p.SourceFile = path
p.SetJobOrigin(path)
return p, nil
} }
// ParseBytes parses YAML from an in-memory byte slice. // ParseBytes parses YAML from an in-memory byte slice.
func ParseBytes(data []byte) (*Pipeline, error) { func ParseBytes(data []byte) (*Pipeline, error) {
// First pass: decode into a raw map to extract job keys. // First pass: parse into a yaml.Node document to extract job keys with
var raw map[string]yaml.Node // their exact source line numbers (key nodes carry the line, value nodes
if err := yaml.Unmarshal(data, &raw); err != nil { // carry the body we decode into Job / map[string]any).
var doc yaml.Node
if err := yaml.Unmarshal(data, &doc); err != nil {
return nil, fmt.Errorf("parsing YAML: %w", err) return nil, fmt.Errorf("parsing YAML: %w", err)
} }
@@ -32,21 +40,36 @@ func ParseBytes(data []byte) (*Pipeline, error) {
p.Jobs = make(map[string]Job) p.Jobs = make(map[string]Job)
p.RawJobs = make(map[string]map[string]any) p.RawJobs = make(map[string]map[string]any)
for key, node := range raw {
if doc.Kind != yaml.DocumentNode || len(doc.Content) == 0 {
return p, nil
}
root := doc.Content[0]
if root.Kind != yaml.MappingNode {
return p, nil
}
// Walk root mapping in key/value pairs.
for i := 0; i+1 < len(root.Content); i += 2 {
keyNode := root.Content[i]
valNode := root.Content[i+1]
key := keyNode.Value
if ReservedKeys[key] { if ReservedKeys[key] {
continue continue
} }
var rawMap map[string]any var rawMap map[string]any
if err := node.Decode(&rawMap); err != nil { if err := valNode.Decode(&rawMap); err != nil {
return nil, fmt.Errorf("parsing raw job %q: %w", key, err) return nil, fmt.Errorf("parsing raw job %q: %w", key, err)
} }
p.RawJobs[key] = rawMap p.RawJobs[key] = rawMap
var j Job var j Job
if err := node.Decode(&j); err != nil { if err := valNode.Decode(&j); err != nil {
return nil, fmt.Errorf("parsing job %q: %w", key, err) return nil, fmt.Errorf("parsing job %q: %w", key, err)
} }
j.Name = key j.Name = key
j.Line = keyNode.Line // exact line of the job name key
p.Jobs[key] = j p.Jobs[key] = j
} }
+25 -10
View File
@@ -3,8 +3,9 @@ package model
// Pipeline represents the top-level structure of a .gitlab-ci.yml file. // Pipeline represents the top-level structure of a .gitlab-ci.yml file.
// Unknown top-level keys are collected into Jobs. // Unknown top-level keys are collected into Jobs.
type Pipeline struct { type Pipeline struct {
SourceFile string // path of the root pipeline file; set by Parse
Stages []string `yaml:"stages"` Stages []string `yaml:"stages"`
Variables map[string]string `yaml:"variables"` Variables map[string]any `yaml:"variables"` // string or {value,description,options} map
Default *DefaultConfig `yaml:"default"` Default *DefaultConfig `yaml:"default"`
Include []any `yaml:"include"` Include []any `yaml:"include"`
Workflow *Workflow `yaml:"workflow"` Workflow *Workflow `yaml:"workflow"`
@@ -13,10 +14,21 @@ type Pipeline struct {
RawJobs map[string]map[string]any `yaml:"-"` // pre-resolution raw maps, used by the resolver RawJobs map[string]map[string]any `yaml:"-"` // pre-resolution raw maps, used by the resolver
} }
// SetJobOrigin sets the File field on all jobs that don't already have one.
// Called after ParseBytes to record which file each job came from.
func (p *Pipeline) SetJobOrigin(file string) {
for name, j := range p.Jobs {
if j.File == "" {
j.File = file
}
p.Jobs[name] = j
}
}
type DefaultConfig struct { type DefaultConfig struct {
Image string `yaml:"image"` Image any `yaml:"image"` // string or {name,pull_policy,...} map
BeforeScript []string `yaml:"before_script"` BeforeScript any `yaml:"before_script"` // []string or string (block scalar)
AfterScript []string `yaml:"after_script"` AfterScript any `yaml:"after_script"` // []string or string
Cache any `yaml:"cache"` Cache any `yaml:"cache"`
Artifacts any `yaml:"artifacts"` Artifacts any `yaml:"artifacts"`
Retry any `yaml:"retry"` Retry any `yaml:"retry"`
@@ -30,14 +42,16 @@ type Workflow struct {
type Job struct { type Job struct {
Name string // set by parser, not from YAML Name string // set by parser, not from YAML
File string // source file; set by Parse / resolver
Line int // line of the job key in its source file; set by parser
Stage string `yaml:"stage"` Stage string `yaml:"stage"`
Script []string `yaml:"script"` Script any `yaml:"script"` // []string or string (block scalar)
Run any `yaml:"run"` // alternative to script (CI steps) Run any `yaml:"run"` // alternative to script (CI steps)
BeforeScript []string `yaml:"before_script"` BeforeScript any `yaml:"before_script"` // []string or string
AfterScript []string `yaml:"after_script"` AfterScript any `yaml:"after_script"` // []string or string
Image any `yaml:"image"` Image any `yaml:"image"`
Services []any `yaml:"services"` Services []any `yaml:"services"`
Variables map[string]string `yaml:"variables"` Variables map[string]any `yaml:"variables"` // string or {value,description,options} map
Rules []Rule `yaml:"rules"` Rules []Rule `yaml:"rules"`
Only any `yaml:"only"` Only any `yaml:"only"`
Except any `yaml:"except"` Except any `yaml:"except"`
@@ -68,8 +82,9 @@ type Job struct {
type Rule struct { type Rule struct {
If string `yaml:"if"` If string `yaml:"if"`
When string `yaml:"when"` When string `yaml:"when"`
Changes []string `yaml:"changes"` Changes any `yaml:"changes"` // []string or {paths,compare_to} map
Exists []string `yaml:"exists"` Exists any `yaml:"exists"` // []string or map form
Variables map[string]any `yaml:"variables"` // set/override variables when rule matches (GitLab CI 15.0+)
} }
// ReservedKeys are top-level GitLab CI keys that are NOT job definitions. // ReservedKeys are top-level GitLab CI keys that are NOT job definitions.
+21 -9
View File
@@ -3,40 +3,52 @@ package resolver
import ( import (
"fmt" "fmt"
"git.k3nny.fr/gitlab-sim/internal/model" "git.k3nny.fr/glint/internal/model"
"gopkg.in/yaml.v3" "gopkg.in/yaml.v3"
) )
// Resolve resolves all extends: references in p.Jobs in place. // Resolve resolves all extends: references in p.Jobs in place.
// Jobs are merged depth-first so that base definitions are resolved before // Jobs are merged depth-first so that base definitions are resolved before
// derived ones. Mutates p.Jobs with the fully merged Job structs. // derived ones. Mutates p.Jobs with the fully merged Job structs.
func Resolve(p *model.Pipeline) error { //
// The first return value lists extends references whose base job could not be
// found (e.g. it lives in a remote include that was not fetched). Those jobs
// are left unmerged but still passed to the linter. A non-nil error is only
// returned for unrecoverable situations such as circular dependencies.
func Resolve(p *model.Pipeline) ([]ExtendWarning, error) {
var extWarnings []ExtendWarning
// Build extends graph: jobName -> ordered list of base job names. // Build extends graph: jobName -> ordered list of base job names.
extendsGraph := make(map[string][]string, len(p.Jobs)) extendsGraph := make(map[string][]string, len(p.Jobs))
for name, job := range p.Jobs { for name, job := range p.Jobs {
bases, err := parseExtends(job.Extends) bases, err := parseExtends(job.Extends)
if err != nil { if err != nil {
return fmt.Errorf("job %q: invalid extends: %w", name, err) return extWarnings, fmt.Errorf("job %q: invalid extends: %w", name, err)
} }
if len(bases) == 0 { if len(bases) == 0 {
continue continue
} }
skip := false
for _, base := range bases { for _, base := range bases {
if _, ok := p.RawJobs[base]; !ok { if _, ok := p.RawJobs[base]; !ok {
return fmt.Errorf("job %q extends unknown job %q", name, base) extWarnings = append(extWarnings, ExtendWarning{Job: name, Base: base})
skip = true
} }
} }
if skip {
continue // leave this job unmerged; still linted with its own fields
}
extendsGraph[name] = bases extendsGraph[name] = bases
} }
if len(extendsGraph) == 0 { if len(extendsGraph) == 0 {
return nil return extWarnings, nil
} }
// Topological sort — bases must be resolved before derived jobs. // Topological sort — bases must be resolved before derived jobs.
order, err := topoSort(extendsGraph) order, err := topoSort(extendsGraph)
if err != nil { if err != nil {
return err return extWarnings, err
} }
// resolved holds the final merged raw map for each processed job. // resolved holds the final merged raw map for each processed job.
@@ -61,17 +73,17 @@ func Resolve(p *model.Pipeline) error {
// Re-decode the merged map into a Job struct. // Re-decode the merged map into a Job struct.
data, err := yaml.Marshal(merged) data, err := yaml.Marshal(merged)
if err != nil { if err != nil {
return fmt.Errorf("job %q: re-encoding merged definition: %w", name, err) return extWarnings, fmt.Errorf("job %q: re-encoding merged definition: %w", name, err)
} }
var j model.Job var j model.Job
if err := yaml.Unmarshal(data, &j); err != nil { if err := yaml.Unmarshal(data, &j); err != nil {
return fmt.Errorf("job %q: re-decoding merged definition: %w", name, err) return extWarnings, fmt.Errorf("job %q: re-decoding merged definition: %w", name, err)
} }
j.Name = name j.Name = name
p.Jobs[name] = j p.Jobs[name] = j
} }
return nil return extWarnings, nil
} }
// parseExtends normalises the extends field (string or []any) into []string. // parseExtends normalises the extends field (string or []any) into []string.
+165 -46
View File
@@ -2,18 +2,18 @@ package resolver
import ( import (
"fmt" "fmt"
"os"
"path/filepath"
"strings" "strings"
"git.k3nny.fr/gitlab-sim/internal/fetcher" "git.k3nny.fr/glint/internal/fetcher"
"git.k3nny.fr/gitlab-sim/internal/model" "git.k3nny.fr/glint/internal/model"
) )
// IncludeWarning describes a remote include entry that could not be resolved. // IncludeWarning describes an include entry that could not be resolved.
// These are surfaced to the user as [WARNING] lines before the lint findings. // These are surfaced to the user as [WARNING] lines before the lint findings.
type IncludeWarning struct { type IncludeWarning struct {
// Label is a short human-readable identifier shown in the warning message, // Label is a short human-readable identifier shown in the warning message.
// e.g. "project my-group/templates:/ci.yml@main" or
// "component gitlab.com/components/golang/build@v1.0".
Label string Label string
Err error // nil when Skipped is true Err error // nil when Skipped is true
Skipped bool // no token available — skipped without attempting a network call Skipped bool // no token available — skipped without attempting a network call
@@ -30,46 +30,155 @@ func (w IncludeWarning) String() string {
w.Label, w.Err) w.Label, w.Err)
} }
// ExtendWarning describes an extends: reference whose base job could not be
// found. This typically means the base is in a remote include that was not
// fetched (e.g. no token). The job's extends chain is skipped; the job itself
// is still linted with whatever fields it directly defines.
type ExtendWarning struct {
Job string // job that declares the extends
Base string // the unknown base job name
}
// ResolveIncludes processes the pipeline's include: block. // ResolveIncludes processes the pipeline's include: block.
// //
// Project includes (include: project: ...) require authentication and are // rootDir is the repository root directory used to resolve local: includes.
// skipped with a warning when no token is configured. // In practice this is the directory of the top-level pipeline file being linted.
// //
// Component includes (include: component: ...) attempt the fetch // Local includes are read from disk and merged recursively.
// unauthenticated first, so public CI/CD catalog components work without a // Project includes require authentication and are skipped with a warning when
// token. A warning is emitted when the fetch fails. // no token is configured. Component includes are attempted unauthenticated.
// Remote and template includes are silently skipped (resolved by GitLab at runtime).
// //
// Non-resolvable include forms (local, remote, template) are silently skipped. // The second return value carries extends warnings discovered while recursively
func ResolveIncludes(p *model.Pipeline, cfg fetcher.GitLabConfig) []IncludeWarning { // processing included files (forwarded from resolver.Resolve calls).
func ResolveIncludes(p *model.Pipeline, cfg fetcher.GitLabConfig, rootDir string) ([]IncludeWarning, []ExtendWarning) {
visited := map[string]bool{}
return resolveIncludes(p, p.Include, cfg, rootDir, visited)
}
// resolveIncludes is the recursive core of ResolveIncludes.
func resolveIncludes(p *model.Pipeline, includes []any, cfg fetcher.GitLabConfig, rootDir string, visited map[string]bool) ([]IncludeWarning, []ExtendWarning) {
var warnings []IncludeWarning var warnings []IncludeWarning
for _, inc := range p.Include { var extWarnings []ExtendWarning
for _, inc := range includes {
entry, ok := normaliseInclude(inc) entry, ok := normaliseInclude(inc)
if !ok { if !ok {
continue continue
} }
if project, _ := entry["project"].(string); project != "" { if project, _ := entry["project"].(string); project != "" {
warnings = append(warnings, resolveProjectInclude(p, entry, project, cfg)...) w, ew := resolveProjectInclude(p, entry, project, cfg, rootDir, visited)
warnings = append(warnings, w...)
extWarnings = append(extWarnings, ew...)
continue continue
} }
if compRef, _ := entry["component"].(string); compRef != "" { if compRef, _ := entry["component"].(string); compRef != "" {
if w, ok := resolveComponentInclude(p, compRef, cfg); ok { w, ew, hadErr := resolveComponentInclude(p, compRef, cfg, rootDir, visited)
if hadErr {
warnings = append(warnings, w) warnings = append(warnings, w)
} }
extWarnings = append(extWarnings, ew...)
continue continue
} }
// local, remote, template — resolved by GitLab at runtime, skip silently. if local, _ := entry["local"].(string); local != "" {
w, ew := resolveLocalInclude(p, local, cfg, rootDir, visited)
warnings = append(warnings, w...)
extWarnings = append(extWarnings, ew...)
continue
} }
return warnings
if remote, _ := entry["remote"].(string); remote != "" {
w, ew := resolveRemoteInclude(p, remote, cfg, rootDir, visited)
warnings = append(warnings, w...)
extWarnings = append(extWarnings, ew...)
continue
}
// template — resolved by GitLab at runtime, skip silently.
}
return warnings, extWarnings
}
// resolveLocalInclude reads a local file from disk (paths are always relative
// to the repository root, with or without a leading slash), recursively
// resolves its own includes, and merges it into p.
func resolveLocalInclude(p *model.Pipeline, rawPath string, cfg fetcher.GitLabConfig, rootDir string, visited map[string]bool) ([]IncludeWarning, []ExtendWarning) {
relPath := strings.TrimPrefix(rawPath, "/")
absPath := filepath.Join(rootDir, relPath)
label := "local " + rawPath
if visited[absPath] {
return nil, nil
}
visited[absPath] = true
data, err := os.ReadFile(absPath)
if err != nil {
return []IncludeWarning{{Label: label, Err: err}}, nil
}
included, err := model.ParseBytes(data)
if err != nil {
return []IncludeWarning{{Label: label, Err: fmt.Errorf("parsing YAML: %w", err)}}, nil
}
included.SetJobOrigin(absPath)
// Recursively resolve the included file's own includes first, merging
// everything into `included` before we merge it into the parent `p`.
var warnings []IncludeWarning
var extWarnings []ExtendWarning
if len(included.Include) > 0 {
w, ew := resolveIncludes(included, included.Include, cfg, rootDir, visited)
warnings = append(warnings, w...)
extWarnings = append(extWarnings, ew...)
}
mergeIncluded(p, included)
return warnings, extWarnings
}
// resolveRemoteInclude fetches a plain HTTPS URL, parses it as CI YAML, and
// merges it into p. Sub-includes of the fetched file are resolved recursively.
func resolveRemoteInclude(p *model.Pipeline, rawURL string, cfg fetcher.GitLabConfig, rootDir string, visited map[string]bool) ([]IncludeWarning, []ExtendWarning) {
label := "remote " + rawURL
if visited[rawURL] {
return nil, nil
}
visited[rawURL] = true
data, err := fetcher.FetchURL(rawURL)
if err != nil {
return []IncludeWarning{{Label: label, Err: err}}, nil
}
included, err := model.ParseBytes(data)
if err != nil {
return []IncludeWarning{{Label: label, Err: fmt.Errorf("parsing YAML: %w", err)}}, nil
}
included.SetJobOrigin(rawURL)
var warnings []IncludeWarning
var extWarnings []ExtendWarning
if len(included.Include) > 0 {
w, ew := resolveIncludes(included, included.Include, cfg, rootDir, visited)
warnings = append(warnings, w...)
extWarnings = append(extWarnings, ew...)
}
mergeIncluded(p, included)
return warnings, extWarnings
} }
// resolveProjectInclude fetches all files listed under a single project: entry // resolveProjectInclude fetches all files listed under a single project: entry
// and merges them into p. // and merges them into p.
func resolveProjectInclude(p *model.Pipeline, entry map[string]any, project string, cfg fetcher.GitLabConfig) []IncludeWarning { func resolveProjectInclude(p *model.Pipeline, entry map[string]any, project string, cfg fetcher.GitLabConfig, rootDir string, visited map[string]bool) ([]IncludeWarning, []ExtendWarning) {
ref, _ := entry["ref"].(string) ref, _ := entry["ref"].(string)
var warnings []IncludeWarning var warnings []IncludeWarning
var extWarnings []ExtendWarning
for _, filePath := range includeFiles(entry) { for _, filePath := range includeFiles(entry) {
label := fmt.Sprintf("project %s:%s", project, filePath) label := fmt.Sprintf("project %s:%s", project, filePath)
@@ -93,57 +202,61 @@ func resolveProjectInclude(p *model.Pipeline, entry map[string]any, project stri
warnings = append(warnings, IncludeWarning{Label: label, Err: fmt.Errorf("parsing YAML: %w", err)}) warnings = append(warnings, IncludeWarning{Label: label, Err: fmt.Errorf("parsing YAML: %w", err)})
continue continue
} }
included.SetJobOrigin(label)
if len(included.Include) > 0 {
w, ew := resolveIncludes(included, included.Include, cfg, rootDir, visited)
warnings = append(warnings, w...)
extWarnings = append(extWarnings, ew...)
}
mergeIncluded(p, included) mergeIncluded(p, included)
} }
return warnings return warnings, extWarnings
} }
// resolveComponentInclude fetches a CI/CD catalog component and merges it into p. // resolveComponentInclude fetches a CI/CD catalog component and merges it into p.
// Returns (warning, true) if something went wrong; (zero, false) on success. // Returns (warning, extWarnings, true) if something went wrong; (zero, nil, false) on success.
func resolveComponentInclude(p *model.Pipeline, ref string, cfg fetcher.GitLabConfig) (IncludeWarning, bool) { func resolveComponentInclude(p *model.Pipeline, ref string, cfg fetcher.GitLabConfig, rootDir string, visited map[string]bool) (IncludeWarning, []ExtendWarning, bool) {
label := "component " + ref label := "component " + ref
// Variable interpolation (e.g. $CI_SERVER_FQDN) cannot be resolved locally.
if strings.ContainsRune(ref, '$') { if strings.ContainsRune(ref, '$') {
return IncludeWarning{ return IncludeWarning{
Label: label, Label: label,
Err: fmt.Errorf("component reference contains a CI variable that cannot be resolved at lint time"), Err: fmt.Errorf("component reference contains a CI variable that cannot be resolved at lint time"),
}, true }, nil, true
} }
host, project, component, version, err := parseComponentRef(ref) host, project, component, version, err := parseComponentRef(ref)
if err != nil { if err != nil {
return IncludeWarning{Label: label, Err: err}, true return IncludeWarning{Label: label, Err: err}, nil, true
} }
hostCfg := cfg.ForHost(host) hostCfg := cfg.ForHost(host)
data, err := fetchComponentFile(hostCfg, project, component, version) data, err := fetchComponentFile(hostCfg, project, component, version)
if err != nil { if err != nil {
return IncludeWarning{Label: label, Err: err}, true return IncludeWarning{Label: label, Err: err}, nil, true
} }
included, err := model.ParseBytes(data) included, err := model.ParseBytes(data)
if err != nil { if err != nil {
return IncludeWarning{Label: label, Err: fmt.Errorf("parsing component YAML: %w", err)}, true return IncludeWarning{Label: label, Err: fmt.Errorf("parsing component YAML: %w", err)}, nil, true
}
included.SetJobOrigin(label)
var extWarnings []ExtendWarning
if len(included.Include) > 0 {
_, ew := resolveIncludes(included, included.Include, cfg, rootDir, visited)
extWarnings = append(extWarnings, ew...)
} }
mergeIncluded(p, included) mergeIncluded(p, included)
return IncludeWarning{}, false return IncludeWarning{}, extWarnings, false
} }
// parseComponentRef parses a CI/CD component reference of the form: // parseComponentRef parses a CI/CD component reference of the form:
// //
// <host>/<project-path>/<component-name>@<version> // <host>/<project-path>/<component-name>@<version>
//
// The host is the GitLab instance FQDN. The last path segment before @ is the
// component name; everything between host and component name is the project path.
//
// Examples:
//
// gitlab.com/components/golang/build@v1.0.0
// gitlab.example.com/my-org/ci-templates/lint@main
// gitlab.com/components/secret-detection/secret-detection@~latest
func parseComponentRef(ref string) (host, project, component, version string, err error) { func parseComponentRef(ref string) (host, project, component, version string, err error) {
atIdx := strings.LastIndex(ref, "@") atIdx := strings.LastIndex(ref, "@")
if atIdx < 0 || atIdx == len(ref)-1 { if atIdx < 0 || atIdx == len(ref)-1 {
@@ -154,7 +267,6 @@ func parseComponentRef(ref string) (host, project, component, version string, er
path := ref[:atIdx] path := ref[:atIdx]
parts := strings.Split(path, "/") parts := strings.Split(path, "/")
// Minimum: host + at least one project segment + component = 3 parts.
if len(parts) < 3 { if len(parts) < 3 {
err = fmt.Errorf("component reference %q must be <host>/<project>/<component>@<version>", ref) err = fmt.Errorf("component reference %q must be <host>/<project>/<component>@<version>", ref)
return return
@@ -167,9 +279,6 @@ func parseComponentRef(ref string) (host, project, component, version string, er
} }
// fetchComponentFile fetches a component's template YAML from a GitLab project. // fetchComponentFile fetches a component's template YAML from a GitLab project.
// GitLab supports two layouts; both are attempted:
// - templates/<component>.yml (single-file component)
// - templates/<component>/template.yml (directory component)
func fetchComponentFile(cfg fetcher.GitLabConfig, project, component, version string) ([]byte, error) { func fetchComponentFile(cfg fetcher.GitLabConfig, project, component, version string) ([]byte, error) {
primary := "templates/" + component + ".yml" primary := "templates/" + component + ".yml"
data, err := cfg.FetchFile(project, primary, version) data, err := cfg.FetchFile(project, primary, version)
@@ -180,14 +289,12 @@ func fetchComponentFile(cfg fetcher.GitLabConfig, project, component, version st
fallback := "templates/" + component + "/template.yml" fallback := "templates/" + component + "/template.yml"
data2, err2 := cfg.FetchFile(project, fallback, version) data2, err2 := cfg.FetchFile(project, fallback, version)
if err2 != nil { if err2 != nil {
// Return the primary error — it refers to the canonical path.
return nil, fmt.Errorf("%v (also tried %s: %v)", err, fallback, err2) return nil, fmt.Errorf("%v (also tried %s: %v)", err, fallback, err2)
} }
return data2, nil return data2, nil
} }
// normaliseInclude converts a raw include: list element to a string-keyed map. // normaliseInclude converts a raw include: list element to a string-keyed map.
// An include: value can be a plain string (shorthand local) or a map.
func normaliseInclude(raw any) (map[string]any, bool) { func normaliseInclude(raw any) (map[string]any, bool) {
switch v := raw.(type) { switch v := raw.(type) {
case map[string]any: case map[string]any:
@@ -199,7 +306,6 @@ func normaliseInclude(raw any) (map[string]any, bool) {
} }
// includeFiles returns the list of file paths from an include entry. // includeFiles returns the list of file paths from an include entry.
// The file: key may be a single string or a list of strings.
func includeFiles(entry map[string]any) []string { func includeFiles(entry map[string]any) []string {
raw, ok := entry["file"] raw, ok := entry["file"]
if !ok { if !ok {
@@ -220,8 +326,9 @@ func includeFiles(entry map[string]any) []string {
return nil return nil
} }
// mergeIncluded copies jobs and stages from src into dst. // mergeIncluded copies jobs, stages, and variables from src into dst.
// dst (the main pipeline) always wins when a key already exists. // dst (the main pipeline) always wins when a key already exists — this matches
// GitLab's precedence rule where root-pipeline values override included templates.
func mergeIncluded(dst, src *model.Pipeline) { func mergeIncluded(dst, src *model.Pipeline) {
stageSet := make(map[string]bool, len(dst.Stages)) stageSet := make(map[string]bool, len(dst.Stages))
for _, s := range dst.Stages { for _, s := range dst.Stages {
@@ -242,4 +349,16 @@ func mergeIncluded(dst, src *model.Pipeline) {
} }
} }
} }
// Merge pipeline-level variables: dst wins on conflict (root overrides includes).
if len(src.Variables) > 0 {
if dst.Variables == nil {
dst.Variables = make(map[string]any, len(src.Variables))
}
for k, v := range src.Variables {
if _, exists := dst.Variables[k]; !exists {
dst.Variables[k] = v
}
}
}
} }
+1 -1
View File
@@ -3,7 +3,7 @@ stages:
- deploy - deploy
# CI/CD catalog components. Without network access or a token the fetches will # CI/CD catalog components. Without network access or a token the fetches will
# fail and gitlab-sim emits a WARNING for each, then lints the local jobs normally. # fail and glint emits a WARNING for each, then lints the local jobs normally.
include: include:
# Standard single-segment component path (file layout: templates/sast.yml) # Standard single-segment component path (file layout: templates/sast.yml)
- component: gitlab.com/components/sast/sast@latest - component: gitlab.com/components/sast/sast@latest
+1 -1
View File
@@ -3,7 +3,7 @@ stages:
- test - test
# This pipeline includes templates from another GitLab project. # This pipeline includes templates from another GitLab project.
# Without a token, gitlab-sim warns but still lints the local jobs. # Without a token, glint warns but still lints the local jobs.
include: include:
- project: my-group/ci-templates - project: my-group/ci-templates
ref: main ref: main
+14
View File
@@ -0,0 +1,14 @@
---
# Exercises include: remote: URL fetching and sub-include recursion.
# The remote file is a real public GitLab CI template; it may contain its own
# includes which should also be resolved. Exit code is 0 (warnings allowed).
stages:
- test
include:
- remote: https://gitlab.com/gitlab-org/gitlab/-/raw/master/lib/gitlab/ci/templates/Bash.gitlab-ci.yml
local-job:
stage: test
script:
- echo "local job alongside remote include"
+9
View File
@@ -38,3 +38,12 @@ missing-needs-job:
- echo "bad" - echo "bad"
needs: needs:
- nonexistent-job # ERROR: job doesn't exist - nonexistent-job # ERROR: job doesn't exist
# optional: true — missing dep should be WARNING not ERROR
optional-needs-job:
stage: build
script:
- echo "I depend on something that may not exist"
needs:
- job: nonexistent-optional-job
optional: true
+51
View File
@@ -0,0 +1,51 @@
---
# rules_if_expr.yml
# Exercises the rules:if: expression evaluator for:
# - Multi-line block-scalar expressions (|| on next line)
# - ${VAR} curly-brace variable syntax
# - Regex flags (/pattern/i case-insensitive)
# - Parenthesised compound expressions
# Expected: exits 0 (lints clean with no context).
stages:
- build
- deploy
variables:
DEPLOY_ENVIRONMENTS:
value: "staging"
description: "Target deployment environment"
build:
stage: build
script: make build
rules:
# Multi-line expression: || on next line
- if: |
$CI_COMMIT_BRANCH == "main" ||
$CI_COMMIT_BRANCH == "develop"
when: on_success
- when: never
deploy-feature:
stage: deploy
script: make deploy
rules:
# ${VAR} curly-brace syntax
- if: '${CI_COMMIT_BRANCH} != null && ${CI_COMMIT_TAG} == null'
when: manual
- when: never
release:
stage: deploy
script: make release
rules:
# Case-insensitive regex flag
- if: '$CI_COMMIT_BRANCH =~ /^(main|master)$/i'
when: on_success
# Parenthesised compound with multi-line
- if: >-
($CI_COMMIT_TAG != null) &&
($CI_PIPELINE_SOURCE == "push" || $CI_PIPELINE_SOURCE == "web")
when: on_success
- when: never
+4
View File
@@ -0,0 +1,4 @@
include:
- /.gitlab-ci-default-runners.yml
# Currently we're happy with the defaults
+12
View File
@@ -0,0 +1,12 @@
# This is just used for the scheduled pipelines in the
# https://gitlab.com/samba-team/samba configuration
#
variables:
SAMBA_CI_FLAVOR: "coverage"
# "--enable-coverage" or ""
SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE: "--enable-coverage"
include:
- /.gitlab-ci-coverage-runners.yml
- /.gitlab-ci-main.yml
+30
View File
@@ -0,0 +1,30 @@
# From https://docs.gitlab.com/ee/ci/runners/hosted_runners/linux.html
#
# ...
#
# Runner Tag vCPUs Memory Storage
# saas-linux-small-amd64 2 8 GB 25 GB
#
# Our current private runner 'docker', 'samba-ci-private', 'shared' and
# 'ubuntu2204'. It runs with an ubuntu2204 kernel (5.15) and provides an
# ext4 filesystem, 2 CPU and 4 GB (shared tag) 8G (samba-ci-private tag) RAM.
#
.shared_runner_build:
# We use saas-linux-small-amd64 shared runners by default.
# We avoid adding explicit tags for them in order
# to work with potential changes in future
#
# In order to generate valid yaml, we define a dummy variable...
variables:
SAMBA_SHARED_RUNNER_BUILD_DUMMY_VARIABLE: shared_runner_build
.shared_runner_test:
# We use saas-linux-small-amd64 shared runners by default.
extends: .shared_runner_build
.private_runner_test:
# We use our private runner only for special tests
tags:
- docker
- samba-ci-private
+10
View File
@@ -0,0 +1,10 @@
variables:
SAMBA_CI_FLAVOR: "default"
# "--enable-coverage" or ""
# See .gitlab-ci-coverage.yml
SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE: ""
AUTOBUILD_SKIP_SAMBA_O3: "0"
include:
- /.gitlab-ci-default-runners.yml
- /.gitlab-ci-main.yml
+688
View File
@@ -0,0 +1,688 @@
# see https://docs.gitlab.com/ce/ci/yaml/README.html for all available options
# Stages explained
#
# images: Build the images with the bootstrap script
# build_first: Build a few things first to find silly errors (fast job)
# (don't pay for 35 machines until something compiles)
# build: The main parallel job
# (keep these to 1hour as we are billed per hour)
# test_only: Tests using the build from prior stages, these typically
# have an explicit dependency defined to a specific build job,
# which means that start as soon as the build job finished.
# test_private: Like test_only, but running on private runners
# report: Code coverage reporting
stages:
- images
- build_first
- build
- test_only
- test_private
- report
variables:
# We want to be resilient to runner failures
ARTIFACT_DOWNLOAD_ATTEMPTS: "3"
EXECUTOR_JOB_SECTION_ATTEMPTS: "3"
GET_SOURCES_ATTEMPTS: "3"
RESTORE_CACHE_ATTEMPTS: "3"
#
GIT_STRATEGY: fetch
GIT_DEPTH: "3"
#
# Use GZip by default, it is fast and is good enough. Other options include --xz
SAMBA_TESTBASE_TAR_OPTIONS: -z
#
# we run autobuild.py inside a samba CI docker image located on gitlab's registry
# overwrite this variable if you want use your own image registry.
#
# Or better ask for access to the shared development repository, see
# https://wiki.samba.org/index.php/Samba_CI_on_gitlab#Getting_Access
#
SAMBA_CI_CONTAINER_REGISTRY: registry.gitlab.com/samba-team/devel/samba
#
# Set this to the contents of bootstrap/sha1sum.txt
# which is generated by bootstrap/template.py --render
#
SAMBA_CI_CONTAINER_TAG: e494a8092a6d0e794223f56ddb2ffbaf76402cf6
#
# We use the ubuntu2204 image as default as
# it matches what we have on atb-devel-224
#
SAMBA_CI_CONTAINER_IMAGE: ubuntu2204
#
# The following images are available
# Please see the samba-o3 sections at the end of this file!
# We should run that for each available image
#
SAMBA_CI_CONTAINER_IMAGE_ubuntu2204: ubuntu2204
SAMBA_CI_CONTAINER_IMAGE_ubuntu2404: ubuntu2404
SAMBA_CI_CONTAINER_IMAGE_ubuntu2604: ubuntu2604
SAMBA_CI_CONTAINER_IMAGE_debian11: debian11
SAMBA_CI_CONTAINER_IMAGE_debian11_32bit: debian11-32bit
SAMBA_CI_CONTAINER_IMAGE_debian12: debian12
SAMBA_CI_CONTAINER_IMAGE_opensuse160: opensuse160
SAMBA_CI_CONTAINER_IMAGE_rocky8: rocky8
SAMBA_CI_CONTAINER_IMAGE_centos9s: centos9s
SAMBA_CI_CONTAINER_IMAGE_fedora43: fedora43
include:
# The image creation details are specified in a separate file
# See bootstrap/README.md for details
- 'bootstrap/.gitlab-ci.yml'
.shared_runner_build_image:
extends: .shared_runner_build
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE}
image: ${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-${SAMBA_CI_JOB_IMAGE}:${SAMBA_CI_CONTAINER_TAG}
.shared_template:
extends: .shared_runner_build_image
# All Samba jobs are interruptible, this avoids burning CPU when a
# newer branch is pushed.
interruptible: true
timeout: 2h
# Otherwise we run twice, once on push and once on MR
# https://forum.gitlab.com/t/new-rules-syntax-and-detached-pipelines/37292
rules:
- if: $CI_MERGE_REQUEST_ID
when: never
- when: on_success
variables:
AUTOBUILD_JOB_NAME: $CI_JOB_NAME
stage: build
cache:
key: ccache.${CI_JOB_NAME}.${SAMBA_CI_JOB_IMAGE}.${SAMBA_CI_FLAVOR}
paths:
- ccache
# This is overridden in many cases, but ensures none of the other
# main jobs start until and unless this build finishes. However
# this also ensures we do not download artifacts from any build
# unless we specifically depend on it, saving bandwidth
needs:
- job: samba-def-build
artifacts: false
before_script:
- uname -a
- ls -l /sys/module/
- ls -l /sys/kernel/security/
- if [ -e /sys/kernel/security/lsm ]; then cat /sys/kernel/security/lsm ; echo; fi
- if [ -e /proc/config.gz ]; then sudo zcat /proc/config.gz; echo; fi
- lsb_release -a
- cat /etc/os-release
- id
- cat /proc/self/status
- lscpu
- cat /proc/cpuinfo
- mount
- df -h
- cat /proc/swaps
- free -h
# ld will fail if coverage enabled, force link ld to ld.bfd
- if [ -n "$SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE" ]; then sudo ln -sf $(which ld.bfd) $(which ld); fi
# See bootstrap/.gitlab-ci.yml how to generate a new image
- echo "SAMBA_CI_CONTAINER_REGISTRY[${SAMBA_CI_CONTAINER_REGISTRY}]"
- echo "SAMBA_CI_CONTAINER_TAG[${SAMBA_CI_CONTAINER_TAG}]"
- echo "SAMBA_CI_JOB_IMAGE[${SAMBA_CI_JOB_IMAGE}]"
- echo "CI_JOB_IMAGE[${CI_JOB_IMAGE}]"
- bootstrap/template.py --sha1sum > /tmp/sha1sum-template.txt
- diff -u bootstrap/sha1sum.txt /tmp/sha1sum-template.txt
- echo "${SAMBA_CI_CONTAINER_TAG}" > /tmp/sha1sum-tag.txt
- diff -u bootstrap/sha1sum.txt /tmp/sha1sum-tag.txt
- diff -u bootstrap/sha1sum.txt /sha1sum.txt
- echo "${CI_COMMIT_SHA} ${CI_COMMIT_TITLE}" > /tmp/commit.txt
- export CCACHE_BASEDIR="${PWD}"
- export CCACHE_DIR="${PWD}/ccache" && mkdir -pv "$CCACHE_DIR"
- export CC="ccache cc"
- export CXX="ccache c++"
- ccache -z -M 500M
- ccache -s
# We are already running .gitlab-ci directives from this repo, remove additional checks that break our CI
- git config --global --add safe.directory '*'
after_script:
- mount
- df -h
- cat /proc/swaps
- free -h
- CCACHE_BASEDIR="${PWD}" CCACHE_DIR="${PWD}/ccache" ccache -s -c
artifacts:
expire_in: 1 week
paths:
- "*.stdout"
- "*.stderr"
- "*.info"
- public
- system-info.txt
retry:
max: 2
when:
- runner_system_failure
- stuck_or_timeout_failure
- api_failure
- runner_unsupported
- stale_schedule
- archived_failure
- scheduler_failure
- data_integrity_failure
script:
# gitlab predefines CI_JOB_NAME for each job. The gitlab job usually matches the
# autobuild name, which means we can define a default template that runs most autobuild jobs
- script/autobuild.py $AUTOBUILD_JOB_NAME $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --verbose --nocleanup --keeplogs --tail --full-testbase /builds/samba-testbase
# Ensure when adding a new job below that you also add it to
# the dependencies for 'pages' below for the code coverage page
# generation.
others:
extends: .shared_template
script:
- script/autobuild.py pidl $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --verbose --nocleanup --keeplogs --tail --full-testbase /builds/samba-testbase/pidl
- script/autobuild.py replace $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --verbose --nocleanup --keeplogs --tail --full-testbase /builds/samba-testbase/replace
- script/autobuild.py talloc $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --verbose --nocleanup --keeplogs --tail --full-testbase /builds/samba-testbase/talloc
- script/autobuild.py tdb $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --verbose --nocleanup --keeplogs --tail --full-testbase /builds/samba-testbase/tdb
- script/autobuild.py tevent $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --verbose --nocleanup --keeplogs --tail --full-testbase /builds/samba-testbase/tevent
- script/autobuild.py samba-xc $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --verbose --nocleanup --keeplogs --tail --full-testbase /builds/samba-testbase/samba-xc
- script/autobuild.py docs-xml $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --verbose --nocleanup --keeplogs --tail --full-testbase /builds/samba-testbase/docs-xml
- make -C coverity
.shared_template_build_only:
extends: .shared_template
timeout: 2h
needs:
artifacts:
expire_in: 1 week
paths:
- "*.stdout"
- "*.stderr"
- "*.info"
- system-info.txt
- samba-testbase.tar
script:
# gitlab predefines CI_JOB_NAME for each job. The gitlab job usually matches the
# autobuild name, which means we can define a default template that runs most autobuild jobs
- script/autobuild.py $AUTOBUILD_JOB_NAME $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --verbose --nocleanup --keeplogs --tail --full-testbase /builds/samba-testbase
# On success we need to pack everything into an artifacts file
# which needs to be in the git checkout.
# As tar doesn't handle hardlink of read-only files,
# we remember the acls and add write permissions
# before creating the archive. The consumer will apply
# the acls again.
- cp -a /sha1sum.txt /builds/samba-testbase/image-sha1sum.txt
- cp -a /tmp/commit.txt /builds/samba-testbase/commit.txt
- ln -s /builds/samba-testbase/${AUTOBUILD_JOB_NAME}/ /builds/samba-testbase/build_subdir_link
- pushd /builds && getfacl -R samba-testbase > samba-testbase.acl.dump && popd
- chmod -R +w /builds/samba-testbase
- mv /builds/samba-testbase.acl.dump /builds/samba-testbase/
- tar $SAMBA_TESTBASE_TAR_OPTIONS -cf samba-testbase.tar /builds/samba-testbase
- ls -la samba-testbase.tar
- sha1sum samba-testbase.tar
.shared_template_test_only:
extends:
- .shared_template
- .shared_runner_test
stage: test_only
script:
# Print the Kerberos version to check we ended up with the right one
# in the runner. We do not have configure output to recognize it
# otherwise.
- if [ -x "$(command -v krb5-config)" ]; then krb5-config --version; fi
# We unpack the artifacts file created by the .shared_template_build_only
# run we depend on
- ls -la samba-testbase.tar
- sha1sum samba-testbase.tar
- tar $SAMBA_TESTBASE_TAR_OPTIONS -xf samba-testbase.tar -C /
- diff -u /builds/samba-testbase/image-sha1sum.txt /sha1sum.txt
- diff -u /builds/samba-testbase/commit.txt /tmp/commit.txt
- mv /builds/samba-testbase/samba-testbase.acl.dump /builds/samba-testbase.acl.dump
- pushd /builds && setfacl --restore=/builds/samba-testbase.acl.dump && popd
- ls -la /builds/samba-testbase/
- ls -la /builds/samba-testbase/build_subdir_link
- ls -la /builds/samba-testbase/build_subdir_link/
- if [ -n "$SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE" ]; then find /builds/samba-testbase/build_subdir_link/ -type d -printf "'%p'\n" | xargs chmod u+w; fi
- ls -la /builds/samba-testbase/build_subdir_link/
# gitlab predefines CI_JOB_NAME for each job. The gitlab job usually matches the
# autobuild name, which means we can define a default template that runs most autobuild jobs
- script/autobuild.py $AUTOBUILD_JOB_NAME $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --skip-dependencies --verbose --nocleanup --keeplogs --tail --full-testbase /builds/samba-testbase
samba-def-build:
extends: .shared_template_build_only
stage: build_first
.needs_samba-def-build:
extends: .shared_template_test_only
needs:
- job: samba-def-build
artifacts: true
- job: samba-codecheck
samba-mit-build:
extends: .shared_template_build_only
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_fedora43}
stage: build_first
.needs_samba-mit-build:
extends: .shared_template_test_only
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_fedora43}
needs:
- job: samba-mit-build
artifacts: true
- job: samba-codecheck
samba-h5l-build:
extends: .shared_template_build_only
.needs_samba-h5l-build:
extends: .shared_template_test_only
needs:
- job: samba-h5l-build
artifacts: true
samba-without-smb1-build:
extends: .shared_template_build_only
.needs_samba-without-smb1-build:
extends: .shared_template_test_only
needs:
- job: samba-without-smb1-build
artifacts: true
samba-nt4-build:
extends: .shared_template_build_only
.needs_samba-nt4-build:
extends: .shared_template_test_only
needs:
- job: samba-nt4-build
artifacts: true
samba-no-opath-build:
extends: .shared_template_build_only
.needs_samba-no-opath-build:
extends: .shared_template_test_only
needs:
- job: samba-no-opath-build
artifacts: true
samba:
extends: .shared_template
samba-mitkrb5:
extends: .shared_template
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_fedora43}
samba-minimal-smbd:
extends: .shared_template
samba-nopython:
extends: .shared_template
samba-admem:
extends: .needs_samba-def-build
samba-ad-dc-2:
extends: .needs_samba-def-build
samba-ad-dc-3:
extends: .needs_samba-def-build
samba-ad-dc-4a:
extends: .needs_samba-def-build
samba-ad-dc-4b:
extends: .needs_samba-def-build
samba-ad-dc-5:
extends: .needs_samba-def-build
samba-ad-dc-6:
extends: .needs_samba-def-build
samba-ad-back1:
extends: .needs_samba-def-build
samba-ad-back2:
extends: .needs_samba-def-build
samba-schemaupgrade:
extends: .needs_samba-def-build
samba-libs:
extends: .shared_template
samba-fuzz:
extends: .shared_template
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_ubuntu2404}
ctdb:
extends: .shared_template
samba-ctdb:
extends: .shared_template
samba-ad-dc-ntvfs:
extends: .needs_samba-def-build
samba-admem-mit:
extends: .needs_samba-mit-build
samba-addc-mit-4a:
extends: .needs_samba-mit-build
samba-addc-mit-4b:
extends: .needs_samba-mit-build
# This task is run first to ensure we compile before we start the
# main run as it is the fastest full compile of Samba.
samba-fips:
extends: .shared_template
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_fedora43}
samba-codecheck:
extends: .shared_template
needs:
stage: build_first
.private_test_only:
extends: .private_runner_test
stage: test_private
rules:
# See above, to avoid a duplicate CI on the MR (these rules override the others)
- if: $CI_MERGE_REQUEST_ID
when: never
# These jobs are only run if the gitlab repo has private runners available.
# To enable private jobs, you must add the following var and value to
# your gitlab repo by navigating to:
# settings -> CI/CD -> Environment variables
- if: $SUPPORT_PRIVATE_TEST == "yes"
.needs_ext4_support:
# All runners provide an ext4 filesystem
#
# Note: we don't use
# extends: .shared_template_test_only
# as that somehow resets the needs section
# and generates problems for something
# like this (which is used below)
#
# .needs_samba-SOME-build-ext4:
# extends:
# - .needs_samba-SOME-build
# - .needs_ext4_support
#
# So we only set stage again instead...
stage: test_only
.needs_5_15_kernel:
# Our private runners are based on
# ubuntu2204 with a 5.15 kernel.
#
# And they also provide an ext4 filesystem
extends: .private_test_only
.needs_samba-def-build-ext4:
extends:
- .needs_samba-def-build
- .needs_ext4_support
.needs_samba-mit-build-ext4:
extends:
- .needs_samba-mit-build
- .needs_ext4_support
.needs_samba-h5l-build-ext4:
extends:
- .needs_samba-h5l-build
- .needs_ext4_support
.needs_samba-without-smb1-build-5_15:
# Currently this doesn't strictly
# require a kernel >= 5.15, but only
# ext4 support.
#
# But we want to make sure that
# our private runners keep working
# and at least do a single job.
#
# In future we'll be able to run
# tests with io_uring in this
# setup, which will requires a
# 5.15 kernel in order to be useful.
extends:
- .needs_samba-without-smb1-build
- .needs_5_15_kernel
.needs_samba-nt4-build-ext4:
extends:
- .needs_samba-nt4-build
- .needs_ext4_support
.needs_samba-no-opath-build-ext4:
extends:
- .needs_samba-no-opath-build
- .needs_ext4_support
samba-fileserver:
extends: .needs_samba-h5l-build-ext4
samba-fileserver-without-smb1:
extends: .needs_samba-without-smb1-build
# This is a full build without the AD DC so we test the build with MIT
# Kerberos from the default system (Ubuntu 22.04 at this stage).
# Runtime behaviour checked via the ktest (static ccache and keytab)
# environment
samba-ktest-mit:
extends: .shared_template
samba-ad-dc-1:
extends: .needs_samba-def-build-ext4
samba-nt4:
extends: .needs_samba-nt4-build-ext4
samba-addc-mit-1:
extends: .needs_samba-mit-build-ext4
samba-no-opath1:
extends: .needs_samba-no-opath-build-ext4
samba-no-opath2:
extends: .needs_samba-no-opath-build-ext4
# 'pages' is a special job which can publish artifacts in `public` dir to gitlab pages
pages:
extends: .shared_runner_build_image
stage: report
dependencies: # tell gitlab to download artifacts for these jobs
- others
- samba
- samba-mitkrb5
- samba-admem
- samba-ad-dc-2
- samba-ad-dc-3
- samba-ad-dc-4a
- samba-ad-dc-4b
- samba-ad-dc-5
- samba-ad-dc-6
- samba-libs
- samba-minimal-smbd
- samba-nopython
- samba-fuzz
# - ctdb # TODO
- samba-ctdb
- samba-ad-dc-ntvfs
- samba-admem-mit
- samba-addc-mit-4a
- samba-addc-mit-4b
- samba-ad-back1
- samba-ad-back2
- samba-fileserver
- samba-fileserver-without-smb1
- samba-ad-dc-1
- samba-nt4
- samba-schemaupgrade
- samba-addc-mit-1
- samba-fips
- samba-no-opath1
- samba-no-opath2
- ubuntu2204-samba-o3
script:
- ls -la *.info
- ./configure.developer
- make -j
- ls -la *.info
- lcov $(ls *.info | xargs -I{} echo -n "-a {} ") -o all.info
- ls -la *.info
- genhtml all.info --ignore-errors source --output-directory public --prefix=$(pwd) --title "coverage report for $CI_COMMIT_REF_NAME $CI_COMMIT_SHORT_SHA"
artifacts:
expire_in: 30 days
paths:
- public
only:
variables:
- $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE == "--enable-coverage"
# Coverity Scan
coverity:
extends: .shared_runner_build_image
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_opensuse160}
stage: build
script:
- wget https://scan.coverity.com/download/linux64 --post-data "token=$COVERITY_SCAN_TOKEN&project=$COVERITY_SCAN_PROJECT_NAME" -O /tmp/coverity_tool.tgz
- tar xf /tmp/coverity_tool.tgz
- ./configure.developer --with-cluster-support
- cov-analysis-linux64-*/bin/coverity capture --dir cov-int --project-dir ./
- tar czf cov-int.tar.gz cov-int
- curl
--form token=$COVERITY_SCAN_TOKEN
--form email=$COVERITY_SCAN_EMAIL
--form file=@cov-int.tar.gz
--form version="`git describe --tags`"
--form description="CI build"
https://scan.coverity.com/builds?project=$COVERITY_SCAN_PROJECT_NAME
only:
refs:
- master
- schedules
variables:
- $COVERITY_SCAN_TOKEN != null
- $COVERITY_SCAN_PROJECT_NAME != null
- $COVERITY_SCAN_EMAIL != null
artifacts:
expire_in: 1 week
when: on_failure
paths:
- cov-int/*.txt
debian11-samba-32bit:
extends: .shared_template
variables:
AUTOBUILD_JOB_NAME: samba-32bit
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_debian11_32bit}
#
# We build samba-o3 on all supported distributions
#
# This job, which matches the main CI, needs to still do coverage so
# we show the coverage on the "none" environment tests
#
# We want --enable-coverage specified here otherwise we will have a
# different set of build options on the coverage build and can fail
# when -O3 gets combined with --enable-coverage in the scheduled
# builds.
ubuntu2204-samba-o3:
extends: .shared_template
variables:
AUTOBUILD_JOB_NAME: samba-o3
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_ubuntu2204}
SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE: "--enable-coverage"
rules:
# See above, to avoid a duplicate CI on the MR (these rules override the others)
- if: $CI_MERGE_REQUEST_ID
when: never
# do not run o3 builds (which run a lot of VMs) if told not to
# (this uses the same variable as autobuild.py)
- if: $AUTOBUILD_SKIP_SAMBA_O3 == "1"
when: never
- when: on_success
# All other jobs do not want code coverage.
.samba-o3-template:
extends: .shared_template
variables:
AUTOBUILD_JOB_NAME: samba-o3
rules:
# See above, to avoid a duplicate CI on the MR (these rules override the others)
- if: $CI_MERGE_REQUEST_ID
when: never
# do not run o3 builds (which run a lot of VMs) if told not to
# (this uses the same variable as autobuild.py)
- if: $AUTOBUILD_SKIP_SAMBA_O3 == "1"
when: never
# do not run o3 for coverage since they are using different images
- if: $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE == ""
ubuntu2404-samba-o3:
extends: .samba-o3-template
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_ubuntu2404}
ubuntu2604-samba-o3:
extends: .samba-o3-template
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_ubuntu2604}
debian11-samba-o3:
extends: .samba-o3-template
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_debian11}
debian12-samba-o3:
extends: .samba-o3-template
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_debian12}
opensuse160-samba-o3:
extends: .samba-o3-template
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_opensuse160}
rocky8-samba-o3:
extends: .samba-o3-template
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_rocky8}
centos9s-samba-o3:
extends: .samba-o3-template
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_centos9s}
fedora43-samba-o3:
extends: .samba-o3-template
variables:
SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_fedora43}
#
# Keep the samba-o3 sections at the end ...
#
+5
View File
@@ -0,0 +1,5 @@
# This is just a legacy alias used by the
# https://gitlab.com/samba-team/devel/samba configuration
#
include:
- '/.gitlab-ci.yml'
+2
View File
@@ -0,0 +1,2 @@
include:
- /.gitlab-ci-default.yml
+16
View File
@@ -0,0 +1,16 @@
#
# GitLeaks Repo Specific Configuration
#
# This allowlist is used to help Red Hat ignore false positives during its code
# scans.
[allowlist]
paths = [
'''docs-xml/manpages/smbstatus.1.xml''',
'''selftests/*''',
'''source3/script/tests/*''',
'''source4/dsdb/tests/*''',
'''source4/torture/*''',
'''testprogs/blackbox/*''',
'''tests/*''',
]
+122
View File
@@ -0,0 +1,122 @@
---
.build_image_template:
image: quay.io/podman/stable:latest
stage: images
tags:
# We need to make sure we only use gitlab.com
# runners and not our own runners, as our current runners
# don't allow 'docker build ...' to run.
- saas-linux-small-amd64
variables:
SAMBA_CI_IS_BROKEN_IMAGE: "no"
SAMBA_CI_TEST_JOB: "samba-o3"
SAMBA_CI_PLATFORM: "linux/amd64"
before_script:
# install prerequisites
- dnf install -qy diffutils
# Ensure we are generating correct the container
- uname -a
- cat /etc/os-release
- echo "SAMBA_CI_CONTAINER_REGISTRY[${SAMBA_CI_CONTAINER_REGISTRY}]"
- echo "SAMBA_CI_CONTAINER_TAG[${SAMBA_CI_CONTAINER_TAG}]"
- echo "SAMBA_CI_IS_BROKEN_IMAGE[${SAMBA_CI_IS_BROKEN_IMAGE}]"
- echo "SAMBA_CI_REBUILD_IMAGES[${SAMBA_CI_REBUILD_IMAGES}]"
- echo "SAMBA_CI_REBUILD_BROKEN_IMAGES[${SAMBA_CI_REBUILD_BROKEN_IMAGES}]"
- echo "GITLAB_USER_LOGIN[${GITLAB_USER_LOGIN}]"
- echo "${SAMBA_CI_CONTAINER_TAG}" > /tmp/sha1sum-tag.txt
- diff -u bootstrap/sha1sum.txt /tmp/sha1sum-tag.txt
script: |
set -xueo pipefail
ci_image_name=samba-ci-${CI_JOB_NAME}
podman build --platform ${SAMBA_CI_PLATFORM} --tag ${ci_image_name} --build-arg SHA1SUM=${SAMBA_CI_CONTAINER_TAG} bootstrap/generated-dists/${CI_JOB_NAME}
ci_image_path="${SAMBA_CI_CONTAINER_REGISTRY}/${ci_image_name}"
timestamp=$(date +%Y%m%d%H%M%S)
container_hash=$(podman image inspect --format='{{ .Id }}' ${ci_image_name} | cut -c 1-9)
timestamp_tag=${SAMBA_CI_CONTAINER_TAG}-${timestamp}-${GITLAB_USER_LOGIN}-${container_hash}
samba_repo_root=/home/samba/samba
# Ensure we are generating the correct container that we expect to be in
echo "${SAMBA_CI_CONTAINER_TAG}" > /tmp/sha1sum-tag.txt
diff -u bootstrap/sha1sum.txt /tmp/sha1sum-tag.txt
podman run --volume $(pwd):${samba_repo_root} --workdir ${samba_repo_root} ${ci_image_name} \
/bin/bash -c "echo \"${SAMBA_CI_CONTAINER_TAG}\" > /tmp/sha1sum-tag.txt; diff -u bootstrap/sha1sum.txt /tmp/sha1sum-tag.txt"
podman run --volume $(pwd):${samba_repo_root} --workdir ${samba_repo_root} ${ci_image_name} \
diff -u bootstrap/sha1sum.txt /sha1sum.txt
podman run --volume $(pwd):${samba_repo_root} --workdir ${samba_repo_root} ${ci_image_name} \
bootstrap/template.py --sha1sum > /tmp/sha1sum-template.txt
diff -u bootstrap/sha1sum.txt /tmp/sha1sum-template.txt
# run smoke test with samba-o3 or samba-fuzz
podman run --volume $(pwd):/src:ro ${ci_image_name} \
/bin/bash -c "git config --global --add safe.directory /src/.git && git clone /src samba && cd samba && export PKG_CONFIG_PATH=/usr/lib64/compat-gnutls34/pkgconfig:/usr/lib64/compat-nettle32/pkgconfig && script/autobuild.py ${SAMBA_CI_TEST_JOB} --verbose --nocleanup --keeplogs --tail --testbase /tmp/samba-testbase"
podman tag ${ci_image_name} ${ci_image_path}:${SAMBA_CI_CONTAINER_TAG}
podman tag ${ci_image_name} ${ci_image_path}:${timestamp_tag}
# We build all images, but only upload is it's not marked as broken
test x"${SAMBA_CI_IS_BROKEN_IMAGE}" = x"yes" || { \
podman login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY; \
podman push ${ci_image_path}:${SAMBA_CI_CONTAINER_TAG}; \
podman push ${ci_image_path}:${timestamp_tag}; \
}
echo "Success for ${ci_image_path}:${timestamp_tag}"
test x"${SAMBA_CI_IS_BROKEN_IMAGE}" = x"no" || { \
echo "The image ${CI_JOB_NAME} is marked as broken and should have failed!"; \
echo "Replace .build_image_template_force_broken with .build_image_template!"; \
echo "Add a .samba-o3-template section at the end of the main .gitlab-ci.yml!"; \
/bin/false; \
}
only:
variables:
#
# You need a custom pipeline which passes
# SAMBA_CI_REBUILD_IMAGES="yes".
#
# https://gitlab.com/samba-team/devel/samba/pipelines/new
#
- $SAMBA_CI_REBUILD_IMAGES == "yes"
.build_image_template_force_broken:
extends: .build_image_template
variables:
SAMBA_CI_IS_BROKEN_IMAGE: "yes"
only:
variables:
#
# You need a custom pipeline which passes
# SAMBA_CI_REBUILD_BROKEN_IMAGES="yes"
# in order to build broken images for debugging
#
# https://gitlab.com/samba-team/devel/samba/pipelines/new
#
- $SAMBA_CI_REBUILD_BROKEN_IMAGES == "yes"
ubuntu2204:
extends: .build_image_template
ubuntu2404:
extends: .build_image_template
ubuntu2604:
extends: .build_image_template
debian11:
extends: .build_image_template
debian12:
extends: .build_image_template
fedora43:
extends: .build_image_template
debian11-32bit:
extends: .build_image_template
variables:
SAMBA_CI_TEST_JOB: "samba-32bit"
SAMBA_CI_PLATFORM: "linux/i386"
rocky8:
extends: .build_image_template
centos9s:
extends: .build_image_template
opensuse160:
extends: .build_image_template
+87
View File
@@ -0,0 +1,87 @@
---
# Exercises multi-line script patterns and extended variable declarations.
# Ref: https://docs.gitlab.com/ci/yaml/script/#split-long-commands
# Ref: https://docs.gitlab.com/ee/ci/yaml/#variablesdescription
# All patterns here must parse cleanly (exit 0).
stages:
- build
- test
- deploy
# Pipeline-level variables: plain strings and extended {value, description} map form.
variables:
PLAIN_VAR: "hello"
DEPLOY_ENV:
value: "staging"
description: "The deployment target. Set to staging or production."
RETRIES:
value: "3"
description: "Number of retry attempts."
options:
- "1"
- "3"
- "5"
default:
# image in map form (name + pull_policy)
image:
name: alpine:latest
pull_policy: if-not-present
# before_script as a block scalar (not a list)
before_script:
- apk add --no-cache curl git
build-literal-block:
stage: build
# script items using literal block scalar (|)
script:
- |
if [[ "$DEPLOY_ENV" == "production" ]]; then
echo "Production build"
else
echo "Non-production build"
fi
- echo "Build step done"
build-folded-block:
stage: build
# script items using folded block scalar (>)
script:
- >
apt-get update -qq &&
apt-get install -y curl wget
- echo "Packages installed"
before_script:
- |
echo "Job-level before_script"
echo "Using literal block scalar"
test-job:
stage: test
script:
- echo "Running tests"
- |
set -e
go test ./...
echo "Tests passed"
# Job-level variable with extended form
variables:
TEST_FLAG:
value: "true"
description: "Enable verbose test output"
# rules.changes in map form (GitLab 15.3+)
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
paths:
- "**/*.go"
compare_to: "main"
when: on_success
- when: on_success
deploy-job:
stage: deploy
script:
- echo "Deploying to $DEPLOY_ENV"
when: manual
+51
View File
@@ -0,0 +1,51 @@
---
# variable_refs.yml
# Verifies that GL032 does not fire for variables that are declared or predefined.
# Covers: pipeline variables:, job variables:, workflow:rules:variables:, and CI_* prefixes.
# Expected: exits 0 (no errors; no GL032 warnings).
stages:
- build
- deploy
variables:
DEPLOY_TARGET: "staging"
FEATURE_FLAG: "false"
workflow:
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
variables:
DEPLOY_TARGET: "production"
- when: always
build:
stage: build
script: make build
rules:
# pipeline-level variable — GL032 must not fire
- if: '$DEPLOY_TARGET == "staging"'
when: on_success
# predefined CI variable — GL032 must not fire
- if: '$CI_COMMIT_BRANCH =~ /^feat\/.+/'
when: manual
deploy:
stage: deploy
script: make deploy
variables:
DEPLOY_REGION: "us-east-1"
rules:
# pipeline-level variable — no warning
- if: '$FEATURE_FLAG == "true"'
when: never
# workflow-rule-injected variable — no warning
- if: '$DEPLOY_TARGET == "production"'
when: on_success
# job-level variable — no warning
- if: '$DEPLOY_REGION == "us-east-1"'
when: on_success
# predefined GITLAB_ variable — no warning
- if: '$GITLAB_USER_LOGIN == "bot"'
when: never
- when: manual
+25
View File
@@ -0,0 +1,25 @@
---
# variable_refs_included.yml
# GL032 must NOT fire for variables declared in an included file.
# The included file (variable_refs_included_template.yml) declares TEMPLATE_VAR.
# Expected: exits 0 (no errors; no GL032 warnings).
stages:
- build
include:
- local: /variable_refs_included_template.yml
build:
stage: build
script: make build
rules:
# TEMPLATE_VAR comes from the included file — GL032 must not fire.
- if: '$TEMPLATE_VAR == "enabled"'
when: on_success
# ROOT_VAR is declared in this file — GL032 must not fire.
- if: '$ROOT_VAR == "yes"'
when: manual
variables:
ROOT_VAR: "yes"
+6
View File
@@ -0,0 +1,6 @@
---
# Included by variable_refs_included.yml — declares a pipeline-level variable
# that the parent pipeline's jobs reference in rules:if: expressions.
variables:
TEMPLATE_VAR: "enabled"
+51
View File
@@ -0,0 +1,51 @@
---
# workflow_vars.yml
# Exercises workflow:rules:variables: injection.
# The matching workflow rule sets DEPLOY_TARGET; job rules use it.
#
# Expected behaviour per context:
# (no context) → all jobs active (no context evaluation)
# --branch main → deploy-prod active, deploy-staging skipped
# --branch develop → deploy-prod skipped, deploy-staging active
# --branch feat/x → deploy-prod skipped, deploy-staging skipped (manual)
stages:
- build
- deploy
variables:
DEPLOY_TARGET:
value: ""
description: "Deployment target — set by workflow rules"
workflow:
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
variables:
DEPLOY_TARGET: production
- if: '$CI_COMMIT_BRANCH == "develop"'
variables:
DEPLOY_TARGET: staging
- when: always
build:
stage: build
script: make build
rules:
- when: always
deploy-prod:
stage: deploy
script: make deploy ENV=production
rules:
- if: '$DEPLOY_TARGET == "production"'
when: on_success
- when: never
deploy-staging:
stage: deploy
script: make deploy ENV=staging
rules:
- if: '$DEPLOY_TARGET == "staging"'
when: on_success
- when: never