12 Commits

Author SHA1 Message Date
k3nny a0e2582cf1 fix(linter): support bare true/false and integer literals in rules:if:
release / Build and publish release (push) Successful in 1m15s
GitLab CI expressions allow unquoted true, false, and integers as
comparison operands (all treated as their string representations):

  $GATEWAY_ENABLED == true    (equivalent to == "true")
  $FEATURE_FLAG == false      (equivalent to == "false")
  $PARALLEL == 4              (equivalent to == "4")
  $ENABLED == 1 / == 0

Previously these fell through to permissive true because parseValue
only recognised $VAR, "${VAR}", quoted strings, and null. Added:
  - true/false keyword branch → returns "true"/"false"
  - integer literal branch (digits only) → returns decimal string

All three new forms are correctly excluded from longer identifier
prefixes (identByte boundary check). Adds 8 new unit tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 22:20:59 +02:00
k3nny e931b9d1c9 fix(linter): improve rules:if: expression evaluator
Four correctness fixes to the GitLab CI expression parser in
internal/cicontext/eval.go:

- Multi-line: \n and \r are now treated as whitespace in skipWS so
  block-scalar or folded-scalar if: values with || / && on continuation
  lines evaluate correctly instead of falling back to permissive true.
- ${VAR} curly-brace variable syntax now supported in parseValue.
- Regex flags (/pattern/i, /pattern/m, /pattern/s) are now consumed and
  translated to Go (?i)/(?m)/(?s) prefixes via applyRegexFlags.
- Variable on RHS of =~ / !~: when the right operand is $VAR, the
  variable's value is interpreted as a /regex/[flags] string via
  extractRegexFromString; non-regex values fall back to permissive true.

Adds 16 new unit tests covering all four cases and a testdata fixture
(rules_if_expr.yml) exercising multi-line, ${VAR}, and /pattern/i in a
real pipeline with context flags.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 22:08:08 +02:00
k3nny b21a7d60dc feat(resolver,graph): fetch and resolve include: remote: HTTPS URLs
release / Build and publish release (push) Successful in 1m14s
Remote includes (include: remote: https://...) were previously skipped
silently in the resolver and rendered as unexpanded leaf nodes in the
graph.

Changes:
- fetcher.FetchURL: new shared unauthenticated HTTP GET helper
- resolver: resolveRemoteInclude fetches the URL, parses YAML, sets job
  origin to the URL string, recursively resolves sub-includes, and emits
  a warning on failure (lint continues on the rest of the pipeline)
- graph: recurseRemote fetches the URL, captures direct job names, and
  recurses into sub-includes so remote nodes expand like local ones

Adds testdata/includes_remote.yml fixture.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 21:42:15 +02:00
k3nny d34c39927d fix(linter): downgrade needs optional:true missing-job to warning
release / Build and publish release (push) Successful in 1m12s
parseNeedJobNames is replaced by parseNeedEntries which preserves the
optional flag from each needs: entry. When a referenced job does not
exist and optional:true is set, the finding is now WARNING instead of
ERROR, matching GitLab CI runtime behavior (the dependency is silently
skipped when the job is absent from a conditional include).

Optional missing deps are also excluded from the cycle-detection graph
since there is no real dependency edge to trace.

Adds a fixture case in testdata/needs.yml to prevent regression.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 21:27:16 +02:00
k3nny a303f63a5e feat(linter): add file/line to findings; downgrade extends missing-script to warning
Every finding now carries the source file and exact line number of the job
key in its YAML file. Format: [ERROR] job "name" (file.yml:12): message.

Pipeline-level findings (workflow rules, no stages) reference p.SourceFile.
Cross-file include jobs (local, project, component) carry the include source
as their File, set via Pipeline.SetJobOrigin after each ParseBytes call in
the resolver.

Line numbers come from the yaml.Node key node (exact job-name line) in a
new document-level first pass in ParseBytes, replacing the previous
map[string]yaml.Node approach which only gave value-node lines.

Also: jobs that declare extends: but have no script after resolution now
emit WARNING instead of ERROR. The script may come from a base in a remote
include that was not fetched (no token, offline), making the error a false
positive in common project setups.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 21:24:18 +02:00
k3nny a962c996c1 feat(graph): show jobs per file in include dependency graph
Each node in 'glint graph includes' now lists the jobs defined directly
in that file. Jobs appear as rounded Mermaid nodes with a distinct
light-purple style, connected with dashed arrows (-.->). This visual
distinction separates ownership (file -.-> job) from the include
hierarchy (file --> included-file).

The root file's jobs are collected by re-parsing it without include
resolution; local and fetched project/component nodes populate their
job list in the existing recurse* methods.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 21:03:50 +02:00
k3nny e5f926b55f docs: 📝 add logo
release / Build and publish release (push) Successful in 1m47s
2026-06-11 20:40:39 +02:00
k3nny 8c3ce050f5 Merge pull request 'fix(model): handle YAML map forms that caused unmarshall errors' (#3) from fix/unmarshall_errors into main
Reviewed-on: #3
2026-06-11 20:31:30 +02:00
k3nny c4ab64391d fix(model): handle YAML map forms that caused unmarshall errors
Variables with value/description/options sub-keys, default.image in map
form, default.before_script / default.after_script as block scalars, and
rules.changes / rules.exists in {paths, compare_to} map form all caused
"yaml: cannot unmarshal !!map into string" because the struct fields were
typed too narrowly.

Changed types in model.Pipeline, model.DefaultConfig, and model.Rule to
accept any to match GitLab CI spec flexibility (13.7+ variable declarations,
15.3+ rules.changes map form, image map form in default block).

Adds testdata/script_multiline.yml covering all these patterns.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 20:25:53 +02:00
k3nny 4cc50afb5f fix(build): replace actions/checkout@v4 with plain git clone
release / Build and publish release (push) Successful in 1m3s
actions/checkout@v4 requires Node.js which is absent in golang:alpine.
Clone the repo directly with git using an oauth2 token in the URL,
removing the Node.js dependency entirely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 01:04:27 +02:00
k3nny dfbafd8ed3 chore(build): use golang:1.26-alpine container instead of ubuntu-latest
release / Build and publish release (push) Failing after 41s
Switch from the full ubuntu runner image to golang:1.26-alpine via the
container: directive. Go is pre-installed so actions/setup-go is dropped;
curl, git, and jq are added with apk in the first step.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 01:00:30 +02:00
k3nny 58cfbb4a57 fix(build): replace upload-artifact@v4 with direct Gitea API upload
upload-artifact@v4 uses a backend API incompatible with Gitea (GHES).
Collapse to a single job that builds both targets sequentially and uploads
directly to a Gitea release via the REST API, removing the need for
artifact passing between jobs entirely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 00:59:09 +02:00
19 changed files with 715 additions and 112 deletions
+31 -38
View File
@@ -6,52 +6,45 @@ on:
- 'v*' - 'v*'
jobs: jobs:
build: release:
name: Build (${{ matrix.goos }}/${{ matrix.goarch }}) name: Build and publish release
runs-on: ubuntu-latest runs-on: ubuntu-latest
strategy: container:
matrix: image: golang:1.26-alpine
include:
- goos: linux
goarch: amd64
suffix: -linux-amd64
- goos: windows
goarch: amd64
suffix: .exe
steps: steps:
- uses: actions/checkout@v4 - name: Install tools
run: apk add --no-cache curl git jq
- uses: actions/setup-go@v5 - name: Checkout
with:
go-version-file: go.mod
- name: Build
env: env:
GOOS: ${{ matrix.goos }} TOKEN: ${{ secrets.GITHUB_TOKEN }}
GOARCH: ${{ matrix.goarch }} SERVER_URL: ${{ github.server_url }}
REPO: ${{ github.repository }}
REF: ${{ github.ref_name }}
run: |
git clone --depth 1 --branch "$REF" \
"$(echo "$SERVER_URL" | sed "s|https://|https://oauth2:${TOKEN}@|")/${REPO}.git" .
- name: Build Linux (amd64)
env:
GOOS: linux
GOARCH: amd64
CGO_ENABLED: "0" CGO_ENABLED: "0"
run: | run: |
go build -trimpath -ldflags="-s -w" \ go build -trimpath -ldflags="-s -w" \
-o glint-${{ github.ref_name }}${{ matrix.suffix }} \ -o glint-${{ github.ref_name }}-linux-amd64 \
./cmd/glint/... ./cmd/glint/...
- uses: actions/upload-artifact@v4 - name: Build Windows (amd64)
with: env:
name: binary-${{ matrix.goos }}-${{ matrix.goarch }} GOOS: windows
path: glint-${{ github.ref_name }}${{ matrix.suffix }} GOARCH: amd64
retention-days: 7 CGO_ENABLED: "0"
run: |
release: go build -trimpath -ldflags="-s -w" \
name: Publish release -o glint-${{ github.ref_name }}.exe \
needs: build ./cmd/glint/...
runs-on: ubuntu-latest
steps:
- uses: actions/download-artifact@v4
with:
pattern: binary-*
merge-multiple: true
- name: Create release and upload assets - name: Create release and upload assets
env: env:
@@ -67,11 +60,11 @@ jobs:
-d "{\"tag_name\":\"$TAG\",\"name\":\"$TAG\",\"draft\":false,\"prerelease\":false}" \ -d "{\"tag_name\":\"$TAG\",\"name\":\"$TAG\",\"draft\":false,\"prerelease\":false}" \
| jq -r .id) | jq -r .id)
for file in glint-*; do for file in glint-${{ github.ref_name }}-linux-amd64 glint-${{ github.ref_name }}.exe; do
curl -sf -X POST \ curl -sf -X POST \
-H "Authorization: token $TOKEN" \ -H "Authorization: token $TOKEN" \
-H "Content-Type: application/octet-stream" \ -H "Content-Type: application/octet-stream" \
"$API_URL/repos/$REPO/releases/$release_id/assets?name=$(basename "$file")" \ "$API_URL/repos/$REPO/releases/$release_id/assets?name=$file" \
--data-binary "@$file" --data-binary "@$file"
echo "uploaded: $file" echo "uploaded: $file"
done done
+27
View File
@@ -7,6 +7,33 @@ This project uses [Semantic Versioning](https://semver.org).
## [Unreleased] ## [Unreleased]
### Added
- **`rules:if:` expression evaluator improvements** — six correctness fixes to the GitLab CI expression parser:
- **Multi-line expressions** — newlines (`\n`, `\r`) are now treated as whitespace between tokens, so block-scalar `if:` values (e.g. `if: | ...`) and folded YAML scalars with `||`/`&&` on a continuation line are parsed correctly instead of falling back to permissive `true`.
- **`${VAR}` curly-brace variable syntax** — `${CI_COMMIT_BRANCH}` is now equivalent to `$CI_COMMIT_BRANCH` everywhere a value is expected.
- **Regex flags** — `/pattern/i`, `/pattern/m`, `/pattern/s` are now honoured; the `i` flag (case-insensitive) is translated to Go's `(?i)` prefix before compiling. Unknown flags are silently ignored.
- **Variable as regex RHS** — `$BRANCH =~ $PATTERN` where `$PATTERN` holds a `/regex/[flags]` string is now evaluated by extracting and compiling the pattern from the variable's value; if the value is empty or does not look like a regex literal the expression falls back to permissive `true`.
- **`true` / `false` keywords** — bare `true` and `false` (without quotes) are now recognised as the string values `"true"` and `"false"`, matching GitLab CI's own behaviour. `$GATEWAY_ENABLED == true` and `$FEATURE_FLAG == false` now evaluate correctly.
- **Integer literals** — bare integers (e.g. `$PARALLEL == 4`, `$ENABLED == 1`, `$DISABLED == 0`) are now parsed as their decimal string representations and compared accordingly.
- **File and line numbers on findings** — every finding now includes the source file and line where the job is defined, e.g. `[ERROR] job "deploy" (src/deploy.yml:14): …`. For jobs that come from local or fetched includes the file reflects the include source. Pipeline-level findings (workflow rules, missing stages) reference the root pipeline file.
- **`glint graph includes` shows jobs per file** — each node in the Mermaid include dependency graph now shows the jobs defined directly in that file. Jobs are rendered as rounded nodes (`(name)`) in a distinct light-purple style, connected with dashed arrows (`-.->`) to distinguish ownership from the include hierarchy (solid `-->` arrows). The root pipeline file always shows its direct jobs; local and fetched project/component nodes show theirs when the file can be read.
### Fixed
- **`include: remote:` URL includes are now fetched and merged** — glint fetches plain HTTPS URLs in `include: remote:` entries (no authentication), parses the resulting YAML, merges its jobs into the pipeline, and recursively resolves any sub-includes the remote file itself declares. Unreachable or unparseable URLs emit a `[WARNING]` and lint continues on the rest of the pipeline. The `glint graph includes` command now expands remote nodes with their jobs and sub-include tree, matching the behaviour of local and project includes.
- **`needs: optional: true` downgraded to warning** — a `needs:` entry that carries `optional: true` and references a job not present in the pipeline now emits `[WARNING]` instead of `[ERROR]`. GitLab CI silently skips such dependencies at runtime (the job is absent when its include was not triggered), so the finding was a false positive. Non-optional missing needs remain errors. Optional missing deps are also excluded from the cycle-detection graph.
- **`extends:` jobs with missing script downgraded to warning** — a job that declares `extends:` but has no `script` after resolution now emits `[WARNING]` instead of `[ERROR]`. The script may legitimately come from a base job in a remote include that could not be fetched at lint time (e.g. no token configured).
- **Variable map form now parses correctly** — `variables:` entries that use the extended `{value, description, options}` form (GitLab CI 13.7+) no longer cause `yaml: cannot unmarshal !!map into string`. Both `Pipeline.Variables` and per-job `Variables` now accept either plain strings or map-form declarations.
- **`default.image` map form now parses correctly** — `default: image: {name: ..., pull_policy: ...}` used to cause `yaml: cannot unmarshal !!map into string`; `DefaultConfig.Image` is now typed as `any` to match `Job.Image`.
- **`default.before_script` / `default.after_script` now accept both list and scalar forms** — previously `DefaultConfig.BeforeScript` and `DefaultConfig.AfterScript` were `[]string`, causing a parse error when the field was written as a block scalar string. They are now typed as `any` to match the corresponding `Job` fields.
- **`rules.changes` / `rules.exists` map form now parses correctly** — extended `changes: {paths: [...], compare_to: "..."}` syntax (GitLab CI 15.3+) used to cause `yaml: cannot unmarshal !!map into []string`.
## [0.2.0] - 2026-06-11 ## [0.2.0] - 2026-06-11
### Added ### Added
+10 -3
View File
@@ -1,7 +1,13 @@
# glint <p align="center">
<img src="assets/glint-logo.png" alt="glint logo" width="220" />
</p>
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE) <h1 align="center">glint</h1>
[![Release](https://img.shields.io/badge/release-v0.2.0-blue.svg)](CHANGELOG.md)
<p align="center">
<a href="LICENSE"><img src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" alt="License"></a>
<a href="CHANGELOG.md"><img src="https://img.shields.io/badge/release-v0.2.0-blue.svg" alt="Release"></a>
</p>
> **Disclaimer:** This tool was built through iterative AI-assisted development with [Claude](https://claude.ai). It is experimental, incomplete, and not intended for production use. Coverage of GitLab CI keywords is best-effort and may lag behind GitLab's evolving spec. Use it at your own discretion — no correctness guarantees are made. Contributions and bug reports are welcome. > **Disclaimer:** This tool was built through iterative AI-assisted development with [Claude](https://claude.ai). It is experimental, incomplete, and not intended for production use. Coverage of GitLab CI keywords is best-effort and may lag behind GitLab's evolving spec. Use it at your own discretion — no correctness guarantees are made. Contributions and bug reports are welcome.
@@ -19,6 +25,7 @@ A local tool to validate and lint `.gitlab-ci.yml` pipelines without needing a G
- **CI/CD catalog components** — resolves `include: component:` references from the GitLab CI/CD Catalog; public components work without a token - **CI/CD catalog components** — resolves `include: component:` references from the GitLab CI/CD Catalog; public components work without a token
- **Deprecation warnings** — flags `only`/`except` usage in favour of `rules` - **Deprecation warnings** — flags `only`/`except` usage in favour of `rules`
- **Local include resolution** — `include: local:` entries are read from disk and recursively merged before linting, so multi-file pipelines are fully validated - **Local include resolution** — `include: local:` entries are read from disk and recursively merged before linting, so multi-file pipelines are fully validated
- **Extended variable declarations** — `variables:` entries may use the `{value, description, options}` map form (GitLab CI 13.7+); `default.image` accepts both string and map form; `rules.changes`/`rules.exists` accept both list and `{paths, compare_to}` map form
- **Graph output** — `glint graph` prints a job tree (stages → jobs) to the terminal; `glint graph includes` emits a Mermaid include dependency diagram; `glint graph pipeline` renders a GitLab CI-style PNG/SVG - **Graph output** — `glint graph` prints a job tree (stages → jobs) to the terminal; `glint graph includes` emits a Mermaid include dependency diagram; `glint graph pipeline` renders a GitLab CI-style PNG/SVG
- **Context simulation** — pass `--branch`, `--tag`, or `--source` to `glint check` or `glint graph` to see which jobs would be active, manual, or skipped for a specific pipeline event; evaluates `rules:if:` expressions and `only`/`except` filters - **Context simulation** — pass `--branch`, `--tag`, or `--source` to `glint check` or `glint graph` to see which jobs would be active, manual, or skipped for a specific pipeline event; evaluates `rules:if:` expressions and `only`/`except` filters
+10
View File
@@ -41,10 +41,14 @@ tasks:
ignore_error: true ignore_error: true
- cmd: ./{{.BINARY}} check testdata/keywords_invalid.yml - cmd: ./{{.BINARY}} check testdata/keywords_invalid.yml
ignore_error: true ignore_error: true
- cmd: ./{{.BINARY}} check testdata/includes_remote.yml
ignore_error: false
- cmd: ./{{.BINARY}} check testdata/includes_project.yml - cmd: ./{{.BINARY}} check testdata/includes_project.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} check testdata/includes_component.yml - cmd: ./{{.BINARY}} check testdata/includes_component.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} check testdata/script_multiline.yml
ignore_error: false
- cmd: ./{{.BINARY}} check testdata/context_rules.yml - cmd: ./{{.BINARY}} check testdata/context_rules.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} check --branch main testdata/context_rules.yml - cmd: ./{{.BINARY}} check --branch main testdata/context_rules.yml
@@ -55,6 +59,12 @@ tasks:
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} check --tag v1.0.0 testdata/context_rules.yml - cmd: ./{{.BINARY}} check --tag v1.0.0 testdata/context_rules.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} check testdata/rules_if_expr.yml
ignore_error: false
- cmd: ./{{.BINARY}} check --branch main testdata/rules_if_expr.yml
ignore_error: false
- cmd: ./{{.BINARY}} check --branch feat/x testdata/rules_if_expr.yml
ignore_error: false
- cmd: ./{{.BINARY}} check samba-testdata/.gitlab-ci.yml - cmd: ./{{.BINARY}} check samba-testdata/.gitlab-ci.yml
ignore_error: false ignore_error: false
- cmd: ./{{.BINARY}} check samba-testdata/.gitlab-ci-coverage.yml - cmd: ./{{.BINARY}} check samba-testdata/.gitlab-ci-coverage.yml
Binary file not shown.

After

Width:  |  Height:  |  Size: 106 KiB

+145 -17
View File
@@ -9,12 +9,15 @@ import (
// variable resolver. // variable resolver.
// //
// Supported: // Supported:
// - Variable references: $VAR_NAME // - Variable references: $VAR_NAME or ${VAR_NAME}
// - String literals: "value" or 'value' // - String literals: "value" or 'value'
// - Null keyword: null // - Null keyword: null
// - Comparison: == != =~ !~ // - Comparison: == != =~ !~
// - Boolean: && || ! // - Boolean: && || !
// - Grouping: ( ) // - Grouping: ( )
// - Regex flags: /pattern/i (case-insensitive), /pattern/m, /pattern/s
// - Multi-line: newlines between tokens are treated as whitespace
// - Variable regex RHS: $VAR =~ $PATTERN when $PATTERN holds a /regex/ string
// //
// Regex patterns use Go's regexp syntax, which covers the common RE2 subset // Regex patterns use Go's regexp syntax, which covers the common RE2 subset
// used by GitLab CI. Unsupported or unparseable expressions fall back to true // used by GitLab CI. Unsupported or unparseable expressions fall back to true
@@ -56,8 +59,13 @@ func (p *exprParser) consume(tok string) bool {
} }
func (p *exprParser) skipWS() { func (p *exprParser) skipWS() {
for p.pos < len(p.s) && (p.s[p.pos] == ' ' || p.s[p.pos] == '\t') { for p.pos < len(p.s) {
b := p.s[p.pos]
if b == ' ' || b == '\t' || b == '\n' || b == '\r' {
p.pos++ p.pos++
continue
}
break
} }
} }
@@ -67,11 +75,11 @@ func (p *exprParser) skipWS() {
// and_expr → not_expr ( '&&' not_expr )* // and_expr → not_expr ( '&&' not_expr )*
// not_expr → '!' not_expr | primary // not_expr → '!' not_expr | primary
// primary → '(' or_expr ')' | comparison // primary → '(' or_expr ')' | comparison
// comparison → value ( op value | regex_op regex )? | value // comparison → value ( op value | regex_op regex_rhs )?
// value → '$' ident | '"' … '"' | "'" … "'" | 'null' // value → '$' '{' ident '}' | '$' ident | '"' … '"' | "'" … "'" | 'null'
// op → '==' | '!=' // op → '==' | '!='
// regex_op → '=~' | '!~' // regex_op → '=~' | '!~'
// regex '/' … '/' // regex_rhs → '/' … '/' flags? | '$' ident (where ident value is '/…/flags')
func (p *exprParser) parseOr() (bool, bool) { func (p *exprParser) parseOr() (bool, bool) {
left, ok := p.parseAnd() left, ok := p.parseAnd()
@@ -165,8 +173,11 @@ func (p *exprParser) parseComparison() (bool, bool) {
case p.consume("=~"): case p.consume("=~"):
p.skipWS() p.skipWS()
pat, ok := p.parseRegexLiteral() pat, patOk, permissive := p.parseRegexRHS()
if !ok { if permissive {
return true, true
}
if !patOk {
return false, false return false, false
} }
re, err := regexp.Compile(pat) re, err := regexp.Compile(pat)
@@ -177,8 +188,11 @@ func (p *exprParser) parseComparison() (bool, bool) {
case p.consume("!~"): case p.consume("!~"):
p.skipWS() p.skipWS()
pat, ok := p.parseRegexLiteral() pat, patOk, permissive := p.parseRegexRHS()
if !ok { if permissive {
return true, true
}
if !patOk {
return false, false return false, false
} }
re, err := regexp.Compile(pat) re, err := regexp.Compile(pat)
@@ -192,13 +206,49 @@ func (p *exprParser) parseComparison() (bool, bool) {
return leftStr != "", true return leftStr != "", true
} }
// parseValue reads $VAR, "string", 'string', or null. // parseRegexRHS parses the right-hand side of =~ / !~ operators.
// null and undefined variables both produce an empty string. // Returns (pattern, ok, permissive):
// - /regex/flags literal → (pattern, true, false)
// - $VAR whose value is /regex/flags → (pattern, true, false)
// - $VAR whose value is empty or not a /regex/ → ("", false, true) — caller uses permissive true
// - parse error → ("", false, false)
func (p *exprParser) parseRegexRHS() (pat string, ok bool, permissive bool) {
if p.peek() == '/' {
pat, ok = p.parseRegexLiteral()
return pat, ok, false
}
if p.peek() == '$' {
varVal, varOk := p.parseValue()
if !varOk {
return "", false, false
}
pat, ok = extractRegexFromString(varVal)
if !ok {
return "", false, true // variable is not a /regex/ value → permissive
}
return pat, true, false
}
return "", false, false
}
// parseValue reads $VAR, ${VAR}, "string", 'string', null, true, false, or an
// integer literal. null and undefined variables both produce an empty string.
// true/false and integers produce their string representations (GitLab CI
// compares all values as strings).
func (p *exprParser) parseValue() (string, bool) { func (p *exprParser) parseValue() (string, bool) {
p.skipWS() p.skipWS()
if p.peek() == '$' { if p.peek() == '$' {
p.pos++ // consume '$' p.pos++ // consume '$'
if p.peek() == '{' {
p.pos++ // consume '{'
name := p.parseIdent()
if name == "" || p.peek() != '}' {
return "", false
}
p.pos++ // consume '}'
return p.vars(name), true
}
name := p.parseIdent() name := p.parseIdent()
if name == "" { if name == "" {
return "", false return "", false
@@ -206,12 +256,18 @@ func (p *exprParser) parseValue() (string, bool) {
return p.vars(name), true return p.vars(name), true
} }
// null keyword — must not be a prefix of a longer identifier. // Keywords and string literals must not be prefixes of longer identifiers.
if p.startsWith("null") { for _, kw := range []struct{ tok, val string }{
end := p.pos + 4 {"null", ""},
{"true", "true"},
{"false", "false"},
} {
if p.startsWith(kw.tok) {
end := p.pos + len(kw.tok)
if end >= len(p.s) || !isIdentByte(p.s[end]) { if end >= len(p.s) || !isIdentByte(p.s[end]) {
p.pos += 4 p.pos += len(kw.tok)
return "", true // null → empty string return kw.val, true
}
} }
} }
@@ -219,6 +275,15 @@ func (p *exprParser) parseValue() (string, bool) {
return p.parseStringLiteral() return p.parseStringLiteral()
} }
// Integer literal — returned as its decimal string for string comparison.
if p.peek() >= '0' && p.peek() <= '9' {
start := p.pos
for p.pos < len(p.s) && p.s[p.pos] >= '0' && p.s[p.pos] <= '9' {
p.pos++
}
return p.s[start:p.pos], true
}
return "", false return "", false
} }
@@ -261,7 +326,8 @@ func (p *exprParser) parseRegexLiteral() (string, bool) {
b := p.s[p.pos] b := p.s[p.pos]
if b == '/' { if b == '/' {
p.pos++ // consume closing '/' p.pos++ // consume closing '/'
return sb.String(), true flags := p.parseRegexFlags()
return applyRegexFlags(flags, sb.String()), true
} }
if b == '\\' && p.pos+1 < len(p.s) { if b == '\\' && p.pos+1 < len(p.s) {
p.pos++ p.pos++
@@ -275,6 +341,68 @@ func (p *exprParser) parseRegexLiteral() (string, bool) {
return "", false // unterminated regex return "", false // unterminated regex
} }
// parseRegexFlags reads zero or more regex flag letters (i, m, s) after the
// closing '/'. Unknown letters are consumed but ignored.
func (p *exprParser) parseRegexFlags() string {
start := p.pos
for p.pos < len(p.s) && isIdentByte(p.s[p.pos]) {
p.pos++
}
return p.s[start:p.pos]
}
// applyRegexFlags prepends Go regexp flag groups to pattern (e.g. (?i) for 'i').
// Unknown flags are silently ignored.
func applyRegexFlags(flags, pattern string) string {
if flags == "" {
return pattern
}
var prefix strings.Builder
for _, f := range flags {
switch f {
case 'i':
prefix.WriteString("(?i)")
case 'm':
prefix.WriteString("(?m)")
case 's':
prefix.WriteString("(?s)")
}
}
return prefix.String() + pattern
}
// extractRegexFromString parses a /pattern/flags string (typically from a CI
// variable) and returns a Go regexp pattern with flags applied.
func extractRegexFromString(s string) (string, bool) {
s = strings.TrimSpace(s)
if len(s) == 0 || s[0] != '/' {
return "", false
}
var sb strings.Builder
i := 1
for i < len(s) {
b := s[i]
if b == '/' {
i++ // past closing '/'
var flags strings.Builder
for i < len(s) && isIdentByte(s[i]) {
flags.WriteByte(s[i])
i++
}
return applyRegexFlags(flags.String(), sb.String()), true
}
if b == '\\' && i+1 < len(s) {
i++
sb.WriteByte('\\')
sb.WriteByte(s[i])
} else {
sb.WriteByte(b)
}
i++
}
return "", false // unterminated
}
func isIdentByte(b byte) bool { func isIdentByte(b byte) bool {
return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || (b >= '0' && b <= '9') || b == '_' return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || (b >= '0' && b <= '9') || b == '_'
} }
+48
View File
@@ -9,6 +9,10 @@ func TestEvalIf(t *testing.T) {
"CI_COMMIT_TAG": "", "CI_COMMIT_TAG": "",
"CI_PIPELINE_SOURCE": "push", "CI_PIPELINE_SOURCE": "push",
"DEPLOY_ENV": "staging", "DEPLOY_ENV": "staging",
"BRANCH_PATTERN": "/^dev/",
"BRANCH_PATTERN_CI": "/^DEV/i",
"EMPTY_PATTERN": "",
"PLAIN_PATTERN": "develop",
} }
return m[key] return m[key]
} }
@@ -69,6 +73,50 @@ func TestEvalIf(t *testing.T) {
{"extra spaces", ` $CI_COMMIT_BRANCH == "develop" `, true}, {"extra spaces", ` $CI_COMMIT_BRANCH == "develop" `, true},
{"tabs", "$CI_COMMIT_BRANCH\t==\t\"develop\"", true}, {"tabs", "$CI_COMMIT_BRANCH\t==\t\"develop\"", true},
// ── Multi-line expressions (newlines between tokens) ──────────────────
{"multiline or true", "$CI_COMMIT_BRANCH == \"develop\" ||\n$CI_COMMIT_TAG != null", true},
{"multiline or false", "$CI_COMMIT_BRANCH == \"main\" ||\n$CI_COMMIT_TAG != null", false},
{"multiline and true", "$CI_COMMIT_BRANCH == \"develop\" &&\n$CI_PIPELINE_SOURCE == \"push\"", true},
{"multiline and false", "$CI_COMMIT_BRANCH == \"main\" &&\n$CI_PIPELINE_SOURCE == \"push\"", false},
{"multiline with crlf", "$CI_COMMIT_BRANCH == \"develop\" ||\r\n$CI_COMMIT_TAG != null", true},
// ── ${VAR} curly-brace syntax ─────────────────────────────────────────
{"curly var eq match", `${CI_COMMIT_BRANCH} == "develop"`, true},
{"curly var eq no match", `${CI_COMMIT_BRANCH} == "main"`, false},
{"curly var truthiness", `${CI_COMMIT_BRANCH}`, true},
{"curly var falsy", `${CI_COMMIT_TAG}`, false},
{"curly var neq null", `${CI_COMMIT_BRANCH} != null`, true},
{"curly mixed", `${CI_COMMIT_BRANCH} == "develop" && $CI_PIPELINE_SOURCE == "push"`, true},
// ── Regex flags (/pattern/i etc.) ─────────────────────────────────────
{"regex flag i match", `$CI_COMMIT_BRANCH =~ /^DEV/i`, true},
{"regex flag i no match", `$CI_COMMIT_BRANCH =~ /^MAIN/i`, false},
{"regex flag i not match", `$CI_COMMIT_BRANCH !~ /^MAIN/i`, true},
{"regex no flag case sensitive", `$CI_COMMIT_BRANCH =~ /^DEV/`, false},
{"regex flag i version tag", `$CI_PIPELINE_SOURCE =~ /^PUSH$/i`, true},
// ── Variable on right side of =~ ──────────────────────────────────────
{"var regex rhs match", `$CI_COMMIT_BRANCH =~ $BRANCH_PATTERN`, true},
{"var regex rhs no match", `$CI_PIPELINE_SOURCE =~ $BRANCH_PATTERN`, false},
{"var regex rhs ci flag match", `$CI_COMMIT_BRANCH =~ $BRANCH_PATTERN_CI`, true},
{"var regex rhs empty permissive", `$CI_COMMIT_BRANCH =~ $EMPTY_PATTERN`, true},
{"var regex rhs plain permissive", `$CI_COMMIT_BRANCH =~ $PLAIN_PATTERN`, true},
{"var regex rhs not match", `$CI_COMMIT_BRANCH !~ $BRANCH_PATTERN`, false},
// ── Bare true/false keywords ─────────────────────────────────────────
// GitLab CI treats true/false as the string values "true"/"false".
{"bare true match", `$CI_PIPELINE_SOURCE == true`, false}, // "push" != "true"
{"bare false match", `$CI_COMMIT_TAG == false`, false}, // "" != "false"
{"bare true var set to true", `$DEPLOY_ENV == true`, false}, // "staging" != "true"
{"bare false neq", `$CI_COMMIT_BRANCH != false`, true}, // "develop" != "false"
{"bare true in compound", `$CI_COMMIT_BRANCH != null && $CI_COMMIT_TAG == false`, false},
// ── Integer literals ──────────────────────────────────────────────────
// Compared as decimal strings (GitLab CI converts integers to strings).
{"int eq match", `$CI_PIPELINE_SOURCE != 0`, true}, // "push" != "0"
{"int eq no match", `$CI_COMMIT_TAG == 0`, false}, // "" != "0"
{"int in compound", `$CI_COMMIT_BRANCH != null && $CI_COMMIT_BRANCH != 0`, true},
// ── Permissive fallback ─────────────────────────────────────────────── // ── Permissive fallback ───────────────────────────────────────────────
{"unparseable returns true", `this is not valid syntax %%%`, true}, {"unparseable returns true", `this is not valid syntax %%%`, true},
{"empty expr returns true", ``, true}, {"empty expr returns true", ``, true},
+18
View File
@@ -142,6 +142,24 @@ func (cfg GitLabConfig) FetchFile(project, filePath, ref string) ([]byte, error)
return body, nil return body, nil
} }
// FetchURL downloads the content at a plain HTTPS URL without authentication.
// Used for include: remote: entries which are public by definition.
func FetchURL(rawURL string) ([]byte, error) {
resp, err := http.Get(rawURL) //nolint:noctx
if err != nil {
return nil, fmt.Errorf("GET %s: %w", rawURL, err)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("reading body: %w", err)
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("GET %s: status %d", rawURL, resp.StatusCode)
}
return body, nil
}
func firstNonEmpty(values ...string) string { func firstNonEmpty(values ...string) string {
for _, v := range values { for _, v := range values {
if v != "" { if v != "" {
+81 -5
View File
@@ -4,6 +4,7 @@ import (
"fmt" "fmt"
"os" "os"
"path/filepath" "path/filepath"
"sort"
"strings" "strings"
"git.k3nny.fr/glint/internal/fetcher" "git.k3nny.fr/glint/internal/fetcher"
@@ -11,6 +12,8 @@ import (
) )
// Includes returns a Mermaid flowchart of the full include dependency tree. // Includes returns a Mermaid flowchart of the full include dependency tree.
// Each node shows the jobs defined directly in that file, connected with
// dashed arrows (solid arrows represent the include hierarchy itself).
// It recurses into project:, component:, and local: includes to expose // It recurses into project:, component:, and local: includes to expose
// transitive dependencies. Includes that cannot be fetched are shown but not expanded. // transitive dependencies. Includes that cannot be fetched are shown but not expanded.
func Includes(sourcePath string, rawIncludes []any, cfg fetcher.GitLabConfig) string { func Includes(sourcePath string, rawIncludes []any, cfg fetcher.GitLabConfig) string {
@@ -19,7 +22,12 @@ func Includes(sourcePath string, rawIncludes []any, cfg fetcher.GitLabConfig) st
cfg: cfg, cfg: cfg,
baseDir: filepath.Dir(sourcePath), baseDir: filepath.Dir(sourcePath),
} }
root := &treeNode{id: "root", label: mermaidLabel(sourcePath), class: "main"} root := &treeNode{
id: "root",
label: mermaidLabel(sourcePath),
class: "main",
jobs: directJobs(sourcePath),
}
b.buildChildren(root, rawIncludes) b.buildChildren(root, rawIncludes)
return renderTree(root) return renderTree(root)
} }
@@ -29,6 +37,7 @@ type treeNode struct {
label string label string
class string class string
children []*treeNode children []*treeNode
jobs []string // job names defined directly in this file
} }
// treeBuilder accumulates state while recursively traversing include entries. // treeBuilder accumulates state while recursively traversing include entries.
@@ -105,7 +114,9 @@ func (b *treeBuilder) parseMap(m map[string]any) []*treeNode {
return []*treeNode{node} return []*treeNode{node}
} }
if remote, ok := m["remote"].(string); ok { if remote, ok := m["remote"].(string); ok {
return []*treeNode{{id: b.nextID(), label: "remote:<br>" + mermaidLabel(remote), class: "remote"}} node := &treeNode{id: b.nextID(), label: "remote:<br>" + mermaidLabel(remote), class: "remote"}
b.recurseRemote(node, remote)
return []*treeNode{node}
} }
if tmpl, ok := m["template"].(string); ok { if tmpl, ok := m["template"].(string); ok {
return []*treeNode{{id: b.nextID(), label: "template:<br>" + mermaidLabel(tmpl), class: "template"}} return []*treeNode{{id: b.nextID(), label: "template:<br>" + mermaidLabel(tmpl), class: "template"}}
@@ -126,7 +137,11 @@ func (b *treeBuilder) recurseLocal(node *treeNode, path string) {
return return
} }
p, err := model.ParseBytes(data) p, err := model.ParseBytes(data)
if err != nil || len(p.Include) == 0 { if err != nil {
return
}
node.jobs = jobNames(p)
if len(p.Include) == 0 {
return return
} }
orig := b.baseDir orig := b.baseDir
@@ -147,7 +162,33 @@ func (b *treeBuilder) recurseProject(node *treeNode, project, filePath, ref stri
return return
} }
p, err := model.ParseBytes(data) p, err := model.ParseBytes(data)
if err != nil || len(p.Include) == 0 { if err != nil {
return
}
node.jobs = jobNames(p)
if len(p.Include) == 0 {
return
}
b.buildChildren(node, p.Include)
}
func (b *treeBuilder) recurseRemote(node *treeNode, rawURL string) {
key := "remote:" + rawURL
if b.visited[key] {
return
}
b.visited[key] = true
data, err := fetcher.FetchURL(rawURL)
if err != nil {
return
}
p, err := model.ParseBytes(data)
if err != nil {
return
}
node.jobs = jobNames(p)
if len(p.Include) == 0 {
return return
} }
b.buildChildren(node, p.Include) b.buildChildren(node, p.Include)
@@ -172,7 +213,11 @@ func (b *treeBuilder) recurseComponent(node *treeNode, ref string) {
return return
} }
p, err := model.ParseBytes(data) p, err := model.ParseBytes(data)
if err != nil || len(p.Include) == 0 { if err != nil {
return
}
node.jobs = jobNames(p)
if len(p.Include) == 0 {
return return
} }
b.buildChildren(node, p.Include) b.buildChildren(node, p.Include)
@@ -193,11 +238,19 @@ func renderTree(root *treeNode) string {
w(" classDef local fill:#428fdc,stroke:#1068bf,color:#fff") w(" classDef local fill:#428fdc,stroke:#1068bf,color:#fff")
w(" classDef remote fill:#868686,stroke:#686868,color:#fff") w(" classDef remote fill:#868686,stroke:#686868,color:#fff")
w(" classDef template fill:#fc6d26,stroke:#e56b1f,color:#fff") w(" classDef template fill:#fc6d26,stroke:#e56b1f,color:#fff")
w(" classDef job fill:#f5f5ff,stroke:#7175a0,color:#333")
w("") w("")
jobCounter := 0
var emit func(n *treeNode) var emit func(n *treeNode)
emit = func(n *treeNode) { emit = func(n *treeNode) {
wf(" %s[\"%s\"]:::%s", n.id, n.label, n.class) wf(" %s[\"%s\"]:::%s", n.id, n.label, n.class)
for _, jobName := range n.jobs {
jobCounter++
jobID := fmt.Sprintf("job%d", jobCounter)
wf(" %s(\"%s\"):::job", jobID, mermaidLabel(jobName))
wf(" %s -.-> %s", n.id, jobID)
}
for _, child := range n.children { for _, child := range n.children {
emit(child) emit(child)
wf(" %s --> %s", n.id, child.id) wf(" %s --> %s", n.id, child.id)
@@ -208,6 +261,29 @@ func renderTree(root *treeNode) string {
return sb.String() return sb.String()
} }
// directJobs parses a single pipeline file (without include resolution) and
// returns the sorted list of job names defined directly in it.
func directJobs(path string) []string {
p, err := model.Parse(path)
if err != nil {
return nil
}
return jobNames(p)
}
// jobNames returns a sorted slice of all job names defined in p.
func jobNames(p *model.Pipeline) []string {
if len(p.Jobs) == 0 {
return nil
}
names := make([]string, 0, len(p.Jobs))
for name := range p.Jobs {
names = append(names, name)
}
sort.Strings(names)
return names
}
// parseComponentRef parses a CI/CD component reference of the form // parseComponentRef parses a CI/CD component reference of the form
// <host>/<project-path>/<component-name>@<version>. // <host>/<project-path>/<component-name>@<version>.
func parseComponentRef(ref string) (host, project, component, version string, err error) { func parseComponentRef(ref string) (host, project, component, version string, err error) {
+4
View File
@@ -24,6 +24,8 @@ func checkDependencies(p *model.Pipeline) []Finding {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Job: name, Job: name,
File: job.File,
Line: job.Line,
Message: fmt.Sprintf("'dependencies' references unknown job %q", dep), Message: fmt.Sprintf("'dependencies' references unknown job %q", dep),
}) })
continue continue
@@ -34,6 +36,8 @@ func checkDependencies(p *model.Pipeline) []Finding {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Job: name, Job: name,
File: job.File,
Line: job.Line,
Message: fmt.Sprintf("'dependencies' job %q must be in an earlier stage (in %q, current job is in %q)", dep, depJob.Stage, job.Stage), Message: fmt.Sprintf("'dependencies' job %q must be in an earlier stage (in %q, current job is in %q)", dep, depJob.Stage, job.Stage),
}) })
} }
+30 -2
View File
@@ -17,12 +17,25 @@ const (
type Finding struct { type Finding struct {
Severity Severity Severity Severity
Job string // empty for pipeline-level findings Job string // empty for pipeline-level findings
File string // source file where the finding originates
Line int // line number in File (0 = unknown)
Message string Message string
} }
func (f Finding) String() string { func (f Finding) String() string {
loc := ""
if f.File != "" {
if f.Line > 0 {
loc = fmt.Sprintf(" (%s:%d)", f.File, f.Line)
} else {
loc = fmt.Sprintf(" (%s)", f.File)
}
}
if f.Job != "" { if f.Job != "" {
return fmt.Sprintf("[%s] job %q: %s", f.Severity, f.Job, f.Message) return fmt.Sprintf("[%s] job %q%s: %s", f.Severity, f.Job, loc, f.Message)
}
if loc != "" {
return fmt.Sprintf("[%s]%s: %s", f.Severity, loc, f.Message)
} }
return fmt.Sprintf("[%s] %s", f.Severity, f.Message) return fmt.Sprintf("[%s] %s", f.Severity, f.Message)
} }
@@ -43,6 +56,7 @@ func checkStages(p *model.Pipeline) []Finding {
if len(p.Stages) == 0 { if len(p.Stages) == 0 {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Warning, Severity: Warning,
File: p.SourceFile,
Message: "no stages defined; GitLab will use default stages (build, test, deploy)", Message: "no stages defined; GitLab will use default stages (build, test, deploy)",
}) })
} }
@@ -58,6 +72,7 @@ func checkWorkflow(p *model.Pipeline) []Finding {
if rule.When != "" && !validWorkflowRuleWhen[rule.When] { if rule.When != "" && !validWorkflowRuleWhen[rule.When] {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
File: p.SourceFile,
Message: fmt.Sprintf("workflow.rules[%d].when has invalid value %q; valid: always, never", i, rule.When), Message: fmt.Sprintf("workflow.rules[%d].when has invalid value %q; valid: always, never", i, rule.When),
}) })
} }
@@ -88,10 +103,16 @@ func checkJob(name string, job model.Job, stageSet map[string]bool) []Finding {
// After extends resolution, a job with no script/run is an error. // After extends resolution, a job with no script/run is an error.
// Exceptions: trigger jobs, pages jobs (use pages: keyword), and template jobs. // Exceptions: trigger jobs, pages jobs (use pages: keyword), and template jobs.
// When the job has extends:, the script may come from a base that couldn't be
// fetched (e.g. a remote include without a token), so downgrade to warning.
hasScript := scriptNonEmpty(job.Script) || job.Run != nil hasScript := scriptNonEmpty(job.Script) || job.Run != nil
if !isTemplate && !isTrigger && job.Pages == nil && !hasScript { if !isTemplate && !isTrigger && job.Pages == nil && !hasScript {
sev := Error
if job.Extends != nil {
sev = Warning
}
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: sev,
Job: name, Job: name,
Message: "missing required field 'script' (or 'run')", Message: "missing required field 'script' (or 'run')",
}) })
@@ -137,6 +158,13 @@ func checkJob(name string, job model.Job, stageSet map[string]bool) []Finding {
findings = append(findings, checkJobKeywords(name, job)...) findings = append(findings, checkJobKeywords(name, job)...)
// Attach source location to every job-scoped finding collected above.
for i := range findings {
if findings[i].Job != "" && findings[i].File == "" {
findings[i].File = job.File
findings[i].Line = job.Line
}
}
return findings return findings
} }
+42 -19
View File
@@ -6,6 +6,12 @@ import (
"git.k3nny.fr/glint/internal/model" "git.k3nny.fr/glint/internal/model"
) )
// needEntry is a parsed element from a job's needs: list.
type needEntry struct {
job string
optional bool // true when the needs entry carries optional: true
}
func checkNeeds(p *model.Pipeline) []Finding { func checkNeeds(p *model.Pipeline) []Finding {
var findings []Finding var findings []Finding
@@ -15,7 +21,7 @@ func checkNeeds(p *model.Pipeline) []Finding {
stageIndex[s] = i stageIndex[s] = i
} }
// needsGraph maps each job to the list of jobs it depends on. // needsGraph maps each job to the jobs it depends on (existing jobs only).
// Used for cycle detection after individual checks. // Used for cycle detection after individual checks.
needsGraph := make(map[string][]string) needsGraph := make(map[string][]string)
@@ -24,22 +30,33 @@ func checkNeeds(p *model.Pipeline) []Finding {
continue continue
} }
neededNames := parseNeedJobNames(job.Needs) entries := parseNeedEntries(job.Needs)
needsGraph[name] = neededNames
jobStageIdx, jobHasStage := stageIndex[job.Stage] jobStageIdx, jobHasStage := stageIndex[job.Stage]
for _, needed := range neededNames { for _, entry := range entries {
neededJob, exists := p.Jobs[needed] neededJob, exists := p.Jobs[entry.job]
if !exists { if !exists {
// optional: true means GitLab CI will silently skip the
// dependency when the job is absent (e.g. from a conditional
// include). Downgrade to warning so users are informed without
// failing the lint.
sev := Error
if entry.optional {
sev = Warning
}
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: sev,
Job: name, Job: name,
Message: fmt.Sprintf("needs unknown job %q", needed), File: job.File,
Line: job.Line,
Message: fmt.Sprintf("needs unknown job %q", entry.job),
}) })
continue continue
} }
// Add to the cycle-detection graph only when the dep exists.
needsGraph[name] = append(needsGraph[name], entry.job)
// A job cannot need a job in a later stage. // A job cannot need a job in a later stage.
if len(p.Stages) > 0 && jobHasStage && neededJob.Stage != "" { if len(p.Stages) > 0 && jobHasStage && neededJob.Stage != "" {
neededStageIdx, neededHasStage := stageIndex[neededJob.Stage] neededStageIdx, neededHasStage := stageIndex[neededJob.Stage]
@@ -47,9 +64,11 @@ func checkNeeds(p *model.Pipeline) []Finding {
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Job: name, Job: name,
File: job.File,
Line: job.Line,
Message: fmt.Sprintf( Message: fmt.Sprintf(
"needs %q which is in a later stage (%q after %q)", "needs %q which is in a later stage (%q after %q)",
needed, neededJob.Stage, job.Stage, entry.job, neededJob.Stage, job.Stage,
), ),
}) })
} }
@@ -57,32 +76,33 @@ func checkNeeds(p *model.Pipeline) []Finding {
} }
} }
findings = append(findings, detectNeedsCycles(needsGraph)...) findings = append(findings, detectNeedsCycles(needsGraph, p.Jobs)...)
return findings return findings
} }
// parseNeedJobNames extracts job names from a needs: list. // parseNeedEntries extracts needs entries from a needs: list, preserving the
// Each element is either a plain string or a map with a "job" key. // optional flag. Each element is a plain string (job name) or a map with a
// Cross-pipeline needs (maps with a "pipeline" key) are skipped. // "job" key. Cross-pipeline needs (maps with a "pipeline" key) are skipped.
func parseNeedJobNames(needs []any) []string { func parseNeedEntries(needs []any) []needEntry {
var names []string var entries []needEntry
for _, n := range needs { for _, n := range needs {
switch v := n.(type) { switch v := n.(type) {
case string: case string:
names = append(names, v) entries = append(entries, needEntry{job: v})
case map[string]any: case map[string]any:
if _, crossPipeline := v["pipeline"]; crossPipeline { if _, crossPipeline := v["pipeline"]; crossPipeline {
continue continue
} }
if job, ok := v["job"].(string); ok { if job, ok := v["job"].(string); ok {
names = append(names, job) optional, _ := v["optional"].(bool)
entries = append(entries, needEntry{job: job, optional: optional})
} }
} }
} }
return names return entries
} }
func detectNeedsCycles(graph map[string][]string) []Finding { func detectNeedsCycles(graph map[string][]string, jobs map[string]model.Job) []Finding {
const ( const (
unvisited = 0 unvisited = 0
visiting = 1 visiting = 1
@@ -101,9 +121,12 @@ func detectNeedsCycles(graph map[string][]string) []Finding {
case visiting: case visiting:
if !reported[name] { if !reported[name] {
reported[name] = true reported[name] = true
j := jobs[name]
findings = append(findings, Finding{ findings = append(findings, Finding{
Severity: Error, Severity: Error,
Job: name, Job: name,
File: j.File,
Line: j.Line,
Message: fmt.Sprintf("circular dependency in needs: %v → %s", path, name), Message: fmt.Sprintf("circular dependency in needs: %v → %s", path, name),
}) })
} }
+30 -7
View File
@@ -13,14 +13,22 @@ func Parse(path string) (*Pipeline, error) {
if err != nil { if err != nil {
return nil, fmt.Errorf("reading file: %w", err) return nil, fmt.Errorf("reading file: %w", err)
} }
return ParseBytes(data) p, err := ParseBytes(data)
if err != nil {
return nil, err
}
p.SourceFile = path
p.SetJobOrigin(path)
return p, nil
} }
// ParseBytes parses YAML from an in-memory byte slice. // ParseBytes parses YAML from an in-memory byte slice.
func ParseBytes(data []byte) (*Pipeline, error) { func ParseBytes(data []byte) (*Pipeline, error) {
// First pass: decode into a raw map to extract job keys. // First pass: parse into a yaml.Node document to extract job keys with
var raw map[string]yaml.Node // their exact source line numbers (key nodes carry the line, value nodes
if err := yaml.Unmarshal(data, &raw); err != nil { // carry the body we decode into Job / map[string]any).
var doc yaml.Node
if err := yaml.Unmarshal(data, &doc); err != nil {
return nil, fmt.Errorf("parsing YAML: %w", err) return nil, fmt.Errorf("parsing YAML: %w", err)
} }
@@ -32,21 +40,36 @@ func ParseBytes(data []byte) (*Pipeline, error) {
p.Jobs = make(map[string]Job) p.Jobs = make(map[string]Job)
p.RawJobs = make(map[string]map[string]any) p.RawJobs = make(map[string]map[string]any)
for key, node := range raw {
if doc.Kind != yaml.DocumentNode || len(doc.Content) == 0 {
return p, nil
}
root := doc.Content[0]
if root.Kind != yaml.MappingNode {
return p, nil
}
// Walk root mapping in key/value pairs.
for i := 0; i+1 < len(root.Content); i += 2 {
keyNode := root.Content[i]
valNode := root.Content[i+1]
key := keyNode.Value
if ReservedKeys[key] { if ReservedKeys[key] {
continue continue
} }
var rawMap map[string]any var rawMap map[string]any
if err := node.Decode(&rawMap); err != nil { if err := valNode.Decode(&rawMap); err != nil {
return nil, fmt.Errorf("parsing raw job %q: %w", key, err) return nil, fmt.Errorf("parsing raw job %q: %w", key, err)
} }
p.RawJobs[key] = rawMap p.RawJobs[key] = rawMap
var j Job var j Job
if err := node.Decode(&j); err != nil { if err := valNode.Decode(&j); err != nil {
return nil, fmt.Errorf("parsing job %q: %w", key, err) return nil, fmt.Errorf("parsing job %q: %w", key, err)
} }
j.Name = key j.Name = key
j.Line = keyNode.Line // exact line of the job name key
p.Jobs[key] = j p.Jobs[key] = j
} }
+21 -7
View File
@@ -3,8 +3,9 @@ package model
// Pipeline represents the top-level structure of a .gitlab-ci.yml file. // Pipeline represents the top-level structure of a .gitlab-ci.yml file.
// Unknown top-level keys are collected into Jobs. // Unknown top-level keys are collected into Jobs.
type Pipeline struct { type Pipeline struct {
SourceFile string // path of the root pipeline file; set by Parse
Stages []string `yaml:"stages"` Stages []string `yaml:"stages"`
Variables map[string]string `yaml:"variables"` Variables map[string]any `yaml:"variables"` // string or {value,description,options} map
Default *DefaultConfig `yaml:"default"` Default *DefaultConfig `yaml:"default"`
Include []any `yaml:"include"` Include []any `yaml:"include"`
Workflow *Workflow `yaml:"workflow"` Workflow *Workflow `yaml:"workflow"`
@@ -13,10 +14,21 @@ type Pipeline struct {
RawJobs map[string]map[string]any `yaml:"-"` // pre-resolution raw maps, used by the resolver RawJobs map[string]map[string]any `yaml:"-"` // pre-resolution raw maps, used by the resolver
} }
// SetJobOrigin sets the File field on all jobs that don't already have one.
// Called after ParseBytes to record which file each job came from.
func (p *Pipeline) SetJobOrigin(file string) {
for name, j := range p.Jobs {
if j.File == "" {
j.File = file
}
p.Jobs[name] = j
}
}
type DefaultConfig struct { type DefaultConfig struct {
Image string `yaml:"image"` Image any `yaml:"image"` // string or {name,pull_policy,...} map
BeforeScript []string `yaml:"before_script"` BeforeScript any `yaml:"before_script"` // []string or string (block scalar)
AfterScript []string `yaml:"after_script"` AfterScript any `yaml:"after_script"` // []string or string
Cache any `yaml:"cache"` Cache any `yaml:"cache"`
Artifacts any `yaml:"artifacts"` Artifacts any `yaml:"artifacts"`
Retry any `yaml:"retry"` Retry any `yaml:"retry"`
@@ -30,6 +42,8 @@ type Workflow struct {
type Job struct { type Job struct {
Name string // set by parser, not from YAML Name string // set by parser, not from YAML
File string // source file; set by Parse / resolver
Line int // line of the job key in its source file; set by parser
Stage string `yaml:"stage"` Stage string `yaml:"stage"`
Script any `yaml:"script"` // []string or string (block scalar) Script any `yaml:"script"` // []string or string (block scalar)
Run any `yaml:"run"` // alternative to script (CI steps) Run any `yaml:"run"` // alternative to script (CI steps)
@@ -37,7 +51,7 @@ type Job struct {
AfterScript any `yaml:"after_script"` // []string or string AfterScript any `yaml:"after_script"` // []string or string
Image any `yaml:"image"` Image any `yaml:"image"`
Services []any `yaml:"services"` Services []any `yaml:"services"`
Variables map[string]string `yaml:"variables"` Variables map[string]any `yaml:"variables"` // string or {value,description,options} map
Rules []Rule `yaml:"rules"` Rules []Rule `yaml:"rules"`
Only any `yaml:"only"` Only any `yaml:"only"`
Except any `yaml:"except"` Except any `yaml:"except"`
@@ -68,8 +82,8 @@ type Job struct {
type Rule struct { type Rule struct {
If string `yaml:"if"` If string `yaml:"if"`
When string `yaml:"when"` When string `yaml:"when"`
Changes []string `yaml:"changes"` Changes any `yaml:"changes"` // []string or {paths,compare_to} map
Exists []string `yaml:"exists"` Exists any `yaml:"exists"` // []string or map form
} }
// ReservedKeys are top-level GitLab CI keys that are NOT job definitions. // ReservedKeys are top-level GitLab CI keys that are NOT job definitions.
+44 -1
View File
@@ -90,7 +90,14 @@ func resolveIncludes(p *model.Pipeline, includes []any, cfg fetcher.GitLabConfig
continue continue
} }
// remote, template — resolved by GitLab at runtime, skip silently. if remote, _ := entry["remote"].(string); remote != "" {
w, ew := resolveRemoteInclude(p, remote, cfg, rootDir, visited)
warnings = append(warnings, w...)
extWarnings = append(extWarnings, ew...)
continue
}
// template — resolved by GitLab at runtime, skip silently.
} }
return warnings, extWarnings return warnings, extWarnings
} }
@@ -117,6 +124,7 @@ func resolveLocalInclude(p *model.Pipeline, rawPath string, cfg fetcher.GitLabCo
if err != nil { if err != nil {
return []IncludeWarning{{Label: label, Err: fmt.Errorf("parsing YAML: %w", err)}}, nil return []IncludeWarning{{Label: label, Err: fmt.Errorf("parsing YAML: %w", err)}}, nil
} }
included.SetJobOrigin(absPath)
// Recursively resolve the included file's own includes first, merging // Recursively resolve the included file's own includes first, merging
// everything into `included` before we merge it into the parent `p`. // everything into `included` before we merge it into the parent `p`.
@@ -132,6 +140,39 @@ func resolveLocalInclude(p *model.Pipeline, rawPath string, cfg fetcher.GitLabCo
return warnings, extWarnings return warnings, extWarnings
} }
// resolveRemoteInclude fetches a plain HTTPS URL, parses it as CI YAML, and
// merges it into p. Sub-includes of the fetched file are resolved recursively.
func resolveRemoteInclude(p *model.Pipeline, rawURL string, cfg fetcher.GitLabConfig, rootDir string, visited map[string]bool) ([]IncludeWarning, []ExtendWarning) {
label := "remote " + rawURL
if visited[rawURL] {
return nil, nil
}
visited[rawURL] = true
data, err := fetcher.FetchURL(rawURL)
if err != nil {
return []IncludeWarning{{Label: label, Err: err}}, nil
}
included, err := model.ParseBytes(data)
if err != nil {
return []IncludeWarning{{Label: label, Err: fmt.Errorf("parsing YAML: %w", err)}}, nil
}
included.SetJobOrigin(rawURL)
var warnings []IncludeWarning
var extWarnings []ExtendWarning
if len(included.Include) > 0 {
w, ew := resolveIncludes(included, included.Include, cfg, rootDir, visited)
warnings = append(warnings, w...)
extWarnings = append(extWarnings, ew...)
}
mergeIncluded(p, included)
return warnings, extWarnings
}
// resolveProjectInclude fetches all files listed under a single project: entry // resolveProjectInclude fetches all files listed under a single project: entry
// and merges them into p. // and merges them into p.
func resolveProjectInclude(p *model.Pipeline, entry map[string]any, project string, cfg fetcher.GitLabConfig, rootDir string, visited map[string]bool) ([]IncludeWarning, []ExtendWarning) { func resolveProjectInclude(p *model.Pipeline, entry map[string]any, project string, cfg fetcher.GitLabConfig, rootDir string, visited map[string]bool) ([]IncludeWarning, []ExtendWarning) {
@@ -161,6 +202,7 @@ func resolveProjectInclude(p *model.Pipeline, entry map[string]any, project stri
warnings = append(warnings, IncludeWarning{Label: label, Err: fmt.Errorf("parsing YAML: %w", err)}) warnings = append(warnings, IncludeWarning{Label: label, Err: fmt.Errorf("parsing YAML: %w", err)})
continue continue
} }
included.SetJobOrigin(label)
if len(included.Include) > 0 { if len(included.Include) > 0 {
w, ew := resolveIncludes(included, included.Include, cfg, rootDir, visited) w, ew := resolveIncludes(included, included.Include, cfg, rootDir, visited)
@@ -200,6 +242,7 @@ func resolveComponentInclude(p *model.Pipeline, ref string, cfg fetcher.GitLabCo
if err != nil { if err != nil {
return IncludeWarning{Label: label, Err: fmt.Errorf("parsing component YAML: %w", err)}, nil, true return IncludeWarning{Label: label, Err: fmt.Errorf("parsing component YAML: %w", err)}, nil, true
} }
included.SetJobOrigin(label)
var extWarnings []ExtendWarning var extWarnings []ExtendWarning
if len(included.Include) > 0 { if len(included.Include) > 0 {
+14
View File
@@ -0,0 +1,14 @@
---
# Exercises include: remote: URL fetching and sub-include recursion.
# The remote file is a real public GitLab CI template; it may contain its own
# includes which should also be resolved. Exit code is 0 (warnings allowed).
stages:
- test
include:
- remote: https://gitlab.com/gitlab-org/gitlab/-/raw/master/lib/gitlab/ci/templates/Bash.gitlab-ci.yml
local-job:
stage: test
script:
- echo "local job alongside remote include"
+9
View File
@@ -38,3 +38,12 @@ missing-needs-job:
- echo "bad" - echo "bad"
needs: needs:
- nonexistent-job # ERROR: job doesn't exist - nonexistent-job # ERROR: job doesn't exist
# optional: true — missing dep should be WARNING not ERROR
optional-needs-job:
stage: build
script:
- echo "I depend on something that may not exist"
needs:
- job: nonexistent-optional-job
optional: true
+51
View File
@@ -0,0 +1,51 @@
---
# rules_if_expr.yml
# Exercises the rules:if: expression evaluator for:
# - Multi-line block-scalar expressions (|| on next line)
# - ${VAR} curly-brace variable syntax
# - Regex flags (/pattern/i case-insensitive)
# - Parenthesised compound expressions
# Expected: exits 0 (lints clean with no context).
stages:
- build
- deploy
variables:
DEPLOY_ENVIRONMENTS:
value: "staging"
description: "Target deployment environment"
build:
stage: build
script: make build
rules:
# Multi-line expression: || on next line
- if: |
$CI_COMMIT_BRANCH == "main" ||
$CI_COMMIT_BRANCH == "develop"
when: on_success
- when: never
deploy-feature:
stage: deploy
script: make deploy
rules:
# ${VAR} curly-brace syntax
- if: '${CI_COMMIT_BRANCH} != null && ${CI_COMMIT_TAG} == null'
when: manual
- when: never
release:
stage: deploy
script: make release
rules:
# Case-insensitive regex flag
- if: '$CI_COMMIT_BRANCH =~ /^(main|master)$/i'
when: on_success
# Parenthesised compound with multi-line
- if: >-
($CI_COMMIT_TAG != null) &&
($CI_PIPELINE_SOURCE == "push" || $CI_PIPELINE_SOURCE == "web")
when: on_success
- when: never
+87
View File
@@ -0,0 +1,87 @@
---
# Exercises multi-line script patterns and extended variable declarations.
# Ref: https://docs.gitlab.com/ci/yaml/script/#split-long-commands
# Ref: https://docs.gitlab.com/ee/ci/yaml/#variablesdescription
# All patterns here must parse cleanly (exit 0).
stages:
- build
- test
- deploy
# Pipeline-level variables: plain strings and extended {value, description} map form.
variables:
PLAIN_VAR: "hello"
DEPLOY_ENV:
value: "staging"
description: "The deployment target. Set to staging or production."
RETRIES:
value: "3"
description: "Number of retry attempts."
options:
- "1"
- "3"
- "5"
default:
# image in map form (name + pull_policy)
image:
name: alpine:latest
pull_policy: if-not-present
# before_script as a block scalar (not a list)
before_script:
- apk add --no-cache curl git
build-literal-block:
stage: build
# script items using literal block scalar (|)
script:
- |
if [[ "$DEPLOY_ENV" == "production" ]]; then
echo "Production build"
else
echo "Non-production build"
fi
- echo "Build step done"
build-folded-block:
stage: build
# script items using folded block scalar (>)
script:
- >
apt-get update -qq &&
apt-get install -y curl wget
- echo "Packages installed"
before_script:
- |
echo "Job-level before_script"
echo "Using literal block scalar"
test-job:
stage: test
script:
- echo "Running tests"
- |
set -e
go test ./...
echo "Tests passed"
# Job-level variable with extended form
variables:
TEST_FLAG:
value: "true"
description: "Enable verbose test output"
# rules.changes in map form (GitLab 15.3+)
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
paths:
- "**/*.go"
compare_to: "main"
when: on_success
- when: on_success
deploy-job:
stage: deploy
script:
- echo "Deploying to $DEPLOY_ENV"
when: manual