diff --git a/.github/workflows/gatekeep-pr.yml b/.github/workflows/gatekeep-pr.yml index d94708c..f47769a 100644 --- a/.github/workflows/gatekeep-pr.yml +++ b/.github/workflows/gatekeep-pr.yml @@ -3,6 +3,12 @@ # # Secret: OPENROUTER_API_KEY (already configured on this repo) # Action: in-repo composite at repository root (uses: ./) +# +# Important (uses: ./): +# - Checkout is the PR head, so Action fixes on main do NOT apply until the +# PR branch rebases/merges main. Stale heads can look "broken" after a fix. +# - Re-run: push an empty commit, or Actions → this workflow → Run workflow +# (workflow_dispatch; uses default branch / manual inputs as configured). name: Gatekeep diff --git a/README.md b/README.md index 40e4d48..1233e41 100644 --- a/README.md +++ b/README.md @@ -22,30 +22,41 @@ AI-assisted PR review for **your own repos**: parallel multi-model review across `layue13/gatekeep` reviews **its own** PRs via [`.github/workflows/gatekeep-pr.yml`](.github/workflows/gatekeep-pr.yml) with `uses: ./` and `secrets.OPENROUTER_API_KEY`. Open a PR against `main` to see a review (Gitea Actions must have a runner — same pool as other `layue13` repos). +Because dogfood uses `uses: ./`, **rebase/merge `main` into the PR branch** after Action fixes land, or the PR still runs the old Action. Re-run via new push or **Actions → Gatekeep → Run workflow**. Free-model 429s are retried; extras in `models:` after the first 3 are fallbacks; risk is force-high only if **zero** models succeed. + Allowlists / Gitea-on-github.com limitations: [`docs/remote-uses.md`](docs/remote-uses.md). ## Providers YAML ```yaml +language: zh-CN # report language: zh-CN | en | … (prompts + published comments) + providers: openrouter: # only include providers you configure key: OPENROUTER_API_KEY # free-form secret/env name — pick any protocol: openai # openai | openai-completions | openai-responses | anthropic # base_url: optional (defaults exist for openrouter / anthropic / openai) models: - - google/gemma-4-31b-it:free - - google/gemma-4-26b-a4b-it:free - cohere/north-mini-code:free - nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free + - google/gemma-4-31b-it:free ``` | Field | Meaning | | --- | --- | +| `language` | Report language for model prose + summary/inline comments (`zh-CN`, `en`, …). Alias: `report_language`. | | `key` | **Free-form** name of the GitHub secret / env var holding the API key. **No** required `gatekeep_` prefix. | | `protocol` | `openai` \| `openai-completions` \| `openai-responses` \| `anthropic` | | `models` | Names to use (validated against the provider’s model list API at runtime) | | `base_url` | Optional; known provider ids get defaults | +### Report language + +- **This repo’s defaults:** `language: zh-CN` in `defaults/providers.yaml`. +- **Override without editing YAML:** Action input `language: en` (or `zh-CN`). +- Resolution order: Action input `language` → YAML `language` / `report_language` → `en`. +- JSON field names and enums stay English; human-facing prose follows the configured language. + Mix providers in one review by listing several entries. Secrets are read from the **workflow step `env`** using each entry’s `key` name. Example suggestion only (still free-form): `key: gatekeep_openrouter` is fine if you named your secret that — not required. diff --git a/README.zh-CN.md b/README.zh-CN.md index a682d20..764bdc5 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -28,31 +28,46 @@ AI 辅助的 PR 审查 Action:多模型并行投票、按严重级别输出可 `layue13/gatekeep` 用 [`.github/workflows/gatekeep-pr.yml`](.github/workflows/gatekeep-pr.yml) 对本仓库 PR 跑 Gatekeep:`uses: ./` + `secrets.OPENROUTER_API_KEY`。向 `main` 开 PR 即可触发(需有可用的 Gitea Actions runner)。 +**注意:** `uses: ./` 用的是 **PR 分支上的 Action**。main 上的修复要进 PR,需要 rebase/merge main,否则看起来像“修了还挂”。重跑:再推一次提交,或 Actions → Gatekeep → **Run workflow**。免费模型 429 会退避重试;`models` 列表第 4 个及以后作备用;**只有全部模型失败**才强制抬高风险档为 high。 + github.com runner 若不允许非 github.com Action 源,见 `docs/remote-uses.md`。 ## 多供应商(providers.yaml) ```yaml +language: zh-CN # 报告语言:zh-CN | en | …(提示词 + 汇总/inline 评论) + providers: openrouter: key: OPENROUTER_API_KEY # 自由命名 — 任意 secret/环境变量名 protocol: openai # openai | openai-completions | openai-responses | anthropic models: - - google/gemma-4-31b-it:free - - google/gemma-4-26b-a4b-it:free - cohere/north-mini-code:free - nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free + - google/gemma-4-31b-it:free ``` +- **`language`:报告语言**(本仓库默认 `zh-CN`)。也可用 Action 输入 `language:` 覆盖。别名 `report_language`。 - **`key`:自由命名**,**不要**要求必须 `gatekeep_` 前缀(你也可以叫 `gatekeep_openrouter`,只是举例)。 - 每个供应商只需:`key` + `protocol`(`base_url` / `models` 按需;已知 id 有默认 base_url)。 - **不内置死模型清单**;启动时拉取供应商模型列表并校验名字。 - 可混用多个供应商;只写你真正配置了的条目。 +### 如何设置语言 + +| 方式 | 示例 | +| --- | --- | +| `providers.yaml` | `language: zh-CN` 或 `language: en` | +| Action 输入(优先) | `with: { language: en }` | +| 缺省 | 无 YAML / 无输入时回落 `en`;本仓库 defaults 已写 `zh-CN` | + +JSON 键名与枚举值(如 `Critical`)保持英文;摘要、结论正文、inline 评论跟人读的文案跟 `language`。 + ## 输入面 | 项 | 含义 | | --- | --- | +| `language` | 报告语言(`zh-CN` / `en` …);覆盖 YAML | | `key` | Secret / 环境变量名(自由) | | `protocol` | `openai` / `openai-completions` / `openai-responses` / `anthropic` | | `models` | 参与并行审查的模型名 | diff --git a/action.yml b/action.yml index 9587cae..31b8053 100644 --- a/action.yml +++ b/action.yml @@ -21,6 +21,12 @@ inputs: (or bare names unique in config). Empty = providers.yaml lists. required: false default: "" + language: + description: > + Report language for model prose + published summary/inline comments + (e.g. zh-CN, en). Empty = providers.yaml `language`, else en. + required: false + default: "" skip-draft: description: Skip draft PRs required: false @@ -52,10 +58,18 @@ runs: # Runner image / consumer workflow must provide Node >= 18. - name: Require Node.js shell: bash + env: + GATEKEEP_LANGUAGE: ${{ inputs.language }} run: | if ! command -v node >/dev/null 2>&1; then - echo "::error::Node.js >= 18 required for Gatekeep (merge-findings.js / publish.js)." - echo "Preinstall Node on the runner, or add a mirrored setup-node step before uses: ./." + lang="${GATEKEEP_LANGUAGE:-}" + if [[ "${lang,,}" == zh* ]]; then + echo "::error::Gatekeep 需要 Node.js ≥ 18(merge-findings.js / publish.js)。" + echo "请在 runner 预装 Node,或在 uses: ./ 之前加本 forge 已镜像的 setup-node;不要依赖未镜像的 actions/setup-node。" + else + echo "::error::Node.js >= 18 required for Gatekeep (merge-findings.js / publish.js)." + echo "Preinstall Node on the runner, or add a mirrored setup-node step before uses: ./." + fi exit 1 fi node --version @@ -102,6 +116,7 @@ runs: GITHUB_TOKEN: ${{ inputs.github-token }} CONFIG_DIR: ${{ inputs.config-dir }} GATEKEEP_MODELS: ${{ inputs.models }} + GATEKEEP_LANGUAGE: ${{ inputs.language }} SKIP_DRAFT: ${{ inputs.skip-draft }} GATEKEEP_ROOT: ${{ github.workspace }} GATEKEEP_ACTION: ${{ github.action_path }} diff --git a/config/providers.yaml b/config/providers.yaml index 6ee7af3..0a59034 100644 --- a/config/providers.yaml +++ b/config/providers.yaml @@ -1,6 +1,8 @@ # Example / owner consumer override — OpenRouter only. # `key` is free-form; map it in the workflow step env from secrets. +language: zh-CN + agreement_min: 2 temperature: 0 max_tokens: 4096 @@ -10,7 +12,7 @@ providers: key: OPENROUTER_API_KEY protocol: openai models: - - google/gemma-4-31b-it:free - - google/gemma-4-26b-a4b-it:free - cohere/north-mini-code:free - nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free + - google/gemma-4-31b-it:free + - google/gemma-4-26b-a4b-it:free diff --git a/defaults/providers.yaml b/defaults/providers.yaml index be39da5..5c33b54 100644 --- a/defaults/providers.yaml +++ b/defaults/providers.yaml @@ -2,6 +2,10 @@ # `key` is a FREE-FORM secret / env var name (no required gatekeep_ prefix). # Put that name in the workflow step env, e.g. env: { OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }} } +# Report language for prompts + published summary/inline comments (BCP-47-ish: zh-CN, en, …). +# Override with Action input `language` when set. +language: zh-CN + agreement_min: 2 temperature: 0 max_tokens: 4096 @@ -10,8 +14,10 @@ providers: openrouter: key: OPENROUTER_API_KEY protocol: openai + # First 3 are used in parallel. Prefer models that tend to answer on free tier; + # put rate-limit-prone free gemma after cohere/nvidia. models: - - google/gemma-4-31b-it:free - - google/gemma-4-26b-a4b-it:free - cohere/north-mini-code:free - nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free + - google/gemma-4-31b-it:free + - google/gemma-4-26b-a4b-it:free diff --git a/docs/dogfood.md b/docs/dogfood.md index 8834176..25d3bfa 100644 --- a/docs/dogfood.md +++ b/docs/dogfood.md @@ -9,13 +9,16 @@ The Action repo itself runs Gatekeep on its own pull requests. | Secret | `OPENROUTER_API_KEY` → step `env` | | Trigger | `pull_request` (non-draft) + `workflow_dispatch` | | Proven | [PR #2](https://git.potatolab.uk/layue13/gatekeep/pulls/2) got ``; run 15833 **success** | +| Report language | `defaults/providers.yaml` → `language: zh-CN` (override with Action `language:`) | -Gitea notes: no `actions/setup-node` mirror — Action requires runner Node ≥18. PyYAML installs via workspace venv (PEP 668). Free OpenRouter models may 429; review still posts (fail-closed tier). +Gitea notes: no `actions/setup-node` mirror — Action requires runner Node ≥18. PyYAML installs via workspace venv (PEP 668). Free OpenRouter models may 429: Action retries with backoff, tries `fallback_models` (list entries after the first 3), and only force-elevates risk to high when **zero** models succeed. Summary is Chinese when `language: zh-CN`. ## How to see a review -1. Branch off `main`, make a tiny change, open a PR on https://git.potatolab.uk/layue13/gatekeep -2. Wait for the **Gatekeep** workflow (same Actions runner pool as other `layue13` repos) -3. Look for the summary comment `` on the PR +1. Branch off **current** `main` (or merge/rebase main into the PR branch — `uses: ./` loads Action code from the PR head) +2. Open/push a PR on https://git.potatolab.uk/layue13/gatekeep +3. Wait for the **Gatekeep** workflow (same Actions runner pool as other `layue13` repos) +4. Look for the summary comment `` (updated in place on re-run) +5. Manual re-run: Actions → Gatekeep → **Run workflow** (`workflow_dispatch`), or push another commit Consumer sample workflows remain under top-level `workflows/` for copy-paste into *other* repos’ `.github/workflows/`. diff --git a/scripts/call-model.sh b/scripts/call-model.sh old mode 100644 new mode 100755 index 275bde7..d1dbb80 --- a/scripts/call-model.sh +++ b/scripts/call-model.sh @@ -2,6 +2,7 @@ # Call one model via provider protocol. # Protocols: openai | openai-completions | openai-responses | anthropic # (openai / openai-compatible → chat completions) +# Retries HTTP 429 with exponential backoff (GATEKEEP_HTTP_RETRIES, default 3). # Args: set -euo pipefail @@ -21,6 +22,7 @@ if [[ -z "$KEY" ]]; then fi MAX_TOKENS="${GATEKEEP_MAX_TOKENS:-4096}" +RETRIES="${GATEKEEP_HTTP_RETRIES:-3}" SYSTEM=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["system"])' "$PROMPT_FILE") USER=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["user"])' "$PROMPT_FILE") @@ -54,13 +56,9 @@ print(json.dumps({ })) PY ) - HTTP_CODE=$(curl -sS -o "$OUT_FILE.raw" -w "%{http_code}" \ - "${BASE_URL%/}/chat/completions" \ - -H "Authorization: Bearer $KEY" \ - -H "Content-Type: application/json" \ - -H "HTTP-Referer: https://github.com/${GITHUB_REPOSITORY:-gatekeep}" \ - -H "X-Title: Gatekeep" \ - -d "$BODY") + URL="${BASE_URL%/}/chat/completions" + EXTRA_HDR=(-H "HTTP-Referer: https://github.com/${GITHUB_REPOSITORY:-gatekeep}" -H "X-Title: Gatekeep") + AUTH=(-H "Authorization: Bearer $KEY") EXTRACT=openai elif [[ "$WIRE" == "openai-responses" ]]; then BODY=$(python3 - "$MODEL" "$SYSTEM" "$USER" "$MAX_TOKENS" <<'PY' @@ -77,11 +75,9 @@ print(json.dumps({ })) PY ) - HTTP_CODE=$(curl -sS -o "$OUT_FILE.raw" -w "%{http_code}" \ - "${BASE_URL%/}/responses" \ - -H "Authorization: Bearer $KEY" \ - -H "Content-Type: application/json" \ - -d "$BODY") + URL="${BASE_URL%/}/responses" + EXTRA_HDR=() + AUTH=(-H "Authorization: Bearer $KEY") EXTRACT=openai-responses elif [[ "$WIRE" == "anthropic" ]]; then BODY=$(python3 - "$MODEL" "$SYSTEM" "$USER" "$MAX_TOKENS" <<'PY' @@ -96,21 +92,36 @@ print(json.dumps({ })) PY ) - HTTP_CODE=$(curl -sS -o "$OUT_FILE.raw" -w "%{http_code}" \ - "${BASE_URL%/}/v1/messages" \ - -H "x-api-key: $KEY" \ - -H "anthropic-version: 2023-06-01" \ - -H "Content-Type: application/json" \ - -d "$BODY") + URL="${BASE_URL%/}/v1/messages" + EXTRA_HDR=(-H "anthropic-version: 2023-06-01") + AUTH=(-H "x-api-key: $KEY") EXTRACT=anthropic fi -if [[ "$HTTP_CODE" != "200" ]]; then +attempt=0 +HTTP_CODE="000" +while true; do + HTTP_CODE=$(curl -sS -o "$OUT_FILE.raw" -w "%{http_code}" \ + "$URL" \ + "${AUTH[@]}" \ + -H "Content-Type: application/json" \ + "${EXTRA_HDR[@]}" \ + -d "$BODY" || echo "000") + if [[ "$HTTP_CODE" == "200" ]]; then + break + fi + if [[ "$HTTP_CODE" == "429" && "$attempt" -lt "$RETRIES" ]]; then + sleep_s=$((2 ** attempt)) + echo "$LABEL HTTP 429 — retry $((attempt + 1))/${RETRIES} after ${sleep_s}s" >&2 + sleep "$sleep_s" + attempt=$((attempt + 1)) + continue + fi echo "$LABEL failed HTTP $HTTP_CODE" >&2 head -c 800 "$OUT_FILE.raw" >&2 || true echo "{\"error\":\"http\",\"model\":\"$MODEL\",\"provider\":\"$PROVIDER\",\"http\":$HTTP_CODE}" > "$OUT_FILE" exit 0 -fi +done python3 - "$OUT_FILE.raw" "$OUT_FILE" "$MODEL" "$PROVIDER" "$EXTRACT" <<'PY' import json,sys @@ -123,7 +134,6 @@ if kind == "anthropic": if c.get("type") == "text" ) elif kind == "openai-responses": - # Best-effort extract from Responses API text = raw.get("output_text") or "" if not text: chunks = [] diff --git a/scripts/merge-findings.js b/scripts/merge-findings.js index 09bb741..0a1ed77 100644 --- a/scripts/merge-findings.js +++ b/scripts/merge-findings.js @@ -17,6 +17,7 @@ const dir = args.dir; const floor = args.floor || "low"; const agreementMin = Number(args["agreement-min"] || 2); const scanHardFail = args["scan-hard-fail"] === "true"; +const language = (args.language || process.env.GATEKEEP_LANGUAGE || "en").trim() || "en"; const outPath = args.out; const tierRank = { low: 0, medium: 1, high: 2 }; @@ -166,10 +167,22 @@ for (const m of models) { } } +// Prefer successful model tiers; only if every call failed, fail-closed high. +const okModels = models.filter((m) => !m.error); +const modelErrors = models.filter((m) => m.error).length; +const degraded = modelErrors > 0 || okModels.length < agreementMin; + +// When some models 429/fail, lower the bar to surface usable findings from survivors +// (still require at least 1 successful model). Full fail-closed high only if zero ok. +let effectiveAgreement = agreementMin; +if (okModels.length > 0 && okModels.length < agreementMin) { + effectiveAgreement = Math.max(1, okModels.length); +} + const agreed = []; const disputed = []; for (const f of bucket.values()) { - if (f.models.length >= agreementMin) agreed.push(f); + if (f.models.length >= effectiveAgreement) agreed.push(f); else disputed.push(f); } @@ -177,8 +190,10 @@ const bySeverity = (a, b) => (sevRank[a.severity] ?? 9) - (sevRank[b.severity] ? agreed.sort(bySeverity); disputed.sort(bySeverity); +const tierSources = okModels.length ? okModels : models; let risk_tier = floor; -for (const m of models) risk_tier = maxTier(risk_tier, m.risk_tier); +for (const m of tierSources) risk_tier = maxTier(risk_tier, m.risk_tier); +if (!okModels.length) risk_tier = maxTier(risk_tier, "high"); for (const f of agreed) { if (f.kind === "security" || f.severity === "Critical") risk_tier = maxTier(risk_tier, "high"); else if (f.severity === "High" || f.tag === "Important") risk_tier = maxTier(risk_tier, "medium"); @@ -192,6 +207,11 @@ const result = { risk_tier, policy_floor: floor, agreement_min: agreementMin, + effective_agreement_min: effectiveAgreement, + language, + degraded, + model_errors: modelErrors, + models_ok: okModels.length, models: models.map((m) => ({ id: m.model, risk_tier: m.risk_tier, diff --git a/scripts/publish.js b/scripts/publish.js index 45c0c03..6bac033 100644 --- a/scripts/publish.js +++ b/scripts/publish.js @@ -2,6 +2,7 @@ /** * Publish Gatekeep summary + agreed inline comments. * Forge-agnostic (GitHub + Gitea) via fetch — no gh CLI required. + * Report language from merged.language / GATEKEEP_LANGUAGE (zh-CN, en, …). */ import { readFileSync, writeFileSync } from "node:fs"; @@ -24,8 +25,133 @@ const checkName = "gatekeep / security"; const apiBase = (process.env.GITHUB_API_URL || "https://api.github.com").replace(/\/$/, ""); const gitea = !apiBase.includes("api.github.com"); +const language = ( + merged.language || + process.env.GATEKEEP_LANGUAGE || + "en" +) + .trim() + .toLowerCase(); +const zh = language.startsWith("zh"); + const SEV_EMOJI = { Critical: "🔴", High: "🟠", Medium: "🟡", Low: "🟢" }; +const T = zh + ? { + title: "## Gatekeep 审查", + intro: "多模型并行审查结果如下。**先看严重级别**,Critical → Low 几秒扫完。", + risk: "PR 风险档", + floor: "策略底线", + agreement: "一致门槛", + agreedDisputed: "一致 / 分歧", + critHigh: "Critical / High(一致)", + modelsHdr: "### 模型(并行)", + modelCol: "模型", + tierCol: "其 PR 档", + findingsCol: "结论数", + agreedHdr: "### 一致结论", + agreedHelp: + "按严重级别排序。每条都写清 **严重级别**、**位置**、**为什么重要**。\n修复,或回复 `dismissed: <原因>` 后 resolve 讨论串。Gatekeep **没有**聊天 / @ 命令入口。", + agreedEmpty: "_模型之间没有达成一致的结论 — 可能很好,也可能该看分歧。_", + disputedHdr: "### 分歧(仅摘要,不上 inline)", + disputedHelp: "只有部分模型看到。有参考价值,本身不挡合并。", + disputedEmpty: "_没有值得单独列出的分歧。_", + scanHdr: "### 确定性安全扫描", + scanEmpty: "_无硬失败命中。_", + policyHdr: "### 策略底线信号", + policyNone: "- 无", + humansHdr: "### 仍需人工", + humansDefault: "- 品味、架构、计费/CI/鉴权、读证据", + footer: + "漏了真问题?开 **gatekeep-incident** issue — 这样才能变成 eval。\n_仅在模型一致时发 inline。PR 档失败关闭。结论以严重级别 + 位置 + 原因为先。_", + where: "位置", + why: "为什么重要", + whyFallback: "未写清影响 — 驳回前请先补一句。", + modelsLabel: "模型", + pre: " · 既有", + block: " · 阻断", + dispute: " · 分歧(未上 inline)", + preInline: "(既有)", + inlineFooter: (models) => + `_由 ${models} 一致认定。请修复,或回复 \`dismissed: <原因>\` 后 resolve。_\n_无聊天命令 — Gatekeep 不响应 @。_`, + reviewBody: "Gatekeep 一致结论(严重级别 · 位置 · 原因)。请修复或驳回 — 无 @ 命令。", + fileLevel: (path) => `_文件级(无线号)于 \`${path}\`。_`, + noLoc: "(无位置)", + approveLow: + "看起来风险低 — Gatekeep + 安全绿灯后可不需人工(仅当 kill switch 打开才可自动合并)。", + approveMed: "需要 **1 名人工**过目。Bot 点赞不算人工。", + approveHigh: "需要 **2 名人工**批准。Bot 不能顶替。", + errNote: (n, ok, degAgree) => + `> **降级说明:** ${n} 个模型调用失败(常见免费档限流 429;已重试/尝试备用模型)。` + + (ok === 0 + ? `全部失败 → 风险档**失败关闭为 high**,**不等于**代码一定有问题。` + : `已用 ${ok} 个成功响应继续审查` + + (degAgree ? `(一致门槛临时降为 ${degAgree})` : "") + + `;失败关闭的 high **仅在零成功响应时**强制抬升。`), + statusBlock: "存在未关闭的安全 / Critical 结论 — 合并阻断", + statusOk: (tier) => `PR 风险 ${tier} — 无未关闭安全阻断`, + modelsAgree: "一致门槛 ≥", + } + : { + title: "## Gatekeep review", + intro: + "Hey — here's the multi-model pass. **Severity first** so you can skim Critical → Low in seconds.", + risk: "PR risk tier", + floor: "Policy floor", + agreement: "Agreement bar", + agreedDisputed: "Agreed / disputed", + critHigh: "Critical / High (agreed)", + modelsHdr: "### Models (parallel)", + modelCol: "Model", + tierCol: "Their PR tier", + findingsCol: "Findings", + agreedHdr: "### Agreed findings", + agreedHelp: + "Sorted by severity. Each one spells out **severity**, **where**, and **why it matters**.\nFix or dismiss with `dismissed: ` (then resolve the thread). Gatekeep has **no** chat / @-command surface.", + agreedEmpty: + "_Nothing the models agreed on — nice, or spooky. Glance at disputed below._", + disputedHdr: "### Disputed (summary only — not posted inline)", + disputedHelp: + "One model saw it; others didn't. Useful signal, not a merge blocker by itself.", + disputedEmpty: "_No disagreements worth parking._", + scanHdr: "### Deterministic security scan", + scanEmpty: "_No hard-fail hits._", + policyHdr: "### Policy floor signals", + policyNone: "- none", + humansHdr: "### Still on humans", + humansDefault: "- taste, architecture, billing/CI/auth, reading the evidence", + footer: + "Missed something real? Open a **gatekeep-incident** issue — that's how it becomes an eval.\n_Inline comments only when models agree. PR tier is fail-closed. Findings lead with severity + location + why._", + where: "Where", + why: "Why it matters", + whyFallback: "Impact not spelled out — please clarify before dismissing.", + modelsLabel: "Models", + pre: " · pre-existing", + block: " · blocking", + dispute: " · disputed (not inlined)", + preInline: " (pre-existing)", + inlineFooter: (models) => + `_Agreed by ${models}. Fix it, or reply \`dismissed: \` and resolve the thread._\n_No chat commands — Gatekeep doesn't take @-mentions._`, + reviewBody: + "Gatekeep agreed findings (severity · where · why). Fix or dismiss — no @-commands.", + fileLevel: (path) => `_File-level (no single line) on \`${path}\`._`, + noLoc: "(no location)", + approveLow: + "Looks low-risk — no human needed once Gatekeep + security are green (auto-merge only if the kill switch is on).", + approveMed: "Needs **1 human** look. Bot thumbs-up don't count as that look.", + approveHigh: "Needs **2 human** approvals. Bots never fill those seats.", + errNote: (n, ok, degAgree) => + `> **Degraded:** ${n} model call(s) failed (often free-tier 429; retried / tried fallbacks). ` + + (ok === 0 + ? `Zero usable responses → risk tier **fail-closed high** (not proof the code is bad).` + : `Reviewing from ${ok} successful response(s)` + + (degAgree ? ` (agreement bar temporarily ${degAgree})` : "") + + `; fail-closed high is forced only when **zero** models succeed.`), + statusBlock: "Open security / Critical findings — merge blocked", + statusOk: (tier) => `PR risk ${tier} — no open security blockers`, + modelsAgree: "≥", + }; + async function forgeApi(method, path, body) { const p = path.startsWith("/") ? path : `/${path}`; const url = `${apiBase}${p}`; @@ -48,50 +174,46 @@ async function forgeApi(method, path, body) { } function approvals(tier) { - if (tier === "low") - return "Looks low-risk — no human needed once Gatekeep + security are green (auto-merge only if the kill switch is on)."; - if (tier === "medium") return "Needs **1 human** look. Bot thumbs-up don't count as that look."; - return "Needs **2 human** approvals. Bots never fill those seats."; + if (tier === "low") return T.approveLow; + if (tier === "medium") return T.approveMed; + return T.approveHigh; } function loc(f) { - return f.location || (f.path ? `${f.path}${f.line ? `:${f.line}` : ""}` : "(no location)"); + return f.location || (f.path ? `${f.path}${f.line ? `:${f.line}` : ""}` : T.noLoc); } function formatFindingBlock(f, { disputed = false } = {}) { const sev = f.severity || "Medium"; const emoji = SEV_EMOJI[sev] || "⚪"; - const pre = f.tag === "Pre-existing" ? " · pre-existing" : ""; - const block = f.blocking ? " · blocking" : ""; - const dispute = disputed ? " · disputed (not inlined)" : ""; + const pre = f.tag === "Pre-existing" ? T.pre : ""; + const block = f.blocking ? T.block : ""; + const dispute = disputed ? T.dispute : ""; const tagNote = f.tag && f.tag !== "Important" ? ` · ${f.tag}` : ""; - const why = f.why_it_matters || "Impact not spelled out — please clarify before dismissing."; + const why = f.why_it_matters || T.whyFallback; const detail = (f.body || "").trim(); const detailBit = detail && detail !== why ? `\n\n${detail}` : ""; return ( `#### ${emoji} ${sev}${pre}${block}${dispute}${tagNote}: ${f.title}\n` + - `- **Where:** \`${loc(f)}\`\n` + - `- **Why it matters:** ${why}` + + `- **${T.where}:** \`${loc(f)}\`\n` + + `- **${T.why}:** ${why}` + detailBit + - `\n- _Models:_ ${f.models.join(", ")}` + `\n- _${T.modelsLabel}:_ ${f.models.join(", ")}` ); } function formatInline(f) { const sev = f.severity || "Medium"; const emoji = SEV_EMOJI[sev] || "⚪"; - const pre = f.tag === "Pre-existing" ? " (pre-existing)" : ""; - const why = f.why_it_matters || "Impact not spelled out — please clarify before dismissing."; + const pre = f.tag === "Pre-existing" ? T.preInline : ""; + const why = f.why_it_matters || T.whyFallback; const detail = (f.body || "").trim(); let out = `### ${emoji} ${sev}${pre} — ${f.title}\n\n` + - `**Where:** \`${loc(f)}\`\n\n` + - `**Why it matters:** ${why}\n`; + `**${T.where}:** \`${loc(f)}\`\n\n` + + `**${T.why}:** ${why}\n`; if (detail && detail !== why) out += `\n${detail}\n`; - out += - `\n---\n` + - `_Agreed by ${f.models.join(", ")}. Fix it, or reply \`dismissed: \` and resolve the thread._\n` + - `_No chat commands — Gatekeep doesn't take @-mentions._`; + out += `\n---\n` + T.inlineFooter(f.models.join(", ")); return out; } @@ -100,12 +222,12 @@ const sortedDisputed = [...(merged.disputed_findings || [])]; const agreedList = sortedAgreed.length === 0 - ? "_Nothing the models agreed on — nice, or spooky. Glance at disputed below._" + ? T.agreedEmpty : sortedAgreed.map((f) => formatFindingBlock(f)).join("\n\n"); const disputedList = sortedDisputed.length === 0 - ? "_No disagreements worth parking._" + ? T.disputedEmpty : sortedDisputed.map((f) => formatFindingBlock(f, { disputed: true })).join("\n\n"); const modelRows = merged.models @@ -118,49 +240,64 @@ const modelRows = merged.models const critCount = sortedAgreed.filter((f) => f.severity === "Critical").length; const highCount = sortedAgreed.filter((f) => f.severity === "High").length; +const errCount = Number(merged.model_errors || 0); +const okCount = Number( + merged.models_ok != null + ? merged.models_ok + : (merged.models || []).filter((m) => !m.error).length, +); +const effAgree = merged.effective_agreement_min; +const degAgree = + merged.degraded && effAgree != null && effAgree < merged.agreement_min ? effAgree : null; +const errBanner = + errCount > 0 || merged.degraded ? `\n${T.errNote(errCount, okCount, degAgree)}\n` : ""; + +const agreementBar = zh + ? `${T.modelsAgree}${merged.agreement_min} 个模型` + + (degAgree ? `(本轮降为 ${degAgree})` : "") + : `≥${merged.agreement_min} models` + (degAgree ? ` (this run: ${degAgree})` : ""); + const body = `${marker} -## Gatekeep review - -Hey — here's the multi-model pass. **Severity first** so you can skim Critical → Low in seconds. +${T.title} +${T.intro} +${errBanner} | | | | --- | --- | -| **PR risk tier** | \`${merged.risk_tier}\` | -| Policy floor | \`${merged.policy_floor}\` | -| Agreement bar | ≥${merged.agreement_min} models | -| Agreed / disputed | ${sortedAgreed.length} / ${sortedDisputed.length} | -| Critical / High (agreed) | ${critCount} / ${highCount} | +| **${T.risk}** | \`${merged.risk_tier}\` | +| ${T.floor} | \`${merged.policy_floor}\` | +| ${T.agreement} | ${agreementBar} | +| ${T.agreedDisputed} | ${sortedAgreed.length} / ${sortedDisputed.length} | +| ${T.critHigh} | ${critCount} / ${highCount} | ${approvals(merged.risk_tier)} -### Models (parallel) -| Model | Their PR tier | Findings | +${T.modelsHdr} +| ${T.modelCol} | ${T.tierCol} | ${T.findingsCol} | | --- | --- | --- | ${modelRows} -### Agreed findings -Sorted by severity. Each one spells out **severity**, **where**, and **why it matters**. -Fix or dismiss with \`dismissed: \` (then resolve the thread). Gatekeep has **no** chat / @-command surface. +${T.agreedHdr} +${T.agreedHelp} ${agreedList} -### Disputed (summary only — not posted inline) -One model saw it; others didn't. Useful signal, not a merge blocker by itself. +${T.disputedHdr} +${T.disputedHelp} ${disputedList} -### Deterministic security scan -${(scan.hits || []).length ? (scan.hits || []).map((h) => `- ${h}`).join("\n") : "_No hard-fail hits._"} +${T.scanHdr} +${(scan.hits || []).length ? (scan.hits || []).map((h) => `- ${h}`).join("\n") : T.scanEmpty} -### Policy floor signals -${(floor.reasons || []).map((r) => `- ${r}`).join("\n") || "- none"} +${T.policyHdr} +${(floor.reasons || []).map((r) => `- ${r}`).join("\n") || T.policyNone} -### Still on humans -${(merged.human_focus || []).map((h) => `- ${h}`).join("\n") || "- taste, architecture, billing/CI/auth, reading the evidence"} +${T.humansHdr} +${(merged.human_focus || []).map((h) => `- ${h}`).join("\n") || T.humansDefault} --- -Missed something real? Open a **gatekeep-incident** issue — that's how it becomes an eval. -_Inline comments only when models agree. PR tier is fail-closed. Findings lead with severity + location + why._ +${T.footer} `; async function main() { @@ -182,7 +319,6 @@ async function main() { .slice(0, 30) .map((f) => ({ path: f.path, - // Gitea uses old_position/new_position in some versions; line works on GitHub line: f.line, body: formatInline(f), })); @@ -192,11 +328,11 @@ async function main() { await forgeApi("POST", `/repos/${owner}/${name}/pulls/${pr.number}/reviews`, { commit_id: pr.head.sha, event: "COMMENT", - body: "Gatekeep agreed findings (severity · where · why). Fix or dismiss — no @-commands.", + body: T.reviewBody, comments: inline, }); } catch (e) { - console.error("Inline review failed; falling back to issue comments:", e.message); + console.error("inline review failed; falling back to issue comments:", e.message); for (const f of inline) { await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/comments`, { body: f.body + `\n\n_\`${f.path}:${f.line}\`_`, @@ -212,7 +348,7 @@ async function main() { f.path ) { await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/comments`, { - body: formatInline(f) + `\n\n_File-level (no single line) on \`${f.path}\`._`, + body: formatInline(f) + `\n\n${T.fileLevel(f.path)}`, }); } } @@ -231,7 +367,6 @@ async function main() { } } try { - // Gitea expects { labels: ["name"] }; GitHub same for issues API await forgeApi("POST", `/repos/${owner}/${name}/issues/${pr.number}/labels`, { labels: [want], }); @@ -242,10 +377,9 @@ async function main() { const conclusion = merged.open_security_blocking ? "failure" : "success"; const statusState = merged.open_security_blocking ? "failure" : "success"; const desc = merged.open_security_blocking - ? "Open security / Critical findings — merge blocked" - : `PR risk ${merged.risk_tier} — no open security blockers`; + ? T.statusBlock + : T.statusOk(merged.risk_tier); - // Prefer Checks API (GitHub); fall back to commit status (Gitea-friendly) try { await forgeApi("POST", `/repos/${owner}/${name}/check-runs`, { name: checkName, diff --git a/scripts/resolve-providers.py b/scripts/resolve-providers.py index 178b9e3..5ba6318 100644 --- a/scripts/resolve-providers.py +++ b/scripts/resolve-providers.py @@ -46,8 +46,14 @@ KNOWN_DEFAULTS = { } -def die(msg: str, code: int = 1) -> None: - print(f"gatekeep: {msg}", file=sys.stderr) +def _zh() -> bool: + lang = (os.environ.get("GATEKEEP_LANGUAGE") or "").strip().lower() + return lang.startswith("zh") + + +def die(msg: str, code: int = 1, msg_zh: str | None = None) -> None: + text = msg_zh if (_zh() and msg_zh) else msg + print(f"gatekeep: {text}", file=sys.stderr) sys.exit(code) @@ -57,7 +63,10 @@ def load_raw(path: Path) -> dict: try: import yaml # type: ignore except ImportError: - die("PyYAML required to load providers.yaml (pip install pyyaml)") + die( + "PyYAML required to load providers.yaml (pip install pyyaml)", + msg_zh="加载 providers.yaml 需要 PyYAML(请 pip install pyyaml 或使用 Action 的 venv 步骤)", + ) raw = yaml.safe_load(text) or {} else: raw = json.loads(text) @@ -109,16 +118,19 @@ def secret_value(key_name: str) -> str: return "" -def http_json(url: str, headers: dict) -> dict | list: +def http_json(url: str, headers: dict, timeout: float = 25) -> dict | list: req = urllib.request.Request(url, headers=headers, method="GET") - with urllib.request.urlopen(req, timeout=60) as resp: + with urllib.request.urlopen(req, timeout=timeout) as resp: return json.loads(resp.read().decode("utf-8")) def normalize_protocol(p: str) -> str: p = (p or "").strip().lower() if p not in PROTOCOL_WIRE: - die(f"unknown protocol {p!r} (want {' | '.join(PROTOCOLS)})") + die( + f"unknown protocol {p!r} (want {' | '.join(PROTOCOLS)})", + msg_zh=f"未知 protocol {p!r}(可选 {' | '.join(PROTOCOLS)})", + ) return p @@ -126,41 +138,72 @@ def wire_protocol(p: str) -> str: return PROTOCOL_WIRE[normalize_protocol(p)] -def discover_model_ids(base_url: str, protocol: str, api_key: str) -> set[str]: +def discover_model_ids(base_url: str, protocol: str, api_key: str) -> set[str] | None: + """Return model id set, or None if discovery failed after retries (skip validation).""" + import time + base = base_url.rstrip("/") wire = wire_protocol(protocol) - ids: set[str] = set() - try: - if wire in ("openai-completions", "openai-responses"): - data = http_json( - f"{base}/models", - {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}, + if wire not in ("openai-completions", "openai-responses", "anthropic"): + die(f"unhandled protocol wire {wire!r}") + + last_err: Exception | None = None + attempts = int(os.environ.get("GATEKEEP_DISCOVER_RETRIES", "3")) + for attempt in range(max(1, attempts)): + try: + ids: set[str] = set() + if wire in ("openai-completions", "openai-responses"): + data = http_json( + f"{base}/models", + { + "Authorization": f"Bearer {api_key}", + "Content-Type": "application/json", + }, + ) + for m in data.get("data") or []: + mid = m.get("id") + if mid: + ids.add(mid) + else: + data = http_json( + f"{base}/v1/models", + { + "x-api-key": api_key, + "anthropic-version": "2023-06-01", + "Content-Type": "application/json", + }, + ) + for m in data.get("data") or []: + mid = m.get("id") + if mid: + ids.add(mid) + return ids + except urllib.error.HTTPError as e: + body = e.read().decode("utf-8", errors="replace")[:400] + last_err = RuntimeError(f"HTTP {e.code} {body}") + # 401/403 are configuration errors — fail closed + if e.code in (401, 403): + die( + f"model discovery failed for {base}: {last_err}", + msg_zh=f"拉取模型列表失败({base}):{last_err} — 请检查 key", + ) + except Exception as e: + last_err = e + if attempt + 1 < attempts: + sleep_s = 2**attempt + print( + f"gatekeep: model discovery retry {attempt + 1}/{attempts} " + f"after {sleep_s}s ({last_err})", + file=sys.stderr, ) - for m in data.get("data") or []: - mid = m.get("id") - if mid: - ids.add(mid) - elif wire == "anthropic": - data = http_json( - f"{base}/v1/models", - { - "x-api-key": api_key, - "anthropic-version": "2023-06-01", - "Content-Type": "application/json", - }, - ) - for m in data.get("data") or []: - mid = m.get("id") - if mid: - ids.add(mid) - else: - die(f"unhandled protocol wire {wire!r}") - except urllib.error.HTTPError as e: - body = e.read().decode("utf-8", errors="replace")[:400] - die(f"model discovery failed for {base}: HTTP {e.code} {body}") - except Exception as e: - die(f"model discovery failed for {base}: {e}") - return ids + time.sleep(sleep_s) + + print( + f"gatekeep: model discovery failed for {base} after {attempts} tries " + f"({last_err}); skipping name validation and continuing", + file=sys.stderr, + ) + return None def enrich_provider(pid: str, pconf: dict) -> dict: @@ -189,6 +232,12 @@ def main() -> None: override_csv = sys.argv[3] if len(sys.argv) > 3 else "" cfg = load_raw(cfg_path) + # Resolve language early so die() can speak Chinese when configured + override_lang = (os.environ.get("GATEKEEP_LANGUAGE") or "").strip() + cfg_lang = str(cfg.get("language") or cfg.get("report_language") or "").strip() + language = override_lang or cfg_lang or "en" + os.environ["GATEKEEP_LANGUAGE"] = language + providers_in = cfg["providers"] providers: dict[str, dict] = { pid: enrich_provider(pid, pconf) for pid, pconf in providers_in.items() @@ -240,54 +289,74 @@ def main() -> None: requested = uniq if len(requested) < 2: - die(f"need 2–3 models for parallel review; got {len(requested)}") + die( + f"need 2–3 models for parallel review; got {len(requested)}", + msg_zh=f"并行审查至少需要 2–3 个模型,当前只有 {len(requested)} 个", + ) + fallback_requested: list[dict] = [] if len(requested) > 3: - print(f"gatekeep: truncating to first 3 of {len(requested)} models", file=sys.stderr) + print( + f"gatekeep: primary parallel set = first 3 of {len(requested)}; " + f"rest kept as fallbacks on 429/failure", + file=sys.stderr, + ) + fallback_requested = requested[3:] requested = requested[:3] provider_meta: dict[str, dict] = {} - for pid in {r["provider"] for r in requested}: + for pid in {r["provider"] for r in requested + fallback_requested}: pconf = providers[pid] key_name = str(pconf["key"]) api_key = secret_value(key_name) if not api_key: die( f"missing secret/env for provider {pid!r}: expected env var named {key_name!r} " - f"(from providers.*.key — free-form; set it on the workflow step env from secrets)." + f"(from providers.*.key — free-form; set it on the workflow step env from secrets).", + msg_zh=( + f"缺少供应商 {pid!r} 的密钥:需要环境变量 {key_name!r}" + f"(来自 providers.*.key;请在 workflow step 的 env 里从 secrets 注入)。" + ), ) available = discover_model_ids(pconf["base_url"], pconf["protocol"], api_key) - if not available: - die(f"provider {pid}: model list empty — check base_url/protocol/key") + if available is not None and len(available) == 0: + die( + f"provider {pid}: model list empty — check base_url/protocol/key", + msg_zh=f"供应商 {pid} 模型列表为空 — 请检查 base_url / protocol / key", + ) provider_meta[pid] = { "base_url": pconf["base_url"].rstrip("/"), "protocol": pconf["protocol"], "wire_protocol": wire_protocol(pconf["protocol"]), "key_name": key_name, "api_key": api_key, - "available_count": len(available), - "_available": available, + "available_count": len(available) if available is not None else -1, + "_available": available, # None ⇒ skip validation } - plan_models = [] - for r in requested: + def to_plan_entry(r: dict) -> dict: pid, model = r["provider"], r["model"] avail = provider_meta[pid]["_available"] - if model not in avail: + if avail is not None and model not in avail: sample = sorted(avail)[:8] die( f"model {model!r} not available on provider {pid} " - f"({provider_meta[pid]['available_count']} listed). Examples: {sample}" + f"({provider_meta[pid]['available_count']} listed). Examples: {sample}", + msg_zh=( + f"模型 {model!r} 在供应商 {pid} 上不可用" + f"(列表共 {provider_meta[pid]['available_count']} 个)。示例:{sample}" + ), ) - plan_models.append( - { - "provider": pid, - "model": model, - "base_url": provider_meta[pid]["base_url"], - "protocol": provider_meta[pid]["protocol"], - "wire_protocol": provider_meta[pid]["wire_protocol"], - "key_name": provider_meta[pid]["key_name"], - } - ) + return { + "provider": pid, + "model": model, + "base_url": provider_meta[pid]["base_url"], + "protocol": provider_meta[pid]["protocol"], + "wire_protocol": provider_meta[pid]["wire_protocol"], + "key_name": provider_meta[pid]["key_name"], + } + + plan_models = [to_plan_entry(r) for r in requested] + fallback_models = [to_plan_entry(r) for r in fallback_requested] secrets_path = out_path.with_suffix(".secrets.json") secrets_path.write_text( @@ -299,7 +368,9 @@ def main() -> None: "agreement_min": int(cfg.get("agreement_min", 2)), "temperature": cfg.get("temperature", 0), "max_tokens": int(cfg.get("max_tokens", 4096)), + "language": language, "models": plan_models, + "fallback_models": fallback_models, "secrets_file": str(secrets_path), "providers_used": sorted(provider_meta.keys()), } @@ -309,8 +380,10 @@ def main() -> None: { "providers": plan["providers_used"], "models": [f"{m['provider']}:{m['model']}" for m in plan_models], + "fallbacks": [f"{m['provider']}:{m['model']}" for m in fallback_models], "keys": {m["provider"]: m["key_name"] for m in plan_models}, "agreement_min": plan["agreement_min"], + "language": plan["language"], } ), flush=True, diff --git a/scripts/review.sh b/scripts/review.sh old mode 100644 new mode 100755 index 9916899..43d9263 --- a/scripts/review.sh +++ b/scripts/review.sh @@ -39,6 +39,9 @@ python3 "$ACTION/scripts/resolve-providers.py" \ "${GATEKEEP_MODELS:-}" AGREEMENT_MIN=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["agreement_min"])' "$PLAN") +# Prefer Action input / env override already applied in plan.language +export GATEKEEP_LANGUAGE +GATEKEEP_LANGUAGE=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("language") or "en")' "$PLAN") export GATEKEEP_MAX_TOKENS GATEKEEP_MAX_TOKENS=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("max_tokens",4096))' "$PLAN") SECRETS_FILE=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["secrets_file"])' "$PLAN") @@ -75,15 +78,25 @@ diff = (out / "diff.txt").read_text() if (out / "diff.txt").exists() else "" discover = json.loads((out / "discover.json").read_text()) if (out / "discover.json").exists() else {} plan = json.loads((out / "provider-plan.json").read_text()) -system = """You are one reviewer in Gatekeep, a multi-model PR review system for the repo owner's own projects. +language = (plan.get("language") or os.environ.get("GATEKEEP_LANGUAGE") or "en").strip() or "en" +lang_note = ( + f"LANGUAGE: Write ALL human-facing prose in {language} " + f"(summary, rationale, finding title / why_it_matters / body, human_focus). " + f"Keep JSON keys and enum tokens in English exactly as specified " + f"(risk_tier, severity Critical|High|Medium|Low, tag, kind)." +) + +system = f"""You are one reviewer in Gatekeep, a multi-model PR review system for the repo owner's own projects. Your job is useful review content that makes human triage cheap — not chat UX. +{lang_note} + Return ONLY valid JSON (no markdown fences) with: -{ +{{ "risk_tier": "low|medium|high", "summary": "short chatty overview", "rationale": "why this PR risk tier", - "findings": [{ + "findings": [{{ "severity": "Critical|High|Medium|Low", "path": "file path", "line": 1, @@ -94,14 +107,14 @@ Return ONLY valid JSON (no markdown fences) with: "why_it_matters": "concrete impact if ignored", "body": "optional extra detail in a friendly tone", "blocking": false - }], + }}], "human_focus": ["string"] -} +}} Finding rules (required on EVERY finding): 1. severity — Critical/High/Medium/Low so humans see critical vs minor instantly. (Distinct from PR risk_tier.) 2. exact location — path + line (use line_end for a range when needed). -3. why_it_matters — the impact/reason, not just \"please fix\". +3. why_it_matters — the impact/reason, not just "please fix". Tone: clear and human; structured fields, not robotic checklist-speak. tag is secondary (Important/Nit/Pre-existing); severity dominates. Prefer real harm over style nits. Drop pure style. kind=security or severity=Critical ⇒ blocking true. @@ -117,6 +130,7 @@ PR risk_tier rules: user = { "policy_floor": floor, "policy_floor_reasons": [r.strip() for r in reasons.split("|") if r.strip()], + "language": language, "discover": discover, "reviewers": [f"{m['provider']}:{m['model']}" for m in plan.get("models", [])], "pull_request": { @@ -133,11 +147,11 @@ user = { }, "diff": diff, } -json.dump({"system": system, "user": json.dumps(user, indent=2)}, open(out / "prompt.json", "w")) -print(f"Prompt ready; floor={floor}; plan={len(plan.get('models',[]))} models", flush=True) +json.dump({"system": system, "user": json.dumps(user, indent=2), "language": language}, open(out / "prompt.json", "w")) +print(f"Prompt ready; floor={floor}; language={language}; plan={len(plan.get('models',[]))} models", flush=True) PY -# Fire models in parallel (may be mixed providers) +# Fire primary models in parallel (may be mixed providers) pids=() i=0 while IFS= read -r line; do @@ -159,6 +173,52 @@ for pid in "${pids[@]}"; do done echo "Parallel model calls finished (wait_ec=$ec)" >&2 +# On partial failure, try fallback_models from providers.yaml (beyond the first 3) +python3 - "$MODELS_DIR" "$PLAN" <<'PY' >"$OUT/fallback-needed.txt" +import json, pathlib, sys +models_dir = pathlib.Path(sys.argv[1]) +plan = json.load(open(sys.argv[2])) +failed = ok = 0 +for p in models_dir.glob("model-*.json"): + try: + j = json.loads(p.read_text()) + except Exception: + failed += 1 + continue + if j.get("error") or j.get("parse_error"): + failed += 1 + else: + ok += 1 +fallbacks = plan.get("fallback_models") or [] +# Aim for ≥2 usable responses; if zero ok, try up to 3 reserves +if ok == 0: + need = min(len(fallbacks), 3) +elif ok < 2: + need = min(len(fallbacks), 2 - ok) +else: + need = 0 +print(need) +print(f"gatekeep: primary ok={ok} failed={failed} fallbacks={len(fallbacks)} need={need}", file=sys.stderr) +PY +NEED_FB=$(head -1 "$OUT/fallback-needed.txt" | tr -d '[:space:]') +NEED_FB=${NEED_FB:-0} +if [[ "$NEED_FB" -gt 0 ]]; then + echo "gatekeep: trying $NEED_FB fallback model(s) after partial failure" >&2 + fb_i=0 + while IFS= read -r line; do + fb_i=$((fb_i + 1)) + [[ "$fb_i" -gt "$NEED_FB" ]] && break + provider=$(printf '%s' "$line" | python3 -c 'import json,sys; print(json.load(sys.stdin)["provider"])') + model=$(printf '%s' "$line" | python3 -c 'import json,sys; print(json.load(sys.stdin)["model"])') + protocol=$(printf '%s' "$line" | python3 -c 'import json,sys; print(json.load(sys.stdin)["protocol"])') + base_url=$(printf '%s' "$line" | python3 -c 'import json,sys; print(json.load(sys.stdin)["base_url"])') + safe=$(echo "fb_${provider}_${model}" | tr '/:' '__') + bash "$ACTION/scripts/call-model.sh" \ + "$provider" "$model" "$protocol" "$base_url" \ + "$OUT/prompt.json" "$MODELS_DIR/model-fb$fb_i-$safe.json" "$SECRETS_FILE" || true + done < <(python3 -c 'import json,sys; [print(json.dumps(m)) for m in json.load(open(sys.argv[1])).get("fallback_models") or []]' "$PLAN") +fi + # Scrub secrets file from workspace artifact path if present rm -f "$SECRETS_FILE" @@ -167,6 +227,7 @@ node "$ACTION/scripts/merge-findings.js" \ --floor "$FLOOR" \ --agreement-min "$AGREEMENT_MIN" \ --scan-hard-fail "$SCAN_HARD" \ + --language "$GATEKEEP_LANGUAGE" \ --out "$OUT/merged.json" | tee "$OUT/merge-meta.json" set +e