fix(ci): harden Submit docs URLs to IndexNow step (#833) #2540
Annotations
1 error and 1 warning
|
run-lint
You have commit messages with errors
⧗ input: fix(ci): harden Submit docs URLs to IndexNow step (#833)
* fix(ci): harden Submit docs URLs to IndexNow step
The existing step had three functional problems that made it silently
broken since PR #827 landed:
1. Hardcoded fallback key `9ded60b43b8b40578588fc42b04c3e5d` was the
stale key deleted from landing/public/ in keploy/landing#186 because
Bing was rejecting it with HTTP 403 UserForbiddedToAccessSite. Every
push-run since PR #827 has been hitting 403 in silence.
2. `continue-on-error: true` combined with a plain `echo` on failure
(no `::error::` annotation, no `exit 1`) meant the failing step
reported green in the Actions UI — no signal, no alerting.
3. No `keyLocation` in the payload, no CURL_EXIT capture, no filter for
legacy versioned docs or auto-generated tag pages (so ~800 URLs were
submitted, mostly low-value /tags/ and /docs/{1,2,3}.0.0/ pages that
robots.txt already disallows).
Changes:
- Default INDEXNOW_KEY to feeeddbe661a4686ba6e4d5cb663b2cb (the key
actually served from keploy.io/<key>.txt). Secret override still
works for future rotations.
- Add keyLocation to the payload.
- Filter sitemap URLs: drop /tags/ pages and legacy /docs/{1,2,3}.0.0/
before submitting — reduces from ~800 to ~206 URLs, all on supported
paths. Verified locally with a real POST, HTTP 200.
- set -euo pipefail, capture CURL_EXIT separately from HTTP_CODE, exit
non-zero with ::error:: on real failure, print response body +
troubleshooting block with keyLocation check and a ready-to-paste
reproducer. 2xx (including 202 accepted-pending-verification) is an
explicit exit 0.
- Keep continue-on-error: true so the IndexNow step never blocks the
deploy, but the ::error:: annotation now surfaces failures in the UI
instead of hiding them.
Dry-run locally against the live sitemap-derived URL list returned
HTTP 200 with 20 real docs URLs.
Signed-off-by: nehagup <15074229+nehagup@users.noreply.github.com>
* fix(ci): Copilot review — robust IndexNow parsing, notice level, correct filter rationale, mktemp+trap
Four fixes from the Copilot review on PR #833:
1. `|| true` on the URL extraction pipeline. Under `set -euo pipefail`,
if the filter chain (`grep -oP | grep -v | grep -vE | sort -u`)
produced no matches, grep's exit 1 would abort the step before the
`[ -z "$URLS" ]` skip branch could run. Trailing `|| true` makes the
empty-result path intentionally non-fatal and the skip branch is
now reachable.
2. `::warning::` → `::notice::` for the two non-error skip paths
(missing sitemap, no URLs after filter). Per repo guidelines, the
Actions UI shouldn't surface these as warnings; they're expected
no-op conditions.
3. Corrected the filter comment. The previous wording claimed legacy
versioned docs were excluded "because robots.txt disallows them and
they're marked noindex anyway", but the current static/robots.txt
on main doesn't have that disallow and there's no noindex rule for
those paths. Replaced with the real reason: keeping submissions
focused on the preferred current doc paths.
4. Replaced the fixed `/tmp/indexnow-resp.txt` path with a `mktemp`-
generated RESPONSE_FILE and a `trap ... EXIT` cleanup, so repeat
runs can't collide and stale data can't leak into later reads.
Dry-ran all three edge cases locally (all-filtered-out, normal URLs,
missing sitemap) + verified the trap fires on EXIT.
Signed-off-by: nehagup <15074229+nehagup@users.noreply.github.com>
* fix(ci): explicit jq check + quoted URL in troubleshooting reproducer
Two Copilot review follow-ups:
1. Guard the payload build under `set -euo pipefail`: explicitly check
`command -v jq` and the `jq` command's exit status, emitting a
`::error::` annotation on either failure before `exit 1`. Previously
a missing or broken jq would abort the step immediately without the
actionable error message, leaving an unannotated "process exited 1"
in the Actions UI.
2. Quote `$INDEXNOW_KEY_LOC` and the IndexNow API URL in
|
|
run-lint
Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/checkout@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Node.js 20 will be removed from the runner on September 16th, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|