Commit f861516
authored
fix: reconnect expired MCP backend sessions transparently, extend server session timeout (#2597)
Long-running agent workflows (~30+ min) fail when the gateway's HTTP
session to a backend (e.g. safeoutputs) expires mid-run. The agent
receives `session not found` from the backend and enters a futile retry
loop rather than recovering.
## Changes
### Client-side session reconnect (`internal/mcp/connection.go`,
`http_transport.go`)
- **Plain JSON-RPC transport**: `sendHTTPRequest` detects HTTP 404 +
`"session not found"` body, calls `reconnectPlainJSON()` to
re-initialize the session, then retries the original request once.
- **SDK transports (streamable / SSE)**: `callSDKMethodWithReconnect()`
wraps all SDK method calls — on a `"session not found"` error it calls
`reconnectSDKTransport()` (closes the dead session, dials a new one) and
retries once. `SendRequestWithServerID` now routes through this wrapper
for SDK transports.
- **Thread safety**: `sessionMu sync.RWMutex` protects `httpSessionID`,
`session`, and `client`. All reads go through `getSDKSession()` /
`getHTTPSessionID()` (under `RLock`); reconnect functions hold the full
`Lock`.
```go
// Plain JSON-RPC: sendHTTPRequest now does this automatically
result, err := c.executeHTTPRequest(...)
if isSessionNotFoundHTTPResponse(result.StatusCode, result.ResponseBody) {
if reconnErr := c.reconnectPlainJSON(); reconnErr == nil {
result, err = c.executeHTTPRequest(...) // retry once
}
}
// SDK transports (streamable / SSE): via callSDKMethodWithReconnect
result, err := c.callSDKMethod(method, params)
if err != nil && isSessionNotFoundError(err) {
if reconnErr := c.reconnectSDKTransport(); reconnErr == nil {
result, err = c.callSDKMethod(method, params)
}
}
```
### Server-side idle timeout (`internal/server/transport.go`)
`SessionTimeout` increased from `30m → 2h` so the agent→gateway inbound
session doesn't expire during extended periods of no MCP activity (e.g.
a long `lake build`).
### Tests (`internal/mcp/http_transport_test.go`)
Added `TestSendHTTPRequest_ReconnectsOnSessionNotFound`,
`TestSendHTTPRequest_ReconnectFailure`,
`TestSendHTTPRequest_NoReconnectOnOtherErrors`, plus unit tests for
`isSessionNotFoundError` and `isSessionNotFoundHTTPResponse`.
> [!WARNING]
>
> <details>
> <summary>Firewall rules blocked me from connecting to one or more
addresses (expand for details)</summary>
>
> #### I tried to connect to the following addresses, but was blocked by
firewall rules:
>
> - `example.com`
> - Triggering command: `/tmp/go-build2012957444/b330/launcher.test
/tmp/go-build2012957444/b330/launcher.test
-test.testlogfile=/tmp/go-build2012957444/b330/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true g_.a --local es
committer.email go` (dns block)
> - Triggering command: `/tmp/go-build916828748/b330/launcher.test
/tmp/go-build916828748/b330/launcher.test
-test.testlogfile=/tmp/go-build916828748/b330/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true
aw-mcpg/internal/config/rules/rules.go
aw-mcpg/internal/config/rules/rules_test.go
ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet . contextprotocol/checkout
--64 ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -I
Scom/odrQfnn_R7r_as4UScom -I
/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu-lang=go1.25 --gdwarf-5
--64 -o 2957444/b291/importcfg` (dns block)
> - Triggering command: `/tmp/go-build1482963661/b334/launcher.test
/tmp/go-build1482963661/b334/launcher.test
-test.testlogfile=/tmp/go-build1482963661/b334/testlog.txt
-test.paniconexit0 -test.timeout=10m0s --ve��
/home/REDACTED/work/gh-aw-mcpg/gh-aw-mcpg/internal/guard/context.go
/home/REDACTED/work/gh-aw-mcpg/gh-aw-mcpg/internal/guard/guard.go ash
b128.go s2rJ-u63W ache/go/1.25.8/x/tmp/go-build3142434740/b172/vet.cfg
nIQTNFClwl8R --ve��` (dns block)
> - `invalid-host-that-does-not-exist-12345.com`
> - Triggering command: `/tmp/go-build2012957444/b315/config.test
/tmp/go-build2012957444/b315/config.test
-test.testlogfile=/tmp/go-build2012957444/b315/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true -c=4 -nolocalimports
-importcfg /tmp/go-build2012957444/b289/importcfg -pack
/home/REDACTED/go/pkg/mod/golang.org/x/oauth2@v0.34.0/deviceauth.go
/home/REDACTED/go/pkg/mod/golang.org/x/oauth2@v0.34.0/oauth2.go ortc��
rLdqHevS3 64/src/internal/byteorder/byteor--gdwarf2 x_amd64/vet
pull.rebase abis` (dns block)
> - Triggering command: `/tmp/go-build588503108/b315/config.test
/tmp/go-build588503108/b315/config.test
-test.testlogfile=/tmp/go-build588503108/b315/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true
ache/go/1.25.8/x64/src/runtime/c-I` (dns block)
> - Triggering command: `/tmp/go-build1482963661/b319/config.test
/tmp/go-build1482963661/b319/config.test
-test.testlogfile=/tmp/go-build1482963661/b319/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -o d -n 10 -importcfg
2957444/b370=> -s -w -buildmode=exe /usr/bin/runc.original --ve�� it
tests..." -extld=gcc x_amd64/vet
64/src/runtime/c/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet
credential.helpe/tmp/go-build3142434740/b239/vet.cfg
ache/go/1.25.8/x-lang=go1.17 x_amd64/vet` (dns block)
> - `nonexistent.local`
> - Triggering command: `/tmp/go-build2012957444/b330/launcher.test
/tmp/go-build2012957444/b330/launcher.test
-test.testlogfile=/tmp/go-build2012957444/b330/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true g_.a --local es
committer.email go` (dns block)
> - Triggering command: `/tmp/go-build916828748/b330/launcher.test
/tmp/go-build916828748/b330/launcher.test
-test.testlogfile=/tmp/go-build916828748/b330/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true
aw-mcpg/internal/config/rules/rules.go
aw-mcpg/internal/config/rules/rules_test.go
ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet . contextprotocol/checkout
--64 ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -I
Scom/odrQfnn_R7r_as4UScom -I
/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu-lang=go1.25 --gdwarf-5
--64 -o 2957444/b291/importcfg` (dns block)
> - Triggering command: `/tmp/go-build1482963661/b334/launcher.test
/tmp/go-build1482963661/b334/launcher.test
-test.testlogfile=/tmp/go-build1482963661/b334/testlog.txt
-test.paniconexit0 -test.timeout=10m0s --ve��
/home/REDACTED/work/gh-aw-mcpg/gh-aw-mcpg/internal/guard/context.go
/home/REDACTED/work/gh-aw-mcpg/gh-aw-mcpg/internal/guard/guard.go ash
b128.go s2rJ-u63W ache/go/1.25.8/x/tmp/go-build3142434740/b172/vet.cfg
nIQTNFClwl8R --ve��` (dns block)
> - `slow.example.com`
> - Triggering command: `/tmp/go-build2012957444/b330/launcher.test
/tmp/go-build2012957444/b330/launcher.test
-test.testlogfile=/tmp/go-build2012957444/b330/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true g_.a --local es
committer.email go` (dns block)
> - Triggering command: `/tmp/go-build916828748/b330/launcher.test
/tmp/go-build916828748/b330/launcher.test
-test.testlogfile=/tmp/go-build916828748/b330/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true
aw-mcpg/internal/config/rules/rules.go
aw-mcpg/internal/config/rules/rules_test.go
ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet . contextprotocol/checkout
--64 ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -I
Scom/odrQfnn_R7r_as4UScom -I
/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu-lang=go1.25 --gdwarf-5
--64 -o 2957444/b291/importcfg` (dns block)
> - Triggering command: `/tmp/go-build1482963661/b334/launcher.test
/tmp/go-build1482963661/b334/launcher.test
-test.testlogfile=/tmp/go-build1482963661/b334/testlog.txt
-test.paniconexit0 -test.timeout=10m0s --ve��
/home/REDACTED/work/gh-aw-mcpg/gh-aw-mcpg/internal/guard/context.go
/home/REDACTED/work/gh-aw-mcpg/gh-aw-mcpg/internal/guard/guard.go ash
b128.go s2rJ-u63W ache/go/1.25.8/x/tmp/go-build3142434740/b172/vet.cfg
nIQTNFClwl8R --ve��` (dns block)
> - `this-host-does-not-exist-12345.com`
> - Triggering command: `/tmp/go-build2012957444/b339/mcp.test
/tmp/go-build2012957444/b339/mcp.test
-test.testlogfile=/tmp/go-build2012957444/b339/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true 64/src/runtime/cgo o
x_amd64/vet` (dns block)
> - Triggering command: `/tmp/go-build916828748/b339/mcp.test
/tmp/go-build916828748/b339/mcp.test
-test.testlogfile=/tmp/go-build916828748/b339/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -test.v=true 2957444/b308/_pkg_.a
/tmp/go-build2012957444/b165/
ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile . contextprotocol/-c
--64 ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -I
2957444/b320/_pkg_.a -I docker-buildx --gdwarf-5 b/gh-aw-mcpg/int--norc
-o docker-buildx` (dns block)
> - Triggering command: `/tmp/go-build1482963661/b343/mcp.test
/tmp/go-build1482963661/b343/mcp.test
-test.testlogfile=/tmp/go-build1482963661/b343/testlog.txt
-test.paniconexit0 -test.timeout=10m0s -o ssue -importcfg x_amd64/vet -s
-w -buildmode=exe x_amd64/vet --no�� --noprofile -extld=gcc x_amd64/vet
ternal/engine/wabash ternal/engine/wa/usr/bin/runc x_amd64/compile
x_amd64/vet` (dns block)
>
> If you need me to access, download, or install something from one of
these locations, you can either:
>
> - Configure [Actions setup
steps](https://gh.io/copilot/actions-setup-steps) to set up my
environment, which run before the firewall is enabled
> - Add the appropriate URLs or hosts to the custom allowlist in this
repository's [Copilot coding agent
settings](https://github.com/github/gh-aw-mcpg/settings/copilot/coding_agent)
(admins only)
>
> </details>
<!-- START COPILOT CODING AGENT TIPS -->
---
⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS
or Windows machine with [Raycast](https://gh.io/cca-raycast-docs).4 files changed
Lines changed: 427 additions & 43 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
71 | 72 | | |
72 | 73 | | |
73 | 74 | | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
74 | 96 | | |
75 | 97 | | |
76 | 98 | | |
| |||
255 | 277 | | |
256 | 278 | | |
257 | 279 | | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
258 | 369 | | |
259 | 370 | | |
260 | 371 | | |
| |||
301 | 412 | | |
302 | 413 | | |
303 | 414 | | |
304 | | - | |
| 415 | + | |
305 | 416 | | |
306 | 417 | | |
307 | 418 | | |
| |||
374 | 485 | | |
375 | 486 | | |
376 | 487 | | |
377 | | - | |
| 488 | + | |
378 | 489 | | |
379 | 490 | | |
380 | 491 | | |
| |||
429 | 540 | | |
430 | 541 | | |
431 | 542 | | |
432 | | - | |
| 543 | + | |
433 | 544 | | |
434 | 545 | | |
435 | 546 | | |
| |||
445 | 556 | | |
446 | 557 | | |
447 | 558 | | |
448 | | - | |
| 559 | + | |
449 | 560 | | |
450 | 561 | | |
451 | 562 | | |
| |||
455 | 566 | | |
456 | 567 | | |
457 | 568 | | |
458 | | - | |
| 569 | + | |
459 | 570 | | |
460 | 571 | | |
461 | 572 | | |
| |||
469 | 580 | | |
470 | 581 | | |
471 | 582 | | |
472 | | - | |
| 583 | + | |
473 | 584 | | |
474 | 585 | | |
475 | 586 | | |
| |||
478 | 589 | | |
479 | 590 | | |
480 | 591 | | |
481 | | - | |
| 592 | + | |
482 | 593 | | |
483 | 594 | | |
484 | 595 | | |
| |||
493 | 604 | | |
494 | 605 | | |
495 | 606 | | |
496 | | - | |
| 607 | + | |
497 | 608 | | |
498 | 609 | | |
499 | 610 | | |
| |||
504 | 615 | | |
505 | 616 | | |
506 | 617 | | |
507 | | - | |
508 | | - | |
| 618 | + | |
| 619 | + | |
509 | 620 | | |
510 | 621 | | |
511 | 622 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
65 | 83 | | |
66 | 84 | | |
67 | 85 | | |
| |||
436 | 454 | | |
437 | 455 | | |
438 | 456 | | |
439 | | - | |
440 | | - | |
441 | | - | |
442 | | - | |
443 | | - | |
444 | | - | |
445 | | - | |
446 | | - | |
447 | | - | |
448 | | - | |
449 | | - | |
450 | | - | |
451 | | - | |
452 | | - | |
453 | | - | |
454 | | - | |
455 | | - | |
456 | | - | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
457 | 466 | | |
458 | | - | |
| 467 | + | |
459 | 468 | | |
460 | 469 | | |
461 | | - | |
462 | | - | |
| 470 | + | |
| 471 | + | |
463 | 472 | | |
464 | 473 | | |
465 | | - | |
466 | 474 | | |
467 | 475 | | |
468 | 476 | | |
469 | 477 | | |
470 | 478 | | |
471 | | - | |
472 | | - | |
473 | | - | |
474 | 479 | | |
| 480 | + | |
475 | 481 | | |
476 | | - | |
477 | | - | |
478 | | - | |
479 | | - | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
480 | 487 | | |
481 | 488 | | |
482 | 489 | | |
483 | 490 | | |
484 | 491 | | |
485 | | - | |
| 492 | + | |
486 | 493 | | |
487 | | - | |
| 494 | + | |
488 | 495 | | |
489 | 496 | | |
490 | | - | |
| 497 | + | |
491 | 498 | | |
492 | 499 | | |
493 | 500 | | |
| |||
499 | 506 | | |
500 | 507 | | |
501 | 508 | | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
0 commit comments