Move pgbackrest-restore test to Kyverno Chainsaw by cbandy · Pull Request #4228 · CrunchyData/postgres-operator

cbandy · 2025-08-01T19:57:28Z

Checklist:

Have you added an explanation of what your changes do and why you'd like them to be included?
Have you updated or added documentation for the change, as applicable?
Have you tested your changes on all related environments with successful results, as applicable?
- Have you added automated tests?

Type of Changes:

Testing enhancement

Other Information:

This test failed in this other PR, and the KUTTL logs did not give me enough information to diagnose the cause. Chainsaw did much better and unblocked me over there.

This test has a number of scripts and jobs that pass and share data. Chainsaw's bindings and templates are a nice way to break this test up, and its "catch" operations provide good context when a step fails. Tested with Kyverno Chainsaw 0.2.12 See: https://kyverno.github.io/chainsaw/main

This helped me reproduce a race in "pg_ctl start" during Postgres recovery. Issue: PGO-1945

ValClarkson

LGTM

benjaminjb · 2025-08-01T20:30:52Z

+      use: { template: '21-lose-data.yaml' }
+
+    - name: 'Point-In-Time Restore'
+      use: { template: '22-point-in-time-restore.yaml' }


Is it worth it to regularize the file/template names? I always liked the numbered files because you can just read through a directory (well, could, before we started to use the subdir files), but if Chainsaw uses this file to sequence the tests, maybe another convention is worth talking about (at a later date!)

Definitely. I think I started with numbers close to the KUTTL ones to ease review/migration, but where I landed is not that!

I like the filenames going in a similar order as the test, too, but now they can get reused so... 🤷🏻

Maybe something more like "scenario one," "... two," 🤔 🌱

Yeah, if the templates are being reused (hopefully!), then putting them in order doesn't make sense. Maybe descriptive titles could make it easier to read?

benjaminjb · 2025-08-01T20:39:28Z

-        run: |
-          curl -Lo /usr/local/bin/kubectl-kuttl https://github.com/kudobuilder/kuttl/releases/download/v0.13.0/kubectl-kuttl_0.13.0_linux_x86_64
-          chmod +x /usr/local/bin/kubectl-kuttl
+


We don't need to install KUTTL any longer?

The Make targets do go run ...@latest by default. The binary download is faster, but not as easy to latest.

I'm on the fence. Do you have a preference?

Faster by how much? My guess is not that much, so this seems reasonable.

Checking the run times of the Run make check-kuttl && exit between this PR and another (with the older style), I see 1 worst case I wouldn't maybe want (abt 2 mins difference), but mostly I see 0-15 secs difference (and with these jobs, not sure where the time is really coming from).

benjaminjb · 2025-08-07T19:16:27Z

+
+This [chainsaw](https://github.com/kyverno/chainsaw) suite tests that CPK can clone and restore through pgBackRest backups.
+
+...other material here...


What else should we put here to explain this to ourselves when we come back to this?

Is there anything specific to these tests that wouldn't go in general chainsaw docs? 🤔

I can't think of anything right now and the original test didn't have anything, so I'll leave this for now as a placeholder.

sfc-gh-jmckulka

couple of questions but looks good!

sfc-gh-jmckulka · 2025-08-07T19:54:58Z

+
+This [chainsaw](https://github.com/kyverno/chainsaw) suite tests that CPK can clone and restore through pgBackRest backups.
+
+...other material here...


Is there anything specific to these tests that wouldn't go in general chainsaw docs? 🤔

sfc-gh-jmckulka · 2025-08-07T19:58:41Z

+apiVersion: chainsaw.kyverno.io/v1alpha2
+kind: Configuration
+metadata:
+  name: end-to-end


Do we need this file?

I think the info in this file, mainly labels and timeouts, can be defined per test 🤔

Yeah, and as needed, I think we might make that change. A lot of this will depend on the second test that gets added/what further development we do to add chainsaw tests.

sfc-gh-jmckulka · 2025-08-07T20:01:34Z

+      value: 'The name of the new PostgresCluster'
+
+  try:
+    -


What are we doing here? Why is there a newline after this - instead of - description: > on a single line 😕

I see that this is a template. Is it related to that?

Not sure why Chris started with this pattern, but it goes through all the files.

It works around a bug in the GitHub YAML rendering: https://www.github.com/atom/language-yaml/issues/109#issuecomment-645831505

sfc-gh-jmckulka · 2025-08-07T20:20:36Z

-          curl -Lo /usr/local/bin/kubectl-kuttl https://github.com/kudobuilder/kuttl/releases/download/v0.13.0/kubectl-kuttl_0.13.0_linux_x86_64
-          chmod +x /usr/local/bin/kubectl-kuttl
+
+      - run: |


This step is still named kuttl-k3d... should we make the name more generic?

cbandy added 3 commits August 1, 2025 14:54

Add a tablespace volume to the restore E2E test

ca9a5b6

Add a template to the restore E2E test

eb10022

This helped me reproduce a race in "pg_ctl start" during Postgres recovery. Issue: PGO-1945

ValClarkson approved these changes Aug 1, 2025

View reviewed changes

benjaminjb reviewed Aug 1, 2025

View reviewed changes

Comment thread testing/chainsaw/e2e/pgbackrest-restore/chainsaw-test.yaml

benjaminjb reviewed Aug 1, 2025

View reviewed changes

Comment thread testing/chainsaw/e2e/pgbackrest-restore/change-parameters.yaml Outdated

benjaminjb added 2 commits August 6, 2025 14:04

Is this easier to read?

3391817

adjust some verbiage, fix

876c9ef

benjaminjb reviewed Aug 7, 2025

View reviewed changes

sfc-gh-jmckulka approved these changes Aug 7, 2025

View reviewed changes

benjaminjb added 2 commits August 8, 2025 12:02

Update README.md

a40d697

Update test.yaml

db20bde

benjaminjb merged commit fae895c into CrunchyData:main Aug 8, 2025
20 checks passed

cbandy mentioned this pull request Aug 11, 2025

Wait for one WAL archive after changing parameters #4235

Merged

5 tasks


		This [chainsaw](https://github.com/kyverno/chainsaw) suite tests that CPK can clone and restore through pgBackRest backups.

		...other material here...

Conversation

cbandy commented Aug 1, 2025

Uh oh!

ValClarkson left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cbandy Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sfc-gh-jmckulka left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cbandy Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cbandy Aug 1, 2025 •

edited

Loading

cbandy Aug 11, 2025 •

edited

Loading