feat(handler) : add format handler for BTRFS stream #1451
Merged
qkaiser merged 2 commits intoonekey-sec:mainfrom Apr 15, 2026
Merged
feat(handler) : add format handler for BTRFS stream #1451qkaiser merged 2 commits intoonekey-sec:mainfrom
qkaiser merged 2 commits intoonekey-sec:mainfrom
Conversation
49d95e3 to
3f84ef2
Compare
qkaiser
reviewed
Mar 25, 2026
fc491ff to
234f4d4
Compare
qkaiser
reviewed
Mar 27, 2026
Contributor
qkaiser
left a comment
There was a problem hiding this comment.
Don't forget unit tests. Add integration samples that exercise:
- different types of compression
- with prefix and suffix data
Contributor
|
@Corv0O don't forget to rebase on upstream main for CI checks to pass |
234f4d4 to
b1963e0
Compare
qkaiser
reviewed
Mar 31, 2026
b1963e0 to
dc37e20
Compare
dc37e20 to
1df9e4d
Compare
qkaiser
reviewed
Apr 14, 2026
Contributor
qkaiser
left a comment
There was a problem hiding this comment.
A few changes and we're good to go.
1df9e4d to
2a48143
Compare
2a48143 to
5bc4e1d
Compare
5bc4e1d to
c3612c9
Compare
These functions are implemented by BTRFS stream and we want to make sure they are properly implemented and secured against path traversal payloads.
Instead of relying on an external CRC32C package, I implemented the algorithm directly in Python. This manual implementation was necessary because existing libraries lacked the flexibility to support the specific requirement used by BTRFS stream. The checksum uses the Castagnoli reflected polynomial with a seed of 0x0 and no final XOR operation. crc32c and crcmod wasn't able to fullfill those requirements and calculate the right checksum.
c3612c9 to
93056e7
Compare
qkaiser
approved these changes
Apr 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#1372
A btrfs stream file is a binary format produced by btrfs send to transfer filesystem snapshots. It encodes filesystem operations as a sequence of TLV-based commands that btrfs receive replays to reconstruct the original snapshot.
Format
Stream header :
Magic : 62 74 72 66 73 2d 73 74 72 65 61 6d 00 (btrfs-stream\0)
Version : 1 or 2
After the stream header, the format consists of a sequence of commands replayed by btrfs receive. Each command has a fixed 10-byte header:
Command header :
Length of data : 4 bytes
Type of the command : 2 bytes
Crc32c : 4 bytes (use Castagnoli reflected polynomial, seed = 0x0, no final XOR)
Command payload (TLV attributes) :
Type : 2 bytes (PATH, UUID, ...)
Length : 2 bytes
Value : [length] bytes
The format has two modes:
This handler only handle Full send file
Sample :
setup
full send
you can verify the content of the btfrs stream with this command
btrfs receive --dump -f sample.bin[References]
https://btrfs.readthedocs.io/en/latest/dev/dev-send-stream.html#btrfs-send-c-encoded-write-25
https://formats.kaitai.io/btrfs_stream/
https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/Design_notes_on_Send/Receive.html