[EPIC] Transaction Support Issues and Pull Requests

[jonathanc-n]

What's the feature are you trying to implement?

There is an accumulation of issues + PRs related to transactions which has made it quite difficult to follow on to the transaction support progress (there are duplicate pull requests, issues, etc. being made ). This is a list to help new contributors be aware of what is being worked on + reviewers have a easy place to access pull requests that can be looked at 😄:

Issues (without pull request)

• Write Support (EPIC): [EPIC] Iceberg-rust Write support #700
• Metadata Tables (EPIC): [EPIC] feat: support metadata tables #823
• Row Delta (row level modifications + concurrency): Support RowDeltaAction #1104
• Integration Tests for Transaction: Add integration tests for transaction APIs #1322
• Retrying Commits: Support commit retries #964
• Union by Name for Schema: Update a TableSchema from a Schema #698
• Initial Default: Implement initial-default #737 (waiting for MergeAppend)
• Validate in SnapshotProduceAction: Support validate and related logic in Snapshot #1344

Pull Requests 🚀

• Position Delete Writer: feat: support position delete writer #704
• Support Entries Metadata Table: feat: Support metadata table "Entries"  #863 (currently blocking metadata table support)
• MergeAppend: feat: support merge append action #902
• Scan Delete Support: Scan Delete Support Part 4: Delete File Loading; Skeleton for Processing #982 + Scan Delete Support Part 5: Positional Delete Parsing #1011 + Scan Delete Support Part 6: Equality Delete Parsing #1017
• Refine Record Batches: feat: refine record batch project #1014
• Partition Splitter: Partition Writer Support Part 1: add partition splitter #1040
• Partition Stats: feat: Support PartitionStats #1111
• Update Schema API: feat: Add basic operations for UpdateSchema #1172
• PartitionMetricsEvaluator: (build up towards Row Delta): feat: Add basic support for PartitionAndMetricsEvaluator #1189
• Rewrite Manifest Action: feat: support rewrite manifest action #1237
• Branch Update Actions: feat: Support branch actions using ManageSnapshots #1298
• Name Mapping: feat: Add IndexByName and IndexById to Namemapping #1299
• Retrieving Latest Table (refresh): Support retrieving the latest Iceberg table on table scan #1297
• Updating Summary Properties: feat: Introduce snapshot summary properties #1336
• Update Table in Catalog: feat: sql catalog support update table #862
• Update statistics: feat(transaction): Add UpdateStatisticsAction #1359
• Snapshot validation: feat(transaction): Support snapshot validation #1353

I have listed out some issues that are dependant on other pull requests. I will be maintaining this as issues + pull requests are being made, feel free to let me know if I need to add something!

[Xuanwo]

Thank you @jonathanc-n for gathering this information. I’ll begin reviewing the pending PRs.

[liurenjie1024]

Thanks @jonathanc-n for this summary. Transaction support is a somewhat large topic. Except reviewing prs, we also need to have a design to split it into smaller issues, so that the community could make progress on it. I'm do the planning recently.

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'