Add parallel distributed algorithms for segmented copy#7253
Add parallel distributed algorithms for segmented copy#7253arpittkhandelwal wants to merge 3 commits intoTheHPXProject:masterfrom
Conversation
Up to standards ✅🟢 Issues
|
There was a problem hiding this comment.
Pull request overview
This PR adds segmented (distributed-aware) implementations of copy and copy_if intended to execute across hpx::partitioned_vector segments/localities, and wires them into the segmented algorithms umbrella headers and public include entry point.
Changes:
- Added a new segmented algorithms header implementing
tag_invokeoverloads forhpx::copy_tandhpx::copy_if_t. - Integrated the new header into
hpx/parallel/segmented_algorithm.hppandhpx/include/parallel_copy.hpp. - Expanded the
partitioned_vector_copy.cppunit test to covercopy_if(sync/async, seq/par policies).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
libs/full/segmented_algorithms/tests/unit/partitioned_vector_copy.cpp |
Adds new copy_if tests for partitioned_vector (sync + task-based async). |
libs/full/segmented_algorithms/include/hpx/parallel/segmented_algorithms/copy.hpp |
New segmented implementations and CPO registrations for copy/copy_if. |
libs/full/segmented_algorithms/include/hpx/parallel/segmented_algorithm.hpp |
Includes the new segmented copy.hpp in the umbrella segmented algorithm header. |
libs/full/include/include/hpx/include/parallel_copy.hpp |
Exposes the segmented copy/copy_if implementation via the public include header. |
9b560be to
ac8a89e
Compare
ac8a89e to
4382751
Compare
4382751 to
85693a0
Compare
hkaiser
left a comment
There was a problem hiding this comment.
Implementing copy_if the way you propose will expose extremely bad performance. I'd rather not have an implementation at all than one that is slow and the user can't improve things. May I suggest removing the copy_if from this PR?
I agree a slow sequential fallback would be worse than nothing here. I've removed copy_if from this PR so we can just focus on getting segmented_copy merged cleanly |
|
The LLM got you (again) ;-) |
Description
This PR extends HPX's distributed algorithms library by implementing parallel, distributed-aware
segmented_copyalgorithm forhpx::partitioned_vector.Previously, applying these algorithms to a partitioned vector would fall back to sequential or non-segmented executions, which resulted in suboptimal performance and cross-node data transfer bottlenecks. This implementation directly maps the execution to the underlying segments (localities) using asynchronous dataflow execution and localized dispatching.
Key Changes:
libs/full/segmented_algorithms/include/hpx/parallel/segmented_algorithms/copy.hpp.tag_invokeoverloads for bothhpx::copy_tandhpx::copy_if_twithin thehpx::segmentednamespace.hpx/parallel/segmented_algorithm.hppand exposed them globally viahpx/include/parallel_copy.hpp.partitioned_vector_copy.cppto validate cross-node data movement, multi-partition behavior, and sequential/parallel policy integration for both standard and_asyncvariants.Testing:
ninja tests.unit.modules.segmented_algorithms.partitioned_vector_copy.hpx::execution::seq) and parallel (hpx::execution::par) execution policies.