Skip to content

Commit 8d37bec

Browse files
author
Anupam Yadav
committed
[SPARK-56374][BUILD] Add missing Arrow shade rule to SBT Connect client assembly
Add org.apache.arrow relocation to SparkConnectClient (jvm) and SparkConnectJdbc assembly shade rules in SparkBuild.scala, matching Maven's client pom.xml. Maven shades Arrow classes to org/sparkproject/org/apache/arrow/ in the Connect client JAR. SBT was missing this rule, leaving 998 Arrow classes unshaded at org/apache/arrow/ inside the assembly JAR. This is a small incremental step toward full SBT/Maven assembly parity (SPARK-56374). The Connect server assembly already matches Maven (identical class count and shaded namespaces). Other gaps (extra unshaded transitive dependencies in SBT client JARs, network-yarn native library renaming, CopyDependencies) are left for follow-up work.
1 parent 6bba551 commit 8d37bec

1 file changed

Lines changed: 2 additions & 0 deletions

File tree

project/SparkBuild.scala

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -926,6 +926,7 @@ object SparkConnectJdbc {
926926
ShadeRule.rename("io.netty.**" -> "org.sparkproject.connect.client.io.netty.@1").inAll,
927927
ShadeRule.rename("io.perfmark.**" -> "org.sparkproject.connect.client.io.perfmark.@1").inAll,
928928
ShadeRule.rename("org.codehaus.**" -> "org.sparkproject.connect.client.org.codehaus.@1").inAll,
929+
ShadeRule.rename("org.apache.arrow.**" -> "org.sparkproject.connect.client.org.apache.arrow.@1").inAll,
929930
ShadeRule.rename("android.annotation.**" -> "org.sparkproject.connect.client.android.annotation.@1").inAll
930931
),
931932

@@ -1007,6 +1008,7 @@ object SparkConnectClient {
10071008
ShadeRule.rename("io.netty.**" -> "org.sparkproject.connect.client.io.netty.@1").inAll,
10081009
ShadeRule.rename("io.perfmark.**" -> "org.sparkproject.connect.client.io.perfmark.@1").inAll,
10091010
ShadeRule.rename("org.codehaus.**" -> "org.sparkproject.connect.client.org.codehaus.@1").inAll,
1011+
ShadeRule.rename("org.apache.arrow.**" -> "org.sparkproject.connect.client.org.apache.arrow.@1").inAll,
10101012
ShadeRule.rename("android.annotation.**" -> "org.sparkproject.connect.client.android.annotation.@1").inAll
10111013
),
10121014

0 commit comments

Comments
 (0)