Skip to content

try building docker image w/ Github URL #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kevinjqliu wants to merge 2 commits into
base: main
Choose a base branch
from
Open

try building docker image w/ Github URL #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5c5a635
fix relative pathing
kevinjqliu Mar 11, 2026
7d6daba
build using github!!
kevinjqliu Mar 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion dev/docker-compose-integration.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@
services:
spark-iceberg:
image: pyiceberg-spark:latest
build: spark/
build:
context: https://github.com/kevinjqliu/iceberg-python.git#kevinjqliu/build-docker-from-github
dockerfile: dev/spark/Dockerfile
Comment on lines +21 to +23
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Building the Docker image from a personal GitHub repository URL introduces a significant risk to the project's stability and reproducibility. This configuration makes the build process dependent on an external fork (kevinjqliu/iceberg-python) and a specific branch that is outside the control of this project's maintainers. If the repository is removed, made private, or the branch is altered, the build will fail. It's strongly recommended to use a local build context to ensure that the build relies only on the code within this repository.

    build: spark/

container_name: pyiceberg-spark
networks:
iceberg_net:
Expand Down
2 changes: 1 addition & 1 deletion dev/spark/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ RUN set -e && \
done

# Copy configuration last (changes more frequently than JARs)
COPY --chown=spark:spark spark-defaults.conf ${SPARK_HOME}/conf/
COPY --chown=spark:spark dev/spark/spark-defaults.conf ${SPARK_HOME}/conf/
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This path change is coupled with the modification in docker-compose-integration.yml to build from a Git URL. While it makes that specific build work, it breaks the ability to build this Docker image locally using a command like docker build dev/spark, as the path dev/spark/spark-defaults.conf will not be found within the dev/spark build context. This change should be reverted along with the change in docker-compose-integration.yml to maintain support for local builds.

COPY --chown=spark:spark spark-defaults.conf ${SPARK_HOME}/conf/


USER spark
WORKDIR ${SPARK_HOME}
Expand Down
Loading