Skip to content

Improvements:

Ottomata requested to merge T309227_improvements into main

  • .bumpversion.cfg should be able to change PACKAGE_VERSION in Dockerfile

  • Specify all deps in one file: conda-environemnt.yml

  • Add bin/spark3 scripts that exec ouut of the conda-analytics env

  • Improve find-spark-home so that it can be source directly. We will use this to aide building the spark yarn archive (AKA assembly file) - T310578

  • Refactor local Dockerfiles to use parallel multi stage build. This speeds up the build process, hopefully also speeding up the .deb development process.

Still TODO: synchronize the build changes in the Dockerfiles to the gitlab CI work being done in workflow_utils!28 (merged)

Edited by Ottomata

Merge request reports