You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Koert Kuipers <ko...@tresata.com> on 2016/02/03 06:35:04 UTC

make-distribution fails due to wrong order of modules

i am seeing make-distribution fail because lib_managed does not exist. what
seems to happen is that sql/hive module gets build and creates this
directory. but after this sometime later module spark-parent gets build,
which includes:

[INFO] Building Spark Project Parent POM 1.6.0-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.6.1:clean (default-clean) @
spark-parent_2.11 ---
[INFO] Deleting /home/koert/src/spark/target
[INFO] Deleting /home/koert/src/spark/lib_managed (includes = [], excludes
= [])

so after maven is done by the time make-distribution tries to access
lib_managed its gone.

now this is with a very slightly modified spark 1.6.0 build (one extra
<dependency> in module assembly). so i am guessing i caused it. anyone know
enough maven foo to tell me how to fix this? i think i need to get the
order of the modules fixed so spark-parent gets build before sql/hive?

thanks! koert