Adam-submit can't use outside spark master

[flangelier]

Problem

Adam-submit is using spark in the localhost, but if we want yo use another spark cluster, adam-submit is failing.

From what I found out, adam-submit (which is calling spark-submit) don't have ways to give args to the spark-submit.

This is the command i used :
(you need a spark master and a hdfs cluster to run it)

docker run --link=spark-master:spark-master --link=hdfs-namenode:hdfs-namenode gelog/adam \ 
adam-submit --master spark://spark-master:7077 \
transform \
hdfs://hdfs-namenode:9000/SRR062634.sam \
hdfs://hdfs-namenode:9000/SRR062634.adam

Solution

We cloud tweak the adam-submit script to pass an ENV variable when it call the spark-submit
The env var cloud be something like $SPARK_SUBMIT_ARGS with an empty default value but we cloud gave a value like

SPARK_SUBMIT_ARGS='--master spark://spark-master:7077'

and use the following command:

docker run -e "SPARK_SUBMIT_ARGS=--master spark://spark-master:7077" \
--link=spark-master:spark-master --link=hdfs-namenode:hdfs-namenode gelog/adam \ 
adam-submit \
transform \
hdfs://hdfs-namenode:9000/SRR062634.sam \
hdfs://hdfs-namenode:9000/SRR062634.adam

[flangelier]

Add

$SPARK_SUBMIT_ARGS \

after

"$SPARK_SUBMIT" \

in /usr/local/adam/bin/adam-submit