You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Xuchen Yao <ya...@gmail.com> on 2017/02/10 19:18:04 UTC
Getting exit code of pipe()
Hello Community,
I have the following Python code that calls an external command:
rdd.pipe('run.sh', env=os.environ).collect()
run.sh can either exit with status 1 or 0, how could I get the exit code
from Python? Thanks!
Xuchen
Re: Getting exit code of pipe()
Posted by Felix Cheung <fe...@hotmail.com>.
I mean if you are running a script instead of exiting with a code it could print out something.
Sounds like checkCode is what you want though.
_____________________________
From: Xuchen Yao <ya...@gmail.com>>
Sent: Sunday, February 12, 2017 8:33 AM
Subject: Re: Getting exit code of pipe()
To: Felix Cheung <fe...@hotmail.com>>
Cc: <us...@spark.apache.org>>
Cool that's exactly what I was looking for! Thanks!
How does one output the status into stdout? I mean, how does one capture the status output of pipe() command?
On Sat, Feb 11, 2017 at 9:50 AM, Felix Cheung <fe...@hotmail.com>> wrote:
Do you want the job to fail if there is an error exit code?
You could set checkCode to True
spark.apache.org/docs/latest/api/python/pyspark.html?highlight=pipe#pyspark.RDD.pipe<http://spark.apache.org/docs/latest/api/python/pyspark.html?highlight=pipe#pyspark.RDD.pipe>
Otherwise maybe you want to output the status into stdout so you could process it individually.
_____________________________
From: Xuchen Yao <ya...@gmail.com>>
Sent: Friday, February 10, 2017 11:18 AM
Subject: Getting exit code of pipe()
To: <us...@spark.apache.org>>
Hello Community,
I have the following Python code that calls an external command:
rdd.pipe('run.sh', env=os.environ).collect()
run.sh can either exit with status 1 or 0, how could I get the exit code from Python? Thanks!
Xuchen
Re: Getting exit code of pipe()
Posted by Xuchen Yao <ya...@gmail.com>.
Cool that's exactly what I was looking for! Thanks!
How does one output the status into stdout? I mean, how does one capture
the status output of pipe() command?
On Sat, Feb 11, 2017 at 9:50 AM, Felix Cheung <fe...@hotmail.com>
wrote:
> Do you want the job to fail if there is an error exit code?
>
> You could set checkCode to True
> spark.apache.org/docs/latest/api/python/pyspark.html?
> highlight=pipe#pyspark.RDD.pipe
>
> Otherwise maybe you want to output the status into stdout so you could
> process it individually.
>
>
> _____________________________
> From: Xuchen Yao <ya...@gmail.com>
> Sent: Friday, February 10, 2017 11:18 AM
> Subject: Getting exit code of pipe()
> To: <us...@spark.apache.org>
>
>
>
> Hello Community,
>
> I have the following Python code that calls an external command:
>
> rdd.pipe('run.sh', env=os.environ).collect()
>
> run.sh can either exit with status 1 or 0, how could I get the exit code
> from Python? Thanks!
>
> Xuchen
>
>
>
Re: Getting exit code of pipe()
Posted by Felix Cheung <fe...@hotmail.com>.
Do you want the job to fail if there is an error exit code?
You could set checkCode to True
spark.apache.org/docs/latest/api/python/pyspark.html?highlight=pipe#pyspark.RDD.pipe<http://spark.apache.org/docs/latest/api/python/pyspark.html?highlight=pipe#pyspark.RDD.pipe>
Otherwise maybe you want to output the status into stdout so you could process it individually.
_____________________________
From: Xuchen Yao <ya...@gmail.com>>
Sent: Friday, February 10, 2017 11:18 AM
Subject: Getting exit code of pipe()
To: <us...@spark.apache.org>>
Hello Community,
I have the following Python code that calls an external command:
rdd.pipe('run.sh', env=os.environ).collect()
run.sh can either exit with status 1 or 0, how could I get the exit code from Python? Thanks!
Xuchen