Re: PySpark Interpretor not working

[moon soo Lee <mo...@apache.org>]

Hi,

https://github.com/apache/incubator-zeppelin/pull/118
is recently merged and i think it helps configuring pyspark with Yarn.

Please try latest branch and let me know if it helps.

Thanks,
moon

On Tue, Jun 30, 2015 at 12:10 AM IT CTO <go...@gmail.com> wrote:

> I have the same configuration but when I run the program the process never
> returns. status remain on running
> log say SEND >> PROGRESS
> forever....
>
> On Sun, Jun 14, 2015 at 2:34 AM MrAsanjar . <af...@gmail.com> wrote:
>
>> hi
>> I had a similar issue, try these:
>> 1) add following settings to zeppelin-env.sh ( it must be added there at
>> this time)
>> export MASTER=yarn-client
>> export HADOOP_CONF_DIR=/etc/hadoop/conf
>> export
>> PYTHONPATH=/usr/lib/spark/python:/usr/lib/spark/python/lib/py4j-0.8.2.1-src.zip
>> export SPARK_YARN_USER_ENV="PYTHONPATH=${PYTHONPATH}"
>> export SPARK_HOME=/usr/lib/spark
>> 2) Make sure /tmp has 1777 access permission
>> 3) restart the daemon "zeppelin-daemon.sh restart"
>>
>> On Thu, Jun 11, 2015 at 2:13 PM, Marcel Hofmann <mh...@gmail.com>
>> wrote:
>>
>>> Hey everybody,
>>>
>>> I'm currently testing Zeppelin, but unfortunatley, I can't really get it
>>> up and running. The example notebook is running just fine, and everything
>>> works there, but a simple pyspark statement like:
>>>
>>> %pyspark
>>> list = range(1,4)
>>> print(list)
>>>
>>> will not execute. Looking at the interpretor-spark logs the only thing
>>> happening is:
>>>
>>>  INFO [2015-06-11 12:05:21,664] ({pool-2-thread-3}
>>> SchedulerFactory.java[jobStarted]:132) - Job
>>> remoteInterpretJob_1434049521664 started by scheduler interpreter_836706052
>>>  INFO [2015-06-11 12:05:21,667] ({pool-2-thread-3}
>>> PySparkInterpreter.java[createPythonScript]:123) - File
>>> /tmp/zeppelin_pyspark.py created
>>>
>>> I have set the following env-variables:
>>> export SPARK_HOME=/home/biadmin/spark-1.3.0/spark-1.3.0-bin-hadoop2.4
>>> export PYSPARK_PYTHON=python
>>> export
>>> PYTHONPATH=$SPARK_HOME/python:$SPARK_HOME/python/lib/py4j-0.8.2.1-src.zip
>>>
>>> Thanks for any advice!
>>>
>>> Marcel
>>>
>>>
>>