You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Tyler Mitchell <Ty...@actian.com> on 2015/05/12 09:25:12 UTC
Recommended versions?
I've been using Python with Spark 1.3.0 and copy of Zeppelin from today, but I'm getting some weirdness. Before digging into it further I also tried Spark 1.3.1 but got other strange things (output of notes not showing anything).
So curious, what versions are recommended or preferred? I'm trying on OSX so far.
Thanks!
Tyler
Re: Recommended versions?
Posted by Marty B <sp...@mjhb.com>.
That does not work for me:
%pyspark
import pyspark_cassandra
(<type 'exceptions.ImportError'>, ImportError('No module named
pyspark_cassandra',), <traceback object at 0x7f9d16537290>)
Here is my conf/zeppelin-env.sh:
PSCC="/path/to/pyspark-cassandra/target"
export
ZEPPELIN_JAVA_OPTS="-Dspark.cassandra.connection.host=cassie
-Dspark.jars=$PSCC/pyspark_cassandra-0.1.4.jar"
export PYTHONPATH="$PSCC/pyspark_cassandra-0.1.3-py2.7.egg"
Interestingly, the logs indicate that they both loaded successfully:
Added JAR
file:/path/to/pyspark-cassandra/target/pyspark_cassandra-0.1.4.jar at
http://192.168.1.132:53081/jars/pyspark_cassandra-0.1.4.jar
Copying /path/to/pyspark-cassandra/target/pyspark_cassandra-0.1.4-py2.7.egg
to
/tmp/spark-11dd4dc8-6a78-4e9a-8073-d2ad2497f8ba/spark-8a841bb9-33c9-42a8-ac53-aab1e024d7d2/pyspark_cassandra-0.1.4-py2.7.egg
Added file
/path/to/pyspark-cassandra/target/pyspark_cassandra-0.1.4-py2.7.egg at
http://192.168.1.132:53081/files/pyspark_cassandra-0.1.4-py2.7.egg
On Tue, May 19, 2015 at 11:39 AM Jonathan Haddad <jo...@jonhaddad.com> wrote:
> export PYTHONPATH=/path/to/module.egg I think is what I was doing.
>
> On Tue, May 19, 2015 at 11:27 AM Marty B <sp...@mjhb.com> wrote:
>
>> Jon,
>>
>> How are you loading the Python module?
>>
>> The only way I could get it to load was to modify
>> spark/src/main/resources/python/zeppelin_pyspark.py and add it to the
>> SparkContext.
>>
>>
>> On Tue, May 19, 2015 at 10:43 AM Jonathan Haddad <jo...@jonhaddad.com>
>> wrote:
>>
>>> Yep, I believe that was the one.
>>>
>>> On Tue, May 19, 2015 at 10:41 AM Marty B <sp...@mjhb.com> wrote:
>>>
>>>> Jon,
>>>>
>>>> Is this the error message you see?
>>>>
>>>> (<class 'py4j.protocol.Py4JJavaError'>, Py4JJavaError(u'An error
>>>> occurred while calling o77.loadClass.\n', JavaObject id=o78), <traceback
>>>> object at 0x7f8949d124d0>)
>>>>
>>>> On Tue, May 19, 2015 at 9:49 AM Jonathan Haddad <jo...@jonhaddad.com>
>>>> wrote:
>>>>
>>>>> I believe was able to make this part work by pointing PYTHONPATH to
>>>>> the egg. The part I couldn't get working was the driver class path, etc.
>>>>> I'm not really a Java guy so it's a lot of stumbling around for me.
>>>>>
>>>>> On Tue, May 19, 2015 at 9:46 AM Marty B <sp...@mjhb.com> wrote:
>>>>>
>>>>>> The problem as I see it is that there appears to be no way to request
>>>>>> Zeppelin to load the .egg file (Python module containing PySpark-Cassandra
>>>>>> interface).
>>>>>>
>>>>>> I opened ZEPPELIN-71 for this, but have seen no progress other than
>>>>>> acknowledgement.
>>>>>>
>>>>>> Perhaps additional feedback in the JIRA ticket would help the
>>>>>> Zeppelin team with prioritization. :-)
>>>>>>
>>>>>> I hope to find time over the upcoming holiday weekend (US) to look
>>>>>> further into fixing it - my first attempt failed.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, May 19, 2015 at 9:21 AM Jonathan Haddad <jo...@jonhaddad.com>
>>>>>> wrote:
>>>>>>
>>>>>>> What I meant was I couldn't figure out how to get the Cassandra
>>>>>>> connector to work in pyspark. I had asked previously (including the flags
>>>>>>> I pass to pyspark) and gotten no response.
>>>>>>>
>>>>>>> If you've got a recommendation as to how to make the equivilent of
>>>>>>> this work in Zeppelin I'd love to give it a try:
>>>>>>>
>>>>>>> PYSPARK_ROOT=/Users/jhaddad/dev/pyspark-cassandra/target
>>>>>>> export PYSPARK_DRIVER_PYTHON=ipython
>>>>>>> PYSPARK_DRIVER_PYTHON_OPTS="notebook"
>>>>>>>
>>>>>>> pyspark \
>>>>>>> --jars ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>>>>>>> --driver-class-path ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>>>>>>> --py-files ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4-py2.7.egg \
>>>>>>> --conf spark.cassandra.connection.host=127.0.0.1 \
>>>>>>> --master spark://127.0.0.1:7077 \
>>>>>>>
>>>>>>> Jon
>>>>>>>
>>>>>>> On Tue, May 19, 2015 at 4:30 AM moon soo Lee <mo...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> First, thanks for trying Zeppelin.
>>>>>>>>
>>>>>>>> You guys mean pyspark is not working?
>>>>>>>> I'm developing zeppelin on OSX, so Zeppelin should work on it.
>>>>>>>>
>>>>>>>> If you can try latest code and post some log files, it'll be
>>>>>>>> helpful to investigate the problems.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> moon
>>>>>>>> On 2015년 5월 12일 (화) at 오후 4:39 Jonathan Haddad <
>>>>>>>> jonathan.haddad@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> How did you get it to work? I tried and failed miserably.
>>>>>>>>>
>>>>>>>>> Jon
>>>>>>>>>
>>>>>>>>> On May 12, 2015, at 12:25 AM, Tyler Mitchell <
>>>>>>>>> Tyler.Mitchell@actian.com> wrote:
>>>>>>>>>
>>>>>>>>> I've been using Python with Spark 1.3.0 and copy of Zeppelin
>>>>>>>>> from today, but I'm getting some weirdness. Before digging into it further
>>>>>>>>> I also tried Spark 1.3.1 but got other strange things (output of notes not
>>>>>>>>> showing anything).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So curious, what versions are recommended or preferred? I'm
>>>>>>>>> trying on OSX so far.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Tyler
>>>>>>>>>
>>>>>>>>>
Re: Recommended versions?
Posted by Jonathan Haddad <jo...@jonhaddad.com>.
export PYTHONPATH=/path/to/module.egg I think is what I was doing.
On Tue, May 19, 2015 at 11:27 AM Marty B <sp...@mjhb.com> wrote:
> Jon,
>
> How are you loading the Python module?
>
> The only way I could get it to load was to modify
> spark/src/main/resources/python/zeppelin_pyspark.py and add it to the
> SparkContext.
>
>
> On Tue, May 19, 2015 at 10:43 AM Jonathan Haddad <jo...@jonhaddad.com>
> wrote:
>
>> Yep, I believe that was the one.
>>
>> On Tue, May 19, 2015 at 10:41 AM Marty B <sp...@mjhb.com> wrote:
>>
>>> Jon,
>>>
>>> Is this the error message you see?
>>>
>>> (<class 'py4j.protocol.Py4JJavaError'>, Py4JJavaError(u'An error
>>> occurred while calling o77.loadClass.\n', JavaObject id=o78), <traceback
>>> object at 0x7f8949d124d0>)
>>>
>>> On Tue, May 19, 2015 at 9:49 AM Jonathan Haddad <jo...@jonhaddad.com>
>>> wrote:
>>>
>>>> I believe was able to make this part work by pointing PYTHONPATH to the
>>>> egg. The part I couldn't get working was the driver class path, etc. I'm
>>>> not really a Java guy so it's a lot of stumbling around for me.
>>>>
>>>> On Tue, May 19, 2015 at 9:46 AM Marty B <sp...@mjhb.com> wrote:
>>>>
>>>>> The problem as I see it is that there appears to be no way to request
>>>>> Zeppelin to load the .egg file (Python module containing PySpark-Cassandra
>>>>> interface).
>>>>>
>>>>> I opened ZEPPELIN-71 for this, but have seen no progress other than
>>>>> acknowledgement.
>>>>>
>>>>> Perhaps additional feedback in the JIRA ticket would help the Zeppelin
>>>>> team with prioritization. :-)
>>>>>
>>>>> I hope to find time over the upcoming holiday weekend (US) to look
>>>>> further into fixing it - my first attempt failed.
>>>>>
>>>>>
>>>>>
>>>>> On Tue, May 19, 2015 at 9:21 AM Jonathan Haddad <jo...@jonhaddad.com>
>>>>> wrote:
>>>>>
>>>>>> What I meant was I couldn't figure out how to get the Cassandra
>>>>>> connector to work in pyspark. I had asked previously (including the flags
>>>>>> I pass to pyspark) and gotten no response.
>>>>>>
>>>>>> If you've got a recommendation as to how to make the equivilent of
>>>>>> this work in Zeppelin I'd love to give it a try:
>>>>>>
>>>>>> PYSPARK_ROOT=/Users/jhaddad/dev/pyspark-cassandra/target
>>>>>> export PYSPARK_DRIVER_PYTHON=ipython
>>>>>> PYSPARK_DRIVER_PYTHON_OPTS="notebook"
>>>>>>
>>>>>> pyspark \
>>>>>> --jars ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>>>>>> --driver-class-path ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>>>>>> --py-files ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4-py2.7.egg \
>>>>>> --conf spark.cassandra.connection.host=127.0.0.1 \
>>>>>> --master spark://127.0.0.1:7077 \
>>>>>>
>>>>>> Jon
>>>>>>
>>>>>> On Tue, May 19, 2015 at 4:30 AM moon soo Lee <mo...@apache.org> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> First, thanks for trying Zeppelin.
>>>>>>>
>>>>>>> You guys mean pyspark is not working?
>>>>>>> I'm developing zeppelin on OSX, so Zeppelin should work on it.
>>>>>>>
>>>>>>> If you can try latest code and post some log files, it'll be helpful
>>>>>>> to investigate the problems.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> moon
>>>>>>> On 2015년 5월 12일 (화) at 오후 4:39 Jonathan Haddad <
>>>>>>> jonathan.haddad@gmail.com> wrote:
>>>>>>>
>>>>>>>> How did you get it to work? I tried and failed miserably.
>>>>>>>>
>>>>>>>> Jon
>>>>>>>>
>>>>>>>> On May 12, 2015, at 12:25 AM, Tyler Mitchell <
>>>>>>>> Tyler.Mitchell@actian.com> wrote:
>>>>>>>>
>>>>>>>> I've been using Python with Spark 1.3.0 and copy of Zeppelin
>>>>>>>> from today, but I'm getting some weirdness. Before digging into it further
>>>>>>>> I also tried Spark 1.3.1 but got other strange things (output of notes not
>>>>>>>> showing anything).
>>>>>>>>
>>>>>>>>
>>>>>>>> So curious, what versions are recommended or preferred? I'm
>>>>>>>> trying on OSX so far.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>
>>>>>>>> Tyler
>>>>>>>>
>>>>>>>>
Re: Recommended versions?
Posted by Marty B <sp...@mjhb.com>.
Jon,
How are you loading the Python module?
The only way I could get it to load was to modify
spark/src/main/resources/python/zeppelin_pyspark.py and add it to the
SparkContext.
On Tue, May 19, 2015 at 10:43 AM Jonathan Haddad <jo...@jonhaddad.com> wrote:
> Yep, I believe that was the one.
>
> On Tue, May 19, 2015 at 10:41 AM Marty B <sp...@mjhb.com> wrote:
>
>> Jon,
>>
>> Is this the error message you see?
>>
>> (<class 'py4j.protocol.Py4JJavaError'>, Py4JJavaError(u'An error occurred
>> while calling o77.loadClass.\n', JavaObject id=o78), <traceback object at
>> 0x7f8949d124d0>)
>>
>> On Tue, May 19, 2015 at 9:49 AM Jonathan Haddad <jo...@jonhaddad.com>
>> wrote:
>>
>>> I believe was able to make this part work by pointing PYTHONPATH to the
>>> egg. The part I couldn't get working was the driver class path, etc. I'm
>>> not really a Java guy so it's a lot of stumbling around for me.
>>>
>>> On Tue, May 19, 2015 at 9:46 AM Marty B <sp...@mjhb.com> wrote:
>>>
>>>> The problem as I see it is that there appears to be no way to request
>>>> Zeppelin to load the .egg file (Python module containing PySpark-Cassandra
>>>> interface).
>>>>
>>>> I opened ZEPPELIN-71 for this, but have seen no progress other than
>>>> acknowledgement.
>>>>
>>>> Perhaps additional feedback in the JIRA ticket would help the Zeppelin
>>>> team with prioritization. :-)
>>>>
>>>> I hope to find time over the upcoming holiday weekend (US) to look
>>>> further into fixing it - my first attempt failed.
>>>>
>>>>
>>>>
>>>> On Tue, May 19, 2015 at 9:21 AM Jonathan Haddad <jo...@jonhaddad.com>
>>>> wrote:
>>>>
>>>>> What I meant was I couldn't figure out how to get the Cassandra
>>>>> connector to work in pyspark. I had asked previously (including the flags
>>>>> I pass to pyspark) and gotten no response.
>>>>>
>>>>> If you've got a recommendation as to how to make the equivilent of
>>>>> this work in Zeppelin I'd love to give it a try:
>>>>>
>>>>> PYSPARK_ROOT=/Users/jhaddad/dev/pyspark-cassandra/target
>>>>> export PYSPARK_DRIVER_PYTHON=ipython
>>>>> PYSPARK_DRIVER_PYTHON_OPTS="notebook"
>>>>>
>>>>> pyspark \
>>>>> --jars ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>>>>> --driver-class-path ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>>>>> --py-files ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4-py2.7.egg \
>>>>> --conf spark.cassandra.connection.host=127.0.0.1 \
>>>>> --master spark://127.0.0.1:7077 \
>>>>>
>>>>> Jon
>>>>>
>>>>> On Tue, May 19, 2015 at 4:30 AM moon soo Lee <mo...@apache.org> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> First, thanks for trying Zeppelin.
>>>>>>
>>>>>> You guys mean pyspark is not working?
>>>>>> I'm developing zeppelin on OSX, so Zeppelin should work on it.
>>>>>>
>>>>>> If you can try latest code and post some log files, it'll be helpful
>>>>>> to investigate the problems.
>>>>>>
>>>>>> Thanks,
>>>>>> moon
>>>>>> On 2015년 5월 12일 (화) at 오후 4:39 Jonathan Haddad <
>>>>>> jonathan.haddad@gmail.com> wrote:
>>>>>>
>>>>>>> How did you get it to work? I tried and failed miserably.
>>>>>>>
>>>>>>> Jon
>>>>>>>
>>>>>>> On May 12, 2015, at 12:25 AM, Tyler Mitchell <
>>>>>>> Tyler.Mitchell@actian.com> wrote:
>>>>>>>
>>>>>>> I've been using Python with Spark 1.3.0 and copy of Zeppelin from
>>>>>>> today, but I'm getting some weirdness. Before digging into it further I
>>>>>>> also tried Spark 1.3.1 but got other strange things (output of notes not
>>>>>>> showing anything).
>>>>>>>
>>>>>>>
>>>>>>> So curious, what versions are recommended or preferred? I'm
>>>>>>> trying on OSX so far.
>>>>>>>
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>> Tyler
>>>>>>>
>>>>>>>
Re: Recommended versions?
Posted by Jonathan Haddad <jo...@jonhaddad.com>.
Yep, I believe that was the one.
On Tue, May 19, 2015 at 10:41 AM Marty B <sp...@mjhb.com> wrote:
> Jon,
>
> Is this the error message you see?
>
> (<class 'py4j.protocol.Py4JJavaError'>, Py4JJavaError(u'An error occurred
> while calling o77.loadClass.\n', JavaObject id=o78), <traceback object at
> 0x7f8949d124d0>)
>
> On Tue, May 19, 2015 at 9:49 AM Jonathan Haddad <jo...@jonhaddad.com> wrote:
>
>> I believe was able to make this part work by pointing PYTHONPATH to the
>> egg. The part I couldn't get working was the driver class path, etc. I'm
>> not really a Java guy so it's a lot of stumbling around for me.
>>
>> On Tue, May 19, 2015 at 9:46 AM Marty B <sp...@mjhb.com> wrote:
>>
>>> The problem as I see it is that there appears to be no way to request
>>> Zeppelin to load the .egg file (Python module containing PySpark-Cassandra
>>> interface).
>>>
>>> I opened ZEPPELIN-71 for this, but have seen no progress other than
>>> acknowledgement.
>>>
>>> Perhaps additional feedback in the JIRA ticket would help the Zeppelin
>>> team with prioritization. :-)
>>>
>>> I hope to find time over the upcoming holiday weekend (US) to look
>>> further into fixing it - my first attempt failed.
>>>
>>>
>>>
>>> On Tue, May 19, 2015 at 9:21 AM Jonathan Haddad <jo...@jonhaddad.com>
>>> wrote:
>>>
>>>> What I meant was I couldn't figure out how to get the Cassandra
>>>> connector to work in pyspark. I had asked previously (including the flags
>>>> I pass to pyspark) and gotten no response.
>>>>
>>>> If you've got a recommendation as to how to make the equivilent of this
>>>> work in Zeppelin I'd love to give it a try:
>>>>
>>>> PYSPARK_ROOT=/Users/jhaddad/dev/pyspark-cassandra/target
>>>> export PYSPARK_DRIVER_PYTHON=ipython
>>>> PYSPARK_DRIVER_PYTHON_OPTS="notebook"
>>>>
>>>> pyspark \
>>>> --jars ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>>>> --driver-class-path ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>>>> --py-files ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4-py2.7.egg \
>>>> --conf spark.cassandra.connection.host=127.0.0.1 \
>>>> --master spark://127.0.0.1:7077 \
>>>>
>>>> Jon
>>>>
>>>> On Tue, May 19, 2015 at 4:30 AM moon soo Lee <mo...@apache.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> First, thanks for trying Zeppelin.
>>>>>
>>>>> You guys mean pyspark is not working?
>>>>> I'm developing zeppelin on OSX, so Zeppelin should work on it.
>>>>>
>>>>> If you can try latest code and post some log files, it'll be helpful
>>>>> to investigate the problems.
>>>>>
>>>>> Thanks,
>>>>> moon
>>>>> On 2015년 5월 12일 (화) at 오후 4:39 Jonathan Haddad <
>>>>> jonathan.haddad@gmail.com> wrote:
>>>>>
>>>>>> How did you get it to work? I tried and failed miserably.
>>>>>>
>>>>>> Jon
>>>>>>
>>>>>> On May 12, 2015, at 12:25 AM, Tyler Mitchell <
>>>>>> Tyler.Mitchell@actian.com> wrote:
>>>>>>
>>>>>> I've been using Python with Spark 1.3.0 and copy of Zeppelin from
>>>>>> today, but I'm getting some weirdness. Before digging into it further I
>>>>>> also tried Spark 1.3.1 but got other strange things (output of notes not
>>>>>> showing anything).
>>>>>>
>>>>>>
>>>>>> So curious, what versions are recommended or preferred? I'm trying
>>>>>> on OSX so far.
>>>>>>
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>> Tyler
>>>>>>
>>>>>>
Re: Recommended versions?
Posted by Marty B <sp...@mjhb.com>.
Jon,
Is this the error message you see?
(<class 'py4j.protocol.Py4JJavaError'>, Py4JJavaError(u'An error occurred
while calling o77.loadClass.\n', JavaObject id=o78), <traceback object at
0x7f8949d124d0>)
On Tue, May 19, 2015 at 9:49 AM Jonathan Haddad <jo...@jonhaddad.com> wrote:
> I believe was able to make this part work by pointing PYTHONPATH to the
> egg. The part I couldn't get working was the driver class path, etc. I'm
> not really a Java guy so it's a lot of stumbling around for me.
>
> On Tue, May 19, 2015 at 9:46 AM Marty B <sp...@mjhb.com> wrote:
>
>> The problem as I see it is that there appears to be no way to request
>> Zeppelin to load the .egg file (Python module containing PySpark-Cassandra
>> interface).
>>
>> I opened ZEPPELIN-71 for this, but have seen no progress other than
>> acknowledgement.
>>
>> Perhaps additional feedback in the JIRA ticket would help the Zeppelin
>> team with prioritization. :-)
>>
>> I hope to find time over the upcoming holiday weekend (US) to look
>> further into fixing it - my first attempt failed.
>>
>>
>>
>> On Tue, May 19, 2015 at 9:21 AM Jonathan Haddad <jo...@jonhaddad.com>
>> wrote:
>>
>>> What I meant was I couldn't figure out how to get the Cassandra
>>> connector to work in pyspark. I had asked previously (including the flags
>>> I pass to pyspark) and gotten no response.
>>>
>>> If you've got a recommendation as to how to make the equivilent of this
>>> work in Zeppelin I'd love to give it a try:
>>>
>>> PYSPARK_ROOT=/Users/jhaddad/dev/pyspark-cassandra/target
>>> export PYSPARK_DRIVER_PYTHON=ipython
>>> PYSPARK_DRIVER_PYTHON_OPTS="notebook"
>>>
>>> pyspark \
>>> --jars ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>>> --driver-class-path ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>>> --py-files ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4-py2.7.egg \
>>> --conf spark.cassandra.connection.host=127.0.0.1 \
>>> --master spark://127.0.0.1:7077 \
>>>
>>> Jon
>>>
>>> On Tue, May 19, 2015 at 4:30 AM moon soo Lee <mo...@apache.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> First, thanks for trying Zeppelin.
>>>>
>>>> You guys mean pyspark is not working?
>>>> I'm developing zeppelin on OSX, so Zeppelin should work on it.
>>>>
>>>> If you can try latest code and post some log files, it'll be helpful to
>>>> investigate the problems.
>>>>
>>>> Thanks,
>>>> moon
>>>> On 2015년 5월 12일 (화) at 오후 4:39 Jonathan Haddad <
>>>> jonathan.haddad@gmail.com> wrote:
>>>>
>>>>> How did you get it to work? I tried and failed miserably.
>>>>>
>>>>> Jon
>>>>>
>>>>> On May 12, 2015, at 12:25 AM, Tyler Mitchell <
>>>>> Tyler.Mitchell@actian.com> wrote:
>>>>>
>>>>> I've been using Python with Spark 1.3.0 and copy of Zeppelin from
>>>>> today, but I'm getting some weirdness. Before digging into it further I
>>>>> also tried Spark 1.3.1 but got other strange things (output of notes not
>>>>> showing anything).
>>>>>
>>>>>
>>>>> So curious, what versions are recommended or preferred? I'm trying
>>>>> on OSX so far.
>>>>>
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> Tyler
>>>>>
>>>>>
Re: Recommended versions?
Posted by Jonathan Haddad <jo...@jonhaddad.com>.
I believe was able to make this part work by pointing PYTHONPATH to the
egg. The part I couldn't get working was the driver class path, etc. I'm
not really a Java guy so it's a lot of stumbling around for me.
On Tue, May 19, 2015 at 9:46 AM Marty B <sp...@mjhb.com> wrote:
> The problem as I see it is that there appears to be no way to request
> Zeppelin to load the .egg file (Python module containing PySpark-Cassandra
> interface).
>
> I opened ZEPPELIN-71 for this, but have seen no progress other than
> acknowledgement.
>
> Perhaps additional feedback in the JIRA ticket would help the Zeppelin
> team with prioritization. :-)
>
> I hope to find time over the upcoming holiday weekend (US) to look further
> into fixing it - my first attempt failed.
>
>
>
> On Tue, May 19, 2015 at 9:21 AM Jonathan Haddad <jo...@jonhaddad.com> wrote:
>
>> What I meant was I couldn't figure out how to get the Cassandra connector
>> to work in pyspark. I had asked previously (including the flags I pass to
>> pyspark) and gotten no response.
>>
>> If you've got a recommendation as to how to make the equivilent of this
>> work in Zeppelin I'd love to give it a try:
>>
>> PYSPARK_ROOT=/Users/jhaddad/dev/pyspark-cassandra/target
>> export PYSPARK_DRIVER_PYTHON=ipython PYSPARK_DRIVER_PYTHON_OPTS="notebook"
>>
>> pyspark \
>> --jars ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>> --driver-class-path ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
>> --py-files ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4-py2.7.egg \
>> --conf spark.cassandra.connection.host=127.0.0.1 \
>> --master spark://127.0.0.1:7077 \
>>
>> Jon
>>
>> On Tue, May 19, 2015 at 4:30 AM moon soo Lee <mo...@apache.org> wrote:
>>
>>> Hi,
>>>
>>> First, thanks for trying Zeppelin.
>>>
>>> You guys mean pyspark is not working?
>>> I'm developing zeppelin on OSX, so Zeppelin should work on it.
>>>
>>> If you can try latest code and post some log files, it'll be helpful to
>>> investigate the problems.
>>>
>>> Thanks,
>>> moon
>>> On 2015년 5월 12일 (화) at 오후 4:39 Jonathan Haddad <
>>> jonathan.haddad@gmail.com> wrote:
>>>
>>>> How did you get it to work? I tried and failed miserably.
>>>>
>>>> Jon
>>>>
>>>> On May 12, 2015, at 12:25 AM, Tyler Mitchell <Ty...@actian.com>
>>>> wrote:
>>>>
>>>> I've been using Python with Spark 1.3.0 and copy of Zeppelin from
>>>> today, but I'm getting some weirdness. Before digging into it further I
>>>> also tried Spark 1.3.1 but got other strange things (output of notes not
>>>> showing anything).
>>>>
>>>>
>>>> So curious, what versions are recommended or preferred? I'm trying
>>>> on OSX so far.
>>>>
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> Tyler
>>>>
>>>>
Re: Recommended versions?
Posted by Marty B <sp...@mjhb.com>.
The problem as I see it is that there appears to be no way to request
Zeppelin to load the .egg file (Python module containing PySpark-Cassandra
interface).
I opened ZEPPELIN-71 for this, but have seen no progress other than
acknowledgement.
Perhaps additional feedback in the JIRA ticket would help the Zeppelin team
with prioritization. :-)
I hope to find time over the upcoming holiday weekend (US) to look further
into fixing it - my first attempt failed.
On Tue, May 19, 2015 at 9:21 AM Jonathan Haddad <jo...@jonhaddad.com> wrote:
> What I meant was I couldn't figure out how to get the Cassandra connector
> to work in pyspark. I had asked previously (including the flags I pass to
> pyspark) and gotten no response.
>
> If you've got a recommendation as to how to make the equivilent of this
> work in Zeppelin I'd love to give it a try:
>
> PYSPARK_ROOT=/Users/jhaddad/dev/pyspark-cassandra/target
> export PYSPARK_DRIVER_PYTHON=ipython PYSPARK_DRIVER_PYTHON_OPTS="notebook"
>
> pyspark \
> --jars ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
> --driver-class-path ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
> --py-files ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4-py2.7.egg \
> --conf spark.cassandra.connection.host=127.0.0.1 \
> --master spark://127.0.0.1:7077 \
>
> Jon
>
> On Tue, May 19, 2015 at 4:30 AM moon soo Lee <mo...@apache.org> wrote:
>
>> Hi,
>>
>> First, thanks for trying Zeppelin.
>>
>> You guys mean pyspark is not working?
>> I'm developing zeppelin on OSX, so Zeppelin should work on it.
>>
>> If you can try latest code and post some log files, it'll be helpful to
>> investigate the problems.
>>
>> Thanks,
>> moon
>> On 2015년 5월 12일 (화) at 오후 4:39 Jonathan Haddad <jo...@gmail.com>
>> wrote:
>>
>>> How did you get it to work? I tried and failed miserably.
>>>
>>> Jon
>>>
>>> On May 12, 2015, at 12:25 AM, Tyler Mitchell <Ty...@actian.com>
>>> wrote:
>>>
>>> I've been using Python with Spark 1.3.0 and copy of Zeppelin from
>>> today, but I'm getting some weirdness. Before digging into it further I
>>> also tried Spark 1.3.1 but got other strange things (output of notes not
>>> showing anything).
>>>
>>>
>>> So curious, what versions are recommended or preferred? I'm trying on
>>> OSX so far.
>>>
>>>
>>> Thanks!
>>>
>>>
>>> Tyler
>>>
>>>
Re: Recommended versions?
Posted by Jonathan Haddad <jo...@jonhaddad.com>.
What I meant was I couldn't figure out how to get the Cassandra connector
to work in pyspark. I had asked previously (including the flags I pass to
pyspark) and gotten no response.
If you've got a recommendation as to how to make the equivilent of this
work in Zeppelin I'd love to give it a try:
PYSPARK_ROOT=/Users/jhaddad/dev/pyspark-cassandra/target
export PYSPARK_DRIVER_PYTHON=ipython PYSPARK_DRIVER_PYTHON_OPTS="notebook"
pyspark \
--jars ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
--driver-class-path ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4.jar \
--py-files ${PYSPARK_ROOT}/pyspark_cassandra-0.1.4-py2.7.egg \
--conf spark.cassandra.connection.host=127.0.0.1 \
--master spark://127.0.0.1:7077 \
Jon
On Tue, May 19, 2015 at 4:30 AM moon soo Lee <mo...@apache.org> wrote:
> Hi,
>
> First, thanks for trying Zeppelin.
>
> You guys mean pyspark is not working?
> I'm developing zeppelin on OSX, so Zeppelin should work on it.
>
> If you can try latest code and post some log files, it'll be helpful to
> investigate the problems.
>
> Thanks,
> moon
> On 2015년 5월 12일 (화) at 오후 4:39 Jonathan Haddad <jo...@gmail.com>
> wrote:
>
>> How did you get it to work? I tried and failed miserably.
>>
>> Jon
>>
>> On May 12, 2015, at 12:25 AM, Tyler Mitchell <Ty...@actian.com>
>> wrote:
>>
>> I've been using Python with Spark 1.3.0 and copy of Zeppelin from
>> today, but I'm getting some weirdness. Before digging into it further I
>> also tried Spark 1.3.1 but got other strange things (output of notes not
>> showing anything).
>>
>>
>> So curious, what versions are recommended or preferred? I'm trying on
>> OSX so far.
>>
>>
>> Thanks!
>>
>>
>> Tyler
>>
>>
Re: Recommended versions?
Posted by Tyler Mitchell <Ty...@actian.com>.
Moon, what versions of Spark are you supporting on OSX with Zeppelin master code?
Tyler
________________________________
From: moon soo Lee <mo...@apache.org>
Sent: May 19, 2015 4:29 AM
To: users@zeppelin.incubator.apache.org
Subject: Re: Recommended versions?
Hi,
First, thanks for trying Zeppelin.
You guys mean pyspark is not working?
I'm developing zeppelin on OSX, so Zeppelin should work on it.
If you can try latest code and post some log files, it'll be helpful to investigate the problems.
Thanks,
moon
On 2015? 5? 12? (?) at ?? 4:39 Jonathan Haddad <jo...@gmail.com>> wrote:
How did you get it to work? I tried and failed miserably.
Jon
On May 12, 2015, at 12:25 AM, Tyler Mitchell <Ty...@actian.com>> wrote:
I've been using Python with Spark 1.3.0 and copy of Zeppelin from today, but I'm getting some weirdness. Before digging into it further I also tried Spark 1.3.1 but got other strange things (output of notes not showing anything).
So curious, what versions are recommended or preferred? I'm trying on OSX so far.
Thanks!
Tyler
Re: Recommended versions?
Posted by moon soo Lee <mo...@apache.org>.
Hi,
First, thanks for trying Zeppelin.
You guys mean pyspark is not working?
I'm developing zeppelin on OSX, so Zeppelin should work on it.
If you can try latest code and post some log files, it'll be helpful to
investigate the problems.
Thanks,
moon
On 2015년 5월 12일 (화) at 오후 4:39 Jonathan Haddad <jo...@gmail.com>
wrote:
> How did you get it to work? I tried and failed miserably.
>
> Jon
>
> On May 12, 2015, at 12:25 AM, Tyler Mitchell <Ty...@actian.com>
> wrote:
>
> I've been using Python with Spark 1.3.0 and copy of Zeppelin from
> today, but I'm getting some weirdness. Before digging into it further I
> also tried Spark 1.3.1 but got other strange things (output of notes not
> showing anything).
>
>
> So curious, what versions are recommended or preferred? I'm trying on
> OSX so far.
>
>
> Thanks!
>
>
> Tyler
>
>
Re: Recommended versions?
Posted by Jonathan Haddad <jo...@gmail.com>.
How did you get it to work? I tried and failed miserably.
Jon
> On May 12, 2015, at 12:25 AM, Tyler Mitchell <Ty...@actian.com> wrote:
>
> I've been using Python with Spark 1.3.0 and copy of Zeppelin from today, but I'm getting some weirdness. Before digging into it further I also tried Spark 1.3.1 but got other strange things (output of notes not showing anything).
>
>
> So curious, what versions are recommended or preferred? I'm trying on OSX so far.
>
>
> Thanks!
>
> Tyler