Re: How to play with built in Pyspark

[Jongyoul Lee <jo...@gmail.com>]

Yes there is no version information. Pyspark follows the version of Spark.

On Friday, July 31, 2015, Zhou Jiang <jz...@gmail.com> wrote:

> Thanks, It worked. One more question, regarding ‘-Ppyspark’ parameters, is
> there any version number associated?
> Such as -Ppyspark-1.4
>
> On Jul 30, 2015, at 8:11 PM, Jongyoul Lee <jongyoul@gmail.com
> <javascript:_e(%7B%7D,'cvml','jongyoul@gmail.com');>> wrote:
>
> You should add a profile like '-Ppyspark' while you build.
>
> On Fri, Jul 31, 2015 at 12:09 PM, Zhou Jiang <jz.belief@gmail.com
> <javascript:_e(%7B%7D,'cvml','jz.belief@gmail.com');>> wrote:
>
>> Anyone knows how to make the *built in* pyspark work?
>>
>> I built the source code with command:
>>
>> mvn clean package -Pspark-1.4 -DskipTests
>>
>> And all the Spark tags worked except %pyspark. No matter what python code
>> I run, it will report
>>
>> pyspark is not responding
>>
>> I checked the logs, only two lines was recorded if I run a HelloWorld
>> python code based on %pyspark:
>>
>>  INFO [2015-07-30 19:26:21,220] ({pool-2-thread-5}
>> SchedulerFactory.java[jobStarted]:132) - Job
>> remoteInterpretJob_1438309581220 started by scheduler interpreter_2047324099
>>  INFO [2015-07-30 19:26:31,233] ({pool-2-thread-5}
>> SchedulerFactory.java[jobFinished]:138) - Job
>> remoteInterpretJob_1438309581220 finished by scheduler
>> interpreter_2047324099
>>
>>
>> Thank you so much.
>>
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
>

-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net