You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Biplob Biswas <re...@gmail.com> on 2016/07/07 15:21:51 UTC
Re: StreamingKmeans Spark doesn't work at all
Hi,
Can anyone care to please look into this issue? I would really love some
assistance here.
Thanks a lot.
Thanks & Regards
Biplob Biswas
On Tue, Jul 5, 2016 at 1:00 PM, Biplob Biswas <re...@gmail.com>
wrote:
>
> Hi,
>
> I implemented the streamingKmeans example provided in the spark website but
> in Java.
> The full implementation is here,
>
> http://pastebin.com/CJQfWNvk
>
> But i am not getting anything in the output except occasional timestamps
> like one below:
>
> -------------------------------------------
> Time: 1466176935000 ms
> -------------------------------------------
>
> Also, i have 2 directories:
> "D:\spark\streaming example\Data Sets\training"
> "D:\spark\streaming example\Data Sets\test"
>
> and inside these directories i have 1 file each "samplegpsdata_train.txt"
> and "samplegpsdata_test.txt" with training data having 500 datapoints and
> test data with 60 datapoints.
>
> I am very new to the spark systems and any help is highly appreciated.
>
>
> //---------------------------------------------------------------------------------------//
>
> Now, I also have now tried using the scala implementation available here:
>
> https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/mllib/StreamingKMeansExample.scala
>
>
> and even had the training and test file provided in the format specified in
> that file as follows:
>
> * The rows of the training text files must be vector data in the form
> * `[x1,x2,x3,...,xn]`
> * Where n is the number of dimensions.
> *
> * The rows of the test text files must be labeled data in the form
> * `(y,[x1,x2,x3,...,xn])`
> * Where y is some identifier. n must be the same for train and test.
>
>
> But I still get no output on my eclipse window ... just the Time!
>
> Can anyone seriously help me with this?
>
> Thank you so much
> Biplob Biswas
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/StreamingKmeans-Spark-doesn-t-work-at-all-tp27286.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>
Re: StreamingKmeans Spark doesn't work at all
Posted by Biplob Biswas <re...@gmail.com>.
Hi Shuai,
Thanks for the reply, I mentioned in the mail that I tried running the
scala example as well from the link I provided and the result is the same.
Thanks & Regards
Biplob Biswas
On Mon, Jul 11, 2016 at 5:52 AM, Shuai Lin <li...@gmail.com> wrote:
> I would suggest you run the scala version of the example first, so you can
> tell whether it's a problem of the data you provided or a problem of the
> java code.
>
> On Mon, Jul 11, 2016 at 2:37 AM, Biplob Biswas <re...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I know i am asking again, but I tried running the same thing on mac as
>> well as some answers on the internet suggested it could be an issue with
>> the windows environment, but still nothing works.
>>
>> Can anyone atleast suggest whether its a bug with spark or is it
>> something else?
>>
>> Would be really grateful! Thanks a lot.
>>
>> Thanks & Regards
>> Biplob Biswas
>>
>> On Thu, Jul 7, 2016 at 5:21 PM, Biplob Biswas <re...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Can anyone care to please look into this issue? I would really love
>>> some assistance here.
>>>
>>> Thanks a lot.
>>>
>>> Thanks & Regards
>>> Biplob Biswas
>>>
>>> On Tue, Jul 5, 2016 at 1:00 PM, Biplob Biswas <re...@gmail.com>
>>> wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> I implemented the streamingKmeans example provided in the spark website
>>>> but
>>>> in Java.
>>>> The full implementation is here,
>>>>
>>>> http://pastebin.com/CJQfWNvk
>>>>
>>>> But i am not getting anything in the output except occasional timestamps
>>>> like one below:
>>>>
>>>> -------------------------------------------
>>>> Time: 1466176935000 ms
>>>> -------------------------------------------
>>>>
>>>> Also, i have 2 directories:
>>>> "D:\spark\streaming example\Data Sets\training"
>>>> "D:\spark\streaming example\Data Sets\test"
>>>>
>>>> and inside these directories i have 1 file each
>>>> "samplegpsdata_train.txt"
>>>> and "samplegpsdata_test.txt" with training data having 500 datapoints
>>>> and
>>>> test data with 60 datapoints.
>>>>
>>>> I am very new to the spark systems and any help is highly appreciated.
>>>>
>>>>
>>>> //---------------------------------------------------------------------------------------//
>>>>
>>>> Now, I also have now tried using the scala implementation available
>>>> here:
>>>>
>>>> https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/mllib/StreamingKMeansExample.scala
>>>>
>>>>
>>>> and even had the training and test file provided in the format
>>>> specified in
>>>> that file as follows:
>>>>
>>>> * The rows of the training text files must be vector data in the form
>>>> * `[x1,x2,x3,...,xn]`
>>>> * Where n is the number of dimensions.
>>>> *
>>>> * The rows of the test text files must be labeled data in the form
>>>> * `(y,[x1,x2,x3,...,xn])`
>>>> * Where y is some identifier. n must be the same for train and test.
>>>>
>>>>
>>>> But I still get no output on my eclipse window ... just the Time!
>>>>
>>>> Can anyone seriously help me with this?
>>>>
>>>> Thank you so much
>>>> Biplob Biswas
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://apache-spark-user-list.1001560.n3.nabble.com/StreamingKmeans-Spark-doesn-t-work-at-all-tp27286.html
>>>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>>>
>>>>
>>>
>>
>
Re: StreamingKmeans Spark doesn't work at all
Posted by Shuai Lin <li...@gmail.com>.
I would suggest you run the scala version of the example first, so you can
tell whether it's a problem of the data you provided or a problem of the
java code.
On Mon, Jul 11, 2016 at 2:37 AM, Biplob Biswas <re...@gmail.com>
wrote:
> Hi,
>
> I know i am asking again, but I tried running the same thing on mac as
> well as some answers on the internet suggested it could be an issue with
> the windows environment, but still nothing works.
>
> Can anyone atleast suggest whether its a bug with spark or is it something
> else?
>
> Would be really grateful! Thanks a lot.
>
> Thanks & Regards
> Biplob Biswas
>
> On Thu, Jul 7, 2016 at 5:21 PM, Biplob Biswas <re...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Can anyone care to please look into this issue? I would really love some
>> assistance here.
>>
>> Thanks a lot.
>>
>> Thanks & Regards
>> Biplob Biswas
>>
>> On Tue, Jul 5, 2016 at 1:00 PM, Biplob Biswas <re...@gmail.com>
>> wrote:
>>
>>>
>>> Hi,
>>>
>>> I implemented the streamingKmeans example provided in the spark website
>>> but
>>> in Java.
>>> The full implementation is here,
>>>
>>> http://pastebin.com/CJQfWNvk
>>>
>>> But i am not getting anything in the output except occasional timestamps
>>> like one below:
>>>
>>> -------------------------------------------
>>> Time: 1466176935000 ms
>>> -------------------------------------------
>>>
>>> Also, i have 2 directories:
>>> "D:\spark\streaming example\Data Sets\training"
>>> "D:\spark\streaming example\Data Sets\test"
>>>
>>> and inside these directories i have 1 file each "samplegpsdata_train.txt"
>>> and "samplegpsdata_test.txt" with training data having 500 datapoints and
>>> test data with 60 datapoints.
>>>
>>> I am very new to the spark systems and any help is highly appreciated.
>>>
>>>
>>> //---------------------------------------------------------------------------------------//
>>>
>>> Now, I also have now tried using the scala implementation available here:
>>>
>>> https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/mllib/StreamingKMeansExample.scala
>>>
>>>
>>> and even had the training and test file provided in the format specified
>>> in
>>> that file as follows:
>>>
>>> * The rows of the training text files must be vector data in the form
>>> * `[x1,x2,x3,...,xn]`
>>> * Where n is the number of dimensions.
>>> *
>>> * The rows of the test text files must be labeled data in the form
>>> * `(y,[x1,x2,x3,...,xn])`
>>> * Where y is some identifier. n must be the same for train and test.
>>>
>>>
>>> But I still get no output on my eclipse window ... just the Time!
>>>
>>> Can anyone seriously help me with this?
>>>
>>> Thank you so much
>>> Biplob Biswas
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://apache-spark-user-list.1001560.n3.nabble.com/StreamingKmeans-Spark-doesn-t-work-at-all-tp27286.html
>>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>>
>>>
>>
>
Re: StreamingKmeans Spark doesn't work at all
Posted by Biplob Biswas <re...@gmail.com>.
Hi,
I know i am asking again, but I tried running the same thing on mac as well
as some answers on the internet suggested it could be an issue with the
windows environment, but still nothing works.
Can anyone atleast suggest whether its a bug with spark or is it something
else?
Would be really grateful! Thanks a lot.
Thanks & Regards
Biplob Biswas
On Thu, Jul 7, 2016 at 5:21 PM, Biplob Biswas <re...@gmail.com>
wrote:
> Hi,
>
> Can anyone care to please look into this issue? I would really love some
> assistance here.
>
> Thanks a lot.
>
> Thanks & Regards
> Biplob Biswas
>
> On Tue, Jul 5, 2016 at 1:00 PM, Biplob Biswas <re...@gmail.com>
> wrote:
>
>>
>> Hi,
>>
>> I implemented the streamingKmeans example provided in the spark website
>> but
>> in Java.
>> The full implementation is here,
>>
>> http://pastebin.com/CJQfWNvk
>>
>> But i am not getting anything in the output except occasional timestamps
>> like one below:
>>
>> -------------------------------------------
>> Time: 1466176935000 ms
>> -------------------------------------------
>>
>> Also, i have 2 directories:
>> "D:\spark\streaming example\Data Sets\training"
>> "D:\spark\streaming example\Data Sets\test"
>>
>> and inside these directories i have 1 file each "samplegpsdata_train.txt"
>> and "samplegpsdata_test.txt" with training data having 500 datapoints and
>> test data with 60 datapoints.
>>
>> I am very new to the spark systems and any help is highly appreciated.
>>
>>
>> //---------------------------------------------------------------------------------------//
>>
>> Now, I also have now tried using the scala implementation available here:
>>
>> https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/mllib/StreamingKMeansExample.scala
>>
>>
>> and even had the training and test file provided in the format specified
>> in
>> that file as follows:
>>
>> * The rows of the training text files must be vector data in the form
>> * `[x1,x2,x3,...,xn]`
>> * Where n is the number of dimensions.
>> *
>> * The rows of the test text files must be labeled data in the form
>> * `(y,[x1,x2,x3,...,xn])`
>> * Where y is some identifier. n must be the same for train and test.
>>
>>
>> But I still get no output on my eclipse window ... just the Time!
>>
>> Can anyone seriously help me with this?
>>
>> Thank you so much
>> Biplob Biswas
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/StreamingKmeans-Spark-doesn-t-work-at-all-tp27286.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>
>>
>