You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tinkerpop.apache.org by Marko Rodriguez <ok...@gmail.com> on 2015/12/02 21:32:28 UTC
Re: [TinkerPop] Re: TP3 3.1 SparkGraphComputer doesn't support spark on YARN?
Hi MeteorMarc,
That is really cool of you to do that.
Can you issue a pull request that updates the respective hadoop-gremlin/conf files with ONLY the necessary properties and then # comment them out so people will be able to see what is required if they want to use Spark+YARN.
Thank you,
Marko.
http://markorodriguez.com
On Dec 2, 2015, at 1:25 PM, MeteorMarc <bi...@xs4all.nl> wrote:
> Hi Christine,
>
> I built TP 3.1 against Hortonworks 2.3.2 and added configs in the conf/hadoop folder of gremlin-console to support spark on yarn. See:
>
> https://github.com/vtslab/incubator-tinkerpop
>
> Good luck,
>
> Marc
>
>
>
> Op maandag 30 november 2015 18:02:24 UTC+1 schreef Christine Li:
> Spark is deployed on Yarn. I've tried blvp sample with by set spark.master=yarn-client and got the follow error:
>
> Caused by: java.lang.ClassNotFoundException: org.apache.spark.deploy.yarn.YarnSparkHadoopUtil
>
> Looks like Spark on Yarn isn't supported. Is it the case?
>
> Thanks,
>
> --
> You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/d7472deb-db15-4ab6-b6fb-e6766ad24c69%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
Re: [TinkerPop] Re: TP3 3.1 SparkGraphComputer doesn't support
spark on YARN?
Posted by HadoopMarc <ma...@gmail.com>.
Hi Cristine,
Did you start the repl with gremlinhdp.sh? It does all the exports needed
and sets the CLASSPATH. Check if the configs are right for your cluster.
https://github.com/vtslab/incubator-tinkerpop/blob/3.1.0-hdp-2.3.2.0-2950/gremlin-console/bin/gremlinhdp.sh
Marc
Op maandag 7 december 2015 21:30:03 UTC+1 schreef Christine Li:
>
> Sorry for this 101 question. When I ran blvp sample with spark on yarn.
> I've got ClassNoFound for org.apache.spark.deploy.yarn.YarnSparkHadoopUtil.
> I built TP3.1 from
> https://github.com/vtslab/incubator-tinkerpop/tree/3.1.0-hdp-2.3.2.0-2950
> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fvtslab%2Fincubator-tinkerpop%2Ftree%2F3.1.0-hdp-2.3.2.0-2950%2Fgremlin-console%2Fconf&sa=D&sntz=1&usg=AFQjCNE1V0fiyL15ISSF2nAFGd7wyIse4g>
> and didn't see jars contains YarnSparkHadoopUtil either. Did I miss
> something?
>
>
> On Friday, December 4, 2015 at 1:16:41 PM UTC-5, Jason Plurad wrote:
>>
>> Christine, look in here
>> https://github.com/vtslab/incubator-tinkerpop/tree/3.1.0-hdp-2.3.2.0-2950/hadoop-gremlin/conf
>>
>> On Friday, December 4, 2015 at 1:03:30 PM UTC-5, Christine Li wrote:
>>>
>>> Hi, Marc
>>>
>>> I don't see a hadoop folder on
>>> https://github.com/vtslab/incubator-tinkerpop/tree/3.1.0-hdp-2.3.2.0-2950/gremlin-console/conf
>>> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fvtslab%2Fincubator-tinkerpop%2Ftree%2F3.1.0-hdp-2.3.2.0-2950%2Fgremlin-console%2Fconf&sa=D&sntz=1&usg=AFQjCNE1V0fiyL15ISSF2nAFGd7wyIse4g>
>>>
>>> Did I miss something?
>>> Thanks,
>>>
>>> On Thursday, December 3, 2015 at 9:51:33 AM UTC-5, HadoopMarc wrote:
>>>>
>>>> Hi Marko,
>>>>
>>>> Thanks for your interest. I will do this, but give me a few days until
>>>> I have an evening free.
>>>>
>>>> Regards, Marc
>>>>
>>>> Op woensdag 2 december 2015 21:32:35 UTC+1 schreef Marko A. Rodriguez:
>>>>>
>>>>> Hi MeteorMarc,
>>>>>
>>>>> That is really cool of you to do that.
>>>>>
>>>>> Can you issue a pull request that updates the respective
>>>>> hadoop-gremlin/conf files with ONLY the necessary properties and then #
>>>>> comment them out so people will be able to see what is required if they
>>>>> want to use Spark+YARN.
>>>>>
>>>>> Thank you,
>>>>> Marko.
>>>>>
>>>>> http://markorodriguez.com
>>>>>
>>>>> On Dec 2, 2015, at 1:25 PM, MeteorMarc <bi...@xs4all.nl> wrote:
>>>>>
>>>>> Hi Christine,
>>>>>
>>>>> I built TP 3.1 against Hortonworks 2.3.2 and added configs in the
>>>>> conf/hadoop folder of gremlin-console to support spark on yarn. See:
>>>>>
>>>>> https://github.com/vtslab/incubator-tinkerpop
>>>>>
>>>>> Good luck,
>>>>>
>>>>> Marc
>>>>>
>>>>>
>>>>>
>>>>> Op maandag 30 november 2015 18:02:24 UTC+1 schreef Christine Li:
>>>>>>
>>>>>> Spark is deployed on Yarn. I've tried blvp sample with by set
>>>>>> spark.master=yarn-client and got the follow error:
>>>>>>
>>>>>> Caused by: java.lang.ClassNotFoundException:
>>>>>> org.apache.spark.deploy.yarn.YarnSparkHadoopUtil
>>>>>>
>>>>>> Looks like Spark on Yarn isn't supported. Is it the case?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Gremlin-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to gremlin-user...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/gremlin-users/d7472deb-db15-4ab6-b6fb-e6766ad24c69%40googlegroups.com
>>>>> <https://groups.google.com/d/msgid/gremlin-users/d7472deb-db15-4ab6-b6fb-e6766ad24c69%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>>
>>>>>
Re: [TinkerPop] Re: TP3 3.1 SparkGraphComputer doesn't support
spark on YARN?
Posted by Jason Plurad <pl...@gmail.com>.
Christine, look in here
https://github.com/vtslab/incubator-tinkerpop/tree/3.1.0-hdp-2.3.2.0-2950/hadoop-gremlin/conf
On Friday, December 4, 2015 at 1:03:30 PM UTC-5, Christine Li wrote:
>
> Hi, Marc
>
> I don't see a hadoop folder on
> https://github.com/vtslab/incubator-tinkerpop/tree/3.1.0-hdp-2.3.2.0-2950/gremlin-console/conf
>
> Did I miss something?
> Thanks,
>
> On Thursday, December 3, 2015 at 9:51:33 AM UTC-5, HadoopMarc wrote:
>>
>> Hi Marko,
>>
>> Thanks for your interest. I will do this, but give me a few days until I
>> have an evening free.
>>
>> Regards, Marc
>>
>> Op woensdag 2 december 2015 21:32:35 UTC+1 schreef Marko A. Rodriguez:
>>>
>>> Hi MeteorMarc,
>>>
>>> That is really cool of you to do that.
>>>
>>> Can you issue a pull request that updates the respective
>>> hadoop-gremlin/conf files with ONLY the necessary properties and then #
>>> comment them out so people will be able to see what is required if they
>>> want to use Spark+YARN.
>>>
>>> Thank you,
>>> Marko.
>>>
>>> http://markorodriguez.com
>>>
>>> On Dec 2, 2015, at 1:25 PM, MeteorMarc <bi...@xs4all.nl> wrote:
>>>
>>> Hi Christine,
>>>
>>> I built TP 3.1 against Hortonworks 2.3.2 and added configs in the
>>> conf/hadoop folder of gremlin-console to support spark on yarn. See:
>>>
>>> https://github.com/vtslab/incubator-tinkerpop
>>>
>>> Good luck,
>>>
>>> Marc
>>>
>>>
>>>
>>> Op maandag 30 november 2015 18:02:24 UTC+1 schreef Christine Li:
>>>>
>>>> Spark is deployed on Yarn. I've tried blvp sample with by set
>>>> spark.master=yarn-client and got the follow error:
>>>>
>>>> Caused by: java.lang.ClassNotFoundException:
>>>> org.apache.spark.deploy.yarn.YarnSparkHadoopUtil
>>>>
>>>> Looks like Spark on Yarn isn't supported. Is it the case?
>>>>
>>>> Thanks,
>>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Gremlin-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to gremlin-user...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/gremlin-users/d7472deb-db15-4ab6-b6fb-e6766ad24c69%40googlegroups.com
>>> <https://groups.google.com/d/msgid/gremlin-users/d7472deb-db15-4ab6-b6fb-e6766ad24c69%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
Re: [TinkerPop] Re: TP3 3.1 SparkGraphComputer doesn't support
spark on YARN?
Posted by Christine Li <ch...@gmail.com>.
Hi, Marc
I don't see a hadoop folder on
https://github.com/vtslab/incubator-tinkerpop/tree/3.1.0-hdp-2.3.2.0-2950/gremlin-console/conf
Did I miss something?
Thanks,
On Thursday, December 3, 2015 at 9:51:33 AM UTC-5, HadoopMarc wrote:
>
> Hi Marko,
>
> Thanks for your interest. I will do this, but give me a few days until I
> have an evening free.
>
> Regards, Marc
>
> Op woensdag 2 december 2015 21:32:35 UTC+1 schreef Marko A. Rodriguez:
>>
>> Hi MeteorMarc,
>>
>> That is really cool of you to do that.
>>
>> Can you issue a pull request that updates the respective
>> hadoop-gremlin/conf files with ONLY the necessary properties and then #
>> comment them out so people will be able to see what is required if they
>> want to use Spark+YARN.
>>
>> Thank you,
>> Marko.
>>
>> http://markorodriguez.com
>>
>> On Dec 2, 2015, at 1:25 PM, MeteorMarc <bi...@xs4all.nl> wrote:
>>
>> Hi Christine,
>>
>> I built TP 3.1 against Hortonworks 2.3.2 and added configs in the
>> conf/hadoop folder of gremlin-console to support spark on yarn. See:
>>
>> https://github.com/vtslab/incubator-tinkerpop
>>
>> Good luck,
>>
>> Marc
>>
>>
>>
>> Op maandag 30 november 2015 18:02:24 UTC+1 schreef Christine Li:
>>>
>>> Spark is deployed on Yarn. I've tried blvp sample with by set
>>> spark.master=yarn-client and got the follow error:
>>>
>>> Caused by: java.lang.ClassNotFoundException:
>>> org.apache.spark.deploy.yarn.YarnSparkHadoopUtil
>>>
>>> Looks like Spark on Yarn isn't supported. Is it the case?
>>>
>>> Thanks,
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to gremlin-user...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/d7472deb-db15-4ab6-b6fb-e6766ad24c69%40googlegroups.com
>> <https://groups.google.com/d/msgid/gremlin-users/d7472deb-db15-4ab6-b6fb-e6766ad24c69%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
Re: [TinkerPop] Re: TP3 3.1 SparkGraphComputer doesn't support
spark on YARN?
Posted by HadoopMarc <ma...@gmail.com>.
Hi Marko,
Thanks for your interest. I will do this, but give me a few days until I
have an evening free.
Regards, Marc
Op woensdag 2 december 2015 21:32:35 UTC+1 schreef Marko A. Rodriguez:
>
> Hi MeteorMarc,
>
> That is really cool of you to do that.
>
> Can you issue a pull request that updates the respective
> hadoop-gremlin/conf files with ONLY the necessary properties and then #
> comment them out so people will be able to see what is required if they
> want to use Spark+YARN.
>
> Thank you,
> Marko.
>
> http://markorodriguez.com
>
> On Dec 2, 2015, at 1:25 PM, MeteorMarc <bi...@xs4all.nl <javascript:>>
> wrote:
>
> Hi Christine,
>
> I built TP 3.1 against Hortonworks 2.3.2 and added configs in the
> conf/hadoop folder of gremlin-console to support spark on yarn. See:
>
> https://github.com/vtslab/incubator-tinkerpop
>
> Good luck,
>
> Marc
>
>
>
> Op maandag 30 november 2015 18:02:24 UTC+1 schreef Christine Li:
>>
>> Spark is deployed on Yarn. I've tried blvp sample with by set
>> spark.master=yarn-client and got the follow error:
>>
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.spark.deploy.yarn.YarnSparkHadoopUtil
>>
>> Looks like Spark on Yarn isn't supported. Is it the case?
>>
>> Thanks,
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to gremlin-user...@googlegroups.com <javascript:>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/d7472deb-db15-4ab6-b6fb-e6766ad24c69%40googlegroups.com
> <https://groups.google.com/d/msgid/gremlin-users/d7472deb-db15-4ab6-b6fb-e6766ad24c69%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>