You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by bhavyateja <bh...@gmail.com> on 2015/01/17 00:17:06 UTC
spark 1.2 compatibility
Is spark 1.2 is compatibly with HDP 2.1
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: spark 1.2 compatibility
Posted by Matei Zaharia <ma...@gmail.com>.
The Apache Spark project should work with it, but I'm not sure you can get support from HDP (if you have that).
Matei
> On Jan 16, 2015, at 5:36 PM, Judy Nash <ju...@exchange.microsoft.com> wrote:
>
> Should clarify on this. I personally have used HDP 2.1 + Spark 1.2 and have not seen a problem.
>
> However officially HDP 2.1 + Spark 1.2 is not a supported scenario.
>
> -----Original Message-----
> From: Judy Nash
> Sent: Friday, January 16, 2015 5:35 PM
> To: 'bhavyateja'; user@spark.apache.org
> Subject: RE: spark 1.2 compatibility
>
> Yes. It's compatible with HDP 2.1
>
> -----Original Message-----
> From: bhavyateja [mailto:bhavyateja.potineni@gmail.com]
> Sent: Friday, January 16, 2015 3:17 PM
> To: user@spark.apache.org
> Subject: spark 1.2 compatibility
>
> Is spark 1.2 is compatibly with HDP 2.1
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
RE: spark 1.2 compatibility
Posted by Judy Nash <ju...@exchange.microsoft.com>.
Should clarify on this. I personally have used HDP 2.1 + Spark 1.2 and have not seen a problem.
However officially HDP 2.1 + Spark 1.2 is not a supported scenario.
-----Original Message-----
From: Judy Nash
Sent: Friday, January 16, 2015 5:35 PM
To: 'bhavyateja'; user@spark.apache.org
Subject: RE: spark 1.2 compatibility
Yes. It's compatible with HDP 2.1
-----Original Message-----
From: bhavyateja [mailto:bhavyateja.potineni@gmail.com]
Sent: Friday, January 16, 2015 3:17 PM
To: user@spark.apache.org
Subject: spark 1.2 compatibility
Is spark 1.2 is compatibly with HDP 2.1
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
RE: spark 1.2 compatibility
Posted by Judy Nash <ju...@exchange.microsoft.com>.
Yes. It's compatible with HDP 2.1
-----Original Message-----
From: bhavyateja [mailto:bhavyateja.potineni@gmail.com]
Sent: Friday, January 16, 2015 3:17 PM
To: user@spark.apache.org
Subject: spark 1.2 compatibility
Is spark 1.2 is compatibly with HDP 2.1
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: spark 1.2 compatibility
Posted by bhavyateja <bh...@gmail.com>.
Yes it works with 2.2 but we are trying to use spark 1.2 on HDP 2.1
On Sat, Jan 17, 2015, 11:18 AM Chitturi Padma [via Apache Spark User List] <
ml-node+s1001560n21208h63@n3.nabble.com> wrote:
> It worked for me. spark 1.2.0 with hadoop 2.2.0
>
> On Sat, Jan 17, 2015 at 9:39 PM, bhavyateja [via Apache Spark User List] <[hidden
> email] <http:///user/SendEmail.jtp?type=node&node=21208&i=0>> wrote:
>
>> Hi all,
>>
>> Thanks for your contribution. We have checked and confirmed that HDP 2.1
>> YARN don't work with Spark 1.2
>>
> On Sat, Jan 17, 2015 at 9:11 AM, bhavya teja potineni <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=21207&i=0>> wrote:
>>
> Hi
>>>
>>> Did you try using spark 1.2 on hdp 2.1 YARN
>>>
>>> Can you please go thru the thread
>>>
>>> http://apache-spark-user-list.1001560.n3.nabble.com/Troubleshooting-Spark-tt21189.html
>>> and check where I am going wrong. As my word count program is erroring
>>> out when using spark 1.2 using YARN but its getting executed using spark
>>> 0.9.1
>>>
>>
>>> On Sat, Jan 17, 2015 at 5:55 AM, Chitturi Padma [via Apache Spark User
>>> List] <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=21207&i=1>> wrote:
>>>
>>>> Yes. I built spar 1.2 with apache hadoop 2.2. No compatibility issues.
>>>>
>>>> On Sat, Jan 17, 2015 at 4:47 AM, bhavyateja [via Apache Spark User
>>>> List] <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=21202&i=0>> wrote:
>>>>
>>>>> Is spark 1.2 is compatibly with HDP 2.1
>>>>>
>>>>> ------------------------------
>>>>> If you reply to this email, your message will be added to the
>>>>> discussion below:
>>>>>
>>>>> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197.html
>>>>> To start a new topic under Apache Spark User List, email [hidden
>>>>> email] <http:///user/SendEmail.jtp?type=node&node=21202&i=1>
>>>>> To unsubscribe from Apache Spark User List, click here.
>>>>> NAML
>>>>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the
>>>> discussion below:
>>>>
>>>> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21202.html
>>>> To unsubscribe from spark 1.2 compatibility, click here.
>>>> NAML
>>>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>
>>> If you reply to this email, your message will be added to the
>> discussion below:
>>
>
>> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21207.html
>>
> To start a new topic under Apache Spark User List, email [hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=21208&i=1>
>> To unsubscribe from Apache Spark User List, click here.
>> NAML
>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21208.html
> To unsubscribe from spark 1.2 compatibility, click here
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=21197&code=YmhhdnlhdGVqYS5wb3RpbmVuaUBnbWFpbC5jb218MjExOTd8LTE1MDEzODgyODY=>
> .
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21209.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: spark 1.2 compatibility
Posted by Chitturi Padma <le...@gmail.com>.
It worked for me. spark 1.2.0 with hadoop 2.2.0
On Sat, Jan 17, 2015 at 9:39 PM, bhavyateja [via Apache Spark User List] <
ml-node+s1001560n21207h62@n3.nabble.com> wrote:
> Hi all,
>
> Thanks for your contribution. We have checked and confirmed that HDP 2.1
> YARN don't work with Spark 1.2
>
> On Sat, Jan 17, 2015 at 9:11 AM, bhavya teja potineni <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=21207&i=0>> wrote:
>
>> Hi
>>
>> Did you try using spark 1.2 on hdp 2.1 YARN
>>
>> Can you please go thru the thread
>>
>> http://apache-spark-user-list.1001560.n3.nabble.com/Troubleshooting-Spark-tt21189.html
>> and check where I am going wrong. As my word count program is erroring
>> out when using spark 1.2 using YARN but its getting executed using spark
>> 0.9.1
>>
>> On Sat, Jan 17, 2015 at 5:55 AM, Chitturi Padma [via Apache Spark User
>> List] <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=21207&i=1>> wrote:
>>
>>> Yes. I built spar 1.2 with apache hadoop 2.2. No compatibility issues.
>>>
>>> On Sat, Jan 17, 2015 at 4:47 AM, bhavyateja [via Apache Spark User List]
>>> <[hidden email] <http:///user/SendEmail.jtp?type=node&node=21202&i=0>>
>>> wrote:
>>>
>>>> Is spark 1.2 is compatibly with HDP 2.1
>>>>
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the
>>>> discussion below:
>>>>
>>>> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197.html
>>>> To start a new topic under Apache Spark User List, email [hidden
>>>> email] <http:///user/SendEmail.jtp?type=node&node=21202&i=1>
>>>> To unsubscribe from Apache Spark User List, click here.
>>>> NAML
>>>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>
>>>
>>>
>>> ------------------------------
>>> If you reply to this email, your message will be added to the
>>> discussion below:
>>>
>>> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21202.html
>>> To unsubscribe from spark 1.2 compatibility, click here.
>>> NAML
>>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>>
>>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21207.html
> To start a new topic under Apache Spark User List, email
> ml-node+s1001560n1h76@n3.nabble.com
> To unsubscribe from Apache Spark User List, click here
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=bGVhcm5pbmdzLmNoaXR0dXJpQGdtYWlsLmNvbXwxfC03NzExMjUwMg==>
> .
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21208.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: spark 1.2 compatibility
Posted by bhavyateja <bh...@gmail.com>.
Hi all,
Thanks for your contribution. We have checked and confirmed that HDP 2.1
YARN don't work with Spark 1.2
On Sat, Jan 17, 2015 at 9:11 AM, bhavya teja potineni <
bhavyateja.potineni@gmail.com> wrote:
> Hi
>
> Did you try using spark 1.2 on hdp 2.1 YARN
>
> Can you please go thru the thread
>
> http://apache-spark-user-list.1001560.n3.nabble.com/Troubleshooting-Spark-tt21189.html
> and check where I am going wrong. As my word count program is erroring out
> when using spark 1.2 using YARN but its getting executed using spark 0.9.1
>
> On Sat, Jan 17, 2015 at 5:55 AM, Chitturi Padma [via Apache Spark User
> List] <ml...@n3.nabble.com> wrote:
>
>> Yes. I built spar 1.2 with apache hadoop 2.2. No compatibility issues.
>>
>> On Sat, Jan 17, 2015 at 4:47 AM, bhavyateja [via Apache Spark User List]
>> <[hidden email] <http:///user/SendEmail.jtp?type=node&node=21202&i=0>>
>> wrote:
>>
>>> Is spark 1.2 is compatibly with HDP 2.1
>>>
>>> ------------------------------
>>> If you reply to this email, your message will be added to the
>>> discussion below:
>>>
>>> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197.html
>>> To start a new topic under Apache Spark User List, email [hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=21202&i=1>
>>> To unsubscribe from Apache Spark User List, click here.
>>> NAML
>>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>>
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the
>> discussion below:
>>
>> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21202.html
>> To unsubscribe from spark 1.2 compatibility, click here
>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=21197&code=YmhhdnlhdGVqYS5wb3RpbmVuaUBnbWFpbC5jb218MjExOTd8LTE1MDEzODgyODY=>
>> .
>> NAML
>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21207.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: spark 1.2 compatibility
Posted by bhavyateja <bh...@gmail.com>.
Hi
Did you try using spark 1.2 on hdp 2.1 YARN
Can you please go thru the thread
http://apache-spark-user-list.1001560.n3.nabble.com/Troubleshooting-Spark-tt21189.html
and check where I am going wrong. As my word count program is erroring out
when using spark 1.2 using YARN but its getting executed using spark 0.9.1
On Sat, Jan 17, 2015 at 5:55 AM, Chitturi Padma [via Apache Spark User
List] <ml...@n3.nabble.com> wrote:
> Yes. I built spar 1.2 with apache hadoop 2.2. No compatibility issues.
>
> On Sat, Jan 17, 2015 at 4:47 AM, bhavyateja [via Apache Spark User List] <[hidden
> email] <http:///user/SendEmail.jtp?type=node&node=21202&i=0>> wrote:
>
>> Is spark 1.2 is compatibly with HDP 2.1
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the
>> discussion below:
>>
>> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197.html
>> To start a new topic under Apache Spark User List, email [hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=21202&i=1>
>> To unsubscribe from Apache Spark User List, click here.
>> NAML
>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21202.html
> To unsubscribe from spark 1.2 compatibility, click here
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=21197&code=YmhhdnlhdGVqYS5wb3RpbmVuaUBnbWFpbC5jb218MjExOTd8LTE1MDEzODgyODY=>
> .
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21205.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: spark 1.2 compatibility
Posted by Chitturi Padma <le...@gmail.com>.
Yes. I built spar 1.2 with apache hadoop 2.2. No compatibility issues.
On Sat, Jan 17, 2015 at 4:47 AM, bhavyateja [via Apache Spark User List] <
ml-node+s1001560n21197h52@n3.nabble.com> wrote:
> Is spark 1.2 is compatibly with HDP 2.1
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197.html
> To start a new topic under Apache Spark User List, email
> ml-node+s1001560n1h76@n3.nabble.com
> To unsubscribe from Apache Spark User List, click here
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=bGVhcm5pbmdzLmNoaXR0dXJpQGdtYWlsLmNvbXwxfC03NzExMjUwMg==>
> .
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-2-compatibility-tp21197p21202.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.