You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Sutanu Das <sd...@att.com> on 2016/02/18 23:22:08 UTC
Spark History Server NOT showing Jobs with Hortonworks
Hi Community,
Challenged with Spark issues with Hortonworks (HDP 2.3.2_Spark 1.4.1) - The Spark History Server is NOT showing the Spark Running Jobs in Local Mode
The local-host:4040/app/v1 is ALSO not working
How can I look at my local Spark job?
# Generated by Apache Ambari. Fri Feb 5 00:37:06 2016
spark.history.kerberos.keytab none
spark.history.kerberos.principal none
spark.history.provider org.apache.spark.deploy.yarn.history.YarnHistoryProvider
spark.history.ui.port 18080
spark.yarn.containerLauncherMaxThreads 25
spark.yarn.driver.memoryOverhead 2048
spark.yarn.executor.memoryOverhead 2048
spark.yarn.historyServer.address has-dal-0001.corp.wayport.net:18080
spark.yarn.max.executor.failures 3
spark.yarn.preserve.staging.files false
spark.yarn.queue default
spark.yarn.scheduler.heartbeat.interval-ms 5000
spark.yarn.services org.apache.spark.deploy.yarn.history.YarnHistoryService
spark.yarn.submit.file.replication 3
History Server
* Timeline Service Location: http://has-dal-0002.corp.wayport.net:8188/
* Last Updated: Feb 18, 2016 10:09:12 PM UTC
* Service Started: Feb 5, 2016 12:37:15 AM UTC
* Current Time: Feb 18, 2016 10:10:46 PM UTC
* Timeline Service: Timeline service is enabled
* History Provider: Apache Hadoop YARN Timeline Service
Re: Spark History Server NOT showing Jobs with Hortonworks
Posted by Divya Gehlot <di...@gmail.com>.
Hi Sutanu ,
When you run your spark shell
you would see below lines in your console
16/02/18 21:43:53 INFO AbstractConnector: Started
SelectChannelConnector@0.0.0.0:4041
16/02/18 21:43:53 INFO Utils: Successfully started service 'SparkUI' on
port 4041.
16/02/18 21:43:54 INFO SparkUI: Started SparkUI at http://xx.xx.xx.xxx:4041
As In my case instead of default port the UI started at 4041 port .
Hope this helps.
Thanks,
Divya
On 19 February 2016 at 07:09, Mich Talebzadeh <mi...@peridale.co.uk> wrote:
> Is 4040 port used in your host? It should be default
>
>
>
> Example
>
>
>
> *netstat -plten|grep 4040*
>
>
>
> tcp 0 0 :::4040
> :::* LISTEN 1009 42748209 *22778*/java
>
>
>
> *ps -ef|grep 22778*
>
>
>
> hduser 22778 22770 0 08:34 pts/1 00:01:18 /usr/java/latest/bin/java
> -cp
> /home/hduser/jars/jconn4.jar:/home/hduser/jars/ojdbc6.jar:/usr/lib/spark-1.5.2-bin-hadoop2.6/conf/:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/spark-assembly-1.5.2-hadoop2.6.0.jar:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/home/hduser/hadoop-2.6.0/etc/hadoop/
> -Dscala.usejavacp=true -Xms1G -Xmx1G -XX:MaxPermSize=256m
> org.apache.spark.deploy.SparkSubmit --master spark://50.140.197.217:7077
> --class org.apache.spark.repl.Main --name Spark shell spark-shell
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Technology Ltd, its subsidiaries nor their
> employees accept any responsibility.
>
>
>
>
>
> *From:* Sutanu Das [mailto:sd2302@att.com]
> *Sent:* 18 February 2016 22:58
> *To:* Mich Talebzadeh <mi...@peridale.co.uk>; user@spark.apache.org
>
> *Subject:* RE: Spark History Server NOT showing Jobs with Hortonworks
>
>
>
> Hi Mich, Community - Do I need to specify it in the properties file in my
> spark-submit ?
>
>
>
> *From:* Mich Talebzadeh [mailto:mich@peridale.co.uk <mi...@peridale.co.uk>]
>
> *Sent:* Thursday, February 18, 2016 4:28 PM
> *To:* Sutanu Das; user@spark.apache.org
> *Subject:* RE: Spark History Server NOT showing Jobs with Hortonworks
>
>
>
> The jobs are normally shown under <HOSTNAME>:4040/jobs/ in a normal set up
> not using any vendor’s flavoiur
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Technology Ltd, its subsidiaries nor their
> employees accept any responsibility.
>
>
>
>
>
> *From:* Sutanu Das [mailto:sd2302@att.com <sd...@att.com>]
> *Sent:* 18 February 2016 22:22
> *To:* user@spark.apache.org
> *Subject:* Spark History Server NOT showing Jobs with Hortonworks
>
>
>
> Hi Community,
>
>
>
> Challenged with Spark issues with *Hortonworks* (HDP 2.3.2_Spark 1.4.1)
> – The Spark History Server is NOT showing the Spark Running Jobs in Local
> Mode
>
>
>
> The local-host:4040/app/v1 is ALSO not working
>
>
>
> How can I look at my local Spark job?
>
>
>
>
>
> # Generated by Apache Ambari. Fri Feb 5 00:37:06 2016
>
>
>
> spark.history.kerberos.keytab none
>
> spark.history.kerberos.principal none
>
> spark.history.provider
> org.apache.spark.deploy.yarn.history.YarnHistoryProvider
>
> spark.history.ui.port 18080
>
> spark.yarn.containerLauncherMaxThreads 25
>
> spark.yarn.driver.memoryOverhead 2048
>
> spark.yarn.executor.memoryOverhead 2048
>
> spark.yarn.historyServer.address has-dal-0001.corp.wayport.net:18080
>
> spark.yarn.max.executor.failures 3
>
> spark.yarn.preserve.staging.files false
>
> spark.yarn.queue default
>
> spark.yarn.scheduler.heartbeat.interval-ms 5000
>
> spark.yarn.services org.apache.spark.deploy.yarn.history.YarnHistoryService
>
> spark.yarn.submit.file.replication 3
>
>
>
> *History Server *
>
> - *Timeline Service Location:*
> http://has-dal-0002.corp.wayport.net:8188/
> - *Last Updated:* Feb 18, 2016 10:09:12 PM UTC
> - *Service Started:* Feb 5, 2016 12:37:15 AM UTC
> - *Current Time:* Feb 18, 2016 10:10:46 PM UTC
> - *Timeline Service:* Timeline service is enabled
> - *History Provider:* Apache Hadoop YARN Timeline Service
>
>
>
RE: Spark History Server NOT showing Jobs with Hortonworks
Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
Is 4040 port used in your host? It should be default
Example
netstat -plten|grep 4040
tcp 0 0 :::4040 :::*
LISTEN 1009 42748209 22778/java
ps -ef|grep 22778
hduser 22778 22770 0 08:34 pts/1 00:01:18 /usr/java/latest/bin/java
-cp
/home/hduser/jars/jconn4.jar:/home/hduser/jars/ojdbc6.jar:/usr/lib/spark-1.5
.2-bin-hadoop2.6/conf/:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/spark-assembly
-1.5.2-hadoop2.6.0.jar:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/datanucleus-co
re-3.2.10.jar:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2
.6.jar:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/h
ome/hduser/hadoop-2.6.0/etc/hadoop/ -Dscala.usejavacp=true -Xms1G -Xmx1G
-XX:MaxPermSize=256m org.apache.spark.deploy.SparkSubmit --master
spark://50.140.197.217:7077 --class org.apache.spark.repl.Main --name Spark
shell spark-shell
Dr Mich Talebzadeh
LinkedIn
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABU
rV8Pw>
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUr
V8Pw
<http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com
NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Technology
Ltd, its subsidiaries or their employees, unless expressly so stated. It is
the responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Technology Ltd, its subsidiaries nor their
employees accept any responsibility.
From: Sutanu Das [mailto:sd2302@att.com]
Sent: 18 February 2016 22:58
To: Mich Talebzadeh <mi...@peridale.co.uk>; user@spark.apache.org
Subject: RE: Spark History Server NOT showing Jobs with Hortonworks
Hi Mich, Community - Do I need to specify it in the properties file in my
spark-submit ?
From: Mich Talebzadeh [mailto:mich@peridale.co.uk]
Sent: Thursday, February 18, 2016 4:28 PM
To: Sutanu Das; user@spark.apache.org <ma...@spark.apache.org>
Subject: RE: Spark History Server NOT showing Jobs with Hortonworks
The jobs are normally shown under <HOSTNAME>:4040/jobs/ in a normal set up
not using any vendor's flavoiur
Dr Mich Talebzadeh
LinkedIn
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABU
rV8Pw>
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUr
V8Pw
<http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com
NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Technology
Ltd, its subsidiaries or their employees, unless expressly so stated. It is
the responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Technology Ltd, its subsidiaries nor their
employees accept any responsibility.
From: Sutanu Das [mailto:sd2302@att.com]
Sent: 18 February 2016 22:22
To: user@spark.apache.org <ma...@spark.apache.org>
Subject: Spark History Server NOT showing Jobs with Hortonworks
Hi Community,
Challenged with Spark issues with Hortonworks (HDP 2.3.2_Spark 1.4.1) - The
Spark History Server is NOT showing the Spark Running Jobs in Local Mode
The local-host:4040/app/v1 is ALSO not working
How can I look at my local Spark job?
# Generated by Apache Ambari. Fri Feb 5 00:37:06 2016
spark.history.kerberos.keytab none
spark.history.kerberos.principal none
spark.history.provider
org.apache.spark.deploy.yarn.history.YarnHistoryProvider
spark.history.ui.port 18080
spark.yarn.containerLauncherMaxThreads 25
spark.yarn.driver.memoryOverhead 2048
spark.yarn.executor.memoryOverhead 2048
spark.yarn.historyServer.address has-dal-0001.corp.wayport.net:18080
spark.yarn.max.executor.failures 3
spark.yarn.preserve.staging.files false
spark.yarn.queue default
spark.yarn.scheduler.heartbeat.interval-ms 5000
spark.yarn.services org.apache.spark.deploy.yarn.history.YarnHistoryService
spark.yarn.submit.file.replication 3
History Server
* Timeline Service Location:
http://has-dal-0002.corp.wayport.net:8188/
* Last Updated: Feb 18, 2016 10:09:12 PM UTC
* Service Started: Feb 5, 2016 12:37:15 AM UTC
* Current Time: Feb 18, 2016 10:10:46 PM UTC
* Timeline Service: Timeline service is enabled
* History Provider: Apache Hadoop YARN Timeline Service
RE: Spark History Server NOT showing Jobs with Hortonworks
Posted by Sutanu Das <sd...@att.com>.
Hi Mich, Community - Do I need to specify it in the properties file in my spark-submit ?
From: Mich Talebzadeh [mailto:mich@peridale.co.uk]
Sent: Thursday, February 18, 2016 4:28 PM
To: Sutanu Das; user@spark.apache.org
Subject: RE: Spark History Server NOT showing Jobs with Hortonworks
The jobs are normally shown under <HOSTNAME>:4040/jobs/ in a normal set up not using any vendor's flavoiur
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
http://talebzadehmich.wordpress.com<http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Technology Ltd, its subsidiaries nor their employees accept any responsibility.
From: Sutanu Das [mailto:sd2302@att.com]
Sent: 18 February 2016 22:22
To: user@spark.apache.org<ma...@spark.apache.org>
Subject: Spark History Server NOT showing Jobs with Hortonworks
Hi Community,
Challenged with Spark issues with Hortonworks (HDP 2.3.2_Spark 1.4.1) - The Spark History Server is NOT showing the Spark Running Jobs in Local Mode
The local-host:4040/app/v1 is ALSO not working
How can I look at my local Spark job?
# Generated by Apache Ambari. Fri Feb 5 00:37:06 2016
spark.history.kerberos.keytab none
spark.history.kerberos.principal none
spark.history.provider org.apache.spark.deploy.yarn.history.YarnHistoryProvider
spark.history.ui.port 18080
spark.yarn.containerLauncherMaxThreads 25
spark.yarn.driver.memoryOverhead 2048
spark.yarn.executor.memoryOverhead 2048
spark.yarn.historyServer.address has-dal-0001.corp.wayport.net:18080
spark.yarn.max.executor.failures 3
spark.yarn.preserve.staging.files false
spark.yarn.queue default
spark.yarn.scheduler.heartbeat.interval-ms 5000
spark.yarn.services org.apache.spark.deploy.yarn.history.YarnHistoryService
spark.yarn.submit.file.replication 3
History Server
* Timeline Service Location: http://has-dal-0002.corp.wayport.net:8188/
* Last Updated: Feb 18, 2016 10:09:12 PM UTC
* Service Started: Feb 5, 2016 12:37:15 AM UTC
* Current Time: Feb 18, 2016 10:10:46 PM UTC
* Timeline Service: Timeline service is enabled
* History Provider: Apache Hadoop YARN Timeline Service
RE: Spark History Server NOT showing Jobs with Hortonworks
Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
The jobs are normally shown under <HOSTNAME>:4040/jobs/ in a normal set up
not using any vendor's flavoiur
Dr Mich Talebzadeh
LinkedIn
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABU
rV8Pw>
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUr
V8Pw
<http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com
NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Technology
Ltd, its subsidiaries or their employees, unless expressly so stated. It is
the responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Technology Ltd, its subsidiaries nor their
employees accept any responsibility.
From: Sutanu Das [mailto:sd2302@att.com]
Sent: 18 February 2016 22:22
To: user@spark.apache.org
Subject: Spark History Server NOT showing Jobs with Hortonworks
Hi Community,
Challenged with Spark issues with Hortonworks (HDP 2.3.2_Spark 1.4.1) - The
Spark History Server is NOT showing the Spark Running Jobs in Local Mode
The local-host:4040/app/v1 is ALSO not working
How can I look at my local Spark job?
# Generated by Apache Ambari. Fri Feb 5 00:37:06 2016
spark.history.kerberos.keytab none
spark.history.kerberos.principal none
spark.history.provider
org.apache.spark.deploy.yarn.history.YarnHistoryProvider
spark.history.ui.port 18080
spark.yarn.containerLauncherMaxThreads 25
spark.yarn.driver.memoryOverhead 2048
spark.yarn.executor.memoryOverhead 2048
spark.yarn.historyServer.address has-dal-0001.corp.wayport.net:18080
spark.yarn.max.executor.failures 3
spark.yarn.preserve.staging.files false
spark.yarn.queue default
spark.yarn.scheduler.heartbeat.interval-ms 5000
spark.yarn.services org.apache.spark.deploy.yarn.history.YarnHistoryService
spark.yarn.submit.file.replication 3
History Server
* Timeline Service Location:
http://has-dal-0002.corp.wayport.net:8188/
* Last Updated: Feb 18, 2016 10:09:12 PM UTC
* Service Started: Feb 5, 2016 12:37:15 AM UTC
* Current Time: Feb 18, 2016 10:10:46 PM UTC
* Timeline Service: Timeline service is enabled
* History Provider: Apache Hadoop YARN Timeline Service
Re: Spark History Server NOT showing Jobs with Hortonworks
Posted by Steve Loughran <st...@hortonworks.com>.
this is set up to save history to the timeline service, something which works provided the applications are all set up to publish there too.
On 18 Feb 2016, at 22:22, Sutanu Das <sd...@att.com>> wrote:
Hi Community,
Challenged with Spark issues with Hortonworks (HDP 2.3.2_Spark 1.4.1) – The Spark History Server is NOT showing the Spark Running Jobs in Local Mode
The local-host:4040/app/v1 is ALSO not working
How can I look at my local Spark job?
# Generated by Apache Ambari. Fri Feb 5 00:37:06 2016
spark.history.kerberos.keytab none
spark.history.kerberos.principal none
this tells the history server to use ATS
spark.history.provider org.apache.spark.deploy.yarn.history.YarnHistoryProvider
spark.history.ui.port 18080
spark.yarn.containerLauncherMaxThreads 25
spark.yarn.driver.memoryOverhead 2048
spark.yarn.executor.memoryOverhead 2048
spark.yarn.historyServer.address has-dal-0001.corp.wayport.net<http://has-dal-0001.corp.wayport.net/>:18080
spark.yarn.max.executor.failures 3
spark.yarn.preserve.staging.files false
spark.yarn.queue default
spark.yarn.scheduler.heartbeat.interval-ms 5000
this says: publish via it
spark.yarn.services org.apache.spark.deploy.yarn.history.YarnHistoryService
spark.yarn.submit.file.replication 3
There's some asynchronous publishing, so things don't appear immediately as the app starts, and the updates can take a bit to trickle out, but things look set up right to work both ways.
I'll email you off the list and see if I can help track down what's happening
-steve