You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by ckhari4u <ck...@gmail.com> on 2017/06/08 05:43:08 UTC
Spark History Server does not redirect to Yarn aggregated logs for
container logs
Hey Guys,
I am hitting the below issue when trying to access the STDOUT/STDERR logs in
Spark History Server for the executors of a Spark application executed in
Yarn mode. I have enabled Yarn log aggregation.
Repro Steps:
1) Run the spark-shell in yarn client mode. Or run Pi job in Yarn mode.
2) Once the job is completed, (in the case of spark shell, exit after doing
some simple operations), try to access the STDOUT or STDERR logs of the
application from the Executors tab in the Spark History Server UI.
3) If yarn log aggregation is enabled, then logs won't be available in node
manager's log location. But history Server is trying to access the logs from
the nodemanager's log
location({yarn.nodemanager.log-dirs}/application_${appid) giving below error
in the UI:
Failed redirect for container_e31_1496881617682_0003_01_000002
ResourceManager
RM Home
NodeManager
Tools
Failed while trying to construct the redirect url to the log server. Log
Server url may not be configured
java.lang.Exception: Unknown container. Container either has not started or
has already completed or doesn't belong to this node at all.
Either Spark History Server should be able to read from the aggregated logs
and display the logs in the UI or it should give a graceful message. As of
now its redirecting to the NM webpage and trying to fetch the logs from the
node managers local location.
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-History-Server-does-not-redirect-to-Yarn-aggregated-logs-for-container-logs-tp21706.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Spark History Server does not redirect to Yarn aggregated logs
for container logs
Posted by HariKrishnan CK <ck...@gmail.com>.
Thanks Sai !
I have created a Jira Bug:
https://issues.apache.org/jira/browse/SPARK-21013
Regards n Thanks
Hari
mob: +1 813-510-8215
On Wed, Jun 7, 2017 at 11:44 PM, Saisai Shao <sa...@gmail.com> wrote:
> Yes, currently if log is aggregated, then accessing through UI is not
> worked, you can create a JIRA to improve this if you would like to.
>
> On Thu, Jun 8, 2017 at 1:43 PM, ckhari4u <ck...@gmail.com> wrote:
>
>> Hey Guys,
>>
>> I am hitting the below issue when trying to access the STDOUT/STDERR logs
>> in
>> Spark History Server for the executors of a Spark application executed in
>> Yarn mode. I have enabled Yarn log aggregation.
>>
>> Repro Steps:
>>
>> 1) Run the spark-shell in yarn client mode. Or run Pi job in Yarn mode.
>> 2) Once the job is completed, (in the case of spark shell, exit after
>> doing
>> some simple operations), try to access the STDOUT or STDERR logs of the
>> application from the Executors tab in the Spark History Server UI.
>> 3) If yarn log aggregation is enabled, then logs won't be available in
>> node
>> manager's log location. But history Server is trying to access the logs
>> from
>> the nodemanager's log
>> location({yarn.nodemanager.log-dirs}/application_${appid) giving below
>> error
>> in the UI:
>>
>>
>>
>> Failed redirect for container_e31_1496881617682_0003_01_000002
>> ResourceManager
>> RM Home
>> NodeManager
>> Tools
>> Failed while trying to construct the redirect url to the log server. Log
>> Server url may not be configured
>> java.lang.Exception: Unknown container. Container either has not started
>> or
>> has already completed or doesn't belong to this node at all.
>>
>>
>> Either Spark History Server should be able to read from the aggregated
>> logs
>> and display the logs in the UI or it should give a graceful message. As of
>> now its redirecting to the NM webpage and trying to fetch the logs from
>> the
>> node managers local location.
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-spark-developers
>> -list.1001551.n3.nabble.com/Spark-History-Server-does-not-
>> redirect-to-Yarn-aggregated-logs-for-container-logs-tp21706.html
>> Sent from the Apache Spark Developers List mailing list archive at
>> Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
>
Re: Spark History Server does not redirect to Yarn aggregated logs
for container logs
Posted by Saisai Shao <sa...@gmail.com>.
Yes, currently if log is aggregated, then accessing through UI is not
worked, you can create a JIRA to improve this if you would like to.
On Thu, Jun 8, 2017 at 1:43 PM, ckhari4u <ck...@gmail.com> wrote:
> Hey Guys,
>
> I am hitting the below issue when trying to access the STDOUT/STDERR logs
> in
> Spark History Server for the executors of a Spark application executed in
> Yarn mode. I have enabled Yarn log aggregation.
>
> Repro Steps:
>
> 1) Run the spark-shell in yarn client mode. Or run Pi job in Yarn mode.
> 2) Once the job is completed, (in the case of spark shell, exit after doing
> some simple operations), try to access the STDOUT or STDERR logs of the
> application from the Executors tab in the Spark History Server UI.
> 3) If yarn log aggregation is enabled, then logs won't be available in node
> manager's log location. But history Server is trying to access the logs
> from
> the nodemanager's log
> location({yarn.nodemanager.log-dirs}/application_${appid) giving below
> error
> in the UI:
>
>
>
> Failed redirect for container_e31_1496881617682_0003_01_000002
> ResourceManager
> RM Home
> NodeManager
> Tools
> Failed while trying to construct the redirect url to the log server. Log
> Server url may not be configured
> java.lang.Exception: Unknown container. Container either has not started or
> has already completed or doesn't belong to this node at all.
>
>
> Either Spark History Server should be able to read from the aggregated logs
> and display the logs in the UI or it should give a graceful message. As of
> now its redirecting to the NM webpage and trying to fetch the logs from the
> node managers local location.
>
>
>
>
>
> --
> View this message in context: http://apache-spark-
> developers-list.1001551.n3.nabble.com/Spark-History-
> Server-does-not-redirect-to-Yarn-aggregated-logs-for-
> container-logs-tp21706.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>