You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by Peeyush Bishnoi <bp...@yahoo.co.in> on 2015/06/04 11:12:44 UTC
Review Request 35058: FALCON-1038: Log mover fails for map-reduce
action
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35058/
-----------------------------------------------------------
Review request for Falcon.
Bugs: FALCON-1038
https://issues.apache.org/jira/browse/FALCON-1038
Repository: falcon-git
Description
-------
Log mover fails for map-reduce action
Diffs
-----
oozie/src/main/java/org/apache/falcon/logging/DefaultTaskLogRetriever.java 962f891
oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java ba669c8
oozie/src/main/java/org/apache/falcon/logging/TaskLogRetrieverYarn.java 61c5afb
Diff: https://reviews.apache.org/r/35058/diff/
Testing
-------
yes
Thanks,
Peeyush Bishnoi
Re: Review Request 35058: FALCON-1038: Log mover fails for map-reduce
action
Posted by Peeyush Bishnoi <bp...@yahoo.co.in>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35058/
-----------------------------------------------------------
(Updated June 10, 2015, 7:57 a.m.)
Review request for Falcon.
Changes
-------
Updated patch after incorporating review comments.
Bugs: FALCON-1038
https://issues.apache.org/jira/browse/FALCON-1038
Repository: falcon-git
Description
-------
Log mover fails for map-reduce action
Diffs (updated)
-----
oozie/src/main/java/org/apache/falcon/logging/DefaultTaskLogRetriever.java 962f891
oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java ba669c8
oozie/src/main/java/org/apache/falcon/logging/TaskLogRetrieverYarn.java 61c5afb
Diff: https://reviews.apache.org/r/35058/diff/
Testing
-------
yes
Thanks,
Peeyush Bishnoi
Re: Review Request 35058: FALCON-1038: Log mover fails for map-reduce
action
Posted by Peeyush Bishnoi <bp...@yahoo.co.in>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35058/
-----------------------------------------------------------
(Updated June 10, 2015, 6:46 a.m.)
Review request for Falcon.
Changes
-------
Updated patch that inlude changes for FALCON-1008.
Bugs: FALCON-1038
https://issues.apache.org/jira/browse/FALCON-1038
Repository: falcon-git
Description
-------
Log mover fails for map-reduce action
Diffs (updated)
-----
oozie/src/main/java/org/apache/falcon/logging/DefaultTaskLogRetriever.java 962f891
oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java ba669c8
oozie/src/main/java/org/apache/falcon/logging/TaskLogRetrieverYarn.java 61c5afb
Diff: https://reviews.apache.org/r/35058/diff/
Testing
-------
yes
Thanks,
Peeyush Bishnoi
Re: Review Request 35058: FALCON-1038: Log mover fails for map-reduce
action
Posted by Peeyush Bishnoi <bp...@yahoo.co.in>.
> On June 8, 2015, 4:36 a.m., Pallavi Rao wrote:
> > Ship It!
Thanks Pallavi for review.
> On June 8, 2015, 4:36 a.m., Pallavi Rao wrote:
> > oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java, line 155
> > <https://reviews.apache.org/r/35058/diff/2/?file=978948#file978948line155>
> >
> > Nit : method name, isSupportedActionType, is more appropriate as the method returns a boolean.
Added isActionTypeSupported.
- Peeyush
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35058/#review86983
-----------------------------------------------------------
On June 10, 2015, 6:46 a.m., Peeyush Bishnoi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35058/
> -----------------------------------------------------------
>
> (Updated June 10, 2015, 6:46 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-1038
> https://issues.apache.org/jira/browse/FALCON-1038
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Log mover fails for map-reduce action
>
>
> Diffs
> -----
>
> oozie/src/main/java/org/apache/falcon/logging/DefaultTaskLogRetriever.java 962f891
> oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java ba669c8
> oozie/src/main/java/org/apache/falcon/logging/TaskLogRetrieverYarn.java 61c5afb
>
> Diff: https://reviews.apache.org/r/35058/diff/
>
>
> Testing
> -------
>
> yes
>
>
> Thanks,
>
> Peeyush Bishnoi
>
>
Re: Review Request 35058: FALCON-1038: Log mover fails for map-reduce
action
Posted by Pallavi Rao <pa...@inmobi.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35058/#review86983
-----------------------------------------------------------
Ship it!
Ship It!
oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java
<https://reviews.apache.org/r/35058/#comment139194>
Nit : method name, isSupportedActionType, is more appropriate as the method returns a boolean.
- Pallavi Rao
On June 4, 2015, 9:19 a.m., Peeyush Bishnoi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35058/
> -----------------------------------------------------------
>
> (Updated June 4, 2015, 9:19 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-1038
> https://issues.apache.org/jira/browse/FALCON-1038
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Log mover fails for map-reduce action
>
>
> Diffs
> -----
>
> oozie/src/main/java/org/apache/falcon/logging/DefaultTaskLogRetriever.java 962f891
> oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java ba669c8
> oozie/src/main/java/org/apache/falcon/logging/TaskLogRetrieverYarn.java 61c5afb
>
> Diff: https://reviews.apache.org/r/35058/diff/
>
>
> Testing
> -------
>
> yes
>
>
> Thanks,
>
> Peeyush Bishnoi
>
>
Re: Review Request 35058: FALCON-1038: Log mover fails for map-reduce
action
Posted by Peeyush Bishnoi <bp...@yahoo.co.in>.
> On June 8, 2015, 6:26 a.m., pavan kumar kolamuri wrote:
> >
Thanks Pavan for review. Please see my comments inline.
> On June 8, 2015, 6:26 a.m., pavan kumar kolamuri wrote:
> > oozie/src/main/java/org/apache/falcon/logging/TaskLogRetrieverYarn.java, line 57
> > <https://reviews.apache.org/r/35058/diff/2/?file=978949#file978949line57>
> >
> > Can we also check whether YARN_LOG_SERVER_URL property was set or not ? Else we should log the message . Otherwise it leads to null pointer exception and user might not understand until he looks into code. Please correct me if there is always default value for this .
Thanks for bringing this up. I have checked the yarn-site.xml and found that yarn.log.server.url is not available as default. I have handled this now.
> On June 8, 2015, 6:26 a.m., pavan kumar kolamuri wrote:
> > oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java, line 146
> > <https://reviews.apache.org/r/35058/diff/2/?file=978948#file978948line146>
> >
> > With this change logs api in falcon might fail . Can you please check whether that is working after this change ? Mostly it won't fail can you please check on safer side.
I have verified this and logs api is not failing. It is returning the path of respective action type log
$ falcon instance -type process -name pig-process -logs -start "2013-11-15T00:05Z"
...
...
actions:
user-action_pig_SUCCEEDED-1.log SUCCEEDED http://c6001.example.com:50070/data/projects/falcon/staging/falcon/workflows/process/pig-process/logs/job-2013-11-15-00-05/000/user-action_pig_SUCCEEDED-1.log
...
- Peeyush
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35058/#review86986
-----------------------------------------------------------
On June 10, 2015, 6:46 a.m., Peeyush Bishnoi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35058/
> -----------------------------------------------------------
>
> (Updated June 10, 2015, 6:46 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-1038
> https://issues.apache.org/jira/browse/FALCON-1038
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Log mover fails for map-reduce action
>
>
> Diffs
> -----
>
> oozie/src/main/java/org/apache/falcon/logging/DefaultTaskLogRetriever.java 962f891
> oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java ba669c8
> oozie/src/main/java/org/apache/falcon/logging/TaskLogRetrieverYarn.java 61c5afb
>
> Diff: https://reviews.apache.org/r/35058/diff/
>
>
> Testing
> -------
>
> yes
>
>
> Thanks,
>
> Peeyush Bishnoi
>
>
Re: Review Request 35058: FALCON-1038: Log mover fails for map-reduce
action
Posted by pavan kumar kolamuri <pa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35058/#review86986
-----------------------------------------------------------
oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java
<https://reviews.apache.org/r/35058/#comment139199>
With this change logs api in falcon might fail . Can you please check whether that is working after this change ? Mostly it won't fail can you please check on safer side.
oozie/src/main/java/org/apache/falcon/logging/TaskLogRetrieverYarn.java
<https://reviews.apache.org/r/35058/#comment139196>
Can we also check whether YARN_LOG_SERVER_URL property was set or not ? Else we should log the message . Otherwise it leads to null pointer exception and user might not understand until he looks into code. Please correct me if there is always default value for this .
- pavan kumar kolamuri
On June 4, 2015, 9:19 a.m., Peeyush Bishnoi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35058/
> -----------------------------------------------------------
>
> (Updated June 4, 2015, 9:19 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-1038
> https://issues.apache.org/jira/browse/FALCON-1038
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Log mover fails for map-reduce action
>
>
> Diffs
> -----
>
> oozie/src/main/java/org/apache/falcon/logging/DefaultTaskLogRetriever.java 962f891
> oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java ba669c8
> oozie/src/main/java/org/apache/falcon/logging/TaskLogRetrieverYarn.java 61c5afb
>
> Diff: https://reviews.apache.org/r/35058/diff/
>
>
> Testing
> -------
>
> yes
>
>
> Thanks,
>
> Peeyush Bishnoi
>
>
Re: Review Request 35058: FALCON-1038: Log mover fails for map-reduce
action
Posted by Peeyush Bishnoi <bp...@yahoo.co.in>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35058/
-----------------------------------------------------------
(Updated June 4, 2015, 9:19 a.m.)
Review request for Falcon.
Bugs: FALCON-1038
https://issues.apache.org/jira/browse/FALCON-1038
Repository: falcon-git
Description
-------
Log mover fails for map-reduce action
Diffs (updated)
-----
oozie/src/main/java/org/apache/falcon/logging/DefaultTaskLogRetriever.java 962f891
oozie/src/main/java/org/apache/falcon/logging/JobLogMover.java ba669c8
oozie/src/main/java/org/apache/falcon/logging/TaskLogRetrieverYarn.java 61c5afb
Diff: https://reviews.apache.org/r/35058/diff/
Testing
-------
yes
Thanks,
Peeyush Bishnoi