You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Mona Chitnis <mo...@yahoo.in> on 2013/11/01 00:23:02 UTC
Review Request 15159: OOZIE-1474 Fix logging issues - latency,
accurate job ids, coord Job UI to show job logs
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15159/
-----------------------------------------------------------
Review request for oozie.
Bugs: OOZIE-1474
https://issues.apache.org/jira/browse/OOZIE-1474
Repository: oozie
Description
-------
Back end change to batch log streaming + front-end change for Coord 'Job' UI (web-console only displays coord action UI).
Diffs
-----
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1536925
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java 1536925
trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1536925
trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java 1536925
trunk/core/src/main/java/org/apache/oozie/service/XLogStreamingService.java 1536925
trunk/core/src/main/java/org/apache/oozie/util/LogUtils.java 1536925
trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java 1536925
trunk/core/src/main/java/org/apache/oozie/util/XLogStreamer.java 1536925
trunk/webapp/src/main/webapp/oozie-console.js 1536925
Diff: https://reviews.apache.org/r/15159/diff/
Testing
-------
Thanks,
Mona Chitnis
Re: Review Request 15159: OOZIE-1474 Fix logging issues - latency,
accurate job ids, coord Job UI to show job logs
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15159/#review29121
-----------------------------------------------------------
trunk/core/src/main/java/org/apache/oozie/service/ZKXLogStreamingService.java
<https://reviews.apache.org/r/15159/#comment56254>
You need to flush after this also based on the buffer length
trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java
<https://reviews.apache.org/r/15159/#comment56248>
Need to revert javadoc
trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java
<https://reviews.apache.org/r/15159/#comment56249>
Need to revert javadoc
trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java
<https://reviews.apache.org/r/15159/#comment56251>
Flushes the log writing
trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java
<https://reviews.apache.org/r/15159/#comment56250>
bytesWritten
trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java
<https://reviews.apache.org/r/15159/#comment56252>
Not required
trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java
<https://reviews.apache.org/r/15159/#comment56253>
Have a writer.flush() after the while loop
- Rohini Palaniswamy
On Nov. 19, 2013, 4:18 a.m., Mona Chitnis wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15159/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2013, 4:18 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1474
> https://issues.apache.org/jira/browse/OOZIE-1474
>
>
> Repository: oozie
>
>
> Description
> -------
>
> Back end change to batch log streaming + front-end change for Coord 'Job' UI (web-console only displays coord action UI).
>
>
> Diffs
> -----
>
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1543301
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java 1543301
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1543301
> trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java 1543301
> trunk/core/src/main/java/org/apache/oozie/service/XLogStreamingService.java 1543301
> trunk/core/src/main/java/org/apache/oozie/service/ZKXLogStreamingService.java 1543301
> trunk/core/src/main/java/org/apache/oozie/util/LogUtils.java 1543301
> trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java 1543301
> trunk/core/src/main/java/org/apache/oozie/util/XLogStreamer.java 1543301
> trunk/core/src/main/resources/oozie-default.xml 1543301
> trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionNotificationXCommand.java 1543301
> trunk/core/src/test/java/org/apache/oozie/command/wf/TestNotificationXCommand.java 1543301
> trunk/core/src/test/java/org/apache/oozie/service/TestXLogStreamingService.java 1543301
> trunk/core/src/test/java/org/apache/oozie/util/TestLogStreamer.java 1543301
> trunk/webapp/src/main/webapp/oozie-console.js 1543301
>
> Diff: https://reviews.apache.org/r/15159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mona Chitnis
>
>
Re: Review Request 15159: OOZIE-1474 Fix logging issues - latency,
accurate job ids, coord Job UI to show job logs
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15159/#review29447
-----------------------------------------------------------
Ship it!
Formatting seems to be off in the new patch uploaded. Just check that.
trunk/core/src/main/java/org/apache/oozie/service/ZKXLogStreamingService.java
<https://reviews.apache.org/r/15159/#comment56660>
This change is not required. But is ok
- Rohini Palaniswamy
On Nov. 26, 2013, 1:04 a.m., Mona Chitnis wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15159/
> -----------------------------------------------------------
>
> (Updated Nov. 26, 2013, 1:04 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1474
> https://issues.apache.org/jira/browse/OOZIE-1474
>
>
> Repository: oozie
>
>
> Description
> -------
>
> Back end change to batch log streaming + front-end change for Coord 'Job' UI (web-console only displays coord action UI).
>
>
> Diffs
> -----
>
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1545415
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java 1545415
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1545415
> trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 1545415
> trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java 1545415
> trunk/core/src/main/java/org/apache/oozie/service/XLogStreamingService.java 1545415
> trunk/core/src/main/java/org/apache/oozie/service/ZKXLogStreamingService.java 1545415
> trunk/core/src/main/java/org/apache/oozie/util/LogUtils.java 1545415
> trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java 1545415
> trunk/core/src/main/java/org/apache/oozie/util/XLogStreamer.java 1545415
> trunk/core/src/main/resources/oozie-default.xml 1545415
> trunk/core/src/test/java/org/apache/oozie/util/TestLogStreamer.java 1545415
> trunk/webapp/src/main/webapp/oozie-console.js 1545415
>
> Diff: https://reviews.apache.org/r/15159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mona Chitnis
>
>
Re: Review Request 15159: OOZIE-1474 Fix logging issues - latency,
accurate job ids, coord Job UI to show job logs
Posted by Mona Chitnis <mo...@yahoo.in>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15159/
-----------------------------------------------------------
(Updated Nov. 26, 2013, 1:04 a.m.)
Review request for oozie.
Changes
-------
addressed review comments. removed 2 unit tests from patch as they are fixed as part of OOZIE-1581.
Bugs: OOZIE-1474
https://issues.apache.org/jira/browse/OOZIE-1474
Repository: oozie
Description
-------
Back end change to batch log streaming + front-end change for Coord 'Job' UI (web-console only displays coord action UI).
Diffs (updated)
-----
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1545415
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java 1545415
trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1545415
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java 1545415
trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java 1545415
trunk/core/src/main/java/org/apache/oozie/service/XLogStreamingService.java 1545415
trunk/core/src/main/java/org/apache/oozie/service/ZKXLogStreamingService.java 1545415
trunk/core/src/main/java/org/apache/oozie/util/LogUtils.java 1545415
trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java 1545415
trunk/core/src/main/java/org/apache/oozie/util/XLogStreamer.java 1545415
trunk/core/src/main/resources/oozie-default.xml 1545415
trunk/core/src/test/java/org/apache/oozie/util/TestLogStreamer.java 1545415
trunk/webapp/src/main/webapp/oozie-console.js 1545415
Diff: https://reviews.apache.org/r/15159/diff/
Testing
-------
Thanks,
Mona Chitnis
Re: Review Request 15159: OOZIE-1474 Fix logging issues - latency,
accurate job ids, coord Job UI to show job logs
Posted by Mona Chitnis <mo...@yahoo.in>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15159/
-----------------------------------------------------------
(Updated Nov. 19, 2013, 4:18 a.m.)
Review request for oozie.
Changes
-------
incorporated review comments, fixed flaky tests, tested jobid + actionid correct with ActionExecutor Logs.
Bugs: OOZIE-1474
https://issues.apache.org/jira/browse/OOZIE-1474
Repository: oozie
Description
-------
Back end change to batch log streaming + front-end change for Coord 'Job' UI (web-console only displays coord action UI).
Diffs (updated)
-----
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1543301
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java 1543301
trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1543301
trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java 1543301
trunk/core/src/main/java/org/apache/oozie/service/XLogStreamingService.java 1543301
trunk/core/src/main/java/org/apache/oozie/service/ZKXLogStreamingService.java 1543301
trunk/core/src/main/java/org/apache/oozie/util/LogUtils.java 1543301
trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java 1543301
trunk/core/src/main/java/org/apache/oozie/util/XLogStreamer.java 1543301
trunk/core/src/main/resources/oozie-default.xml 1543301
trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionNotificationXCommand.java 1543301
trunk/core/src/test/java/org/apache/oozie/command/wf/TestNotificationXCommand.java 1543301
trunk/core/src/test/java/org/apache/oozie/service/TestXLogStreamingService.java 1543301
trunk/core/src/test/java/org/apache/oozie/util/TestLogStreamer.java 1543301
trunk/webapp/src/main/webapp/oozie-console.js 1543301
Diff: https://reviews.apache.org/r/15159/diff/
Testing
-------
Thanks,
Mona Chitnis
Re: Review Request 15159: OOZIE-1474 Fix logging issues - latency,
accurate job ids, coord Job UI to show job logs
Posted by Mona Chitnis <mo...@yahoo.in>.
> On Nov. 13, 2013, 6:31 p.m., Rohini Palaniswamy wrote:
> > trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java, lines 780-781
> > <https://reviews.apache.org/r/15159/diff/2/?file=381356#file381356line780>
> >
> > Should this be done in other places in this class like start(..) as well?
moving it to the caller function start() itself, then it will get applied in the beginning itself to all callee methods - submitLauncher, check etc
> On Nov. 13, 2013, 6:31 p.m., Rohini Palaniswamy wrote:
> > trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java, lines 659-660
> > <https://reviews.apache.org/r/15159/diff/2/?file=381359#file381359line659>
> >
> > why remove this?
discussed offline, retained.
- Mona
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15159/#review28763
-----------------------------------------------------------
On Nov. 8, 2013, 11:12 p.m., Mona Chitnis wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15159/
> -----------------------------------------------------------
>
> (Updated Nov. 8, 2013, 11:12 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1474
> https://issues.apache.org/jira/browse/OOZIE-1474
>
>
> Repository: oozie
>
>
> Description
> -------
>
> Back end change to batch log streaming + front-end change for Coord 'Job' UI (web-console only displays coord action UI).
>
>
> Diffs
> -----
>
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1538746
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java 1538746
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1538746
> trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java 1538746
> trunk/core/src/main/java/org/apache/oozie/service/XLogStreamingService.java 1538746
> trunk/core/src/main/java/org/apache/oozie/util/LogUtils.java 1538746
> trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java 1538746
> trunk/core/src/main/java/org/apache/oozie/util/XLogStreamer.java 1538746
> trunk/webapp/src/main/webapp/oozie-console.js 1538746
>
> Diff: https://reviews.apache.org/r/15159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mona Chitnis
>
>
Re: Review Request 15159: OOZIE-1474 Fix logging issues - latency,
accurate job ids, coord Job UI to show job logs
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15159/#review28763
-----------------------------------------------------------
This does not handle HA log streaming. Need to fix that as well.
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/15159/#comment55771>
With this change, does the user and job information appear in these logging statements?
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/15159/#comment55773>
Should this be done in other places in this class like start(..) as well?
trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java
<https://reviews.apache.org/r/15159/#comment55774>
why remove this?
trunk/core/src/main/java/org/apache/oozie/service/XLogStreamingService.java
<https://reviews.apache.org/r/15159/#comment55775>
Have 4K as default instead of -1
trunk/webapp/src/main/webapp/oozie-console.js
<https://reviews.apache.org/r/15159/#comment55776>
|| actionsText == ""
- Rohini Palaniswamy
On Nov. 8, 2013, 11:12 p.m., Mona Chitnis wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15159/
> -----------------------------------------------------------
>
> (Updated Nov. 8, 2013, 11:12 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1474
> https://issues.apache.org/jira/browse/OOZIE-1474
>
>
> Repository: oozie
>
>
> Description
> -------
>
> Back end change to batch log streaming + front-end change for Coord 'Job' UI (web-console only displays coord action UI).
>
>
> Diffs
> -----
>
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1538746
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java 1538746
> trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1538746
> trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java 1538746
> trunk/core/src/main/java/org/apache/oozie/service/XLogStreamingService.java 1538746
> trunk/core/src/main/java/org/apache/oozie/util/LogUtils.java 1538746
> trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java 1538746
> trunk/core/src/main/java/org/apache/oozie/util/XLogStreamer.java 1538746
> trunk/webapp/src/main/webapp/oozie-console.js 1538746
>
> Diff: https://reviews.apache.org/r/15159/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mona Chitnis
>
>
Re: Review Request 15159: OOZIE-1474 Fix logging issues - latency,
accurate job ids, coord Job UI to show job logs
Posted by Mona Chitnis <mo...@yahoo.in>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15159/
-----------------------------------------------------------
(Updated Nov. 8, 2013, 11:12 p.m.)
Review request for oozie.
Changes
-------
simplified buffering logic
Bugs: OOZIE-1474
https://issues.apache.org/jira/browse/OOZIE-1474
Repository: oozie
Description
-------
Back end change to batch log streaming + front-end change for Coord 'Job' UI (web-console only displays coord action UI).
Diffs (updated)
-----
trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 1538746
trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java 1538746
trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java 1538746
trunk/core/src/main/java/org/apache/oozie/service/CallableQueueService.java 1538746
trunk/core/src/main/java/org/apache/oozie/service/XLogStreamingService.java 1538746
trunk/core/src/main/java/org/apache/oozie/util/LogUtils.java 1538746
trunk/core/src/main/java/org/apache/oozie/util/TimestampedMessageParser.java 1538746
trunk/core/src/main/java/org/apache/oozie/util/XLogStreamer.java 1538746
trunk/webapp/src/main/webapp/oozie-console.js 1538746
Diff: https://reviews.apache.org/r/15159/diff/
Testing
-------
Thanks,
Mona Chitnis