You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Rajesh Balamohan (JIRA)" <ji...@apache.org> on 2010/02/05 06:54:28 UTC

[jira] Created: (MAPREDUCE-1461) Feature to instruct rumen-folder utility to skip jobs worth of specific duration

Feature to instruct rumen-folder utility to skip jobs worth of specific duration
--------------------------------------------------------------------------------

                 Key: MAPREDUCE-1461
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1461
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
            Reporter: Rajesh Balamohan
             Fix For: 0.22.0


JSON outputs of rumen on production logs can be huge in the order of multiple GB. Rumen's folder utility helps in getting a smaller snapshot of this JSON data.
It would be helpful to have an option in rumen-folder, wherein user can specify a duration from which rumen-folder should start processing data.

Related JIRA link: https://issues.apache.org/jira/browse/MAPREDUCE-1295

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1461) Feature to instruct rumen-folder utility to skip jobs worth of specific duration

Posted by "Rajesh Balamohan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rajesh Balamohan updated MAPREDUCE-1461:
----------------------------------------

    Attachment: mapreduce-1461--2010-02-05.patch

The attached patch implements this feature. User can specify  the time duration to be skipped by specifying "-starts-after" commandline argument.

> Feature to instruct rumen-folder utility to skip jobs worth of specific duration
> --------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1461
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1461
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Rajesh Balamohan
>             Fix For: 0.22.0
>
>         Attachments: mapreduce-1461--2010-02-05.patch
>
>
> JSON outputs of rumen on production logs can be huge in the order of multiple GB. Rumen's folder utility helps in getting a smaller snapshot of this JSON data.
> It would be helpful to have an option in rumen-folder, wherein user can specify a duration from which rumen-folder should start processing data.
> Related JIRA link: https://issues.apache.org/jira/browse/MAPREDUCE-1295

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1461) Feature to instruct rumen-folder utility to skip jobs worth of specific duration

Posted by "Rajesh Balamohan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rajesh Balamohan updated MAPREDUCE-1461:
----------------------------------------

    Attachment: mapreduce-1461--2010-03-04.patch

I took the trunk version and generated the patch. Please refer the attached file.

> Feature to instruct rumen-folder utility to skip jobs worth of specific duration
> --------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1461
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1461
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Rajesh Balamohan
>             Fix For: 0.22.0
>
>         Attachments: mapreduce-1461--2010-02-05.patch, mapreduce-1461--2010-03-04.patch
>
>
> JSON outputs of rumen on production logs can be huge in the order of multiple GB. Rumen's folder utility helps in getting a smaller snapshot of this JSON data.
> It would be helpful to have an option in rumen-folder, wherein user can specify a duration from which rumen-folder should start processing data.
> Related JIRA link: https://issues.apache.org/jira/browse/MAPREDUCE-1295

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1461) Feature to instruct rumen-folder utility to skip jobs worth of specific duration

Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amar Kamat updated MAPREDUCE-1461:
----------------------------------

    Component/s: tools/rumen

> Feature to instruct rumen-folder utility to skip jobs worth of specific duration
> --------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1461
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1461
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: tools/rumen
>            Reporter: Rajesh Balamohan
>             Fix For: 0.22.0
>
>         Attachments: mapreduce-1461--2010-02-05.patch, mapreduce-1461--2010-03-04.patch
>
>
> JSON outputs of rumen on production logs can be huge in the order of multiple GB. Rumen's folder utility helps in getting a smaller snapshot of this JSON data.
> It would be helpful to have an option in rumen-folder, wherein user can specify a duration from which rumen-folder should start processing data.
> Related JIRA link: https://issues.apache.org/jira/browse/MAPREDUCE-1295

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1461) Feature to instruct rumen-folder utility to skip jobs worth of specific duration

Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hong Tang updated MAPREDUCE-1461:
---------------------------------

    Status: Patch Available  (was: Open)

Mark patch available for hudson to pick up.

> Feature to instruct rumen-folder utility to skip jobs worth of specific duration
> --------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1461
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1461
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Rajesh Balamohan
>             Fix For: 0.22.0
>
>         Attachments: mapreduce-1461--2010-02-05.patch
>
>
> JSON outputs of rumen on production logs can be huge in the order of multiple GB. Rumen's folder utility helps in getting a smaller snapshot of this JSON data.
> It would be helpful to have an option in rumen-folder, wherein user can specify a duration from which rumen-folder should start processing data.
> Related JIRA link: https://issues.apache.org/jira/browse/MAPREDUCE-1295

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (MAPREDUCE-1461) Feature to instruct rumen-folder utility to skip jobs worth of specific duration

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAPREDUCE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832908#action_12832908 ] 

Hadoop QA commented on MAPREDUCE-1461:
--------------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12434931/mapreduce-1461--2010-02-05.patch
  against trunk revision 909241.

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    -1 patch.  The patch command could not apply the patch.

Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/318/console

This message is automatically generated.

> Feature to instruct rumen-folder utility to skip jobs worth of specific duration
> --------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1461
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1461
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Rajesh Balamohan
>             Fix For: 0.22.0
>
>         Attachments: mapreduce-1461--2010-02-05.patch
>
>
> JSON outputs of rumen on production logs can be huge in the order of multiple GB. Rumen's folder utility helps in getting a smaller snapshot of this JSON data.
> It would be helpful to have an option in rumen-folder, wherein user can specify a duration from which rumen-folder should start processing data.
> Related JIRA link: https://issues.apache.org/jira/browse/MAPREDUCE-1295

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (MAPREDUCE-1461) Feature to instruct rumen-folder utility to skip jobs worth of specific duration

Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated MAPREDUCE-1461:
-------------------------------------

    Status: Open  (was: Patch Available)

Please regenerate the patch relative to the root of the source tree

> Feature to instruct rumen-folder utility to skip jobs worth of specific duration
> --------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1461
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1461
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Rajesh Balamohan
>             Fix For: 0.22.0
>
>         Attachments: mapreduce-1461--2010-02-05.patch
>
>
> JSON outputs of rumen on production logs can be huge in the order of multiple GB. Rumen's folder utility helps in getting a smaller snapshot of this JSON data.
> It would be helpful to have an option in rumen-folder, wherein user can specify a duration from which rumen-folder should start processing data.
> Related JIRA link: https://issues.apache.org/jira/browse/MAPREDUCE-1295

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.