You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Supreeth (JIRA)" <ji...@apache.org> on 2009/05/14 09:32:45 UTC
[jira] Created: (PIG-809) number of input lines it processed,
number of output lines it produced for PIG job
number of input lines it processed, number of output lines it produced for PIG job
----------------------------------------------------------------------------------
Key: PIG-809
URL: https://issues.apache.org/jira/browse/PIG-809
Project: Pig
Issue Type: Improvement
Components: impl
Environment: Linux
Reporter: Supreeth
Excerpt from the mail conversation.
It will be a great addition to Pig. Hadoop currently provides all these
counters. All Pig has to do is to add them up for all Hadoop jobs in the
script, and emit them at the end of the script. File a jira ?
- Milind
On 5/13/09 8:16 AM, "Supreeth Hosur Nagesh Rao" <su...@yahoo-inc.com>
wrote:
> > Hi Olga
> >
> > With every PIG job is there any way for us to trap into the operational
> > stats of that job, like number of input lines it processed, number of
> > output lines it produced?
> >
> > I dont want to have a separate PIG script to do the same as it may be
> > additional parsing, so is there such a stat. If not can that be
> > provided, and exposed as a config parameter?
> >
> > -Supreeth
This will be a great feature to have for our processing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-809) number of input lines it processed,
number of output lines it produced for PIG job
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding resolved PIG-809.
------------------------------
Resolution: Fixed
> number of input lines it processed, number of output lines it produced for PIG job
> ----------------------------------------------------------------------------------
>
> Key: PIG-809
> URL: https://issues.apache.org/jira/browse/PIG-809
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Environment: Linux
> Reporter: Supreeth
> Assignee: Richard Ding
> Fix For: 0.8.0
>
>
> Excerpt from the mail conversation.
> It will be a great addition to Pig. Hadoop currently provides all these
> counters. All Pig has to do is to add them up for all Hadoop jobs in the
> script, and emit them at the end of the script. File a jira ?
> - Milind
> On 5/13/09 8:16 AM, "Supreeth Hosur Nagesh Rao" <su...@yahoo-inc.com>
> wrote:
> > > Hi Olga
> > >
> > > With every PIG job is there any way for us to trap into the operational
> > > stats of that job, like number of input lines it processed, number of
> > > output lines it produced?
> > >
> > > I dont want to have a separate PIG script to do the same as it may be
> > > additional parsing, so is there such a stat. If not can that be
> > > provided, and exposed as a config parameter?
> > >
> > > -Supreeth
> This will be a great feature to have for our processing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-809) number of input lines it processed,
number of output lines it produced for PIG job
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding reassigned PIG-809:
--------------------------------
Assignee: Richard Ding
> number of input lines it processed, number of output lines it produced for PIG job
> ----------------------------------------------------------------------------------
>
> Key: PIG-809
> URL: https://issues.apache.org/jira/browse/PIG-809
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Environment: Linux
> Reporter: Supreeth
> Assignee: Richard Ding
>
> Excerpt from the mail conversation.
> It will be a great addition to Pig. Hadoop currently provides all these
> counters. All Pig has to do is to add them up for all Hadoop jobs in the
> script, and emit them at the end of the script. File a jira ?
> - Milind
> On 5/13/09 8:16 AM, "Supreeth Hosur Nagesh Rao" <su...@yahoo-inc.com>
> wrote:
> > > Hi Olga
> > >
> > > With every PIG job is there any way for us to trap into the operational
> > > stats of that job, like number of input lines it processed, number of
> > > output lines it produced?
> > >
> > > I dont want to have a separate PIG script to do the same as it may be
> > > additional parsing, so is there such a stat. If not can that be
> > > provided, and exposed as a config parameter?
> > >
> > > -Supreeth
> This will be a great feature to have for our processing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-809) number of input lines it processed,
number of output lines it produced for PIG job
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-809:
-------------------------------
Fix Version/s: 0.8.0
> number of input lines it processed, number of output lines it produced for PIG job
> ----------------------------------------------------------------------------------
>
> Key: PIG-809
> URL: https://issues.apache.org/jira/browse/PIG-809
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Environment: Linux
> Reporter: Supreeth
> Assignee: Richard Ding
> Fix For: 0.8.0
>
>
> Excerpt from the mail conversation.
> It will be a great addition to Pig. Hadoop currently provides all these
> counters. All Pig has to do is to add them up for all Hadoop jobs in the
> script, and emit them at the end of the script. File a jira ?
> - Milind
> On 5/13/09 8:16 AM, "Supreeth Hosur Nagesh Rao" <su...@yahoo-inc.com>
> wrote:
> > > Hi Olga
> > >
> > > With every PIG job is there any way for us to trap into the operational
> > > stats of that job, like number of input lines it processed, number of
> > > output lines it produced?
> > >
> > > I dont want to have a separate PIG script to do the same as it may be
> > > additional parsing, so is there such a stat. If not can that be
> > > provided, and exposed as a config parameter?
> > >
> > > -Supreeth
> This will be a great feature to have for our processing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-809) number of input lines it processed,
number of output lines it produced for PIG job
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882667#action_12882667 ]
Richard Ding commented on PIG-809:
----------------------------------
PIG-1299 & PIG-1389 address this requirement: the number of records read from each user input and written to each user output in a script will be written to the Pig log at the end of execution.
> number of input lines it processed, number of output lines it produced for PIG job
> ----------------------------------------------------------------------------------
>
> Key: PIG-809
> URL: https://issues.apache.org/jira/browse/PIG-809
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Environment: Linux
> Reporter: Supreeth
> Assignee: Richard Ding
> Fix For: 0.8.0
>
>
> Excerpt from the mail conversation.
> It will be a great addition to Pig. Hadoop currently provides all these
> counters. All Pig has to do is to add them up for all Hadoop jobs in the
> script, and emit them at the end of the script. File a jira ?
> - Milind
> On 5/13/09 8:16 AM, "Supreeth Hosur Nagesh Rao" <su...@yahoo-inc.com>
> wrote:
> > > Hi Olga
> > >
> > > With every PIG job is there any way for us to trap into the operational
> > > stats of that job, like number of input lines it processed, number of
> > > output lines it produced?
> > >
> > > I dont want to have a separate PIG script to do the same as it may be
> > > additional parsing, so is there such a stat. If not can that be
> > > provided, and exposed as a config parameter?
> > >
> > > -Supreeth
> This will be a great feature to have for our processing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-809) number of input lines it processed,
number of output lines it produced for PIG job
Posted by "Supreeth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709702#action_12709702 ]
Supreeth commented on PIG-809:
------------------------------
Hi Alan
I see the details in PIG-619, however could you summarize how we can get these numbers for our PIG scripts?
-Supreeth
> number of input lines it processed, number of output lines it produced for PIG job
> ----------------------------------------------------------------------------------
>
> Key: PIG-809
> URL: https://issues.apache.org/jira/browse/PIG-809
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Environment: Linux
> Reporter: Supreeth
>
> Excerpt from the mail conversation.
> It will be a great addition to Pig. Hadoop currently provides all these
> counters. All Pig has to do is to add them up for all Hadoop jobs in the
> script, and emit them at the end of the script. File a jira ?
> - Milind
> On 5/13/09 8:16 AM, "Supreeth Hosur Nagesh Rao" <su...@yahoo-inc.com>
> wrote:
> > > Hi Olga
> > >
> > > With every PIG job is there any way for us to trap into the operational
> > > stats of that job, like number of input lines it processed, number of
> > > output lines it produced?
> > >
> > > I dont want to have a separate PIG script to do the same as it may be
> > > additional parsing, so is there such a stat. If not can that be
> > > provided, and exposed as a config parameter?
> > >
> > > -Supreeth
> This will be a great feature to have for our processing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-809) number of input lines it processed,
number of output lines it produced for PIG job
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709440#action_12709440 ]
Alan Gates commented on PIG-809:
--------------------------------
Is this a duplicate of PIG-619, which was just committed?
> number of input lines it processed, number of output lines it produced for PIG job
> ----------------------------------------------------------------------------------
>
> Key: PIG-809
> URL: https://issues.apache.org/jira/browse/PIG-809
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Environment: Linux
> Reporter: Supreeth
>
> Excerpt from the mail conversation.
> It will be a great addition to Pig. Hadoop currently provides all these
> counters. All Pig has to do is to add them up for all Hadoop jobs in the
> script, and emit them at the end of the script. File a jira ?
> - Milind
> On 5/13/09 8:16 AM, "Supreeth Hosur Nagesh Rao" <su...@yahoo-inc.com>
> wrote:
> > > Hi Olga
> > >
> > > With every PIG job is there any way for us to trap into the operational
> > > stats of that job, like number of input lines it processed, number of
> > > output lines it produced?
> > >
> > > I dont want to have a separate PIG script to do the same as it may be
> > > additional parsing, so is there such a stat. If not can that be
> > > provided, and exposed as a config parameter?
> > >
> > > -Supreeth
> This will be a great feature to have for our processing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-809) number of input lines it processed,
number of output lines it produced for PIG job
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715709#action_12715709 ]
Alan Gates commented on PIG-809:
--------------------------------
Sorry, I referenced the wrong jira in the previous comment. I meant PIG-626.
> number of input lines it processed, number of output lines it produced for PIG job
> ----------------------------------------------------------------------------------
>
> Key: PIG-809
> URL: https://issues.apache.org/jira/browse/PIG-809
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Environment: Linux
> Reporter: Supreeth
>
> Excerpt from the mail conversation.
> It will be a great addition to Pig. Hadoop currently provides all these
> counters. All Pig has to do is to add them up for all Hadoop jobs in the
> script, and emit them at the end of the script. File a jira ?
> - Milind
> On 5/13/09 8:16 AM, "Supreeth Hosur Nagesh Rao" <su...@yahoo-inc.com>
> wrote:
> > > Hi Olga
> > >
> > > With every PIG job is there any way for us to trap into the operational
> > > stats of that job, like number of input lines it processed, number of
> > > output lines it produced?
> > >
> > > I dont want to have a separate PIG script to do the same as it may be
> > > additional parsing, so is there such a stat. If not can that be
> > > provided, and exposed as a config parameter?
> > >
> > > -Supreeth
> This will be a great feature to have for our processing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.