You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Vivek Padmanabhan (Created) (JIRA)" <ji...@apache.org> on 2011/11/11 10:42:51 UTC
[jira] [Created] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
JobStats.getHadoopCounters() is never set and always returns null
-----------------------------------------------------------------
Key: PIG-2358
URL: https://issues.apache.org/jira/browse/PIG-2358
Project: Pig
Issue Type: Bug
Affects Versions: 0.9.1
Reporter: Vivek Padmanabhan
The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
{code}
PigStats stats = PigRunner.run(args, null);
PigStats.JobGraph jobGraph = stats.getJobGraph();
for (JobStats jobStats : jobGraph) {
Counters counters = jobStats.getHadoopCounters();
System.out.println(counters);// prints null
}
{code}
But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "Olga Natkovich (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-2358:
--------------------------------
Alan, Daniel - could you please confirm if this patch is ready to go and commit if possible, thanks
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358-1.patch, PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "Alan Gates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-2358:
----------------------------
Status: Open (was: Patch Available)
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "Daniel Dai (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158625#comment-13158625 ]
Daniel Dai commented on PIG-2358:
---------------------------------
Unit tests pass for me.
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuting zhao updated PIG-2358:
-----------------------------
Status: Open (was: Patch Available)
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "Daniel Dai (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2358:
----------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Unit tests pass. test-patch:
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] -1 release audit. The applied patch generated 463 release audit warnings (more than the trunk's current 456 warnings).
There is no new file added, ignore release audit warning.
Patch committed to both trunk and 0.10 branch.
Thanks Xuting!
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358-1.patch, PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154692#comment-13154692 ]
xuting zhao commented on PIG-2358:
----------------------------------
Ant test-commit has been successfully run on this patch.
A Junit test has been added on the getHadoopCounter method.
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuting zhao updated PIG-2358:
-----------------------------
Attachment: PIG-2358-1.patch
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358-1.patch, PIG-2358.patch, PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuting zhao updated PIG-2358:
-----------------------------
Status: Patch Available (was: Open)
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "Virag Kothari (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148653#comment-13148653 ]
Virag Kothari commented on PIG-2358:
------------------------------------
The API JobStats.getHadoopCounters() is very important as I can't get all hadoop counters through other API's (E.g. FILE_BYTES_READ, FILE_BYTES_WRITTEN, COMBINE_INPUT_RECORDS and many more)
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "Alan Gates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-2358:
----------------------------
Attachment: TEST-org.apache.pig.test.TestPigRunner.txt
Log file from failing test.
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuting zhao updated PIG-2358:
-----------------------------
Attachment: PIG-2358.patch
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuting zhao updated PIG-2358:
-----------------------------
Affects Version/s: (was: 0.9.1)
0.10
Status: Patch Available (was: Open)
I have successfully run the additional unit test and the ant test-commit on both trunk and 0.10 branch. Please let me know if there is any problems. Thanks :)
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358-1.patch, PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuting zhao updated PIG-2358:
-----------------------------
Status: Patch Available (was: Open)
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159003#comment-13159003 ]
xuting zhao commented on PIG-2358:
----------------------------------
I modified the unit test and run successfully the test-commit as well as the TestPigRunner Junit test on trunk.
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch, PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuting zhao updated PIG-2358:
-----------------------------
Attachment: PIG-2358.patch
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch, PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156339#comment-13156339 ]
xuting zhao commented on PIG-2358:
----------------------------------
Hi Alan,
Thanks for the comments. I downloaded the trunk code from svn, applied the patch and run " ant test -Dtestcase=TestPigRunner" and it seems ran successfully. May I know what is your build environment? Thanks.
Xuting
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "Daniel Dai (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161087#comment-13161087 ]
Daniel Dai commented on PIG-2358:
---------------------------------
Running tests now.
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358-1.patch, PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuting zhao updated PIG-2358:
-----------------------------
Attachment: (was: PIG-2358.patch)
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358-1.patch, PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuting zhao updated PIG-2358:
-----------------------------
Attachment: PIG-2358.patch
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "Alan Gates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156225#comment-13156225 ]
Alan Gates commented on PIG-2358:
---------------------------------
I see quite a few failures in TestPigRunner when I apply this patch. I'll upload the log file from the test.
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358.patch
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "Daniel Dai (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161488#comment-13161488 ]
Daniel Dai commented on PIG-2358:
---------------------------------
Tests pass. Patch looks good. I will commit it.
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
> Attachments: PIG-2358-1.patch, PIG-2358.patch, TEST-org.apache.pig.test.TestPigRunner.txt
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "xuting zhao (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuting zhao updated PIG-2358:
-----------------------------
Attachment: (was: PIG-2358.patch)
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never
set and always returns null
Posted by "Olga Natkovich (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-2358:
--------------------------------
Fix Version/s: 0.10
Assignee: xuting zhao
> JobStats.getHadoopCounters() is never set and always returns null
> -----------------------------------------------------------------
>
> Key: PIG-2358
> URL: https://issues.apache.org/jira/browse/PIG-2358
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.1
> Reporter: Vivek Padmanabhan
> Assignee: xuting zhao
> Fix For: 0.10
>
>
> The api JobStats.getHadoopCounters() always returns null since the instance variable returned from the method ("counters") is never set.
> {code}
> PigStats stats = PigRunner.run(args, null);
> PigStats.JobGraph jobGraph = stats.getJobGraph();
>
> for (JobStats jobStats : jobGraph) {
> Counters counters = jobStats.getHadoopCounters();
> System.out.println(counters);// prints null
> }
> {code}
> But of course I can get individual counter information through the other APIs (getHdfsBytesWritten(),getMapInputRecords() etc).
> I guess the change came in as part of PIG-1389 (the instance variable got replaced with method local variable in JobStats.addCounters).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira