You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Yan Zhou (JIRA)" <ji...@apache.org> on 2010/01/22 22:56:21 UTC
[jira] Created: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
[zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
------------------------------------------------------------------------------------------------------------------------------
Key: PIG-1201
URL: https://issues.apache.org/jira/browse/PIG-1201
Project: Pig
Issue Type: Bug
Affects Versions: 0.6.0
Reporter: Yan Zhou
Assignee: Yan Zhou
Priority: Minor
Fix For: 0.7.0
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yan Zhou updated PIG-1201:
--------------------------
Resolution: Fixed
Fix Version/s: (was: 0.7.0)
Status: Resolved (was: Patch Available)
Patch committed to 0.6 branch
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: PIG-1201.patch, PIG-1201.patch, PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yan Zhou updated PIG-1201:
--------------------------
Status: Open (was: Patch Available)
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch, PIG-1201.patch, PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1201) [zebra] HDFS meta queries are issued
by all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835291#action_12835291 ]
Yan Zhou commented on PIG-1201:
-------------------------------
Patch committed to the load-store-redesign branch.
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: PIG-1201.patch, PIG-1201.patch, PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yan Zhou updated PIG-1201:
--------------------------
Attachment: PIG-1201.patch
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1201) [zebra] HDFS meta queries are issued
by all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805812#action_12805812 ]
Hadoop QA commented on PIG-1201:
--------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12431631/PIG-1201.patch
against trunk revision 903030.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/181/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/181/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/181/console
This message is automatically generated.
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch, PIG-1201.patch, PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1201) [zebra] HDFS meta queries are issued
by all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805195#action_12805195 ]
Yan Zhou commented on PIG-1201:
-------------------------------
HDFS listStatus calls by every mapper to the name node is costly, particularly if the target has huge number of disk entries, i.e., files and directories. Zebra has the problem in a couple of ways:
1) for unsorted tables, the index is not built on disk. The input split which is a tfile row split has file index that needs to be mapped to the file name using the index, which contains file names in order and their sizes, by each and every mapper. Building the index makes the listStatus call as it needs info of all files. And if the number of files are huge, this caused name node resource cramps. Instead, the file index can be well replaced with the file name so that the mapping, and consequently the index, is not needed at all for the routine ops like queries against the tables. For other informational requests like dumpInfo where a comprehensive picture is required, the index could be built as needed. The on-disk index is still preferred as it will save one listStatus call by the front end. But it would require more changes to support backward compatibility and the meta file that holds the index does not support versioning. Consequently, this work is deferred to a future release, although the on-disk index will be built for future convinience;
2) Each BasicTable.Reader, at construction, will check and mark all deleted CGs in the SchemaFile.setCGDeletedFlags method, which makes the listStatus call. This may not be as bad as the one in 1), but for the tables with lots of CGs, it could present a problem. Instead, the check can only be made by a front end and passed to mappers the info.
The huge JobConf serialization size in Pig loader implementation will be fixed by only serializing the few configuration variables that Zebra need.
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1201) [zebra] HDFS meta queries are issued
by all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12804443#action_12804443 ]
Vinod K V commented on PIG-1201:
--------------------------------
I am not at all familiar with zebra or pig code, but just checked the patch to verify the JobConf issue.
{code}
- Configuration localConf = new Configuration();
+ Configuration localConf = ConfigurationUtil.toConfiguration(store.getConfiguration());
{code}
{{ConfigurationUtil.toConfiguration()}} still uses a {{new Configuration()}} which will initialize a new object with *all* the default properties from the default resource(*-default.xml)s also populated. So the size of the config object will still be huge, or atleast not small. Instead, I think we should use {{new Configuration(false)}}.
Am I making any sense or are my observations completely invalid?
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yan Zhou updated PIG-1201:
--------------------------
Status: Patch Available (was: Open)
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch, PIG-1201.patch, PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1201) [zebra] HDFS meta queries are issued
by all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12804606#action_12804606 ]
Yan Zhou commented on PIG-1201:
-------------------------------
This is a performance/resource fix. Existing tests are good enough for correctness and sanity coverage and no new tests are required.
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yan Zhou updated PIG-1201:
--------------------------
Status: Patch Available (was: Open)
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1201) [zebra] HDFS meta queries are issued
by all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12804055#action_12804055 ]
Hadoop QA commented on PIG-1201:
--------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12431202/PIG-1201.patch
against trunk revision 902253.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/188/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/188/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/188/console
This message is automatically generated.
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yan Zhou updated PIG-1201:
--------------------------
Status: Open (was: Patch Available)
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Chao Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chao Wang updated PIG-1201:
---------------------------
Patch looks good +1
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch, PIG-1201.patch, PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yan Zhou updated PIG-1201:
--------------------------
Fix Version/s: 0.6.0
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1201) [zebra] HDFS meta queries are issued
by all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12804600#action_12804600 ]
Yan Zhou commented on PIG-1201:
-------------------------------
The point is to minimize the extra serialized size on disk. In other words, the "configMap" serialized to disk in "writeObject" method is the one that caused the problem. localConf is never serialized to disk.
We have tested this change and saw the serialized size on disk drops significantly from around 140K to 1.2K, proving the correctness of the current fix.
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1201) [zebra] HDFS meta queries are issued
by all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805366#action_12805366 ]
Hadoop QA commented on PIG-1201:
--------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12431488/PIG-1201.patch
against trunk revision 903030.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/178/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/178/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/178/console
This message is automatically generated.
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch, PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yan Zhou updated PIG-1201:
--------------------------
Attachment: PIG-1201.patch
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch, PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yan Zhou updated PIG-1201:
--------------------------
Status: Patch Available (was: Open)
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch, PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1201) [zebra] HDFS meta queries are issued by
all mappers; Pig Loader serialize all JobConf contents including those
unused by zebra
Posted by "Yan Zhou (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yan Zhou updated PIG-1201:
--------------------------
Attachment: PIG-1201.patch
> [zebra] HDFS meta queries are issued by all mappers; Pig Loader serialize all JobConf contents including those unused by zebra
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-1201
> URL: https://issues.apache.org/jira/browse/PIG-1201
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Yan Zhou
> Assignee: Yan Zhou
> Priority: Minor
> Fix For: 0.6.0, 0.7.0
>
> Attachments: PIG-1201.patch, PIG-1201.patch, PIG-1201.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.