You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Kanna Karanam (JIRA)" <ji...@apache.org> on 2012/06/13 19:55:42 UTC
[jira] [Created] (HIVE-3128) Support the CAB format to accommodate
the absence of tar command on windows
Kanna Karanam created HIVE-3128:
-----------------------------------
Summary: Support the CAB format to accommodate the absence of tar command on windows
Key: HIVE-3128
URL: https://issues.apache.org/jira/browse/HIVE-3128
Project: Hive
Issue Type: Bug
Components: CLI, Query Processor
Reporter: Kanna Karanam
TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) use commons-compress instead of
forking tar process
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399851#comment-13399851 ]
Hudson commented on HIVE-3128:
------------------------------
Integrated in Hive-trunk-h0.21 #1506 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1506/])
HIVE-3128 Use commons-compress instead of forking tar process (Kanna Karanam via egc) (Revision 1353044)
Result = FAILURE
ecapriolo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353044
Files :
* /hive/trunk/common/ivy.xml
* /hive/trunk/common/src/java/org/apache/hadoop/hive/common/FileUtils.java
* /hive/trunk/ivy/libraries.properties
> use commons-compress instead of forking tar process
> ---------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Assignee: Kanna Karanam
> Fix For: 0.10.0
>
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) use commons-compress instead of
forking tar process
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399859#comment-13399859 ]
Kanna Karanam commented on HIVE-3128:
-------------------------------------
Hi Edward - It looks like a random failure not related to these changes. Please let me know if you see something that I am missing.
Thanks
> use commons-compress instead of forking tar process
> ---------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Assignee: Kanna Karanam
> Fix For: 0.10.0
>
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) use commons-compress instead of
forking tar process
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401891#comment-13401891 ]
Hudson commented on HIVE-3128:
------------------------------
Integrated in Hive-trunk-h0.21 #1515 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1515/])
HIVE-3180 Fix Eclipse classpath template broken in HIVE-3128. Carl Steinbach (via egc) (Revision 1354187)
Result = FAILURE
ecapriolo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1354187
Files :
* /hive/trunk/eclipse-templates/.classpath
> use commons-compress instead of forking tar process
> ---------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Assignee: Kanna Karanam
> Fix For: 0.10.0
>
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398275#comment-13398275 ]
Carl Steinbach commented on HIVE-3128:
--------------------------------------
Review request: https://reviews.apache.org/r/5474/
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3128) use commons-compress instead of
forking tar process
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Capriolo updated HIVE-3128:
----------------------------------
Summary: use commons-compress instead of forking tar process (was: Support the CAB format to accommodate the absence of tar command on windows)
> use commons-compress instead of forking tar process
> ---------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3128) use commons-compress instead of
forking tar process
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Capriolo updated HIVE-3128:
----------------------------------
Resolution: Fixed
Fix Version/s: 0.10.0
Assignee: Kanna Karanam
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks! Windows support here we come.
> use commons-compress instead of forking tar process
> ---------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Assignee: Kanna Karanam
> Fix For: 0.10.0
>
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13395871#comment-13395871 ]
Edward Capriolo commented on HIVE-3128:
---------------------------------------
If you think you need a follow up Jira please create one, but as this issue is concerned: Do not worry about minimizing the changes. Typically we make large changes rather then a series of smaller ones. This is because the review and test process takes a fair amount of time. That being said it is better to do the changes once correctly. Sometimes there is a good reason that we are doing something round-about like forking a process but many times (including this time as far as I can tell) the code should be changed drastically.
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393552#comment-13393552 ]
Edward Capriolo commented on HIVE-3128:
---------------------------------------
This is great work. The larger problem becomes how are we going to unit test these things and ensure they stay working. Also I think we are talking the wrong approach on this. Rather then making the actual problem better we are making it works. I am -1 on this approach.
Java has tools to build tar and zip files. We should be re-writing these tools in Java rather then writing hooks to do everything the wrong way twice.
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398273#comment-13398273 ]
Kanna Karanam commented on HIVE-3128:
-------------------------------------
Thanks Carl. I submitted the review request.
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3128) Support the CAB format to accommodate
the absence of tar command on windows
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-3128:
---------------------------------
Status: Patch Available (was: Open)
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398570#comment-13398570 ]
Kanna Karanam commented on HIVE-3128:
-------------------------------------
Thanks Edward, My initial observation is It took approximately 235 seconds extra time to complete the full test run on my unix test box. Is there any specific test case that I can run and capture the numbers with/without my changes?
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398425#comment-13398425 ]
Edward Capriolo commented on HIVE-3128:
---------------------------------------
When you say small performance hit. What are the numbers you are seeing? I am going to play with the feature review later today.
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393602#comment-13393602 ]
Edward Capriolo commented on HIVE-3128:
---------------------------------------
We should also avoid forking shells, and doing things like IF (WINDOWS) unless it is unavoidable. In all these windows fixes we can clean some technical debt.
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398239#comment-13398239 ]
Kanna Karanam commented on HIVE-3128:
-------------------------------------
Attached the patch with JAVA based archiving & compression using apache common compression library (commons-compress-1.4.1.jar).
Note: All unit tests are passing but CLI tests are taking little bit extra time. It seems to be a small perf hit because of JAVA vs Native tools.
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3128) Support the CAB format to accommodate
the absence of tar command on windows
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kanna Karanam updated HIVE-3128:
--------------------------------
Attachment: HIVE-3128.1.patch.txt
Attached the patch
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393598#comment-13393598 ]
Ashutosh Chauhan commented on HIVE-3128:
----------------------------------------
I agree with Ed. Pure java based solution is more desirable then a platform specific tools like tar or cabarc. Apache already has one such library which could be used http://commons.apache.org/compress/
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393664#comment-13393664 ]
Kanna Karanam commented on HIVE-3128:
-------------------------------------
Thanks Edward, Ashutosh, It is a good suggestion. In fact, I prefer the platform independent code but I targeted to minimize the number of changes in my initial patches to get it working on Windows and avoid any regressions on UNIX. Let me explore the option to use code based archiving too. Is it ok if we address this in follow-up jira?
Thanks
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398246#comment-13398246 ]
Carl Steinbach commented on HIVE-3128:
--------------------------------------
@Kanna: Please submit a review request on reviews.apache.org. Thanks.
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3128) Support the CAB format to accommodate
the absence of tar command on windows
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kanna Karanam updated HIVE-3128:
--------------------------------
Attachment: HIVE-3128.2.patch.txt
JAVA based archiving & compression using apache common compression library (commons-compress-1.4.1.jar).
Note: All unit tests are passing but CLI tests are taking little bit extra time. It seems to be a small perf hit because of JAVA vs Native tools.
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399620#comment-13399620 ]
Edward Capriolo commented on HIVE-3128:
---------------------------------------
The unit tests usually run multiple hours, if this patch adds less then 5 minutes this is not a big deal.
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) use commons-compress instead of
forking tar process
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399971#comment-13399971 ]
Edward Capriolo commented on HIVE-3128:
---------------------------------------
Unrelated all is well.
> use commons-compress instead of forking tar process
> ---------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Assignee: Kanna Karanam
> Fix For: 0.10.0
>
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) use commons-compress instead of
forking tar process
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422322#comment-13422322 ]
Hudson commented on HIVE-3128:
------------------------------
Integrated in Hive-trunk-h0.21 #1567 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1567/])
HIVE-3295. HIVE-3128 introduced bug causing dynamic partitioning to fail. (kevinwilfong reviewed by njain, ashutoshc) (Revision 1365460)
Result = FAILURE
kevinwilfong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365460
Files :
* /hive/trunk/common/src/java/org/apache/hadoop/hive/common/CompressionUtils.java
* /hive/trunk/common/src/java/org/apache/hadoop/hive/common/FileUtils.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
> use commons-compress instead of forking tar process
> ---------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Assignee: Kanna Karanam
> Fix For: 0.10.0
>
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) use commons-compress instead of
forking tar process
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399665#comment-13399665 ]
Kanna Karanam commented on HIVE-3128:
-------------------------------------
Thanks Edward.
> use commons-compress instead of forking tar process
> ---------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Assignee: Kanna Karanam
> Fix For: 0.10.0
>
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3128) Support the CAB format to
accommodate the absence of tar command on windows
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399651#comment-13399651 ]
Edward Capriolo commented on HIVE-3128:
---------------------------------------
+1 . Will commit
The surface time of tests swings so dramatically the performance of tar is not an issue. In fact a test I did ran faster with your patch then without.
{noformat}
[junit] diff -a /home/edward/hive/trunk/build/ql/test/logs/clientpositive/timestamp_lazy.q.out /home/edward/hive/trunk/ql/src/test/results/clientpositive/timestamp_lazy.q.out
[junit] Done query: timestamp_lazy.q elapsedTime=32s
[junit] diff -a /home/edward/hive/trunk/build/ql/test/logs/clientpositive/timestamp_lazy.q.out /home/edward/hive/trunk/ql/src/test/results/clientpositive/timestamp_lazy.q.out
[junit] Done query: timestamp_lazy.q elapsedTime=38s
{noformat}
> Support the CAB format to accommodate the absence of tar command on windows
> ---------------------------------------------------------------------------
>
> Key: HIVE-3128
> URL: https://issues.apache.org/jira/browse/HIVE-3128
> Project: Hive
> Issue Type: Bug
> Components: CLI, Query Processor
> Reporter: Kanna Karanam
> Attachments: HIVE-3128.1.patch.txt, HIVE-3128.2.patch.txt
>
>
> TAR tool doesn’t exist by default on windows systems so use the CAB files on windows
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira