You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Brock Noland (JIRA)" <ji...@apache.org> on 2013/06/06 18:05:20 UTC
[jira] [Commented] (HIVE-4675) Create new parallel unit test
environment
[ https://issues.apache.org/jira/browse/HIVE-4675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13677173#comment-13677173 ]
Brock Noland commented on HIVE-4675:
------------------------------------
First off, sorry for the large patch. I created this tool as a personal side project. It eventually became quite useful as an internal tool.
I would like to contribute the hive parallel unit testing framework to hive proper. Currently it's on github: https://github.com/brockn/hive-ptest/ but I'd love to generate a patch for hive and upload it here if it's of interest. It was the following features:
-Does not require an NFS filer
-Utilizes multiple disks and multiple cpus on a slave host
-Entire source directory for any test failure is saved
-Highly configurable. A properties file is used for base configuration and many items can be overridden at runtime
-Patch builds can take a URL meaning the patch can be pulled directly from JIRA
-Reliable. We are using this to test all patches internally
-Tests can be easily isolated or ignored
-The framework itself is tested via unit tests
Utilizing 8 hosts internally we have been able to get the trunk unit tests down to 60 minutes.
> Create new parallel unit test environment
> -----------------------------------------
>
> Key: HIVE-4675
> URL: https://issues.apache.org/jira/browse/HIVE-4675
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
> Reporter: Brock Noland
> Assignee: Brock Noland
>
> The current ptest tool is great, but it has the following limitations:
> -Required an NFS filer
> -Unless the NFS filer is dedicated ptests can become IO bound easily
> -Investigating of failures is troublesome because the source directory for the failure is not saved
> -Ignoring or isolated tests is not supported
> -No unit tests for the ptest framework exist
> It'd be great to have a ptest tool that addresses this limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira