You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Swarnim Kulkarni (JIRA)" <ji...@apache.org> on 2012/09/10 16:36:07 UTC

[jira] [Created] (HIVE-3449) Speed up ant builds with the ant uptodate task

Swarnim Kulkarni created HIVE-3449:
--------------------------------------

             Summary: Speed up ant builds with the ant uptodate task
                 Key: HIVE-3449
                 URL: https://issues.apache.org/jira/browse/HIVE-3449
             Project: Hive
          Issue Type: Improvement
          Components: Build Infrastructure
    Affects Versions: 0.9.0
            Reporter: Swarnim Kulkarni


Given that the hive build is an enormously loooong build (~ 6hrs), it might be very helpful if there are some checkpointing capabilities available to be able to resume a build from  failed point and not have to restart everything on a single test failure.

One possible way to do this would be to use the ant uptodate task to check set of target files to be more up-to-date than given set of source files and execute a target only if that is true. 

By default this capability could be off but can be activated with the -Dresume=true argument.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3449) Speed up ant builds with the ant uptodate task

Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471144#comment-13471144 ] 

Carl Steinbach commented on HIVE-3449:
--------------------------------------

@Harsh: You can also do this:

% ant clean package -Divy.checkmodified=false


                
> Speed up ant builds with the ant uptodate task
> ----------------------------------------------
>
>                 Key: HIVE-3449
>                 URL: https://issues.apache.org/jira/browse/HIVE-3449
>             Project: Hive
>          Issue Type: Improvement
>          Components: Build Infrastructure
>    Affects Versions: 0.9.0
>            Reporter: Swarnim Kulkarni
>
> Given that the hive build is an enormously loooong build (~ 6hrs), it might be very helpful if there are some checkpointing capabilities available to be able to resume a build from  failed point and not have to restart everything on a single test failure.
> One possible way to do this would be to use the ant uptodate task to check set of target files to be more up-to-date than given set of source files and execute a target only if that is true. 
> By default this capability could be off but can be activated with the -Dresume=true argument.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3449) Speed up ant builds with the ant uptodate task

Posted by "Harsh J (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471088#comment-13471088 ] 

Harsh J commented on HIVE-3449:
-------------------------------

Here's a quick speed hack I did: Delete the {{checkmodified="${ivy.checkmodified}"}} lines in ivy/settings.xml file. This made Hive not try to resolve everything under the sun every single time, dropping the {{clean package}} target build time from over 10 minutes to 1 minute.
                
> Speed up ant builds with the ant uptodate task
> ----------------------------------------------
>
>                 Key: HIVE-3449
>                 URL: https://issues.apache.org/jira/browse/HIVE-3449
>             Project: Hive
>          Issue Type: Improvement
>          Components: Build Infrastructure
>    Affects Versions: 0.9.0
>            Reporter: Swarnim Kulkarni
>
> Given that the hive build is an enormously loooong build (~ 6hrs), it might be very helpful if there are some checkpointing capabilities available to be able to resume a build from  failed point and not have to restart everything on a single test failure.
> One possible way to do this would be to use the ant uptodate task to check set of target files to be more up-to-date than given set of source files and execute a target only if that is true. 
> By default this capability could be off but can be activated with the -Dresume=true argument.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira