You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hcatalog-commits@incubator.apache.org by "Carl Steinbach (JIRA)" <ji...@apache.org> on 2011/04/28 21:46:03 UTC
[jira] [Created] (HCATALOG-14) Make it possible to build HCatalog
using the HCatalog Git mirror
Make it possible to build HCatalog using the HCatalog Git mirror
----------------------------------------------------------------
Key: HCATALOG-14
URL: https://issues.apache.org/jira/browse/HCATALOG-14
Project: HCatalog
Issue Type: Improvement
Reporter: Carl Steinbach
There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
{noformat}
% ant
Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
hive.jar:
BUILD FAILED
/Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
Total time: 0 seconds
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HCATALOG-14) Make it possible to build HCatalog
using the HCatalog Git mirror
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HCATALOG-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HCATALOG-14:
-----------------------------------
Description:
There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
% ant
Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
hive.jar:
BUILD FAILED
/Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
Total time: 0 seconds
was:
There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
{noformat}
% ant
Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
hive.jar:
BUILD FAILED
/Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
Total time: 0 seconds
{noformat}
> Make it possible to build HCatalog using the HCatalog Git mirror
> ----------------------------------------------------------------
>
> Key: HCATALOG-14
> URL: https://issues.apache.org/jira/browse/HCATALOG-14
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Carl Steinbach
>
> There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
> However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
> % ant
> Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
> hive.jar:
> BUILD FAILED
> /Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
> Total time: 0 seconds
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HCATALOG-14) Make it possible to build HCatalog
using the HCatalog Git mirror
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HCATALOG-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038277#comment-13038277 ]
Carl Steinbach commented on HCATALOG-14:
----------------------------------------
bq. When there's a new feature in Hive's trunk that we need how do we get a new snapshot uploaded?
I modified Hive's Hudson job to automatically run the 'maven-build' and 'maven-publish' tasks, which should hopefully push artifacts to the snapshot repository on every successful build. I'll let you know if I can get this to work. Assuming it does, will this satisfy HCatalog's requirements? Besides the Hive JARs, do you need access to anything else like configuration files or shell scripts?
> Make it possible to build HCatalog using the HCatalog Git mirror
> ----------------------------------------------------------------
>
> Key: HCATALOG-14
> URL: https://issues.apache.org/jira/browse/HCATALOG-14
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Carl Steinbach
> Attachments: hcat-git-2.patch, hcat-git.patch
>
>
> There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
> However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
> % ant
> Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
> hive.jar:
> BUILD FAILED
> /Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
> Total time: 0 seconds
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HCATALOG-14) Make it possible to build HCatalog
using the HCatalog Git mirror
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HCATALOG-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated HCATALOG-14:
--------------------------------
Attachment: hcat-git.patch
Attached patch adds hive as a git submodule, which gits it to build for me under git. Also adds a git-init ant task which would be necessary initially.
Git submodules aren't exactly the same as svn externs, particularly in that you can't point them at specific revisions (per my reading), so this will work as long as we're tracking with trunk. Cutting release branches will be more difficult, but I think this patch should go in to save developers from svn hell.
I cloned the hcat repository at https://github.com/jghoman/hcatalog and applied this patch, if anyone wants to test it. I had trouble applying it against the svn checkout as this patch tries to create hive/extern, which already exists. But this shouldn't cause a problem.
I'm not a git expert, so there may be issues I'm not aware of, but this works for me.
> Make it possible to build HCatalog using the HCatalog Git mirror
> ----------------------------------------------------------------
>
> Key: HCATALOG-14
> URL: https://issues.apache.org/jira/browse/HCATALOG-14
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Carl Steinbach
> Attachments: hcat-git.patch
>
>
> There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
> However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
> % ant
> Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
> hive.jar:
> BUILD FAILED
> /Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
> Total time: 0 seconds
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HCATALOG-14) Make it possible to build HCatalog
using the HCatalog Git mirror
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HCATALOG-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027268#comment-13027268 ]
Alan Gates commented on HCATALOG-14:
------------------------------------
This won't work. In general we plan to have the extern tied to a particular version of Hive, rather than floating with trunk. Otherwise it will get too hard to keep hcat builds going and tests passing.
> Make it possible to build HCatalog using the HCatalog Git mirror
> ----------------------------------------------------------------
>
> Key: HCATALOG-14
> URL: https://issues.apache.org/jira/browse/HCATALOG-14
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Carl Steinbach
> Attachments: hcat-git.patch
>
>
> There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
> However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
> % ant
> Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
> hive.jar:
> BUILD FAILED
> /Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
> Total time: 0 seconds
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HCATALOG-14) Make it possible to
build HCatalog using the HCatalog Git mirror
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HCATALOG-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038277#comment-13038277 ]
Carl Steinbach edited comment on HCATALOG-14 at 5/23/11 10:19 PM:
------------------------------------------------------------------
bq. When there's a new feature in Hive's trunk that we need how do we get a new snapshot uploaded?
I modified Hive's Hudson job to automatically run the 'maven-build' and 'maven-publish' tasks, which should hopefully push artifacts to the snapshot repository on every successful build. I'll let you know if I can get this to work. Assuming it does, will this satisfy HCatalog's requirements? Besides the Hive JARs, do you need access to anything else like configuration files or shell scripts?
PS: Can you ask the INFRA folks to turn on wiki markup for the HCatalog JIRA group?
was (Author: cwsteinbach):
bq. When there's a new feature in Hive's trunk that we need how do we get a new snapshot uploaded?
I modified Hive's Hudson job to automatically run the 'maven-build' and 'maven-publish' tasks, which should hopefully push artifacts to the snapshot repository on every successful build. I'll let you know if I can get this to work. Assuming it does, will this satisfy HCatalog's requirements? Besides the Hive JARs, do you need access to anything else like configuration files or shell scripts?
> Make it possible to build HCatalog using the HCatalog Git mirror
> ----------------------------------------------------------------
>
> Key: HCATALOG-14
> URL: https://issues.apache.org/jira/browse/HCATALOG-14
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Carl Steinbach
> Attachments: hcat-git-2.patch, hcat-git.patch
>
>
> There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
> However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
> % ant
> Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
> hive.jar:
> BUILD FAILED
> /Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
> Total time: 0 seconds
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HCATALOG-14) Make it possible to build HCatalog
using the HCatalog Git mirror
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HCATALOG-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036995#comment-13036995 ]
Jakob Homan commented on HCATALOG-14:
-------------------------------------
Since there's been no response to my query about making svn do what we need it to do (http://bit.ly/jb1TF0), I was wondering why it's necessary to have the extern to Hive included at all, rather than just including a jar to Hive? Hive's now published to Maven: https://repository.apache.org/content/repositories/snapshots/org/apache/hive/, so snapshots should be available. Are developers expecting to create patches against Hive and HCatalog at the same time? What's the expected timeframe for removing the Hive external?
> Make it possible to build HCatalog using the HCatalog Git mirror
> ----------------------------------------------------------------
>
> Key: HCATALOG-14
> URL: https://issues.apache.org/jira/browse/HCATALOG-14
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Carl Steinbach
> Attachments: hcat-git-2.patch, hcat-git.patch
>
>
> There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
> However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
> % ant
> Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
> hive.jar:
> BUILD FAILED
> /Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
> Total time: 0 seconds
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HCATALOG-14) Make it possible to build HCatalog
using the HCatalog Git mirror
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HCATALOG-14?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated HCATALOG-14:
--------------------------------
Attachment: hcat-git-2.patch
Actually, I was wrong. Git tracks specific revisions, which is want we want in the long term but not in the short, since pointing at the 7 branch breaks the compile. Here's a patch that does this. Unfortunately, there's no way to apply it via subversion since git sets a property on the external folder (the specific revision to point to, currently at 6e7d9afd4b6f2118fb173aa516f9e53ae8f6f2f6, the 7 branch).
I can't see a way to set this git property via svn. Anyone know? In the meantime, it works for me to apply this against my working branches, which lets me work on hcat from git. I can then un-apply this patch before generating the actual patch I'm aiming for.
In the long term, since git doesn't support externals, it's probably best to focus on removing the hive extern. I'll ask on Stack Overflow about the svn property, in case I'm missing something.
> Make it possible to build HCatalog using the HCatalog Git mirror
> ----------------------------------------------------------------
>
> Key: HCATALOG-14
> URL: https://issues.apache.org/jira/browse/HCATALOG-14
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Carl Steinbach
> Attachments: hcat-git-2.patch, hcat-git.patch
>
>
> There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
> However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
> % ant
> Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
> hive.jar:
> BUILD FAILED
> /Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
> Total time: 0 seconds
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HCATALOG-14) Make it possible to build HCatalog
using the HCatalog Git mirror
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HCATALOG-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038263#comment-13038263 ]
Alan Gates commented on HCATALOG-14:
------------------------------------
It doesn't appear that this is automatically updated, since the last update is May 17th. I'm not clear who has the power to update these snapshot releases. When there's a new feature in Hive's trunk that we need how do we get a new snapshot uploaded?
In general, I thought Carl had a solution for git users where a target would be added to build that would read the hive version needed from SVN and then check out the proper git reversion. This depended on git knowing the SVN revision numbers so you could check out properly. Carl, did you get a chance to explore whether this would work?
> Make it possible to build HCatalog using the HCatalog Git mirror
> ----------------------------------------------------------------
>
> Key: HCATALOG-14
> URL: https://issues.apache.org/jira/browse/HCATALOG-14
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Carl Steinbach
> Attachments: hcat-git-2.patch, hcat-git.patch
>
>
> There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
> However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
> % ant
> Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
> hive.jar:
> BUILD FAILED
> /Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
> Total time: 0 seconds
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HCATALOG-14) Make it possible to build HCatalog
using the HCatalog Git mirror
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HCATALOG-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038961#comment-13038961 ]
Carl Steinbach commented on HCATALOG-14:
----------------------------------------
The Hive Hudson jobs are now publishing SNAPSHOT artifacts to the apache repository on every successful build:
https://repository.apache.org/content/repositories/snapshots/org/apache/hive/
> Make it possible to build HCatalog using the HCatalog Git mirror
> ----------------------------------------------------------------
>
> Key: HCATALOG-14
> URL: https://issues.apache.org/jira/browse/HCATALOG-14
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Carl Steinbach
> Attachments: hcat-git-2.patch, hcat-git.patch
>
>
> There's a Git mirror of the HCatalog SVN repository available here: git://git.apache.org/hcatalog.git
> However, right now it's not possible to build HCatalog using the Git mirror because of the svn:external dependency on Hive:
> % ant
> Buildfile: /Users/carl/Work/repos/hcatalog/build.xml
> hive.jar:
> BUILD FAILED
> /Users/carl/Work/repos/hcatalog/build.xml:202: Basedir /Users/carl/Work/repos/hcatalog/hive/external does not exist
> Total time: 0 seconds
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira