You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Lars Francke (JIRA)" <ji...@apache.org> on 2010/01/15 02:50:54 UTC
[jira] Created: (HBASE-2128) ant tar build broken since switch to
Ivy
ant tar build broken since switch to Ivy
----------------------------------------
Key: HBASE-2128
URL: https://issues.apache.org/jira/browse/HBASE-2128
Project: Hadoop HBase
Issue Type: Bug
Components: build
Affects Versions: 0.21.0
Reporter: Lars Francke
Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
{code:xml}
<mkdir dir="${dist.dir}/lib"/>
<copy todir="${dist.dir}/lib">
<fileset dir="${build.lib}" />
<fileset dir="${common.ivy.lib.dir}"/>
</copy>
{code]
The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801258#action_12801258 ]
Andrew Purtell commented on HBASE-2128:
---------------------------------------
bq. i think it would be better if ec2 wasnt part of the core build stuff. seems kind of weird...
It's not. It's a contrib -- src/contrib/ec2/ ... Or did I miss the point?
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated HBASE-2128:
---------------------------
Attachment: HBASE-2128.patch
Do one at a time. Do not deal with javadoc issue here. See HBASE-2135 for javadoc cp issue
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-2128.
--------------------------
Resolution: Fixed
Fix Version/s: 0.21.0
Assignee: Kay Kay
Hadoop Flags: [Reviewed]
Applied to TRUNK. Thanks for the patch Kay Kay.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Assignee: Kay Kay
> Fix For: 0.21.0
>
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Lars Francke (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Francke updated HBASE-2128:
--------------------------------
Description:
Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
{code:xml}
<mkdir dir="${dist.dir}/lib"/>
<copy todir="${dist.dir}/lib">
<fileset dir="${build.lib}" />
<fileset dir="${common.ivy.lib.dir}"/>
</copy>
{code}
The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
was:
Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
{code:xml}
<mkdir dir="${dist.dir}/lib"/>
<copy todir="${dist.dir}/lib">
<fileset dir="${build.lib}" />
<fileset dir="${common.ivy.lib.dir}"/>
</copy>
{code]
The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801101#action_12801101 ]
Kay Kay commented on HBASE-2128:
--------------------------------
{quote}
My preferred way of doing this would be to keep only the extra dependencies for the contribs in their own folders.
{quote}
That would be the ideal thing I guess. That would mean - revisiting build.xml / build-contrib.xml to reuse the files, to create only the 'diff' dependencies. Given that 'tar' is broken - I am trying to look for something quick and immediate.
{quote}
In all likelihood I'm missing something important here but I don't see why all jars would have to be duplicated
{quote}
Agree - that was just meant to have contrib-s as separated namespace. If the previous issue were resolved - then having core and contribs separate would be ideal.
{quote}
All dependencies in one folder doesn't seem right either.
{quote}
Yup. that was my main concern.
So - did you look at the most recent patch ( all in one lib/*.jar) minimally intrusive to the scripts - but need to be revisited in the long term though.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated HBASE-2128:
---------------------------
Attachment: HBASE-2128.patch
Patch that gets all jars in pre-Ivy stage - ( all jars in lib/*.jar ) .
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833240#action_12833240 ]
stack commented on HBASE-2128:
------------------------------
I should commit this?
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Lars Francke (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833218#action_12833218 ]
Lars Francke commented on HBASE-2128:
-------------------------------------
Kay Kay's latest patch seems to work perfect in regards to the broken ant tar build. Is it ready to be committed?
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800810#action_12800810 ]
Andrew Purtell commented on HBASE-2128:
---------------------------------------
bq. The src/contrib/stargate/lib folder is empty and can be deleted/does not need to be created at all.
I found if it was not there, then ivy would error out.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801277#action_12801277 ]
Kay Kay commented on HBASE-2128:
--------------------------------
{quote}
while we are at it, we should turn stdout/stderr output buffering, since during an ivy build the 'download' does not actually print increment progress during 1 line.
{quote}
makes sense. would it be ok to take it in a separate issue though ?
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801137#action_12801137 ]
Andrew Purtell commented on HBASE-2128:
---------------------------------------
bq. For 0.21 we'll probably have three contribs with dependencies (I'm not counting ec2): stargate, transactional and thrift.
Don't count out EC2. At some point the bash stuff will be deprecated and there will be a Python/libcloud based approach replacing it, like what Hadoop Core is doing. So the EC2 stuff is likely to pull dependencies.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Lars Francke (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801095#action_12801095 ]
Lars Francke commented on HBASE-2128:
-------------------------------------
My preferred way of doing this would be to keep only the extra dependencies for the contribs in their own folders.
So all core dependencies in lib/core all extra dependencies for stargate in lib/stargate but not the core dependencies. If I remember correctly this was the way it was done before. At least almost. If one decides to use stargate all you need to do is to add one directory to the classpath.
In all likelihood I'm missing something important here but I don't see why all jars would have to be duplicated. For 0.21 we'll probably have three contribs with dependencies (I'm not counting ec2): stargate, transactional and thrift. This would mean that all common jars would be included four times in the final tar.
All dependencies in one folder doesn't seem right either.
But I'm still very new to this project so feel free to disregard all I've said :)
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated HBASE-2128:
---------------------------
Attachment: HBASE-2128.patch
next version of the same
In addition to the previous ,
- added lib/core/*.jar to CP in the hbase script as well- that should work with the release tarball.
The CP entries are -
lib/(thrift/zookeeper)
lib/core/*.jar
HBase team - feel free to swap the order if that does not sound right.
Meanwhile - the CP jars of stargate / transactional are kept in a separate namespace altogether - not affecting the scripts.
As the contrib packages increase - it would be useful to keep core and the launching script in a separate namespace .
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Lars Francke (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800741#action_12800741 ]
Lars Francke commented on HBASE-2128:
-------------------------------------
bq. Meanwhile - the CP jars of stargate / transactional are kept in a separate namespace altogether - not affecting the scripts.
I don't know if you are doing this intentionally but now the core jars are duplicated in the lib/* folders. This not only doubles the size of the final tar (from 35 MB to 70 MB) but it seems to cause problems with SLF4J (which will have to be introduced as a new dependency for Thrift 0.2). The latter may be my fault though, I'm struggling to get everything running as it was before the switch to Ivy. I'll comment again if it turns out the problem was caused by me.
Either way the duplication should be unnecessary. At least I can't imagine why it would be required.
The src/contrib/stargate/lib folder is empty and can be deleted/does not need to be created at all.
Thanks for your quick fix!
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833257#action_12833257 ]
ryan rawson commented on HBASE-2128:
------------------------------------
stack: do it, make it happen.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kay Kay updated HBASE-2128:
---------------------------
Attachment: HBASE-2128.patch
*) tar now maintains 3 different sets - lib/core/*.jar , lib/transactional/*.jar , lib/stargate/*.jar
Thrift + zookeeper , that are common to all of them is in lib only.
*) Bug in init target due to space in libthrift* , not being copied
*) unnecessary classpath element in javadoc target removed since no such file exists ( contrib/**/*.jar) in the tree anymore.
Having said that - the scripts need to be modified (or we need to identify a file-layout structure) since the scripts are meant for build/ivy/common - build/contrib/stargate/ivy/lib/common etc. , for a development environment.
When we release - may be we need to revisit what structure we would want and refactor scripts accordingly.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800957#action_12800957 ]
Kay Kay commented on HBASE-2128:
--------------------------------
So - what is the consensus we have regarding the directory structure of release , as far as lib directory is concerned ?
If we want to have all the jars in lib/*.jar as before - I am ok , but keep in mind as we add more contribs we may want to separate the CP namespaces for them to make thing easy for debugging.
| The src/contrib/stargate/lib folder is empty and can be deleted/does not need to be created at all.
| Maybe we need to add a mkdir somewhere? mkdir after copy of stargate to build for compile?
I am ok with one way or another , but if it is distracting at this point - ok with maintaining current status quo while removing any CP reference in build.xml files to contrib/**/lib/*.jar , since no such file exists / should be there in future.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801138#action_12801138 ]
ryan rawson commented on HBASE-2128:
------------------------------------
i think it would be better if ec2 wasnt part of the core build stuff. seems kind of weird...
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801142#action_12801142 ]
ryan rawson commented on HBASE-2128:
------------------------------------
while we are at it, we should turn stdout/stderr output buffering, since during an ivy build the 'download' does not actually print increment progress during 1 line.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800519#action_12800519 ]
ryan rawson commented on HBASE-2128:
------------------------------------
Tar is supposed to be self contained because it is the basis of our packaging and release. We are not going to force our users to download all the deps on their prod machines.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800818#action_12800818 ]
stack commented on HBASE-2128:
------------------------------
Maybe we need to add a mkdir somewhere? mkdir after copy of stargate to build for compile?
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800777#action_12800777 ]
Kay Kay commented on HBASE-2128:
--------------------------------
{quote}
I don't know if you are doing this intentionally but now the core jars are duplicated in the lib/* folders.
{quote}
As I mentioned earlier - it was intentional to separate the CP namespace for core / transactional / .. <other contribs> .
If that is too much of a complexity then - modify the tar patch such that all the target files - dependency jars , from ivy, get copied to lib/*.jar directory.
{quote}
The src/contrib/stargate/lib folder is empty and can be deleted/does not need to be created at all.
{quote}
true. thanks.
{quote}
I'm struggling to get everything running as it was before the switch to Ivy. I'll comment again if it turns out the problem was caused by me.
{quote}
Feel free to log tickets / add more details to the broken parts so that we can fix the same.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800512#action_12800512 ]
Kay Kay commented on HBASE-2128:
--------------------------------
Will try to look further.
But the tarball for a release is expected to be smaller after HBASE-1433, because the lib\*.jars are not supposed to be released , but will be retrieved on demand.
zk and thrift are retained only because their artifacts have not yet been published.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "Kay Kay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833347#action_12833347 ]
Kay Kay commented on HBASE-2128:
--------------------------------
Thanks for taking this patch. When I submitted this patch - there were only 2 contribs in this - that I had hardcoded ( stargate / transactional ). But with the addition of mdc_replication now, an entry might need to be added for that as well. I can track it separately.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Assignee: Kay Kay
> Fix For: 0.21.0
>
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2128) ant tar build broken since switch to
Ivy
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801105#action_12801105 ]
ryan rawson commented on HBASE-2128:
------------------------------------
the contrib method in general needs to be reworked, i think its way out of scope here, so if we can do more or less what we were doing before, then i say it is fixed. I'm not sure how important it is to tease out the Classpaths, my initial thought it 'doesnt matter'.
> ant tar build broken since switch to Ivy
> ----------------------------------------
>
> Key: HBASE-2128
> URL: https://issues.apache.org/jira/browse/HBASE-2128
> Project: Hadoop HBase
> Issue Type: Bug
> Components: build
> Affects Versions: 0.21.0
> Reporter: Lars Francke
> Attachments: HBASE-2128.patch, HBASE-2128.patch, HBASE-2128.patch
>
>
> Running ant tar produces a very small tar file because all .jar dependencies are missing. This happens since the switch to Ivy.
> Adding common.ivy.lib.dir to the build.xml fixes some of it but some things still don't work:
> {code:xml}
> <mkdir dir="${dist.dir}/lib"/>
> <copy todir="${dist.dir}/lib">
> <fileset dir="${build.lib}" />
> <fileset dir="${common.ivy.lib.dir}"/>
> </copy>
> {code}
> The jars for the contrib apps still seem to be missing. At the moment this is only stargate but the I've got the same problem for the new thrift contrib. I am afraid I don't know enough about Ant or Ivy to be of any further assistance.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.