You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by "Josh Wills (JIRA)" <ji...@apache.org> on 2012/09/24 22:44:07 UTC
[jira] [Created] (CRUNCH-76) Create binary release artifacts
Josh Wills created CRUNCH-76:
--------------------------------
Summary: Create binary release artifacts
Key: CRUNCH-76
URL: https://issues.apache.org/jira/browse/CRUNCH-76
Project: Crunch
Issue Type: Task
Reporter: Josh Wills
Assignee: Matthias Friedrich
Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-76) Create binary release artifacts
Posted by "Josh Wills (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466688#comment-13466688 ]
Josh Wills commented on CRUNCH-76:
----------------------------------
(Therefore, +1.)
> Create binary release artifacts
> -------------------------------
>
> Key: CRUNCH-76
> URL: https://issues.apache.org/jira/browse/CRUNCH-76
> Project: Crunch
> Issue Type: Task
> Reporter: Josh Wills
> Assignee: Matthias Friedrich
> Labels: artifact
> Attachments: CRUNCH-76-Create-binary-release.patch, CRUNCH-76-Create-binary-release-v2.patch
>
>
> Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-76) Create binary release artifacts
Posted by "Matthias Friedrich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464122#comment-13464122 ]
Matthias Friedrich commented on CRUNCH-76:
------------------------------------------
If people need a binaries, I think we should provide them. Questions:
* Create a separate artifact for binaries?
* Ship the transitive closure of our dependencies in a lib dir?
* Include Javadoc, too?
* Any legal issues or Apache customs to consider?
I think we should find an Apache project to emulate. Any ideas? Whirr perhaps?
> Create binary release artifacts
> -------------------------------
>
> Key: CRUNCH-76
> URL: https://issues.apache.org/jira/browse/CRUNCH-76
> Project: Crunch
> Issue Type: Task
> Reporter: Josh Wills
> Assignee: Matthias Friedrich
> Labels: artifact
>
> Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-76) Create binary release artifacts
Posted by "Josh Wills (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466579#comment-13466579 ]
Josh Wills commented on CRUNCH-76:
----------------------------------
Cool, ran it successfully. Quibble: do we need to ship mockito/hamcrest in the libs? I thought we only needed those for crunch unit tests, I wasn't aware that any of the bits depend on them.
> Create binary release artifacts
> -------------------------------
>
> Key: CRUNCH-76
> URL: https://issues.apache.org/jira/browse/CRUNCH-76
> Project: Crunch
> Issue Type: Task
> Reporter: Josh Wills
> Assignee: Matthias Friedrich
> Labels: artifact
> Attachments: CRUNCH-76-Create-binary-release.patch, CRUNCH-76-Create-binary-release-v2.patch
>
>
> Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-76) Create binary release artifacts
Posted by "Matthias Friedrich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466432#comment-13466432 ]
Matthias Friedrich commented on CRUNCH-76:
------------------------------------------
I had a look at this, it seems we'll need a new maven module just for creating the binary artifact. When the parent module is built, the children's artifacts aren't available yet, so we need a new module that depends on every module we want to include in the binary distribution. See http://maven.apache.org/plugins/maven-assembly-plugin/faq.html#module-binaries
> Create binary release artifacts
> -------------------------------
>
> Key: CRUNCH-76
> URL: https://issues.apache.org/jira/browse/CRUNCH-76
> Project: Crunch
> Issue Type: Task
> Reporter: Josh Wills
> Assignee: Matthias Friedrich
> Labels: artifact
>
> Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-76) Create binary release artifacts
Posted by "Josh Wills (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466687#comment-13466687 ]
Josh Wills commented on CRUNCH-76:
----------------------------------
Yeah, we should duplicate the dependencies. I'll create another JIRA for it, but I don't think it should block this one.
> Create binary release artifacts
> -------------------------------
>
> Key: CRUNCH-76
> URL: https://issues.apache.org/jira/browse/CRUNCH-76
> Project: Crunch
> Issue Type: Task
> Reporter: Josh Wills
> Assignee: Matthias Friedrich
> Labels: artifact
> Attachments: CRUNCH-76-Create-binary-release.patch, CRUNCH-76-Create-binary-release-v2.patch
>
>
> Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CRUNCH-76) Create binary release artifacts
Posted by "Matthias Friedrich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-76?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Friedrich resolved CRUNCH-76.
--------------------------------------
Resolution: Fixed
Fix Version/s: 0.4.0
Committed. I haven't heard from Bigtop but I hope this will meet their requirements. If not we can re-open this issue.
> Create binary release artifacts
> -------------------------------
>
> Key: CRUNCH-76
> URL: https://issues.apache.org/jira/browse/CRUNCH-76
> Project: Crunch
> Issue Type: Task
> Reporter: Josh Wills
> Assignee: Matthias Friedrich
> Labels: artifact
> Fix For: 0.4.0
>
> Attachments: CRUNCH-76-Create-binary-release.patch, CRUNCH-76-Create-binary-release-v2.patch
>
>
> Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-76) Create binary release artifacts
Posted by "Matthias Friedrich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466573#comment-13466573 ]
Matthias Friedrich commented on CRUNCH-76:
------------------------------------------
You can use the full release process, but "mvn package -Papache-release" should be enough for inspecting the release artifacts (see crunch-dist/target).
> Create binary release artifacts
> -------------------------------
>
> Key: CRUNCH-76
> URL: https://issues.apache.org/jira/browse/CRUNCH-76
> Project: Crunch
> Issue Type: Task
> Reporter: Josh Wills
> Assignee: Matthias Friedrich
> Labels: artifact
> Attachments: CRUNCH-76-Create-binary-release.patch, CRUNCH-76-Create-binary-release-v2.patch
>
>
> Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-76) Create binary release artifacts
Posted by "Josh Wills (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466570#comment-13466570 ]
Josh Wills commented on CRUNCH-76:
----------------------------------
Just created a branch to test this-- how do I run it? Like a regular release?
> Create binary release artifacts
> -------------------------------
>
> Key: CRUNCH-76
> URL: https://issues.apache.org/jira/browse/CRUNCH-76
> Project: Crunch
> Issue Type: Task
> Reporter: Josh Wills
> Assignee: Matthias Friedrich
> Labels: artifact
> Attachments: CRUNCH-76-Create-binary-release.patch, CRUNCH-76-Create-binary-release-v2.patch
>
>
> Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CRUNCH-76) Create binary release artifacts
Posted by "Matthias Friedrich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-76?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Friedrich updated CRUNCH-76:
-------------------------------------
Attachment: CRUNCH-76-Create-binary-release.patch
First try at creating a binary release, containing license files, our artifacts, and dependencies, based on Flume's setup. I had to rebuild the source distribution in the process because it doesn't work from a sub-module (the resulting archives are almost identical).
Is there anything else to include? We still have the old scrunch artifacts that we dropped from the last release. Where do we want to go with that?
> Create binary release artifacts
> -------------------------------
>
> Key: CRUNCH-76
> URL: https://issues.apache.org/jira/browse/CRUNCH-76
> Project: Crunch
> Issue Type: Task
> Reporter: Josh Wills
> Assignee: Matthias Friedrich
> Labels: artifact
> Attachments: CRUNCH-76-Create-binary-release.patch
>
>
> Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CRUNCH-76) Create binary release artifacts
Posted by "Matthias Friedrich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-76?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Friedrich updated CRUNCH-76:
-------------------------------------
Attachment: CRUNCH-76-Create-binary-release-v2.patch
Updated patch due to changes in CRUNCH-83.
> Create binary release artifacts
> -------------------------------
>
> Key: CRUNCH-76
> URL: https://issues.apache.org/jira/browse/CRUNCH-76
> Project: Crunch
> Issue Type: Task
> Reporter: Josh Wills
> Assignee: Matthias Friedrich
> Labels: artifact
> Attachments: CRUNCH-76-Create-binary-release.patch, CRUNCH-76-Create-binary-release-v2.patch
>
>
> Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CRUNCH-76) Create binary release artifacts
Posted by "Matthias Friedrich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CRUNCH-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466652#comment-13466652 ]
Matthias Friedrich commented on CRUNCH-76:
------------------------------------------
You're right, we don't really need to ship them. The problem is that one job of crunch-test is to act as a collection of dependencies for other modules, that's why Mockito and Hamcrest are on the "compile" scope. We could duplicate dependencies of JUnit, Hamcrest, and Mockito in all modules that need them (on the "test" scope"),then they would disappear from the distribution. I think "mvn dependencies:analyze" wants us to do that, BTW.
Just adding an exclusion in the assembly descriptor would be looking for trouble. If we'd ever use them in crunch-test, we'd probably forget to remove the exclusion.
> Create binary release artifacts
> -------------------------------
>
> Key: CRUNCH-76
> URL: https://issues.apache.org/jira/browse/CRUNCH-76
> Project: Crunch
> Issue Type: Task
> Reporter: Josh Wills
> Assignee: Matthias Friedrich
> Labels: artifact
> Attachments: CRUNCH-76-Create-binary-release.patch, CRUNCH-76-Create-binary-release-v2.patch
>
>
> Per BIGTOP-612, it would be helpful for Crunch to include compiled jars as a convenience as part of our release artifacts, like most of the other Hadoop projects. There are some pros and cons to doing this, so this JIRA is here to track the discussion from both the Crunch and Bigtop perspectives.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira