You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by minahlee <gi...@git.apache.org> on 2016/06/10 21:31:19 UTC
[GitHub] zeppelin pull request #994: [WIP][ZEPPELIN-986] Create publish release scrip...
GitHub user minahlee opened a pull request:
https://github.com/apache/zeppelin/pull/994
[WIP][ZEPPELIN-986] Create publish release script
### What is this PR for?
This PR is to automate release publish to maven repository.
We used to use maven-deploy-plugin and maven-release-plugin for release but somehow it didn't work well with Zeppelin so 0.5.5 and 0.5.6 haven't been published to maven repository.
Publishing release to maven repository will eventaully help zeppelin to reduce binary package size by leading users to use Dynamic interpreter loading(#908).
### What type of PR is it?
Infra
### What is the Jira issue?
[ZEPPELIN-986](https://issues.apache.org/jira/browse/ZEPPELIN-986)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes, https://cwiki.apache.org/confluence/display/ZEPPELIN/Preparing+Zeppelin+Release will be updated accordingly once this pr is merged.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/minahlee/zeppelin ZEPPELIN-986
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/994.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #994
----
commit e03e03ef0fd7e073c941871468627d4371d13888
Author: Mina Lee <mi...@apache.org>
Date: 2016-06-10T20:35:42Z
Add maven publish release script
commit 488d98cc48f2ca335c4a11a1f413b76a59d42de8
Author: Mina Lee <mi...@apache.org>
Date: 2016-06-10T20:35:51Z
Refactor create release script
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [WIP][ZEPPELIN-986] Create publish release script
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/994
CI failiure is unrelated. Looks good to me
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/994
Got it, thank for reminding! Let's merge this guy then
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/994
@bzz Thanks for the review. I removed mistakenly pushed file.
I included geode, scalding interpreter in maven publish script so that geode, scalding users can take advantage of #1042.
To be honest, I don't know why these two interpreters were not included in last release. Can someone explain?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/994
Merge if there is no more discussion
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/994
Great, I think if checking error conditions could be added to network-related commands - it would be even more awesome, as this is quite common failure pattern.
Other then that, looks great to me and thank you for automating this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/994
@minahlee Great!
I think it should be ready to merge as soon as issues highlighted above
a. failing networking `curl`
b. build configuration duplication
are addressed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/994
Looks great, I think there are just few things left:
- rebased on latest master
- check that code conforms the [conventions from our style guide](https://github.com/apache/zeppelin/blob/master/CONTRIBUTING.md#code-convention)
@minahlee if we want to remove `maven release plugin`, could you also point to the place that does automate the version switch in POM files (as AFAIK that is the only purpose we used that plugin so far)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/994
@echarles
- This script is written exactly because of the reason that @bzz explained above.
> calling it multiple times with different arguments, instead of single script was quite error prone.
- AFAIK, repo is under zeppelin-zengine module, not zeppelin-server. But if there is any other use case for zeppelin-server other than notebook repo I don't see the reason not to publish. If you come up with other use case, please let me know
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #994: [WIP][ZEPPELIN-986] Create publish release scrip...
Posted by minahlee <gi...@git.apache.org>.
GitHub user minahlee reopened a pull request:
https://github.com/apache/zeppelin/pull/994
[WIP][ZEPPELIN-986] Create publish release script
### What is this PR for?
This PR is to automate release publish to maven repository.
We used to use maven-deploy-plugin and maven-release-plugin for release but somehow it didn't work well with Zeppelin so 0.5.5 and 0.5.6 haven't been published to maven repository.
Publishing release to maven repository will eventually help zeppelin to reduce binary package size by leading users to use Dynamic interpreter loading(#908).
Originally below modules were skipped for maven release
- all interpreters(except spark)
- zeppelin-display
- zeppelin-server
- zeppelin-distribution
on the other hand this pr will skip only:
- zeppelin-server
- zeppelin-distribution
### What type of PR is it?
Infra
### What is the Jira issue?
[ZEPPELIN-986](https://issues.apache.org/jira/browse/ZEPPELIN-986)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes, https://cwiki.apache.org/confluence/display/ZEPPELIN/Preparing+Zeppelin+Release will be updated accordingly once this pr is merged.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/minahlee/zeppelin ZEPPELIN-986
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/994.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #994
----
commit e03e03ef0fd7e073c941871468627d4371d13888
Author: Mina Lee <mi...@apache.org>
Date: 2016-06-10T20:35:42Z
Add maven publish release script
commit 488d98cc48f2ca335c4a11a1f413b76a59d42de8
Author: Mina Lee <mi...@apache.org>
Date: 2016-06-10T20:35:51Z
Refactor create release script
commit eba94a72e83ebed9ad7809a25ce48cffc14f3fa0
Author: Mina Lee <mi...@apache.org>
Date: 2016-06-10T21:45:16Z
remove deploy plugin from pom since custom script will be used instead for deploy
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by echarles <gi...@git.apache.org>.
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/994
why spark is not published. is it related to the assembly?
Btw, spark 2 doesn't use assembly any more and favor dependencies in separate jars. See https://issues.apache.org/jira/browse/SPARK-11157 for more details.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/994
Looks awesome to me, thank you @minahlee !
2 minor things are a bit confusing:
- the purpose of `dev/publish_release.sh.hehe` file
- some artifacts (namely, integration for Geode and Scalding) we publish in maven, but AFAIK they are not part of the convenience binary for the release
```
publish: -Pspark-1.6 -Phadoop-2.4 -Pyarn -Ppyspark -Psparkr -Pr -Pgeode -Pscalding
release: -Pspark-1.6 -Phadoop-2.4 -Pyarn -Ppyspark -Psparkr -Pr
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by echarles <gi...@git.apache.org>.
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/994
@minahlee thx for this precision and context.
True, I just realized after sending the comment that repo is not part of zeppelin-server.
However, I need zeppelin-server in my project (I kind off embed the web and websocket server) - Publishing all artifcats is also in my understanding a common practice. WDYT. Is there any issue you face to publish the server?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/994
@echarles If there are such needs, nothing stops us from publishing it. I will make change accordingly. Thank you for sharing your situation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/994
@bzz Thank you for the review, I handled style in 21ca610. Please let me know if you have any other suggestions :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #994: [WIP][ZEPPELIN-986] Create publish release scrip...
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee closed the pull request at:
https://github.com/apache/zeppelin/pull/994
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #994: [ZEPPELIN-986] Create publish release script
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/994
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by echarles <gi...@git.apache.org>.
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/994
Didn't look at the code... just saw in the description (and @minahlee just confirmed it would be published).
Thinking more about this PR:
- Why the deploy and release plugins were not working? Using scrips instead of existing plugins is not ideal to me.
- I understand distribution should not be published, but server should be. It contains the repo components that can be used elsewhere. Apart from the fact we could have a separate module for the repo, all components (excpect distribution) should be published. That's a rule of thumb and if we don't follow it, someone is gonna come sooner or later and ask for the missing bits.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/994
@minahlee Here's related issues why geode and scalding is not in maven publish script.
Geode - https://issues.apache.org/jira/browse/ZEPPELIN-375
Scalding - https://issues.apache.org/jira/browse/ZEPPELIN-972
Please take a look. To include them, bin_license/LICENSE file need to be updated with their transitive dependencies(geode, scalding interpreter), and remove 3rd party repository (scalding interpreter)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/994
@echarles I think my description caused confusion. Spark will be included in release since most of Zeppelin user is based on Spark. The modules I wrote in description is skipped modules for publish and Spark always hasn't been skipped from release.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/994
@bzz Sorry for late response, I am working on it and going to push commits by today. Thanks for heads up :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by echarles <gi...@git.apache.org>.
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/994
Good to hear the release plugin is working fine.
The release manager is responsible for the tools he wants. Fine to me if it is a script or a plugin, but at the end the artifacts must find their way to maven central.
For this PR, is it possible to ensure `zeppelin-server` will be also published?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/994
@Leemoonsoo Thanks for pointing it out! I reverted commit of adding geode, scalding
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #994: [ZEPPELIN-986] Create publish release script
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/994
Thank you for the feedback @echarles !
I think it should be fine to add missing bits, as soon as they are identified and there is somebody with actual use case for it later on. Would you be willing to log it as a JIRA issue?
As person who did release management for Zeppelin before - I had no troubles with release plugin, although calling it multiple times with different arguments, instead of single script was quite error prone.
If a manager for a current release though wants to try another solution and provides a robust one (as in this PR) - it should be fine, especially given that we can always get back to the plugins if desired.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---