You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/03/12 08:09:32 UTC
[GitHub] [incubator-pinot] fx19880617 opened a new pull request #5146:
update pinot assembly scripts
fx19880617 opened a new pull request #5146: update pinot assembly scripts
URL: https://github.com/apache/incubator-pinot/pull/5146
This is for release 0.3.0
`mvn release` command doesn't honor output relocation, so we need to manually specify those in assembly scripts for `pinot-plugins`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request
#5146: Update pinot assembly scripts for release 0.3.0
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on a change in pull request #5146: Update pinot assembly scripts for release 0.3.0
URL: https://github.com/apache/incubator-pinot/pull/5146#discussion_r391854918
##########
File path: pinot-distribution/pinot-assembly.xml
##########
@@ -43,10 +43,74 @@
<file>
<source>${pinot.root}/DISCLAIMER</source>
</file>
+ <!-- Include Pinot All-in-one jar -->
<file>
<source>${pinot.root}/pinot-distribution/target/pinot-distribution-${project.version}-shaded.jar</source>
<destName>lib/pinot-all-${project.version}-jar-with-dependencies.jar</destName>
</file>
+ <!-- Start Include Pinot Plugins-->
+ <!-- Start Include Pinot Stream Ingestion Plugins-->
+ <!-- Only Include Specified Kafka Version Plugin-->
+ <file>
+ <source>${pinot.root}/pinot-plugins/pinot-stream-ingestion/pinot-kafka-${kafka.version}/target/pinot-kafka-${kafka.version}-${project.version}-shaded.jar</source>
Review comment:
That is how we did it previously by gathering all the jars together then copy directory, see line `<outputDirectory>${pinot.root}/pinot-plugins/target/plugins/${plugin.type}/${project.artifactId}</outputDirectory>` in `pinot-plugins/pom.xml`.
However `mvn release` doesn't honor this, and cannot find the built jars if we relocated them, that's why we need to hand-pick all those jars again here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] snleee commented on a change in pull request
#5146: Update pinot assembly scripts for release 0.3.0
Posted by GitBox <gi...@apache.org>.
snleee commented on a change in pull request #5146: Update pinot assembly scripts for release 0.3.0
URL: https://github.com/apache/incubator-pinot/pull/5146#discussion_r391849492
##########
File path: pinot-distribution/pinot-assembly.xml
##########
@@ -43,10 +43,74 @@
<file>
<source>${pinot.root}/DISCLAIMER</source>
</file>
+ <!-- Include Pinot All-in-one jar -->
<file>
<source>${pinot.root}/pinot-distribution/target/pinot-distribution-${project.version}-shaded.jar</source>
<destName>lib/pinot-all-${project.version}-jar-with-dependencies.jar</destName>
</file>
+ <!-- Start Include Pinot Plugins-->
+ <!-- Start Include Pinot Stream Ingestion Plugins-->
+ <!-- Only Include Specified Kafka Version Plugin-->
+ <file>
+ <source>${pinot.root}/pinot-plugins/pinot-stream-ingestion/pinot-kafka-${kafka.version}/target/pinot-kafka-${kafka.version}-${project.version}-shaded.jar</source>
Review comment:
After I run `mvn install`, I see `pinot-plugins/target/plugins` has all shaded plugins copied. Is it possible to copy this directory at once instead of listing all files?
```
~/workspace/pinot/pinot-plugins/target/plugins
❯ pwd
/Users/snlee/workspace/pinot/pinot-plugins/target/plugins
~/workspace/pinot/pinot-plugins/target/plugins
❯ ll
total 0
drwxr-xr-x 11 snlee LINKEDIN\eng 352B Mar 10 21:33 pinot-batch-ingestion
drwxr-xr-x 6 snlee LINKEDIN\eng 192B Mar 10 21:30 pinot-file-system
drwxr-xr-x 9 snlee LINKEDIN\eng 288B Mar 10 21:30 pinot-input-format
drwxr-xr-x 2 snlee LINKEDIN\eng 64B Mar 10 21:29 pinot-plugins
drwxr-xr-x 6 snlee LINKEDIN\eng 192B Mar 10 21:33 pinot-stream-ingestion
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #5146: Update pinot
assembly scripts for release 0.3.0
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #5146: Update pinot assembly scripts for release 0.3.0
URL: https://github.com/apache/incubator-pinot/pull/5146#issuecomment-598998494
> Whenever we add the new plugin and make it as part of the distribution, we will need to add the line here. We can periodically double check on this as part of the release process (similar that we clean up license for every release).
>
> Can we update the release doc to include this? Otherwise, LGTM! Thank you for working on this.
Yes, also for required for plugin developer, this is how they could package plugin into the distribution. I will add this into not only release process doc, but also dev guide.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] snleee commented on issue #5146: Update pinot
assembly scripts for release 0.3.0
Posted by GitBox <gi...@apache.org>.
snleee commented on issue #5146: Update pinot assembly scripts for release 0.3.0
URL: https://github.com/apache/incubator-pinot/pull/5146#issuecomment-598981186
Whenever we add the new plugin and make it as part of the distribution, we will need to add the line here. We can periodically double check on this as part of the release process (similar that we clean up license for every release).
Can we update the release doc to include this? Otherwise, LGTM! Thank you for working on this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 merged pull request #5146: Update
pinot assembly scripts for release 0.3.0
Posted by GitBox <gi...@apache.org>.
fx19880617 merged pull request #5146: Update pinot assembly scripts for release 0.3.0
URL: https://github.com/apache/incubator-pinot/pull/5146
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] snleee commented on a change in pull request
#5146: Update pinot assembly scripts for release 0.3.0
Posted by GitBox <gi...@apache.org>.
snleee commented on a change in pull request #5146: Update pinot assembly scripts for release 0.3.0
URL: https://github.com/apache/incubator-pinot/pull/5146#discussion_r391849492
##########
File path: pinot-distribution/pinot-assembly.xml
##########
@@ -43,10 +43,74 @@
<file>
<source>${pinot.root}/DISCLAIMER</source>
</file>
+ <!-- Include Pinot All-in-one jar -->
<file>
<source>${pinot.root}/pinot-distribution/target/pinot-distribution-${project.version}-shaded.jar</source>
<destName>lib/pinot-all-${project.version}-jar-with-dependencies.jar</destName>
</file>
+ <!-- Start Include Pinot Plugins-->
+ <!-- Start Include Pinot Stream Ingestion Plugins-->
+ <!-- Only Include Specified Kafka Version Plugin-->
+ <file>
+ <source>${pinot.root}/pinot-plugins/pinot-stream-ingestion/pinot-kafka-${kafka.version}/target/pinot-kafka-${kafka.version}-${project.version}-shaded.jar</source>
Review comment:
After I run `mvn install`, I see `pinot-plugins/target/plugins` has all shaded plugins copied. Is it possible to copy this directory at once instead of listing all files?
```
~/workspace/pinot/pinot-plugins/target/plugins adls-gen2*
❯ pwd
/Users/snlee/workspace/pinot/pinot-plugins/target/plugins
~/workspace/pinot/pinot-plugins/target/plugins adls-gen2*
❯ ll
total 0
drwxr-xr-x 11 snlee LINKEDIN\eng 352B Mar 10 21:33 pinot-batch-ingestion
drwxr-xr-x 6 snlee LINKEDIN\eng 192B Mar 10 21:30 pinot-file-system
drwxr-xr-x 9 snlee LINKEDIN\eng 288B Mar 10 21:30 pinot-input-format
drwxr-xr-x 2 snlee LINKEDIN\eng 64B Mar 10 21:29 pinot-plugins
drwxr-xr-x 6 snlee LINKEDIN\eng 192B Mar 10 21:33 pinot-stream-ingestion
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org