You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "nodece (via GitHub)" <gi...@apache.org> on 2024/03/21 08:33:46 UTC
[I] [Native-image] Build a native-image for pulsar-admin [pulsar]
nodece opened a new issue, #22318:
URL: https://github.com/apache/pulsar/issues/22318
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Motivation
Using https://www.graalvm.org/ to build a native-image for pulsar-admin reduces the running time.
Comparison of running time:
```
❯ ./native-pulsar-admin.sh
+ echo native-pulsar-admin
native-pulsar-admin
+ ./target/./native-pulsar-admin ../conf/client.conf tenants list
public
pulsar
real 0m0.264s
user 0m0.208s
sys 0m0.049s
+ echo ----------------
----------------
+ echo 'default pulsar-admin'
default pulsar-admin
+ ../bin/pulsar-admin tenants list
2024-03-21T08:27:30.697167Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release
2024-03-21T08:27:30.703654Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release
2024-03-21T08:27:30.729035Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release
2024-03-21T08:27:30.732848Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release
public
pulsar
real 0m2.429s
user 0m4.667s
sys 0m0.318s
```
This `native-pulsar-admin` reduces real by 9 times compared to `pulsar-admin.sh`.
### Solution
_No response_
### Alternatives
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Native-image] Build a native-image for pulsar-admin [pulsar]
Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on issue #22318:
URL: https://github.com/apache/pulsar/issues/22318#issuecomment-2016498358
Let me try CDS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Native-image] Build a native-image for pulsar-admin [pulsar]
Posted by "nicoloboschi (via GitHub)" <gi...@apache.org>.
nicoloboschi commented on issue #22318:
URL: https://github.com/apache/pulsar/issues/22318#issuecomment-2014701503
I've tried in the past to build a native binary. The main issue is that all the [reflection calls must be intercepted and declared in the configuration file](https://www.graalvm.org/latest/reference-manual/native-image/dynamic-features/Reflection/). There's a [tracing mechanism in graalvm](https://www.graalvm.org/latest/reference-manual/native-image/metadata/AutomaticMetadataCollection/#tracing-agent) to auto-generate those configurations, however not all the code paths are triggered and therefore some runtime errors could be generated at runtime (blocking users in making some operations).
That's why I think it's not worth it and instead the Pulsar community should start using Pulsar shell as single CLI tool (and that's why it was born)
My attempts were made more than 1 year ago and things could have changed meanwhile; however I've read the same complaints in other ASF projects
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Native-image] Build a native-image for pulsar-admin [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #22318:
URL: https://github.com/apache/pulsar/issues/22318#issuecomment-2016443625
Have you tested CDS? That would be easy to add.
https://docs.oracle.com/en/java/javase/21/vm/class-data-sharing.html
For example, Spring is leveraging this. Blog post https://spring.io/blog/2023/12/04/cds-with-spring-framework-6-1 .
> As mentioned by Brian Goetz in [his Project Leyden talk at Devoxx](https://www.youtube.com/watch?v=O1Oz2-AXKKM): “Most people don’t use CDS today but probably should since they can get a reasonable startup improvement for relatively little work”.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Native-image] Build a native-image for pulsar-admin [pulsar]
Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on issue #22318:
URL: https://github.com/apache/pulsar/issues/22318#issuecomment-2014810552
> How long does it take to build the native image?
Environment:
```
MacBookPro16,1
6H Intel Core i7
2.6 GHz
```
Only build for pulsar-admin: [INFO] Total time: 05:04 min
Pulsar admin is a commonly used tool, so I plan to support that. Or we can export the native-image config to the jar, the developers can use the Graalvm to build the native-image tool.
https://github.com/remkop/picocli/tree/main/picocli-codegen can help us to generate the CLI config of native-image, it is simpler, I have already done it, and I can open a PR to preview.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Native-image] Build a native-image for pulsar-admin [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #22318:
URL: https://github.com/apache/pulsar/issues/22318#issuecomment-2014679270
Yes, this could be useful. Also for pulsar-shell. The only concern is about the increased build time and I wouldn't like to increase CI build times for building the native images. How long does it take to build the native image?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Native-image] Build a native-image for pulsar-admin [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #22318:
URL: https://github.com/apache/pulsar/issues/22318#issuecomment-2014888216
> Only build for pulsar-admin: [INFO] Total time: 05:04 min
>
> Pulsar admin is a commonly used tool, so I plan to support that. Or we can export the native-image config to the jar, the developers can use the Graalvm to build the native-image tool.
It would be worth checking with GitHub Actions CI.
One possibility is to setup the CI in a way that this isn't run for PR builds at all and let the scheduled build handle building the native image & running tests for it for catching possible regressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org