[GitHub] [pulsar] nicoloboschi opened a new issue, #16250: PIP-181: Pulsar Shell

[GitBox <gi...@apache.org>]

nicoloboschi opened a new issue, #16250:
URL: https://github.com/apache/pulsar/issues/16250

   ## Motivation
   
   Currently Pulsar comes with a couple of utility scripts with the goal of managing an existing cluster, test behaviours and verify performances: these tools are available as SH script inside the `bin` directory.
   The `pulsar-admin` is the CLI tool supposed to help users and operators to configure the system, operate over policies, install functions and much else.
   
   This proposal basically aims to solve two different problems:
   
   1. `pulsar-admin` is terribly slow. Every time the script is triggered, a new JVM process is spawned. The JVM process creation is heavy and most of the time is spent by the JVM initializion process. A very common use case for cluster operators is to create scripts with several commands with the goal of initialize the cluster, initialize a specific tenant (namespaces, topics, policies, functions..); in this case, one JVM is initialized for each scripts leads to waste of time and resources.
   
   2. User experience. The current design of the Pulsar CLIs can be improved. There are a couple of aspects that may be annoying for a user and can discourage a user to use Pulsar. 
       1. Poking around available commands and options in a CLI tool (`pulsar-admin` for instance, but it's the same for `pulsar-perf` and `pulsar-client`) is slow and hard. In order to discover commands and options you need to use `-h` option and, since the performance issue pointed at 1., it can be annoying and time-consuming. Autocomplete feature could be a real game-changer in this context.
       2. Different CLI tools. There are a couple of different shell scripts. They have different goals and it's okay to keep them separated. However, they raise a barrier for a non Pulsar expert that doesn't have a convenient entry-point. 
   
   ## Goal
   
   Address all the issues in the previous section with a single solution.
   
   ## API Changes
   
   A new shell script `bin/pulsar-shell` will be introduced. 
   
   ## Implementation
   
   ### Concepts
   The new script `pulsar-shell` will differ from the existing for the following reasons:
   
   1. It's a shell. When you start it, it will wait for commands to be executed. After the command has been executed, despite on its result, the shell session will not be destroyed and it will wait for another command.
   2. Unifies all the CLI scripts. In `pulsar-shell` you'll be able to run all the existing CLI commands. This will be done in a way that when a new command/option is added, the pulsar-shell will be updated accordingly.
   3. It comes with sofisticated autocompletion and command history to highly improve the UX.
   4. Performance. Since JVM is initiated once, it will gain on performance thanks to the JVM warmup and internal libraries bootstraps. 
   5. It will accept a file or a list of commands (parameter and stdin) to start a shell, run the commands and close the shell. We'll call it the `non-interactive` mode and it will ease the cluster operations automations.
   
   Note that existing tools will not be removed/changed.
   
   ### Implementation
   
   The shell implementation will be developed in Java, using a well-knonw library called [JLine 3](https://github.com/jline/jline3) for the shell support.
   There will be a new main class that will extends the existing class tools (e.g. `PulsarAdminTool`)
   
   #### Configuration
   The configuration file took by default will be `client.conf`, like current CLI tools. The env setup will be exacly the same as for `pulsar-admin` and `pulsar-client`.
   
   #### Autocompletion
   JLine3 has great support for autocompletion. The shell java class will translate current `JCommander` tools to the JLine3 completion API. This will ensure that all APIs will be up-to-date and covered.
   
   #### History
   JLine3 has built-in support for history, both in-memory and persisted (on local FS). By default the history will be persisted in the user home directory. This feature can be turned off for security concerns with a configuration property.
   
   
   ## Rejected Alternatives
   
   - Create a shell mode for `pulsar-admin`. This won't be a flexible solution because then we may need the same for other tools. Also it doesn't cover the CLI unification part.
   - Create a brand-new shell mode. Not compatible with current CLI and another tool to mantain.
   - Use another shell library. There are a couple of valid alternatives to JLine3 but it's wide-spread and the autocompletion API is much more flexible than others since it's not done with annotations (the shell needs to translate current JCommander definitions from existing tools)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] codelipenghui commented on issue #16250: PIP-181: Pulsar Shell

[GitBox <gi...@apache.org>]

codelipenghui commented on issue #16250:
URL: https://github.com/apache/pulsar/issues/16250#issuecomment-1211637626

   Close since all the PRs get merged.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] codelipenghui closed issue #16250: PIP-181: Pulsar Shell

[GitBox <gi...@apache.org>]

codelipenghui closed issue #16250: PIP-181: Pulsar Shell
URL: https://github.com/apache/pulsar/issues/16250


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] MonicaMagoniCom commented on issue #16250: PIP-181: Pulsar Shell

[GitBox <gi...@apache.org>]

MonicaMagoniCom commented on issue #16250:
URL: https://github.com/apache/pulsar/issues/16250#issuecomment-1199366649

   In our enviroment based on Kubernetes we deploy pulsar with Helm; authentication and authorization are enabled and this means that when the broker is created it will run some pulsar admin commands in order to:
   - create the tenant
   - create the token
   - create the namespace
   - grant permission
   In version 2.9.3, it seems that these commands are really slow (the broker takes up to 6 minutes to be ready); until 2.7.2 they were quite fast (in the order of 2 minutes max). any suggestion?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org