You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/05/01 09:11:02 UTC
Slack digest for #general - 2019-05-01
2019-04-30 16:10:13 UTC - Sébastien de Melo: Hi guys! Is it supported to have multiple functions workers separated from the brokers?
----
2019-04-30 16:13:53 UTC - Matteo Merli: Yes
----
2019-04-30 16:14:32 UTC - Matteo Merli: You just start that as a separated component with `bin/pulsar functions-worker`
----
2019-04-30 16:23:08 UTC - Sébastien de Melo: Thank you! That's what I did but the stats subcommand failed with the following error:
$ pulsar-admin --admin-url <http://lucky-star-pulsar-functions-worker:6750> functions stats --name Router
javax.ws.rs.ProcessingException: lucky-star-pulsar-functions-worker-5988889989-s7zzm: Name or service not known
Reason: HTTP 500 Internal Server Error
command terminated with exit code 1
----
2019-04-30 16:23:22 UTC - Sébastien de Melo: I guess I have to try with a StatefulSet
----
2019-04-30 16:27:44 UTC - Matteo Merli: Yes, it will redirect you to the worker that is running the particular function
----
2019-04-30 16:27:58 UTC - Matteo Merli: so that advertised name needs to be routable
----
2019-04-30 16:29:09 UTC - Poule: do Function support Python3 or just 2
----
2019-04-30 16:33:31 UTC - Poule: what means "Function in trigger function is not ready"
----
2019-04-30 16:58:01 UTC - Sanjeev Kulkarni: Functions support ython3
----
2019-04-30 16:58:16 UTC - Sanjeev Kulkarni: That means that function is not yet started
----
2019-04-30 17:20:59 UTC - Poule: 10 minutes is long to start
----
2019-04-30 17:21:03 UTC - Poule: :0)
----
2019-04-30 17:25:35 UTC - Poule: Do I need to start pulsar in a Python3 environment to insert python3 functions?
----
2019-04-30 17:26:13 UTC - Poule: Or all is taken care of whenever which python version I use
----
2019-04-30 17:27:30 UTC - Ali Ahmed: @Poule the functions just the the system python
----
2019-04-30 17:27:56 UTC - Ali Ahmed: it should point to python3 if you need to use that.
----
2019-04-30 17:28:46 UTC - Poule: it takes python in the path I guess?
----
2019-04-30 17:42:12 UTC - Sanjeev Kulkarni: That is correct
----
2019-04-30 17:42:40 UTC - Sanjeev Kulkarni: Wrt function not starting after a long time, check for any errors in the broker or function worker side
----
2019-04-30 17:42:57 UTC - Sanjeev Kulkarni: Are you running in threaded or process mode?
----
2019-04-30 17:43:42 UTC - Poule: I `tail -f` the logs but nothing interesting
----
2019-04-30 17:43:51 UTC - Poule: no idea what is threaded vs process mode
----
2019-04-30 17:44:07 UTC - Poule: i just started basic `pulsar standalone`
----
2019-04-30 17:45:14 UTC - Poule: in a `pipenv shell` env
----
2019-04-30 17:47:14 UTC - Poule: so far it works in python2
----
2019-04-30 17:49:26 UTC - Sanjeev Kulkarni: It’s python right so it has to be process mode
----
2019-04-30 17:49:46 UTC - Sanjeev Kulkarni: Do you have function logs?
----
2019-04-30 17:55:51 UTC - Poule: it does not log it fails before with ```13:54:55.548 [function-timer-thread-82-1] ERROR org.apache.pulsar.functions.runtime.ProcessRuntime - Extracted Process death exception
java.lang.RuntimeException:
at org.apache.pulsar.functions.runtime.ProcessRuntime.tryExtractingDeathException(ProcessRuntime.java:378) ~[org.apache.pulsar-pulsar-functions-runtime-2.3.1.jar:2.3.1]```
----
2019-04-30 17:57:42 UTC - Poule: my guess is wrong python version, need to investigate
----
2019-04-30 17:59:55 UTC - Devin G. Bost: We've had some concerns about how well Pulsar might perform (in terms of availability) in Kubernetes due to the fact that Pulsar is a stateful service. Does anyone have some wisdom they could please share about any issues with Pulsar in Kubernetes they've experienced and what they've done about them (and how well those solutions worked)?
----
2019-04-30 18:01:35 UTC - Poule: do you like Kubernetes?
----
2019-04-30 18:02:47 UTC - Devin G. Bost: I personally don't have enough experience with Kubernetes to give a strong opinion about it, but my colleagues (and those on my team) like it.
----
2019-04-30 18:03:54 UTC - Matteo Merli: @Devin G. Bost We have been running Pulsar in Kubernetes in production for a while with very good results.
Sure, it takes a while for one to get acquainted with Kubernetes and there sure is a learning curve (as I guess the same could be said of Pulsar…)
Though, in my view, the current status is very solid and flexible.
----
2019-04-30 18:05:54 UTC - Poule: what's the advantages of running P on K?
----
2019-04-30 18:06:38 UTC - Matteo Merli: Simplified operations and integration with modern ecosystem.
----
2019-04-30 18:07:02 UTC - Devin G. Bost: @Matteo Merli Thanks for the feedback. Are there any specific issues that you needed to address to keep everything stable (such as dealing with failure states, edge cases, etc.)?
----
2019-04-30 18:07:24 UTC - Matteo Merli: There are a loot of very good tools that are coming up with out-of-the-box K8S integrations.
----
2019-04-30 18:09:21 UTC - Matteo Merli: I’d say the biggest “hurdle” was to “learn” kubernetes :slightly_smiling_face: and understand all the options and how to configure them. For example the semantics of stateful sets (what’s being preserved and what not). Or the monitoring readinessProbes and livenessProbe..
----
2019-04-30 18:10:54 UTC - Poule: merlimat you installed K by hand or through Rancher/others
----
2019-04-30 18:11:21 UTC - Matteo Merli: Mostly EKS or GKE
----
2019-04-30 18:11:59 UTC - Poule: ok, good idea
----
2019-04-30 18:12:14 UTC - Matteo Merli: I guess that helps keeping the scary parts far away
----
2019-04-30 18:12:25 UTC - Poule: yup!
----
2019-04-30 18:26:13 UTC - Thor Sigurjonsson: Yep. I love K8S to be sure - I guess on our end I'm thinking we're needing a multi-site instance of pulsar -- we foresee different workloads and different latency and uptime requirements. also -- I'm betting on doing k8s for anything but the "core" multi-site and highest uptime requirements of that solution..
----
2019-04-30 18:27:14 UTC - Thor Sigurjonsson: I think in our world that means we might have a "bare-metal" core cluster at each site with the high uptime requirements.
----
2019-04-30 18:27:47 UTC - Poule: @Thor Sigurjonsson are you on EKS or GKE
----
2019-04-30 18:27:49 UTC - Thor Sigurjonsson: We're also looking to do workloads in functions that might not play nice with that core cluster but would live nicely in a k8s cluster.
----
2019-04-30 18:28:01 UTC - Thor Sigurjonsson: We've got one GKE cluster at the moment
----
2019-04-30 18:28:08 UTC - Thor Sigurjonsson: but we have on-prem stuff too.
----
2019-04-30 18:28:50 UTC - Thor Sigurjonsson: we also have machine learning workloads that we're exploring integrating somehow into pulsar functions...
+1 : Dan C, Jon Bock
----
2019-04-30 18:30:13 UTC - Thor Sigurjonsson: Our bare metal solution right now is just Docker with host networking and local volumes mounted and all that simply orchestrated with salt. Docker is set up also for live-restore and nicer prod settings like that.
----
2019-04-30 18:32:51 UTC - Poule: do you like salt better than ansible
----
2019-04-30 18:44:06 UTC - Thor Sigurjonsson: Personally -- I guess so yes. I have more experience with it -- I guess it has more "features" which make it more usable for more than just setup of infrastructure / classic config mgmt.
----
2019-04-30 18:44:42 UTC - Thor Sigurjonsson: More reactive stuff I guess that way.
----
2019-04-30 19:11:50 UTC - Chris Bartholomew: I run Pulsar in GKE using pre-emptible nodes, which means my underlying nodes get destroyed and recreated every day. For a stateful service that sounds scary, but Pulsar on k8s handles it just fine. K8s reschedules everything, and Pulsar's architecture means it survives this daily chaos with minimal impact.
----
2019-05-01 05:35:54 UTC - Shivji Kumar Jha: I am trying to run a broker from my IDE (PulsarBrokerStarter.java)
I have changed the broker.conf to use standalone.
```
# Flag to control features that are meant to be used when running in standalone mode
isRunningStandalone=true
# Name of the cluster to which this broker belongs to
clusterName=shiv-local```
how do I tell standalone broker to run its own zookeeper?
I keep getting errors saying
```Required zookeeperServers is null```
----
2019-05-01 05:36:26 UTC - jose farfan: @jose farfan has joined the channel
----
2019-05-01 05:37:35 UTC - Matt Slack: @Matt Slack has joined the channel
----
2019-05-01 05:38:59 UTC - Shivji Kumar Jha: If i run an explicit zookeeper on my laptop and point standalone.conf to use that
```# Zookeeper quorum connection string
zookeeperServers=localhost:32770```
I still get an error saying
```Required zookeeperServers is null```
----
2019-05-01 05:49:49 UTC - Shivji Kumar Jha: Is this documented somewhere?
----
2019-05-01 06:12:52 UTC - Shivji Kumar Jha: Got it, pulsarstandalonestarter it is it seems..
----
2019-05-01 06:12:56 UTC - Sijie Guo: @Shivji Kumar Jha not quite understand your requirements. doesn’t bin/standalone start a standalone cluster?
----
2019-05-01 06:12:58 UTC - Shivji Kumar Jha: looking there now!
----
2019-05-01 06:13:11 UTC - Sijie Guo: if you want to start a standalone cluster in program, use `PulsarStandaloneStarter`
----
2019-05-01 06:13:30 UTC - Shivji Kumar Jha: I want to start pulsar from IDE, to debug bugs like the one you helped me with yesterday
----
2019-05-01 06:13:42 UTC - Shivji Kumar Jha: RIght figured that out just now
----
2019-05-01 06:14:43 UTC - Sijie Guo: i see :+1:
----
2019-05-01 06:37:36 UTC - Shivji Kumar Jha: @Sijie Guo I see the only required field is config
This is my args
``` "--config" "/Users/shivaji.jha/workspace/pulsar/conf/standalone.conf" "--only-broker" "true"```
But jcommander keeps failing to parse (last line below)
```
public PulsarStandaloneStarter(String[] args) throws Exception {
JCommander jcommander = new JCommander();
try {
jcommander.addObject(this);
jcommander.parse(args);
```
----
2019-05-01 07:14:46 UTC - Shivji Kumar Jha: I changed args to
“--config” “/Users/shivaji.jha/workspace/pulsar/conf/standalone.conf”
And there are other crashes now…
----
2019-05-01 07:26:12 UTC - Shivji Kumar Jha: 12:52:30.137 [main] ERROR org.apache.pulsar.functions.worker.WorkerService - Error Starting up in worker
java.lang.RuntimeException: No JavaInstanceJar specified
at org.apache.pulsar.functions.runtime.ProcessRuntimeFactory.<init>(ProcessRuntimeFactory.java:73) ~[classes/:?]
at org.apache.pulsar.functions.worker.FunctionRuntimeManager.<init>(FunctionRuntimeManager.java:145) ~[classes/:?]
at org.apache.pulsar.functions.worker.WorkerService.start(WorkerService.java:165) [classes/:?]
at org.apache.pulsar.broker.PulsarService.startWorkerService(PulsarService.java:1044) [classes/:?]
at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:441) [classes/:?]
at org.apache.pulsar.PulsarStandalone.start(PulsarStandalone.java:305) [classes/:?]
at org.apache.pulsar.PulsarStandaloneStarter.main(PulsarStandaloneStarter.java:102) [classes/:?]
----
2019-05-01 07:41:57 UTC - Shivji Kumar Jha: worked it seems finally :slightly_smiling_face:
----