You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2018/07/09 09:11:02 UTC
Slack digest for #general - 2018-07-09
2018-07-08 09:37:55 UTC - Yuvaraj Loganathan: I trying to start global-zookeeper using `apache-pulsar-2.0.1-incubating/bin# ./pulsar global-zookeeper`
----
2018-07-08 09:38:33 UTC - Yuvaraj Loganathan: I am not able to start as i am getting following error
```[AppClassLoader@6d03e736] info AspectJ Weaver Version 1.8.9 built on Monday Mar 14, 2016 at 21:18:16 GMT
[AppClassLoader@6d03e736] info register classloader sun.misc.Launcher$AppClassLoader@6d03e736
[AppClassLoader@6d03e736] info using configuration file:/tmp/apache-pulsar-2.0.1-incubating/lib/org.apache.pulsar-pulsar-zookeeper-2.0.1-incubating.jar!/META-INF/aop.xml
[AppClassLoader@6d03e736] info using configuration file:/tmp/apache-pulsar-2.0.1-incubating/lib/org.apache.pulsar-pulsar-broker-2.0.1-incubating.jar!/META-INF/aop.xml
[AppClassLoader@6d03e736] info register aspect org.apache.pulsar.zookeeper.FinalRequestProcessorAspect
[AppClassLoader@6d03e736] info register aspect org.apache.pulsar.zookeeper.ZooKeeperServerAspect
[AppClassLoader@6d03e736] info register aspect org.apache.pulsar.broker.zookeeper.aspectj.ClientCnxnAspect
09:37:10.202 [main] INFO org.apache.pulsar.zookeeper.ZooKeeperStarter - Starting ZK stats HTTP server at port 8001
09:37:10.223 [main] INFO org.eclipse.jetty.util.log - Logging initialized @2536ms
09:37:10.294 [main] INFO org.eclipse.jetty.server.Server - jetty-9.3.11.v20160721
09:37:10.331 [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@c2db68f{/,null,AVAILABLE}
09:37:10.349 [main] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@208a9c91{HTTP/1.1,[http/1.1]}{0.0.0.0:8001}
09:37:10.350 [main] INFO org.eclipse.jetty.server.Server - Started @2665ms
09:37:10.356 [main] INFO org.apache.zookeeper.server.DatadirCleanupManager - autopurge.snapRetainCount set to 3
09:37:10.357 [main] INFO org.apache.zookeeper.server.DatadirCleanupManager - autopurge.purgeInterval set to 0
09:37:10.357 [main] INFO org.apache.zookeeper.server.DatadirCleanupManager - Purge task is not scheduled.
09:37:10.358 [main] WARN org.apache.zookeeper.server.quorum.QuorumPeerMain - Either no config or no quorum defined in config, running in standalone mode
09:37:10.407 [main] ERROR org.apache.zookeeper.server.ZooKeeperServerMain - Invalid arguments, exiting abnormally
java.lang.IllegalArgumentException: Invalid number of arguments:[]
at org.apache.zookeeper.server.ServerConfig.parse(ServerConfig.java:56) ~[org.apache.pulsar-pulsar-zookeeper-2.0.1-incubating.jar:2.0.1-incubating]
at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:84) ~[org.apache.pulsar-pulsar-zookeeper-2.0.1-incubating.jar:2.0.1-incubating]
at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53) [org.apache.pulsar-pulsar-zookeeper-2.0.1-incubating.jar:2.0.1-incubating]
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) [org.apache.pulsar-pulsar-zookeeper-2.0.1-incubating.jar:2.0.1-incubating]
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78) [org.apache.pulsar-pulsar-zookeeper-2.0.1-incubating.jar:2.0.1-incubating]
at org.apache.pulsar.zookeeper.ZooKeeperStarter.start(ZooKeeperStarter.java:68) [org.apache.pulsar-pulsar-zookeeper-2.0.1-incubating.jar:2.0.1-incubating]
at org.apache.pulsar.zookeeper.ConfigurationStoreStarter.main(ConfigurationStoreStarter.java:26) [org.apache.pulsar-pulsar-zookeeper-2.0.1-incubating.jar:2.0.1-incubating]
09:37:10.411 [main] INFO org.apache.zookeeper.server.ZooKeeperServerMain - Usage: ZooKeeperServerMain configfile | port datadir [ticktime] [maxcnxns]
Usage: ZooKeeperServerMain configfile | port datadir [ticktime] [maxcnxns]
```
----
2018-07-08 09:39:01 UTC - Yuvaraj Loganathan: @Ali Ahmed any help would be really helpful. Thanks
----
2018-07-08 09:40:57 UTC - Ali Ahmed: I would try this instead
```pulsar-daemon start zookeeper```
----
2018-07-08 09:46:39 UTC - Yuvaraj Loganathan: Zookeeper is running fine. I am looking to start global-zookeeper aka configuration-store. I am starting the process through supervisorctl. I don’t want to process to fork . `./bin/pulsar configuration-store` worked now. Thanks for the help.
----
2018-07-08 12:12:04 UTC - Tomer Lev: Hi how can I address pulsar\bookeeper logs to file instead of journalctl ?
----
2018-07-08 12:15:45 UTC - Tomer Lev: actually the log4j.yml is ponting to files but when I try execute ```sudo cat /opt/pulsar/logs/*``` I get empty results
----
2018-07-08 12:16:16 UTC - Tomer Lev: and when executing ```sudo journalctl -ex -u pulsar.service``` I can see the logs ...
----
2018-07-08 14:14:04 UTC - Matteo Merli: @Tomer Lev The systemd config files are starting pulsar/bk in “simple” mode. That means just running the process in foreground, like `bin/pulsar broker`. By default in this mode, pulsar logs are printed to console and thus collected by systemd.
To enables logs to file, you just need to set these variables:
```
export PULSAR_LOG_DIR="$PULSAR_HOME/logs"
export PULSAR_LOG_APPENDER="RollingFile"
export PULSAR_LOG_FILE=pulsar-broker.log
```
The other alternative is to run Pulsar in daemon mode, `bin/pulsar-daemon start broker`, in which case the logging is sent to file by default.
----
2018-07-08 14:15:42 UTC - Matteo Merli: @Yuvaraj Loganathan, for ZK (same as for broker/bookies) the JVM mem settings are defined by the `PULSAR_MEM` variable in `conf/pulsar_env.sh`
----
2018-07-08 14:16:44 UTC - Yuvaraj Loganathan: @Matteo Merli @Sijie Guo Thank you :thumbsup:
----
2018-07-08 14:25:54 UTC - Tomer Lev: Thanks
----
2018-07-08 16:52:41 UTC - Idan: Hi, is it possible to measure how long it took to a message to be on queue until it was consumed?
heavy_plus_sign : Igor Zubchenok
----
2018-07-08 16:52:47 UTC - Idan: do we have such param in the Message objecT?
----
2018-07-08 16:53:09 UTC - Idan: i know we have publishTime param. but do you populate some consumer recieveTime aswell?
----
2018-07-08 16:54:13 UTC - Matteo Merli: do you mean the instant when the consumer received the message before passing it on to the application?
----
2018-07-08 16:55:59 UTC - Idan: exactlly
----
2018-07-08 16:57:21 UTC - Matteo Merli: I see. I guess it would be easy to add to current logic
+1 : Igor Zubchenok
----
2018-07-08 16:57:40 UTC - Idan: not supported yet?
----
2018-07-08 16:58:44 UTC - Matteo Merli: No, the receive timestamp is not being tracked at the moment
----
2018-07-08 17:02:35 UTC - Idan: ok thanks
----
2018-07-08 17:05:05 UTC - Matteo Merli: though in any case that won’t tell the whole story: messages are pushed to consumers up to a certain level and then backpressure applies if the application is not dequeuing fast enough.
The time spent in queue might then show up as lower, since the message was delivered later, only when the consumer had enough space to receive it.
----
2018-07-08 17:05:38 UTC - Idan: but i think that time should be populated as soon as the message is hitting the client
----
2018-07-08 17:05:50 UTC - Idan: to eliminate the client totally
----
2018-07-08 17:06:18 UTC - Matteo Merli: yes
----
2018-07-09 08:31:25 UTC - Idan: Morning
----
2018-07-09 08:31:27 UTC - Idan: ```2018-07-09 08:00:01,887 WARN org.apache.pulsar.shade.com.scurrilous.circe.checksum.Crc32cIntChecksum - Failed to load Circe JNI library. Falling back to Java based CRC32c provider```
----
2018-07-09 08:31:32 UTC - Idan: what that warning would mean?
----
2018-07-09 08:31:37 UTC - Idan: anything we need to pay attention to?
----
2018-07-09 08:34:33 UTC - Sijie Guo: @Idan: pulsar is using a native crc32 implementation for checksum. we only ship the native implementation in linux platform. if you are running the pulsar client in a non-linux platform, it will fall back to java based crc32c implementation.
nothing major to worry
----
2018-07-09 08:37:24 UTC - Idan: we use Ubuntu
----
2018-07-09 08:37:30 UTC - Idan: thats linux platform
----
2018-07-09 08:37:35 UTC - Idan: why would I get that warning?
----
2018-07-09 08:37:53 UTC - Idan: Ubuntu 16
----
2018-07-09 08:43:40 UTC - Sijie Guo: interesting..
----
2018-07-09 08:43:47 UTC - Sijie Guo: that might be related to shading
----
2018-07-09 08:44:44 UTC - Sijie Guo: do you mind filing a github issue for this? will take a look in the morning (it is close to 2am PST here)
----
2018-07-09 08:56:54 UTC - Ivan Kelly: @Sijie Guo I see them regularly on linux too
----
2018-07-09 09:00:43 UTC - Sijie Guo: does it happen when using pulsar-client-original?
----
2018-07-09 09:01:58 UTC - Ivan Kelly: I see it in the logs when starting the proxy
----