You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2020/01/11 09:11:03 UTC
Slack digest for #general - 2020-01-11
2020-01-10 12:45:16 UTC - Eira S: Trying to connect to pulsar client using pulsar+ssl (using pulsar-client-go), I followed all the steps <https://pulsar.apache.org/docs/en/security-tls-authentication/> But I get this error:
```INFO[0000] Connecting to broker remote_addr="<pulsar+ssl://localhost:6651>"
WARN[0000] Failed to connect to broker. error="x509: certificate signed by unknown authority" remote_addr="<pulsar+ssl://localhost:6651>"
INFO[0000] Connection closed ```
----
2020-01-10 13:19:40 UTC - Fabien LD: Not an error, a warn because it is a self signed certificate
----
2020-01-10 13:20:19 UTC - Fabien LD: It should work with that log in my understanding
----
2020-01-10 16:38:27 UTC - Eira S: Thanks @Fabien LD . It still didn't connect with Broker, I used the certs provided with pulsar-client-go instead of using self-signed CA. Works for now!
+1 : Fabien LD
----
2020-01-10 17:38:02 UTC - Fernando: does anyone know if there’s a `kafkacat` equivalent for pulsar?
----
2020-01-10 17:38:51 UTC - Jerry Peng: What does kafkacat do? Tail a topic?
----
2020-01-10 17:39:15 UTC - Jerry Peng: If so you can use the pulsar-client cli to consume from a topic
----
2020-01-10 17:39:21 UTC - Fernando: <https://github.com/edenhill/kafkacat>
----
2020-01-10 17:39:47 UTC - Jerry Peng: Yup the equivalent is pulsar-client cli tool
----
2020-01-10 17:40:13 UTC - Jerry Peng: Found in the bin/ folder
----
2020-01-10 17:40:16 UTC - Fernando: this is `bin/pulsar-client`?
----
2020-01-10 17:40:18 UTC - Fernando: ok cool
----
2020-01-10 17:40:22 UTC - Jerry Peng: Yes
----
2020-01-10 17:40:57 UTC - Jerry Peng: There are also examples of usage on the Apache pulsar website
----
2020-01-10 17:44:41 UTC - Fernando: For future reference: <https://pulsar.apache.org/docs/en/reference-cli-tools/>
----
2020-01-10 17:45:52 UTC - Fernando: Thanks! I ended up doing that
----
2020-01-10 18:56:03 UTC - Eira S: pulsar-client-go: Is it possible to have two pulsar instances running on 2 different docker containers of same docker image?
This is how I have two containers set up:
```bash-4.2# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8c3128d34f4d pulsar-client-go-test:latest "/pulsar/bin/pulsar …" 41 seconds ago Up 40 seconds 0.0.0.0:6650-6651->6650-6651/tcp, 0.0.0.0:8080->8080/tcp, 0.0.0.0:8443->8843/tcp pulsar-client-go-test
f896860278a7 a698de63982b "/pulsar/bin/pulsar …" 8 minutes ago Up 8 minutes 0.0.0.0:7650-7651->7650-7651/tcp, 0.0.0.0:8081->8081/tcp, 0.0.0.0:8444->8844/tcp pulsar-client-go-2nd```
I get these errors on both docker containers while trying to produce messages using CLI
```18:54:38.080 [pulsar-client-io-1-1] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0xbb0539f5, L:/127.0.0.1:36174 - R:localhost/127.0.0.1:6651] Received error from server: org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available
18:54:38.083 [pulsar-client-io-1-1] ERROR org.apache.pulsar.client.impl.ProducerImpl - [my-topic] [null] Failed to create producer: org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available
18:54:38.084 [main] ERROR org.apache.pulsar.client.cli.PulsarClientTool - Error while producing messages
18:54:38.084 [main] ERROR org.apache.pulsar.client.cli.PulsarClientTool - org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available```
----
2020-01-10 19:47:33 UTC - Piotr Jander: @Piotr Jander has joined the channel
----
2020-01-10 20:17:41 UTC - Jordan Widdison: @Jordan Widdison has joined the channel
----
2020-01-10 20:18:13 UTC - Daniel: @Daniel has joined the channel
----
2020-01-10 20:21:44 UTC - Roman Popenov: so bastion is like the admin pod if I understand it correctly?
----
2020-01-10 20:37:41 UTC - Jake: @Jake has joined the channel
----
2020-01-10 21:08:57 UTC - Roman Popenov: I keep getting out of heap exception thrown in local run function
----
2020-01-10 21:12:41 UTC - Roman Popenov: Is a function processing one message at a time in localrun or are there many threads that are spawn to try and process al the messages at once?
----
2020-01-10 23:38:51 UTC - Jerry Peng: > Is a function processing one message at a time in localrun
yes but the pulsar client might be running multiple threads
----
2020-01-10 23:39:07 UTC - Jerry Peng: try increasing the direct memory set for the JVM
----
2020-01-10 23:41:16 UTC - Roman Popenov: Hm… I have an IO Connectors, it is streaming an archived log file from S3, archived the file is roughly ~300 Mbs, and unarchived the log file is about 9 Gs
----
2020-01-10 23:41:54 UTC - Roman Popenov: Basically I stream the archived file through a buffered reader and write a message that is a string of roughly 100 lines onto a topic.
----
2020-01-10 23:42:40 UTC - Roman Popenov: It seems like Pulsar doesn’t have any issues reading and streaming it to a topic
----
2020-01-10 23:43:09 UTC - Roman Popenov: But when I launch the function, which reads the message, splits it into 100 lines and does some ETL on each
----
2020-01-10 23:43:22 UTC - Roman Popenov: and pushes it back to S3
----
2020-01-10 23:43:27 UTC - Roman Popenov: that is when the heap issue arrives
----
2020-01-10 23:45:18 UTC - Roman Popenov: really weird
----
2020-01-10 23:48:17 UTC - Jerry Peng: Can do a heap dump?
----
2020-01-10 23:50:48 UTC - Jerry Peng: how are you starting localrun?
----
2020-01-10 23:51:38 UTC - Roman Popenov: `/pulsar# /pulsar/bin/pulsar-admin functions localrun --jar /tmp/Pulsar-Demo-1.0-SNAPSHOT-uberjar.jar --classname com.fp.gem.pulsar.functions.PulsarSysmonTopicETL --namespace test-ns --tenant fp-tenant --inputs <persistent://fp-tenant/test-ns/demo-topic-sysmon> --output <persistent://fp-tenant/test-ns/demo-topic-sysmon-cim> --subs-name my-function-subscription`
----
2020-01-10 23:51:42 UTC - Roman Popenov: Running that command
----
2020-01-10 23:52:50 UTC - Jerry Peng: did you adjust the memory settings accordingly here:
<https://github.com/apache/pulsar/blob/master/conf/pulsar_tools_env.sh#L45>
----
2020-01-10 23:53:46 UTC - Roman Popenov: Where can I find those?
----
2020-01-10 23:54:25 UTC - Roman Popenov: I only adjusted the memory and cpu values in the heml values file
----
2020-01-10 23:55:33 UTC - Roman Popenov: ```# Extra options to be passed to the jvm
PULSAR_MEM="-Xmx128m -XX:MaxDirectMemorySize=128m"```
----
2020-01-10 23:55:53 UTC - Roman Popenov: Those are the values I find in `/pulsar/conf` on the pod
----
2020-01-10 23:59:58 UTC - Jerry Peng: Can you check what the heap or direct memory is set for the jvm process spawned from your localrun cmd
----
2020-01-11 00:01:03 UTC - Roman Popenov: Not sure how to do it for localrun
----
2020-01-11 00:01:13 UTC - Jerry Peng: jps -v
----
2020-01-11 00:04:22 UTC - Roman Popenov: ```root@good-koala-pulsar-broker-5f56c84c4d-tgm4z:/pulsar# jps -v
389 LocalRunner -Dlog4j.configurationFile=log4j2.yaml -<http://Djava.net|Djava.net>.preferIPv4Stack=true -Xmx128m -XX:MaxDirectMemorySize=128m -Dio.netty.leakDetectionLevel=disabled -Dpulsar.log.appender=RoutingAppender -Dpulsar.log.dir=/pulsar/logs -Dpulsar.log.level=info -Dpulsar.routing.appender.default=Console -Dpulsar.functions.java.instance.jar=/pulsar/instances/java-instance.jar -Dpulsar.functions.python.instance.file=/pulsar/instances/python-instance/python_instance_main.py
441 Jps -Dapplication.home=/usr/local/openjdk-8 -Xms8m
364 PulsarAdminTool -Dlog4j.configurationFile=log4j2.yaml -<http://Djava.net|Djava.net>.preferIPv4Stack=true -Xmx128m -XX:MaxDirectMemorySize=128m -Dio.netty.leakDetectionLevel=disabled -Dpulsar.log.appender=RoutingAppender -Dpulsar.log.dir=/pulsar/logs -Dpulsar.log.level=info -Dpulsar.routing.appender.default=Console```
----
2020-01-11 00:06:13 UTC - Jerry Peng: are you running a python function?
----
2020-01-11 00:06:18 UTC - Roman Popenov: no
----
2020-01-11 00:06:20 UTC - Roman Popenov: Java
----
2020-01-11 00:06:48 UTC - Jerry Peng: nvm I miss read
----
2020-01-11 00:07:03 UTC - Jerry Peng: try setting Xmx128m -XX:MaxDirectMemorySize=128m to be higher
----
2020-01-11 00:07:27 UTC - Roman Popenov: Ok. I can try
----
2020-01-11 00:10:46 UTC - Roman Popenov: I am also thinking that perhaps it has something to do with the env settings:
```# Extra options to be passed to the jvm
PULSAR_MEM="-Xms512m -Xmx1g -XX:MaxDirectMemorySize=512m -Dio.netty.leakDetectionLevel=disabled -Dio.netty.recycler.linkCapacity=1024 -XX:+ParallelRefProcEnabled -XX:+UnlockExperimentalVMOptions -XX:+AggressiveOpts -XX:+DoEscapeAnalysis -XX:ParallelGCThreads=32 -XX:ConcGCThreads=32 -XX:G1NewSizePercent=50 -XX:+DisableExplicitGC -XX:-ResizePLAB -XX:+ExitOnOutOfMemoryError -XX:+PerfDisableSharedMem"
# Garbage collection options
PULSAR_GC="-XX:+UseG1GC -XX:MaxGCPauseMillis=10"```
----
2020-01-11 00:12:44 UTC - Jerry Peng: Thats the broker configurations?
----
2020-01-11 00:12:51 UTC - Roman Popenov: yes
----
2020-01-11 00:13:32 UTC - Jerry Peng: The pulsar-admin tool will be different and will use the settings in conf/pulsar_tools_env.sh
----
2020-01-11 00:16:02 UTC - Roman Popenov: ok, I will try to double the settings there and see what happens
----
2020-01-11 00:21:40 UTC - Roman Popenov: Yeah, looks like it did solve the issue
----
2020-01-11 00:22:32 UTC - Roman Popenov: hopefully I can process the whole archived stream :crossed_fingers:
----
2020-01-11 00:23:18 UTC - Jerry Peng: :+1:
----
2020-01-11 01:34:45 UTC - ravi satya durga prasad Yenugula: @ravi satya durga prasad Yenugula has joined the channel
----
2020-01-11 01:39:02 UTC - ravi satya durga prasad Yenugula: Hi, I am trying to install Pulsar on a Ec2 instance (centos) and I am getting memory error
----
2020-01-11 01:39:28 UTC - ravi satya durga prasad Yenugula: ```OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000080000000, 2147483648, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 2147483648 bytes for committing reserved memory.
# An error report file with more information is saved as:```
----
2020-01-11 01:44:18 UTC - ravi satya durga prasad Yenugula: Error Logs
----
2020-01-11 01:44:43 UTC - ravi satya durga prasad Yenugula: any Idea?
----
2020-01-11 02:39:34 UTC - Alexandre DUVAL: Hi, you need to set the $PULSAR_MEM env var.
----
2020-01-11 03:10:55 UTC - Sijie Guo: is the event sequence external sequence or pulsar sequence id?
----