You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/04/11 09:11:02 UTC
Slack digest for #general - 2019-04-11
2019-04-10 09:11:54 UTC - Olivier Chicha: @Ali Ahmed Thank you for you answer.
I am not familiar with Erlang / Elixir, but another entity of our company is using Elixir, and we would like to make sure that they will be able to use pulsar.
If I understands correctly what you say is that from Elixir they could access through a wrapper (they would need to develop, I guess) to the C++ implementation of the client.
Is that correct?
----
2019-04-10 13:21:09 UTC - bhagesharora: @Matteo Merli
I am implementing message deduplication through java, I have one produce class which is reading a text file, text file hold(1, 2, 3, 4, 5) String type Data, and we have consume class which is reading same Data.
When I am running/pushing messages from producer class I am getting error..
message..1
[main] WARN org.apache.pulsar.shade.com.scurrilous.circe.checksum.Crc32cIntChecksum - Failed to load Circe JNI library. Falling back to Java based CRC32c provider
message..3
message..5
Exception in thread "main" java.util.NoSuchElementException
at java.util.Scanner.throwFor(Scanner.java:862)
at java.util.Scanner.next(Scanner.java:1371)
at pulsar_producer_consumer.pulsar_producer_consumer.MessageDeduplicationProducer1.main(MessageDeduplicationProducer1.java:83)
[pulsar-timer-4-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - [<persistent://public/default/topic-1>] [standalone-7-3] Pending messages: 0 --- Publish throughput: 0.03 msg/s --- 0.00 Mbit/s --- Latency: med: 207.656 ms - 95pct: 207.656 ms - 99pct: 207.656 ms - 99.9pct: 207.656 ms - max: 207.656 ms --- Ack received rate: 0.03 ack/s --- Failed messages: 0
When I am running Consumer class or trying to consume same messages, so I can see in output, it's consuming only two messages, see output below
Created Pulsar consumer
[pulsar-client-io-1-1] WARN org.apache.pulsar.shade.com.scurrilous.circe.checksum.Crc32cIntChecksum - Failed to load Circe JNI library. Falling back to Java based CRC32c provider
inside loop 2
inside loop 4
[pulsar-timer-4-1] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - [<persistent://public/default/topic-1>] [tutorial-subscription] [05a87] Prefetched messages: 0 --- Consume throughput: 0.03 msgs/s --- Throughput received: 0.00 msg/s --- 0.03 Mbit/s --- Ack sent rate: 0 ack/s --- Failed messages: 0 --- Failed acks: {}
Why Its not able to consume all messages, Why It's consuming even no, or producing odd no. ??
----
2019-04-10 13:23:43 UTC - Matteo Merli: Created <https://github.com/apache/pulsar/pull/4015> as a pre-req
----
2019-04-10 14:21:46 UTC - Olivier Chicha: Hello I would like, for the purpose of unit test to embed multiple pulsar broker (+1zookeeper +1bookeeper) inside a single JVM
I assume this is feasible by copying and modifying the code of PulsarStandalone, but I was wondering if there is already a project / class that would provide that.
----
2019-04-10 17:08:30 UTC - Bicou: @Bicou has joined the channel
----
2019-04-10 18:04:11 UTC - Ali Ahmed: currently yes that wrapper has not been implemented yet since no one requested it.
----
2019-04-10 18:37:56 UTC - Steve Kim: Thanks for your help. I looked at `BlobStoreBackedInputStreamImpl` and `BlobStoreBackedReadHandleImpl` to understand how performance could be improved. I think that reads from blob store would be faster with speculative read-ahead, so that `refillBufferIfNeeded` can swap an already-populated buffer instead of blocking on a call to `blobStore.getBlob`.
----
2019-04-10 19:35:58 UTC - Douglas Mutart: @Douglas Mutart has joined the channel
----
2019-04-10 22:34:49 UTC - Ali Ahmed: @Olivier Chicha you can use this
<https://github.com/streamlio/pulsar-embedded-tutorial/blob/master/src/test/java/org/apache/pulsar/PulsarEmbeddedTest.java>
----
2019-04-11 02:27:59 UTC - hao zhang: @hao zhang has joined the channel
----
2019-04-11 04:01:09 UTC - jia zhai: :+1:
----
2019-04-11 04:05:48 UTC - jia zhai: Yes, we should do that.
Are you willing to contribute to this feature?:slightly_smiling_face:
----
2019-04-11 04:08:02 UTC - jia zhai: Opened Issue <https://github.com/apache/pulsar/issues/4025> to track it.
----
2019-04-11 07:52:30 UTC - Kev Jackson: just reading the deployment instructions and we would like to use a pre-deployed/defined zookeeper cluster - is this comment in the documentation valid " You'll need to install the Pulsar binary package on each machine in the cluster, including machines running ZooKeeper and BookKeeper."
----
2019-04-11 07:53:35 UTC - Ali Ahmed: you can just connect to an existing zookeeper instance
----
2019-04-11 07:55:18 UTC - Kev Jackson: ok so I'm going to deploy a standard zookeeper cluster and then deploy pulsar as a separate cluster - I will still need to download the pulsar installation tarball so I can run this step as it requires a local connection <https://pulsar.apache.org/docs/en/deploy-bare-metal/#initializing-cluster-metadata>
----
2019-04-11 08:14:27 UTC - Kev Jackson: what happens if I execute: `bin/initialize-cluster-metadata` with the same configuration twice? is it idempotent (ie. safe, the second initialize is a no-op?)
----
2019-04-11 08:37:54 UTC - Sijie Guo: it should be idemponent. the second one is a no-op
----