You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2018/06/18 09:11:02 UTC
Slack digest for #general - 2018-06-18
2018-06-17 14:23:18 UTC - Daniel Ferreira Jorge: Hi, are there short term plans to bring the python client up to speed with the java api?
----
2018-06-17 14:27:55 UTC - Sijie Guo: Yes we are catching up with the cpp features in 2.1 and 2.2 release.
----
2018-06-17 14:28:40 UTC - Sijie Guo: 2.1 is on the way, planning to start the release process this week.
----
2018-06-17 14:31:33 UTC - Daniel Ferreira Jorge: Great! Also, are there any plans to "promote" the cpp/python to a first class citizen? Meaning, always being updated with the latest release as it happens and not always catching up?
----
2018-06-17 14:33:09 UTC - Daniel Ferreira Jorge: In my humble opinion, python is popular enough for this to happen. Even <http://streaml.io|streaml.io> uses python in some blog posts and main examples instead of java...
----
2018-06-17 14:36:52 UTC - Sijie Guo: Yeah, we wish we can do that as changes happen, not catching up. However the fact is people in the community have more experiences on Java, the hands on cpp/python are lesser than java. So features end up happen faster at java :) will bring this to dev@ mailing list to see how we can improve :)
----
2018-06-17 14:37:45 UTC - Daniel Ferreira Jorge: Great @Sijie Guo, thank you!
slightly_smiling_face : Sijie Guo
----
2018-06-17 21:10:50 UTC - Rob V: guys, how do you manage state in Pulsar? I came from Kafka where you can use Kafka Streams to aggregate messages and even save the actual state back into a topic. Is anything like that available in Pulsar?
----
2018-06-17 21:17:07 UTC - Sijie Guo: @Rob V currently pulsar doesn’t kafka streams like things. however pulsar provides a module called pulsar functions, which you can run serverless functions in pulsar to do in-stream processing. 2.0 support stateless functions, in 2.1 we will have stateful functions ready for preview, which you can use for achieve kafka streams functionality but it is a more generic serverless approach, where you can write your own functions to achieve much comprehensive functionalities.
you can checkout pulsar functions here:
<https://pulsar.incubator.apache.org/docs/latest/functions/overview/>
and streamlio folks write some blogs about it:
<https://streaml.io/blog/pulsar-functions-for-developers/>
<https://streaml.io/blog/pulsar-functions/>
----
2018-06-17 21:17:12 UTC - Sijie Guo: hope this helps
----
2018-06-17 21:32:12 UTC - Rob V: thank you @Sijie Guo
----
2018-06-18 02:04:04 UTC - Jonathan: @Jonathan has joined the channel
----
2018-06-18 02:09:00 UTC - Jonathan: Hi, I'm doing a proof of concept of being able to write 1,000,000 messages per second into a database using Pulsar and Spark Streaming. However, I'm trying to understand Pulsar first by playing around with it.
```
<https://github.com/streamlio/pulsar-java-tutorial>
```
If I increased the for loop to 10,000 or even 100,000 it takes a longer than a second to produce and consume. Understandable because there needs to be more clusters. I'm still learning big data, but looking through the faqs
```
<https://github.com/apache/incubator-pulsar/blob/master/faq.md#i-tested-the-performance-using-performanceproducer-between-two-server-node-with-10000mbits-nicand-i-tested-tcp-throughput-can-be-larger-than-1gbs-i-saw-that-the-max-msg-throughput-is-around-1000000-msgs-when-using-little-msg_sizesuch-as-64128bytes-when-i-increased-the-msg_size-to-1028-or-larger--then-the-msgs-will-decreased-sharply-to-150000msgs-and-both-has-max-throughput-around---1600mbits-which-is-far-from-1gbs--and-im-curious-that-the-throughput-between-producer-and-broker-why-cant-excess-1600mbits---it-seems-that-the-producer-executor-only-use-one-thread-is-this-the-reasonthen-i-start-two-producer-client-jvm-the-throughput-increased-not-much-just-about-little-beyond-1600mbits-any-other-reasons>
```
I'm assuming one of the example in the repo shows how to produce/consume 100,000 messages within a second. I was trying to run it and dissect it afterwards, but I'm running into issues. When I do ```bin/pulsar standalone``` I keep getting
```
ERROR org.apache.pulsar.broker.ServiceConfigurationUtils - <http://unallocated.barefruit.co.uk|unallocated.barefruit.co.uk>: <http://unallocated.barefruit.co.uk|unallocated.barefruit.co.uk>: nodename nor servname provided, or not known
java.net.UnknownHostException: <http://unallocated.barefruit.co.uk|unallocated.barefruit.co.uk>: <http://unallocated.barefruit.co.uk|unallocated.barefruit.co.uk>: nodename nor servname provided, or not known
at java.net.InetAddress.getLocalHost(InetAddress.java:1505) ~[?:1.8.0_172]
at org.apache.pulsar.broker.ServiceConfigurationUtils.unsafeLocalhostResolve(ServiceConfigurationUtils.java:42) [pulsar-broker-common.jar:2.1.0-incubating-SNAPSHOT]
at org.apache.pulsar.PulsarStandaloneStarter.<init>(PulsarStandaloneStarter.java:129) [pulsar-broker.jar:2.1.0-incubating-SNAPSHOT]
at org.apache.pulsar.PulsarStandaloneStarter.main(PulsarStandaloneStarter.java:265) [pulsar-broker.jar:2.1.0-incubating-SNAPSHOT]
Caused by: java.net.UnknownHostException: <http://unallocated.barefruit.co.uk|unallocated.barefruit.co.uk>: nodename nor servname provided, or not known
```
Can someone please help me?
----
2018-06-18 02:12:23 UTC - Ali Ahmed: what is this host ?
----
2018-06-18 02:12:27 UTC - Ali Ahmed: ```<http://unallocated.barefruit.co.uk|unallocated.barefruit.co.uk>```
----
2018-06-18 02:50:06 UTC - Jonathan: i have no idea
----
2018-06-18 02:50:27 UTC - Jonathan: i'm googling it right now and it has something to do with my ISP
----
2018-06-18 03:03:38 UTC - Ali Ahmed: can you type hostname on the machine where you are running this
----
2018-06-18 03:04:01 UTC - Ali Ahmed: as an alternative you can also use the pulsar docker image
----
2018-06-18 03:05:51 UTC - Jonathan: actually i fixed the issue. ```<https://stackoverflow.com/questions/1881546/inetaddress-getlocalhost-throws-unknownhostexception>``` I also had an instance of zookeeper running so I killed it and started pulsar in standalone
----
2018-06-18 03:05:53 UTC - Jonathan: everything works
----
2018-06-18 03:07:00 UTC - Ali Ahmed: ok
----
2018-06-18 06:32:43 UTC - Sagar Gaur: Is there any way of knowing that which release is my apache pulsar source or binary...?
----
2018-06-18 06:44:46 UTC - jia zhai: @Sagar Gaur In the source, the pom.xml contains the version info : `<version>2.1.0-incubating-SNAPSHOT</version>`
----