You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2018/10/03 09:11:04 UTC
Slack digest for #general - 2018-10-03
2018-10-02 14:16:26 UTC - Henrik Andersen: @Henrik Andersen has joined the channel
----
2018-10-02 17:57:42 UTC - sfescape: @sfescape has joined the channel
----
2018-10-02 18:02:08 UTC - sfescape: Hi, couple of days ago I filed an issue in the github project on the websocket reader and haven't received a response. Is that the right place for me to report issues? I just saw this slack channel and am wondering if this is the right place. :thinking_face:
----
2018-10-02 18:03:22 UTC - Sijie Guo: @sfescape github is the right place for reporting issues.
----
2018-10-02 18:06:40 UTC - Sijie Guo: the reader api currently is not available in websocket. there are features lagging in websocket. it is in the backlog for catching up those features. however if there is anyone in the community interested in contribution, you are welcome to help us
----
2018-10-02 18:07:08 UTC - Matteo Merli: correction: reader API is available on WebSocket! :slightly_smiling_face:
----
2018-10-02 18:07:28 UTC - Matteo Merli: multi-topic consumptions is not there
----
2018-10-02 18:07:45 UTC - Sijie Guo: he is asking for ReachedEndOfTopic
----
2018-10-02 18:07:53 UTC - Sijie Guo: and haveMessagesAvailable
----
2018-10-02 18:07:58 UTC - Sijie Guo: I think these are missing
----
2018-10-02 18:08:18 UTC - Matteo Merli: oh, that’s true
----
2018-10-02 18:19:42 UTC - sfescape: @Sijie Guo thank you for the responses. I'm most concerned about the reader issue where if you specify a messageId it's returning messages starting with the first message after the one you specify rather then including the message you pass in.
----
2018-10-02 18:20:53 UTC - Matteo Merli: That is the case in the reader API also in Java/C++/Python and Go
----
2018-10-02 18:27:36 UTC - Sijie Guo: @sfescape I am trying to understand your request at <https://github.com/apache/pulsar/issues/2682> .
typically when a consumer consumed a message, it has the message id; when it connects back with the message id, that means it will need the next message. that’s probably the reason why we are returning the next message.
what is your use case of returning current one? we can discuss how to support your use case.
----
2018-10-02 18:51:14 UTC - sfescape: Yes, and when you specify "latest" it is exclusive of the actual "latest" message. But if you specify "earliest" it is inclusive of the earliest message. When specifying a specific messageId the documentation (and I think the correct behavior) is (or should be) inclusive. Otherwise you have no way to retrieve a message by messageId.
----
2018-10-02 18:52:05 UTC - sfescape: My use case is to store a series of messages that can be retrieved by other systems. So I store the "first" messageId and the "last" messageId in the series which can then later be used to retrieve the range of messages in the series.
----
2018-10-02 18:57:38 UTC - sfescape: The documentation says: "Message ID to start from, earliest or latest (default: latest)". I read that to be inclusive of the messageId.
----
2018-10-02 18:58:57 UTC - sfescape: If the intended behavior is exclusive, then how can you retrieve a message by messageId? You would need to retrieve from "earliest" and just discard all the messages until you got the one you want. Which, of course, would perform more and more slowly over time.
----
2018-10-02 19:00:40 UTC - Matteo Merli: Sure we can think of a way to let the application specify either included/excluded.
----
2018-10-02 19:01:38 UTC - Matteo Merli: regarding docs, I think it’s explicit in the javadocs <http://pulsar.apache.org/api/client/org/apache/pulsar/client/api/ReaderBuilder.html#startMessageId-org.apache.pulsar.client.api.MessageId->
```
The initial reader positioning is done by specifying a message id. The options are:
* MessageId.earliest : Start reading from the earliest message available in the topic
* MessageId.latest : Start reading from the end topic, only getting messages published after the reader was created
* MessageId : When passing a particular message id, the reader will position itself on that specific position. The first message to be read will be the message next to the specified messageId.
```
----
2018-10-02 19:04:23 UTC - sfescape: I see, I wasn't looking at the Java Client API. Would definitely like to be able to specify inclusive/exclusive (would be useful for "latest" case as well).
+1 : Matteo Merli, Ali Ahmed, Nathanial Murphy
----
2018-10-02 19:15:27 UTC - Mark Addison: I am trying to test some code using a standalone instance of pulsar
However, when trying to publish a message to a namespace and exception is thrown
>>>org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: Namespace does not have any clusters configured : local_cluster=standalone ns=tenant/test-ns
----
2018-10-02 19:16:25 UTC - Mark Addison: Does this imply that any testing of code requiring the utilisation of namespaces must be performed against a cluster?
----
2018-10-02 19:18:19 UTC - Mark Addison: This seems quite restrictive especially for development teams who might want to run test code on a local workstation
----
2018-10-02 19:19:32 UTC - Matteo Merli: No, you just have to specify that namespace is assigned to the `standalone` cluster
----
2018-10-02 19:21:15 UTC - Matteo Merli: Or use `public/default` namespace which is already setup, and can be used with short notation (eg: `my-topic` is translated into `<persistent://public/default/my-topic>`
----
2018-10-02 19:55:27 UTC - Grant Wu: Does the C++ client build against musl?
----
2018-10-02 19:55:49 UTC - Grant Wu: I’m curious, because the `pulsar-client` Python package does not install on Alpine based docker images
----
2018-10-02 19:56:14 UTC - Grant Wu: <https://github.com/docker-library/docs/issues/904> seems to be the issue
----
2018-10-02 19:56:20 UTC - Matteo Merli: I think you would have to compile from scratch
----
2018-10-02 19:56:28 UTC - Grant Wu: i don’t understand 100% what’s going on but I think it has something to do with the glibc/musl thing
----
2018-10-02 19:56:44 UTC - Matteo Merli: And most issues would be coming from dependencies to compile with musl
----
2018-10-02 19:58:13 UTC - Grant Wu: Oh, I guess I meant to ask more about the client
----
2018-10-02 19:58:16 UTC - Grant Wu: Same goes for the client?
----
2018-10-02 19:58:43 UTC - Matteo Merli: Yes, client is the only c++ based
----
2018-10-02 19:59:41 UTC - Grant Wu: Do you mean to say that clients based on the official C++ client will have this issue too?
----
2018-10-02 20:00:48 UTC - Matteo Merli: Yes, you would have to build the c++ client lib to make it work. I tried once and gave up after a while.
----
2018-10-02 20:01:11 UTC - Grant Wu: Unfortunate but reasonable
----
2018-10-02 20:34:30 UTC - Greg Stein: @Greg Stein has joined the channel
----
2018-10-03 01:35:17 UTC - Grant Wu: Has anyone dealt with
```
# example-client/vendor/github.com/apache/incubator-pulsar/pulsar-client-go/pulsar
example-client/vendor/github.com/apache/incubator-pulsar/pulsar-client-go/pulsar/c_consumer.go:129:4: could not determine kind of name for C.pulsar_consumer_configuration_set_property
example-client/vendor/github.com/apache/incubator-pulsar/pulsar-client-go/pulsar/c_consumer.go:136:2: could not determine kind of name for C.pulsar_consumer_set_read_compacted
```
before?
----
2018-10-03 01:35:24 UTC - Grant Wu: This is with the Go client.
----
2018-10-03 01:45:28 UTC - Matteo Merli: It seems you have the 2.1 C++ library but the latest Go code. You should pin the Go dependency to the same version as the c++ shared library
----
2018-10-03 01:56:22 UTC - Grant Wu: ah
----
2018-10-03 01:56:26 UTC - Grant Wu: so I need `2.1`?
----
2018-10-03 01:57:17 UTC - Grant Wu: ah, that worked, thanks!
----
2018-10-03 02:03:08 UTC - Grant Wu: Made an issue to improve the documentation around this. i can hopefully get around to it
----
2018-10-03 02:44:37 UTC - Marvin Cai: @Marvin Cai has joined the channel
----
2018-10-03 07:08:45 UTC - Mark Addison: We have to use namespaces and therefore need the ability to test them
----
2018-10-03 07:09:12 UTC - Mark Addison: thanks
----