You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/11/27 09:11:02 UTC
Slack digest for #general - 2019-11-27
2019-11-26 09:28:13 UTC - xue: How to use skywalking in pulse function？
----
2019-11-26 09:34:29 UTC - Alexandre DUVAL: Fyi, <https://github.com/apache/pulsar-manager/issues/14>
----
2019-11-26 09:38:25 UTC - Keith: thanks
----
2019-11-26 09:59:18 UTC - xue: Is this a bug?
----
2019-11-26 11:48:50 UTC - jia zhai: <https://docs.google.com/document/d/1FBAb6zFRsTwhJ79YjCzlyh1sF_2h9ke3arspJpYdSCI/edit>
<https://medium.com/streamnative/use-apache-skywalking-to-trace-apache-pulsar-messages-b543ac253053>
There were some docs that written by @Penghui Li.
----
2019-11-26 11:49:35 UTC - jia zhai: For Pulsar function, there may need some further work.
----
2019-11-26 11:52:00 UTC - xue: The example in this PDF document I tested is OK.
----
2019-11-26 11:52:37 UTC - xue: Pulsar function，I changed the RuntimeUtils.java code, skywalking can be used in pulsar function
----
2019-11-26 11:53:09 UTC - jia zhai: :thumbsup_all:
----
2019-11-26 11:53:37 UTC - Penghui Li: Cool,
----
2019-11-26 11:53:51 UTC - xue:
----
2019-11-26 11:54:16 UTC - jia zhai: It would be great if you could write some doc about how you achieve and use this. :grinning:
----
2019-11-26 11:55:23 UTC - xue: Now it's for testing。I will write this modification in the document and share it
----
2019-11-26 11:56:15 UTC - jia zhai: :+1:
----
2019-11-26 11:56:57 UTC - xue: Chinese is OK
----
2019-11-26 11:57:14 UTC - jia zhai: :grinning:
----
2019-11-26 11:57:17 UTC - Penghui Li: 是的你可以写中文的文档
----
2019-11-26 11:58:54 UTC - xue: 好的，我先修改一下，因为我用function功能，先改成满足自己需求的
100 : Sijie Guo
----
2019-11-26 12:04:43 UTC - Penghui Li: 欢迎贡献回pulsar社区:grin:
----
2019-11-26 12:41:37 UTC - Christophe Bornet: I need to know the current consumer lag for a message published some time ago. The idea being estimating how much time there is before a given message for which I know the publish time will be consumed. Anyone knows how I could achieve this ?
----
2019-11-26 14:47:13 UTC - Christophe Bornet: Done : <https://github.com/apache/pulsar/issues/5752>
----
2019-11-26 16:11:00 UTC - Britt Bolen: Thanks… I think the comments in the interface should be updated then…
/**
* Asynchronously close the PulsarClient and release all the resources.
* <p>
* This operation will trigger a graceful close of all producer, consumer and reader instances that this client has
* currently active. That implies that close and wait, asynchronously, until all pending producer send requests are
* persisted.
*
* @throws PulsarClientException
* if the close operation fails
*/
CompletableFuture<Void> closeAsync();
----
2019-11-26 16:49:55 UTC - David Kjerrumgaard: Thanks, I will review the issue on github
----
2019-11-26 16:51:18 UTC - Joshua Dunham: The basics are the record reader / record writer seems new in the nifi connector. Camt figure out how to make it work. :(
----
2019-11-26 16:54:41 UTC - Joshua Dunham: I havent said thanks for the help, its appreciated. :)
----
2019-11-26 17:00:48 UTC - David Kjerrumgaard: No worries. Glad to help
----
2019-11-26 17:07:22 UTC - David Kjerrumgaard: The RecordReader and RecordWriters were added after I wrote the blog post you are referring to. I will update the issue with the proper steps
----
2019-11-26 17:17:08 UTC - Luke Lu: ```zk: localhost:2181(CONNECTED) 0] get /managed-ledgers/<topic-path>
????? ?????-*???????????????????2?*
s34
bucket*<bucket>
region us-west-2.
endpoint"<https://s3.us-west-2.amazonaws.com4>
bucket*<bucket>
region us-west-2.
endpoint"<https://s3.us-west-2.amazonaws.com4>
bucket*<bucket>
region us-west-2.```
The same endpoint, bucket, region info are repeated many times in the same entries.
----
2019-11-26 17:27:01 UTC - David Kjerrumgaard: In the meantime, you may want to look at the following; <https://community.cloudera.com/t5/Community-Articles/Record-based-processors-in-Apache-NiFi-1-2/ta-p/245683> which describes the reader/writer pattern in general.
----
2019-11-26 17:30:59 UTC - David Kjerrumgaard: <https://bryanbende.com/development/2017/06/20/apache-nifi-records-and-schema-registries> is another good resource
----
2019-11-26 17:48:52 UTC - Sijie Guo: hmm interesting. this looks like a bug.
----
2019-11-26 17:49:00 UTC - Sijie Guo: Can you file a github issue?
----
2019-11-26 18:05:11 UTC - Joshua Dunham: Spunds great, I'll look at both.
----
2019-11-26 20:27:22 UTC - Britt Bolen: Actually I think there might be a bug… <https://github.com/apache/pulsar/blob/cc746322d51cea14b04092370bcebf695721fe70/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ProducerImpl.java#L633> It really looks like we’re discarding all pending messages after completing the close without calling the callbacks… so when I do a quick test like….
`future = producer.sendAsync();`
`producer.close();`
`future.get();`
The future blocks forever… because the pending message is being discarded… I think.
Calling `producer.flush()` before close fixes my test.
@Matteo Merli thoughts?
----
2019-11-26 20:27:30 UTC - Alexandre DUVAL: There is a configuration point to enable auth on public/default?
----
2019-11-26 20:27:38 UTC - Alexandre DUVAL: Or disable public/default.
----
2019-11-26 20:28:19 UTC - Matteo Merli: if authz is turned on, it's automatically required for public/default as well
----
2019-11-26 20:29:42 UTC - Alexandre DUVAL: true, did a bad usage.
----
2019-11-27 02:34:04 UTC - xue: Tiered Storage，I created <persistent://jmm/test/my-topic> , execute the cmd : bin/pulsar-admin topics offload --size-threshold 10M jmm/test/my-topic, return: Topic not found
----
2019-11-27 03:08:12 UTC - xue: In the pulsar document, use this command to execute the specified topic name, but my topic has two partitions. If you do not specify a partition, the specified topic will appear topic not found, and then I specify the partition number, which can be executed normally.
----
2019-11-27 03:10:20 UTC - xue: In the pulsar document, is the topic specified in the command of Tiered Storage not partitioned?
----
2019-11-27 06:06:50 UTC - Sijie Guo: the `offload` command only applies on a partition of partitioned topic or a non-partitioned topic.
----
2019-11-27 07:22:33 UTC - xue: The description in the document is not comprehensive enough
----