You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/08/29 09:11:03 UTC
Slack digest for #general - 2019-08-29
2019-08-28 09:30:39 UTC - xue: If Topic uploads schema, when deleting Topic, schema will not be deleted automatically. You need to connect to zookeeper to delete schema information manually. Is this a bug?
----
2019-08-28 13:17:09 UTC - Diego Salvi: Hi! I've a problem with one topic. I cannot read it anymore... I get "Failed to create consumer: Topic is temporarily unavailable" every time I attempt to create a reader but other than that I don't see any more errors on logs Could you help me to pin point where the problem could be? (I didn't still rebooted the broker)
----
2019-08-28 15:02:43 UTC - Alexandre DUVAL: @Jerry Peng Hi, for pulsar sql, how we should provide jwt token to auth presto to pulsar? On the pulsar connector config (which fields?) or using presto --access-token, but what should it looks like?
----
2019-08-28 15:19:09 UTC - Alexandre DUVAL: I tried using admin token like `pulsar sql --access-token "<TOKEN>"`, or `"token:<TOKEN>"` but still get 401 from pulsar :confused:.
----
2019-08-28 16:51:00 UTC - Thor Sigurjonsson: I noticed the go function/client instructions require use of the c++ client, then also that for macOS there are instructions for using homebrew to install it. It appears that the homebrew formula is still the older c++ client version 2.3.2 something and the go client build is now on 2.4.0. Is there a way to get a newer homebrew formula, for that to be updated or getting my c++ client built on mac for this (didn't see docs directly for that)?
----
2019-08-28 16:52:36 UTC - Poule: ```
b'Cannot construct instance of `org.apache.pulsar.common.protocol.schema.PostSchemaPayload` (although at least one Creator exists): no String-argument constructor/factory method to deserialize from String value (\'{"name": "b", "type": "AVRO", "schema": "", "properties": {}}\')\n at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 1]'
```
what am i doing wrong?
----
2019-08-28 16:53:06 UTC - Poule: it's on POST to `/v2/schemas/{tenant}/{namespace}/{topic}/schema`
----
2019-08-28 17:24:37 UTC - Sijie Guo: @xue in `topic delete` command, you can specify an option to delete schema as welll.
----
2019-08-28 17:25:52 UTC - Sijie Guo: @Diego Salvi
it might be worth to do the followings:
- run `topic stats` to get the topic stats
- run `topic internal-stats` to get the internal stats
- get a jstack of the broker
These information can help understand why a topic is temporarily unavailable.
----
2019-08-28 17:27:52 UTC - Sijie Guo: @Ali Ahmed knows the best about the homebrew part. It is probably that we didn’t update homebrew.
----
2019-08-28 17:29:44 UTC - Sijie Guo: there is no “name” property in `PostSchemaPayload` structure.
----
2019-08-28 17:29:52 UTC - Sijie Guo: <http://pulsar.apache.org/docs/en/next/schema-manage/#upload-a-schema>
----
2019-08-28 17:30:04 UTC - Sijie Guo: please refer to the new schema documentation.
----
2019-08-28 18:04:16 UTC - Thor Sigurjonsson: Thanks. Found the issue has been reported: <https://github.com/apache/pulsar/issues/4727>
----
2019-08-28 19:01:57 UTC - Rajiv Abraham: @Sijie Guo I just wanted to confirm that when you say projection function above, you mean a Pulsar Function and not any debezium(or Kafka Connect) specific projection setting?
----
2019-08-28 19:03:54 UTC - Sijie Guo: @Rajiv Abraham I meant a pulsar function. although I am not sure if debezium supports projects or not, if debezium already supports that, you can just enable it through debezium settings. If debezium doesn’t support projection, a Pulsar function can be introduced to project the columns.
----
2019-08-28 19:06:22 UTC - Rajiv Abraham: @Sijie Guo Thank you.
----
2019-08-28 19:57:46 UTC - Rajiv Abraham: Hi, I was wondering if there is a Pulsar Roadmap page? In particular for now, I was wondering if there was any plans to support Python project archives instead of the current single file constraint(or PEX files) and if you guys had any tentative time plan?
----
2019-08-28 20:01:57 UTC - Poule: With Avro and `pulsar-admin functions trigger --trigger-file `, what is the content format i need to put in the trigger-file ? I try JSON, JSON as string..
----
2019-08-28 21:09:11 UTC - Jerry Peng: @Alexandre DUVAL support for auth in Pulsar SQL was just added and not in an official release. Feel free to compile the latest code and try it out
----
2019-08-28 21:09:39 UTC - Jerry Peng: <https://github.com/apache/pulsar/blob/master/conf/presto/catalog/pulsar.properties#L59>
----
2019-08-28 21:10:04 UTC - Jerry Peng: There is no support for multitenancy right now but have plans to add it in the future
----
2019-08-28 21:10:44 UTC - Jerry Peng: You can also submit python wheel file or a tar.gz package for running Python Pulsar Functions
----
2019-08-28 21:19:53 UTC - Alexandre DUVAL: Cool! Thanks a lot.
----
2019-08-28 21:38:20 UTC - Matteo Merli: <https://pulsar.apache.org/docs/en/functions-quickstart/#package-python-dependencies>
----
2019-08-28 22:07:16 UTC - Rajiv Abraham: @Jerry Peng @Matteo Merli Thank you very much.
----
2019-08-28 22:52:29 UTC - derek: @derek has joined the channel
----
2019-08-29 00:46:38 UTC - xue: @Sijie Guo I know it can be deleted through the option, but I did not delete it after execution.
----
2019-08-29 01:56:43 UTC - Sijie Guo: ah I see you might be hitting a bug in 2.4.0. there is a bug fix coming up in 2.4.1
----
2019-08-29 02:00:10 UTC - xue: @Sijie Guo yes, pulsar 2.4.0
----
2019-08-29 02:02:08 UTC - Sijie Guo: <https://github.com/apache/pulsar/issues/4977>
----
2019-08-29 02:02:36 UTC - Sijie Guo: The fix was <https://github.com/apache/pulsar/pull/4669> and will be released in 2.4.1
----
2019-08-29 02:03:29 UTC - xue: ok,thank you.
----
2019-08-29 07:44:57 UTC - Diego Salvi: Hi @Sijie Guo, I've got some informations from admin api:
----
2019-08-29 07:45:24 UTC - Diego Salvi: \/admin/v2/persistent/idc4352/ns/events/stats
{
"msgRateIn":0.0,
"msgThroughputIn":0.0,
"msgRateOut":0.0,
"msgThroughputOut":0.0,
"averageMsgSize":0.0,
"storageSize":0,
"publishers":[
],
"subscriptions":{
},
"replication":{
},
"deduplicationStatus":"Disabled"
}
----
2019-08-29 07:45:31 UTC - Diego Salvi: \/admin/v2/persistent/idc4352/ns/events/internalStats
{
"entriesAddedCounter":0,
"numberOfEntries":0,
"totalSize":0,
"currentLedgerEntries":0,
"currentLedgerSize":0,
"lastLedgerCreatedTimestamp":"2019-08-28T13:02:17.668+02:00",
"waitingCursorsCount":5,
"pendingAddEntriesCount":0,
"lastConfirmedEntry":"3207702:-1",
"state":"Fenced",
"ledgers":[
{
"ledgerId":3207702,
"entries":0,
"size":0,
"offloaded":false
}
],
"cursors":{
}
}
----
2019-08-29 07:45:39 UTC - Diego Salvi: \/admin/v2/persistent/idc4352/ns/events/internal-info
{
"version":1,
"creationDate":"2019-08-28T13:02:17.411+02:00",
"modificationDate":"2019-08-28T13:02:17.687+02:00",
"ledgers":[
{
"ledgerId":3207702
}
],
"cursors":{
}
}
----
2019-08-29 07:45:51 UTC - Diego Salvi: \/admin/v2/persistent/idc4352/ns
["<persistent://idc4352/ns/api>","<persistent://idc4352/ns/consent>","<persistent://idc4352/ns/events>"]
----
2019-08-29 07:46:40 UTC - Diego Salvi: I'm looking at the "state":"Fenced"
----
2019-08-29 07:46:55 UTC - Diego Salvi: how could be recovered?
----
2019-08-29 08:32:43 UTC - xue: Will the newly added bookie node transfer historical messages from other nodes to the newly added node? Or will only new messages be routed to the newly added bookies node?
----
2019-08-29 08:43:23 UTC - Richard Sherman: Only new messages will end up there. Bookkeeper does not do rebalancing when adding nodes, unlike other systems. This means adding a new bookie doesn't have a performance hit due to the impact of rebalancing.
But if one of the other bookies goes down then as part of recovering from underreplication of existing data some of the historic data would then be copied to the new node.
----
2019-08-29 09:05:22 UTC - geal: @geal has joined the channel
----
2019-08-29 09:08:50 UTC - geal: hi all! I’m working at Clever Cloud with @Alexandre DUVAL and I’ve been improving the Rust client: <https://github.com/wyyerd/pulsar-rs>
----
2019-08-29 09:09:19 UTC - geal: I’m wondering if it could become an official client at some point. What would be required for that?
----
2019-08-29 09:10:32 UTC - Ali Ahmed: The code would have to be licensed under the Apache license
----
2019-08-29 09:10:50 UTC - Ali Ahmed: A repo under the apache org can be created
----
2019-08-29 09:10:55 UTC - geal: right now it’s dual licensed in APache and MIT
----