You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/08/24 09:11:03 UTC
Slack digest for #general - 2019-08-24
2019-08-23 09:19:58 UTC - Vinay Aggarwal: @Bon: I read the entire documentation again on geo replication and still not able to figure why replication is not working:
Namespace policies :
./bin/pulsar-admin namespaces policies tenant-poc/ns1
{
"auth_policies" : {
"namespace_auth" : { },
"destination_auth" : { },
"subscription_auth_roles" : { }
},
"replication_clusters" : [ "us-west", "us-central" ],
"bundles" : {
"boundaries" : [ "0x00000000", "0x40000000", "0x80000000", "0xc0000000", "0xffffffff" ],
"numBundles" : 4
},
"backlog_quota_map" : {
"destination_storage" : {
"limit" : -1073741824,
"policy" : "producer_request_hold"
}
},
"topicDispatchRate" : {
"us-west" : {
"dispatchThrottlingRateInMsg" : 0,
"dispatchThrottlingRateInByte" : 0,
"ratePeriodInSecond" : 1
}
},
"subscriptionDispatchRate" : {
"us-west" : {
"dispatchThrottlingRateInMsg" : 0,
"dispatchThrottlingRateInByte" : 0,
"ratePeriodInSecond" : 1
}
},
"replicatorDispatchRate" : { },
"clusterSubscribeRate" : {
"us-west" : {
"subscribeThrottlingRatePerConsumer" : 0,
"ratePeriodInSecond" : 30
}
},
"latency_stats_sample_rate" : { },
"message_ttl_in_seconds" : 0,
"deleted" : false,
"encryption_required" : false,
"subscription_auth_mode" : "None",
"max_producers_per_topic" : 0,
"max_consumers_per_topic" : 0,
"max_consumers_per_subscription" : 0,
"compaction_threshold" : 0,
"offload_threshold" : -1,
"schema_auto_update_compatibility_strategy" : "Full",
"schema_validation_enforced" : false
}
and still all messages are in replication backlog
bin/pulsar-admin persistent stats <persistent://tenant-poc/ns1/default>
{
"msgRateIn" : 0.0,
"msgThroughputIn" : 0.0,
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"averageMsgSize" : 0.0,
"storageSize" : 476,
"publishers" : [ {
"msgRateIn" : 0.0,
"msgThroughputIn" : 0.0,
"averageMsgSize" : 0.0,
"producerId" : 0,
"metadata" : { },
"address" : "/##########:48684",
"producerName" : "us-west-0-8",
"connectedSince" : "2019-08-23T08:42:19.91Z",
"clientVersion" : "2.4.0"
} ],
"subscriptions" : {
"defaultSubscription" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"msgRateRedeliver" : 0.0,
"msgBacklog" : 3,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"unackedMessages" : 0,
"type" : "Exclusive",
"activeConsumerName" : "29838",
"msgRateExpired" : 0.0,
"consumers" : [ {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"msgRateRedeliver" : 0.0,
"consumerName" : "29838",
"availablePermits" : 997,
"unackedMessages" : 0,
"blockedConsumerOnUnackedMsgs" : false,
"metadata" : { },
"address" : "/##########:48688",
"connectedSince" : "2019-08-23T08:42:20.011Z",
"clientVersion" : "2.4.0"
} ],
"isReplicated" : false
}
},
"replication" : {
"us-central" : {
"msgRateIn" : 0.0,
"msgThroughputIn" : 0.0,
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"msgRateExpired" : 0.0,
"replicationBacklog" : 4,
"connected" : false,
"replicationDelayInSeconds" : 0
}
},
"deduplicationStatus" : "Disabled"
}
----
2019-08-23 09:52:56 UTC - Walter: I am seeing different values between pulsar dashboard home tenants and tenants namespace screen. For example i have an one tenant and one namespace under same tenant. But the value showing tenant home screen is 114 namespaces and 826 subscription. In namespace screen it's showing correct value. I have added the screen shots.
----
2019-08-23 09:54:39 UTC - sunxiaojian926: @sunxiaojian926 has joined the channel
----
2019-08-23 10:00:36 UTC - Richard Sherman: Yes that is a bug. @Sijie Guo I should have time to look at fixing this and raising a PR next week
+1 : jia zhai
----
2019-08-23 10:12:07 UTC - xue: I did the Geo Replication test, built two local clusters, modified global_zookeeper.conf, added the observer role, according to the operation steps on the deployment document, started the first cluster when starting the broker, it can start the broker successfully, but then start the broker of the second cluster when reporting the error. What is the reason?
----
2019-08-23 11:37:07 UTC - dba: Hi guys. We decided to create an early alpha release.
The source is here: <https://github.com/danske-commodities/dotpulsar>
and we have a NuGet package here: <https://www.nuget.org/packages/DotPulsar/>
Currently we depend on .NET Core 2.2, but we will move to .NET Standard 2.1 when it is released.
Enjoy :slightly_smiling_face:
+1 : Karthik Ramasamy, Ali Ahmed, Jerry Peng, Sinan Bir
tada : Matteo Merli, Sinan Bir
----
2019-08-23 11:38:36 UTC - dba: Hey you two. We have published the code and a nuget package. Enjoy :slightly_smiling_face:
----
2019-08-23 11:41:36 UTC - Jan-Pieter George: Hi @dba, awesome, thanks! Will try and block some time to play.
+1 : dba
----
2019-08-23 12:12:43 UTC - Chris Bartholomew: @xue When you update the list of clusters on the namespace, you need to include the local cluster as well as the remote cluster name. The namespace needs to list all the clusters involved in the geo-replication. Based the log message, I think you can fix this using the command ```pulsar-admin namespaces set-clusters -c pulsar-cluster,pulsar-cluster2 public/functions```
----
2019-08-23 14:42:15 UTC - Eugene: @Eugene has joined the channel
----
2019-08-23 16:17:25 UTC - Addison Higham: updated the link, <https://docs.google.com/document/d/18HPgFN8LOsxSBIScrldKWTmJvFkUvRemMoit1KQjcik/edit?usp=sharing>, should work now...
----
2019-08-23 16:17:36 UTC - Matteo Merli: :+1:
----
2019-08-23 16:21:54 UTC - Addison Higham: also, I think I call it out there, but I am sure that punts on a few things that interface would need, ACLs seem like they might be the most tricky, but IDK how complex the usage of ACLs are in pulsar
----
2019-08-24 02:36:40 UTC - Igor Zubchenok: ping
----
2019-08-24 05:58:18 UTC - caohao: @caohao has joined the channel
----