Zhen-hao opened a new issue #11631:
URL: https://github.com/apache/pulsar/issues/11631
I see a runtime failure of a Pulsar producer with the log below. To make this issue easier to understand, I attach here a graphic view of the difference between the two schemas. The original schema is a union type and the new version adds a few record types to the union type.
Although the newly added types occur in two chunks in the schema surrounded by the old ones, the new version should still be backward compatible with the old version. Because it is straightforward to translate data serialized with the old version into data for the new version.
The error message is not very helpful because it doesn't mention if it fails the forward or the backward compatibility check.
Now, I don't know if there is a bug in the backward compatibility check or it is because both backward and forward checks are enabled by default.
Error log:
```
Aug 10 22:57:59 bookkeeper2 nt-ui[84774]: [production/projection_nt_ui_akka_journal/User_SellerProfileUpdated] [pairtime-prd-48-4] GetOrCreateSchema error
Aug 10 22:57:59 bookkeeper2 nt-ui[84774]: [nt-ui-user] Failed to process envelope with offset [TimeBasedUUID(6ebc77f0-f9ee-11eb-9687-21cd63c940db)]. Projection will stop as defined by recovery strategy.
Aug 10 22:57:59 bookkeeper2 nt-ui[84774]: Restarting graph due to failure. stack_trace:
org.apache.pulsar.client.api.PulsarClientException$IncompatibleSchemaException: org.apache.pulsar.broker.service.schema.exceptions.IncompatibleSchemaException: org.apache.avro.SchemaValidationException: Unable to read schema:
{
"type" : "record",
"name" : "SellerProfileUpdated",
"namespace" : "com.nt.back.user.Event",
"fields" : [ {
"name" : "update",
"type" : [ {
"type" : "record",
"name" : "DefaultCurrencyUpdate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : [ "null", {
"type" : "record",
"name" : "CurrencyUnitRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "code",
"type" : "string"
} ]
} ]
} ]
}, {
"type" : "record",
"name" : "DisableAllowDirectMessaging",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "DisableFilterRequestsWithAvailability",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "DisableFilterRequestsWithDuration",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "DisableFilterRequestsWithHourlyRates",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "DisableSearchable",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "DurationPreferenceUpdate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "durationPreference",
"type" : {
"type" : "record",
"name" : "DurationPreference",
"namespace" : "com.nt.core.Meeting",
"doc" : "custom schema",
"fields" : [ {
"name" : "minDurationInMinutes",
"type" : [ "null", "int" ]
}, {
"name" : "maxDurationInMinutes",
"type" : [ "null", "int" ]
} ]
}
} ]
}, {
"type" : "record",
"name" : "EnableAllowDirectMessaging",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "EnableFilterRequestsWithAvailability",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "EnableFilterRequestsWithDuration",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "EnableFilterRequestsWithHourlyRates",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "EnableSearchable",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "NewHourlyRate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : "com.nt.util.serialization.AvroTypeMappings.CurrencyUnitRep"
}, {
"name" : "amount",
"type" : {
"type" : "record",
"name" : "JodaMoneyRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "currency",
"type" : "string"
}, {
"name" : "amount",
"type" : {
"type" : "bytes",
"logicalType" : "decimal",
"precision" : 8,
"scale" : 2
}
} ]
}
} ]
}, {
"type" : "record",
"name" : "NewInterval",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "interval",
"type" : {
"type" : "record",
"name" : "Interval",
"namespace" : "com.nt.core.Time",
"doc" : "custom schema",
"fields" : [ {
"name" : "start",
"type" : "int"
}, {
"name" : "end",
"type" : "int"
}, {
"name" : "recuring",
"type" : [ "null", {
"type" : "record",
"name" : "RecurringRules",
"fields" : [ {
"name" : "zoneId",
"type" : {
"type" : "record",
"name" : "ZoneIdRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "zoneId",
"type" : "string"
} ]
}
}, {
"name" : "pattern",
"type" : {
"type" : "enum",
"name" : "RecurringPattern",
"symbols" : [ "EveryDay", "EveryMonth", "EveryWeek", "EveryWorkDay" ]
}
}, {
"name" : "end",
"type" : [ "null", "int" ]
}, {
"name" : "exceptions",
"type" : {
"type" : "array",
"items" : "int"
},
"default" : [ ]
} ]
} ],
"default" : null
} ]
}
} ]
}, {
"type" : "record",
"name" : "NewTag",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "name",
"type" : "string"
}, {
"name" : "modelTagId",
"type" : [ "null", "string" ]
} ]
}, {
"type" : "record",
"name" : "RmoveHourlyRate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : "com.nt.util.serialization.AvroTypeMappings.CurrencyUnitRep"
} ]
}, {
"type" : "record",
"name" : "RmoveInterval",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "interval",
"type" : "com.nt.core.Time.Interval"
} ]
}, {
"type" : "record",
"name" : "RmoveTag",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "name",
"type" : "string"
} ]
} ]
}, {
"name" : "timestamp",
"type" : "long"
} ]
}
using schema:
{
"type" : "record",
"name" : "SellerProfileUpdated",
"namespace" : "com.nt.back.user.Event",
"fields" : [ {
"name" : "update",
"type" : [ {
"type" : "record",
"name" : "DefaultCurrencyUpdate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : [ "null", {
"type" : "record",
"name" : "CurrencyUnitRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "code",
"type" : "string"
} ]
} ]
} ]
}, {
"type" : "record",
"name" : "DurationPreferenceUpdate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "durationPreference",
"type" : {
"type" : "record",
"name" : "DurationPreference",
"namespace" : "com.nt.core.Meeting",
"doc" : "custom schema",
"fields" : [ {
"name" : "minDurationInMinutes",
"type" : [ "null", "int" ]
}, {
"name" : "maxDurationInMinutes",
"type" : [ "null", "int" ]
} ]
}
} ]
}, {
"type" : "record",
"name" : "NewHourlyRate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : "com.nt.util.serialization.AvroTypeMappings.CurrencyUnitRep"
}, {
"name" : "amount",
"type" : {
"type" : "record",
"name" : "JodaMoneyRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "currency",
"type" : "string"
}, {
"name" : "amount",
"type" : {
"type" : "bytes",
"logicalType" : "decimal",
"precision" : 8,
"scale" : 2
}
} ]
}
} ]
}, {
"type" : "record",
"name" : "NewInterval",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "interval",
"type" : {
"type" : "record",
"name" : "Interval",
"namespace" : "com.nt.core.Time",
"doc" : "custom schema",
"fields" : [ {
"name" : "start",
"type" : "int"
}, {
"name" : "end",
"type" : "int"
}, {
"name" : "recuring",
"type" : [ "null", {
"type" : "record",
"name" : "RecurringRules",
"fields" : [ {
"name" : "zoneId",
"type" : {
"type" : "record",
"name" : "ZoneIdRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "zoneId",
"type" : "string"
} ]
}
}, {
"name" : "pattern",
"type" : {
"type" : "enum",
"name" : "RecurringPattern",
"symbols" : [ "EveryDay", "EveryMonth", "EveryWeek", "EveryWorkDay" ]
}
}, {
"name" : "end",
"type" : [ "null", "int" ]
}, {
"name" : "exceptions",
"type" : {
"type" : "array",
"items" : "int"
},
"default" : [ ]
} ]
} ],
"default" : null
} ]
}
} ]
}, {
"type" : "record",
"name" : "NewTag",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "name",
"type" : "string"
}, {
"name" : "modelTagId",
"type" : [ "null", "string" ]
} ]
}, {
"type" : "record",
"name" : "RmoveHourlyRate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : "com.nt.util.serialization.AvroTypeMappings.CurrencyUnitRep"
} ]
}, {
"type" : "record",
"name" : "RmoveInterval",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "interval",
"type" : "com.nt.core.Time.Interval"
} ]
}, {
"type" : "record",
"name" : "RmoveTag",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "name",
"type" : "string"
} ]
} ]
}, {
"name" : "timestamp",
"type" : "long"
} ]
} caused by org.apache.pulsar.broker.service.schema.exceptions.IncompatibleSchemaException: org.apache.avro.SchemaValidationException: Unable to read schema:
{
"type" : "record",
"name" : "SellerProfileUpdated",
"namespace" : "com.nt.back.user.Event",
"fields" : [ {
"name" : "update",
"type" : [ {
"type" : "record",
"name" : "DefaultCurrencyUpdate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : [ "null", {
"type" : "record",
"name" : "CurrencyUnitRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "code",
"type" : "string"
} ]
} ]
} ]
}, {
"type" : "record",
"name" : "DisableAllowDirectMessaging",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "DisableFilterRequestsWithAvailability",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "DisableFilterRequestsWithDuration",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "DisableFilterRequestsWithHourlyRates",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "DisableSearchable",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "DurationPreferenceUpdate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "durationPreference",
"type" : {
"type" : "record",
"name" : "DurationPreference",
"namespace" : "com.nt.core.Meeting",
"doc" : "custom schema",
"fields" : [ {
"name" : "minDurationInMinutes",
"type" : [ "null", "int" ]
}, {
"name" : "maxDurationInMinutes",
"type" : [ "null", "int" ]
} ]
}
} ]
}, {
"type" : "record",
"name" : "EnableAllowDirectMessaging",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "EnableFilterRequestsWithAvailability",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "EnableFilterRequestsWithDuration",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "EnableFilterRequestsWithHourlyRates",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "EnableSearchable",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ ]
}, {
"type" : "record",
"name" : "NewHourlyRate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : "com.nt.util.serialization.AvroTypeMappings.CurrencyUnitRep"
}, {
"name" : "amount",
"type" : {
"type" : "record",
"name" : "JodaMoneyRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "currency",
"type" : "string"
}, {
"name" : "amount",
"type" : {
"type" : "bytes",
"logicalType" : "decimal",
"precision" : 8,
"scale" : 2
}
} ]
}
} ]
}, {
"type" : "record",
"name" : "NewInterval",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "interval",
"type" : {
"type" : "record",
"name" : "Interval",
"namespace" : "com.nt.core.Time",
"doc" : "custom schema",
"fields" : [ {
"name" : "start",
"type" : "int"
}, {
"name" : "end",
"type" : "int"
}, {
"name" : "recuring",
"type" : [ "null", {
"type" : "record",
"name" : "RecurringRules",
"fields" : [ {
"name" : "zoneId",
"type" : {
"type" : "record",
"name" : "ZoneIdRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "zoneId",
"type" : "string"
} ]
}
}, {
"name" : "pattern",
"type" : {
"type" : "enum",
"name" : "RecurringPattern",
"symbols" : [ "EveryDay", "EveryMonth", "EveryWeek", "EveryWorkDay" ]
}
}, {
"name" : "end",
"type" : [ "null", "int" ]
}, {
"name" : "exceptions",
"type" : {
"type" : "array",
"items" : "int"
},
"default" : [ ]
} ]
} ],
"default" : null
} ]
}
} ]
}, {
"type" : "record",
"name" : "NewTag",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "name",
"type" : "string"
}, {
"name" : "modelTagId",
"type" : [ "null", "string" ]
} ]
}, {
"type" : "record",
"name" : "RmoveHourlyRate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : "com.nt.util.serialization.AvroTypeMappings.CurrencyUnitRep"
} ]
}, {
"type" : "record",
"name" : "RmoveInterval",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "interval",
"type" : "com.nt.core.Time.Interval"
} ]
}, {
"type" : "record",
"name" : "RmoveTag",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "name",
"type" : "string"
} ]
} ]
}, {
"name" : "timestamp",
"type" : "long"
} ]
}
using schema:
{
"type" : "record",
"name" : "SellerProfileUpdated",
"namespace" : "com.nt.back.user.Event",
"fields" : [ {
"name" : "update",
"type" : [ {
"type" : "record",
"name" : "DefaultCurrencyUpdate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : [ "null", {
"type" : "record",
"name" : "CurrencyUnitRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "code",
"type" : "string"
} ]
} ]
} ]
}, {
"type" : "record",
"name" : "DurationPreferenceUpdate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "durationPreference",
"type" : {
"type" : "record",
"name" : "DurationPreference",
"namespace" : "com.nt.core.Meeting",
"doc" : "custom schema",
"fields" : [ {
"name" : "minDurationInMinutes",
"type" : [ "null", "int" ]
}, {
"name" : "maxDurationInMinutes",
"type" : [ "null", "int" ]
} ]
}
} ]
}, {
"type" : "record",
"name" : "NewHourlyRate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : "com.nt.util.serialization.AvroTypeMappings.CurrencyUnitRep"
}, {
"name" : "amount",
"type" : {
"type" : "record",
"name" : "JodaMoneyRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "currency",
"type" : "string"
}, {
"name" : "amount",
"type" : {
"type" : "bytes",
"logicalType" : "decimal",
"precision" : 8,
"scale" : 2
}
} ]
}
} ]
}, {
"type" : "record",
"name" : "NewInterval",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "interval",
"type" : {
"type" : "record",
"name" : "Interval",
"namespace" : "com.nt.core.Time",
"doc" : "custom schema",
"fields" : [ {
"name" : "start",
"type" : "int"
}, {
"name" : "end",
"type" : "int"
}, {
"name" : "recuring",
"type" : [ "null", {
"type" : "record",
"name" : "RecurringRules",
"fields" : [ {
"name" : "zoneId",
"type" : {
"type" : "record",
"name" : "ZoneIdRep",
"namespace" : "com.nt.util.serialization.AvroTypeMappings",
"fields" : [ {
"name" : "zoneId",
"type" : "string"
} ]
}
}, {
"name" : "pattern",
"type" : {
"type" : "enum",
"name" : "RecurringPattern",
"symbols" : [ "EveryDay", "EveryMonth", "EveryWeek", "EveryWorkDay" ]
}
}, {
"name" : "end",
"type" : [ "null", "int" ]
}, {
"name" : "exceptions",
"type" : {
"type" : "array",
"items" : "int"
},
"default" : [ ]
} ]
} ],
"default" : null
} ]
}
} ]
}, {
"type" : "record",
"name" : "NewTag",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "name",
"type" : "string"
}, {
"name" : "modelTagId",
"type" : [ "null", "string" ]
} ]
}, {
"type" : "record",
"name" : "RmoveHourlyRate",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "currency",
"type" : "com.nt.util.serialization.AvroTypeMappings.CurrencyUnitRep"
} ]
}, {
"type" : "record",
"name" : "RmoveInterval",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "interval",
"type" : "com.nt.core.Time.Interval"
} ]
}, {
"type" : "record",
"name" : "RmoveTag",
"namespace" : "com.nt.core.SellerProfileUpdate",
"fields" : [ {
"name" : "name",
"type" : "string"
} ]
} ]
}, {
"name" : "timestamp",
"type" : "long"
} ]
}
at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:967)
at org.apache.pulsar.client.impl.TypedMessageBuilderImpl.send(TypedMessageBuilderImpl.java:91)
at com.nt.projection.PublishEventsProjectionHandler.$anonfun$process$1(PublishEventsProjectionHandler.scala:90)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:672)
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:431)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63)
at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Zhen-hao commented on issue #11631: URL: https://github.com/apache/pulsar/issues/11631#issuecomment-896577213 found the solution I need. https://pulsar.apache.org/docs/en/schema-manage/#adjust-compatibility I will make a PR to make the error message a bit easier to read when I find the time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
Zhen-hao commented on issue #11631: URL: https://github.com/apache/pulsar/issues/11631#issuecomment-896521118 OK. I think I know the real issue here. [The Pulsar documentation](https://pulsar.apache.org/docs/en/schema-evolution-compatibility/) makes it clear ``` For Avro and JSON, the default schema compatibility check strategy is FULL. ``` My problem is to find a way to set the default compatibility level to `BACKWARD` on the namespace level. Because I need to create a lot of topics dynamically. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
Zhen-hao closed issue #11631: URL: https://github.com/apache/pulsar/issues/11631 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org