You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/04/11 09:11:40 UTC
[GitHub] [pulsar] cbornet opened a new issue, #15118: Cache key and value schema versions in KeyValueSchemaImpl
cbornet opened a new issue, #15118:
URL: https://github.com/apache/pulsar/issues/15118
**Is your enhancement request related to a problem? Please describe.**
While doing performance profiling, @eolivelli found that `KeyValue::decode` is called twice each time `KeyValueSchemaImpl::atSchemaVersion` is called (once for the key, the other time for the value).
This can be for each message you receive from an AUTO_CONSUME consumer for instance.
Eg. in the following code the decoding happens for each call to `getReaderSchema`
```java
@Test
public void test() throws Exception {
String topic = "my-property/my-ns/schema-test" + UUID.randomUUID();
Schema<KeyValue<V1Data, V1Data>> pojoSchema = Schema.KeyValue(
Schema.AVRO(V1Data.class),
Schema.AVRO(V1Data.class),
KeyValueEncodingType.INLINE);
try (Producer<KeyValue<V1Data, V1Data>> p = pulsarClient.newProducer(pojoSchema)
.topic(topic)
.create();
Consumer<GenericRecord> c0 = pulsarClient.newConsumer(Schema.AUTO_CONSUME())
.topic(topic)
.subscriptionName("sub0")
.subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
.subscribe();
) {
p.send(new KeyValue<>(new V1Data(1), new V1Data(2)));
p.send(new KeyValue<>(new V1Data(1), new V1Data(2)));
c0.receive().getReaderSchema();
c0.receive().getReaderSchema();
}
}
```
So it would be beneficial to cache the result of [the key and value atSchemaVersion calls](https://github.com/apache/pulsar/blob/18f2f4a9c1dab7eec5c7c9590b76aca17ee1ead8/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/KeyValueSchemaImpl.java#L285) to spare some CPU cycles.
**Describe the solution you'd like**
Add `ConcurrentMap<SchemaVersion, Schema<?>> keySchemaMap` and `ConcurrentMap<SchemaVersion, Schema<?>> valueSchemaMap` in `KeyValueSchemaImpl` and get/put from/to cache in `KeyValueSchemaImpl::atSchemaVersion`
**Describe alternatives you've considered**
none
**Additional context**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari closed issue #15118: Cache key and value schema versions in KeyValueSchemaImpl
Posted by GitBox <gi...@apache.org>.
lhotari closed issue #15118: Cache key and value schema versions in KeyValueSchemaImpl
URL: https://github.com/apache/pulsar/issues/15118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] cbornet commented on issue #15118: Cache key and value schema versions in KeyValueSchemaImpl
Posted by GitBox <gi...@apache.org>.
cbornet commented on issue #15118:
URL: https://github.com/apache/pulsar/issues/15118#issuecomment-1094779635
I'm working on a PR for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org