You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/05 07:33:41 UTC
[GitHub] [pulsar] objecttrouve opened a new issue #11562: Can't configure a `cryptoKeyReader` via `ReaderBuilder.loadConf`
objecttrouve opened a new issue #11562:
URL: https://github.com/apache/pulsar/issues/11562
### Describe the bug
I'm trying to configure a `CryptoKeyReader` with a [`FlinkPulsarSource`](https://github.com/streamnative/pulsar-flink#source).
The [Pulsar docs](https://pulsar.apache.org/docs/en/client-libraries-java/#configure-reader) suggest you can configure such a crypto key reader by passing a value for property called `cryptoKeyReader`.
[Here's](https://github.com/streamnative/pulsar-flink/blob/a38b8095d3bd96470af07f4fd20d830983c0a07c/pulsar-flink-connector/src/main/java/org/apache/flink/streaming/connectors/pulsar/internal/ReaderThread.java#L126) where those properties are used (on creating the Flink Pulsar source).
The evaluating code tries to _deserialize_ a config object from the property map.
But what would be the value in such a map? It's _not_ the class name, as I naively assumed.
If I understand the code correctly, the value doesn't even matter, because the object mapper tries to instantiate an _interface_. Resulting in the following error:
```com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.apache.pulsar.client.api.CryptoKeyReader` (no Creators, like default constructor, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information```
[I also asked about this in the `flink-connector` channel on Slack.](https://apache-pulsar.slack.com/archives/CA678TN9G/p1627910055002100?thread_ts=1627909730.002000&cid=CA678TN9G) But unless I'm missing something, I think it's a bug in Pulsar. Or at least a misleading documentation. Raising this issue to clarify.
(If the calling code was under my control, I could, of course, just use the [setter for the crypto key reader](https://github.com/apache/pulsar/blob/ce9a6d2392fdf9f3b50afa37f4d871d5360362ca/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ReaderBuilderImpl.java#L153). I might do a PR about it here or in the [`pulsar-flink` project](https://github.com/streamnative/pulsar-flink). However, I want to clarify the intended usage first.)
### To Reproduce
[Here](https://github.com/objecttrouve/pulsar/blob/90f35be79735a9a53add4093d0a403e326761403/pulsar-client/src/test/java/org/apache/pulsar/client/impl/BuildersTest.java#L121)'s a unit test illustrating what I was trying to do:
```
@SuppressWarnings("unused")
public static class MyCryptoKeyReader implements CryptoKeyReader {
@Override
public EncryptionKeyInfo getPublicKey(String keyName, Map<String, String> metadata) {
return null;
}
@Override
public EncryptionKeyInfo getPrivateKey(String keyName, Map<String, String> metadata) {
return null;
}
}
@Ignore("Illustrates wrong usage.")
@Test
public void readerBuilderLoadConfWithCryptoKeyReaderClassValueTest() throws Exception {
@Cleanup
PulsarClient client = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
Map<String, Object> config = new HashMap<>();
/*
* This is what I first assumed.
* This ISN'T how it works!
* The `loadConf` method expects a _serialized_ `CryptoKeyReader` at this point.
*/
config.put("cryptoKeyReader", "org.apache.pulsar.client.impl.BuildersTest.MyCryptoKeyReader");
ReaderBuilderImpl<byte[]> builder = (ReaderBuilderImpl<byte[]>) client.newReader()
.loadConf(config);
Class<?> clazz = builder.getClass();
Field conf = clazz.getDeclaredField("conf");
conf.setAccessible(true);
Object obj = conf.get(builder);
assertTrue(obj instanceof ReaderConfigurationData);
//noinspection rawtypes
assertTrue(((ReaderConfigurationData)obj).getCryptoKeyReader() instanceof MyCryptoKeyReader);
}
```
It reproduces the error mentioned above.
### Expected behavior
I inspected a property where I could pass a class name of a `CryptoKeyReader` implementation with a no-arg constructor, which would be instantiated on loading the config.
### Desktop (please complete the following information):
```
uname -a
Linux otto 5.4.0-80-generic #90-Ubuntu SMP Fri Jul 9 22:49:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
```
### Additional context
[Same question in the Pulsar Slack.](https://apache-pulsar.slack.com/archives/CA678TN9G/p1627910055002100?thread_ts=1627909730.002000&cid=CA678TN9G)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #11562: Can't configure a `cryptoKeyReader` via `ReaderBuilder.loadConf`
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #11562:
URL: https://github.com/apache/pulsar/issues/11562#issuecomment-1058887349
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] objecttrouve commented on issue #11562: Can't configure a `cryptoKeyReader` via `ReaderBuilder.loadConf`
Posted by GitBox <gi...@apache.org>.
objecttrouve commented on issue #11562:
URL: https://github.com/apache/pulsar/issues/11562#issuecomment-894782036
Created [#386](https://github.com/streamnative/pulsar-flink/issues/386) in the [streamnative/pulsar-flink](https://github.com/streamnative/pulsar-flink) project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org