You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/10/10 22:08:13 UTC
[GitHub] [pulsar] harissecic opened a new issue #8233: Custom ObjectMapper for JSON schema writer/reader for Java client?
harissecic opened a new issue #8233:
URL: https://github.com/apache/pulsar/issues/8233
**Describe the solution you'd like**
I'm trying to create module/plugin for Micronaut. I've gone far enough and it might be ready soon as snapshot/release candidate. However Micronauts philosophy is no reflection at all or as less as possible when combining with libraries that use such thing. I've seen that majority of the code actually uses reflection and ObjectMapper is not original one from jackons but rather your own wrapped version. Could this be modified to support it. So far I've managed to schema that skips using Construction<T> class but in the end it will use your own ObjectMapper which is probably also relying on reflection while Micronaut has modifications that eliminate this and if your library could easily be accept type of com.fasterxml.jackson.databind instead of current one.
This could also be considered for all Schemas (to expose a bit more simple approach).
**Describe alternatives you've considered**
I could write my own readers and writters and schema in total but it will take enormous amount of time until I get there and I'm not sure how would Pulsar fit with such modifications as I don't know how will it influence the schema stored at brookers. I would love to have all schemas customised for Micronaut but I don't have enough deep knowledge about Pulsar like what do topics rely on when it comes to schema and how does it process internally this stuff.
**Additional context**
Using latest stable library at this point which is 2.6.1.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #8233: Custom ObjectMapper for JSON schema writer/reader for Java client?
Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #8233:
URL: https://github.com/apache/pulsar/issues/8233#issuecomment-786134188
> Sorry it took some time but unshaded lib is colliding because of some netty loop. So it cannot be used with this approach. Not sure should I close this issue as it will not be resolved or to wait maybe some good idea comes along.
@harissecic The Netty conflict might be addressable if the other code you have is compatible with Netty 4.1.x. In that case, the Netty conflict might be caused by "misaligned" versions which can be resolved by importing the netty-bom input Maven's dependency management (or with Gradle's support for BOM).
```
<properties>
<netty.version>4.1.51.Final</netty.version
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-bom</artifactId>
<version>${netty.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
...
</dependencies>
</dependencyManagement>
```
You can ask for more help for such usage problems on [Pulsar Slack](https://pulsar.apache.org/en/contact/).
I hope this helps.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] harissecic commented on issue #8233: Custom ObjectMapper for JSON schema writer/reader for Java client?
Posted by GitBox <gi...@apache.org>.
harissecic commented on issue #8233:
URL: https://github.com/apache/pulsar/issues/8233#issuecomment-786148412
> > Sorry it took some time but unshaded lib is colliding because of some netty loop. So it cannot be used with this approach. Not sure should I close this issue as it will not be resolved or to wait maybe some good idea comes along.
>
> @harissecic The Netty conflict might be addressable if the other code you have is compatible with Netty 4.1.x. In that case, the Netty conflict might be caused by "misaligned" versions which can be resolved by importing the netty-bom input Maven's dependency management (or with Gradle's support for BOM).
>
> ```
> <properties>
> <netty.version>4.1.51.Final</netty.version>
> <!-- ... more ... -->
> </properties>
> <dependencyManagement>
> <dependencies>
> <dependency>
> <groupId>io.netty</groupId>
> <artifactId>netty-bom</artifactId>
> <version>${netty.version}</version>
> <type>pom</type>
> <scope>import</scope>
> </dependency>
> <!-- ... more ... -->
> </dependencies>
> </dependencyManagement>
> ```
>
> You can ask for more help for such usage problems on [Pulsar Slack](https://pulsar.apache.org/en/contact/).
>
> I hope this helps.
Thanks a lot. It does help. I need to recheck this with Micronaut team and how could I adapt this to their rules. If this is not possible to change then I stick with what is given by Pulsar. If it's possible to change which I doubt then I can adapt this. Anyways anyone having any quality input how to do some advance stuff can suggest here https://github.com/micronaut-projects/micronaut-pulsar/. I'm pushing soon Java example with OAuth2 connecting to pulsar so hope I get some quality check on that code regarding pulsar or at least more hands on deck. Since Micronaut is reflection-free (or striving to be as much) it would help to adapt all SchemaTypes (instead of my silly switch-case solution actually use types and adapt to use Micronaut tools for json, string, byte, proto-buf...
Anyways thanks a lot I'll investigate more and worst-case I have to wait until I get time to implement Schema myself and use ObjectMapper in there which is quite easy just a lot of code retyping.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #8233: Custom ObjectMapper for JSON schema writer/reader for Java client?
Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #8233:
URL: https://github.com/apache/pulsar/issues/8233#issuecomment-713645158
@harissecic This feature has already been implemented by #6842 which is included since 2.6.0+.
There's a Kotlin code example in https://github.com/apache/pulsar/issues/6528#issuecomment-701410483 about how to use it. I haven't tried it myself, but there's a note that it doesn't work with the shaded client and you should use pulsar-client-original dependency instead of pulsar-client.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] harissecic commented on issue #8233: Custom ObjectMapper for JSON schema writer/reader for Java client?
Posted by GitBox <gi...@apache.org>.
harissecic commented on issue #8233:
URL: https://github.com/apache/pulsar/issues/8233#issuecomment-785995445
> > I'm unaware about pulsar-client-original and what's the difference between non "original" one.
>
> [pulsar-client](https://search.maven.org/artifact/org.apache.pulsar/pulsar-client/2.6.1/jar) shades most dependencies. [pulsar-client-original](https://search.maven.org/artifact/org.apache.pulsar/pulsar-client-original/2.6.1/jar) is the same code, but the dependencies aren't shaded. You can notice the difference in dependencies for example by comparing the pom.xml files.
Sorry it took some time but unshaded lib is colliding because of some netty loop. So it cannot be used with this approach. Not sure should I close this issue as it will not be resolved or to wait maybe some good idea comes along.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on issue #8233: Custom ObjectMapper for JSON schema writer/reader for Java client?
Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #8233:
URL: https://github.com/apache/pulsar/issues/8233#issuecomment-713675936
> I'm unaware about pulsar-client-original and what's the difference between non "original" one.
[pulsar-client](https://search.maven.org/artifact/org.apache.pulsar/pulsar-client/2.6.1/jar) shades most dependencies. [pulsar-client-original](https://search.maven.org/artifact/org.apache.pulsar/pulsar-client-original/2.6.1/jar) is the same code, but the dependencies aren't shaded. You can notice the difference in dependencies for example by comparing the pom.xml files.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] harissecic commented on issue #8233: Custom ObjectMapper for JSON schema writer/reader for Java client?
Posted by GitBox <gi...@apache.org>.
harissecic commented on issue #8233:
URL: https://github.com/apache/pulsar/issues/8233#issuecomment-713651608
> @harissecic This feature has already been implemented by #6842 which is included since 2.6.0+.
>
> There's a Kotlin code example in [#6528 (comment)](https://github.com/apache/pulsar/issues/6528#issuecomment-701410483) about how to use it. I haven't tried it myself, but there's a note that it doesn't work with the shaded client and you should use pulsar-client-original dependency instead of pulsar-client.
Thanks, I'll review this and check if it's possible to use it. I'm unaware about pulsar-client-original and what's the difference between non "original" one. If they are the same then I can switch to it if it passes tests. However this is not only for JSON it's also mentioning other Schema translator substitutions so it would help if they have more than just ObjectMapper.. However I'll check it out even just JSON is good enough for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari edited a comment on issue #8233: Custom ObjectMapper for JSON schema writer/reader for Java client?
Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on issue #8233:
URL: https://github.com/apache/pulsar/issues/8233#issuecomment-786134188
> Sorry it took some time but unshaded lib is colliding because of some netty loop. So it cannot be used with this approach. Not sure should I close this issue as it will not be resolved or to wait maybe some good idea comes along.
@harissecic The Netty conflict might be addressable if the other code you have is compatible with Netty 4.1.x. In that case, the Netty conflict might be caused by "misaligned" versions which can be resolved by importing the netty-bom input Maven's dependency management (or with Gradle's support for BOM).
```
<properties>
<netty.version>4.1.51.Final</netty.version>
<!-- ... more ... -->
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-bom</artifactId>
<version>${netty.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<!-- ... more ... -->
</dependencies>
</dependencyManagement>
```
You can ask for more help for such usage problems on [Pulsar Slack](https://pulsar.apache.org/en/contact/).
I hope this helps.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari edited a comment on issue #8233: Custom ObjectMapper for JSON schema writer/reader for Java client?
Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on issue #8233:
URL: https://github.com/apache/pulsar/issues/8233#issuecomment-786134188
> Sorry it took some time but unshaded lib is colliding because of some netty loop. So it cannot be used with this approach. Not sure should I close this issue as it will not be resolved or to wait maybe some good idea comes along.
@harissecic The Netty conflict might be addressable if the other code you have is compatible with Netty 4.1.x. In that case, the Netty conflict might be caused by "misaligned" versions which can be resolved by importing the netty-bom input Maven's dependency management (or with Gradle's support for BOM).
```
<properties>
<netty.version>4.1.51.Final</netty.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-bom</artifactId>
<version>${netty.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<!-- ... -->
</dependencies>
</dependencyManagement>
```
You can ask for more help for such usage problems on [Pulsar Slack](https://pulsar.apache.org/en/contact/).
I hope this helps.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] harissecic closed issue #8233: Custom ObjectMapper for JSON schema writer/reader for Java client?
Posted by GitBox <gi...@apache.org>.
harissecic closed issue #8233:
URL: https://github.com/apache/pulsar/issues/8233
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org