You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/10/06 08:01:38 UTC

[GitHub] [pulsar-client-node] dionjansen opened a new issue #129: Metadata properties incompatible with pulsar-dotnet

dionjansen opened a new issue #129:
URL: https://github.com/apache/pulsar-client-node/issues/129


   When producing messages with message properties from the node client producer and consuming them with the [pulsar-dotpulsar](https://github.com/apache/pulsar-dotpulsar) consumer it seems that the properties are mixed lost inside the message body.
   
   #### Reproduce:
   
   1. Modify the [producer.js](https://github.com/apache/pulsar-client-node/blob/master/examples/producer.js) to include message properties, say: `{foo: "bar"}`
   1. Modify the [Consuming](https://github.com/apache/pulsar-dotpulsar/blob/master/samples/Consuming/Program.cs) sample in dotpulsar to read and log the properties, e.g.: ` Console.WriteLine("Received: " + data + ", Props Count: " + message.Properties.Count);`
   1. Run both
   
   You'd expect the consumer to log to read:
   
   ```bash
   Received my-message-0, Props Count: 1
   Received my-message-1, Props Count: 1
   # ... etc
   ```
   
   But what we actually see:
   
   ```bash
   Received 
   
   foobar
          my-message-0, Props Count: 0
   Received 
   
   foobar
          my-message-1, Props Count: 0
   # ... etc
   ```
   
   According to the maintainer of [pulsar-dotpulsar](https://github.com/apache/pulsar-dotpulsar) there seems to be a difference in how the raw payload is created by the node-lib, in that the metadata properties of the message are being send as part of the payload. It is unclear if this is a bug or perhaps a missing feature that [pulsar-dotpulsar](https://github.com/apache/pulsar-dotpulsar) is not making use of (yet).
   
   #### Additional info
   
   I did the following checks between different clients to check if this issue can be seen in different combinations:
   
   - node-client > dotnet-client: does not work 👎 
   - node-client > python-client: works 👍 
   - node-client > java-client: works 👍 
   - python-client > dotnet-client: works 👍 
   
   Also this issue is only reproducible if the messages are not batched, sending a batch of messages in the above scenario works as expected.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar-client-node] massakam commented on issue #129: Metadata properties incompatible with pulsar-dotnet

Posted by GitBox <gi...@apache.org>.
massakam commented on issue #129:
URL: https://github.com/apache/pulsar-client-node/issues/129#issuecomment-705963659


   Oh, I haven't joined `#dev-dotpulsar` yet.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar-client-node] massakam commented on issue #129: Metadata properties incompatible with pulsar-dotnet

Posted by GitBox <gi...@apache.org>.
massakam commented on issue #129:
URL: https://github.com/apache/pulsar-client-node/issues/129#issuecomment-705960146


   A description of "batching" in Pulsar is here:
   https://pulsar.apache.org/docs/en/2.6.1/concepts-messaging/#batching


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar-client-node] massakam commented on issue #129: Metadata properties incompatible with pulsar-dotnet

Posted by GitBox <gi...@apache.org>.
massakam commented on issue #129:
URL: https://github.com/apache/pulsar-client-node/issues/129#issuecomment-705958062


   @dionjansen @sijie 
   Apparently, no matter what language the producer is written in, pulsar-dotpulsar doesn't seem to be able to get the properties of the received message when batching is enabled.
   
   When I sent a message using the Node.js producer with batching disabled, the pulsar-dotpulsar consumer was able to handle it successfully.
   
   To disable batching:
   ```js
   const producer = await client.createProducer({
     topic: 'persistent://public/default/test',
     batchingEnabled: false, // Disable batching
   });
   ```
   
   I suspect that pulsar-dotpulsar may not be able to properly deserialize batched messages.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar-client-node] sijie commented on issue #129: Metadata properties incompatible with pulsar-dotnet

Posted by GitBox <gi...@apache.org>.
sijie commented on issue #129:
URL: https://github.com/apache/pulsar-client-node/issues/129#issuecomment-705960431


   @massakam that was what I suspected and suggested in the slack channel. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar-client-node] nkurihar closed issue #129: Metadata properties incompatible with pulsar-dotnet

Posted by GitBox <gi...@apache.org>.
nkurihar closed issue #129:
URL: https://github.com/apache/pulsar-client-node/issues/129


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar-client-node] sijie commented on issue #129: Metadata properties incompatible with pulsar-dotnet

Posted by GitBox <gi...@apache.org>.
sijie commented on issue #129:
URL: https://github.com/apache/pulsar-client-node/issues/129#issuecomment-705960431


   @massakam that was what I suspected and suggested in the slack channel. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar-client-node] sijie commented on issue #129: Metadata properties incompatible with pulsar-dotnet

Posted by GitBox <gi...@apache.org>.
sijie commented on issue #129:
URL: https://github.com/apache/pulsar-client-node/issues/129#issuecomment-704108499


   @nkurihar Can you help with this issue?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar-client-node] massakam commented on issue #129: Metadata properties incompatible with pulsar-dotnet

Posted by GitBox <gi...@apache.org>.
massakam commented on issue #129:
URL: https://github.com/apache/pulsar-client-node/issues/129#issuecomment-705958062






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar-client-node] nkurihar commented on issue #129: Metadata properties incompatible with pulsar-dotnet

Posted by GitBox <gi...@apache.org>.
nkurihar commented on issue #129:
URL: https://github.com/apache/pulsar-client-node/issues/129#issuecomment-707439502


   https://github.com/apache/pulsar-dotpulsar/issues/61


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar-client-node] dionjansen commented on issue #129: Metadata properties incompatible with pulsar-dotnet

Posted by GitBox <gi...@apache.org>.
dionjansen commented on issue #129:
URL: https://github.com/apache/pulsar-client-node/issues/129#issuecomment-706709038


   @massakam that's definitely it. Turning batching off for both python and node clients works for the [pulsar-dotpulsar](https://github.com/apache/pulsar-dotpulsar) consumer, yet turning it on (for either producer) exposes the issue. It must be on the dotpulsar side then. I created an issue #69.
   
   I believe this one can be closed then, thanks for the help 👍 .


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org