You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/06/25 02:46:46 UTC
[GitHub] [pulsar] zhanghaou opened a new pull request, #16215: Add first message's properties to batch message metadata.
zhanghaou opened a new pull request, #16215:
URL: https://github.com/apache/pulsar/pull/16215
**Motivation**
When use java client to build batch message, properties in single message can not add to batch message’s metadata.
**Modifications**
Add first message's properties to batch message's metadata.
**Verifying this change**
New unit test added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1230181524
>
@eolivelli Yes，make it configurable in pulsar client is better. When I make this PR, I checked Go client SDK, first message's properties was added in batch message's metadata. Should we keep the same behave on java and Go client SDK?
[https://github.com/apache/pulsar-client-go/blob/2d5f6fcfa22af4184793a22ca353731a02b54ca9/pulsar/internal/batch_builder.go#L214](url)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
Jason918 commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1229150899
> When use java client to build batch message, properties in single message can not add to batch message’s metadata.
Sorry I don't get why we need to add the properties to batch meta? Can you clarify your use case?
@zhanghaou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
eolivelli commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1230185156
I think that it is a bug in the GO client.
The metadata of each message can be different from the first message.
If you add these values on the main header, in any case, they cannot be used by anyone.
Because the broker cannot know about the behaviour of the client. In the broker you cannot know if the message was produced by a client who copies the metadata, so in any case you MUST unpack all the messages.
Why don't you implement this feature (that I guess is for some specific usecase) with a Interceptor ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1230169520
> > When use java client to build batch message, properties in single message can not add to batch message’s metadata.
>
> Sorry I don't get why we need to add the properties to batch meta? Can you clarify your use case? @zhanghaou
@Jason918 We need to get the properties in pulsar broker side, but if use batch message in java client, we can not get batch message's properties if we don't decode the batch message. Some other metadata were copied to batch message's from first message's metadata, so I think properties can also added to batch message's metadata(Maybe make it configurable is better).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1166300069
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1226691806
The pr had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1166305150
@zhanghaou Please provide a correct documentation label for your PR.
Instructions see [Pulsar Documentation Label Guide](https://docs.google.com/document/d/1Qw7LHQdXWBW9t2-r-A7QdFDBwmZh6ytB4guwMoXHqc0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1166305013
@zhanghaou Please provide a correct documentation label for your PR.
Instructions see [Pulsar Documentation Label Guide](https://docs.google.com/document/d/1Qw7LHQdXWBW9t2-r-A7QdFDBwmZh6ytB4guwMoXHqc0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1166754906
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1166801652
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1230170945
> > When use java client to build batch message, properties in single message can not add to batch message’s metadata.
>
> Sorry I don't get why we need to add the properties to batch meta? Can you clarify your use case? @zhanghaou
@Jason918 We need to get the properties in pulsar broker side, but if use batch message in java client, we can not get batch message's properties if we don't decode the batch message. Some other metadata were copied to batch message's from first message's metadata, so I think properties can also added to batch message's metadata(Maybe make it configurable is better).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1231061699
@eolivelli I want to added this feature only for pulsar broker, and all client had enabled this feature. My case is to support flow dyeing, which messages from same environment should be delivered to consumers in same environment(Shared or Key_shared mode), so Interceptor on client side seems useless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1166390995
@zhanghaou Please provide a correct documentation label for your PR.
Instructions see [Pulsar Documentation Label Guide](https://docs.google.com/document/d/1Qw7LHQdXWBW9t2-r-A7QdFDBwmZh6ytB4guwMoXHqc0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
tisonkun commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1345424374
Closed as stale and no consensus. Please rebase and resubmit the patch if it's still relevant.
@zhanghaou I suggest you talk about more about your use case (like https://github.com/apache/pulsar/pull/16215#issuecomment-1231061699) instead of details (https://github.com/apache/pulsar/pull/16215#issue-1284416728). We should first get your motivation and then determinate whether this change it good or not.
If it's for your use case specifically, Pulsar as an open-source project already allow you to modify and distribution as you like. If it's a common use case that you believe can benefit the community, you can start a discussion at [the forum](https://github.com/apache/pulsar/discussions/categories/general) or the dev@ mailing list to reach a consensus first.
This way, you may find better approaches to meet your requirements, or at least avoid [the XY problem](https://xyproblem.info/).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1166176347
@zhanghaou Please provide a correct documentation label for your PR.
Instructions see [Pulsar Documentation Label Guide](https://docs.google.com/document/d/1Qw7LHQdXWBW9t2-r-A7QdFDBwmZh6ytB4guwMoXHqc0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1230181932
>
@eolivelli Yes，make it configurable in pulsar client is better. When I make this PR, I checked Go client SDK, first message's properties was added in batch message's metadata. Should we keep the same behave on java and Go client SDK?
[https://github.com/apache/pulsar-client-go/blob/2d5f6fcfa22af4184793a22ca353731a02b54ca9/pulsar/internal/batch_builder.go#L214](url)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun closed pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
tisonkun closed pull request #16215: Add first message's properties to batch message metadata.
URL: https://github.com/apache/pulsar/pull/16215
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
congbobo184 commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1318541432
@zhanghaou hi, I move this PR to release/2.9.5, if you have any questions, please ping me. thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16215: Add first message's properties to batch message metadata.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #16215:
URL: https://github.com/apache/pulsar/pull/16215#issuecomment-1261657955
The pr had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org