You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/07/14 01:47:24 UTC
[GitHub] [pulsar-client-go] dferstay opened a new pull request #324: Different MessageID implementations for message Production and Consumption
dferstay opened a new pull request #324:
URL: https://github.com/apache/pulsar-client-go/pull/324
This change splits the `MessageID` implementation in two:
1. `messageID` - A 24 byte structure that re contains message identification
information only; to be used during message production
2. `trackingMessageID` - A 72 byte structucture that shares the same
message identification information as `messageID`
and adds `ackTracker`, `acker`, and `receivedTime`
fields; to be used during message consumption
Micro benchmarks show that passing arguments by value that are less-than
four words of memory are optimized by the Go runtime. Results from the
`pulsar/impl_message_bench_test.go` module are below.
```
name time/op
ProducerCall 1.46ns ± 5%
ProducerCall-4 1.47ns ± 5%
ConsumerCall 7.62ns ± 1%
ConsumerCall-4 7.53ns ± 5%
```
### Motivation
The messageID structure in pulsar-client-go has the following fields:
```
type messageID struct {
ledgerID int64
entryID int64
batchIdx int32
partitionIdx int32
tracker *ackTracker
consumer acker
receivedTime time.Time
}
```
The above consumes 72 bytes.
Consider the following:
- The Go runtime optimizes copying of values that are less than or equal to 4 words (32 bytes on 64-bit architectures).
- The tracker (8 bytes), consumer (16 bytes), and receivedTime (24 bytes) fields are not used when producing messages.
It would be advantageous to split the MessageID implementation into two structures: one used for message production, the other used for message consumption
### Modifications
Split the `MessageID` implementation into two structures; one used during message production:
```
type messageID struct {
ledgerID int64
entryID int64
batchIdx int32
partitionIdx int32
```
One used during message consumption:
```
type trackingMessageID struct {
messageID
tracker *ackTracker
consumer acker
receivedTime time.Time
}
```
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
This change is already covered by existing tests, such as:
- pulsar/consumer_multitopic_test.go
- pulsar/consumer_partition_test.go
- pulsar/consumer_regex_test.go
- pulsar/consumer_test.go
- pulsar/impl_message_test.go
- pulsar/negative_acks_tracker_test.go
- pulsar/producer_test.go
- pulsar/reader_test.go
### Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API: no
- The schema: no
- The default values of configurations: no
- The wire protocol: no
### Documentation
- Does this pull request introduce a new feature? no
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] wolfstudy edited a comment on pull request #324: Different MessageID implementations for message Production and Consumption
Posted by GitBox <gi...@apache.org>.
wolfstudy edited a comment on pull request #324:
URL: https://github.com/apache/pulsar-client-go/pull/324#issuecomment-657924547
Thanks @dferstay work for this. When we first added other fields to the messageID structure, I had the same considerations. This is the result of our final discussion. Reference to #82
@cckellogg What do you think of it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] wolfstudy edited a comment on pull request #324: Different MessageID implementations for message Production and Consumption
Posted by GitBox <gi...@apache.org>.
wolfstudy edited a comment on pull request #324:
URL: https://github.com/apache/pulsar-client-go/pull/324#issuecomment-657924547
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] wolfstudy commented on pull request #324: Different MessageID implementations for message Production and Consumption
Posted by GitBox <gi...@apache.org>.
wolfstudy commented on pull request #324:
URL: https://github.com/apache/pulsar-client-go/pull/324#issuecomment-657924547
Thanks @dferstay work for this. When I first added other fields to the messageID structure, I had the same considerations. This is the result of our final discussion. Reference to #82 , @cckellogg What do you think of it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] merlimat merged pull request #324: Different MessageID implementations for message Production and Consumption
Posted by GitBox <gi...@apache.org>.
merlimat merged pull request #324:
URL: https://github.com/apache/pulsar-client-go/pull/324
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org