You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by 太上玄元道君 <da...@apache.org> on 2024/03/09 22:41:17 UTC
[DISCUSS] PIP-345: Optimize finding message by timestamp
Hi Pulsar community,
A new PIP is opened, this thread is to discuss PIP-345: Optimize finding
message by timestamp.
Motivation:
Finding message by timestamp is widely used in Pulsar:
* It is used by the `pulsar-admin` tool to get the message id by
timestamp, expire messages by timestamp, and reset cursor.
* It is used by the `pulsar-client` to reset the subscription to a specific
timestamp.
* And also used by the `expiry-monitor` to find the messages that are
expired.
Even though the current implementation is correct, and using binary search
to speed-up, but it's still not efficient *enough*.
The current implementation is to scan all the ledgers to find the message
by timestamp.
This is a performance bottleneck, especially for large topics with many
messages.
Say, if there is a topic which has 1m entries, through the binary search,
it will take 20 iterations to find the message.
In some extreme cases, it may lead to a timeout, and the client will not be
able to seeking by timestamp.
PIP: https://github.com/apache/pulsar/pull/22234
Your feedback is very important to us, please take a moment to review the
proposal and provide your thoughts.
Thanks,
Tao Jiuming
Re: [DISCUSS] PIP-345: Optimize finding message by timestamp
Posted by 太上玄元道君 <da...@apache.org>.
Hi Yubiao,
Thanks for your review!
1. As I mentioned in the *Alternatives* part of the PIP
`LedgerInfo#timestamp` is broker's timestamp, finding message by timestamp
is client's timestamp(publishTimestamp).
We have to consider about client's clock doesn't sync to broker's clock,
and we are finding a message by publishTimestamp, but pre-filter by
broker's timestamp, it doesn't make sense.
2. Why do we need both `minPublishTimestamp` and `maxPublishTimestamp`
As the email I replied to Girish, the next ledger's `minPublishTimestamp`
might be less than it's previous ledger's `maxPublishTimestamp`,
so we can't use next ledger's `minPublishTimestamp` as it's previous
ledger's `maxPublishTimestamp`, I want to make it clear.
Thanks,
Tao Jiuming
Yubiao Feng <yu...@streamnative.io.invalid> 于2024年3月16日周六 20:29写道：
> Hi Jiuming
>
> Firstly, I think the idea you provided is great.
>
> It seems the field `beginPublishTimestamp` is also not needed, and there is
> an existing field `ledgerInfo.timestamp` to use,
> - The current ledger's timestamp can be used as `beginPublishTimestamp`
> - The next ledger's timestamp can be used as `endPublishTimestamp`
>
> Thanks
> Yubiao Feng
>
>
> On Fri, Mar 15, 2024 at 1:57 PM 太上玄元道君 <da...@apache.org> wrote:
>
> > Hi, Girish,
> >
> > Thanks for your feedback!
> >
> > In general, it's a very good suggestion, we can just use one single
> > `beginPublishTimestamp` to achieve our goal,
> > but the actual problem will be a bit more complex.
> >
> > Actually, the naming of `beginPublishTimestamp` and `endPublishTimestamp`
> > has a little problem,
> > it should be `minPublishTimestamp` and `maxPublishTimestamp`.
> >
> > In some cases, next ledger's `minPublishTimestamp` may less than it's
> > previous ledger's `maxPublishTimestamp`,
> > so we have to maintain both `minPublishTimestamp` and
> > `maxPublishTimestamp`.
> >
> > Say, there are 2 producers publishing to the topic, Producer1 send
> > *message1* to the topic, broker received
> > *message1* immediately and persist it to the ledger. Producer2 send
> > *message2* to the broker *before* *message1*,
> > but for some reason, broker received *message2* after a while.
> > At the same time, Ledger switching happens, the previous ledger's
> > `maxPublishTimestamp` is *message1*'s publishTimestamp,
> > and the current ledger's `minPublishTimestamp` is *message2*'s
> > publishTimestamp,
> > so the current ledger's `minPublishTimestamp` is less than the previous
> > ledger's `maxPublishTimestamp`, right?
> >
> > If we just have a single field `minPublishTimestamp`, it will have a
> > hidden meaning: the next ledger's `minPublishTimestamp`
> > is it's previous ledger's `maxPublishTimestamp`, it's incorrect.
> > So we want to introduce `minPublishTimestamp` and `maxPublishTimestamp`
> to
> > make it clear.
> >
> > Thanks,
> > Tao Jiuming
> >
> > Girish Sharma <sc...@gmail.com> 于2024年3月15日周五 12:14写道：
> >
> > > One suggestion, I think you can make do with storing just begin
> > timestamp.
> > > Any search utilising these values will work the same way with just one
> of
> > > those timestamps compared to both begin and end.
> > >
> > > Any particular reason you need both the timestamps?
> > >
> > > Regards
> > >
> > > On Fri, Mar 15, 2024, 9:39 AM 太上玄元道君 <da...@apache.org> wrote:
> > >
> > > > bump
> > > >
> > > > 太上玄元道君 <da...@apache.org>于2024年3月10日 周日06:41写道：
> > > >
> > > > > Hi Pulsar community,
> > > > >
> > > > > A new PIP is opened, this thread is to discuss PIP-345: Optimize
> > > finding
> > > > > message by timestamp.
> > > > >
> > > > > Motivation:
> > > > > Finding message by timestamp is widely used in Pulsar:
> > > > > * It is used by the `pulsar-admin` tool to get the message id by
> > > > > timestamp, expire messages by timestamp, and reset cursor.
> > > > > * It is used by the `pulsar-client` to reset the subscription to a
> > > > > specific timestamp.
> > > > > * And also used by the `expiry-monitor` to find the messages that
> are
> > > > > expired.
> > > > > Even though the current implementation is correct, and using binary
> > > > search
> > > > > to speed-up, but it's still not efficient *enough*.
> > > > > The current implementation is to scan all the ledgers to find the
> > > message
> > > > > by timestamp.
> > > > > This is a performance bottleneck, especially for large topics with
> > many
> > > > > messages.
> > > > > Say, if there is a topic which has 1m entries, through the binary
> > > search,
> > > > > it will take 20 iterations to find the message.
> > > > > In some extreme cases, it may lead to a timeout, and the client
> will
> > > not
> > > > > be able to seeking by timestamp.
> > > > >
> > > > > PIP: https://github.com/apache/pulsar/pull/22234
> > > > >
> > > > > Your feedback is very important to us, please take a moment to
> review
> > > the
> > > > > proposal and provide your thoughts.
> > > > >
> > > > > Thanks,
> > > > > Tao Jiuming
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] PIP-345: Optimize finding message by timestamp
Posted by Yubiao Feng <yu...@streamnative.io.INVALID>.
Hi Jiuming
Firstly, I think the idea you provided is great.
It seems the field `beginPublishTimestamp` is also not needed, and there is
an existing field `ledgerInfo.timestamp` to use,
- The current ledger's timestamp can be used as `beginPublishTimestamp`
- The next ledger's timestamp can be used as `endPublishTimestamp`
Thanks
Yubiao Feng
On Fri, Mar 15, 2024 at 1:57 PM 太上玄元道君 <da...@apache.org> wrote:
> Hi, Girish,
>
> Thanks for your feedback!
>
> In general, it's a very good suggestion, we can just use one single
> `beginPublishTimestamp` to achieve our goal,
> but the actual problem will be a bit more complex.
>
> Actually, the naming of `beginPublishTimestamp` and `endPublishTimestamp`
> has a little problem,
> it should be `minPublishTimestamp` and `maxPublishTimestamp`.
>
> In some cases, next ledger's `minPublishTimestamp` may less than it's
> previous ledger's `maxPublishTimestamp`,
> so we have to maintain both `minPublishTimestamp` and
> `maxPublishTimestamp`.
>
> Say, there are 2 producers publishing to the topic, Producer1 send
> *message1* to the topic, broker received
> *message1* immediately and persist it to the ledger. Producer2 send
> *message2* to the broker *before* *message1*,
> but for some reason, broker received *message2* after a while.
> At the same time, Ledger switching happens, the previous ledger's
> `maxPublishTimestamp` is *message1*'s publishTimestamp,
> and the current ledger's `minPublishTimestamp` is *message2*'s
> publishTimestamp,
> so the current ledger's `minPublishTimestamp` is less than the previous
> ledger's `maxPublishTimestamp`, right?
>
> If we just have a single field `minPublishTimestamp`, it will have a
> hidden meaning: the next ledger's `minPublishTimestamp`
> is it's previous ledger's `maxPublishTimestamp`, it's incorrect.
> So we want to introduce `minPublishTimestamp` and `maxPublishTimestamp` to
> make it clear.
>
> Thanks,
> Tao Jiuming
>
> Girish Sharma <sc...@gmail.com> 于2024年3月15日周五 12:14写道：
>
> > One suggestion, I think you can make do with storing just begin
> timestamp.
> > Any search utilising these values will work the same way with just one of
> > those timestamps compared to both begin and end.
> >
> > Any particular reason you need both the timestamps?
> >
> > Regards
> >
> > On Fri, Mar 15, 2024, 9:39 AM 太上玄元道君 <da...@apache.org> wrote:
> >
> > > bump
> > >
> > > 太上玄元道君 <da...@apache.org>于2024年3月10日 周日06:41写道：
> > >
> > > > Hi Pulsar community,
> > > >
> > > > A new PIP is opened, this thread is to discuss PIP-345: Optimize
> > finding
> > > > message by timestamp.
> > > >
> > > > Motivation:
> > > > Finding message by timestamp is widely used in Pulsar:
> > > > * It is used by the `pulsar-admin` tool to get the message id by
> > > > timestamp, expire messages by timestamp, and reset cursor.
> > > > * It is used by the `pulsar-client` to reset the subscription to a
> > > > specific timestamp.
> > > > * And also used by the `expiry-monitor` to find the messages that are
> > > > expired.
> > > > Even though the current implementation is correct, and using binary
> > > search
> > > > to speed-up, but it's still not efficient *enough*.
> > > > The current implementation is to scan all the ledgers to find the
> > message
> > > > by timestamp.
> > > > This is a performance bottleneck, especially for large topics with
> many
> > > > messages.
> > > > Say, if there is a topic which has 1m entries, through the binary
> > search,
> > > > it will take 20 iterations to find the message.
> > > > In some extreme cases, it may lead to a timeout, and the client will
> > not
> > > > be able to seeking by timestamp.
> > > >
> > > > PIP: https://github.com/apache/pulsar/pull/22234
> > > >
> > > > Your feedback is very important to us, please take a moment to review
> > the
> > > > proposal and provide your thoughts.
> > > >
> > > > Thanks,
> > > > Tao Jiuming
> > > >
> > >
> >
>
Re: [DISCUSS] PIP-345: Optimize finding message by timestamp
Posted by 太上玄元道君 <da...@apache.org>.
Hi, Girish,
Thanks for your feedback!
In general, it's a very good suggestion, we can just use one single
`beginPublishTimestamp` to achieve our goal,
but the actual problem will be a bit more complex.
Actually, the naming of `beginPublishTimestamp` and `endPublishTimestamp`
has a little problem,
it should be `minPublishTimestamp` and `maxPublishTimestamp`.
In some cases, next ledger's `minPublishTimestamp` may less than it's
previous ledger's `maxPublishTimestamp`,
so we have to maintain both `minPublishTimestamp` and `maxPublishTimestamp`.
Say, there are 2 producers publishing to the topic, Producer1 send
*message1* to the topic, broker received
*message1* immediately and persist it to the ledger. Producer2 send
*message2* to the broker *before* *message1*,
but for some reason, broker received *message2* after a while.
At the same time, Ledger switching happens, the previous ledger's
`maxPublishTimestamp` is *message1*'s publishTimestamp,
and the current ledger's `minPublishTimestamp` is *message2*'s
publishTimestamp,
so the current ledger's `minPublishTimestamp` is less than the previous
ledger's `maxPublishTimestamp`, right?
If we just have a single field `minPublishTimestamp`, it will have a
hidden meaning: the next ledger's `minPublishTimestamp`
is it's previous ledger's `maxPublishTimestamp`, it's incorrect.
So we want to introduce `minPublishTimestamp` and `maxPublishTimestamp` to
make it clear.
Thanks,
Tao Jiuming
Girish Sharma <sc...@gmail.com> 于2024年3月15日周五 12:14写道：
> One suggestion, I think you can make do with storing just begin timestamp.
> Any search utilising these values will work the same way with just one of
> those timestamps compared to both begin and end.
>
> Any particular reason you need both the timestamps?
>
> Regards
>
> On Fri, Mar 15, 2024, 9:39 AM 太上玄元道君 <da...@apache.org> wrote:
>
> > bump
> >
> > 太上玄元道君 <da...@apache.org>于2024年3月10日 周日06:41写道：
> >
> > > Hi Pulsar community,
> > >
> > > A new PIP is opened, this thread is to discuss PIP-345: Optimize
> finding
> > > message by timestamp.
> > >
> > > Motivation:
> > > Finding message by timestamp is widely used in Pulsar:
> > > * It is used by the `pulsar-admin` tool to get the message id by
> > > timestamp, expire messages by timestamp, and reset cursor.
> > > * It is used by the `pulsar-client` to reset the subscription to a
> > > specific timestamp.
> > > * And also used by the `expiry-monitor` to find the messages that are
> > > expired.
> > > Even though the current implementation is correct, and using binary
> > search
> > > to speed-up, but it's still not efficient *enough*.
> > > The current implementation is to scan all the ledgers to find the
> message
> > > by timestamp.
> > > This is a performance bottleneck, especially for large topics with many
> > > messages.
> > > Say, if there is a topic which has 1m entries, through the binary
> search,
> > > it will take 20 iterations to find the message.
> > > In some extreme cases, it may lead to a timeout, and the client will
> not
> > > be able to seeking by timestamp.
> > >
> > > PIP: https://github.com/apache/pulsar/pull/22234
> > >
> > > Your feedback is very important to us, please take a moment to review
> the
> > > proposal and provide your thoughts.
> > >
> > > Thanks,
> > > Tao Jiuming
> > >
> >
>
Re: [DISCUSS] PIP-345: Optimize finding message by timestamp
Posted by Girish Sharma <sc...@gmail.com>.
One suggestion, I think you can make do with storing just begin timestamp.
Any search utilising these values will work the same way with just one of
those timestamps compared to both begin and end.
Any particular reason you need both the timestamps?
Regards
On Fri, Mar 15, 2024, 9:39 AM 太上玄元道君 <da...@apache.org> wrote:
> bump
>
> 太上玄元道君 <da...@apache.org>于2024年3月10日 周日06:41写道：
>
> > Hi Pulsar community,
> >
> > A new PIP is opened, this thread is to discuss PIP-345: Optimize finding
> > message by timestamp.
> >
> > Motivation:
> > Finding message by timestamp is widely used in Pulsar:
> > * It is used by the `pulsar-admin` tool to get the message id by
> > timestamp, expire messages by timestamp, and reset cursor.
> > * It is used by the `pulsar-client` to reset the subscription to a
> > specific timestamp.
> > * And also used by the `expiry-monitor` to find the messages that are
> > expired.
> > Even though the current implementation is correct, and using binary
> search
> > to speed-up, but it's still not efficient *enough*.
> > The current implementation is to scan all the ledgers to find the message
> > by timestamp.
> > This is a performance bottleneck, especially for large topics with many
> > messages.
> > Say, if there is a topic which has 1m entries, through the binary search,
> > it will take 20 iterations to find the message.
> > In some extreme cases, it may lead to a timeout, and the client will not
> > be able to seeking by timestamp.
> >
> > PIP: https://github.com/apache/pulsar/pull/22234
> >
> > Your feedback is very important to us, please take a moment to review the
> > proposal and provide your thoughts.
> >
> > Thanks,
> > Tao Jiuming
> >
>
Re: [DISCUSS] PIP-345: Optimize finding message by timestamp
Posted by 太上玄元道君 <da...@apache.org>.
bump
太上玄元道君 <da...@apache.org>于2024年3月10日 周日06:41写道：
> Hi Pulsar community,
>
> A new PIP is opened, this thread is to discuss PIP-345: Optimize finding
> message by timestamp.
>
> Motivation:
> Finding message by timestamp is widely used in Pulsar:
> * It is used by the `pulsar-admin` tool to get the message id by
> timestamp, expire messages by timestamp, and reset cursor.
> * It is used by the `pulsar-client` to reset the subscription to a
> specific timestamp.
> * And also used by the `expiry-monitor` to find the messages that are
> expired.
> Even though the current implementation is correct, and using binary search
> to speed-up, but it's still not efficient *enough*.
> The current implementation is to scan all the ledgers to find the message
> by timestamp.
> This is a performance bottleneck, especially for large topics with many
> messages.
> Say, if there is a topic which has 1m entries, through the binary search,
> it will take 20 iterations to find the message.
> In some extreme cases, it may lead to a timeout, and the client will not
> be able to seeking by timestamp.
>
> PIP: https://github.com/apache/pulsar/pull/22234
>
> Your feedback is very important to us, please take a moment to review the
> proposal and provide your thoughts.
>
> Thanks,
> Tao Jiuming
>
Re: [DISCUSS] PIP-345: Optimize finding message by timestamp
Posted by 太上玄元道君 <da...@apache.org>.
Hi Dave,
Thanks for your review!
Perhaps it's because I wrote more detailed steps, but the key points is:
1. Deserialize MessageMetadata once broker received message
2. Pass MessageMetadata to `PublishContext`
3. After add entries finished, get `publishTimestamp` from
`PublishContext#messageMetadata`, and
update `beginPublishTimestamp`,`endPublishTimestamp` of the `Ledger`
Since we might deserialize MessageMetadata in message publishing in the
broker side(PersistentTopic#isExceedMaximumDeliveryDelay,
MessageDeduplication), deserialize MessageMetadata once broker received
message will help
to reduce the number of MessageMetadata deserializing in some cases.
About maintain these new ledger fields, it just like
```
public class ManagedLedgerImpl {
// New field
// Add a map to record the begin/end publish timestamp of the ledger
private final NavigableMap<Long, MutablePair</* begin publish
timestamp*/Long, /* end publish timestamp*/Long>> publishTimestamps
= new ConcurrentSkipListMap<>();
// Update the begin/end publish timestamp of the ledger after the entry
is added to the ledger.
// New method
protected void updatePublishTimestamp(long ledgerId, long
publishTimestamp) {
MutablePair<Long, Long> pair =
publishTimestamps.computeIfAbsent(ledgerId, k -> new
MutablePair<>(Long.MAX_VALUE, Long.MIN_VALUE));
pair.setLeft(Math.min(pair.getLeft(), publishTimestamp));
pair.setRight(Math.max(pair.getRight(), publishTimestamp));
}
}
```
I just use a Map to maintain it, when closing Ledger, set
`beginPublishTimestamp`,`endPublishTimestamp` to `LedgerInfo`.
Besides, no additional expenses were introduced.
So, if you are asking about `the time spent`, I would say, *nearly* zero.
Thanks,
Tao Jiuming
Dave Fisher <wa...@apache.org> 于2024年3月12日周二 10:50写道：
> What can you say about the time spent to maintain these new ledger fields?
> I think you are asking to modify the main message logic which is highly
> optimized., but I’m not sure. Have you tried your code on your own
> hardware? Do you have performance comparisons of the normal flow?
>
> > On Mar 11, 2024, at 7:41 PM, 太上玄元道君 <da...@apache.org> wrote:
> >
> > bump
> >
> > 太上玄元道君 <da...@apache.org>于2024年3月11日 周一17:55写道：
> >
> >> bump
> >>
> >> 太上玄元道君 <da...@apache.org> 于2024年3月10日周日 06:41写道：
> >>
> >>> Hi Pulsar community,
> >>>
> >>> A new PIP is opened, this thread is to discuss PIP-345: Optimize
> finding
> >>> message by timestamp.
> >>>
> >>> Motivation:
> >>> Finding message by timestamp is widely used in Pulsar:
> >>> * It is used by the `pulsar-admin` tool to get the message id by
> >>> timestamp, expire messages by timestamp, and reset cursor.
> >>> * It is used by the `pulsar-client` to reset the subscription to a
> >>> specific timestamp.
> >>> * And also used by the `expiry-monitor` to find the messages that are
> >>> expired.
> >>> Even though the current implementation is correct, and using binary
> >>> search to speed-up, but it's still not efficient *enough*.
> >>> The current implementation is to scan all the ledgers to find the
> message
> >>> by timestamp.
> >>> This is a performance bottleneck, especially for large topics with many
> >>> messages.
> >>> Say, if there is a topic which has 1m entries, through the binary
> search,
> >>> it will take 20 iterations to find the message.
> >>> In some extreme cases, it may lead to a timeout, and the client will
> not
> >>> be able to seeking by timestamp.
> >>>
> >>> PIP: https://github.com/apache/pulsar/pull/22234
> >>>
> >>> Your feedback is very important to us, please take a moment to review
> the
> >>> proposal and provide your thoughts.
> >>>
> >>> Thanks,
> >>> Tao Jiuming
> >>>
> >>
>
>
Re: [DISCUSS] PIP-345: Optimize finding message by timestamp
Posted by Dave Fisher <wa...@apache.org>.
What can you say about the time spent to maintain these new ledger fields? I think you are asking to modify the main message logic which is highly optimized., but I’m not sure. Have you tried your code on your own hardware? Do you have performance comparisons of the normal flow?
> On Mar 11, 2024, at 7:41 PM, 太上玄元道君 <da...@apache.org> wrote:
>
> bump
>
> 太上玄元道君 <da...@apache.org>于2024年3月11日 周一17:55写道：
>
>> bump
>>
>> 太上玄元道君 <da...@apache.org> 于2024年3月10日周日 06:41写道：
>>
>>> Hi Pulsar community,
>>>
>>> A new PIP is opened, this thread is to discuss PIP-345: Optimize finding
>>> message by timestamp.
>>>
>>> Motivation:
>>> Finding message by timestamp is widely used in Pulsar:
>>> * It is used by the `pulsar-admin` tool to get the message id by
>>> timestamp, expire messages by timestamp, and reset cursor.
>>> * It is used by the `pulsar-client` to reset the subscription to a
>>> specific timestamp.
>>> * And also used by the `expiry-monitor` to find the messages that are
>>> expired.
>>> Even though the current implementation is correct, and using binary
>>> search to speed-up, but it's still not efficient *enough*.
>>> The current implementation is to scan all the ledgers to find the message
>>> by timestamp.
>>> This is a performance bottleneck, especially for large topics with many
>>> messages.
>>> Say, if there is a topic which has 1m entries, through the binary search,
>>> it will take 20 iterations to find the message.
>>> In some extreme cases, it may lead to a timeout, and the client will not
>>> be able to seeking by timestamp.
>>>
>>> PIP: https://github.com/apache/pulsar/pull/22234
>>>
>>> Your feedback is very important to us, please take a moment to review the
>>> proposal and provide your thoughts.
>>>
>>> Thanks,
>>> Tao Jiuming
>>>
>>
Re: [DISCUSS] PIP-345: Optimize finding message by timestamp
Posted by 太上玄元道君 <da...@apache.org>.
bump
太上玄元道君 <da...@apache.org>于2024年3月11日 周一17:55写道：
> bump
>
> 太上玄元道君 <da...@apache.org> 于2024年3月10日周日 06:41写道：
>
>> Hi Pulsar community,
>>
>> A new PIP is opened, this thread is to discuss PIP-345: Optimize finding
>> message by timestamp.
>>
>> Motivation:
>> Finding message by timestamp is widely used in Pulsar:
>> * It is used by the `pulsar-admin` tool to get the message id by
>> timestamp, expire messages by timestamp, and reset cursor.
>> * It is used by the `pulsar-client` to reset the subscription to a
>> specific timestamp.
>> * And also used by the `expiry-monitor` to find the messages that are
>> expired.
>> Even though the current implementation is correct, and using binary
>> search to speed-up, but it's still not efficient *enough*.
>> The current implementation is to scan all the ledgers to find the message
>> by timestamp.
>> This is a performance bottleneck, especially for large topics with many
>> messages.
>> Say, if there is a topic which has 1m entries, through the binary search,
>> it will take 20 iterations to find the message.
>> In some extreme cases, it may lead to a timeout, and the client will not
>> be able to seeking by timestamp.
>>
>> PIP: https://github.com/apache/pulsar/pull/22234
>>
>> Your feedback is very important to us, please take a moment to review the
>> proposal and provide your thoughts.
>>
>> Thanks,
>> Tao Jiuming
>>
>
Re: [DISCUSS] PIP-345: Optimize finding message by timestamp
Posted by 太上玄元道君 <da...@apache.org>.
bump
太上玄元道君 <da...@apache.org> 于2024年3月10日周日 06:41写道：
> Hi Pulsar community,
>
> A new PIP is opened, this thread is to discuss PIP-345: Optimize finding
> message by timestamp.
>
> Motivation:
> Finding message by timestamp is widely used in Pulsar:
> * It is used by the `pulsar-admin` tool to get the message id by
> timestamp, expire messages by timestamp, and reset cursor.
> * It is used by the `pulsar-client` to reset the subscription to a
> specific timestamp.
> * And also used by the `expiry-monitor` to find the messages that are
> expired.
> Even though the current implementation is correct, and using binary search
> to speed-up, but it's still not efficient *enough*.
> The current implementation is to scan all the ledgers to find the message
> by timestamp.
> This is a performance bottleneck, especially for large topics with many
> messages.
> Say, if there is a topic which has 1m entries, through the binary search,
> it will take 20 iterations to find the message.
> In some extreme cases, it may lead to a timeout, and the client will not
> be able to seeking by timestamp.
>
> PIP: https://github.com/apache/pulsar/pull/22234
>
> Your feedback is very important to us, please take a moment to review the
> proposal and provide your thoughts.
>
> Thanks,
> Tao Jiuming
>