You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Cong Zhao <zh...@apache.org> on 2023/07/25 04:12:09 UTC
[DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API
Hi Pulsar Community, I am writing to start the discussion on PIP 286 PR
with PIP contents: https://github.com/apache/pulsar/pull/20867 Thanks,
Cong Zhao
Re: [DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API
Posted by Cong Zhao <zh...@apache.org>.
Hi Penghui,
Thanks for the suggestion.
First, I changed the method to findFirstEntryMetadata because I think it's more common to look for the first matching entry compared to finding the entry of the last match, e.g. to look for the first position greater than or equal to a specific publish time.
As to return of 'RawEntryMetadata' is to reduce the complexity of the implementation of considerations, because when we use binary search managing the payload reference counter is complex. I think this method just returns entry metadata, If we need to get payload then we can be called again to call the readCompactedEntries method.
Thanks,
Cong Zhao
On 2023/08/28 01:31:14 PengHui Li wrote:
> Hi Cong,
>
> Thanks for driving the proposal.
> Is it better to use `readNewestCompactedEntry(@Nonnull
> Predicate<RawEntryMetadata> condition)`
>
> So that you don't need to introduce a new interface, RawEntryMetadata
> And it's a more general solution for the TopicCompactionService.
>
> WDYT?
>
> Regards,
> Penghui
>
> On Tue, Jul 25, 2023 at 12:12 PM Cong Zhao <zh...@apache.org> wrote:
>
> > Hi Pulsar Community, I am writing to start the discussion on PIP 286 PR
> > with PIP contents: https://github.com/apache/pulsar/pull/20867 Thanks,
> > Cong Zhao
> >
>
Re: [DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API
Posted by PengHui Li <pe...@apache.org>.
Hi Cong,
Thanks for driving the proposal.
Is it better to use `readNewestCompactedEntry(@Nonnull
Predicate<RawEntryMetadata> condition)`
So that you don't need to introduce a new interface, RawEntryMetadata
And it's a more general solution for the TopicCompactionService.
WDYT?
Regards,
Penghui
On Tue, Jul 25, 2023 at 12:12 PM Cong Zhao <zh...@apache.org> wrote:
> Hi Pulsar Community, I am writing to start the discussion on PIP 286 PR
> with PIP contents: https://github.com/apache/pulsar/pull/20867 Thanks,
> Cong Zhao
>
Re: [DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API
Posted by Zike Yang <zi...@apache.org>.
+1 for me.
Thanks,
Zike Yang
On Mon, Sep 11, 2023 at 11:13 AM Cong Zhao <zh...@apache.org> wrote:
>
> Hi, guys
>
> I noticed that the only globally ordered fields in the metadata are 'publishTime' and 'entryIndex' (offset), so I split the previous method into `findEntryByPublishTime` and `findEntryByEntryIndex`,
> this will make method definitions more precise.
>
> The method returns the entry directly instead of RawEntryMetadata to avoid introducing a new interface, it's important to note that the caller needs to release it manually.
>
> I already updated the proposal, please help review it.
>
> Thanks,
> Cong Zhao
>
> On 2023/07/25 04:12:09 Cong Zhao wrote:
> > Hi Pulsar Community, I am writing to start the discussion on PIP 286 PR
> > with PIP contents: https://github.com/apache/pulsar/pull/20867 Thanks,
> > Cong Zhao
> >
Re: [DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API
Posted by Cong Zhao <zh...@apache.org>.
Hi, guys
I noticed that the only globally ordered fields in the metadata are 'publishTime' and 'entryIndex' (offset), so I split the previous method into `findEntryByPublishTime` and `findEntryByEntryIndex`,
this will make method definitions more precise.
The method returns the entry directly instead of RawEntryMetadata to avoid introducing a new interface, it's important to note that the caller needs to release it manually.
I already updated the proposal, please help review it.
Thanks,
Cong Zhao
On 2023/07/25 04:12:09 Cong Zhao wrote:
> Hi Pulsar Community, I am writing to start the discussion on PIP 286 PR
> with PIP contents: https://github.com/apache/pulsar/pull/20867 Thanks,
> Cong Zhao
>