You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Cong Zhao <zh...@apache.org> on 2023/07/25 04:12:09 UTC

[DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API

Hi Pulsar Community, I am writing to start the discussion on PIP 286  PR
with PIP contents: https://github.com/apache/pulsar/pull/20867  Thanks,
Cong Zhao

Re: [DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API

Posted by Cong Zhao <zh...@apache.org>.
Hi Penghui,

Thanks for the suggestion.

First, I changed the method to findFirstEntryMetadata because I think it's more common to look for the first matching entry compared to finding the entry of the last match, e.g. to look for the first position greater than or equal to a specific publish time.

As to return of 'RawEntryMetadata' is to reduce the complexity of the implementation of considerations, because when we use binary search managing the payload reference counter is complex. I think this method just returns entry metadata, If we need to get payload then we can be called again to call the readCompactedEntries method.

Thanks,
Cong Zhao

On 2023/08/28 01:31:14 PengHui Li wrote:
> Hi Cong,
> 
> Thanks for driving the proposal.
> Is it better to use `readNewestCompactedEntry(@Nonnull
> Predicate<RawEntryMetadata> condition)`
> 
> So that you don't need to introduce a new interface, RawEntryMetadata
> And it's a more general solution for the TopicCompactionService.
> 
> WDYT?
> 
> Regards,
> Penghui
> 
> On Tue, Jul 25, 2023 at 12:12 PM Cong Zhao <zh...@apache.org> wrote:
> 
> > Hi Pulsar Community, I am writing to start the discussion on PIP 286  PR
> > with PIP contents: https://github.com/apache/pulsar/pull/20867  Thanks,
> > Cong Zhao
> >
> 

Re: [DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API

Posted by PengHui Li <pe...@apache.org>.
Hi Cong,

Thanks for driving the proposal.
Is it better to use `readNewestCompactedEntry(@Nonnull
Predicate<RawEntryMetadata> condition)`

So that you don't need to introduce a new interface, RawEntryMetadata
And it's a more general solution for the TopicCompactionService.

WDYT?

Regards,
Penghui

On Tue, Jul 25, 2023 at 12:12 PM Cong Zhao <zh...@apache.org> wrote:

> Hi Pulsar Community, I am writing to start the discussion on PIP 286  PR
> with PIP contents: https://github.com/apache/pulsar/pull/20867  Thanks,
> Cong Zhao
>

Re: [DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API

Posted by Zike Yang <zi...@apache.org>.
+1 for me.

Thanks,
Zike Yang

On Mon, Sep 11, 2023 at 11:13 AM Cong Zhao <zh...@apache.org> wrote:
>
> Hi, guys
>
> I noticed that the only globally ordered fields in the metadata are 'publishTime' and 'entryIndex' (offset), so I split the previous method into `findEntryByPublishTime` and `findEntryByEntryIndex`,
> this will make method definitions more precise.
>
> The method returns the entry directly instead of RawEntryMetadata to avoid introducing a new interface, it's important to note that the caller needs to release it manually.
>
> I already updated the proposal, please help review it.
>
> Thanks,
> Cong Zhao
>
> On 2023/07/25 04:12:09 Cong Zhao wrote:
> > Hi Pulsar Community, I am writing to start the discussion on PIP 286  PR
> > with PIP contents: https://github.com/apache/pulsar/pull/20867  Thanks,
> > Cong Zhao
> >

Re: [DISCUSS] PIP-286: Add findNewestPosition method in the TopicCompactionService API

Posted by Cong Zhao <zh...@apache.org>.
Hi, guys

I noticed that the only globally ordered fields in the metadata are 'publishTime' and 'entryIndex' (offset), so I split the previous method into `findEntryByPublishTime` and `findEntryByEntryIndex`,
this will make method definitions more precise.

The method returns the entry directly instead of RawEntryMetadata to avoid introducing a new interface, it's important to note that the caller needs to release it manually.

I already updated the proposal, please help review it.

Thanks,
Cong Zhao

On 2023/07/25 04:12:09 Cong Zhao wrote:
> Hi Pulsar Community, I am writing to start the discussion on PIP 286  PR
> with PIP contents: https://github.com/apache/pulsar/pull/20867  Thanks,
> Cong Zhao
>