You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by Sijie Guo <gu...@gmail.com> on 2017/11/03 01:39:21 UTC
Iterable
Hi Enrico,
In the new ledger api, we return Iterable<LedgerEntry> instead of
Iterator<LedgerEntry>. In general, it is good to return Iterable, however
it seems to be problematic in following use case:
Iterable<LedgerEntry> entries = rh.read(...);
Iterator<LedgerEntry> iter1 = entries.iterator();
while (iter1.hasNext()) {
LedgerEntry le = iter1.next;
le.getEntryBuffer().release();
}
Iterator<LedgerEntry> iter2 = entries.iterator();
while (iter2.hasNext()) {
LedgerEntry le = iter2.next();
le.getEntryBuffer().release();
}
Using Iterable opens up the chance that people will create multiple
iterators and use them concurrently. I think we should not use
Iterable<LedgerEntry>, which it might actually cause problems.
I am thinking if we can create an interface LedgerEntries to extend
Iterable<LedgerEntry>. so:
- the implementation can create increase right buffer reference when
creating an iterator
- we can make LedgerEntries a recycleable object, and it is returned to the
tool when people called #close()
- on #close(), it releases all the references to release resources
interface LedgerEntries extends Iterable<LedgerEntry>, AutoCloseable {
long getEntry(long entryId);
Iterator<LedgerEntry> iterator();
}
Re: Iterable
Posted by Sijie Guo <gu...@gmail.com>.
On Thu, Nov 2, 2017 at 10:49 PM, Enrico Olivelli <eo...@gmail.com>
wrote:
>
>
> Il ven 3 nov 2017, 02:39 Sijie Guo <gu...@gmail.com> ha scritto:
>
>> Hi Enrico,
>>
>> In the new ledger api, we return Iterable<LedgerEntry> instead of
>> Iterator<LedgerEntry>. In general, it is good to return Iterable, however
>> it seems to be problematic in following use case:
>>
>> Iterable<LedgerEntry> entries = rh.read(...);
>>
>> Iterator<LedgerEntry> iter1 = entries.iterator();
>> while (iter1.hasNext()) {
>>
>> LedgerEntry le = iter1.next;
>> le.getEntryBuffer().release();
>>
>> }
>>
>> Iterator<LedgerEntry> iter2 = entries.iterator();
>> while (iter2.hasNext()) {
>>
>> LedgerEntry le = iter2.next();
>> le.getEntryBuffer().release();
>>
>> }
>>
>> Using Iterable opens up the chance that people will create multiple
>> iterators and use them concurrently. I think we should not use
>> Iterable<LedgerEntry>, which it might actually cause problems.
>>
>> I am thinking if we can create an interface LedgerEntries to extend
>> Iterable<LedgerEntry>. so:
>>
>> - the implementation can create increase right buffer reference when
>> creating an iterator
>> - we can make LedgerEntries a recycleable object, and it is returned to
>> the tool when people called #close()
>> - on #close(), it releases all the references to release resources
>>
>> interface LedgerEntries extends Iterable<LedgerEntry>, AutoCloseable {
>>
>> long getEntry(long entryId);
>>
>> Iterator<LedgerEntry> iterator();
>>
>>
>> }
>>
>
>
> I think this is a great idea. I remember we already talked about something
> similar when we discovered the need to release buffers for v2 protocol but
> it was not possible for 4.5.
>
filed issue https://github.com/apache/bookkeeper/issues/693 for this.
>
> Enrico
>
>> --
>
>
> -- Enrico Olivelli
>
Re: Iterable
Posted by Enrico Olivelli <eo...@gmail.com>.
Jia
I think we must do this before releasing 4.6
Enrico
Il ven 3 nov 2017, 06:48 Enrico Olivelli <eo...@gmail.com> ha scritto:
>
>
> Il ven 3 nov 2017, 02:39 Sijie Guo <gu...@gmail.com> ha scritto:
>
>> Hi Enrico,
>>
>> In the new ledger api, we return Iterable<LedgerEntry> instead of
>> Iterator<LedgerEntry>. In general, it is good to return Iterable, however
>> it seems to be problematic in following use case:
>>
>> Iterable<LedgerEntry> entries = rh.read(...);
>>
>> Iterator<LedgerEntry> iter1 = entries.iterator();
>> while (iter1.hasNext()) {
>>
>> LedgerEntry le = iter1.next;
>> le.getEntryBuffer().release();
>>
>> }
>>
>> Iterator<LedgerEntry> iter2 = entries.iterator();
>> while (iter2.hasNext()) {
>>
>> LedgerEntry le = iter2.next();
>> le.getEntryBuffer().release();
>>
>> }
>>
>> Using Iterable opens up the chance that people will create multiple
>> iterators and use them concurrently. I think we should not use
>> Iterable<LedgerEntry>, which it might actually cause problems.
>>
>> I am thinking if we can create an interface LedgerEntries to extend
>> Iterable<LedgerEntry>. so:
>>
>> - the implementation can create increase right buffer reference when
>> creating an iterator
>> - we can make LedgerEntries a recycleable object, and it is returned to
>> the tool when people called #close()
>> - on #close(), it releases all the references to release resources
>>
>> interface LedgerEntries extends Iterable<LedgerEntry>, AutoCloseable {
>>
>> long getEntry(long entryId);
>>
>> Iterator<LedgerEntry> iterator();
>>
>>
>> }
>>
>
>
> I think this is a great idea. I remember we already talked about something
> similar when we discovered the need to release buffers for v2 protocol but
> it was not possible for 4.5.
>
> Enrico
>
>> --
>
>
> -- Enrico Olivelli
>
--
-- Enrico Olivelli
Re: Iterable
Posted by Enrico Olivelli <eo...@gmail.com>.
Il ven 3 nov 2017, 02:39 Sijie Guo <gu...@gmail.com> ha scritto:
> Hi Enrico,
>
> In the new ledger api, we return Iterable<LedgerEntry> instead of
> Iterator<LedgerEntry>. In general, it is good to return Iterable, however
> it seems to be problematic in following use case:
>
> Iterable<LedgerEntry> entries = rh.read(...);
>
> Iterator<LedgerEntry> iter1 = entries.iterator();
> while (iter1.hasNext()) {
>
> LedgerEntry le = iter1.next;
> le.getEntryBuffer().release();
>
> }
>
> Iterator<LedgerEntry> iter2 = entries.iterator();
> while (iter2.hasNext()) {
>
> LedgerEntry le = iter2.next();
> le.getEntryBuffer().release();
>
> }
>
> Using Iterable opens up the chance that people will create multiple
> iterators and use them concurrently. I think we should not use
> Iterable<LedgerEntry>, which it might actually cause problems.
>
> I am thinking if we can create an interface LedgerEntries to extend
> Iterable<LedgerEntry>. so:
>
> - the implementation can create increase right buffer reference when
> creating an iterator
> - we can make LedgerEntries a recycleable object, and it is returned to
> the tool when people called #close()
> - on #close(), it releases all the references to release resources
>
> interface LedgerEntries extends Iterable<LedgerEntry>, AutoCloseable {
>
> long getEntry(long entryId);
>
> Iterator<LedgerEntry> iterator();
>
>
> }
>
I think this is a great idea. I remember we already talked about something
similar when we discovered the need to release buffers for v2 protocol but
it was not possible for 4.5.
Enrico
> --
-- Enrico Olivelli