DL LogReader.readNext blocked forever

[Lothruin Mirwen <lo...@gmail.com>]

Hi BookKeepers!

I have an issue with DL LogReader.readNext(false) [ie: blocking]

Such call should "block until return a record if there are records in the
stream (aka catching up). Otherwise it would wait up to {@link
DistributedLogConfiguration#getReadAheadWaitTime()} milliseconds and return
null if there isn't any more records in the stream." (from javadoc).

It normally works but with empty logs (no data ever written to DL/BK) it
blocks forever. It seems that with no data it never "catchUp" and wait
forever.

As workaround I forcefully write a dummy record into the log. I'm missing
something? Is there any missing configuration I need to provide to BK?

Best regards

Diego Salvi

Re: DL LogReader.readNext blocked forever

[Lothruin Mirwen <lo...@gmail.com>]

Actually I create logs and writer through Namespace:

Namespace namespace = ...
if (!namespace.logExists(logName)) {
        namespace.createLog(logName);
}
DistributedLogManager logManager = namespace.openLog(logName);
AsyncLogWriter asyncLogWriter =
FutureUtils.result(logManager.openAsyncLogWriter());


Then I take a LogReader and start reads

// Actually reader is opened just one time and then reused.
LogReader reader = logManager.openLogReader(DLSN.NonInclusiveLowerBound);

LogRecordWithDLSN record;
while ((record = reader.readNext(!blocking)) != null) {
        ... business logic ... (i was expecting to get a null and exit from
reading after some timeout)
}


That reader.readNext blocks until I write something into the log:

// Generate the record

ByteBuf buf = Unpooled.wrappedBuffer(os.getByteArray(), 0, os.size());
LogRecord record = new LogRecord(System.currentTimeMillis(), buf);

// Actual write request

final CompletableFuture<DLSN> write = asyncLogWriter.write(record);



Il giorno mar 26 feb 2019 alle ore 12:44 Sijie Guo <gu...@gmail.com> ha
scritto:

> On Tue, Feb 26, 2019 at 6:37 PM Lothruin Mirwen <lothruin.mirwen@gmail.com
> >
> wrote:
>
> > Hi BookKeepers!
> >
> > I have an issue with DL LogReader.readNext(false) [ie: blocking]
> >
> > Such call should "block until return a record if there are records in the
> > stream (aka catching up). Otherwise it would wait up to {@link
> > DistributedLogConfiguration#getReadAheadWaitTime()} milliseconds and
> return
> > null if there isn't any more records in the stream." (from javadoc).
> >
> > It normally works but with empty logs (no data ever written to DL/BK) it
> > blocks forever. It seems that with no data it never "catchUp" and wait
> > forever.
> >
>
> How do you create the empty logs? Did you just open a log writer without
> writing records, or you use Namespace to create a log?
>
> The difference is 1) first approach will have a log segment with zero
> entries; 2) second approach might not even have any log segments.
>
> I guess the difference between 1) and 2) cause the differences at handling
> `readNext(false)`. If you can share more details about your setup, I have a
> better idea of the direction at looking into the problem.
>
>
> > As workaround I forcefully write a dummy record into the log. I'm missing
> > something? Is there any missing configuration I need to provide to BK?
> >
> > Best regards
> >
> > Diego Salvi
> >
>

Re: DL LogReader.readNext blocked forever

[Sijie Guo <gu...@gmail.com>]

On Tue, Feb 26, 2019 at 6:37 PM Lothruin Mirwen <lo...@gmail.com>
wrote:

> Hi BookKeepers!
>
> I have an issue with DL LogReader.readNext(false) [ie: blocking]
>
> Such call should "block until return a record if there are records in the
> stream (aka catching up). Otherwise it would wait up to {@link
> DistributedLogConfiguration#getReadAheadWaitTime()} milliseconds and return
> null if there isn't any more records in the stream." (from javadoc).
>
> It normally works but with empty logs (no data ever written to DL/BK) it
> blocks forever. It seems that with no data it never "catchUp" and wait
> forever.
>

How do you create the empty logs? Did you just open a log writer without
writing records, or you use Namespace to create a log?

The difference is 1) first approach will have a log segment with zero
entries; 2) second approach might not even have any log segments.

I guess the difference between 1) and 2) cause the differences at handling
`readNext(false)`. If you can share more details about your setup, I have a
better idea of the direction at looking into the problem.


> As workaround I forcefully write a dummy record into the log. I'm missing
> something? Is there any missing configuration I need to provide to BK?
>
> Best regards
>
> Diego Salvi
>