You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Álvaro Recuero <al...@gmail.com> on 2013/05/27 14:15:03 UTC
Replication warning
Does anyone get this when replication is enabled in HBase? If it is a
warning but does not stop coming up, I wonder what that loop is that
iterating through and if this is really a critical problem.
cheers,
Álvaro
2013-05-27 13:10:40,406 WARN
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Waited
too long for this file, considering dumping
2013-05-27 13:10:40,406 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Unable
to open a reader, sleeping 1000 times 10
2013-05-27 13:10:50,411 WARN
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: 1 Got:
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:55)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:177)
at org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:721)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.openReader(ReplicationHLogReaderManager.java:66)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:562)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:339)
Re: Replication warning
Posted by Álvaro Recuero <al...@gmail.com>.
Thank you Ted. I have patched and the warning is gone now, that's right. I
will take a look as well at 0.94.8 then if you think is worthwhile for
replication purposes, although I would try to stick to my current set up if
possible.
cheers,
Álvaro
On 28 May 2013 13:41, Ted Yu <yu...@gmail.com> wrote:
> You can also try upgrading to 0.94.8 - there are other fixes for
> replication that you will benefit.
>
> Cheers
>
> On May 28, 2013, at 5:23 AM, Álvaro Recuero <al...@gmail.com> wrote:
>
> > Hi Ted,
> >
> > I see, let me check that out then and try to integrate that patch-code
> into
> > my release. For sure it will make a difference just by seeing the lines
> in
> > the patch mentioned. Thank you.
> >
> > Best,
> > Álvaro
> >
> >
> > On 28 May 2013 04:26, Ted Yu <yu...@gmail.com> wrote:
> >
> >> There was a recently integrated JIRA (0.94.8):
> >> HBASE-7122 Proper warning message when opening a log file with no
> entries
> >> (idle cluster)
> >>
> >> The stack trace in above JIRA looks very similar to yours.
> >>
> >> On Mon, May 27, 2013 at 8:40 AM, Álvaro Recuero <al...@gmail.com>
> >> wrote:
> >>
> >>> Hi All,
> >>>
> >>> From the shell:
> >>>
> >>> ~/hbase-0.94.4$ ./bin/hbase shell
> >>> HBase Shell; enter 'help<RETURN>' for list of supported commands.
> >>> Type "exit<RETURN>" to leave the HBase Shell
> >>> Version 0.94.4, r, Mon May 27 15:29:11 WEST 2013
> >>>
> >>> I have seen today it comes up as soon as there is a new item to be
> >>> replicated, and even happens if an old Hlog is rolled. I am not really
> >> sure
> >>> what is going there behind the scenes of Hbase.
> >>>
> >>> Regards,
> >>> Álvaro
> >>>
> >>> On 27 May 2013 14:43, Ted Yu <yu...@gmail.com> wrote:
> >>>
> >>>> The warning came from this loop in ReplicationSource:
> >>>>
> >>>> // Loop until we close down
> >>>> while (isActive()) {
> >>>> ...
> >>>> // Open a reader on it
> >>>> if (!openReader(sleepMultiplier)) {
> >>>> // Reset the sleep multiplier, else it'd be reused for the next
> >>>> file
> >>>> sleepMultiplier = 1;
> >>>> continue;
> >>>> }
> >>>>
> >>>> // If we got a null reader but didn't continue, then sleep and
> >>>> continue
> >>>> if (this.reader == null) {
> >>>> if (sleepForRetries("Unable to open a reader",
> >> sleepMultiplier))
> >>> {
> >>>> sleepMultiplier++;
> >>>> }
> >>>> continue;
> >>>> }
> >>>>
> >>>> How often did the following warning come up in the server log ?
> >>>> Which version of HBase are you using ?
> >>>>
> >>>> Cheers
> >>>>
> >>>> On Mon, May 27, 2013 at 5:15 AM, Álvaro Recuero <al...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Does anyone get this when replication is enabled in HBase? If it is a
> >>>>> warning but does not stop coming up, I wonder what that loop is that
> >>>>> iterating through and if this is really a critical problem.
> >>>>>
> >>>>> cheers,
> >>>>> Álvaro
> >>>>>
> >>>>> 2013-05-27 13:10:40,406 WARN
> >>>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> >>>> Waited
> >>>>> too long for this file, considering dumping
> >>>>> 2013-05-27 13:10:40,406 DEBUG
> >>>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> >>>> Unable
> >>>>> to open a reader, sleeping 1000 times 10
> >>>>> 2013-05-27 13:10:50,411 WARN
> >>>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: 1
> >>>> Got:
> >>>>> java.io.EOFException
> >>>>> at java.io.DataInputStream.readFully(DataInputStream.java:197)
> >>>>> at java.io.DataInputStream.readFully(DataInputStream.java:169)
> >>>>> at
> >>> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> >>>>> at
> >>>>
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
> >>>>> at
> >>>>
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
> >>>>> at
> >>>>
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
> >>>>> at
> >>
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:55)
> >>>>> at
> >>
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:177)
> >>>>> at
> >>> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:721)
> >>>>> at
> >>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.openReader(ReplicationHLogReaderManager.java:66)
> >>>>> at
> >>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:562)
> >>>>> at
> >>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:339)
> >>
>
Re: Replication warning
Posted by Ted Yu <yu...@gmail.com>.
You can also try upgrading to 0.94.8 - there are other fixes for replication that you will benefit.
Cheers
On May 28, 2013, at 5:23 AM, Álvaro Recuero <al...@gmail.com> wrote:
> Hi Ted,
>
> I see, let me check that out then and try to integrate that patch-code into
> my release. For sure it will make a difference just by seeing the lines in
> the patch mentioned. Thank you.
>
> Best,
> Álvaro
>
>
> On 28 May 2013 04:26, Ted Yu <yu...@gmail.com> wrote:
>
>> There was a recently integrated JIRA (0.94.8):
>> HBASE-7122 Proper warning message when opening a log file with no entries
>> (idle cluster)
>>
>> The stack trace in above JIRA looks very similar to yours.
>>
>> On Mon, May 27, 2013 at 8:40 AM, Álvaro Recuero <al...@gmail.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> From the shell:
>>>
>>> ~/hbase-0.94.4$ ./bin/hbase shell
>>> HBase Shell; enter 'help<RETURN>' for list of supported commands.
>>> Type "exit<RETURN>" to leave the HBase Shell
>>> Version 0.94.4, r, Mon May 27 15:29:11 WEST 2013
>>>
>>> I have seen today it comes up as soon as there is a new item to be
>>> replicated, and even happens if an old Hlog is rolled. I am not really
>> sure
>>> what is going there behind the scenes of Hbase.
>>>
>>> Regards,
>>> Álvaro
>>>
>>> On 27 May 2013 14:43, Ted Yu <yu...@gmail.com> wrote:
>>>
>>>> The warning came from this loop in ReplicationSource:
>>>>
>>>> // Loop until we close down
>>>> while (isActive()) {
>>>> ...
>>>> // Open a reader on it
>>>> if (!openReader(sleepMultiplier)) {
>>>> // Reset the sleep multiplier, else it'd be reused for the next
>>>> file
>>>> sleepMultiplier = 1;
>>>> continue;
>>>> }
>>>>
>>>> // If we got a null reader but didn't continue, then sleep and
>>>> continue
>>>> if (this.reader == null) {
>>>> if (sleepForRetries("Unable to open a reader",
>> sleepMultiplier))
>>> {
>>>> sleepMultiplier++;
>>>> }
>>>> continue;
>>>> }
>>>>
>>>> How often did the following warning come up in the server log ?
>>>> Which version of HBase are you using ?
>>>>
>>>> Cheers
>>>>
>>>> On Mon, May 27, 2013 at 5:15 AM, Álvaro Recuero <al...@gmail.com>
>>>> wrote:
>>>>
>>>>> Does anyone get this when replication is enabled in HBase? If it is a
>>>>> warning but does not stop coming up, I wonder what that loop is that
>>>>> iterating through and if this is really a critical problem.
>>>>>
>>>>> cheers,
>>>>> Álvaro
>>>>>
>>>>> 2013-05-27 13:10:40,406 WARN
>>>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
>>>> Waited
>>>>> too long for this file, considering dumping
>>>>> 2013-05-27 13:10:40,406 DEBUG
>>>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
>>>> Unable
>>>>> to open a reader, sleeping 1000 times 10
>>>>> 2013-05-27 13:10:50,411 WARN
>>>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: 1
>>>> Got:
>>>>> java.io.EOFException
>>>>> at java.io.DataInputStream.readFully(DataInputStream.java:197)
>>>>> at java.io.DataInputStream.readFully(DataInputStream.java:169)
>>>>> at
>>> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
>>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
>>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
>>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
>>>>> at
>> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:55)
>>>>> at
>> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:177)
>>>>> at
>>> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:721)
>>>>> at
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.openReader(ReplicationHLogReaderManager.java:66)
>>>>> at
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:562)
>>>>> at
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:339)
>>
Re: Replication warning
Posted by Álvaro Recuero <al...@gmail.com>.
Hi Ted,
I see, let me check that out then and try to integrate that patch-code into
my release. For sure it will make a difference just by seeing the lines in
the patch mentioned. Thank you.
Best,
Álvaro
On 28 May 2013 04:26, Ted Yu <yu...@gmail.com> wrote:
> There was a recently integrated JIRA (0.94.8):
> HBASE-7122 Proper warning message when opening a log file with no entries
> (idle cluster)
>
> The stack trace in above JIRA looks very similar to yours.
>
> On Mon, May 27, 2013 at 8:40 AM, Álvaro Recuero <al...@gmail.com>
> wrote:
>
> > Hi All,
> >
> > From the shell:
> >
> > ~/hbase-0.94.4$ ./bin/hbase shell
> > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> > Type "exit<RETURN>" to leave the HBase Shell
> > Version 0.94.4, r, Mon May 27 15:29:11 WEST 2013
> >
> > I have seen today it comes up as soon as there is a new item to be
> > replicated, and even happens if an old Hlog is rolled. I am not really
> sure
> > what is going there behind the scenes of Hbase.
> >
> > Regards,
> > Álvaro
> >
> > On 27 May 2013 14:43, Ted Yu <yu...@gmail.com> wrote:
> >
> > > The warning came from this loop in ReplicationSource:
> > >
> > > // Loop until we close down
> > > while (isActive()) {
> > > ...
> > > // Open a reader on it
> > > if (!openReader(sleepMultiplier)) {
> > > // Reset the sleep multiplier, else it'd be reused for the next
> > > file
> > > sleepMultiplier = 1;
> > > continue;
> > > }
> > >
> > > // If we got a null reader but didn't continue, then sleep and
> > > continue
> > > if (this.reader == null) {
> > > if (sleepForRetries("Unable to open a reader",
> sleepMultiplier))
> > {
> > > sleepMultiplier++;
> > > }
> > > continue;
> > > }
> > >
> > > How often did the following warning come up in the server log ?
> > > Which version of HBase are you using ?
> > >
> > > Cheers
> > >
> > > On Mon, May 27, 2013 at 5:15 AM, Álvaro Recuero <al...@gmail.com>
> > > wrote:
> > >
> > > > Does anyone get this when replication is enabled in HBase? If it is a
> > > > warning but does not stop coming up, I wonder what that loop is that
> > > > iterating through and if this is really a critical problem.
> > > >
> > > > cheers,
> > > > Álvaro
> > > >
> > > > 2013-05-27 13:10:40,406 WARN
> > > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Waited
> > > > too long for this file, considering dumping
> > > > 2013-05-27 13:10:40,406 DEBUG
> > > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Unable
> > > > to open a reader, sleeping 1000 times 10
> > > > 2013-05-27 13:10:50,411 WARN
> > > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: 1
> > > Got:
> > > > java.io.EOFException
> > > > at java.io.DataInputStream.readFully(DataInputStream.java:197)
> > > > at java.io.DataInputStream.readFully(DataInputStream.java:169)
> > > > at
> > org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> > > > at
> > > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
> > > > at
> > > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
> > > > at
> > > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:55)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:177)
> > > > at
> > org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:721)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.openReader(ReplicationHLogReaderManager.java:66)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:562)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:339)
> > > >
> > >
> >
>
Re: Replication warning
Posted by Ted Yu <yu...@gmail.com>.
There was a recently integrated JIRA (0.94.8):
HBASE-7122 Proper warning message when opening a log file with no entries
(idle cluster)
The stack trace in above JIRA looks very similar to yours.
On Mon, May 27, 2013 at 8:40 AM, Álvaro Recuero <al...@gmail.com> wrote:
> Hi All,
>
> From the shell:
>
> ~/hbase-0.94.4$ ./bin/hbase shell
> HBase Shell; enter 'help<RETURN>' for list of supported commands.
> Type "exit<RETURN>" to leave the HBase Shell
> Version 0.94.4, r, Mon May 27 15:29:11 WEST 2013
>
> I have seen today it comes up as soon as there is a new item to be
> replicated, and even happens if an old Hlog is rolled. I am not really sure
> what is going there behind the scenes of Hbase.
>
> Regards,
> Álvaro
>
> On 27 May 2013 14:43, Ted Yu <yu...@gmail.com> wrote:
>
> > The warning came from this loop in ReplicationSource:
> >
> > // Loop until we close down
> > while (isActive()) {
> > ...
> > // Open a reader on it
> > if (!openReader(sleepMultiplier)) {
> > // Reset the sleep multiplier, else it'd be reused for the next
> > file
> > sleepMultiplier = 1;
> > continue;
> > }
> >
> > // If we got a null reader but didn't continue, then sleep and
> > continue
> > if (this.reader == null) {
> > if (sleepForRetries("Unable to open a reader", sleepMultiplier))
> {
> > sleepMultiplier++;
> > }
> > continue;
> > }
> >
> > How often did the following warning come up in the server log ?
> > Which version of HBase are you using ?
> >
> > Cheers
> >
> > On Mon, May 27, 2013 at 5:15 AM, Álvaro Recuero <al...@gmail.com>
> > wrote:
> >
> > > Does anyone get this when replication is enabled in HBase? If it is a
> > > warning but does not stop coming up, I wonder what that loop is that
> > > iterating through and if this is really a critical problem.
> > >
> > > cheers,
> > > Álvaro
> > >
> > > 2013-05-27 13:10:40,406 WARN
> > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > Waited
> > > too long for this file, considering dumping
> > > 2013-05-27 13:10:40,406 DEBUG
> > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > Unable
> > > to open a reader, sleeping 1000 times 10
> > > 2013-05-27 13:10:50,411 WARN
> > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: 1
> > Got:
> > > java.io.EOFException
> > > at java.io.DataInputStream.readFully(DataInputStream.java:197)
> > > at java.io.DataInputStream.readFully(DataInputStream.java:169)
> > > at
> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> > > at
> > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
> > > at
> > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
> > > at
> > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:55)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:177)
> > > at
> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:721)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.openReader(ReplicationHLogReaderManager.java:66)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:562)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:339)
> > >
> >
>
Re: Replication warning
Posted by Álvaro Recuero <al...@gmail.com>.
Hi All,
>From the shell:
~/hbase-0.94.4$ ./bin/hbase shell
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.94.4, r, Mon May 27 15:29:11 WEST 2013
I have seen today it comes up as soon as there is a new item to be
replicated, and even happens if an old Hlog is rolled. I am not really sure
what is going there behind the scenes of Hbase.
Regards,
Álvaro
On 27 May 2013 14:43, Ted Yu <yu...@gmail.com> wrote:
> The warning came from this loop in ReplicationSource:
>
> // Loop until we close down
> while (isActive()) {
> ...
> // Open a reader on it
> if (!openReader(sleepMultiplier)) {
> // Reset the sleep multiplier, else it'd be reused for the next
> file
> sleepMultiplier = 1;
> continue;
> }
>
> // If we got a null reader but didn't continue, then sleep and
> continue
> if (this.reader == null) {
> if (sleepForRetries("Unable to open a reader", sleepMultiplier)) {
> sleepMultiplier++;
> }
> continue;
> }
>
> How often did the following warning come up in the server log ?
> Which version of HBase are you using ?
>
> Cheers
>
> On Mon, May 27, 2013 at 5:15 AM, Álvaro Recuero <al...@gmail.com>
> wrote:
>
> > Does anyone get this when replication is enabled in HBase? If it is a
> > warning but does not stop coming up, I wonder what that loop is that
> > iterating through and if this is really a critical problem.
> >
> > cheers,
> > Álvaro
> >
> > 2013-05-27 13:10:40,406 WARN
> > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Waited
> > too long for this file, considering dumping
> > 2013-05-27 13:10:40,406 DEBUG
> > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Unable
> > to open a reader, sleeping 1000 times 10
> > 2013-05-27 13:10:50,411 WARN
> > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: 1
> Got:
> > java.io.EOFException
> > at java.io.DataInputStream.readFully(DataInputStream.java:197)
> > at java.io.DataInputStream.readFully(DataInputStream.java:169)
> > at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> > at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
> > at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
> > at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
> > at
> >
> >
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:55)
> > at
> >
> >
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:177)
> > at org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:721)
> > at
> >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.openReader(ReplicationHLogReaderManager.java:66)
> > at
> >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:562)
> > at
> >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:339)
> >
>
Re: Replication warning
Posted by Ted Yu <yu...@gmail.com>.
The warning came from this loop in ReplicationSource:
// Loop until we close down
while (isActive()) {
...
// Open a reader on it
if (!openReader(sleepMultiplier)) {
// Reset the sleep multiplier, else it'd be reused for the next file
sleepMultiplier = 1;
continue;
}
// If we got a null reader but didn't continue, then sleep and
continue
if (this.reader == null) {
if (sleepForRetries("Unable to open a reader", sleepMultiplier)) {
sleepMultiplier++;
}
continue;
}
How often did the following warning come up in the server log ?
Which version of HBase are you using ?
Cheers
On Mon, May 27, 2013 at 5:15 AM, Álvaro Recuero <al...@gmail.com> wrote:
> Does anyone get this when replication is enabled in HBase? If it is a
> warning but does not stop coming up, I wonder what that loop is that
> iterating through and if this is really a critical problem.
>
> cheers,
> Álvaro
>
> 2013-05-27 13:10:40,406 WARN
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Waited
> too long for this file, considering dumping
> 2013-05-27 13:10:40,406 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Unable
> to open a reader, sleeping 1000 times 10
> 2013-05-27 13:10:50,411 WARN
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: 1 Got:
> java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:197)
> at java.io.DataInputStream.readFully(DataInputStream.java:169)
> at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
> at
>
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:55)
> at
>
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:177)
> at org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:721)
> at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.openReader(ReplicationHLogReaderManager.java:66)
> at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:562)
> at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:339)
>