uassigned tablets issue

["Ligade, Shailesh [USA]" <Li...@bah.com>]

Hello,

For some strange reason, monitor master page (version 1.10.0) shows unassigned tablets going up and down, typically goes up to 4 and down to 0
The master log shows unassigned tablet count and waiting for it to go to zero.

This just started happening, is this a cause for concern? I checked for tabletserverlocks, checkformetadataproblems, and other usual debugging commands, everything looks good.

-S

Re: uassigned tablets issue

[dev1 <de...@etcoleman.com>]

Unassigned tablets for a short duration is not unusual.  Tablets can become "unassigned" for a number of reasons during normal operations, including, but not limited to:

  *   tserver failure - all tablets assigned to the newly deceased tserver will be reassigned.
  *   balancer migrations - the balancer will try to even out tablet assignments and will select a number of tablets from "busy" servers to servers that look like they have spare capacity - the exact behavior will be decided by the balancer that you have configured.
  *   new table / add splits.  If you create a table and then add splits, the splits first land on the tserver where the table is first assigned - the balancer will then migrate these over time.  To for a faster rebalance, on 1.x line, you can create the table, add splits, offline the table, online the table - this will force the splits to be assigned throughout the cluster without need to wait for the balancer - this can help distribute ingest if you are going to immediate start ingests after the table creation..  The is an option to create and add splits to an offline table, but I'm not sure that is available in 1.10.
  *   tablet(s) exceed splits threshold - if during ingest, either continuous (batch writes) or bulk imports, if a tablet size exceeds the tablet split threshold, the tablet will split into 1 or more tablets - the balance may then see this as unbalanced and begin a migration.

The best way to determine what is occurring should be in the master / main debug log - you may want to filter out replication related messages if that helps you find the relevant messages.


________________________________
From: Shailesh Ligade <SL...@FBI.GOV>
Sent: Wednesday, January 26, 2022 11:52 AM
To: 'user@accumulo.apache.org' <us...@accumulo.apache.org>
Subject: Re: uassigned tablets issue

Sorry for late reply, from the log i can't tell if it is migrating tablets..we have replication on so logs are very chatty, but whatever i saw I didn't see it as migrating tablets.

Thanks

S
________________________________
From: dev1 <de...@etcoleman.com>
Sent: Tuesday, January 25, 2022 10:30 AM
To: 'user@accumulo.apache.org' <us...@accumulo.apache.org>
Subject: [EXTERNAL EMAIL] - RE: uassigned tablets issue


Can you look at the master log?  Do you see migrations occurring?



Ed Coleman



From: Ligade, Shailesh [USA] <Li...@bah.com>
Sent: Tuesday, January 25, 2022 9:23 AM
To: user@accumulo.apache.org
Subject: uassigned tablets issue



Hello,



For some strange reason, monitor master page (version 1.10.0) shows unassigned tablets going up and down, typically goes up to 4 and down to 0

The master log shows unassigned tablet count and waiting for it to go to zero.



This just started happening, is this a cause for concern? I checked for tabletserverlocks, checkformetadataproblems, and other usual debugging commands, everything looks good.



-S

Re: uassigned tablets issue

[Shailesh Ligade <SL...@FBI.GOV>]

Sorry for late reply, from the log i can't tell if it is migrating tablets..we have replication on so logs are very chatty, but whatever i saw I didn't see it as migrating tablets.

Thanks

S
________________________________
From: dev1 <de...@etcoleman.com>
Sent: Tuesday, January 25, 2022 10:30 AM
To: 'user@accumulo.apache.org' <us...@accumulo.apache.org>
Subject: [EXTERNAL EMAIL] - RE: uassigned tablets issue


Can you look at the master log?  Do you see migrations occurring?



Ed Coleman



From: Ligade, Shailesh [USA] <Li...@bah.com>
Sent: Tuesday, January 25, 2022 9:23 AM
To: user@accumulo.apache.org
Subject: uassigned tablets issue



Hello,



For some strange reason, monitor master page (version 1.10.0) shows unassigned tablets going up and down, typically goes up to 4 and down to 0

The master log shows unassigned tablet count and waiting for it to go to zero.



This just started happening, is this a cause for concern? I checked for tabletserverlocks, checkformetadataproblems, and other usual debugging commands, everything looks good.



-S

RE: uassigned tablets issue

[dev1 <de...@etcoleman.com>]

Can you look at the master log?  Do you see migrations occurring?

Ed Coleman

From: Ligade, Shailesh [USA] <Li...@bah.com>
Sent: Tuesday, January 25, 2022 9:23 AM
To: user@accumulo.apache.org
Subject: uassigned tablets issue

Hello,

For some strange reason, monitor master page (version 1.10.0) shows unassigned tablets going up and down, typically goes up to 4 and down to 0
The master log shows unassigned tablet count and waiting for it to go to zero.

This just started happening, is this a cause for concern? I checked for tabletserverlocks, checkformetadataproblems, and other usual debugging commands, everything looks good.

-S