You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "keith-turner (via GitHub)" <gi...@apache.org> on 2023/07/31 17:00:13 UTC
[GitHub] [accumulo] keith-turner opened a new issue, #3670: Explore moving all tablet in memory state an synchronization into a single class
keith-turner opened a new issue, #3670:
URL: https://github.com/apache/accumulo/issues/3670
The Tablet code has a mixture of I/O code and in memory state (like active walogs, active in memory maps, current files, minor compactions in progress, refreshes in progress, etc). Its hard to analyze this synchronization code to ensure things like does any I/O happen while holding a lock (which is not desired).
Moving all of the tablets in memory state+sync into a single class that could be unit tested and analyzed may be one way to improve this.
Doing this work in the elasticity branch will be easier as a lot of functionality has been removed from the tablet server.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Explore moving all tablet in memory state an synchronization into a single class [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on issue #3670:
URL: https://github.com/apache/accumulo/issues/3670#issuecomment-2251259108
> Should this target an earlier branch? Maybe even a 2.1.x?
It would be more difficult to make this change in 2.1 because the Tablet has alot more code related to synchronizing compaction, splits, and bulk imports that does not exists in 4.0.0. So I think its good to do the refactoring for 4.0.0 or later. If any existing bugs are noticed during the refactoring, then those could possibly be fixed in 2.1.x.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Explore moving all tablet in memory state an synchronization into a single class [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on issue #3670:
URL: https://github.com/apache/accumulo/issues/3670#issuecomment-2176937770
While working on trying to determine the best angle to get started with this, one important thing I realized is that a tablet only has the following types of threads that interact with it.
* Scan thread
* Write thread
* Minor compaction thread
* Refresh thread
* Close thread
* Instantiation thread that creates a tablet on a tserver
Prior to elasticity there were more types of threads to worry about. The threads above need to synchronize on different in memory state and this interleaved with I/O. Started experimenting with [this structure](https://github.com/keith-turner/accumulo/blob/60b129d1f0999ff56942d793e74bab5ba012ee65/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/TabletState.java). The idea behind this structure is that each of the thread type uses a coordination object to interact with the shared in memory state of the tablet. Maybe this structure would cleanly separate the in memory state+sync from the I/O+compute code, not sure though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] EdColeman commented on issue #3670: Explore moving all tablet in memory state an synchronization into a single class
Posted by "EdColeman (via GitHub)" <gi...@apache.org>.
EdColeman commented on issue #3670:
URL: https://github.com/apache/accumulo/issues/3670#issuecomment-1660255178
Should this target an earlier branch? Maybe even a 2.1.x? While it may be easier to implement for elasticity, if this is addressing a source of possible race conditions (so, a bug fix) or reduces the change of introducing errors, having this earlier would benefit users (and help us) until elasticity is adopted. 2.1.x is an LTM release, so we are going to need to support it for quite a while.
This may be more complicated for something targeting 2.1.2, but targeting a 2.1.3 and then merging forward seems that it could provide long-term improvements for this complicated code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org