You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "ddanielr (via GitHub)" <gi...@apache.org> on 2024/12/10 16:52:08 UTC
[I] Add variable to resource groups for "dynamic" vs "static" scaling of tservers so tracking "Dead tservers" is disabled per resource group. [accumulo]
ddanielr opened a new issue, #5162:
URL: https://github.com/apache/accumulo/issues/5162
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support differentiating between static and dynamic resource groups [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion closed issue #5162: Support differentiating between static and dynamic resource groups
URL: https://github.com/apache/accumulo/issues/5162
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support differentiating between static and dynamic resource groups [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion closed issue #5162: Support differentiating between static and dynamic resource groups
URL: https://github.com/apache/accumulo/issues/5162
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support differentiating between static and dynamic resource groups [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on issue #5162:
URL: https://github.com/apache/accumulo/issues/5162#issuecomment-2546373523
Another way to approach this would be to remove the tablet servers ZooKeeper entry on a normal shutdown. I believe that the tserver did not do this in earlier (< 4.0) versions because the configuration when using cluster.yaml because the deployment was meant to be static.
In fact, this is a larger issue when a users uses something like Kubernetes. It's possible that old server paths might linger for all the server processes. We may want to consider removing the server paths from ZooKeeper for all server types when performing a normal shutdown.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support differentiating between static and dynamic resource groups [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on issue #5162:
URL: https://github.com/apache/accumulo/issues/5162#issuecomment-2555166403
> Another way to approach this would be to remove the tablet servers ZooKeeper entry on a normal shutdown.
In the ServiceLock.unlock code (below) in the case of Compactors, Scan Servers, and TabletServers `path` is `/accumulo/<instanceId>/(compactors|sservers|tservers)/<resourceGroup>/<address>` and `localLock` is the ephemeral node that is created. The code is currently only deleting the ephemeral node.
We could modify `ServiceLock.unlock` to accept a boolean, which when `true` could perform a recursive delete on `path`. We would only set it to `true` in the case of Compactors, ScanServers, and TabletServers when ServiceLock.unlock is called. This would mean that the LiveTServerSet running in the Manager would not find any TabletServers without locks when they are shut down gracefully. Graceful shutdown is being addressed in #5193.
https://github.com/apache/accumulo/blob/e5370d1d5cd436d86170d16b0e8c0986d3275e15/core/src/main/java/org/apache/accumulo/core/lock/ServiceLock.java#L531-L558
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support differentiating between static and dynamic resource groups [accumulo]
Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on issue #5162:
URL: https://github.com/apache/accumulo/issues/5162#issuecomment-2578379789
> I created #5226 to suppress the display of dead servers for configured resource groups in the Monitor. I think this is the simplest approach at this point. My earlier comments about removing the lock in ZooKeeper may not fully resolve the issue you raise.
Agreed, I think that #5226 is a good solution for this issue as it's not tied to the cluster.yaml definition and also covers the situation where the user might want to shut down tservers faster than the "graceful shutdown" process allows and would end up generating dead tserver locks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support differentiating between static and dynamic resource groups [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on issue #5162:
URL: https://github.com/apache/accumulo/issues/5162#issuecomment-2539389401
From what I can tell, the logic in `LiveTServerSet.checkServer` only adds the tablet server to the dead list if the tablet server path in ZooKeeper exists but there is no lock data, or if the instance does not match (like the tserver was restarted at the same host/port location). The ZooKeeper node for the tserver is deleted after 10 minutes if there is no lock data.
This information is only shown on the Monitor. We could add a Monitor property that is a list of resource groups to ignore, then use that when calling `DeadServerList.getList` to skip dead servers with a matching resource group in the ServiceLockPath.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support differentiating between static and dynamic resource groups [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on issue #5162:
URL: https://github.com/apache/accumulo/issues/5162#issuecomment-2574106327
I created #5226 to suppress the display of dead servers for configured resource groups in the Monitor. I think this is the simplest approach at this point. My earlier comments about removing the lock in ZooKeeper may not fully resolve the issue you raise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org