You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "ivakegg (via GitHub)" <gi...@apache.org> on 2023/04/28 15:44:09 UTC
[GitHub] [accumulo] ivakegg opened a new issue, #3363: The accumulo monitor no longer clears out dead tablet servers when they are brought back
ivakegg opened a new issue, #3363:
URL: https://github.com/apache/accumulo/issues/3363
The accumulo monitor in 1.10 would clear out a tablet server from the dead tablet servers list when it was brought back on line. This is critical in our systems where we rely on that in operations.
You should be able to reproduce this by starting a system, taking down the tserver, see it in the monitor dead tablet server list, and then bring the server back up. I would expect the dead tablet server line in the monitor to go away like it used to.
accumulo 2.1.1-SNAPSHOT
CentOS 7.3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo commented on issue #3363: The accumulo monitor no longer clears out dead tablet servers when they are brought back
Posted by "DomGarguilo (via GitHub)" <gi...@apache.org>.
DomGarguilo commented on issue #3363:
URL: https://github.com/apache/accumulo/issues/3363#issuecomment-1535231966
I will start looking into this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ivakegg closed issue #3363: The accumulo monitor no longer clears out dead tablet servers when they are brought back
Posted by "ivakegg (via GitHub)" <gi...@apache.org>.
ivakegg closed issue #3363: The accumulo monitor no longer clears out dead tablet servers when they are brought back
URL: https://github.com/apache/accumulo/issues/3363
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #3363: The accumulo monitor no longer clears out dead tablet servers when they are brought back
Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on issue #3363:
URL: https://github.com/apache/accumulo/issues/3363#issuecomment-1527777664
This bug sounds familiar. I thought we addressed this, or something very similar, in a previous bugfix for 2.0 or 2.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #3363: The accumulo monitor no longer clears out dead tablet servers when they are brought back
Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on issue #3363:
URL: https://github.com/apache/accumulo/issues/3363#issuecomment-1536005346
> I believe this may be because when we do a accumulo-cluster start, all of the tservers start up with the short hostname. When I restart a server using accumulo-service tserver start on the nodes where a tserver fell over, then it start with the fully qualified hostname instead. Hence the monitor may not be figuring out that the tserver has been restarted.
Do you mean the hostname you're specifying in the accumulo-cluster's `conf/cluster.yaml` file is different from the hostname that is automatically determined by Accumulo? Is the monitor showing the newly started servers with their FQDN?
If you can confirm this is being caused by that discrepancy on how the server is identified, that would be helpful. However, if that is the case, I'm not sure there's much we can do about this, because it's possible you could have one host named "something" and another host named "something.else", and Accumulo has no way of knowing whether "something" and "something.else" are the same hosts or separate. That's up to the name service, which might not even be configured the same on every node. Accumulo just has to use what is provided in the config, or else try to auto-discover it on the node.
If you prefer the short name, you could try starting with the preferred short name on the host. That's all the `accumulo-cluster` script does, getting the preferred name from its config file. But, the `accumulo-service` script is much simpler, and isn't aware of the config file. It just does what it's explicitly told to do. If not provided a preferred name, it will try to auto-discover it. You can specify it using:
```bash
$ bin/accumulo-service tserver start -o general.process.bind.addr=$(hostname -s)
```
Or, you can set `general.process.bind.addr` in `conf/accumulo.properties` to use an environment variable, as in `general.process.bind.addr=${env:SHORTHOSTNAME}`, and set the environment in `conf/accumulo-env.sh` with something like `SHORTHOSTNAME=$(hostname -s)`
(Note: the configuration for the bind address name replaces the old `-a` command you might be familiar with. In future, we'll probably add a new config option to specify the advertisement address separately from the bind address. Right now, the service advertisement address assumes that it's the same as the bind address, as it did with `-a`.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ivakegg commented on issue #3363: The accumulo monitor no longer clears out dead tablet servers when they are brought back
Posted by "ivakegg (via GitHub)" <gi...@apache.org>.
ivakegg commented on issue #3363:
URL: https://github.com/apache/accumulo/issues/3363#issuecomment-1534710580
I believe this may be because when we do a accumulo-cluster start, all of the tservers start up with the short hostname. When I restart a server using accumulo-service tserver start on the nodes where a tserver fell over, then it start with the fully qualified hostname instead. Hence the monitor may not be figuring out that the tserver has been restarted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ivakegg commented on issue #3363: The accumulo monitor no longer clears out dead tablet servers when they are brought back
Posted by "ivakegg (via GitHub)" <gi...@apache.org>.
ivakegg commented on issue #3363:
URL: https://github.com/apache/accumulo/issues/3363#issuecomment-1545706158
ok, we may update the accumulo-service command to force it to use the short name. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org