Hint transfer during decommission

[Caleb Rackliffe <ca...@gmail.com>]

Hey friends,

I'm looking for some feedback on whether what's been proposed in
CASSANDRA-17808 <https://issues.apache.org/jira/browse/CASSANDRA-17808> is
reasonable. To summarize...

1.) We continue to transfer hints by default during decommission, but at a
higher rate. We could, for instance, stop having DispatchHintsTask divide
its effective rate by the number of nodes in the cluster.

int nodesCount = Math.max(1,
StorageService.instance.getTokenMetadata().getAllEndpoints().size() -
1);
double throttleInBytes =
DatabaseDescriptor.getHintedHandoffThrottleInKiB() * 1024.0 /
nodesCount;
this.rateLimiter = RateLimiter.create(throttleInBytes == 0 ?
Double.MAX_VALUE : throttleInBytes);

2.) We provide an option to simply avoid transferring hints during
unbootstrap. Even this would only take the BatchLog from "best effort" to
"slightly less effort"