You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2021/07/09 15:26:46 UTC
[GitHub] [accumulo] DomGarguilo opened a new issue #2196: Add sanity check to table properties prior to creation of fate operation
DomGarguilo opened a new issue #2196:
URL: https://github.com/apache/accumulo/issues/2196
Based on JIRA ticket: [4773](https://issues.apache.org/jira/browse/ACCUMULO-4773)
Quote from the ticket:
> On the server side, table properties could be sanity checked before the fate operation is created.
> Could be added prior to
> https://github.com/apache/accumulo/blob/rel/1.8.1/server/master/src/main/java/org/apache/accumulo/master/FateServiceHandler.java#L151
> This would prevent situations where user assumes they have set table properties when in fact the table properties are mis-configured and ignored by the fate operation.
It seems like this could still be useful. I wanted to see if anyone had suggestions regarding how table properties might be verified here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882818126
I would like to be assigned to this ticket please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo edited a comment on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
DomGarguilo edited a comment on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882772978
> For the code you posted, it could be placed in a static method and called from those two places.
https://github.com/apache/accumulo/blob/80c8ee78342cfaea978882c37ebfcfc4af990fa1/server/manager/src/main/java/org/apache/accumulo/manager/tableOps/create/PopulateZookeeper.java#L61-L63
In PopulateZookeeper.java we could potentially make use of `TablePropUtil.setTableProperty` which performs the same check. Would look like this:
```java
for (Entry<String,String> entry : tableInfo.props.entrySet()) {
if (!TablePropUtil.setTableProperty(manager.getContext(), tableInfo.getTableId(),
entry.getKey(), entry.getValue())) {
throw new ThriftTableOperationException(null, tableInfo.getTableName(),
TableOperation.CREATE, TableOperationExceptionType.OTHER,
"Property or value not valid " + entry.getKey() + "=" + entry.getValue());
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882756191
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882764925
> Might work if placed in FateServiceHandler.java and PopulateZookeeper.java?
For the code you posted, it could be placed in a static method and called from those two places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-881551643
> We could handle that redundant validation and cause the FaTE operation to roll back
What do you mean by "roll back" here, @ctubbsii?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882817966
So each would be slightly different. Not sure which approach would be best, either two different code blocks (using whats returned from `TablePropUtil.setTableProperty` in PopulateZookeeper.java and `Property.isTablePropertyValid` in FateServiceHandler.java) or making a static method to use in both places. If we make a static method it seems the check would end up happening twice in FateServiceHandler anyways since `TablePropUtil.setTableProperty` will be called regardless (even though its return value is currently ignored).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-879415745
> That's my concern as well. I think it's better to validate the properties without kicking it off and making any persistent changes at all, even those to kick off the FaTE op.
One nice way to do that would be to call the same method in both places to do the validation. Thinking this will make the code more maintainable. This method could throw an error when the validation fails. Would not expect it to ever throw an exception in the FATE op, but if it does it will cause a failure instead of silently ignoring it.
If we only wanted to do the check in the FATE op, would probably need an IT that ensures no persistent data is left behind.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] EdColeman commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
EdColeman commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-881563124
FATE operations can be composed of multiple "steps" . If an error occurs, the FATE transaction stack tries to roll back and unwind any steps that may have completed so the state is consistent with the state when the FATE transaction started.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882756191
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-877307961
This issue was not very specific in what kind of sanity checks it wanted. I suppose we could check each one to see if they are valid per-table properties (to be allowed to be stored in ZK for the table), and that each property value conforms to the constraints for the type. But, it might be best to get more info from the original reporter, which I think was @jmark99
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-878593172
> So it seems if a user passed in invalid props to a table create op that those may be silently ignored. Would probably be better if the table was not created and the create table operation client side got an error.
I agree, but we probably shouldn't wait for it to fail inside the FaTE operation. It seems the original proposal was to validate them before kicking off the FaTE operation, and return an exception in the FateServiceHandler before it starts. I think that seems reasonable, but then we'd be left with a second validation check inside the FaTE operation. We could handle that redundant validation and cause the FaTE operation to roll back (safest, but redundant), or continue to silently ignore it there (easiest, but redundant), or we could try to bypass the validation checks there, since we've already done the validation (least redundant, but involves creating a validation bypass, which could be abused later).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-880182286
> Would not expect it to ever throw an exception in the FATE op, but if it does it will cause a failure instead of silently ignoring it.
Yeah, I'd hate for this to validate redundantly, but I think that's probably the best way to go, so we don't code in a bypass to validation, and can reuse the existing code. (This is the option I called "safest, but redundant" above)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-879395731
> The thing I am uncertain about is if the throwing an exception in the FATE op will properly cleanup the partially created table and result in a client side exception.
That's my concern as well. I think it's better to validate the properties without kicking it off and making any persistent changes at all, even those to kick off the FaTE op.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner edited a comment on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
keith-turner edited a comment on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-878342245
As part of the FATE op to create a table the following executes to set any new table props.
https://github.com/apache/accumulo/blob/80c8ee78342cfaea978882c37ebfcfc4af990fa1/server/manager/src/main/java/org/apache/accumulo/manager/tableOps/create/PopulateZookeeper.java#L61-L63
The setTableProp method does a validity check and if that fails it returns false.
https://github.com/apache/accumulo/blob/80c8ee78342cfaea978882c37ebfcfc4af990fa1/server/base/src/main/java/org/apache/accumulo/server/util/TablePropUtil.java#L42-L43
So it seems if a user passed in invalid props to a table create op that those may be silently ignored. Would probably be better if the table was not created and the create table operation client side got an error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo edited a comment on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
DomGarguilo edited a comment on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882772978
> For the code you posted, it could be placed in a static method and called from those two places.
https://github.com/apache/accumulo/blob/80c8ee78342cfaea978882c37ebfcfc4af990fa1/server/manager/src/main/java/org/apache/accumulo/manager/tableOps/create/PopulateZookeeper.java#L61-L63
In PopulateZookeeper.java we could potentially make use of `TablePropUtil.setTableProperty` which performs the same check. Would look like this:
```java
for (Entry<String,String> entry : tableInfo.props.entrySet()) {
if (!TablePropUtil.setTableProperty(manager.getContext(), tableInfo.getTableId(),
entry.getKey(), entry.getValue())) {
throw new ThriftTableOperationException(null, tableInfo.getTableName(),
TableOperation.CREATE, TableOperationExceptionType.OTHER,
"Property or value not valid " + entry.getKey() + "=" + entry.getValue());
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo edited a comment on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
DomGarguilo edited a comment on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882772978
> For the code you posted, it could be placed in a static method and called from those two places.
https://github.com/apache/accumulo/blob/80c8ee78342cfaea978882c37ebfcfc4af990fa1/server/manager/src/main/java/org/apache/accumulo/manager/tableOps/create/PopulateZookeeper.java#L61-L63
In PopulateZookeeper.java we could potentially make use of `TablePropUtil.setTableProperty` which performs the same check. Would look like this:
```java
for (Entry<String,String> entry : tableInfo.props.entrySet()) {
if (!TablePropUtil.setTableProperty(manager.getContext(), tableInfo.getTableId(),
entry.getKey(), entry.getValue())) {
throw new ThriftTableOperationException(null, tableInfo.getTableName(),
TableOperation.CREATE, TableOperationExceptionType.OTHER,
"Property or value not valid " + entry.getKey() + "=" + entry.getValue());
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-879341594
> we probably shouldn't wait for it to fail inside the FaTE operation
Not sure if its possible, but if we could just throw an exception in the FATE op that would avoid the redundant code outside the FATE op which may result in more maintainable code. The thing I am uncertain about is if the throwing an exception in the FATE op will properly cleanup the partially created table and result in a client side exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] EdColeman closed issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
EdColeman closed issue #2196:
URL: https://github.com/apache/accumulo/issues/2196
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882764925
> Might work if placed in FateServiceHandler.java and PopulateZookeeper.java?
For the code you posted, it could be placed in a static method and called from those two places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882764925
> Might work if placed in FateServiceHandler.java and PopulateZookeeper.java?
For the code you posted, it could be placed in a static method and called from those two places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882772978
> For the code you posted, it could be placed in a static method and called from those two places.
In FateServiceHandler.java
https://github.com/apache/accumulo/blob/80c8ee78342cfaea978882c37ebfcfc4af990fa1/server/manager/src/main/java/org/apache/accumulo/manager/tableOps/create/PopulateZookeeper.java#L61-L63
we could potentially make use of `TablePropUtil.setTableProperty` which performs the same check. Would look like this:
```java
for (Entry<String,String> entry : tableInfo.props.entrySet()) {
if (!TablePropUtil.setTableProperty(manager.getContext(), tableInfo.getTableId(),
entry.getKey(), entry.getValue())) {
throw new ThriftTableOperationException(null, tableInfo.getTableName(),
TableOperation.CREATE, TableOperationExceptionType.OTHER,
"Property or value not valid " + entry.getKey() + "=" + entry.getValue());
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-878342245
As part of the FATE op to create a table the following executes to set any new table props.
https://github.com/apache/accumulo/blob/80c8ee78342cfaea978882c37ebfcfc4af990fa1/server/manager/src/main/java/org/apache/accumulo/manager/tableOps/create/PopulateZookeeper.java#L61-L63
The setTableProp method does a validity check and if that fails it returns false.
https://github.com/apache/accumulo/blob/80c8ee78342cfaea978882c37ebfcfc4af990fa1/server/base/src/main/java/org/apache/accumulo/server/util/TablePropUtil.java#L42-L43
So it seems if a user passed in invalid props to a table create op that those may be silently ignored. Would probably be better if the table was not create and the create table operation client side got an error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo commented on issue #2196: Add sanity check to table properties prior to creation of fate operation
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on issue #2196:
URL: https://github.com/apache/accumulo/issues/2196#issuecomment-882756191
Does it seem like something similar to:
```java
for (Map.Entry<String,String> entry : options.entrySet()) {
if (!Property.isTablePropertyValid(entry.getKey(), entry.getValue())) {
throw new ThriftTableOperationException(null, tableName, tableOp,
TableOperationExceptionType.OTHER,
"Property or value not valid " + entry.getKey() + "=" + entry.getValue());
}
}
```
Might work if placed in FateServiceHandler.java and PopulateZookeeper.java?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org