You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jesse Yates (Created) (JIRA)" <ji...@apache.org> on 2011/10/17 20:23:10 UTC
[jira] [Created] (HBASE-4605) Add constraints as a top-level
feature
Add constraints as a top-level feature
--------------------------------------
Key: HBASE-4605
URL: https://issues.apache.org/jira/browse/HBASE-4605
Project: HBase
Issue Type: Improvement
Components: client, coprocessors
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
>From Jesse's comment on dev:
{quote}
What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-4605:
-------------------------------
Summary: Constraints (was: Add constraints as a top-level feature)
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158121#comment-13158121 ]
Hadoop QA commented on HBASE-4605:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505275/java_HBASE-4605_v1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/384//console
This message is automatically generated.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136498#comment-13136498 ]
Ted Yu commented on HBASE-4605:
-------------------------------
@Jesse:
Can you publish your patch on review board ?
I see some nits which can be better expressed there.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-4605:
------------------------------
Comment: was deleted
(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504935/4605.v7
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 68 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.client.TestInstantSchemaChange
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/352//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/352//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/352//console
This message is automatically generated.)
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168910#comment-13168910 ]
Zhihong Yu commented on HBASE-4605:
-----------------------------------
The above message from QA meant that the patch must be applied with -p1.
I tried to produce a patch which can be applied with -p0 but I got:
{code}
patching file src/docbkx/book.xml
Reversed (or previously applied) patch detected! Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file src/docbkx/book.xml.rej
{code}
@Jesse:
Do you mind refreshing your patch using --no-prefix ?
Thanks
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-4605:
------------------------------
Status: Patch Available (was: Open)
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139843#comment-13139843 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review2934
-----------------------------------------------------------
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
<https://reviews.apache.org/r/2579/#comment6568>
Whoops - missed that one.
- Jesse
On 2011-10-31 00:48:49, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-10-31 00:48:49)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156287#comment-13156287 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
Yeah, sure. I actually just ran into the same issue trying to work on the shell stuff.
Pushing up new version shortly.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152226#comment-13152226 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
bq. I got compilation error trying to run integration test:
Hmmm, something was a little wonky - had test/ excluded to ignore some build issues, but that neglected all the tests I added. But FWIW it passes locally ;) Throwing up the correct version momentarily
Also, please ignore package-info.java - its on the TODO list and just using a placeholder for the moment.
{quote}
SERIALIZE_KEY and SERIALIZE_VALUE perform the same transformation. I think they should be combined into one Function, possibly named serializer so that naming for Functions is consistent (with transformToStrings).
Same with DESERIALIZE_KEY and DESERIALIZE_VALUE.
{quote}
I was thinking it would be better to split them into two different elements since we may want to change how we do the serialization for each in the future, and having the independence makes it much easier to swap and adds negligible overhead.
bq. getValueAsBytes() is only used once. I think we can just inline what it does in getKeyValueForClass().
+1 thought it might be useful later, but we can always add it as needed. Making the fix in the latest patch.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Gary Helmling (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131937#comment-13131937 ]
Gary Helmling commented on HBASE-4605:
--------------------------------------
Summarizing some comments from a related thread in the dev@hbase.apache.org mailing list:
http://search-hadoop.com/m/oGvg82YEzA82
Adding special configuration hooks to {{HTableDescriptor}}, say {{HTableDescriptor.addConstraint(Class c)}}, is likely to lead to long-term clutter if we follow this strategy for each special purpose coprocessor implementation that comes up.
So instead, I'd suggest we invert this and make the coprocessor implementation responsible for setting it's configuration as attributes on {{HTableDescriptor}}. This would follow a pattern something like the following:
{code}
HTableDescriptor htd = new HTableDescriptor(...);
Constraints.add(htd, MyConstraintImpl.class);
HBaseAdmin admin = new HBaseAdmin();
admin.createTable(htd);
{code}
The important part being that the {{Constraints}} class handles marshalling it's required configuration as attributes on {{HTableDescriptor}}, and, in addition, is free to define those configuration helper methods however best makes sense for it's own needs. For the example above, I'm guessing this would be something like:
{code}
public static void add(HTableDescriptor htd, Class<? extends Constraint>... clz)
{code}
This keeps {{HTableDescriptor}} handling just the common needs (through the arbitrary key/value attributes), while keeping the specific configuration logic with each coprocessor implementation.
There is an additional challenge, however, in how we can cleanly incorporate access to set the configurations into the HBase shell. Ideally, you'd be able to do something like (exact syntax aside):
{noformat}
hbase> alter 'mytable', METHOD => 'Constraints.add', VALUE => com.my.MyConstraint
{noformat}
or maybe
{noformat}
hbase> alter 'mytable', TYPE => 'Constraints', METHOD => 'add', VALUE => com.my.MyConstraint
{noformat}
I think the key problem is figuring out how the shell actually knows about the {{Constraints}} class. Some options are:
# use the full class name, e.g. "org.apache.hadoop.hbase.coprocessor.Constraints". This is clunky but probably simplest to do.
# allowed the Constraints class to register itself via some SPI interface that the shell code can examine for possible providers. This would probably still need the class to be statically initialized somehow.
# provide a extensions dir for the shell:
#* extensions drop a simple jruby scriptlet in a file in the dir (say lib/ruby/hbase/ext)
#* the scriptlet does some simple registration of the available methods/commands
#* the shell code loads all files
There may be better options as well, but hopefully this can serve as the basis for some discussion.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-4605:
------------------------------
Comment: was deleted
(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505275/java_HBASE-4605_v1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/384//console
This message is automatically generated.)
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-4605:
-------------------------------
Attachment: java_HBASE-4605_v5.patch
Ok, this time --no-prefix (duh).
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139854#comment-13139854 ]
Ted Yu commented on HBASE-4605:
-------------------------------
In the above patch, why do we need two loops in Constraints.remove(HTableDescriptor) ?
Can we call desc.remove(e.getKey()) in the first loop ?
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139313#comment-13139313 ]
Ted Yu commented on HBASE-4605:
-------------------------------
Should we add method for enabling/disabling individual constraints to Constraints.java ?
{code}
public static void enableConstraint(HTableDescriptor desc, Class<? extends Constraint> clazz) {
...
public static void disableConstraint(HTableDescriptor desc, Class<? extends Constraint> clazz) {
{code}
Using the above form, we cannot simply remove the constraint processor.
We shouldn't remove the constraint corresponding to clazz either because the constraint may have unique Configuration object associated with it.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136461#comment-13136461 ]
Ted Yu commented on HBASE-4605:
-------------------------------
All the new files miss license.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169648#comment-13169648 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
bq. On 2011-12-14 00:40:59, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 335
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line335>
bq. >
bq. > Can we call ByteArrayOutputStream.toString() directly ?
No, because there is no assurance which encoding will be used. This way, we can be sure that it is UTF-8 encoded and conforms to the rest of the project. Also, the overhead here is just one more, short-lived pointer (not a big deal).
bq. On 2011-12-14 00:40:59, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 196
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line196>
bq. >
bq. > Priority is updated at the end.
bq. > I think we should document concurrency assumption in javadoc.
Added a section to package-info talking about concurrency.
bq. On 2011-12-14 00:40:59, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 179
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line179>
bq. >
bq. > Multiple Constraints are supported. Please update javadoc.
done.
bq. On 2011-12-14 00:40:59, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 355
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line355>
bq. >
bq. > The dash is not needed.
done.
- Jesse
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3897
-----------------------------------------------------------
On 2011-12-13 21:38:03, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-13 21:38:03)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/docbkx/book.xml 9617950
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159513#comment-13159513 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
Latest patch does _not_ have labeling of tests since I wanted to wait and see what dev@ thinks about the latest testing stuff.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Add constraints as a top-level
feature
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129064#comment-13129064 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
I've been thinking about how to go about implementing this and have two ways to go about it.
Method 1:
My idea is to write a ConstraintProcessor that is a system level CP with system wide support for setting constraints on a table. this requires add 'top-level' configuration values that the user would set for constraints to run (which would be ordered like coprocessors), but they would just implement the 'Constraint' interface. This means modifying HTD and the shell to enable all these
This means people need to distribute the jars and set conf values similar to what they would have to do before, but we would handle making sure the implemented Constraints get run in the right place, and propagate the errors (e.g. ConstraintFailedExecption) back to the users.
Pluses:
1) Allows users to get back multiple reasons why a put failed.
2) Allows a ConstraintImpl to be a subclass of any arbitrary class and not bound to some abstract constraint.
3) People don't have to worry about it being a coprocessor - it is notionally divorced.
Minuses:
1) Requires changing a bunch of code in HTableDescriptor and essentially duplicating a lot the checking/setting already done for coprocessors. This can be gotten around by generalizing the mechanism for storing classes in the HTD.
Method 2 (already implemented, patch coming):
Add superclass AbstractConstraint which only exposes a check(Put) method. It is actually a Coprocoessor which is loaded, processes the check and then returns the error to the client (wrapped in an IOException) on failure.
Pluses:
1) We don't have to implement any new mechanisms for specifying the constraint, people just have to add it as a coprocessor.
Minuses:
1) It could be confusing since with this mechanism, you just want people to think in terms of Constraints, not coprocessors
2) You are bound to extending the AbstractCoprocessor, not just implementing the interface
3) If just one constraint fails, then the put is rejected, so you can't find out all the reasons it would fail (useful if cleaning data).
4) It doesn't really help 'simplify' the use of HBase. In fact, it increases the complexity.
> Add constraints as a top-level feature
> --------------------------------------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-4605:
------------------------------
Attachment: (was: java_HBASE-4605_v5.patch)
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158139#comment-13158139 ]
Hadoop QA commented on HBASE-4605:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505280/java_HBASE-4605_v3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 69 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestRollingRestart
org.apache.hadoop.hbase.util.TestRegionSplitter
org.apache.hadoop.hbase.io.hfile.TestLruBlockCache
org.apache.hadoop.hbase.client.TestMultiParallel
org.apache.hadoop.hbase.master.TestRestartCluster
org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.regionserver.TestStore
org.apache.hadoop.hbase.regionserver.wal.TestHLogBench
org.apache.hadoop.hbase.rest.TestGzipFilter
org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.rest.TestScannersWithFilters
org.apache.hadoop.hbase.TestInfoServers
org.apache.hadoop.hbase.regionserver.TestParallelPut
org.apache.hadoop.hbase.coprocessor.TestClassLoading
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
org.apache.hadoop.hbase.filter.TestColumnRangeFilter
org.apache.hadoop.hbase.mapred.TestTableInputFormat
org.apache.hadoop.hbase.client.TestHCM
org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
org.apache.hadoop.hbase.rest.TestStatusResource
org.apache.hadoop.hbase.TestRegionRebalancing
org.apache.hadoop.hbase.regionserver.wal.TestLogRollAbort
org.apache.hadoop.hbase.rest.TestVersionResource
org.apache.hadoop.hbase.client.TestScannerTimeout
org.apache.hadoop.hbase.client.TestFromClientSide
org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed
org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol
org.apache.hadoop.hbase.regionserver.TestSplitTransaction
org.apache.hadoop.hbase.rest.TestRowResource
org.apache.hadoop.hbase.rest.TestScannerResource
org.apache.hadoop.hbase.ipc.TestDelayedRpc
org.apache.hadoop.hbase.rest.client.TestRemoteAdmin
org.apache.hadoop.hbase.util.TestFSUtils
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.rest.TestTableResource
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
org.apache.hadoop.hbase.master.TestHMasterRPCException
org.apache.hadoop.hbase.util.TestIdLock
org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster
org.apache.hadoop.hbase.rest.TestTransform
org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint
org.apache.hadoop.hbase.client.TestInstantSchemaChangeSplit
org.apache.hadoop.hbase.regionserver.TestHRegion
org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyControl
org.apache.hadoop.hbase.client.TestMultipleTimestamps
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
org.apache.hadoop.hbase.catalog.TestMetaReaderEditor
org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
org.apache.hadoop.hbase.regionserver.TestKeepDeletes
org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed
org.apache.hadoop.hbase.client.TestMetaScanner
org.apache.hadoop.hbase.io.hfile.TestHFileBlock
org.apache.hadoop.hbase.client.TestTimestampsFilter
org.apache.hadoop.hbase.client.TestInstantSchemaChangeFailover
org.apache.hadoop.hbase.master.TestMaster
org.apache.hadoop.hbase.client.TestShell
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
org.apache.hadoop.hbase.rest.TestSchemaResource
org.apache.hadoop.hbase.TestAcidGuarantees
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan
org.apache.hadoop.hbase.regionserver.handler.TestOpenRegionHandler
org.apache.hadoop.hbase.regionserver.TestRpcMetrics
org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase
org.apache.hadoop.hbase.avro.TestAvroServer
org.apache.hadoop.hbase.rest.client.TestRemoteTable
org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
org.apache.hadoop.hbase.util.TestHBaseFsck
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove
org.apache.hadoop.hbase.client.TestHTableUtil
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit
org.apache.hadoop.hbase.thrift.TestThriftServer
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface
org.apache.hadoop.hbase.util.TestMergeTool
org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics
org.apache.hadoop.hbase.util.TestMergeTable
org.apache.hadoop.hbase.master.TestMasterFailover
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.regionserver.wal.TestHLog
org.apache.hadoop.hbase.rest.TestMultiRowResource
org.apache.hadoop.hbase.TestMultiVersions
org.apache.hadoop.hbase.filter.TestFilter
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.master.TestMasterTransitions
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.master.TestSplitLogManager
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove
org.apache.hadoop.hbase.coprocessor.TestWALObserver
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
org.apache.hadoop.hbase.master.TestOpenedRegionHandler
org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/386//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/386//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/386//console
This message is automatically generated.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136563#comment-13136563 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review2870
-----------------------------------------------------------
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
<https://reviews.apache.org/r/2579/#comment6424>
This condition can be expressed as !constraints.isEmpty()
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
<https://reviews.apache.org/r/2579/#comment6423>
Please use curly braces for line 60.
This log can be combined with log on line 63.
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
<https://reviews.apache.org/r/2579/#comment6425>
This has been done, right ?
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
<https://reviews.apache.org/r/2579/#comment6426>
Contents of put should be included.
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment6427>
We can either add removeCoprocessor() to HTD or, persist a flag to tell ConstraintProcessor that it is disabled.
I think the second approach may be better in our case.
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment6428>
Should be 'desc HTableDescriptor to read from'
- Ted
On 2011-10-26 22:34:35, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-10-26 22:34:35)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171468#comment-13171468 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
@Stack: yeah, we can definitely made mods later. As far as docs go, there is extensive information in the package-info. If you want we can also port/copy it to the book or have a comment in the book about looking at the package info.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Add constraints as a top-level
feature
Posted by "nkeywal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129155#comment-13129155 ]
nkeywal commented on HBASE-4605:
--------------------------------
It would be a very useful features.
For what it worth, I am quite fine with method 2.
Some points I found useful in the SQL engines:
- capacity to disable the constraints for a connection (basically because yoy're doing a big set of operation and you want the best possible performances
- capacity to disable globally (upgrade)
I also wonder how we should manage the evolution of the constraint. On an SQL db, there is only one possible set; considering the amount of data, the traditional upgrade if the traditional sql db could not be possible here; so managing evolution of constraint would make sense.
A comment as well: in a SQL system, the constraints are linked to the transaction: it's checked once the transaction is committed. This is important for cross table constraint checks, as there is no transaction between tables in HBase. I would tend to believe that's mainly a question of documentation (insert in the right order), but it's something to remember anyway (especially as you want to duplicate the relationships with HBase more than with a sql db)...
I don't know the status of HCatalog, but I think the HCatalog schema will be transformable as HBase constraints, adding value to the two of them...
> Add constraints as a top-level feature
> --------------------------------------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171469#comment-13171469 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
@stack: and we kept guava out still and basically went back to the original HTD + removeCoprocessor()
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152400#comment-13152400 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/
-----------------------------------------------------------
(Updated 2011-11-17 22:23:09.015159)
Review request for hbase.
Changes
-------
Fixed test bugs from last 2 sets of changes (thx Ted!).
Ran tests: mvn clean test -Dtest=*Constraint*
They all pass locally.
Summary
-------
Most of the implementation for adding constraints as a coprocessor.
Looking for general comments on style/structure, though nitpicks are ok too.
Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
This addresses bug HBASE-4605.
https://issues.apache.org/jira/browse/HBASE-4605
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
Diff: https://reviews.apache.org/r/2579/diff
Testing
-------
Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
Thanks,
Jesse
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159508#comment-13159508 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
bq. On 2011-11-29 08:36:29, Gary Helmling wrote:
bq. > Most of what I see is just style issues, and a number of the HTableDescriptor changes seem unnecessary. If we clean up those, and assuming the latest patch has some documentation, I think this is looking pretty good.
Uploading the latest (with some of your recommended changes) after this. Couple of concerns may still need to be worked out around IntegerConstraint and the mods to HTD. Let me know what you think.
bq. On 2011-11-29 08:36:29, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/package-info.java, line 22
bq. > <https://reviews.apache.org/r/2579/diff/6/?file=59947#file59947line22>
bq. >
bq. > This differs from what seems to be the latest patch in JIRA (java_HBASE-4605_v3.txt). Are there other differences than the javadoc?
bq. >
bq. > For a patch of this size, review still belongs here in review board (even with HadoopQA patch testing), so we should keep both versions in sync. Please update the diff here with the latest patch.
Yeah, this is the old version. Whats up on the ticket is just documentation. I'll put up the latest here after making the changes.
bq. On 2011-11-29 08:36:29, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 250
bq. > <https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line250>
bq. >
bq. > if / else bodies should be contained in braces.
done.
bq. On 2011-11-29 08:36:29, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 233
bq. > <https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line233>
bq. >
bq. > Be consistent with braces here
Just following the requirements of java :) Done.
bq. On 2011-11-29 08:36:29, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 22
bq. > <https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line22>
bq. >
bq. > I see the usage outside of HTD now, but still don't see where this provides more utility than Bytes.toString()
By pulling the functionality out of the HTD we can be sure that outside classes don't have to worry about making sure they use the 'right' (de)serialization of they keys. Right now we don't want to to anything more than just Bytes.toString(), but in the future we could make it something more complex. For example, take the case where user specified keys are prepended with a special identifier, so we can tell the difference between the two. Alternatively, we could do some cool stuff to cut down on object creation (potentially) with the key conversion or make it faster or more compact or w/e.
Basically, I'm just adding the encapsulation so classes outside of the HTD don't have to know about the fact that the HTD is using Bytes.toString() (and encapsulation is good, right?). This way it is obvious, to an outside class, what is going - oh, it just serializing the key. This makes the addCP code much cleaner too.
bq. On 2011-11-29 08:36:29, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 86
bq. > <https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line86>
bq. >
bq. > style nit: for loop body should be enclosed in braces
Done
bq. On 2011-11-29 08:36:29, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 76
bq. > <https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line76>
bq. >
bq. > Add constraints.size() to this message and you don't need the separate LOG.debug() message.
Done
bq. On 2011-11-29 08:36:29, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 61
bq. > <https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line61>
bq. >
bq. > style nit: if body should be contained in braces
Done.
bq. On 2011-11-29 08:36:29, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 32
bq. > <https://reviews.apache.org/r/2579/diff/6/?file=59946#file59946line32>
bq. >
bq. > This still seems odd to me. I can't see storing an integer value encoded as a string, in order to be able to check for it being a valid integer.
bq. >
bq. > I would think most users would be storing integer values using Bytes.toInt(), for which the checks here would not work.
The only problem with using the Bytes deserialization checking is that even if you put a string in as bytes, it can be read out as an integer (in certain cases). Therefore, you could get a case where something that should not be an integer is allowed when it really ought not to be.
bq. On 2011-11-29 08:36:29, Gary Helmling wrote:
bq. > src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java, line 90
bq. > <https://reviews.apache.org/r/2579/diff/6/?file=59954#file59954line90>
bq. >
bq. > brace alignment and indentation
Done.
bq. On 2011-11-29 08:36:29, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java, line 1166
bq. > <https://reviews.apache.org/r/2579/diff/6/?file=59940#file59940line1166>
bq. >
bq. > I don't see why these changes are necessary when I don't see them being used anywhere outside of HTD? Do these additions buy us anything? In my opinion, they're not making the code any clearer than the current Byte.toString(), etc.
See response below.
- Jesse
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3551
-----------------------------------------------------------
On 2011-11-23 21:19:56, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-23 21:19:56)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162559#comment-13162559 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3610
-----------------------------------------------------------
Main concerns as same as previously -- the HTableDescriptor changes and IntegerConstraint:
1) HTD changes introduce a dependency on Guava. I thought we had eliminated all client-side dependencies on Guava?
2) I think serializing constraint configurations as XML would be useful, as they would then be human readable and require less care to preserve previous serialization formats on upgrade when the property implementations change.
3) I'm still not seeing the use of IntegerConstraint as the one Constraint implementation that we ship as part of core (as opposed to it being an example for documentation). Is there a specific use where you're targeting this?
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
<https://reviews.apache.org/r/2579/#comment8047>
I still don't think this gives us true encapsulation, since the HTD.setValue() allows arbitrary byte[] for values, so there's no way of knowing if (DE)SERIALIZE_KEY was actually used. So is this really better than Bytes.toString()?
I also thought that we had eliminated Guava as a client side dependency? This would re-introduce it. That seems the bigger concern unless we've agreed to a client-side Guava dependency somewhere.
Why not handle the values as Strings and serialize the Contraint configuration objects as XML? In this case, I think human readable is actually good.
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
<https://reviews.apache.org/r/2579/#comment8059>
Omit copyright line
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment8058>
Omit copyright line
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment8055>
Should constraints apply to Increments and Deletes as well? I don't know if they should or not, and even if they do, it doesn't need to be part of this issue. Just something to consider.
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java
<https://reviews.apache.org/r/2579/#comment8057>
Omit copyright line
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
<https://reviews.apache.org/r/2579/#comment8056>
Copyright line not necessary in file headers.
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment8060>
Having HTD.getValue() convert to String here and then immediately converting back to byte[] is odd. I think either add HTD.getValueAsBytes() returning the raw byte[] or serialize the Configuration as XML as I suggested and just handle it as a String here.
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment8053>
The manual serialization of properties here seems likely to be brittle and subject to breakage on upgrade if the implementation changes (unless backwards compatibility is explicitly accounted for).
This actually seems like a decent case for using the Configuration class' XML serialization support. The output will be much larger, but more resistant to changes in the properties serialized. Just clone the input configuration object, set the "enabled" and "priority" properties under known keys, and serialize to a string using Configuration.writeXML().
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java
<https://reviews.apache.org/r/2579/#comment8054>
Is there a reason why this has to be included in code as a built in constraint implementation (the only one), instead of just included in documentation as an example?
As an example it makes sense to me. As a the only built-in, it doesn't, since the way it requires integers to be stored is at odds with all current usage, as I've previously point out.
- Gary
On 2011-11-29 20:19:41, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-29 20:19:41)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/docbkx/book.xml 3c12169
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139380#comment-13139380 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
{quote}
w.r.t. performance after several rounds of enabling/disabling, the following operations are both O(1):
1. retrieve Configuration object from HTD.values map for given Constraint
2. retrieve the boolean in this Configuration object for "hbase.constraint.enabled"
There would be impact on performance if the number of constraints is high.
{quote}
I was thinking you meant just using a boolean to enable/disable the entire constraint mechanism, rather than an individual constraint. It is faster to remove the ConstraintProcessor from the CP list as you save the method calls into to ConstraintProcessor to check the enabled/disabled boolean.
bq. Should we add method for enabling/disabling individual constraints to Constraints.java ?
This starts to be a bit painful if we want to actually do enforcement of order of constraint application you have to traverse the entire map to check for values (however, if we assume that the map isn't big its not that big of a deal). However, I think, in general, that it is a good idea.
I think we should also have a remove method as well as the disable method for individual constraints - this is more of a usage consideration rather than a runtime one. Say you added a constraint that was wrong - removing it rather than disabling it is definitely necessary.
Also, we should also consider having:
{code}
public static void setConfiguration(HTableDescriptor desc, Class<? extends Constraint> clazz, Configuration conf){}
{code}
That overrides the configuration set for a constraint.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-4605:
------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139054#comment-13139054 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review2917
-----------------------------------------------------------
Jessie, this looks like a pretty good start. In addition to the individual comments, a couple of general questions:
1) Should ConstraintProcessor support some standard way of mapping Constraint implementations to the families/qualifiers they should apply to?
2) Should we ship a base set of Constraint implementations for common cases?
Even if the answer to both is "yes", they could be addressed as follow-up JIRAs. But it would be good to think through the end goal here.
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment6534>
I think this sentence needs updating to reference Constraints/ConstraintProcessor.
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment6535>
Is this true? The HTD values are stored in a HashMap, so ordering wouldn't be preserved.
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment6536>
Should reference Constraint since BaseConstraint is just a convenience class.
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment6521>
Should we define a base exception here that is thrown? I'm thinking something like:
ConstraintViolationException extends DoNotRetryIOException
This way we don't need to wrap in DoNotRetryIOException later and we give implementers a bit more guidance on what to do.
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
<https://reviews.apache.org/r/2579/#comment6522>
If we use a defined exception class for constraint violations (see comment above), I think we can omit this try/catch block. Expected exceptions would already subclass DoNotRetryIOException, so no need to wrap it here. Other (non-expected) Throwables would be handled higher up by RegionCoprocessorHost.
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment6525>
I think this and the other addConstraints() method could just be named add().
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment6523>
I get the idea of being able to enable/disable constraints on demand, without removing the individual constraints configured. But why do we need this and the private addConstraintProcessor() method? Why not move the addConstraintProcessor() implementation here and make addConstraints() call this instead?
Also, I would just name this enable().
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment6524>
In addition to being able to disable constraint checking by removing the ConstraintProcessor CP, we probably also need a removeConstraints(HTableDescriptor) method that removes the ConstraintProcessor CP and all the "constraint $" attributes on HTD for complete cleanup.
By the way, since the name of the class is Constraints, I think this method could just be named disable() -- the additional "Constraints" is redundant.
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java
<https://reviews.apache.org/r/2579/#comment6530>
Is this intended as a test class or as a usable implementation? If a test class, should be under src/test/java/...
If a usable implementation, then, looking at it, I wonder how useful it is without a way of mapping constraints to individual qualifiers. Do you think that's something ConstraintProcessor should provide in general?
It could be left up to end user implementations of Constraint to match which KeyValues they should apply to. But then I don't think any implementations we can bundle will have much value, unless they provide for their own qualifier matching via configuration values.
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java
<https://reviews.apache.org/r/2579/#comment6532>
This doesn't seem right. Wouldn't most ints be stored as Bytes.toBytes(int), not as a String?
- Gary
On 2011-10-26 22:34:35, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-10-26 22:34:35)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156290#comment-13156290 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/
-----------------------------------------------------------
(Updated 2011-11-23 21:19:56.263794)
Review request for hbase.
Changes
-------
Updating to current trunk to take into account changes in HTD and for Hadoop QA. Otherwise, no changes from last diff.
Summary
-------
Most of the implementation for adding constraints as a coprocessor.
Looking for general comments on style/structure, though nitpicks are ok too.
Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
This addresses bug HBASE-4605.
https://issues.apache.org/jira/browse/HBASE-4605
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
Diff: https://reviews.apache.org/r/2579/diff
Testing
-------
Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
Thanks,
Jesse
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171462#comment-13171462 ]
stack commented on HBASE-4605:
------------------------------
Did this patch add guava as a client-side dependency in the end?
Doc needs boosting for this new feature; as is, a bunch of detail on howto is missing.
IMO, this patch could have baked out one RB a little longer. Its a new feature. I didn't realize it going in (was slow to get around to review). I'd like to have gone over it before it went in. I could review now I suppose but feels a bit late...
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152166#comment-13152166 ]
Ted Yu commented on HBASE-4605:
-------------------------------
I went through HTableDescriptor where the serialization/deserialization logic is defined.
SERIALIZE_KEY and SERIALIZE_VALUE perform the same transformation. I think they should be combined into one Function, possibly named serializer so that naming for Functions is consistent (with transformToStrings).
Same with DESERIALIZE_KEY and DESERIALIZE_VALUE.
getValueAsBytes() is only used once. I think we can just inline what it does in getKeyValueForClass().
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140503#comment-13140503 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > Jessie, this looks like a pretty good start. In addition to the individual comments, a couple of general questions:
bq. >
bq. > 1) Should ConstraintProcessor support some standard way of mapping Constraint implementations to the families/qualifiers they should apply to?
bq. > 2) Should we ship a base set of Constraint implementations for common cases?
bq. >
bq. > Even if the answer to both is "yes", they could be addressed as follow-up JIRAs. But it would be good to think through the end goal here.
bq.
bq. Jesse Yates wrote:
bq. 1) I was thinking an individual constriant could check any/all of the cf/cq being used. We can do it via (2) where we have an abstract Constraint that makes it easy to check a given cf/cq.
bq.
bq. So, no (kinda) and yes. For (2), I think it will be community usage/findings as to what people find useful - we can decide on a case-by-case basis if it is a valid contribution.
Okay, seems fine to me. Just wanted to raise the questions.
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 80
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line80>
bq. >
bq. > If we use a defined exception class for constraint violations (see comment above), I think we can omit this try/catch block. Expected exceptions would already subclass DoNotRetryIOException, so no need to wrap it here. Other (non-expected) Throwables would be handled higher up by RegionCoprocessorHost.
bq.
bq. Jesse Yates wrote:
bq. I don't think that runtime exceptions on a constraint should be allowed to propagate up the CPHost - if the constraint fails then that put should fail, but not anything else.
Some runtime exceptions could indicate programming errors or bugs, in which case I think it's best to handle them the same way we handle unexpected errors in coprocessors -- by either unloading or aborting, depending on configuration. We need to be careful for what we allow from user code running in process on region servers.
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 13
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53676#file53676line13>
bq. >
bq. > Is this intended as a test class or as a usable implementation? If a test class, should be under src/test/java/...
bq. >
bq. > If a usable implementation, then, looking at it, I wonder how useful it is without a way of mapping constraints to individual qualifiers. Do you think that's something ConstraintProcessor should provide in general?
bq. >
bq. > It could be left up to end user implementations of Constraint to match which KeyValues they should apply to. But then I don't think any implementations we can bundle will have much value, unless they provide for their own qualifier matching via configuration values.
bq.
bq. Jesse Yates wrote:
bq. Its meant to be both an example and a test util. I can move it into test, but I can see cases where people might want to just check to make sure they only store values. Yes, it a really simple use case, but it also helps people to build more complex ones
Maybe it should just go in the documentation then? Either in package level javadoc or a section of the HBase book? Maybe longer term we need a separate "examples" maven module for this and other extensions points. But the current implementation doesn't seem useful out of the box, since it forces integers to be stored as strings... in order to check that they're integers. Seems odd.
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 39
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53673#file53673line39>
bq. >
bq. > Should we define a base exception here that is thrown? I'm thinking something like:
bq. >
bq. > ConstraintViolationException extends DoNotRetryIOException
bq. >
bq. > This way we don't need to wrap in DoNotRetryIOException later and we give implementers a bit more guidance on what to do.
bq.
bq. Jesse Yates wrote:
bq. I wanted to let people throw whatever exception they wanted to so they can be specific as to what happened with minimal overhead. For instance, the IntegerConstraint lets the constraint just check to and then throw a NumberFormatException if it fails. However, the code for the user (the actual constraint implementation) is super concise and easy.
I disagree here. Treating all possible exceptions the same seems wrong. An OutOfMemoryError is not equivalent to a NumberFormatException. We need some structure in place for what types of errors are allowed/expected, instead of swallowing everything and assuming it's just a constraint violation. I don't think asking the Constraint implementations to think about what errors they want to throw is much of a burden.
- Gary
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review2917
-----------------------------------------------------------
On 2011-10-31 00:48:49, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-10-31 00:48:49)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Add constraints as a top-level
feature
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129456#comment-13129456 ]
Ted Yu commented on HBASE-4605:
-------------------------------
See the following from HBASE-4014 w.r.t. exceptions and coprocessors:
The general gist here is to wrap each of {Master,RegionServer}CoprocessorHost's coprocessor call inside a
"try { ... } catch (Throwable e) { handleCoprocessorThrowable(e) }"
block.
handleCoprocessorThrowable() is responsible for either passing 'e' along to the client (if 'e' is an IOException) or, otherwise, aborting the service (Regionserver or Master).
> Add constraints as a top-level feature
> --------------------------------------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163944#comment-13163944 ]
Zhihong Yu commented on HBASE-4605:
-----------------------------------
w.r.t. atomicity, it is outside the initial scope of this JIRA.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163939#comment-13163939 ]
Ted Yu commented on HBASE-4605:
-------------------------------
Renaming IntegrationTestConstraint as described above makes sense.
Thanks Jesse.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-4605:
-------------------------------
Attachment: java_HBASE-4605_v2.patch
oops, wrong file. This should be right.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163926#comment-13163926 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
@Gary:
Guava- I didn't know about the effort to get rid of guava client side, it can be pulled if it is really a big issue. I just read through 3264 - is it the case that only the client side dependencies are used for the MR stuff? For this it works out really nicely for doing the auto-serializing (see changes to the CP stuff). We could say that the set bytes for key is at your own risk, and make strings the default.
getValueAsBytes() - this was -1'ed in a previous iteration as constraints was the only use case for it - wrapping it with the explicit functions seemed a decent solution.
Configuration - As far as human readable, how often are people going to need to check those values? The HTD already just keeps around bytes, so people don't check those over the wire - constraints would remain in the same style.
How would you feel about making that configurable? I'm thinking setting a debug flag in Constraints/configuration for that value.
check(Put) - this could definitely be an extension. +1 on future considerations.
IntegerConstraint - In the end, I can let this go. You're right that having in the docs should be enough. Also, combined with the fact that we don't have any others by default (and that removing is far harder than adding) I'll drop it into just the docs
@Ted:
IntegrationTestConstraint - based on the recent discussion on dev@ about testing, this should be renamed to just TestConstraint and labeled as @MediumTest. The rest should be moved to @SmallTest
Atomicity - since per-row modifications are done essentially serially on the region, is this really that big of an issue? If not, then another jira isn't a necessary, but maybe something to consider going forward. Taking the row lock is going to be a big slowdown - it just depends on what the common use case is going to be.
(this is reproducing a some of what was commented on RB, but is nice to have succinctly).
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145266#comment-13145266 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
@Ted:
{quote}
I only see synchronized keyword on Constraints.add().
Have you tried using synchronization on other methods ?
{quote}
No I haven't, but this was the beginnings of making the setting synchronized. However, after thinking about it, that doesn't really make all that much sense for a bunch of static setting methods. I was going to push this off onto the user.
{quote}
Also, HTableDescriptor.values is protected field. We can change its actual implementation to ConcurrentHashMap, etc to accommodate for the concurrency you described.
{quote}
Seems a little excessive especially since all the synchronization on HTD has been done by users when necessary in the past. My hunch is that this is rarely an issue.
{quote}
If we store metadata about constraints in the Configuration object as I described @ 29/Oct/11 04:20, we utilize the available serialization mechanism.
The current approach deals with serialization itself. This is not as flexible as the above approach.
{quote}
+1 Making the changes. My initial thought was to mimic how CPs are added to the table, which handle writing their own bytes. But, yeah your recommendation would be makes it much easier (and consistent).
@Gary:
{quote}
Some runtime exceptions could indicate programming errors or bugs, in which case I think it's best to handle them the same way we handle unexpected errors in coprocessors – by either unloading or aborting, depending on configuration. We need to be careful for what we allow from user code running in process on region servers.
{quote}
AND
{quote}
I disagree here. Treating all possible exceptions the same seems wrong. An OutOfMemoryError is not equivalent to a NumberFormatException. We need some structure in place for what types of errors are allowed/expected, instead of swallowing everything and assuming it's just a constraint violation. I don't think asking the Constraint implementations to think about what errors they want to throw is much of a burden.
{quote}
+1 having the user propagate a DoNoRetryException subclass (eg. ConstraintException) - you make a good point. Working on the best way to handle unload/total failure.
{quote}
Sometimes people do weird things (I've seen people doing exactly what I was testing against), so while ugly, is not unheard of. An examples module would be great - I'll comment on 4336 about that. In the mean time, maybe just having an examples package (main/java/…hbase.constraint.examples) that can be updated as people find more uses for constraints? Though I'm not adverse to the book either.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168926#comment-13168926 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3897
-----------------------------------------------------------
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment8770>
Multiple Constraints are supported. Please update javadoc.
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment8772>
Priority is updated at the end.
I think we should document concurrency assumption in javadoc.
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment8768>
Can we call ByteArrayOutputStream.toString() directly ?
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment8769>
The dash is not needed.
- Ted
On 2011-12-13 21:38:03, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-13 21:38:03)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/docbkx/book.xml 9617950
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139858#comment-13139858 ]
Ted Yu commented on HBASE-4605:
-------------------------------
I think the name of table should be added to the following exception:
{code}
throw new IllegalArgumentException("Constraint: " + clazz.getName()
+ " is not associated with this table.");
{code}
Constraints.getKeyValueForClass() may return null. I think we should check for null in remove(HTableDescriptor, Class):
{code}
desc.remove(e.getKey().get());
{code}
and other methods.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172785#comment-13172785 ]
Zhihong Yu commented on HBASE-4605:
-----------------------------------
I created HBASE-5070 where further code changes should be made.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163923#comment-13163923 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3653
-----------------------------------------------------------
Answering most of the big issues on HBASE-4605 since there is a clean summary there by Ted.
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
<https://reviews.apache.org/r/2579/#comment8150>
Done
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment8151>
Done
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment8153>
this could definitely be an extension. +1 on future considerations.
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java
<https://reviews.apache.org/r/2579/#comment8152>
Done
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
<https://reviews.apache.org/r/2579/#comment8154>
done
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment8147>
I think we pulled getValueAsBytes() from HTD since this was the only use case. Agreed that it is a bit awkward, but we figured that since this is really the only use case for it, it didn't make sense to keep that function around.
Adding this back in makes it easier to remove guava, since that ser/deser can be kept entirely under the covers.
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment8148>
+1 on using the configuration.
Originally, this was done to ensure that the configuration could be completely open to the user, but constraining (no pun intended) the conf by a couple values is not a big deal.
However, again we are at the question of keeping it human readable or not. See top comment.
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java
<https://reviews.apache.org/r/2579/#comment8149>
Moved to documentation.
- Jesse
On 2011-11-29 20:19:41, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-29 20:19:41)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/docbkx/book.xml 3c12169
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139099#comment-13139099 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
bq. On 2011-10-26 23:01:06, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 49
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line49>
bq. >
bq. > This condition can be expressed as !constraints.isEmpty()
I was concerned with speed here - maybe a premature optimization and bit of assumption on my part that the check of is empty takes longer than a boolean, and that my way has minimal memory overhead
bq. On 2011-10-26 23:01:06, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 77
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line77>
bq. >
bq. > This has been done, right ?
yup, just some cruft.
bq. On 2011-10-26 23:01:06, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 83
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line83>
bq. >
bq. > Contents of put should be included.
+1
bq. On 2011-10-26 23:01:06, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 135
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line135>
bq. >
bq. > We can either add removeCoprocessor() to HTD or, persist a flag to tell ConstraintProcessor that it is disabled.
bq. >
bq. > I think the second approach may be better in our case.
see comments on Gary's comments.
bq. On 2011-10-26 23:01:06, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 140
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line140>
bq. >
bq. > Should be 'desc HTableDescriptor to read from'
+1
- Jesse
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review2870
-----------------------------------------------------------
On 2011-10-26 22:34:35, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-10-26 22:34:35)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159153#comment-13159153 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3551
-----------------------------------------------------------
Most of what I see is just style issues, and a number of the HTableDescriptor changes seem unnecessary. If we clean up those, and assuming the latest patch has some documentation, I think this is looking pretty good.
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
<https://reviews.apache.org/r/2579/#comment7921>
Nice addition to have
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
<https://reviews.apache.org/r/2579/#comment7913>
I don't see why these changes are necessary when I don't see them being used anywhere outside of HTD? Do these additions buy us anything? In my opinion, they're not making the code any clearer than the current Byte.toString(), etc.
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
<https://reviews.apache.org/r/2579/#comment7916>
style nit: if body should be contained in braces
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
<https://reviews.apache.org/r/2579/#comment7914>
Add constraints.size() to this message and you don't need the separate LOG.debug() message.
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
<https://reviews.apache.org/r/2579/#comment7912>
style nit: for loop body should be enclosed in braces
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment7917>
I see the usage outside of HTD now, but still don't see where this provides more utility than Bytes.toString()
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment7918>
Be consistent with braces here
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment7915>
if / else bodies should be contained in braces.
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java
<https://reviews.apache.org/r/2579/#comment7922>
This still seems odd to me. I can't see storing an integer value encoded as a string, in order to be able to check for it being a valid integer.
I would think most users would be storing integer values using Bytes.toInt(), for which the checks here would not work.
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
<https://reviews.apache.org/r/2579/#comment7919>
This differs from what seems to be the latest patch in JIRA (java_HBASE-4605_v3.txt). Are there other differences than the javadoc?
For a patch of this size, review still belongs here in review board (even with HadoopQA patch testing), so we should keep both versions in sync. Please update the diff here with the latest patch.
src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java
<https://reviews.apache.org/r/2579/#comment7920>
brace alignment and indentation
- Gary
On 2011-11-23 21:19:56, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-23 21:19:56)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139887#comment-13139887 ]
Ted Yu commented on HBASE-4605:
-------------------------------
@Jesse:
I only see synchronized keyword on Constraints.add().
Have you tried using synchronization on other methods ?
Also, HTableDescriptor.values is protected field. We can change its actual implementation to ConcurrentHashMap, etc to accommodate for the concurrency you described.
If we store metadata about constraints in the Configuration object as I described @ 29/Oct/11 04:20, we utilize the available serialization mechanism.
The current approach deals with serialization itself. This is not as flexible as the above approach.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158126#comment-13158126 ]
Hadoop QA commented on HBASE-4605:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505277/java_HBASE-4605_v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/385//console
This message is automatically generated.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152233#comment-13152233 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/
-----------------------------------------------------------
(Updated 2011-11-17 18:56:42.730664)
Review request for hbase.
Changes
-------
Adding in test classes that we accidentally dropped in last patch.
Also, removing HTableDescriptor.getValueAsBytes(String) in favor of on demand, in-line conversion using static calls.
Summary
-------
Most of the implementation for adding constraints as a coprocessor.
Looking for general comments on style/structure, though nitpicks are ok too.
Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
This addresses bug HBASE-4605.
https://issues.apache.org/jira/browse/HBASE-4605
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
Diff: https://reviews.apache.org/r/2579/diff
Testing
-------
Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
Thanks,
Jesse
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Add constraints as a top-level
feature
Posted by "Gary Helmling (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129094#comment-13129094 ]
Gary Helmling commented on HBASE-4605:
--------------------------------------
{quote}
My idea is to write a ConstraintProcessor that is a system level CP with system wide support for setting constraints on a table. this requires add 'top-level' configuration values that the user would set for constraints to run (which would be ordered like coprocessors), but they would just implement the 'Constraint' interface. This means modifying HTD and the shell to enable all these
{quote}
I like the idea of using a system level coprocessor with a minimal extension interface for the checks to be performed. For the actual interface, you could even use Predicate from the google guava lib, or have Constraint just be a named interface that extends Predicate<Put>. Not critical, but plugging in to a standard interface instead of doing a one-off may enable future uses...
For setting the constraint implementations to be applied per table, I agree that using table attributes is probably easiest. But I don't see why we need to modify HTableDescriptor to enable this? We currently have HBASE-4554, which is looking to enable setting table attributes for coprocessors from the shell. It seems like we could make that sufficiently generic to enable both the coprocessors case and this with just changes to the shell code?
> Add constraints as a top-level feature
> --------------------------------------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136514#comment-13136514 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
@Ted: its just supposed to be general design review, hence the lack of licenses (the "finishing touches" mentioned above). I can throw it up on RB though.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152312#comment-13152312 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3324
-----------------------------------------------------------
Quick question: the security coprocessor lives in a separate directory -- /security/src/... --- should this coprocessor live in its own as well?
These coprocessors could eventually be put into separate submodules to keep the core thinner.
This could likely be handled by follow on jira.
- jmhsieh
On 2011-11-17 18:56:42, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-17 18:56:42)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139110#comment-13139110 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
bq. To preserve order of constraints, we can embed their ordinal in HTD key prefix, such as "constraint 1 $", "constraint 2 $", etc.
Exactly what I was planning on doing and where I got the inspiration
{quote}
To enable/disable a constraint, the value (a Configuration object) for the key can carry a special entry:
"hbase.constraint.enabled" -> true/false
{quote}
I was thinking I would just remove the constraint processor from the list of coprocessors. This also gives you complete control over runtime slowdown - if you want to enable constraints, you pay a price, but if not it shouldn't hurt the system speed at all. This would be a good idiom for later, CP based features to follow.
If we have the true/false it would require doing checking on the regionserver to see if constraints are enabled AND that the CP is loaded to have the same potential, which leads to a lot of code bloat.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-4605:
-------------------------------
Attachment: constraint_as_cp.txt
Initial stab at implementing constraints as a coprocessor to processor to process all the constraints passed to a table.
Includes:
- Adding a varargs of constraints to a table
- Adding a varags of constraint and their own configuration to a table
- just enabling Constraints on a table (would use any constraints currently stored in the htd)
- skeleton for disabling constraints (debated feature, since also requires adding a removeCoprocessor feature to htd)
- integration and unit tests.
This is pretty close to what I was envisioning, it just needs a couple finishing touches (I think).
What does everyone think of the impl?
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Add constraints as a top-level
feature
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129506#comment-13129506 ]
Ted Yu commented on HBASE-4605:
-------------------------------
In processBatchCallback():
{code}
!(results[i] instanceof DoNotRetryIOException))) {
retry = true;
{code}
So in your example, you can wrap NumberFormatException in DoNotRetryIOException so that the retries are skipped.
> Add constraints as a top-level feature
> --------------------------------------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139519#comment-13139519 ]
Ted Yu commented on HBASE-4605:
-------------------------------
@Jesse:
You can decide what to put in this JIRA as long as future enhancement can be easily accommodated.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Gary Helmling (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168145#comment-13168145 ]
Gary Helmling commented on HBASE-4605:
--------------------------------------
{quote}
Guava- I didn't know about the effort to get rid of guava client side, it can be pulled if it is really a big issue. I just read through 3264 - is it the case that only the client side dependencies are used for the MR stuff? For this it works out really nicely for doing the auto-serializing (see changes to the CP stuff). We could say that the set bytes for key is at your own risk, and make strings the default.
{quote}
{quote}
getValueAsBytes() - this was -1'ed in a previous iteration as constraints was the only use case for it - wrapping it with the explicit functions seemed a decent solution.
{quote}
{quote}
Configuration - As far as human readable, how often are people going to need to check those values? The HTD already just keeps around bytes, so people don't check those over the wire - constraints would remain in the same style.
How would you feel about making that configurable? I'm thinking setting a debug flag in Constraints/configuration for that value.
{quote}
Wouldn't all of these parts be solved by just serializing the Configuration as XML? Then it's treated as a string and you don't need the HTD changes that introduce Guava, or getValueAsBytes(), and you get human readability as a nice side-effect. I don't see any benefit to serializing Configuration as a Writable, given these issues.
{quote}
check(Put) - this could definitely be an extension. +1 on future considerations.
{quote}
I agree that this extending constraints to cover {{Delete}} and {{Increment}} could be handled as additional issues. I just wanted to raise the question. It's worth thinking about here, since this is introducing a new client-exposed API. But there's still time to make additions if desired prior to 0.94.
{quote}
IntegerConstraint - In the end, I can let this go. You're right that having in the docs should be enough. Also, combined with the fact that we don't have any others by default (and that removing is far harder than adding) I'll drop it into just the docs
{quote}
Thanks, I think it's a valuable example.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158227#comment-13158227 ]
Ted Yu commented on HBASE-4605:
-------------------------------
>From https://builds.apache.org/job/PreCommit-HBASE-Build/386//testReport/org.apache.hadoop.hbase/TestRegionRebalancing/org_apache_hadoop_hbase_TestRegionRebalancing/:
{code}
java.lang.OutOfMemoryError: unable to create new native thread
{code}
Report from HadoopQA was not reliable these days.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156280#comment-13156280 ]
Ted Yu commented on HBASE-4605:
-------------------------------
@Jesse:
Patch v6 doesn't apply cleanly:
{code}
Hunk #13 FAILED at 1135.
1 out of 13 hunks FAILED -- saving rejects to file src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java.rej
{code}
Do you mind uploading a patch (--no-prefix) which applies to TRUNK so that HadoopQA can run through it ?
Thanks
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139101#comment-13139101 ]
Ted Yu commented on HBASE-4605:
-------------------------------
To preserve order of constraints, we can embed their ordinal in HTD key prefix, such as "constraint 1 $", "constraint 2 $", etc.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169032#comment-13169032 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
Hmm, didn't touch any of the admin code, but it looks like its another instance of the 'too many open files' issue coming up again.
No new warnings on any of the code in this patch (as far as I could tell).
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172753#comment-13172753 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3980
-----------------------------------------------------------
Very nice. Some comments below.
src/docbkx/book.xml
<https://reviews.apache.org/r/2579/#comment8986>
This needs filling out some; how do I add a constraint. Perhaps this is in package info? If so, add link to it from here?
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
<https://reviews.apache.org/r/2579/#comment8987>
We're adding white space here
Should there be an answering String add method?
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
<https://reviews.apache.org/r/2579/#comment8988>
This just removes cp from HTD list. If loaded on a regionserver, it continues to be so?
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
<https://reviews.apache.org/r/2579/#comment8989>
Constraints are in a constraint package? This sets up a precedent where each cp gets its own package? Can't we have constraint in coprocessor package?
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
<https://reviews.apache.org/r/2579/#comment8990>
Should have used http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/conf/Configurable.html?
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment8994>
Nit: Missing 'of', 'not' should be 'no'
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment8995>
'not' should be 'no'
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment8996>
Missing 'should'?
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment8997>
Nice class doc
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment8998>
I can't have a constraint on a Delete? Should this be a Mutation?
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9003>
key for sure is not going to be null?
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9002>
Style: Braces if on separate line. No need of braces if 'if' test and if statement are on same line. In fact this could have been written:
return value == null? null: new Pair<String, String>(key, value);
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9006>
This method is about adding constraints but the body of the method seems to be about upping priority. Should explain what priority upping has to do with constraint adding?
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9007>
Ain't these really fat? Ain't this going to bloat HTD?
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9008>
This method is about Strings, not byte arrays?
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9010>
This is going to serialize a fat Configuration into an HTD? If I list an HTD in shell, its going to have this Configuration pollution in it?
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9012>
Incomplete sentence
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9014>
This is expensive call. Its done infrequently?
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9015>
Can I ask whether a constraint enabled or disabled?
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9016>
Braces.
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9017>
For sure not null?
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9026>
Where do I get one of these? Which classloader to use?
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9027>
Should this be a priority comparator rather than a constraintcomparator?
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
<https://reviews.apache.org/r/2579/#comment9028>
Should there be protection against constraints being unloaded? For an administrator only?
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
<https://reviews.apache.org/r/2579/#comment9029>
This text could do w/ a read-through/edit (see things like the curly-brace here)
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
<https://reviews.apache.org/r/2579/#comment9032>
Do I have to alter the table after doing this?
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
<https://reviews.apache.org/r/2579/#comment9033>
Need a rolling restart
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
<https://reviews.apache.org/r/2579/#comment9034>
Is above missing the reenable of table after HTD editing?
This doc is excellent. Needs a pointer from book to here.
- Michael
On 2011-12-13 21:38:03, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-13 21:38:03)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/docbkx/book.xml 9617950
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Add constraints as a top-level
feature
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129495#comment-13129495 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
@Ted: yeah, I found that wrapping. However, tracing the execution back up, a simple put (or delete) ends up being called wrapped in a retry loop that tries the put every time a Throwable is received. So we end up with a RetriesExhaustedException, rather than the original exception being propagated back out. The path down for the put was HTable.put->doPut->flushCommits->HConnectionMananger.processBatchOfPuts, which does the retries. The end result is it takes minutes to do a commit because we are doing this checking and retrying, rather than failing out as the CP docs would imply. I feel like this could have serious impacts outside of this patch too..
Unless I'm missing something?
> Add constraints as a top-level feature
> --------------------------------------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172993#comment-13172993 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
bq. On 2011-12-20 00:20:28, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278>
bq. >
bq. > I wish we had brought up this argument to Gary earlier.
bq.
bq. Gary Helmling wrote:
bq. I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are:
bq.
bq. 1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies
bq. 2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants.
bq.
bq. Jesse Yates wrote:
bq. What do you mean by "to the side"? Ted mentions a centralized repo in HBASE-5070. Are you thinking of essentially a system-level table for just this stuff - kind of an aux-meta table that can be used to store just arbitrary data?
bq.
bq. On one hand cool, on the other, but big are the confs really going to be in the HTD? Probably not more than a couple lines, and you may or may not want to see it, but at the very least you will always get it, as opposed to worrying about another moving part, that may or may not be there "on the side".
Adding conf as String to HTD is for sure convenient. Just wondering what it'll look like when we describe a table. What happens when we want to alter a table? Maybe this a non-issue/just aesthetics. Have you tried it?
- Michael
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3990
-----------------------------------------------------------
On 2011-12-13 21:38:03, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-13 21:38:03)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/docbkx/book.xml 9617950
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169075#comment-13169075 ]
Zhihong Yu commented on HBASE-4605:
-----------------------------------
TestHTableDescriptor misses category.
Other than that I vote +1 on latest patch.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158947#comment-13158947 ]
Ted Yu commented on HBASE-4605:
-------------------------------
About the changes to book.xml:
{code}
+ Constraints could be also be used to ...
{code}
The above should read 'Constraints could also be used to'
{code}
where that checking enabled
{code}
The above should read 'where integrity checking is enabled'
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172979#comment-13172979 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
bq. On 2011-12-20 00:20:28, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 73
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64470#file64470line73>
bq. >
bq. > I was thinking about this too.
bq. > Since the current Constraint is for one table only, I couldn't see how deletion of certain row would break constraint.
If you look at HBASE-4999 you could, theoretically, break a constraint via a delete, since it considers the state of the underlying table. However, that is a later consideration. We already talked about moving Constraints to modifying Mutations in a later ticket.
bq. On 2011-12-20 00:20:28, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278>
bq. >
bq. > I wish we had brought up this argument to Gary earlier.
bq.
bq. Gary Helmling wrote:
bq. I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are:
bq.
bq. 1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies
bq. 2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants.
What do you mean by "to the side"? Ted mentions a centralized repo in HBASE-5070. Are you thinking of essentially a system-level table for just this stuff - kind of an aux-meta table that can be used to store just arbitrary data?
On one hand cool, on the other, but big are the confs really going to be in the HTD? Probably not more than a couple lines, and you may or may not want to see it, but at the very least you will always get it, as opposed to worrying about another moving part, that may or may not be there "on the side".
- Jesse
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3990
-----------------------------------------------------------
On 2011-12-13 21:38:03, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-13 21:38:03)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/docbkx/book.xml 9617950
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-4605:
-------------------------------
Attachment: java_HBASE-4605_v5.patch
For Hadoop-QA
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151892#comment-13151892 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/
-----------------------------------------------------------
(Updated 2011-11-17 07:53:56.106789)
Review request for hbase.
Changes
-------
Based on Ted's suggestion of using the inherent serialization in HTD, I attempted to switch over. However, this was met with some problems of going to/from String/byte[] in the value. This lead to the large number of changes in the current patch.
Essentially, I abstracted out the serialization/deserialization mechanism from the methods into two sets of guava functions (one set for keys, one for values) so we can encapsulate the serialization and in the future make it easier to change. With this encapsulation, I also updated the way CPs are stored to also use the built-in HTD serialization/deserialization mechanisms.
This patch also attempts to fix the remaining issues recently brought up in 4605, namely:
- Using a ConstraintException to specify an allowed constraint
- Just unloading the entire ConstraintProcessor on a runtimeexception
- dropping all the attempts at making Constraints (the class) thread safe
I'm working on the shell stuff (been busy), but figured I would throw this up in the meantime so we can get the java stuff solid.
Summary
-------
Most of the implementation for adding constraints as a coprocessor.
Looking for general comments on style/structure, though nitpicks are ok too.
Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
This addresses bug HBASE-4605.
https://issues.apache.org/jira/browse/HBASE-4605
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java ef47d0d
src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
Diff: https://reviews.apache.org/r/2579/diff
Testing
-------
Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
Thanks,
Jesse
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-4605:
-------------------------------
Attachment: java_HBASE-4605_v7.patch
Updated patch with fixed javadocs as per Ted's comments.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Add constraints as a top-level
feature
Posted by "Andrew Purtell (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129393#comment-13129393 ]
Andrew Purtell commented on HBASE-4605:
---------------------------------------
+1 on choice #1, with generalization of argument passing to coprocessors via table attributes. HBASE-4048 and HBASE-4554 are a start there.
> Add constraints as a top-level feature
> --------------------------------------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-4605:
------------------------------
Attachment: 4605-v6.txt
Patch that adds category to TestHTableDescriptor
I plan to integrate v6 if there is no objection from Gary.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169769#comment-13169769 ]
Zhihong Yu commented on HBASE-4605:
-----------------------------------
Test failures were due to 'Too many open files'
Will commit tomorrow.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152381#comment-13152381 ]
Ted Yu commented on HBASE-4605:
-------------------------------
I got the following for patch v5:
{code}
testEnableDisableRemove(org.apache.hadoop.hbase.constraint.TestConstraints) Time elapsed: 0.005 sec <<< ERROR!
java.lang.NullPointerException
at org.apache.hadoop.hbase.util.Bytes.toBytes(Bytes.java:425)
at org.apache.hadoop.hbase.HTableDescriptor$3.apply(HTableDescriptor.java:1153)
at org.apache.hadoop.hbase.HTableDescriptor$3.apply(HTableDescriptor.java:1150)
at org.apache.hadoop.hbase.constraint.Constraints.getKeyValueForClass(Constraints.java:151)
at org.apache.hadoop.hbase.constraint.Constraints.has(Constraints.java:137)
at org.apache.hadoop.hbase.constraint.TestConstraints.testEnableDisableRemove(TestConstraints.java:121)
{code}
{code}
testIntegerConstraint(org.apache.hadoop.hbase.constraint.TestIntegerConstraint): Unexpected exception, expected<java.lang.NumberFormatException> but was<org.apache.hadoop.hbase.constraint.ConstraintException>
{code}
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13162565#comment-13162565 ]
Ted Yu commented on HBASE-4605:
-------------------------------
Thanks Gary for the detailed review.
For the check() API:
{code}
public void check(Put p) throws ConstraintException;
{code}
We can abstract the input parameter as Mutation.
I think we need to reach agreement on the following major issues:
1. introduction of dependency of Guava in client library - Jonathan Gray would strongly disagree with this practice.
2. whether IntegerConstraint should be included in the constraint package
3. constraint configuration serialization method
If the current implementation for some of the above isn't critical to this feature (#1 and #2), we can defer to future JIRAs.
Actually a fourth issue (raised by Suraj under the discussion of 'Question on Coprocessors and Atomicity') is more important: if we don't provide atomicity by holding rowlock during the check, the use cases for Constraints would decrease.
Issue #4 definitely doesn't have to be covered by this JIRA.
The fifth issue is how IntegrationTestConstraint.java should be tagged.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136729#comment-13136729 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
Also, this patch (clearly) does not include changes to the shell or book documentation. Any pointers on where to look for the former would be really helpful, as I haven't looked into the shell much before (and don't have a ton of ruby experience). However, going to be looking into that next week.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136531#comment-13136531 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/
-----------------------------------------------------------
Review request for hbase.
Summary
-------
Most of the implementation for adding constraints as a coprocessor.
Looking for general comments on style/structure, though nitpicks are ok too.
Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
This addresses bug HBASE-4605.
https://issues.apache.org/jira/browse/HBASE-4605
Diffs
-----
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
Diff: https://reviews.apache.org/r/2579/diff
Testing
-------
Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
Thanks,
Jesse
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169570#comment-13169570 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
I'd like to get the comments updated as per the latest comments on RB. Otherwise, +1
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Add constraints as a top-level
feature
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129203#comment-13129203 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
@gary:
{quote}
I like the idea of using a system level coprocessor with a minimal extension interface for the checks to be performed. For the actual interface, you could even use Predicate from the google guava lib, or have Constraint just be a named interface that extends Predicate<Put>. Not critical, but plugging in to a standard interface instead of doing a one-off may enable future uses...
{quote}
That is exactly what I was thinking for the "top-level" implementation
{quote}
seems like we could make that sufficiently generic to enable both the coprocessors case and this with just changes to the shell code
{quote}
+1
Right now coprocessors have a special syntax for loading on table level, which feels kind of clunky to do by hand (specifying COPROCESSOR$). I feel like we could definitely help enable setting values with a more concrete syntax (like a setCoprocessor method that we have on the HTableDescriptor now), which should handle the numbering, etc.
So using an abstract version of the stuff from 4554 would definitely help with that. I don't know if we can just the use update shell though - we would probably need to update the java connection as well.
Right now the code for storing things in the conf would be fine, we just need to abstract it a little bit, so it would look something like:
{code}
public void addCoprocessor(name){
addProcessingElement("coprocessor$", name);}
public void addConstriant(name){
addProcessingElement("constriant$", name);}
public void addProcessingElement(String tag, String value){
...//all the checking/add currently in addCoprocessor
}
{code}
@keywal:
Since they are just table configuration values, turning them on/off will be relatively painless.
Cross-table transactions is separate can of worms and really goes against the whole design paradigm of HBase (see discussion on dev about this). This would be optimized to do single table checking, though people could implement cross table checks at serious cost (and later we can build in more optimized mechanisms if it is a common thing people do).
{quote}
HCatalog schema will be transformable as HBase constraints, adding value to the two of them...
{quote}
That should be super simple, it would just take a simple tool to create the corresponding constraints. I would use constraints to enforce things like data sanitation, rather than schema enforcement (its the last ditch barrier to things going into a table properly, since shipping things across the wire is expensive), which should be done client side, but it could definitely work.
> Add constraints as a top-level feature
> --------------------------------------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169764#comment-13169764 ]
Hadoop QA commented on HBASE-4605:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12507427/java_HBASE-4605_v7.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -152 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 75 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.client.TestInstantSchemaChange
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/508//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/508//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/508//console
This message is automatically generated.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Add constraints as a top-level
feature
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129516#comment-13129516 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
oh... yeah, that would work. First time digging into the code and missed that. Thanks Ted!
> Add constraints as a top-level feature
> --------------------------------------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169025#comment-13169025 ]
Hadoop QA commented on HBASE-4605:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12507271/java_HBASE-4605_v5.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -150 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 75 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAdmin
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/501//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/501//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/501//console
This message is automatically generated.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139100#comment-13139100 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > Jessie, this looks like a pretty good start. In addition to the individual comments, a couple of general questions:
bq. >
bq. > 1) Should ConstraintProcessor support some standard way of mapping Constraint implementations to the families/qualifiers they should apply to?
bq. > 2) Should we ship a base set of Constraint implementations for common cases?
bq. >
bq. > Even if the answer to both is "yes", they could be addressed as follow-up JIRAs. But it would be good to think through the end goal here.
1) I was thinking an individual constriant could check any/all of the cf/cq being used. We can do it via (2) where we have an abstract Constraint that makes it easy to check a given cf/cq.
So, no (kinda) and yes. For (2), I think it will be community usage/findings as to what people find useful - we can decide on a case-by-case basis if it is a valid contribution.
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 12
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53673#file53673line12>
bq. >
bq. > Is this true? The HTD values are stored in a HashMap, so ordering wouldn't be preserved.
We can add order preserving, if that makes sense. I can see it for cases where you want to check the cheap constraints first and then do the expensive only if the other ones pass.
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 39
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53673#file53673line39>
bq. >
bq. > Should we define a base exception here that is thrown? I'm thinking something like:
bq. >
bq. > ConstraintViolationException extends DoNotRetryIOException
bq. >
bq. > This way we don't need to wrap in DoNotRetryIOException later and we give implementers a bit more guidance on what to do.
I wanted to let people throw whatever exception they wanted to so they can be specific as to what happened with minimal overhead. For instance, the IntegerConstraint lets the constraint just check to and then throw a NumberFormatException if it fails. However, the code for the user (the actual constraint implementation) is super concise and easy.
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 80
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line80>
bq. >
bq. > If we use a defined exception class for constraint violations (see comment above), I think we can omit this try/catch block. Expected exceptions would already subclass DoNotRetryIOException, so no need to wrap it here. Other (non-expected) Throwables would be handled higher up by RegionCoprocessorHost.
I don't think that runtime exceptions on a constraint should be allowed to propagate up the CPHost - if the constraint fails then that put should fail, but not anything else.
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 75
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line75>
bq. >
bq. > I think this and the other addConstraints() method could just be named add().
+1
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 124
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line124>
bq. >
bq. > I get the idea of being able to enable/disable constraints on demand, without removing the individual constraints configured. But why do we need this and the private addConstraintProcessor() method? Why not move the addConstraintProcessor() implementation here and make addConstraints() call this instead?
bq. >
bq. > Also, I would just name this enable().
+1 That was some leftover cruft from development.
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 134
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line134>
bq. >
bq. > In addition to being able to disable constraint checking by removing the ConstraintProcessor CP, we probably also need a removeConstraints(HTableDescriptor) method that removes the ConstraintProcessor CP and all the "constraint $" attributes on HTD for complete cleanup.
bq. >
bq. > By the way, since the name of the class is Constraints, I think this method could just be named disable() -- the additional "Constraints" is redundant.
Just trying to be explicit with the naming.
I like the idea of the remove, but we just need to make sure it gets well documented as to what happens - I could see a people being confused.
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 13
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53676#file53676line13>
bq. >
bq. > Is this intended as a test class or as a usable implementation? If a test class, should be under src/test/java/...
bq. >
bq. > If a usable implementation, then, looking at it, I wonder how useful it is without a way of mapping constraints to individual qualifiers. Do you think that's something ConstraintProcessor should provide in general?
bq. >
bq. > It could be left up to end user implementations of Constraint to match which KeyValues they should apply to. But then I don't think any implementations we can bundle will have much value, unless they provide for their own qualifier matching via configuration values.
Its meant to be both an example and a test util. I can move it into test, but I can see cases where people might want to just check to make sure they only store values. Yes, it a really simple use case, but it also helps people to build more complex ones
bq. On 2011-10-29 02:11:25, Gary Helmling wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 23
bq. > <https://reviews.apache.org/r/2579/diff/1/?file=53676#file53676line23>
bq. >
bq. > This doesn't seem right. Wouldn't most ints be stored as Bytes.toBytes(int), not as a String?
Yeah, maybe. This is just an example. Depends on what people want to do...Maybe the fact that the impl can vary so widely makes the case for this to be a test class.
- Jesse
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review2917
-----------------------------------------------------------
On 2011-10-26 22:34:35, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-10-26 22:34:35)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171466#comment-13171466 ]
Zhihong Yu commented on HBASE-4605:
-----------------------------------
Guava dependency has been removed.
Feel free to provide your comments. I and Jesse would accommodate them.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172978#comment-13172978 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 413
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line413>
bq. >
bq. > This is expensive call. Its done infrequently?
Essentially, only when we update the configuration for a constraint - adding, changing enabled or not, setting priority. The alternative was to keep around the those values just as bytes in the value, but Gary's argument for keeping them in the configuration for ease and compatibility going forward was the basis for this change. These confs are expected to be very small, so this shouldn't really be _that_ bad to write.
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/package-info.java, line 43
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64474#file64474line43>
bq. >
bq. > Should there be protection against constraints being unloaded? For an administrator only?
Are there ways we can enforce that just admins can do certain things? Also, I feel like if people are creating their own tables, they should be free to modify their own tables. Clearly this gets into the security stuff of which tables you should able to modify, which could be a bigger proposition. Thoughts?
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 479
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line479>
bq. >
bq. > Can I ask whether a constraint enabled or disabled?
Yup - see enabled(HTD, Constraint).
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278>
bq. >
bq. > Ain't these really fat? Ain't this going to bloat HTD?
Yeah, it will bloat the HTD. Let's move this discussion down to comments on Ted's review or onto 5070.
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 166
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line166>
bq. >
bq. > key for sure is not going to be null?
it'll propagate a null pointer from the from the class->key conversion before the return value here is null. What's the standard on handling the null pointer issues client side? Ok to just let that propogate up? Defensive checks? Wrapping it in some other exception? ??
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java, line 1
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64469#file64469line1>
bq. >
bq. > Constraints are in a constraint package? This sets up a precedent where each cp gets its own package? Can't we have constraint in coprocessor package?
Having it in its own package was part of what I would consider making constraints a "top-level" feature. It really depends on how we want to handle doing constraints and CP related features going forward. For instance, the security stuff is not only its own package, its going into its own module. I would argue that constraints are less key than security, but important enough to warrant its own subsection. Otherwise, we can get into a situation where we either have a ton of classes in the coprocessors package (bad) or a ton of packages - for each new feature (potentially way worse). I think we need to treat them on a case-by-case basis going forward.
There are significant docs on constraints, making them not a great thing to just drop into the coprocessor folder. Maybe that can be the metric - number of lines of necessary documentation (as opposed to just all docs, to avoid people creating filler just to have its own package).
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/docbkx/book.xml, line 874
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64467#file64467line874>
bq. >
bq. > This needs filling out some; how do I add a constraint. Perhaps this is in package info? If so, add link to it from here?
Should the link just be to the javadocs (which we could forget to update when this release gets cut) or should it just be a comment that usage is further explained in the javadocs/package-info?
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java, line 480
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64468#file64468line480>
bq. >
bq. > We're adding white space here
bq. >
bq. > Should there be an answering String add method?
We don't have an addValue for bytes, so I figured it was unnecessary.
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 299
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line299>
bq. >
bq. > This method is about Strings, not byte arrays?
yup, this is from an earlier impl
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 588
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line588>
bq. >
bq. > Should this be a priority comparator rather than a constraintcomparator?
Internally, it is comparing the priorities, but those are local to the constraints. What we really care about in the end is the order the constraints are applied, hence, constraintComparator. But priority comparator could work too.
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/package-info.java, line 65
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64474#file64474line65>
bq. >
bq. > This text could do w/ a read-through/edit (see things like the curly-brace here)
hmmm, that line didn't make it into trunk, but giving it another once-over.
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java, line 26
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64469#file64469line26>
bq. >
bq. > Should have used http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/conf/Configurable.html?
Even more so, probably Configured? Kinda makes this class a little obsolete... I'm partial to keeping it in there, just to tie the two hierarchies together - by implemnting BaseConstraint, all they need to take care of is overriding the methods specified.
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 173
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line173>
bq. >
bq. > This method is about adding constraints but the body of the method seems to be about upping priority. Should explain what priority upping has to do with constraint adding?
Its in the package-info, but ok, here works too.
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 317
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line317>
bq. >
bq. > This is going to serialize a fat Configuration into an HTD? If I list an HTD in shell, its going to have this Configuration pollution in it?
Yeah, but there isn't a really good way of getting them across otherwise. Follow-up should be in 5070 and to ted's comment below.
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java, line 1072
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64468#file64468line1072>
bq. >
bq. > This just removes cp from HTD list. If loaded on a regionserver, it continues to be so?
I believe so - you should have to disable and then renable the table for it to take effect (though it may be that you actually need to roll the RS).
bq. On 2011-12-19 23:56:21, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 518
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line518>
bq. >
bq. > For sure not null?
yes...private method, so we should be able to expect valid elements - people shouldn't be able to mess with the values here/put bad data.
- Jesse
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3980
-----------------------------------------------------------
On 2011-12-13 21:38:03, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-13 21:38:03)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/docbkx/book.xml 9617950
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158215#comment-13158215 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
Hmmm, so it looks like most of them failed. So far, its passing on my local machine (OSX). Could this be a build system overload issue? Or something along those lines?
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172996#comment-13172996 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
bq. On 2011-12-20 00:20:28, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278>
bq. >
bq. > I wish we had brought up this argument to Gary earlier.
bq.
bq. Gary Helmling wrote:
bq. I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are:
bq.
bq. 1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies
bq. 2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants.
bq.
bq. Jesse Yates wrote:
bq. What do you mean by "to the side"? Ted mentions a centralized repo in HBASE-5070. Are you thinking of essentially a system-level table for just this stuff - kind of an aux-meta table that can be used to store just arbitrary data?
bq.
bq. On one hand cool, on the other, but big are the confs really going to be in the HTD? Probably not more than a couple lines, and you may or may not want to see it, but at the very least you will always get it, as opposed to worrying about another moving part, that may or may not be there "on the side".
bq.
bq. Michael Stack wrote:
bq. Adding conf as String to HTD is for sure convenient. Just wondering what it'll look like when we describe a table. What happens when we want to alter a table? Maybe this a non-issue/just aesthetics. Have you tried it?
Not yet - I was putting all the shell work off until 4879. Figured if it looked gross/crazy, we could handle it later. It also depends on what people are actually doing with the configuration stuff - if its not much, then this point is really moot - we can't really tell until people try it out and complain (or not smile).
- Jesse
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3990
-----------------------------------------------------------
On 2011-12-13 21:38:03, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-13 21:38:03)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/docbkx/book.xml 9617950
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168752#comment-13168752 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/
-----------------------------------------------------------
(Updated 2011-12-13 21:38:03.559611)
Review request for hbase.
Changes
-------
Making changes as per last comments on last diff:
1) Removing copywrite lines
2) Adding test categorization
3) Removing guava from HTD
4) Serializing conf to xml
5) Moving integerConstraint to package-info
Summary
-------
Most of the implementation for adding constraints as a coprocessor.
Looking for general comments on style/structure, though nitpicks are ok too.
Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
This addresses bug HBASE-4605.
https://issues.apache.org/jira/browse/HBASE-4605
Diffs (updated)
-----
src/docbkx/book.xml 9617950
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
Diff: https://reviews.apache.org/r/2579/diff
Testing
-------
Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
Thanks,
Jesse
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170371#comment-13170371 ]
Zhihong Yu commented on HBASE-4605:
-----------------------------------
Integrated to TRUNK.
Thanks for the patch Jesse.
Thanks for the review Gary.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4605:
--------------------------
Status: Patch Available (was: Open)
Patch testing v7.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170777#comment-13170777 ]
Hudson commented on HBASE-4605:
-------------------------------
Integrated in HBase-TRUNK-security #33 (See [https://builds.apache.org/job/HBase-TRUNK-security/33/])
HBASE-4605 Constraints, remove .orig files accidentally added in previous commit
HBASE-4605 Constraints (Jesse Yates)
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig
tedyu :
Files :
* /hbase/trunk/src/docbkx/book.xml
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159516#comment-13159516 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/
-----------------------------------------------------------
(Updated 2011-11-29 20:19:41.250509)
Review request for hbase.
Changes
-------
Updating to incorporate Gary and Ted's latest (as of 11/29) comments.
Mostly cleanup in syntax. Also pushing up latest documentation changes that we previously on the Jira.
Summary
-------
Most of the implementation for adding constraints as a coprocessor.
Looking for general comments on style/structure, though nitpicks are ok too.
Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
This addresses bug HBASE-4605.
https://issues.apache.org/jira/browse/HBASE-4605
Diffs (updated)
-----
src/docbkx/book.xml 3c12169
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
Diff: https://reviews.apache.org/r/2579/diff
Testing
-------
Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
Thanks,
Jesse
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172774#comment-13172774 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3990
-----------------------------------------------------------
Thanks for the valuable comments, Stack.
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
<https://reviews.apache.org/r/2579/#comment9036>
I was thinking about this too.
Since the current Constraint is for one table only, I couldn't see how deletion of certain row would break constraint.
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9035>
I wish we had brought up this argument to Gary earlier.
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
<https://reviews.apache.org/r/2579/#comment9037>
Currently this method is called by testUpdateConstraint()
- Ted
On 2011-12-13 21:38:03, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-13 21:38:03)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/docbkx/book.xml 9617950
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172781#comment-13172781 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
bq. On 2011-12-20 00:20:28, Ted Yu wrote:
bq. > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278
bq. > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278>
bq. >
bq. > I wish we had brought up this argument to Gary earlier.
I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are:
1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies
2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants.
- Gary
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3990
-----------------------------------------------------------
On 2011-12-13 21:38:03, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-13 21:38:03)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/docbkx/book.xml 9617950
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139113#comment-13139113 ]
Ted Yu commented on HBASE-4605:
-------------------------------
w.r.t. performance after several rounds of enabling/disabling, the following operations are both O(1):
1. retrieve Configuration object from HTD.values map for given Constraint
2. retrieve the boolean in this Configuration object for "hbase.constraint.enabled"
There would be impact on performance if the number of constraints is high.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Ted Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4605:
--------------------------
Attachment: 4605.v7
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152390#comment-13152390 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
Sorry about the quality of these patches Ted, something must be messed up with my IDE to not be catching some of this...
Anyways, yes, your right, that is broken. Comes from the switch to throwing the ConstraintException rather allowing an exception to be thrown.
Also, it looks like testEnableDisableRemove has gotten messed up from dropping the getValueAsBytes(). Fixing and reposting.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139859#comment-13139859 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
@Ted: concurrent modification exception - an inherent problem of using the iterator on the hash map (actually tried that first though :))
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139104#comment-13139104 ]
Ted Yu commented on HBASE-4605:
-------------------------------
To enable/disable a constraint, the value (a Configuration object) for the key can carry a special entry:
"hbase.constraint.enabled" -> true/false
This way the ordinal for the constraints would be preserved after arbitrary rounds of enabling/disabling.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137329#comment-13137329 ]
Ted Yu commented on HBASE-4605:
-------------------------------
See HBASE-4554 which can be used as example for setting coprocessor table attributes.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-4605:
------------------------------
Status: Open (was: Patch Available)
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169645#comment-13169645 ]
Hadoop QA commented on HBASE-4605:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12507384/4605-v6.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -150 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 75 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.client.TestInstantSchemaChange
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/506//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/506//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/506//console
This message is automatically generated.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-4605:
------------------------------
Comment: was deleted
(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505277/java_HBASE-4605_v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/385//console
This message is automatically generated.)
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152368#comment-13152368 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
bq. On 2011-11-17 20:51:45, jmhsieh wrote:
bq. > Quick question: the security coprocessor lives in a separate directory -- /security/src/... --- should this coprocessor live in its own as well?
bq. >
bq. > These coprocessors could eventually be put into separate submodules to keep the core thinner.
bq. >
bq. > This could likely be handled by follow on jira.
I was debating the idea of moving into it's own module. However, that feels a bit heavy weight for what is honestly just a handful of classes.
There could also be the argument that constraints need to be 'core' piece of a database (at least that is the feeling I get from many ex-SQLers).
Follow on ticket would be a good place, if we decide its the right move - pulling it out wouldn't be that bad.
- Jesse
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/#review3324
-----------------------------------------------------------
On 2011-11-17 18:56:42, Jesse Yates wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/2579/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-11-17 18:56:42)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Most of the implementation for adding constraints as a coprocessor.
bq.
bq. Looking for general comments on style/structure, though nitpicks are ok too.
bq.
bq. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
bq.
bq.
bq. This addresses bug HBASE-4605.
bq. https://issues.apache.org/jira/browse/HBASE-4605
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
bq. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
bq. src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/2579/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
bq.
bq.
bq. Thanks,
bq.
bq. Jesse
bq.
bq.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156411#comment-13156411 ]
Hadoop QA commented on HBASE-4605:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504935/4605.v7
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 68 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.client.TestInstantSchemaChange
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/352//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/352//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/352//console
This message is automatically generated.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139842#comment-13139842 ]
jiraposter@reviews.apache.org commented on HBASE-4605:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2579/
-----------------------------------------------------------
(Updated 2011-10-31 00:48:49.738577)
Review request for hbase.
Changes
-------
Updated based on comments on the last patch and in HBASE-4605 (corresponding JIRA).
Added feautures:
- enable
- disable
- remove
- enable constraint
- disable constraint
- remove constraint
- actually enforcing ordering of constraints based on add
- update constraint
- removal of CPs in HTD
Also, updated tests to cover new cases (which all pass locally).
Still TODO: tie constraints into shell
Summary
-------
Most of the implementation for adding constraints as a coprocessor.
Looking for general comments on style/structure, though nitpicks are ok too.
Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).
This addresses bug HBASE-4605.
https://issues.apache.org/jira/browse/HBASE-4605
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
Diff: https://reviews.apache.org/r/2579/diff
Testing
-------
Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.
Thanks,
Jesse
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13170720#comment-13170720 ]
Hudson commented on HBASE-4605:
-------------------------------
Integrated in HBase-TRUNK #2548 (See [https://builds.apache.org/job/HBase-TRUNK/2548/])
HBASE-4605 Constraints, remove .orig files accidentally added in previous commit
HBASE-4605 Constraints (Jesse Yates)
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig
tedyu :
Files :
* /hbase/trunk/src/docbkx/book.xml
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Add constraints as a top-level
feature
Posted by "Andrew Purtell (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130093#comment-13130093 ]
Andrew Purtell commented on HBASE-4605:
---------------------------------------
@Jesse: Can we change the title of this issue to simply "Constraints" ?
> Add constraints as a top-level feature
> --------------------------------------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152148#comment-13152148 ]
Ted Yu commented on HBASE-4605:
-------------------------------
I got compilation error trying to run integration test:
{code}
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile (default-testCompile) on project hbase: Compilation failure: Compilation failure:
[ERROR] /Users/zhihyu/trunk-hbase/src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java:[107,26] cannot find symbol
[ERROR] symbol : class AllFailConstraint
{code}
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168763#comment-13168763 ]
Hadoop QA commented on HBASE-4605:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12507251/java_HBASE-4605_v5.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/500//console
This message is automatically generated.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139709#comment-13139709 ]
Jesse Yates commented on HBASE-4605:
------------------------------------
@Ted: ok, fair enough. I'm just trying to work through the possibilities and get feedback (thanks btw!).
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: constraint_as_cp.txt, java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-4605:
-------------------------------
Attachment: java_HBASE-4605_v3.patch
Forgot --no-prefix. Let's try this again.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-4605) Constraints
Posted by "Ted Yu (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158139#comment-13158139 ]
Ted Yu edited comment on HBASE-4605 at 11/28/11 11:47 PM:
----------------------------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505280/java_HBASE-4605_v3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 69 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestRollingRestart
org.apache.hadoop.hbase.util.TestRegionSplitter
org.apache.hadoop.hbase.io.hfile.TestLruBlockCache
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/386//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/386//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/386//console
This message is automatically generated.
was (Author: hadoopqa):
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505280/java_HBASE-4605_v3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 69 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestRollingRestart
org.apache.hadoop.hbase.util.TestRegionSplitter
org.apache.hadoop.hbase.io.hfile.TestLruBlockCache
org.apache.hadoop.hbase.client.TestMultiParallel
org.apache.hadoop.hbase.master.TestRestartCluster
org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler
org.apache.hadoop.hbase.client.TestInstantSchemaChange
org.apache.hadoop.hbase.regionserver.TestStore
org.apache.hadoop.hbase.regionserver.wal.TestHLogBench
org.apache.hadoop.hbase.rest.TestGzipFilter
org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.rest.TestScannersWithFilters
org.apache.hadoop.hbase.TestInfoServers
org.apache.hadoop.hbase.regionserver.TestParallelPut
org.apache.hadoop.hbase.coprocessor.TestClassLoading
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
org.apache.hadoop.hbase.filter.TestColumnRangeFilter
org.apache.hadoop.hbase.mapred.TestTableInputFormat
org.apache.hadoop.hbase.client.TestHCM
org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
org.apache.hadoop.hbase.rest.TestStatusResource
org.apache.hadoop.hbase.TestRegionRebalancing
org.apache.hadoop.hbase.regionserver.wal.TestLogRollAbort
org.apache.hadoop.hbase.rest.TestVersionResource
org.apache.hadoop.hbase.client.TestScannerTimeout
org.apache.hadoop.hbase.client.TestFromClientSide
org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed
org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol
org.apache.hadoop.hbase.regionserver.TestSplitTransaction
org.apache.hadoop.hbase.rest.TestRowResource
org.apache.hadoop.hbase.rest.TestScannerResource
org.apache.hadoop.hbase.ipc.TestDelayedRpc
org.apache.hadoop.hbase.rest.client.TestRemoteAdmin
org.apache.hadoop.hbase.util.TestFSUtils
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.rest.TestTableResource
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
org.apache.hadoop.hbase.master.TestHMasterRPCException
org.apache.hadoop.hbase.util.TestIdLock
org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster
org.apache.hadoop.hbase.rest.TestTransform
org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint
org.apache.hadoop.hbase.client.TestInstantSchemaChangeSplit
org.apache.hadoop.hbase.regionserver.TestHRegion
org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyControl
org.apache.hadoop.hbase.client.TestMultipleTimestamps
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
org.apache.hadoop.hbase.catalog.TestMetaReaderEditor
org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
org.apache.hadoop.hbase.regionserver.TestKeepDeletes
org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed
org.apache.hadoop.hbase.client.TestMetaScanner
org.apache.hadoop.hbase.io.hfile.TestHFileBlock
org.apache.hadoop.hbase.client.TestTimestampsFilter
org.apache.hadoop.hbase.client.TestInstantSchemaChangeFailover
org.apache.hadoop.hbase.master.TestMaster
org.apache.hadoop.hbase.client.TestShell
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
org.apache.hadoop.hbase.rest.TestSchemaResource
org.apache.hadoop.hbase.TestAcidGuarantees
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan
org.apache.hadoop.hbase.regionserver.handler.TestOpenRegionHandler
org.apache.hadoop.hbase.regionserver.TestRpcMetrics
org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase
org.apache.hadoop.hbase.avro.TestAvroServer
org.apache.hadoop.hbase.rest.client.TestRemoteTable
org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
org.apache.hadoop.hbase.util.TestHBaseFsck
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove
org.apache.hadoop.hbase.client.TestHTableUtil
org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit
org.apache.hadoop.hbase.thrift.TestThriftServer
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface
org.apache.hadoop.hbase.util.TestMergeTool
org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics
org.apache.hadoop.hbase.util.TestMergeTable
org.apache.hadoop.hbase.master.TestMasterFailover
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.regionserver.wal.TestHLog
org.apache.hadoop.hbase.rest.TestMultiRowResource
org.apache.hadoop.hbase.TestMultiVersions
org.apache.hadoop.hbase.filter.TestFilter
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.master.TestMasterTransitions
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.master.TestSplitLogManager
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove
org.apache.hadoop.hbase.coprocessor.TestWALObserver
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
org.apache.hadoop.hbase.master.TestOpenedRegionHandler
org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/386//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/386//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/386//console
This message is automatically generated.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Add constraints as a top-level
feature
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-4605:
-------------------------------
Attachment: java_Constraint_v2.patch
Constraint implementation that is just added as a coprocessor. Not implemented as a Precondition for ease, though it could be ported over to that fairly easily. Basically, putting this up for posterity since the consensus seems to be pursuing #1 above.
Also, is there a better way to pass back exceptions from coprocessors? Right now, the exception causes a retry which is a huge timeout problem
> Add constraints as a top-level feature
> --------------------------------------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_Constraint_v2.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171708#comment-13171708 ]
stack commented on HBASE-4605:
------------------------------
Ok. Thanks lads. Good there is not extra dependency client-side (its a pain MR'ing)
I'll do a bit of review.....
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-4605:
------------------------------
Comment: was deleted
(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12507251/java_HBASE-4605_v5.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/500//console
This message is automatically generated.)
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164765#comment-13164765 ]
Hadoop QA commented on HBASE-4605:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505280/java_HBASE-4605_v3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -160 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 74 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/465//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/465//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/465//console
This message is automatically generated.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4605) Constraints
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated HBASE-4605:
-------------------------------
Attachment: java_HBASE-4605_v1.patch
Attaching patch.
Includes java implementation and documentation.
Moving shell modification to a follow-on ticket.
> Constraints
> -----------
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
> Affects Versions: 0.94.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch, java_HBASE-4605_v1.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira