You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2016/11/30 23:13:43 UTC
[kudu-CR] Enforce a max number of columns and reasonable identifier lengths
Hello Dan Burkert, Matthew Jacobs, Will Berkeley,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/5290
to review the following change.
Change subject: Enforce a max number of columns and reasonable identifier lengths
......................................................................
Enforce a max number of columns and reasonable identifier lengths
* Disallows unreasonable identifiers such as empty strings or strings
longer than a configurable maximum size. I set the default max size to
256. MySQL[1] and PostgreSQL[2] seem to limit identifiers to 64
characters, and Oracle[3] seems to limit to 30. So 256 seemed like it
should be plenty. The flag is marked as unsafe but if anyone has a
compatibility issue on upgrade, the limit can be raised.
* Enforces a maximum number of columns, set here to 300 as a default.
Really wide tables are an anti-pattern, and although some initial
testing shows that we won't completely fail until a few thousand
columns, we probably want to put a guard rail up well before the
failure point.
Again, if users are depending on a higher number, this can be worked
around by raising the default through an unsafe flag.
[1] http://dev.mysql.com/doc/refman/5.7/en/identifiers.html
[2] https://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html
[3] http://stackoverflow.com/questions/18248318/change-table-column-index-names-size-in-oracle-11g-or-12c
Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
---
M src/kudu/client/client-test.cc
M src/kudu/master/catalog_manager.cc
2 files changed, 184 insertions(+), 37 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/5290/1
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.
Change subject: KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged.
Change subject: KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
......................................................................
KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
* Disallows unreasonable identifiers such as empty strings or strings
longer than a configurable maximum size. I set the default max size to
256. MySQL[1] and PostgreSQL[2] seem to limit identifiers to 64
characters, and Oracle[3] seems to limit to 30. So 256 seemed like it
should be plenty. The flag is marked as unsafe but if anyone has a
compatibility issue on upgrade, the limit can be raised.
* Enforces a maximum number of columns, set here to 300 as a default.
Really wide tables are an anti-pattern, and although some initial
testing shows that we won't completely fail until a few thousand
columns, we probably want to put a guard rail up well before the
failure point.
Again, if users are depending on a higher number, this can be worked
around by raising the default through an unsafe flag.
[1] http://dev.mysql.com/doc/refman/5.7/en/identifiers.html
[2] https://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html
[3] http://stackoverflow.com/questions/18248318/change-table-column-index-names-size-in-oracle-11g-or-12c
Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Reviewed-on: http://gerrit.cloudera.org:8080/5290
Reviewed-by: Todd Lipcon <to...@apache.org>
Reviewed-by: Dan Burkert <da...@apache.org>
Tested-by: Kudu Jenkins
---
M java/kudu-client/pom.xml
M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java
M java/pom.xml
M src/kudu/client/client-test.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/master-test.cc
6 files changed, 225 insertions(+), 59 deletions(-)
Approvals:
Dan Burkert: Looks good to me, approved
Todd Lipcon: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 6
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change.
Change subject: KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 4
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Adar Dembo, Will Berkeley, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/5290
to look at the new patch set (#5).
Change subject: KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
......................................................................
KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
* Disallows unreasonable identifiers such as empty strings or strings
longer than a configurable maximum size. I set the default max size to
256. MySQL[1] and PostgreSQL[2] seem to limit identifiers to 64
characters, and Oracle[3] seems to limit to 30. So 256 seemed like it
should be plenty. The flag is marked as unsafe but if anyone has a
compatibility issue on upgrade, the limit can be raised.
* Enforces a maximum number of columns, set here to 300 as a default.
Really wide tables are an anti-pattern, and although some initial
testing shows that we won't completely fail until a few thousand
columns, we probably want to put a guard rail up well before the
failure point.
Again, if users are depending on a higher number, this can be worked
around by raising the default through an unsafe flag.
[1] http://dev.mysql.com/doc/refman/5.7/en/identifiers.html
[2] https://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html
[3] http://stackoverflow.com/questions/18248318/change-table-column-index-names-size-in-oracle-11g-or-12c
Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
---
M java/kudu-client/pom.xml
M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java
M java/pom.xml
M src/kudu/client/client-test.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/master-test.cc
6 files changed, 225 insertions(+), 59 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/5290/5
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Dan Burkert, Matthew Jacobs, Will Berkeley, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/5290
to look at the new patch set (#2).
Change subject: KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
......................................................................
KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
* Disallows unreasonable identifiers such as empty strings or strings
longer than a configurable maximum size. I set the default max size to
256. MySQL[1] and PostgreSQL[2] seem to limit identifiers to 64
characters, and Oracle[3] seems to limit to 30. So 256 seemed like it
should be plenty. The flag is marked as unsafe but if anyone has a
compatibility issue on upgrade, the limit can be raised.
* Enforces a maximum number of columns, set here to 300 as a default.
Really wide tables are an anti-pattern, and although some initial
testing shows that we won't completely fail until a few thousand
columns, we probably want to put a guard rail up well before the
failure point.
Again, if users are depending on a higher number, this can be worked
around by raising the default through an unsafe flag.
[1] http://dev.mysql.com/doc/refman/5.7/en/identifiers.html
[2] https://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html
[3] http://stackoverflow.com/questions/18248318/change-table-column-index-names-size-in-oracle-11g-or-12c
Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
---
M src/kudu/client/client-test.cc
M src/kudu/master/catalog_manager.cc
2 files changed, 184 insertions(+), 37 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/5290/2
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Dan Burkert, Matthew Jacobs, Will Berkeley, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/5290
to look at the new patch set (#3).
Change subject: KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
......................................................................
KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
* Disallows unreasonable identifiers such as empty strings or strings
longer than a configurable maximum size. I set the default max size to
256. MySQL[1] and PostgreSQL[2] seem to limit identifiers to 64
characters, and Oracle[3] seems to limit to 30. So 256 seemed like it
should be plenty. The flag is marked as unsafe but if anyone has a
compatibility issue on upgrade, the limit can be raised.
* Enforces a maximum number of columns, set here to 300 as a default.
Really wide tables are an anti-pattern, and although some initial
testing shows that we won't completely fail until a few thousand
columns, we probably want to put a guard rail up well before the
failure point.
Again, if users are depending on a higher number, this can be worked
around by raising the default through an unsafe flag.
[1] http://dev.mysql.com/doc/refman/5.7/en/identifiers.html
[2] https://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html
[3] http://stackoverflow.com/questions/18248318/change-table-column-index-names-size-in-oracle-11g-or-12c
Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
---
M java/kudu-client/pom.xml
M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java
M java/pom.xml
M src/kudu/client/client-test.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/master-test.cc
6 files changed, 225 insertions(+), 59 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/5290/3
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.
Change subject: KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
......................................................................
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/5290/4/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
PS4, Line 131: e");
missing period
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 4
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.
Change subject: KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
......................................................................
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/5290/2//COMMIT_MSG
Commit Message:
PS2, Line 9: * Disallows unreasonable identifiers such as empty strings or strings
: longer than a configurable maximum size.
I understand the argument for doing this for column names, (since, as you note in the patch, schemas are copied a lot), but I'm not a huge fan of this for table names. I imagine the limitations in other databases stem from the fact that their early versions are so old, and nobody bothered to (or couldn't) lift the limitations since.
But, 256 is a lot, and since it can be overridden, I don't think it's a huge deal either way.
http://gerrit.cloudera.org:8080/#/c/5290/1/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
PS1, Line 862: K
k
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.
Change subject: KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
......................................................................
Patch Set 3: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
......................................................................
Patch Set 5: Code-Review+2
carrying +2 from adar/will/dan (just added a period)
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: KUDU-1775 (part 2). Enforce a max number of columns and reasonable identifier lengths
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/5290/3/java/pom.xml
File java/pom.xml:
Line 68: <mockito-core.version>1.9.0</mockito-core.version>
had to switch to mockito-core and pull in hamcrest-core, since I wanted to use a hamcrest matcher from within junit, and junit and mockito seemed to disagree about matcher versions on the classpath
--
To view, visit http://gerrit.cloudera.org:8080/5290
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I9cab30dd1ea78a160403c79442b08739ddc92f12
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes