You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metamodel.apache.org by LosD <gi...@git.apache.org> on 2016/10/10 14:58:25 UTC
[GitHub] metamodel pull request #133: Don't get killed by Hive SQLExceptions when fin...
GitHub user LosD opened a pull request:
https://github.com/apache/metamodel/pull/133
Don't get killed by Hive SQLExceptions when finding default schema name
This will allow Hive to throw its exceptions while we're trying to determine the default schema name, without that failing.
Also fixes a problem where columns types without a width would fail by specifying maximum allowed column width for VARCHAR types. (Maybe I should have fixed this in another PR, but it broke the Hive integration test, which drove me mad... Let me know if I should split them).
Fixes METAMODEL-1119
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/LosD/metamodel bug/METAMODEL-1119-workaround-hive-issue
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metamodel/pull/133.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #133
----
commit 4d81aef06eb9a93fbb70fb63b8d9645f25741615
Author: Dennis Du Kr�ger <d...@hp23c.dk>
Date: 2016-10-10T14:32:43Z
Don't get killed by Hive SQLExceptions when finding default schema name
This will allow Hive to throw its exceptions while we�re trying to determine the default schema name, without that failing.
Also fixes a problem where columns types without a width would fail by specifying maximum allowed column width for VARCHAR types. (Maybe I should have fixed this in another PR, but it broke the Hive integration test, which drove me mad... Let me know if I should split them).
Fixes METAMODEL-1119
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] metamodel pull request #133: Don't get killed by Hive SQLExceptions when fin...
Posted by kaspersorensen <gi...@git.apache.org>.
Github user kaspersorensen commented on a diff in the pull request:
https://github.com/apache/metamodel/pull/133#discussion_r82646932
--- Diff: jdbc/src/main/java/org/apache/metamodel/jdbc/dialects/HiveQueryRewriter.java ---
@@ -35,6 +35,10 @@ public String rewriteColumnType(ColumnType columnType, Integer columnSize) {
if (columnType == ColumnType.INTEGER) {
return "INT";
}
+ // Hive does not support VARCHAR without a width, nor VARCHAR(MAX).
+ if (columnType == ColumnType.STRING && columnSize == null) {
--- End diff --
Maybe checking with STRING isn't quite enough here. I could imagine a lot of Hive users using the more specific VARCHAR column type.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] metamodel pull request #133: Don't get killed by Hive SQLExceptions when fin...
Posted by kaspersorensen <gi...@git.apache.org>.
Github user kaspersorensen commented on a diff in the pull request:
https://github.com/apache/metamodel/pull/133#discussion_r82675958
--- Diff: jdbc/src/main/java/org/apache/metamodel/jdbc/dialects/HiveQueryRewriter.java ---
@@ -35,6 +35,10 @@ public String rewriteColumnType(ColumnType columnType, Integer columnSize) {
if (columnType == ColumnType.INTEGER) {
return "INT";
}
+ // Hive does not support VARCHAR without a width, nor VARCHAR(MAX).
+ if (columnType == ColumnType.STRING && columnSize == null) {
--- End diff --
No I wouldn't worry about it. There was a bug once about not rewriting STRING properly but I think that's just history.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] metamodel issue #133: Don't get killed by Hive SQLExceptions when finding de...
Posted by LosD <gi...@git.apache.org>.
Github user LosD commented on the issue:
https://github.com/apache/metamodel/pull/133
Okay, that should fix it.
BTW, another curious thing with Hive on JDBC: Hive inserts are _very_ slow, since it's starting a MapReduce job for every insert... So you'd expect them to prioritize the batch feature? Nope. No support. Every... Single... Insert... Becomes... It's... Own... Job... I think the integration test job uses around 10-15 seconds per insert on my laptop (granted, with Hive running in the MapR sandbox on VirtualBox, but the VM overhead shouldn't really be noticable).
I have a feeling that JDBC isn't really prioritized, so we might at some point see if there's another more low-level API to connect to Hive.
On the other hand, querying is fast, so in that regard it's fine.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] metamodel issue #133: Don't get killed by Hive SQLExceptions when finding de...
Posted by kaspersorensen <gi...@git.apache.org>.
Github user kaspersorensen commented on the issue:
https://github.com/apache/metamodel/pull/133
LGTM, will merge.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] metamodel pull request #133: Don't get killed by Hive SQLExceptions when fin...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/metamodel/pull/133
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---