You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "mdayakar (via GitHub)" <gi...@apache.org> on 2024/01/09 16:18:43 UTC
[PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
mdayakar opened a new pull request, #4990:
URL: https://github.com/apache/hive/pull/4990
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/Hive/HowToContribute
2. Ensure that you have created an issue on the Hive project JIRA: https://issues.apache.org/jira/projects/HIVE/summary
3. Ensure you have added or run the appropriate tests for your PR:
4. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]HIVE-XXXXX: Your PR title ...'.
5. Be sure to keep the PR description updated to reflect all changes.
6. Please write your PR title to summarize what this PR proposes.
7. If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
For supporting Table type feature for HPL/SQL as a part of [HIVE-24383](https://issues.apache.org/jira/browse/HIVE-24383) to access table attributes like FIRST, LAST,.. **expr_dot** section is introduced in _Hplsql.g4_ file which got impacted _<table-name>.<column-name>_ expression. So currently as a part of _expr_dot_ section parsing, if table attribute is not used then considering it as column name and using the same instead of using _<table-name>.<column-name>_.
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
To support _table-name.column-name_ or _table-alias-name.column-name_ in the cursors(loops).
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce _any_ user-facing change?
No
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description, screenshot and/or a reproducable example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Hive versions or within the unreleased branches such as master.
If no, write 'No'.
-->
### Is the change a dependency upgrade?
No
<!--
If yes, please attach a file with output from mvn dependency:tree to validate a complete upgrade of dependency.
If no, write 'No'.
-->
### How was this patch tested?
Test cases are added.
_mvn test -Dtest=TestHplSqlViaBeeLine -pl itests/hive-unit -Pitests_
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
Posted by "mdayakar (via GitHub)" <gi...@apache.org>.
mdayakar commented on code in PR #4990:
URL: https://github.com/apache/hive/pull/4990#discussion_r1448953926
##########
itests/hive-unit/src/test/java/org/apache/hive/beeline/TestHplSqlViaBeeLine.java:
##########
@@ -610,6 +610,25 @@ public void testNullCast() throws Throwable {
testScriptFile(SCRIPT_TEXT, args(), "^(.(?!(NullPointerException)))*$", OutStream.ERR);
}
+ @Test
+ public void testTableAliasInColumnName() throws Throwable {
+ String SCRIPT_TEXT =
+ "DROP TABLE IF EXISTS input;\n" +
+ "DROP TABLE IF EXISTS result;\n" +
+ "CREATE TABLE input (col1 string, col2 int);\n" +
+ "CREATE TABLE result (res string);\n" +
+ "INSERT INTO input VALUES('Hive', 3);\n" +
+ "CREATE PROCEDURE p1() AS\n" +
+ " BEGIN\n" +
+ " FOR rec IN (select tab.col1, tab.col2 num from input tab) LOOP\n" +
Review Comment:
It is just giving 'Unhandled exception in HPL/SQL' and in HS2 log file also no useful information. I will raise a separate issue to handle it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
Posted by "mdayakar (via GitHub)" <gi...@apache.org>.
mdayakar commented on code in PR #4990:
URL: https://github.com/apache/hive/pull/4990#discussion_r1452004940
##########
itests/hive-unit/pom.xml:
##########
@@ -494,6 +494,12 @@
<artifactId>postgresql</artifactId>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>net.sf.supercsv</groupId>
+ <artifactId>super-csv</artifactId>
+ <version>${super-csv.version}</version>
+ <scope>test</scope>
+ </dependency>
Review Comment:
I was getting below error without this dependency so added above dependency.
```
[INFO] Running org.apache.hive.beeline.TestHplSqlViaBeeLine
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 27.295 s <<< FAILURE! - in org.apache.hive.beeline.TestHplSqlViaBeeLine
[ERROR] org.apache.hive.beeline.TestHplSqlViaBeeLine.testTableAliasInColumnName Time elapsed: 0.31 s <<< ERROR!
java.lang.NoClassDefFoundError: org/supercsv/encoder/CsvEncoder
at org.apache.hive.beeline.BeeLine.<init>(BeeLine.java:186)
at org.apache.hive.beeline.BeeLine.<init>(BeeLine.java:566)
at org.apache.hive.beeline.TestBeeLineWithArgs.testCommandLineScript(TestBeeLineWithArgs.java:164)
at org.apache.hive.beeline.TestHplSqlViaBeeLine.testScriptFile(TestHplSqlViaBeeLine.java:653)
at org.apache.hive.beeline.TestHplSqlViaBeeLine.testScriptFile(TestHplSqlViaBeeLine.java:640)
at org.apache.hive.beeline.TestHplSqlViaBeeLine.testTableAliasInColumnName(TestHplSqlViaBeeLine.java:629)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
Caused by: java.lang.ClassNotFoundException: org.supercsv.encoder.CsvEncoder
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 36 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4990:
URL: https://github.com/apache/hive/pull/4990#issuecomment-1883719374
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4990) **Quality Gate passed**
The SonarCloud Quality Gate passed, but some issues were introduced.
[7 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4990&resolved=false&inNewCodePeriod=true)
[0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4990&resolved=false&inNewCodePeriod=true)
No data about Coverage
No data about Duplication
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4990)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
Posted by "mdayakar (via GitHub)" <gi...@apache.org>.
mdayakar commented on code in PR #4990:
URL: https://github.com/apache/hive/pull/4990#discussion_r1446443586
##########
hplsql/src/main/java/org/apache/hive/hplsql/objects/TableClass.java:
##########
@@ -136,4 +155,8 @@ public MethodDictionary methodDictionary() {
public boolean rowType() {
return rowType;
}
+
+ public static boolean isTableAttributeExists(String property) {
+ return TABLE_ATTRIBUTES_LIST.contains(property);
Review Comment:
Changed the code accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
Posted by "mdayakar (via GitHub)" <gi...@apache.org>.
mdayakar commented on code in PR #4990:
URL: https://github.com/apache/hive/pull/4990#discussion_r1452012896
##########
hplsql/src/main/java/org/apache/hive/hplsql/objects/TableClass.java:
##########
@@ -96,9 +96,9 @@ public class TableClass implements HplClass {
} else { // single column
Row row = new Row();
row.addColumn(
- self.hplClass().columns().get(0).columnName(),
- self.hplClass().columns.get(0).columnTypeString(),
- args.get(1));
Review Comment:
Actually there is no change in this class. As a part of initial solution some changes were in this class later they are reverted. It looks this change some format change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
Posted by "mdayakar (via GitHub)" <gi...@apache.org>.
mdayakar closed pull request #4990: HIVE-27489: HPL/SQL does not support table aliases on column names in loops.
URL: https://github.com/apache/hive/pull/4990
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
Posted by "kasakrisz (via GitHub)" <gi...@apache.org>.
kasakrisz commented on code in PR #4990:
URL: https://github.com/apache/hive/pull/4990#discussion_r1446329347
##########
hplsql/src/main/java/org/apache/hive/hplsql/objects/TableClass.java:
##########
@@ -136,4 +155,8 @@ public MethodDictionary methodDictionary() {
public boolean rowType() {
return rowType;
}
+
+ public static boolean isTableAttributeExists(String property) {
+ return TABLE_ATTRIBUTES_LIST.contains(property);
Review Comment:
TABLE_ATTRIBUTES_LIST is more like a Set<String>.
Can `methodDictionary.containsKey(property)` be used instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4990:
URL: https://github.com/apache/hive/pull/4990#issuecomment-1884025006
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4990) **Quality Gate passed**
The SonarCloud Quality Gate passed, but some issues were introduced.
[7 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4990&resolved=false&inNewCodePeriod=true)
[0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4990&resolved=false&inNewCodePeriod=true)
No data about Coverage
No data about Duplication
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4990)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4990:
URL: https://github.com/apache/hive/pull/4990#issuecomment-1887438429
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4990) **Quality Gate passed**
Kudos, no new issues were introduced!
[0 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4990&resolved=false&inNewCodePeriod=true)
[0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4990&resolved=false&inNewCodePeriod=true)
No data about Coverage
No data about Duplication
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4990)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
Posted by "amansinha100 (via GitHub)" <gi...@apache.org>.
amansinha100 commented on code in PR #4990:
URL: https://github.com/apache/hive/pull/4990#discussion_r1446874029
##########
itests/hive-unit/src/test/java/org/apache/hive/beeline/TestHplSqlViaBeeLine.java:
##########
@@ -610,6 +610,25 @@ public void testNullCast() throws Throwable {
testScriptFile(SCRIPT_TEXT, args(), "^(.(?!(NullPointerException)))*$", OutStream.ERR);
}
+ @Test
+ public void testTableAliasInColumnName() throws Throwable {
+ String SCRIPT_TEXT =
+ "DROP TABLE IF EXISTS input;\n" +
+ "DROP TABLE IF EXISTS result;\n" +
+ "CREATE TABLE input (col1 string, col2 int);\n" +
+ "CREATE TABLE result (res string);\n" +
+ "INSERT INTO input VALUES('Hive', 3);\n" +
+ "CREATE PROCEDURE p1() AS\n" +
+ " BEGIN\n" +
+ " FOR rec IN (select tab.col1, tab.col2 num from input tab) LOOP\n" +
Review Comment:
Will this fail if 2 columns have the same alias name defined and within the LOOP we try to reference it ? It would be good to have a meaningful error such as 'ambiguous column reference'.
If this is not supported in the current code, it is ok to file a separate Jira and go ahead with this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-27489: HPL/SQL does not support table aliases on column names in loops. [hive]
Posted by "kasakrisz (via GitHub)" <gi...@apache.org>.
kasakrisz commented on code in PR #4990:
URL: https://github.com/apache/hive/pull/4990#discussion_r1451875835
##########
hplsql/src/main/java/org/apache/hive/hplsql/objects/TableClass.java:
##########
@@ -96,9 +96,9 @@ public class TableClass implements HplClass {
} else { // single column
Row row = new Row();
row.addColumn(
- self.hplClass().columns().get(0).columnName(),
- self.hplClass().columns.get(0).columnTypeString(),
- args.get(1));
Review Comment:
Seems to be an unrelated change.
##########
itests/hive-unit/pom.xml:
##########
@@ -494,6 +494,12 @@
<artifactId>postgresql</artifactId>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>net.sf.supercsv</groupId>
+ <artifactId>super-csv</artifactId>
+ <version>${super-csv.version}</version>
+ <scope>test</scope>
+ </dependency>
Review Comment:
Is this a necessary change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org