You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@calcite.apache.org by GitBox <gi...@apache.org> on 2020/03/21 08:11:49 UTC
[GitHub] [calcite] ritesh-kapoor opened a new pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
ritesh-kapoor opened a new pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r396330273
##########
File path: site/_docs/reference.md
##########
@@ -2362,6 +2362,7 @@ semantics.
| o p | TO_DATE(string, format) | Converts *string* to a date using the format *format*
| o p | TO_TIMESTAMP(string, format) | Converts *string* to a timestamp using the format *format*
| o p | TRANSLATE(expr, fromString, toString) | Returns *expr* with all occurrences of each character in *fromString* replaced by its corresponding character in *toString*. Characters in *expr* that are not in *fromString* are not replaced
+| m | UNCOMPRESS(binary) | UnCompresses binary using zlib decompression and returns the result as a string.
| o | XMLTRANSFORM(xml, xslt) | Returns a string after applying xslt to supplied xml.
Review comment:
Alphabetical order is maintained only for Function name :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] danny0405 commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
danny0405 commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r408017646
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
For Apache Spark and Flink, it is just `string.getBytes()`, but for MySQL, PostgreSQL and MS-SQL, the outputs is different, i didn't have time to do a deeper research, maybe you can have a try.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] danny0405 commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
danny0405 commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r396205999
##########
File path: site/_docs/reference.md
##########
@@ -2362,6 +2362,7 @@ semantics.
| o p | TO_DATE(string, format) | Converts *string* to a date using the format *format*
| o p | TO_TIMESTAMP(string, format) | Converts *string* to a timestamp using the format *format*
| o p | TRANSLATE(expr, fromString, toString) | Returns *expr* with all occurrences of each character in *fromString* replaced by its corresponding character in *toString*. Characters in *expr* that are not in *fromString* are not replaced
+| m | UNCOMPRESS(binary) | UnCompresses binary using zlib decompression and returns the result as a string.
| o | XMLTRANSFORM(xml, xslt) | Returns a string after applying xslt to supplied xml.
Review comment:
The `m` should be in its alphabetical order.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on issue #1865: [CALCITE-3648]
MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on issue #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#issuecomment-604780316
> hi, @ritesh-kapoor Can this PR add a description?
Added.. Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] XuQianJin-Stars commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r396856276
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,23 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkNull("UNCOMPRESS(x'')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
Review comment:
I think in order to cover the test, we need to add a non-ByteString type test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r398998091
##########
File path: core/src/main/java/org/apache/calcite/runtime/CompressionFunctions.java
##########
@@ -62,4 +65,21 @@ public static ByteString compress(String data) {
}
}
+ /**
+ * MySql Decompression is based on zlib.
+ * <a href="https://docs.oracle.com/javase/8/docs/api/java/util/zip/Inflater.html">Inflater</a>
+ * is used to implement decompression.
+ */
+ public static String uncompress(ByteString data) {
+ try {
+ if (data == null || data.length() == 0) {
Review comment:
Added new test case. Thanks :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r401327869
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
`Error while executing SQL "select UNCOMPRESS(p0) from (values ('test')) as t(p0)": Error while compiling generated Java code:
public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root) {
final org.apache.calcite.linq4j.Enumerable _inputEnumerable = org.apache.calcite.linq4j.Linq4j.asEnumerable(new String[] {
"test"});
return new org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
return new org.apache.calcite.linq4j.Enumerator(){
public final org.apache.calcite.linq4j.Enumerator inputEnumerator = _inputEnumerable.enumerator();
public void reset() {
inputEnumerator.reset();
}
public boolean moveNext() {
return inputEnumerator.moveNext();
}
public void close() {
inputEnumerator.close();
}
public Object current() {
return org.apache.calcite.runtime.CompressionFunctions.uncompress((org.apache.calcite.avatica.util.ByteString) (inputEnumerator.current() == null ? (String) null : inputEnumerator.current().toString()));
}
};
}
};
}
public Class getElementType() {
return java.lang.String.class;
}
];`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r398385503
##########
File path: site/_docs/reference.md
##########
@@ -2362,6 +2362,7 @@ semantics.
| o p | TO_DATE(string, format) | Converts *string* to a date using the format *format*
| o p | TO_TIMESTAMP(string, format) | Converts *string* to a timestamp using the format *format*
| o p | TRANSLATE(expr, fromString, toString) | Returns *expr* with all occurrences of each character in *fromString* replaced by its corresponding character in *toString*. Characters in *expr* that are not in *fromString* are not replaced
+| m | UNCOMPRESS(binary) | UnCompresses binary using zlib decompression and returns the result as a string.
| o | XMLTRANSFORM(xml, xslt) | Returns a string after applying xslt to supplied xml.
Review comment:
Function name to be in alphabetical order makes more sense to me, I can raise separate PR if we want to change the order.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r401327869
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
[uncompress-test-error.txt](https://github.com/apache/calcite/files/4412537/uncompress-test-error.txt)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] XuQianJin-Stars commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r398304749
##########
File path: core/src/main/java/org/apache/calcite/runtime/CompressionFunctions.java
##########
@@ -62,4 +65,21 @@ public static ByteString compress(String data) {
}
}
+ /**
+ * MySql Decompression is based on zlib.
+ * <a href="https://docs.oracle.com/javase/8/docs/api/java/util/zip/Inflater.html">Inflater</a>
+ * is used to implement decompression.
+ */
+ public static String uncompress(ByteString data) {
+ try {
+ if (data == null || data.length() == 0) {
Review comment:
> For `data.length==0`, is it possible to be an empty string?
```
mysql> select uncompress(null),uncompress('');
+------------------+----------------+
| uncompress(null) | uncompress('') |
+------------------+----------------+
| NULL | |
+------------------+----------------+
1 row in set (0.00 sec)
```
It should be an empty string
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] danny0405 commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
danny0405 commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r396351187
##########
File path: site/_docs/reference.md
##########
@@ -2362,6 +2362,7 @@ semantics.
| o p | TO_DATE(string, format) | Converts *string* to a date using the format *format*
| o p | TO_TIMESTAMP(string, format) | Converts *string* to a timestamp using the format *format*
| o p | TRANSLATE(expr, fromString, toString) | Returns *expr* with all occurrences of each character in *fromString* replaced by its corresponding character in *toString*. Characters in *expr* that are not in *fromString* are not replaced
+| m | UNCOMPRESS(binary) | UnCompresses binary using zlib decompression and returns the result as a string.
| o | XMLTRANSFORM(xml, xslt) | Returns a string after applying xslt to supplied xml.
Review comment:
Personally i think each column should be in the alphabetical order, from left to right.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407853338
##########
File path: core/src/main/java/org/apache/calcite/runtime/CompressionFunctions.java
##########
@@ -62,4 +65,30 @@ public static ByteString compress(String data) {
}
}
+ /**
+ * MySql Decompression is based on zlib.
+ * <a href="https://docs.oracle.com/javase/8/docs/api/java/util/zip/Inflater.html">Inflater</a>
+ * is used to implement decompression.
+ */
+ public static String uncompress(ByteString data) {
+ try {
+ if (data == null) {
+ return null;
+ } else if (data.length() == 0) {
+ return "";
+ }
+ InflaterInputStream inflaterStream = new InflaterInputStream(
+ new ByteArrayInputStream(data.getBytes(), 4, data.length()));
+ return IOUtils.toString(inflaterStream);
+ } catch (IOException e) {
+ return null;
+ }
+ }
+
+ /**
+ * MySQl Supports uncompress function for string which returns NULL value.
+ */
+ public static String uncompress(String data) {
+ return null;
+ }
Review comment:
> This is not necessary, Calcite supports implicit type coercion, if you declare to support `OperandTypes.BINARY` only and pass a `STRING` actually, Calcite would try to coerce the `STRING` to binary before passing through to the runtime code.
Also Implicit type coercion from string to binary is not supported as of now
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] danny0405 commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
danny0405 commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407839984
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
I guess MySQL may trigger the implicit type coercion for `STRING` params.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] DonnyZone commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
DonnyZone commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407315042
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
I'm inclined to fix the validation issue first. If the problem can be reproduced outside of this PR, we can file another JIRA.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] XuQianJin-Stars commented on issue #1865: [CALCITE-3648]
MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on issue #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#issuecomment-604232418
hi, @ritesh-kapoor Can this PR add a description?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] danny0405 commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
danny0405 commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407841054
##########
File path: core/src/main/java/org/apache/calcite/runtime/CompressionFunctions.java
##########
@@ -62,4 +65,30 @@ public static ByteString compress(String data) {
}
}
+ /**
+ * MySql Decompression is based on zlib.
+ * <a href="https://docs.oracle.com/javase/8/docs/api/java/util/zip/Inflater.html">Inflater</a>
+ * is used to implement decompression.
+ */
+ public static String uncompress(ByteString data) {
+ try {
+ if (data == null) {
+ return null;
+ } else if (data.length() == 0) {
+ return "";
+ }
+ InflaterInputStream inflaterStream = new InflaterInputStream(
+ new ByteArrayInputStream(data.getBytes(), 4, data.length()));
+ return IOUtils.toString(inflaterStream);
+ } catch (IOException e) {
+ return null;
+ }
+ }
+
+ /**
+ * MySQl Supports uncompress function for string which returns NULL value.
+ */
+ public static String uncompress(String data) {
+ return null;
+ }
Review comment:
This is not necessary, Calcite supports implicit type coercion, if you declare to support `OperandTypes.BINARY` only and pass a `STRING` actually, Calcite would try to coerce the `STRING` to binary before passing through to the runtime code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407481593
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
MySQL Supports string values in uncompress function which returns NULL. Please refer:
`mysql> SELECT UNCOMPRESS('string');
+----------------------+
| UNCOMPRESS('string') |
+----------------------+
| NULL |
+----------------------+
1 row in set, 1 warning (0.00 sec)
`
So, This should not be failure test case
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] danny0405 commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
danny0405 commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407950565
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
The implicit type coercion did triggered for `UNCOMPRESS('string')` during SQL validation, but the runtime code gen does not do it well. I suggest to move the `UNCOMPRESS('string')` test to `SqlValidatorTest`, a data type validation test is enough.
We can add the operator test if we really support the `CAST(VARCHAR as BINARY)` in runtime. Maybe @DonnyZone can give a help.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r398998393
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,23 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkNull("UNCOMPRESS(x'')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
Review comment:
The contract doestnot allow string type as an argument. Instead I have added another negative test case. Thanks :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407843891
##########
File path: core/src/main/java/org/apache/calcite/runtime/CompressionFunctions.java
##########
@@ -62,4 +65,30 @@ public static ByteString compress(String data) {
}
}
+ /**
+ * MySql Decompression is based on zlib.
+ * <a href="https://docs.oracle.com/javase/8/docs/api/java/util/zip/Inflater.html">Inflater</a>
+ * is used to implement decompression.
+ */
+ public static String uncompress(ByteString data) {
+ try {
+ if (data == null) {
+ return null;
+ } else if (data.length() == 0) {
+ return "";
+ }
+ InflaterInputStream inflaterStream = new InflaterInputStream(
+ new ByteArrayInputStream(data.getBytes(), 4, data.length()));
+ return IOUtils.toString(inflaterStream);
+ } catch (IOException e) {
+ return null;
+ }
+ }
+
+ /**
+ * MySQl Supports uncompress function for string which returns NULL value.
+ */
+ public static String uncompress(String data) {
+ return null;
+ }
Review comment:
@danny0405 Can you please look for attached error, and help me understand the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on issue #1865: [CALCITE-3648]
MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on issue #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#issuecomment-612901844
All the review comments are resolved. Please let me know if something else is pending
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407843967
##########
File path: core/src/main/java/org/apache/calcite/runtime/CompressionFunctions.java
##########
@@ -62,4 +65,30 @@ public static ByteString compress(String data) {
}
}
+ /**
+ * MySql Decompression is based on zlib.
+ * <a href="https://docs.oracle.com/javase/8/docs/api/java/util/zip/Inflater.html">Inflater</a>
+ * is used to implement decompression.
+ */
+ public static String uncompress(ByteString data) {
+ try {
+ if (data == null) {
+ return null;
+ } else if (data.length() == 0) {
+ return "";
+ }
+ InflaterInputStream inflaterStream = new InflaterInputStream(
+ new ByteArrayInputStream(data.getBytes(), 4, data.length()));
+ return IOUtils.toString(inflaterStream);
+ } catch (IOException e) {
+ return null;
+ }
+ }
+
+ /**
+ * MySQl Supports uncompress function for string which returns NULL value.
+ */
+ public static String uncompress(String data) {
+ return null;
+ }
Review comment:
`Error while executing SQL "select UNCOMPRESS(p0) from (values ('test')) as t(p0)": Error while compiling generated Java code:
public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root) {
final org.apache.calcite.linq4j.Enumerable _inputEnumerable = org.apache.calcite.linq4j.Linq4j.asEnumerable(new String[] {
"test"});
return new org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
return new org.apache.calcite.linq4j.Enumerator(){
public final org.apache.calcite.linq4j.Enumerator inputEnumerator = _inputEnumerable.enumerator();
public void reset() {
inputEnumerator.reset();
}
public boolean moveNext() {
return inputEnumerator.moveNext();
}
public void close() {
inputEnumerator.close();
}
public Object current() {
return org.apache.calcite.runtime.CompressionFunctions.uncompress((org.apache.calcite.avatica.util.ByteString) (inputEnumerator.current() == null ? (String) null : inputEnumerator.current().toString()));
}
};
}
};
}
public Class getElementType() {
return java.lang.String.class;
}
];`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r400681825
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
@XuQianJin-Stars After master merge I'm facing below error
`[Error while executing SQL "select UNCOMPRESS(p0) from (values ('test')) as t(p0)": Error while compiling generated Java code:`
Which makes it difficult to write test case
` sqlTester.checkFails("UNCOMPRESS('test')","(?s).*Only ByteString.*", true);`
So I have removed it for now, Also as per contract only bytestring is allowed as an argument, Not sure if we really need this test case
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor closed pull request #1865: [CALCITE-3648]
MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor closed pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] XuQianJin-Stars commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r399242294
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
> Done :)
My fault, I listed an example. You need to increase the corresponding error message capture.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] DonnyZone commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
DonnyZone commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r401344853
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
The query should fail in validation phase.
It is too late to throw the exception in codegen phase.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] danny0405 commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
danny0405 commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407840348
##########
File path: core/src/main/java/org/apache/calcite/sql/fun/SqlLibraryOperators.java
##########
@@ -177,6 +177,11 @@ private SqlLibraryOperators() {
ReturnTypes.cascade(ReturnTypes.explicit(SqlTypeName.VARBINARY),
SqlTypeTransforms.TO_NULLABLE), null, OperandTypes.STRING, SqlFunctionCategory.STRING);
+ @LibraryOperator(libraries = {MYSQL})
+ public static final SqlFunction UNCOMPRESS = new SqlFunction("UNCOMPRESS", SqlKind.OTHER_FUNCTION,
+ ReturnTypes.cascade(ReturnTypes.explicit(SqlTypeName.VARCHAR),
+ SqlTypeTransforms.FORCE_NULLABLE), null,
+ OperandTypes.or(OperandTypes.STRING, OperandTypes.BINARY), SqlFunctionCategory.STRING);
Review comment:
Why we declare to support a `STRING` params if there is no runtime implementation ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r406782315
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
So should we unblock this feature or resolve validation issue first?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r399011539
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
Done :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407844565
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
@danny0405 Can you please look for attached error, and help me understand the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor opened a new pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor opened a new pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865
https://dev.mysql.com/doc/refman/8.0/en/encryption-functions.html#function_uncompress
e.g.:
SELECT UNCOMPRESS(COMPRESS('any string'));
-> 'any string'
SELECT UNCOMPRESS('any string');
-> NULL
Uncompresses a string compressed by the COMPRESS() function. If the argument is not a compressed value, the result is NULL. This function uses zlib library for decompression (https://docs.oracle.com/javase/8/docs/api/java/util/zip/Inflater.html). Otherwise, the return value is always NULL.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] DonnyZone commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
DonnyZone commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407968727
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
The problem in runtime lies in: `org.apache.calcite.avatica.util.ByteString) xxx`
This piece of code is generated by `EnumUtils.convert`. If we want to fix it, we need to add the converting logic (for `String` to `ByteString`) in `EnumUtils.convert`.
I am just wondering how to implement a general logic? `null` or `Exception` or other implementations?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r407853338
##########
File path: core/src/main/java/org/apache/calcite/runtime/CompressionFunctions.java
##########
@@ -62,4 +65,30 @@ public static ByteString compress(String data) {
}
}
+ /**
+ * MySql Decompression is based on zlib.
+ * <a href="https://docs.oracle.com/javase/8/docs/api/java/util/zip/Inflater.html">Inflater</a>
+ * is used to implement decompression.
+ */
+ public static String uncompress(ByteString data) {
+ try {
+ if (data == null) {
+ return null;
+ } else if (data.length() == 0) {
+ return "";
+ }
+ InflaterInputStream inflaterStream = new InflaterInputStream(
+ new ByteArrayInputStream(data.getBytes(), 4, data.length()));
+ return IOUtils.toString(inflaterStream);
+ } catch (IOException e) {
+ return null;
+ }
+ }
+
+ /**
+ * MySQl Supports uncompress function for string which returns NULL value.
+ */
+ public static String uncompress(String data) {
+ return null;
+ }
Review comment:
> This is not necessary, Calcite supports implicit type coercion, if you declare to support `OperandTypes.BINARY` only and pass a `STRING` actually, Calcite would try to coerce the `STRING` to binary before passing through to the runtime code.
Also Implicit type coercion from binary to string is not supported as of now
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] DonnyZone commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
DonnyZone commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r398296250
##########
File path: core/src/main/java/org/apache/calcite/runtime/CompressionFunctions.java
##########
@@ -62,4 +65,21 @@ public static ByteString compress(String data) {
}
}
+ /**
+ * MySql Decompression is based on zlib.
+ * <a href="https://docs.oracle.com/javase/8/docs/api/java/util/zip/Inflater.html">Inflater</a>
+ * is used to implement decompression.
+ */
+ public static String uncompress(ByteString data) {
+ try {
+ if (data == null || data.length() == 0) {
Review comment:
For `data.length==0`, is it possible to be an empty string?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] XuQianJin-Stars commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r399001523
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
Can add tests based on error？
`sqlTester.checkFails("UNCOMPRESS('test')","(?s).*Only ByteString.*", true);`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] ritesh-kapoor commented on a change in pull request
#1865: [CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
ritesh-kapoor commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r401327869
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
`rror while executing SQL "select UNCOMPRESS(p0) from (values ('test')) as t(p0)": Error while compiling generated Java code:
public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root) {
final org.apache.calcite.linq4j.Enumerable _inputEnumerable = org.apache.calcite.linq4j.Linq4j.asEnumerable(new String[] {
"test"});
return new org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
return new org.apache.calcite.linq4j.Enumerator(){
public final org.apache.calcite.linq4j.Enumerator inputEnumerator = _inputEnumerable.enumerator();
public void reset() {
inputEnumerator.reset();
}
public boolean moveNext() {
return inputEnumerator.moveNext();
}
public void close() {
inputEnumerator.close();
}
public Object current() {
return org.apache.calcite.runtime.CompressionFunctions.uncompress((org.apache.calcite.avatica.util.ByteString) (inputEnumerator.current() == null ? (String) null : inputEnumerator.current().toString()));
}
};
}
};
}
public Class getElementType() {
return java.lang.String.class;
}
];`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [calcite] danny0405 commented on a change in pull request #1865:
[CALCITE-3648] MySQL UNCOMPRESS function support
Posted by GitBox <gi...@apache.org>.
danny0405 commented on a change in pull request #1865: [CALCITE-3648] MySQL UNCOMPRESS function support
URL: https://github.com/apache/calcite/pull/1865#discussion_r401315074
##########
File path: core/src/test/java/org/apache/calcite/sql/test/SqlOperatorBaseTest.java
##########
@@ -5192,6 +5192,25 @@ private void checkNullOperand(SqlTester tester, String op) {
"07000000789c4bad48cc2dc84905000bc002ed", "VARBINARY NOT NULL");
}
+ @Test public void testUncompress() {
+ SqlTester sqlTester = tester(SqlLibrary.MYSQL);
+ sqlTester.checkNull("UNCOMPRESS(NULL)");
+ sqlTester.checkString("UNCOMPRESS(x'')", "", "VARCHAR");
+
+ sqlTester.checkNull("UNCOMPRESS(x'1233')");
+
+ sqlTester.checkString("UNCOMPRESS(COMPRESS('test'))",
+ "test", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'10000000789c4b4c44050033980611')",
+ "aaaaaaaaaaaaaaaa", "VARCHAR");
+
+ sqlTester.checkString("UNCOMPRESS(x'06000000789c2b4ecc2dc849050008de0283' )",
+ "sample", "VARCHAR");
+ sqlTester.checkString("UNCOMPRESS(x'07000000789c4bad48cc2dc84905000bc002ed')",
+ "example", "VARCHAR");
+ }
Review comment:
What is the error Java code like ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services