You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu> on 2021/12/02 17:17:51 UTC
Change in asterixdb[master]: ASTERIXDB-2993: Proper error message for "parquet" on datalake and gcs
From Hussain Towaileb <hu...@gmail.com>:
Hussain Towaileb has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/14303 )
Change subject: ASTERIXDB-2993: Proper error message for "parquet" on datalake and gcs
......................................................................
ASTERIXDB-2993: Proper error message for "parquet" on datalake and gcs
Details:
- Parquet format external datasets are not supported for azure
datalake and google cloud storage, this change ensure
failing with a proper error message.
Change-Id: Id178275eec09d64582149f33d1f8e9c0cf79fc4a
---
M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
1 file changed, 14 insertions(+), 0 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/14303/1
diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
index 5194dae..d1f740e 100644
--- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
+++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
@@ -21,6 +21,7 @@
import static com.google.cloud.storage.Storage.BlobListOption;
import static java.nio.charset.StandardCharsets.UTF_8;
import static org.apache.asterix.common.exceptions.ErrorCode.EXTERNAL_SOURCE_ERROR;
+import static org.apache.asterix.common.exceptions.ErrorCode.INVALID_REQ_PARAM_VAL;
import static org.apache.asterix.common.exceptions.ErrorCode.PARAMETERS_NOT_ALLOWED_AT_SAME_TIME;
import static org.apache.asterix.common.exceptions.ErrorCode.PARAMETERS_REQUIRED;
import static org.apache.asterix.common.exceptions.ErrorCode.PARAM_NOT_ALLOWED_IF_PARAM_IS_PRESENT;
@@ -58,6 +59,7 @@
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_ESCAPE;
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_EXCLUDE;
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_EXTERNAL_SCAN_BUFFER_SIZE;
+import static org.apache.asterix.external.util.ExternalDataConstants.KEY_FORMAT;
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_INCLUDE;
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_QUOTE;
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_RECORD_END;
@@ -1675,6 +1677,12 @@
throw new CompilationException(ErrorCode.PARAMETERS_REQUIRED, srcLoc, ExternalDataConstants.KEY_FORMAT);
}
+ // parquet is not supported for azure datalake
+ if (isParquetFormat(configuration)) {
+ throw new CompilationException(INVALID_REQ_PARAM_VAL, srcLoc, KEY_FORMAT,
+ configuration.get(KEY_FORMAT));
+ }
+
validateIncludeExclude(configuration);
// Check if the bucket is present
@@ -1786,6 +1794,12 @@
throw new CompilationException(ErrorCode.PARAMETERS_REQUIRED, srcLoc, ExternalDataConstants.KEY_FORMAT);
}
+ // parquet is not supported for google cloud storage
+ if (isParquetFormat(configuration)) {
+ throw new CompilationException(INVALID_REQ_PARAM_VAL, srcLoc, KEY_FORMAT,
+ configuration.get(KEY_FORMAT));
+ }
+
validateIncludeExclude(configuration);
String container = configuration.get(ExternalDataConstants.CONTAINER_NAME_FIELD_NAME);
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/14303
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Id178275eec09d64582149f33d1f8e9c0cf79fc4a
Gerrit-Change-Number: 14303
Gerrit-PatchSet: 1
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-MessageType: newchange
Change in asterixdb[master]: ASTERIXDB-2993: Proper error message for "parquet" on datalake and gcs
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/14303 )
Change subject: ASTERIXDB-2993: Proper error message for "parquet" on datalake and gcs
......................................................................
Patch Set 1: Contrib+1
Analytics Compatibility Tests Successful
https://cbjenkins.page.link/qD2EGQnq1rEhx7pt9 : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/14303
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Id178275eec09d64582149f33d1f8e9c0cf79fc4a
Gerrit-Change-Number: 14303
Gerrit-PatchSet: 1
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Thu, 02 Dec 2021 19:39:46 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment
Change in asterixdb[master]: ASTERIXDB-2993: Proper error message for "parquet" on datalake and gcs
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
From Hussain Towaileb <hu...@gmail.com>:
Hussain Towaileb has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/14303 )
Change subject: ASTERIXDB-2993: Proper error message for "parquet" on datalake and gcs
......................................................................
ASTERIXDB-2993: Proper error message for "parquet" on datalake and gcs
Details:
- Parquet format external datasets are not supported for azure
datalake and google cloud storage, this change ensure
failing with a proper error message.
Change-Id: Id178275eec09d64582149f33d1f8e9c0cf79fc4a
---
M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
1 file changed, 14 insertions(+), 0 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/03/14303/1
diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
index 5194dae..d1f740e 100644
--- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
+++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
@@ -21,6 +21,7 @@
import static com.google.cloud.storage.Storage.BlobListOption;
import static java.nio.charset.StandardCharsets.UTF_8;
import static org.apache.asterix.common.exceptions.ErrorCode.EXTERNAL_SOURCE_ERROR;
+import static org.apache.asterix.common.exceptions.ErrorCode.INVALID_REQ_PARAM_VAL;
import static org.apache.asterix.common.exceptions.ErrorCode.PARAMETERS_NOT_ALLOWED_AT_SAME_TIME;
import static org.apache.asterix.common.exceptions.ErrorCode.PARAMETERS_REQUIRED;
import static org.apache.asterix.common.exceptions.ErrorCode.PARAM_NOT_ALLOWED_IF_PARAM_IS_PRESENT;
@@ -58,6 +59,7 @@
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_ESCAPE;
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_EXCLUDE;
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_EXTERNAL_SCAN_BUFFER_SIZE;
+import static org.apache.asterix.external.util.ExternalDataConstants.KEY_FORMAT;
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_INCLUDE;
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_QUOTE;
import static org.apache.asterix.external.util.ExternalDataConstants.KEY_RECORD_END;
@@ -1675,6 +1677,12 @@
throw new CompilationException(ErrorCode.PARAMETERS_REQUIRED, srcLoc, ExternalDataConstants.KEY_FORMAT);
}
+ // parquet is not supported for azure datalake
+ if (isParquetFormat(configuration)) {
+ throw new CompilationException(INVALID_REQ_PARAM_VAL, srcLoc, KEY_FORMAT,
+ configuration.get(KEY_FORMAT));
+ }
+
validateIncludeExclude(configuration);
// Check if the bucket is present
@@ -1786,6 +1794,12 @@
throw new CompilationException(ErrorCode.PARAMETERS_REQUIRED, srcLoc, ExternalDataConstants.KEY_FORMAT);
}
+ // parquet is not supported for google cloud storage
+ if (isParquetFormat(configuration)) {
+ throw new CompilationException(INVALID_REQ_PARAM_VAL, srcLoc, KEY_FORMAT,
+ configuration.get(KEY_FORMAT));
+ }
+
validateIncludeExclude(configuration);
String container = configuration.get(ExternalDataConstants.CONTAINER_NAME_FIELD_NAME);
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/14303
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Id178275eec09d64582149f33d1f8e9c0cf79fc4a
Gerrit-Change-Number: 14303
Gerrit-PatchSet: 1
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-MessageType: newchange
Change in asterixdb[master]: ASTERIXDB-2993: Proper error message for "parquet" on datalake and gcs
Posted by AsterixDB Code Review <do...@asterix-gerrit.ics.uci.edu>.
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/14303 )
Change subject: ASTERIXDB-2993: Proper error message for "parquet" on datalake and gcs
......................................................................
Patch Set 1:
Analytics Compatibility Compilation Successful
https://cbjenkins.page.link/aS8DoP9RDN1aRf2w5 : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/14303
To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Id178275eec09d64582149f33d1f8e9c0cf79fc4a
Gerrit-Change-Number: 14303
Gerrit-PatchSet: 1
Gerrit-Owner: Hussain Towaileb <hu...@gmail.com>
Gerrit-CC: Anon. E. Moose #1000171
Gerrit-CC: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Comment-Date: Thu, 02 Dec 2021 17:25:47 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: No
Gerrit-MessageType: comment