You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by GitBox <gi...@apache.org> on 2019/05/21 08:28:49 UTC
[GitHub] [carbondata] xubo245 commented on a change in pull request #3188:
[CARBONDATA-3358] Support configurable decode for loading binary data,
support base64 and Hex decode.
xubo245 commented on a change in pull request #3188: [CARBONDATA-3358] Support configurable decode for loading binary data, support base64 and Hex decode.
URL: https://github.com/apache/carbondata/pull/3188#discussion_r285907453
##########
File path: processing/src/main/java/org/apache/carbondata/processing/loading/model/CarbonLoadModelBuilder.java
##########
@@ -426,6 +432,24 @@ private void validateAndSetColumnCompressor(CarbonLoadModel carbonLoadModel)
}
}
+
+ private void validateAndSetBinaryDecoder(CarbonLoadModel carbonLoadModel)
+ throws InvalidLoadOptionException {
+ try {
+ String binaryDecoder = carbonLoadModel.getBinaryDecoder();
+ if (StringUtils.isBlank(binaryDecoder)) {
Review comment:
it should set current value or set default value if didn't set hex and base64 in here, otherwise it will throw exception:
```
DataLoad failure: null
java.lang.Exception: DataLoad failure: null
at org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.loadCarbonData(CarbonDataRDDFactory.scala:491)
at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadData(CarbonLoadDataCommand.scala:630)
at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.processData(CarbonLoadDataCommand.scala:357)
at org.apache.spark.sql.execution.command.AtomicRunnableCommand$$anonfun$run$3.apply(package.scala:148)
at org.apache.spark.sql.execution.command.AtomicRunnableCommand$$anonfun$run$3.apply(package.scala:145)
at org.apache.spark.sql.execution.command.Auditable$class.runWithAudit(package.scala:104)
at org.apache.spark.sql.execution.command.AtomicRunnableCommand.runWithAudit(package.scala:141)
at org.apache.spark.sql.execution.command.AtomicRunnableCommand.run(package.scala:145)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services