You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "Manish Gupta (JIRA)" <ji...@apache.org> on 2017/12/12 05:26:00 UTC
[jira] [Resolved] (CARBONDATA-1869) (Carbon1.3.0 - Spark 2.2) Null
pointer exception thrown when concurrent load and select queries executed
for table with dictionary exclude or NO_INVERTED_INDEX
[ https://issues.apache.org/jira/browse/CARBONDATA-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manish Gupta resolved CARBONDATA-1869.
--------------------------------------
Resolution: Fixed
> (Carbon1.3.0 - Spark 2.2) Null pointer exception thrown when concurrent load and select queries executed for table with dictionary exclude or NO_INVERTED_INDEX
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CARBONDATA-1869
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1869
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
> Affects Versions: 1.3.0
> Environment: 3 node ant cluster
> Reporter: Chetan Bhat
> Assignee: dhatchayani
> Labels: DFX
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Steps -
> From beeline terminal a table is created with with table properties having dictionary exclude or NO_INVERTED_INDEX-
> create table test(a1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ('DICTIONARY_EXCLUDE'='a1'); or
> create table test(a1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ('NO_INVERTED_INDEX'='a1');
> From 3 concurrent beeline terminals the below sequence of insert into select and select queries are executed 120 times.
> insert into test select 2147483647;
> select * from test;
> select count(*) from test;
> select a1 from test;
> select round(a1),bround(a1),floor(a1),ceil(a1),rand(),exp(a1),ln(a1),log10(a1),log2(1),log(a1),pow(a1,a1),sqrt(a1),bin(a1),pmod(a1,a1),sin(a1),asin(a1),cos(a1),tan(a1),atan(a1),degrees(a1),radians(a1),positive(a1),negative(a1),sign(a1),factorial(a1),cbrt(a1) from test;
> 【Expected Output】：The insert into select query should be successful and the null pointer exception should not be thrown when concurrent load and select queries executed for table with dictionary exclude or NO_INVERTED_INDEX.
> 【Actual Output】：Null pointer exception thrown when concurrent load and select queries executed with table properties having dictionary exclude or NO_INVERTED_INDEX
> 0: jdbc:hive2://10.18.98.136:23040> insert into test select 2147483647;
> Error: java.lang.NullPointerException (state=,code=0)
> *+{color:red}Stacktrace:{color}+*
> java.lang.NullPointerException
> at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.delete(AbstractDFSCarbonFile.java:152)
> at org.apache.carbondata.processing.util.DeleteLoadFolders.physicalFactAndMeasureMetadataDeletion(DeleteLoadFolders.java:90)
> at org.apache.carbondata.processing.util.DeleteLoadFolders.deleteLoadFoldersFromFileSystem(DeleteLoadFolders.java:134)
> at org.apache.carbondata.spark.rdd.DataManagementFunc$.deleteLoadsAndUpdateMetadata(DataManagementFunc.scala:187)
> at org.apache.carbondata.spark.rdd.CarbonDataRDDFactory$.loadCarbonData(CarbonDataRDDFactory.scala:296)
> at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadData(CarbonLoadDataCommand.scala:362)
> at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.processData(CarbonLoadDataCommand.scala:193)
> at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.run(CarbonLoadDataCommand.scala:65)
> at org.apache.spark.sql.execution.command.management.CarbonInsertIntoCommand.processData(CarbonInsertIntoCommand.scala:43)
> at org.apache.spark.sql.execution.command.DataCommand.run(package.scala:71)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67)
> at org.apache.spark.sql.Dataset.<init>(Dataset.scala:182)
> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:67)
> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:623)
> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:691)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)