You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "Kunal Kapoor (Jira)" <ji...@apache.org> on 2021/02/18 15:26:00 UTC
[jira] [Resolved] (CARBONDATA-4126) Concurrent Compaction fails
with Load on table with SI
[ https://issues.apache.org/jira/browse/CARBONDATA-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kunal Kapoor resolved CARBONDATA-4126.
--------------------------------------
Fix Version/s: 2.1.1
Resolution: Fixed
> Concurrent Compaction fails with Load on table with SI
> ------------------------------------------------------
>
> Key: CARBONDATA-4126
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4126
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
> Affects Versions: 2.1.0
> Environment: Spark 2.4.5
> Reporter: Chetan Bhat
> Priority: Major
> Fix For: 2.1.1
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> [Steps] :-
> Create table, load data and create SI.
> create table brinjal (imei string,AMSize string,channelsId string,ActiveCountry string, Activecity string,gamePointId double,deviceInformationId double,productionDate Timestamp,deliveryDate timestamp,deliverycharge double) stored as carbondata TBLPROPERTIES('table_blocksize'='1');
> LOAD DATA INPATH 'hdfs://hacluster/chetan/vardhandaterestruct.csv' INTO TABLE brinjal OPTIONS('DELIMITER'=',', 'QUOTECHAR'= '"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'= 'imei,deviceInformationId,AMSize,channelsId,ActiveCountry,Activecity,gamePointId,productionDate,deliveryDate,deliverycharge');
> create index indextable1 ON TABLE brinjal (AMSize) AS 'carbondata';
>
> From one terminal load data to table and other terminal perform minor and major compaction on the table concurrently for some time.
> LOAD DATA INPATH 'hdfs://hacluster/chetan/vardhandaterestruct.csv' INTO TABLE brinjal OPTIONS('DELIMITER'=',', 'QUOTECHAR'= '"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'= 'imei,deviceInformationId,AMSize,channelsId,ActiveCountry,Activecity,gamePointId,productionDate,deliveryDate,deliverycharge');
> alter table brinjal compact 'minor';
> alter table brinjal compact 'major';
>
> [Expected Result] :- Concurrent Compaction should be success with Load on table with SI
>
> [Actual Issue] : - Concurrent Compaction fails with Load on table with SI
> *0: jdbc:hive2://linux-32:22550/> alter table brinjal compact 'major';*
> *Error: org.apache.spark.sql.AnalysisException: Compaction failed. Please check logs for more info. Exception in compaction Failed to acquire lock on segment 2, during compaction of table test.brinjal; (state=,code=0)*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)