You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "Chetan Bhat (JIRA)" <ji...@apache.org> on 2018/08/23 06:54:00 UTC
[jira] [Assigned] (CARBONDATA-2875) Two different threads are
overwriting the same carbondatafile
[ https://issues.apache.org/jira/browse/CARBONDATA-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chetan Bhat reassigned CARBONDATA-2875:
---------------------------------------
Assignee: Ajantha Bhat
> Two different threads are overwriting the same carbondatafile
> -------------------------------------------------------------
>
> Key: CARBONDATA-2875
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2875
> Project: CarbonData
> Issue Type: Bug
> Components: data-load
> Affects Versions: 1.5.0
> Environment: Spark 2.2
> Reporter: Supritha
> Assignee: Ajantha Bhat
> Priority: Minor
>
> Steps :
> sparS.sql("create external table if not exists thread_x stored by 'carbondata' location 'D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread'").show();
> sparS.sql("select count(*) from thread_x").show(); ----> 300000 record count
> try{
> System.out.println("count called after write");
> Thread t1 = new Thread(new Trial(sparS));
> Thread t2 = new Thread(new Trial(sparS));
> //Each thread is holding 2 records each
> t1.setName("T1");
> t2.setName("T2");
> t1.start();
> Thread.sleep(10);
> t2.start();
> t1.join();
> t2.join();
> sparS.sql("select count(*) from thread_x").show();
> System.out.println("counrt called after load");
>
> Actual Issue : sparS.sql("select count(*) from thread_x").show(); -----> 300002
> Which implies Two different threads are overwriting the same carbondatafile.
> Expected : 300004 record count should be displayed.
> (ie) A carbonfile has to be created for each thread. Concurrent load is failing.
>
> From logs its is seen that two different threads are overwriting the same carbondatafile
> Line 298: 18/08/22 16:58:10 INFO CarbonUtil: WriterForwardPool: thread_x Copying C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096431985350_0\Fact\Part0\Segment_1534937289563\0\part-0-0_batchno0-0-1534937289563-1534937289561.carbondata --> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread
> Line 302: 18/08/22 16:58:10 INFO CarbonUtil: WriterForwardPool: thread_x Copying C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096431985350_0\Fact\Part0\Segment_1534937289563\0\0_batchno0-0-1534937289563-1534937289561.carbonindex --> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread
> Line 374: 18/08/22 16:58:11 INFO CarbonUtil: WriterForwardPool: thread_x Copying C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096980581610_0\Fact\Part0\Segment_1534937289563\0\part-0-0_batchno0-0-1534937289563-1534937289561.carbondata --> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread
> Line 378: 18/08/22 16:58:11 INFO CarbonUtil: WriterForwardPool: thread_x Copying C:\Users\SWX593~1\AppData\Local\Temp\\carbon16096980581610_0\Fact\Part0\Segment_1534937289563\0\0_batchno0-0-1534937289563-1534937289561.carbonindex --> D:/KernelTest_New/KernetTest/KernelTest_New/KernetTest/thread/thread
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)