You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "xuchuanyin (JIRA)" <ji...@apache.org> on 2018/02/26 06:30:00 UTC
[jira] [Assigned] (CARBONDATA-2204) Access tablestatus file too
many times during query
[ https://issues.apache.org/jira/browse/CARBONDATA-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuchuanyin reassigned CARBONDATA-2204:
--------------------------------------
Assignee: (was: xuchuanyin)
Description:
* Problems
Currently in carbondata, a single query will access tablestatus file 7 times, which will definitely slow down the query performance especially when this file is in remote cluster since reading this file is purely client side operation.
* Steps to reproduce
1. Add logger in `AtomicFileOperationsImpl.openForRead` and printout the file name to read.
2. Run a query on carbondata table. Here I ran `TestLoadDataGeneral.test("test data loading CSV file without extension name")`.
3. Observe the output log and search the keyword 'tablestatus'.
was:
# Problems
Currently in carbondata, a single query will access tablestatus file 7 times, which will definitely slow down the query performance especially when this file is in remote cluster since reading this file is purely client side operation.
# Steps to reproduce
1. Add logger in `AtomicFileOperationsImpl.openForRead` and printout the file name to read.
2. Run a query on carbondata table. Here I ran `TestLoadDataGeneral.test("test data loading CSV file without extension name")`.
3. Observe the output log and search the keyword 'tablestatus'.
> Access tablestatus file too many times during query
> ---------------------------------------------------
>
> Key: CARBONDATA-2204
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2204
> Project: CarbonData
> Issue Type: Improvement
> Components: data-query
> Affects Versions: 1.3.0
> Reporter: xuchuanyin
> Priority: Major
>
> * Problems
> Currently in carbondata, a single query will access tablestatus file 7 times, which will definitely slow down the query performance especially when this file is in remote cluster since reading this file is purely client side operation.
>
> * Steps to reproduce
> 1. Add logger in `AtomicFileOperationsImpl.openForRead` and printout the file name to read.
> 2. Run a query on carbondata table. Here I ran `TestLoadDataGeneral.test("test data loading CSV file without extension name")`.
> 3. Observe the output log and search the keyword 'tablestatus'.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)