You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@carbondata.apache.org by "Kunal Kapoor (JIRA)" <ji...@apache.org> on 2018/08/02 08:43:00 UTC

[jira] [Updated] (CARBONDATA-2814) Refactor FileFactory and related code to take hadoop configuration from spark sessoion

     [ https://issues.apache.org/jira/browse/CARBONDATA-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kunal Kapoor updated CARBONDATA-2814:
-------------------------------------
    Description: 
Currently File Factory in core module creates HadoopConf using core-site.xml and as this is done in a static block therefore it is unable to get any dynamically set properties like SK and AK when set using spark set command. 

 

The proposed solution is to refactor the code of FileFactory so that it can accept configuration from the caller and the same configuration should be passed to the executors from driver.

> Refactor FileFactory and related code to take hadoop configuration from spark sessoion
> --------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-2814
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2814
>             Project: CarbonData
>          Issue Type: Sub-task
>            Reporter: Kunal Kapoor
>            Assignee: Kunal Kapoor
>            Priority: Critical
>
> Currently File Factory in core module creates HadoopConf using core-site.xml and as this is done in a static block therefore it is unable to get any dynamically set properties like SK and AK when set using spark set command. 
>  
> The proposed solution is to refactor the code of FileFactory so that it can accept configuration from the caller and the same configuration should be passed to the executors from driver.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)