[jira] [Updated] (PHOENIX-5197) Use extraOptions to set the configuration for Spark Workers

["Chinmay Kulkarni (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PHOENIX-5197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chinmay Kulkarni updated PHOENIX-5197:
--------------------------------------
    Fix Version/s: connectors-1.0.0

> Use extraOptions to set the configuration for Spark Workers
> -----------------------------------------------------------
>
>                 Key: PHOENIX-5197
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5197
>             Project: Phoenix
>          Issue Type: Improvement
>    Affects Versions: 5.0.0, 4.15.0
>            Reporter: Chinmay Kulkarni
>            Assignee: Chinmay Kulkarni
>            Priority: Major
>             Fix For: connectors-1.0.0
>
>         Attachments: PHOENIX-5197-v1.patch, PHOENIX-5197.patch
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> We need a standardized way to set configurations for Spark driver and workers when reading and writing data.
> Quoting from my offline discussion with [~twdsilva@gmail.com]:
> _"if we use phoenixTableAsRDD to create the rdd then we can pass in a config_
>  _if we use the standard df.write.format() we can't pass in a conf_
>  _maybe the best was to do this is to put config in the option map when we call read/write .format and then set those in the config"_



--
This message was sent by Atlassian Jira
(v8.3.4#803005)