[jira] [Assigned] (DRILL-4052) Allow variables in drill-override.conf

["Arina Ielchiieva (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DRILL-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arina Ielchiieva reassigned DRILL-4052:
---------------------------------------

    Assignee: Arina Ielchiieva

> Allow variables in drill-override.conf
> --------------------------------------
>
>                 Key: DRILL-4052
>                 URL: https://issues.apache.org/jira/browse/DRILL-4052
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Tools, Build & Test
>    Affects Versions: 1.1.0
>         Environment: All
>            Reporter: John Omernik
>            Assignee: Arina Ielchiieva
>              Labels: features
>             Fix For: Future
>
>
> Basic variables, starting with "$HOSTNAME" if that simplified things would be a huge help to managing clusters of drill bits, especially in multi-tenant environments.  
> (Note: Perhaps being able to use all ENV variables through a $ENV:ENVVARNAME would address the issue completely, giving ultimate flexibility). 
> The use case/story I have is a drill cluster where I am running the individual drill bits in Marathon on Mesos. Since I am using MapR-FS (although this could work without MapR through Marathon downloading the config at run time) I point all drill bits to one config location.   This allows me to make a change in one place, making for easy changes and configuration management. 
> For the drill-env.sh, since that is shell script, I found I could easily put in items like specifying log locations that allowed me to use one shared log location for all drillbits, and then specifying that each logfile had the hostname of the drillbit, so I could keep all logs together and thus troubleshoot much easier.  
> When it came to drill-override.conf, I wanted to specify spill directory locations. Specifically, I wanted to use local volumes in MapR-FS.  It allows me to use a clustered filesystem, but have that data only exit on the node that the drill bit is running on.  So, Scheme: maprfs works, but what to specify for the location? I didn't want to specify the same location, /tmp/spill as then it would be shared by drillbits, and not guaranteed to be local to the drill bit (giving up on performance).  I wanted to create a volume that could be mounted /tmp/$hostname/drillspill   I could create the volume in drill-env.sh, but I could not specify that in the drill-override without creating a separate override file for each node. 
> Given scaling and multiple users/clusters, I really want to avoid that situation (a config for each drill bit), hence creating this JIRA for the ability to use variables in the drill-override. Happy to discuss other use cases as well. 
>    



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)