[jira] [Comment Edited] (LENS-123) Ability to load different instances of same driver class

["Himanshu Gahlaut (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LENS-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14262873#comment-14262873 ] 

Himanshu Gahlaut edited comment on LENS-123 at 1/2/15 1:08 PM:
---------------------------------------------------------------

I'm thinking to move to a structured xml file which will look like below and will also support multiple drivers of same/different type. Code will first look for a driver specific configuration value from drivers tag and if it is not available, code will pick up default value from driver-default-config tag. e.g.: In below configuration hive driver in dc2 has a driver specific request-failure-retry-limit of 10 where as the dc1 hive driver will pick up the same from driver-default-config and get request-failure-retry-limit as 3. This structured file will allow multiple drivers and also help us in maintaining common properties across drivers at one place. Let me know your thoughts on it.

{code:xml}
<configuration>
    <drivers>
        <driver type="hive">
            <host>abcd4101.grid.dc1.mycompany.com</host>
            <storages>dc1,local,dc1_pie</storages>
        </driver>

        <driver type="hive">
            <host>abcd4101.grid.dc2.mycompany.com</host>
            <storages>dc1,local,dc1_pie</storages>
            <request-failure-retry-limit>10</request-failure-retry-limit>
        </driver>

        <driver type="jdbc">
            <db-uri>jdbc:mysql://abcd1002.dc1.mycompany.com:2347/my_global_dw</db-uri>
            <db-user>lens_readonly</db-user>
            <db-password>xxxxxxx</db-password>
            <storages>mydb,dc1_ib</storages>
        </driver>
    </drivers>

    <driver-default-config>
        <hive>
            <port>10000</port>
            <connection-class>org.apache.lens.driver.hive.RemoteThriftConnection</connection-class>
            <request-failure-retry-limit>3</request-failure-retry-limit>
            <connection-failure-retry-limit>3</connection-failure-retry-limit>
            <retry-delay-seconds>1</retry-delay-seconds>
            <socket-read-timeout>10</socket-read-timeout>
            <join-type>LEFTOUTER</join-type>
            <priority-ranges>HIGH,7,NORMAL,30,LOW,90,VERY_LOW</priority-ranges>
        </hive>

        <jdbc>
            <class>com.mysql.jdbc.Driver</class>
            <explain-keyword>roughly</jdbc-explain-keyword>
            <explain-before-select>false</explain-before-select>
            <join-type>INNER</join-type>
        </jdbc>

    </driver-default-config>
</configuration>
{code}

Also thinking to use Apache Commons Configuration to read the xml file.


was (Author: himanshu.gahlaut):
I'm thinking to move to a structured xml file which will look like below and will also support multiple drivers of same/different type. Code will first look for a driver specific configuration value from drivers tag and if it is not available, code will pick up default value from driver-default-config tag. e.g.: In below configuration hive driver in dc2 has a driver specific request-failure-retry-limit of 10 where as the dc1 hive driver will pick up the same from driver-default-config and get request-failure-retry-limit as 3. This structured file will allow multiple drivers and also help us in maintaining common properties across drivers at one place. Let me know your thoughts on it.

{code:xml}
<configuration>
    <drivers>
        <driver type="hive">
            <host>abcd4101.grid.dc1.mycompany.com</host>
            <storages>dc1,local,dc1_pie</storages>
        </driver>

        <driver type="hive">
            <host>abcd4101.grid.dc2.mycompany.com</host>
            <storages>dc1,local,dc1_pie</storages>
            <request-failure-retry-limit>10</request-failure-retry-limit>
        </driver>

        <driver type="jdbc">
            <db-uri>jdbc:mysql://abcd1002.dc1.mycompany.com:2347/my_global_dw</db-uri>
            <db-user>lens_readonly</db-user>
            <db-password>xxxxxxx</db-password>
            <storages>mydb,dc1_ib</storages>
        </driver>
    </drivers>

    <driver-default-config>
        <hive>
            <port>10000</port>
            <connection-class>org.apache.lens.driver.hive.RemoteThriftConnection</connection-class>
            <request-failure-retry-limit>3</request-failure-retry-limit>
            <connection-failure-retry-limit>3</connection-failure-retry-limit>
            <retry-delay-seconds>1</retry-delay-seconds>
            <socket-read-timeout>10</socket-read-timeout>
            <join-type>LEFTOUTER</join-type>
            <priority-ranges>HIGH,7,NORMAL,30,LOW,90,VERY_LOW</priority-ranges>
        </hive>

        <jdbc>
            <class>com.mysql.jdbc.Driver</class>
            <explain-keyword>roughly</jdbc-explain-keyword>
            <explain-before-select>false</explain-before-select>
            <join-type>INNER</join-type>
        </jdbc>

    </driver-default-config>
</configuration>
{code}

> Ability to load different instances of same driver class
> --------------------------------------------------------
>
>                 Key: LENS-123
>                 URL: https://issues.apache.org/jira/browse/LENS-123
>             Project: Apache Lens
>          Issue Type: Improvement
>          Components: server
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Himanshu Gahlaut
>             Fix For: 2.0
>
>
> Currently we are loading only one driver instance per class. We should be able to load multiple different instances of a driver class.
> For example, There can be multiple HiveDrivers talking to different HiveServers sitting on each colo. Or different JDBCDrivers talking to different dbs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)