HIVE-1555 [JDBCStorageHandler]

[Dmitry Zagorulkin <za...@me.com>]

Hello everyone!

I have started work on HIVE-1555 feature.

There are few questions:

1)  Current module structure looks like:

├── jdbc-handler.iml
├── pom.xml
└── src
    ├── java
    │   └── org
    │       └── apache
    │           └── hive
    │               └── storagehandler
    │                   ├── JDBCStorageHandler.java
    │                   ├── db
    │                   │   ├── HiveJDBCTypeBridge.java
    │                   │   ├── HiveJDBCVendorBridge.java
    │                   │   └── mysql
    │                   │       ├── HiveMySqlTypeBridge.java
    │                   │       └── HiveMySqlVendorBridge.java
    │                   └── serde
    │                       └── JDBCSerde.java
    ├── main
    │   └── resources
    └── test
        └── java

pom.xml fragment:

  <build>
        <sourceDirectory>${basedir}/src/java</sourceDirectory>
        <testSourceDirectory>${basedir}/src/test</testSourceDirectory>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>test-jar</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

So this module builds fine, but i could not see jdbc-handler-XXX-.jar inside lib directory.
What should i add in assembly build.xml?

2) What is the main difference between HiveStoragePredicateHandler and HiveStorageHandler? I’ve found that we need to implement HiveStoragePredicateHandler if storage handler needs support decomposition of predicates being pushed down into table scans. Is it true that RDBMS does not have this feature and i need to implement HiveStorageHandler?

3) I’ve found many useful classes like DBInputFormat,DBOutputFormat,{Oracle|Mysql*.Reader} which exists in org.apache.hadoop.mapreduce.lib.db but org.apache.hadoop packages are excluded from build. Should i use this classes in my code? Should i need to exclude filtering for org.apache.hadoop group?


Thanks,

Dmitry Zagorulkin

Re: HIVE-1555 [JDBCStorageHandler]

[Dmitry Zagorulkin <za...@me.com>]

I have solved the first problem.
Looking forward for your clarification about other problems.

Thanks in advance.

> On 8 Nov 2016, at 13:08, Dmitry Zagorulkin <za...@me.com> wrote:
> 
> Hello everyone!
> 
> I have started work on HIVE-1555 feature.
> 
> There are few questions:
> 
> 1)  Current module structure looks like:
> 
> ├── jdbc-handler.iml
> ├── pom.xml
> └── src
>    ├── java
>    │   └── org
>    │       └── apache
>    │           └── hive
>    │               └── storagehandler
>    │                   ├── JDBCStorageHandler.java
>    │                   ├── db
>    │                   │   ├── HiveJDBCTypeBridge.java
>    │                   │   ├── HiveJDBCVendorBridge.java
>    │                   │   └── mysql
>    │                   │       ├── HiveMySqlTypeBridge.java
>    │                   │       └── HiveMySqlVendorBridge.java
>    │                   └── serde
>    │                       └── JDBCSerde.java
>    ├── main
>    │   └── resources
>    └── test
>        └── java
> 
> pom.xml fragment:
> 
>  <build>
>        <sourceDirectory>${basedir}/src/java</sourceDirectory>
>        <testSourceDirectory>${basedir}/src/test</testSourceDirectory>
>        <plugins>
>            <plugin>
>                <groupId>org.apache.maven.plugins</groupId>
>                <artifactId>maven-jar-plugin</artifactId>
>                <executions>
>                    <execution>
>                        <goals>
>                            <goal>test-jar</goal>
>                        </goals>
>                    </execution>
>                </executions>
>            </plugin>
>        </plugins>
>    </build>
> 
> So this module builds fine, but i could not see jdbc-handler-XXX-.jar inside lib directory.
> What should i add in assembly build.xml?
> 
> 2) What is the main difference between HiveStoragePredicateHandler and HiveStorageHandler? I’ve found that we need to implement HiveStoragePredicateHandler if storage handler needs support decomposition of predicates being pushed down into table scans. Is it true that RDBMS does not have this feature and i need to implement HiveStorageHandler?
> 
> 3) I’ve found many useful classes like DBInputFormat,DBOutputFormat,{Oracle|Mysql*.Reader} which exists in org.apache.hadoop.mapreduce.lib.db but org.apache.hadoop packages are excluded from build. Should i use this classes in my code? Should i need to exclude filtering for org.apache.hadoop group?
> 
> 
> Thanks,
> 
> Dmitry Zagorulkin
> 
> 
> 
> 
>