Kudu Reader is not working with Java

[Ankit Kumar Mishra <An...@wu.com>]

Hello Team,

My Kudu Reader Java implementation is getting failed with exception - “java.lang.ClassNotFoundException: Failed to find data source: kudu. Please find packages at http://spark.apache.org/third-party-projects.html”

Java Implementation

String kuduTableName = "impala::" + kuduConfig.getDatabaseName() + "." + kuduConfig.getInTableName();

Map<String,String> kuduWriterMap = new HashMap<String, String>();
kuduWriterMap.put("kudu.master",kuduConfig.getMasterAddress());
kuduWriterMap.put("kudu.table",kuduTableName);
kuduWriterMap.put("kudu.socketReadTimeoutMs","720000");

Dataset<Row> readerDS = sparkSession.read().options(kuduWriterMap).format("kudu").load();

I have already defined below dependencies in my Pom.xml and jars are present in my External library.


<dependency>
    <groupId>org.apache.kudu</groupId>
    <artifactId>kudu-client</artifactId>
    <version>1.8.0</version>
    <scope>test</scope>
</dependency>

<dependency>
    <groupId>org.apache.kudu</groupId>
    <artifactId>kudu-spark2_2.11</artifactId>
    <version>1.8.0</version>
</dependency>

<dependency>
    <groupId>org.apache.kudu</groupId>
    <artifactId>kudu-spark2-tools_2.11</artifactId>
    <version>1.8.0</version>
</dependency>

<dependency>
    <groupId>org.apache.kudu</groupId>
    <artifactId>kudu-client</artifactId>
    <version>1.8.0</version>
    <type>test-jar</type>
    <scope>test</scope>
</dependency>



Please help.



Regards,

Ankit Mishra

The information transmitted, including any content in this communication is confidential, is intended only for the use of the intended recipient and is the property of the Western Union Company or its affiliates and subsidiaries. If you are not the intended recipient, you are hereby notified that any use of the information contained in or transmitted with the communication or dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the Western Union sender immediately by replying to this message and delete the original message

Re: Kudu Reader is not working with Java

[Grant Henke <gh...@cloudera.com>]

The ability to specify the format as simply "kudu" was added in 1.9.0. See
the release notes here:
https://kudu.apache.org/releases/1.9.0/docs/release_notes.html#rn_1.9.0_improvements

I would recommend upgrading to 1.9.0 or using the fully qualified name
"org.apache.kudu.spark.kudu"

Thanks,
Grant

On Mon, Apr 22, 2019 at 10:23 AM Ankit Kumar Mishra <An...@wu.com>
wrote:

> Hello Team,
>
>
>
> My Kudu Reader Java implementation is getting failed with exception - “java.lang.ClassNotFoundException:
> Failed to find data source: kudu. Please find packages at
> http://spark.apache.org/third-party-projects.html”
>
>
>
> *Java Implementation*
>
>
>
> String kuduTableName = *"impala::" *+ *kuduConfig*.getDatabaseName() + *"."
> *+ *kuduConfig*.getInTableName();
>
> Map<String,String> kuduWriterMap = *new *HashMap<String, String>();
> kuduWriterMap.put(*"kudu.master"*,*kuduConfig*.getMasterAddress());
> kuduWriterMap.put(*"kudu.table"*,kuduTableName);
> kuduWriterMap.put(*"kudu.socketReadTimeoutMs"*,*"720000"*);
>
> Dataset<Row> readerDS = *sparkSession*
> .read().options(kuduWriterMap).format(*"kudu"*).load();
>
>
>
> I have already defined below dependencies in my Pom.xml and jars are
> present in my External library.
>
>
>
> <*dependency*>
>     <*groupId*>org.apache.kudu</*groupId*>
>     <*artifactId*>kudu-client</*artifactId*>
>     <*version*>1.8.0</*version*>
>     <*scope*>test</*scope*>
> </*dependency*>
>
> <*dependency*>
>     <*groupId*>org.apache.kudu</*groupId*>
>     <*artifactId*>kudu-spark2_2.11</*artifactId*>
>     <*version*>1.8.0</*version*>
> </*dependency*>
>
> <*dependency*>
>     <*groupId*>org.apache.kudu</*groupId*>
>     <*artifactId*>kudu-spark2-tools_2.11</*artifactId*>
>     <*version*>1.8.0</*version*>
> </*dependency*>
>
> <*dependency*>
>     <*groupId*>org.apache.kudu</*groupId*>
>     <*artifactId*>kudu-client</*artifactId*>
>     <*version*>1.8.0</*version*>
>     <*type*>test-jar</*type*>
>     <*scope*>test</*scope*>
> </*dependency*>
>
>
>
> Please help.
>
>
>
> Regards,
>
> Ankit Mishra
>
>
> The information transmitted, including any content in this communication
> is confidential, is intended only for the use of the intended recipient and
> is the property of the Western Union Company or its affiliates and
> subsidiaries. If you are not the intended recipient, you are hereby
> notified that any use of the information contained in or transmitted with
> the communication or dissemination, distribution, or copying of this
> communication is strictly prohibited. If you have received this
> communication in error, please notify the Western Union sender immediately
> by replying to this message and delete the original message
>


-- 
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke