You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Dima Fadeyev <li...@gmail.com> on 2016/07/14 13:26:47 UTC
export from Teradata
Hello, everyone,
I'm new to nifi and this mailing list, I'm evaluating if we could extract
data (tables) from Teradata to local fs or HDFS with nifi. Is that possible?
What I've done so far was creating an "Execute SQL" processor with the
query and the a database connection pooling service with the following
configuration:
connection url: jdbc:teradata://teradata.host/database=mydb
class name: com.teradata.jdbc.TeraDriver
jar url: file:///root/nifi-0.7.0/lib/terajdbc4.jar
...
I'm seeing this error in the log file:
org.apache.nifi.processor.exception.ProcessException:
org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of
class 'com.teradata.jdbc.TeraDriver' for connect URL '
jdbc:teradata://teradata.host/database=mydb'
at
org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:225)
~[na:na]
...
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC
driver of class 'com.teradata.jdbc.TeraDriver' for connect URL '
jdbc:teradata://teradata.host/database=mydb'
...
Caused by: java.sql.SQLException: No suitable driver
It looks like nifi can't find or load the driver, even though the jar is
located in /root/nifi-0.7.0/lib/terajdbc4.jar
Please, help me resolve this.
Thanks in advance.
Re: export from Teradata
Posted by Matt Burgess <ma...@gmail.com>.
+1 for additional classpath property.
> On Jul 20, 2016, at 8:41 AM, Oleg Zhurakousky <oz...@hortonworks.com> wrote:
>
> Just to add to what Mark said, perhaps you can raise a JIRA (as new Feature) to add an optional property to the processor config where such property would point to an additional class path (to support your case and many others).
> We are doing it with few processors already.
>
> Cheers
> Oleg
>
>> On Jul 20, 2016, at 8:37 AM, Mark Payne <ma...@hotmail.com> wrote:
>>
>> Hi Dima,
>>
>> That will work, however it is very dangerous to put jar files into NiFi's lib/ directory. We generally recommend
>> only putting NAR's into the lib/ directory, as any jar that is there will be inherited by all NARs and all classloaders.
>> So you may end up getting some really funny results from other NARs if any of the classes in those jars conflicts
>> with the classes in one of the NARs.
>>
>> The preferred approach is to create the uber jar, as described by Anuj, but NOT put it into the NiFi lib/ directory.
>> Instead, it should be placed outside of the lib/ directory and then referenced by the JAR URL property.
>>
>> Thanks
>> -Mark
>>
>>
>>> On Jul 20, 2016, at 4:10 AM, Dima Fadeyev <li...@gmail.com> wrote:
>>>
>>> Just in case someone has the same problem. Copying both teradata jars to nifi's lib directory and leaving the "jar url" blank solved the problem for us. I was able to start exporting data from Teradata.
>>>
>>> Best regards,
>>> Dima
>>>
>>>> On Thu, Jul 14, 2016 at 4:34 PM, Anuj Handa <an...@gmail.com> wrote:
>>>> Hi Dima,
>>>>
>>>> You will have to create an über jar from the JDBC drivers provided by Teradata and copy the uberjar into the lib folder of nifi.
>>>>
>>>> As Matt pointed out the instructions are in the email thread. You can refer them.
>>>>
>>>> Anuj
>>>>
>>>> > On Jul 14, 2016, at 9:35 AM, Matt Burgess <ma...@gmail.com> wrote:
>>>> >
>>>> > Dima,
>>>> >
>>>> > There was a discussion on how to get the SQL processors working with
>>>> > Teradata a little while ago:
>>>> >
>>>> > http://mail-archives.apache.org/mod_mbox/nifi-users/201605.mbox/%3CCAEXY4srXZkb2pMGiOFGs%3DrSc_mHCFx%2BvjW32RjPhz_K1pMr%2B%2Bg%40mail.gmail.com%3E
>>>> >
>>>> > Looks like it involves making a fat JAR to include the Teradata driver
>>>> > and all its dependencies, since the DBCPControllerService asks for a
>>>> > single JAR containing the driver class
>>>> >
>>>> > Regards,
>>>> > Matt
>>>> >
>>>> >> On Thu, Jul 14, 2016 at 9:26 AM, Dima Fadeyev <li...@gmail.com> wrote:
>>>> >> Hello, everyone,
>>>> >>
>>>> >> I'm new to nifi and this mailing list, I'm evaluating if we could extract
>>>> >> data (tables) from Teradata to local fs or HDFS with nifi. Is that possible?
>>>> >>
>>>> >> What I've done so far was creating an "Execute SQL" processor with the query
>>>> >> and the a database connection pooling service with the following
>>>> >> configuration:
>>>> >>
>>>> >> connection url: jdbc:teradata://teradata.host/database=mydb
>>>> >> class name: com.teradata.jdbc.TeraDriver
>>>> >> jar url: file:///root/nifi-0.7.0/lib/terajdbc4.jar
>>>> >> ...
>>>> >>
>>>> >> I'm seeing this error in the log file:
>>>> >>
>>>> >> org.apache.nifi.processor.exception.ProcessException:
>>>> >> org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of
>>>> >> class 'com.teradata.jdbc.TeraDriver' for connect URL '
>>>> >> jdbc:teradata://teradata.host/database=mydb'
>>>> >> at
>>>> >> org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:225)
>>>> >> ~[na:na]
>>>> >> ...
>>>> >> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC
>>>> >> driver of class 'com.teradata.jdbc.TeraDriver' for connect URL '
>>>> >> jdbc:teradata://teradata.host/database=mydb'
>>>> >> ...
>>>> >> Caused by: java.sql.SQLException: No suitable driver
>>>> >>
>>>> >> It looks like nifi can't find or load the driver, even though the jar is
>>>> >> located in /root/nifi-0.7.0/lib/terajdbc4.jar
>>>> >>
>>>> >> Please, help me resolve this.
>>>> >> Thanks in advance.
>
Re: export from Teradata
Posted by Oleg Zhurakousky <oz...@hortonworks.com>.
Just to add to what Mark said, perhaps you can raise a JIRA (as new Feature) to add an optional property to the processor config where such property would point to an additional class path (to support your case and many others).
We are doing it with few processors already.
Cheers
Oleg
On Jul 20, 2016, at 8:37 AM, Mark Payne <ma...@hotmail.com>> wrote:
Hi Dima,
That will work, however it is very dangerous to put jar files into NiFi's lib/ directory. We generally recommend
only putting NAR's into the lib/ directory, as any jar that is there will be inherited by all NARs and all classloaders.
So you may end up getting some really funny results from other NARs if any of the classes in those jars conflicts
with the classes in one of the NARs.
The preferred approach is to create the uber jar, as described by Anuj, but NOT put it into the NiFi lib/ directory.
Instead, it should be placed outside of the lib/ directory and then referenced by the JAR URL property.
Thanks
-Mark
On Jul 20, 2016, at 4:10 AM, Dima Fadeyev <li...@gmail.com>> wrote:
Just in case someone has the same problem. Copying both teradata jars to nifi's lib directory and leaving the "jar url" blank solved the problem for us. I was able to start exporting data from Teradata.
Best regards,
Dima
On Thu, Jul 14, 2016 at 4:34 PM, Anuj Handa <an...@gmail.com>> wrote:
Hi Dima,
You will have to create an über jar from the JDBC drivers provided by Teradata and copy the uberjar into the lib folder of nifi.
As Matt pointed out the instructions are in the email thread. You can refer them.
Anuj
> On Jul 14, 2016, at 9:35 AM, Matt Burgess <ma...@gmail.com>> wrote:
>
> Dima,
>
> There was a discussion on how to get the SQL processors working with
> Teradata a little while ago:
>
> http://mail-archives.apache.org/mod_mbox/nifi-users/201605.mbox/%3CCAEXY4srXZkb2pMGiOFGs%3DrSc_mHCFx%2BvjW32RjPhz_K1pMr%2B%2Bg%40mail.gmail.com%3E
>
> Looks like it involves making a fat JAR to include the Teradata driver
> and all its dependencies, since the DBCPControllerService asks for a
> single JAR containing the driver class
>
> Regards,
> Matt
>
>> On Thu, Jul 14, 2016 at 9:26 AM, Dima Fadeyev <li...@gmail.com>> wrote:
>> Hello, everyone,
>>
>> I'm new to nifi and this mailing list, I'm evaluating if we could extract
>> data (tables) from Teradata to local fs or HDFS with nifi. Is that possible?
>>
>> What I've done so far was creating an "Execute SQL" processor with the query
>> and the a database connection pooling service with the following
>> configuration:
>>
>> connection url: jdbc:teradata://teradata.host/database=mydb
>> class name: com.teradata.jdbc.TeraDriver
>> jar url: file:///root/nifi-0.7.0/lib/terajdbc4.jar
>> ...
>>
>> I'm seeing this error in the log file:
>>
>> org.apache.nifi.processor.exception.ProcessException:
>> org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of
>> class 'com.teradata.jdbc.TeraDriver' for connect URL '
>> jdbc:teradata://teradata.host/database=mydb'
>> at
>> org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:225)
>> ~[na:na]
>> ...
>> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC
>> driver of class 'com.teradata.jdbc.TeraDriver' for connect URL '
>> jdbc:teradata://teradata.host/database=mydb'
>> ...
>> Caused by: java.sql.SQLException: No suitable driver
>>
>> It looks like nifi can't find or load the driver, even though the jar is
>> located in /root/nifi-0.7.0/lib/terajdbc4.jar
>>
>> Please, help me resolve this.
>> Thanks in advance.
Re: export from Teradata
Posted by Mark Payne <ma...@hotmail.com>.
Hi Dima,
That will work, however it is very dangerous to put jar files into NiFi's lib/ directory. We generally recommend
only putting NAR's into the lib/ directory, as any jar that is there will be inherited by all NARs and all classloaders.
So you may end up getting some really funny results from other NARs if any of the classes in those jars conflicts
with the classes in one of the NARs.
The preferred approach is to create the uber jar, as described by Anuj, but NOT put it into the NiFi lib/ directory.
Instead, it should be placed outside of the lib/ directory and then referenced by the JAR URL property.
Thanks
-Mark
> On Jul 20, 2016, at 4:10 AM, Dima Fadeyev <li...@gmail.com> wrote:
>
> Just in case someone has the same problem. Copying both teradata jars to nifi's lib directory and leaving the "jar url" blank solved the problem for us. I was able to start exporting data from Teradata.
>
> Best regards,
> Dima
>
> On Thu, Jul 14, 2016 at 4:34 PM, Anuj Handa <anujhanda@gmail.com <ma...@gmail.com>> wrote:
> Hi Dima,
>
> You will have to create an über jar from the JDBC drivers provided by Teradata and copy the uberjar into the lib folder of nifi.
>
> As Matt pointed out the instructions are in the email thread. You can refer them.
>
> Anuj
>
> > On Jul 14, 2016, at 9:35 AM, Matt Burgess <mattyb149@gmail.com <ma...@gmail.com>> wrote:
> >
> > Dima,
> >
> > There was a discussion on how to get the SQL processors working with
> > Teradata a little while ago:
> >
> > http://mail-archives.apache.org/mod_mbox/nifi-users/201605.mbox/%3CCAEXY4srXZkb2pMGiOFGs%3DrSc_mHCFx%2BvjW32RjPhz_K1pMr%2B%2Bg%40mail.gmail.com%3E <http://mail-archives.apache.org/mod_mbox/nifi-users/201605.mbox/%3CCAEXY4srXZkb2pMGiOFGs%3DrSc_mHCFx%2BvjW32RjPhz_K1pMr%2B%2Bg%40mail.gmail.com%3E>
> >
> > Looks like it involves making a fat JAR to include the Teradata driver
> > and all its dependencies, since the DBCPControllerService asks for a
> > single JAR containing the driver class
> >
> > Regards,
> > Matt
> >
> >> On Thu, Jul 14, 2016 at 9:26 AM, Dima Fadeyev <linuxremote@gmail.com <ma...@gmail.com>> wrote:
> >> Hello, everyone,
> >>
> >> I'm new to nifi and this mailing list, I'm evaluating if we could extract
> >> data (tables) from Teradata to local fs or HDFS with nifi. Is that possible?
> >>
> >> What I've done so far was creating an "Execute SQL" processor with the query
> >> and the a database connection pooling service with the following
> >> configuration:
> >>
> >> connection url: jdbc:teradata://teradata.host/database=mydb
> >> class name: com.teradata.jdbc.TeraDriver
> >> jar url: file:///root/nifi-0.7.0/lib/terajdbc4.jar
> >> ...
> >>
> >> I'm seeing this error in the log file:
> >>
> >> org.apache.nifi.processor.exception.ProcessException:
> >> org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of
> >> class 'com.teradata.jdbc.TeraDriver' for connect URL '
> >> jdbc:teradata://teradata.host/database=mydb'
> >> at
> >> org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:225)
> >> ~[na:na]
> >> ...
> >> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC
> >> driver of class 'com.teradata.jdbc.TeraDriver' for connect URL '
> >> jdbc:teradata://teradata.host/database=mydb'
> >> ...
> >> Caused by: java.sql.SQLException: No suitable driver
> >>
> >> It looks like nifi can't find or load the driver, even though the jar is
> >> located in /root/nifi-0.7.0/lib/terajdbc4.jar
> >>
> >> Please, help me resolve this.
> >> Thanks in advance.
>
Re: export from Teradata
Posted by Dima Fadeyev <li...@gmail.com>.
Just in case someone has the same problem. Copying both teradata jars to
nifi's lib directory and leaving the "jar url" blank solved the problem for
us. I was able to start exporting data from Teradata.
Best regards,
Dima
On Thu, Jul 14, 2016 at 4:34 PM, Anuj Handa <an...@gmail.com> wrote:
> Hi Dima,
>
> You will have to create an über jar from the JDBC drivers provided by
> Teradata and copy the uberjar into the lib folder of nifi.
>
> As Matt pointed out the instructions are in the email thread. You can
> refer them.
>
> Anuj
>
> > On Jul 14, 2016, at 9:35 AM, Matt Burgess <ma...@gmail.com> wrote:
> >
> > Dima,
> >
> > There was a discussion on how to get the SQL processors working with
> > Teradata a little while ago:
> >
> >
> http://mail-archives.apache.org/mod_mbox/nifi-users/201605.mbox/%3CCAEXY4srXZkb2pMGiOFGs%3DrSc_mHCFx%2BvjW32RjPhz_K1pMr%2B%2Bg%40mail.gmail.com%3E
> >
> > Looks like it involves making a fat JAR to include the Teradata driver
> > and all its dependencies, since the DBCPControllerService asks for a
> > single JAR containing the driver class
> >
> > Regards,
> > Matt
> >
> >> On Thu, Jul 14, 2016 at 9:26 AM, Dima Fadeyev <li...@gmail.com>
> wrote:
> >> Hello, everyone,
> >>
> >> I'm new to nifi and this mailing list, I'm evaluating if we could
> extract
> >> data (tables) from Teradata to local fs or HDFS with nifi. Is that
> possible?
> >>
> >> What I've done so far was creating an "Execute SQL" processor with the
> query
> >> and the a database connection pooling service with the following
> >> configuration:
> >>
> >> connection url: jdbc:teradata://teradata.host/database=mydb
> >> class name: com.teradata.jdbc.TeraDriver
> >> jar url: file:///root/nifi-0.7.0/lib/terajdbc4.jar
> >> ...
> >>
> >> I'm seeing this error in the log file:
> >>
> >> org.apache.nifi.processor.exception.ProcessException:
> >> org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of
> >> class 'com.teradata.jdbc.TeraDriver' for connect URL '
> >> jdbc:teradata://teradata.host/database=mydb'
> >> at
> >>
> org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:225)
> >> ~[na:na]
> >> ...
> >> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create
> JDBC
> >> driver of class 'com.teradata.jdbc.TeraDriver' for connect URL '
> >> jdbc:teradata://teradata.host/database=mydb'
> >> ...
> >> Caused by: java.sql.SQLException: No suitable driver
> >>
> >> It looks like nifi can't find or load the driver, even though the jar is
> >> located in /root/nifi-0.7.0/lib/terajdbc4.jar
> >>
> >> Please, help me resolve this.
> >> Thanks in advance.
>
Re: export from Teradata
Posted by Anuj Handa <an...@gmail.com>.
Hi Dima,
You will have to create an über jar from the JDBC drivers provided by Teradata and copy the uberjar into the lib folder of nifi.
As Matt pointed out the instructions are in the email thread. You can refer them.
Anuj
> On Jul 14, 2016, at 9:35 AM, Matt Burgess <ma...@gmail.com> wrote:
>
> Dima,
>
> There was a discussion on how to get the SQL processors working with
> Teradata a little while ago:
>
> http://mail-archives.apache.org/mod_mbox/nifi-users/201605.mbox/%3CCAEXY4srXZkb2pMGiOFGs%3DrSc_mHCFx%2BvjW32RjPhz_K1pMr%2B%2Bg%40mail.gmail.com%3E
>
> Looks like it involves making a fat JAR to include the Teradata driver
> and all its dependencies, since the DBCPControllerService asks for a
> single JAR containing the driver class
>
> Regards,
> Matt
>
>> On Thu, Jul 14, 2016 at 9:26 AM, Dima Fadeyev <li...@gmail.com> wrote:
>> Hello, everyone,
>>
>> I'm new to nifi and this mailing list, I'm evaluating if we could extract
>> data (tables) from Teradata to local fs or HDFS with nifi. Is that possible?
>>
>> What I've done so far was creating an "Execute SQL" processor with the query
>> and the a database connection pooling service with the following
>> configuration:
>>
>> connection url: jdbc:teradata://teradata.host/database=mydb
>> class name: com.teradata.jdbc.TeraDriver
>> jar url: file:///root/nifi-0.7.0/lib/terajdbc4.jar
>> ...
>>
>> I'm seeing this error in the log file:
>>
>> org.apache.nifi.processor.exception.ProcessException:
>> org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of
>> class 'com.teradata.jdbc.TeraDriver' for connect URL '
>> jdbc:teradata://teradata.host/database=mydb'
>> at
>> org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:225)
>> ~[na:na]
>> ...
>> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC
>> driver of class 'com.teradata.jdbc.TeraDriver' for connect URL '
>> jdbc:teradata://teradata.host/database=mydb'
>> ...
>> Caused by: java.sql.SQLException: No suitable driver
>>
>> It looks like nifi can't find or load the driver, even though the jar is
>> located in /root/nifi-0.7.0/lib/terajdbc4.jar
>>
>> Please, help me resolve this.
>> Thanks in advance.
Re: export from Teradata
Posted by Matt Burgess <ma...@gmail.com>.
Dima,
There was a discussion on how to get the SQL processors working with
Teradata a little while ago:
http://mail-archives.apache.org/mod_mbox/nifi-users/201605.mbox/%3CCAEXY4srXZkb2pMGiOFGs%3DrSc_mHCFx%2BvjW32RjPhz_K1pMr%2B%2Bg%40mail.gmail.com%3E
Looks like it involves making a fat JAR to include the Teradata driver
and all its dependencies, since the DBCPControllerService asks for a
single JAR containing the driver class
Regards,
Matt
On Thu, Jul 14, 2016 at 9:26 AM, Dima Fadeyev <li...@gmail.com> wrote:
> Hello, everyone,
>
> I'm new to nifi and this mailing list, I'm evaluating if we could extract
> data (tables) from Teradata to local fs or HDFS with nifi. Is that possible?
>
> What I've done so far was creating an "Execute SQL" processor with the query
> and the a database connection pooling service with the following
> configuration:
>
> connection url: jdbc:teradata://teradata.host/database=mydb
> class name: com.teradata.jdbc.TeraDriver
> jar url: file:///root/nifi-0.7.0/lib/terajdbc4.jar
> ...
>
> I'm seeing this error in the log file:
>
> org.apache.nifi.processor.exception.ProcessException:
> org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC driver of
> class 'com.teradata.jdbc.TeraDriver' for connect URL '
> jdbc:teradata://teradata.host/database=mydb'
> at
> org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:225)
> ~[na:na]
> ...
> Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create JDBC
> driver of class 'com.teradata.jdbc.TeraDriver' for connect URL '
> jdbc:teradata://teradata.host/database=mydb'
> ...
> Caused by: java.sql.SQLException: No suitable driver
>
> It looks like nifi can't find or load the driver, even though the jar is
> located in /root/nifi-0.7.0/lib/terajdbc4.jar
>
> Please, help me resolve this.
> Thanks in advance.