You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Tom Gullo <to...@gmail.com> on 2016/08/18 20:33:12 UTC
adding dependencies like jdbc drivers to the build
If I want to add a jdbc driver or any third party dependency where should I
add that dependency in the Maven build for Nifi?
Thanks
-Tom
Re: adding dependencies like jdbc drivers to the build
Posted by Matt Burgess <ma...@gmail.com>.
All,
I took a shot at adding the ability to specify multiple URLs, files,
and folders to the DBCPConnectionPool configuration (NIFI-2604). The
branch is here if you'd like to build and try:
https://github.com/mattyb149/nifi/tree/NIFI-2604
The property name, description, etc. has changed, which will
invalidate your current DBCPConnectionPool instances. However I think
this is a necessary change for a better user experience going forward,
and you simply copy the old property value to the new property value
and all should return to normal. I haven't issued a PR yet as it
depends on a previous PR to be merged.
If you give this a try, please let me know how/if it works for you.
Thanks,
Matt
On Fri, Aug 19, 2016 at 2:00 PM, Sumanth Chinthagunta <xm...@gmail.com> wrote:
> Sorry I mean ./driver_jars
> I had similar relative directories ./module_jars etc ., which works fine for
> ExecuteScript processor
>
> Sent from my iPhone
>
> On Aug 19, 2016, at 4:27 AM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
>
> Sumanth,
>
>
>
> If the driver is in your lib directory already then you should leave the
> path empty. All jar’s in your lib directory are loaded in the classpath for
> all NAR’s.
>
>
>
> Personally I have three different JDBC drivers in my lib directory to make
> them available for whoever needs them (MS SQL, SAP Hana, Teradata, and will
> add Oracle soon).
>
>
>
> --Peter
>
>
>
> From: Sumanth Chinthagunta [mailto:xmlking@gmail.com]
> Sent: Thursday, August 18, 2016 8:56 PM
> To: users@nifi.apache.org
> Subject: Re: adding dependencies like jdbc drivers to the build
>
>
>
> It would be nice if we support relative paths for driver jar. E.g.,
> ./lib/mariadb-java-client-1.1.7.jar
>
> This let flow templet portable (dev -> prod)
>
>
>
>
>
> Sent from my iPhone
>
>
> On Aug 18, 2016, at 2:25 PM, Bryan Bende <bb...@gmail.com> wrote:
>
> For JDBC, if you are talking about the DBConnectionPool, you should be able
> to reference a driver as an external file such as
> file:///var/tmp/mariadb-java-client-1.1.7.jar'
>
>
>
> If you are talking about something different besides the DBConnectionPool
> then it depends what processor/component...
>
> If you look in the lib directory you will see all the NAR files, each NAR
> has one or more components along with all of the other JARs it needs, and
> each NAR has isolated class loading so that they will not interfere with
> each other.
>
>
>
> You would need to figure out which NAR you are dealing with and add a
> dependency to one of the poms related to that NAR.
>
>
>
> -Bryan
>
>
>
>
>
> On Thu, Aug 18, 2016 at 4:33 PM, Tom Gullo <to...@gmail.com> wrote:
>
> If I want to add a jdbc driver or any third party dependency where should I
> add that dependency in the Maven build for Nifi?
>
>
>
> Thanks
>
> -Tom
>
>
Re: adding dependencies like jdbc drivers to the build
Posted by Sumanth Chinthagunta <xm...@gmail.com>.
Sorry I mean ./driver_jars
I had similar relative directories ./module_jars etc ., which works fine for ExecuteScript processor
Sent from my iPhone
> On Aug 19, 2016, at 4:27 AM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
>
> Sumanth,
>
> If the driver is in your lib directory already then you should leave the path empty. All jar’s in your lib directory are loaded in the classpath for all NAR’s.
>
> Personally I have three different JDBC drivers in my lib directory to make them available for whoever needs them (MS SQL, SAP Hana, Teradata, and will add Oracle soon).
>
> --Peter
>
> From: Sumanth Chinthagunta [mailto:xmlking@gmail.com]
> Sent: Thursday, August 18, 2016 8:56 PM
> To: users@nifi.apache.org
> Subject: Re: adding dependencies like jdbc drivers to the build
>
> It would be nice if we support relative paths for driver jar. E.g., ./lib/mariadb-java-client-1.1.7.jar
> This let flow templet portable (dev -> prod)
>
>
>
>
> Sent from my iPhone
>
> On Aug 18, 2016, at 2:25 PM, Bryan Bende <bb...@gmail.com> wrote:
>
> For JDBC, if you are talking about the DBConnectionPool, you should be able to reference a driver as an external file such as file:///var/tmp/mariadb-java-client-1.1.7.jar'
>
> If you are talking about something different besides the DBConnectionPool then it depends what processor/component...
> If you look in the lib directory you will see all the NAR files, each NAR has one or more components along with all of the other JARs it needs, and each NAR has isolated class loading so that they will not interfere with each other.
>
> You would need to figure out which NAR you are dealing with and add a dependency to one of the poms related to that NAR.
>
> -Bryan
>
>
> On Thu, Aug 18, 2016 at 4:33 PM, Tom Gullo <to...@gmail.com> wrote:
> If I want to add a jdbc driver or any third party dependency where should I add that dependency in the Maven build for Nifi?
>
> Thanks
> -Tom
>
Re: adding dependencies like jdbc drivers to the build
Posted by Joe Witt <jo...@gmail.com>.
Captured the elements of this thread
https://issues.apache.org/jira/browse/NIFI-2604 Please alter/update
as needed.
On Fri, Aug 19, 2016 at 5:51 AM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
> While you are at it… can you make it so it supports more than one file in
> that field, probably comma delimited? I have one JDBC driver that for
> whatever reason requires two separate JAR’s.
>
>
>
> From: Joe Witt [mailto:joe.witt@gmail.com]
> Sent: Friday, August 19, 2016 5:30 AM
> To: users@nifi.apache.org
> Subject: RE: adding dependencies like jdbc drivers to the build
>
>
>
> Adding jars to the lib directory is not ideal as it pollutes all
> classloaders. We should add expression language support to the path
> property if it isn't already as that makes variable registry access
> available which makes use of the same template or flow in different
> environments easier.
>
>
>
> On Aug 19, 2016 7:27 AM, "Peter Wicks (pwicks)" <pw...@micron.com> wrote:
>
> Sumanth,
>
>
>
> If the driver is in your lib directory already then you should leave the
> path empty. All jar’s in your lib directory are loaded in the classpath for
> all NAR’s.
>
>
>
> Personally I have three different JDBC drivers in my lib directory to make
> them available for whoever needs them (MS SQL, SAP Hana, Teradata, and will
> add Oracle soon).
>
>
>
> --Peter
>
>
>
> From: Sumanth Chinthagunta [mailto:xmlking@gmail.com]
> Sent: Thursday, August 18, 2016 8:56 PM
> To: users@nifi.apache.org
> Subject: Re: adding dependencies like jdbc drivers to the build
>
>
>
> It would be nice if we support relative paths for driver jar. E.g.,
> ./lib/mariadb-java-client-1.1.7.jar
>
> This let flow templet portable (dev -> prod)
>
>
>
>
>
> Sent from my iPhone
>
>
> On Aug 18, 2016, at 2:25 PM, Bryan Bende <bb...@gmail.com> wrote:
>
> For JDBC, if you are talking about the DBConnectionPool, you should be able
> to reference a driver as an external file such as
> file:///var/tmp/mariadb-java-client-1.1.7.jar'
>
>
>
> If you are talking about something different besides the DBConnectionPool
> then it depends what processor/component...
>
> If you look in the lib directory you will see all the NAR files, each NAR
> has one or more components along with all of the other JARs it needs, and
> each NAR has isolated class loading so that they will not interfere with
> each other.
>
>
>
> You would need to figure out which NAR you are dealing with and add a
> dependency to one of the poms related to that NAR.
>
>
>
> -Bryan
>
>
>
>
>
> On Thu, Aug 18, 2016 at 4:33 PM, Tom Gullo <to...@gmail.com> wrote:
>
> If I want to add a jdbc driver or any third party dependency where should I
> add that dependency in the Maven build for Nifi?
>
>
>
> Thanks
>
> -Tom
>
>
>
>
RE: adding dependencies like jdbc drivers to the build
Posted by "Peter Wicks (pwicks)" <pw...@micron.com>.
While you are at it… can you make it so it supports more than one file in that field, probably comma delimited? I have one JDBC driver that for whatever reason requires two separate JAR’s.
From: Joe Witt [mailto:joe.witt@gmail.com]
Sent: Friday, August 19, 2016 5:30 AM
To: users@nifi.apache.org
Subject: RE: adding dependencies like jdbc drivers to the build
Adding jars to the lib directory is not ideal as it pollutes all classloaders. We should add expression language support to the path property if it isn't already as that makes variable registry access available which makes use of the same template or flow in different environments easier.
On Aug 19, 2016 7:27 AM, "Peter Wicks (pwicks)" <pw...@micron.com>> wrote:
Sumanth,
If the driver is in your lib directory already then you should leave the path empty. All jar’s in your lib directory are loaded in the classpath for all NAR’s.
Personally I have three different JDBC drivers in my lib directory to make them available for whoever needs them (MS SQL, SAP Hana, Teradata, and will add Oracle soon).
--Peter
From: Sumanth Chinthagunta [mailto:xmlking@gmail.com<ma...@gmail.com>]
Sent: Thursday, August 18, 2016 8:56 PM
To: users@nifi.apache.org<ma...@nifi.apache.org>
Subject: Re: adding dependencies like jdbc drivers to the build
It would be nice if we support relative paths for driver jar. E.g., ./lib/mariadb-java-client-1.1.7.jar
This let flow templet portable (dev -> prod)
Sent from my iPhone
On Aug 18, 2016, at 2:25 PM, Bryan Bende <bb...@gmail.com>> wrote:
For JDBC, if you are talking about the DBConnectionPool, you should be able to reference a driver as an external file such as file:///var/tmp/mariadb-java-client-1.1.7.jar<file:///\\var\tmp\mariadb-java-client-1.1.7.jar>'
If you are talking about something different besides the DBConnectionPool then it depends what processor/component...
If you look in the lib directory you will see all the NAR files, each NAR has one or more components along with all of the other JARs it needs, and each NAR has isolated class loading so that they will not interfere with each other.
You would need to figure out which NAR you are dealing with and add a dependency to one of the poms related to that NAR.
-Bryan
On Thu, Aug 18, 2016 at 4:33 PM, Tom Gullo <to...@gmail.com>> wrote:
If I want to add a jdbc driver or any third party dependency where should I add that dependency in the Maven build for Nifi?
Thanks
-Tom
RE: adding dependencies like jdbc drivers to the build
Posted by Joe Witt <jo...@gmail.com>.
Adding jars to the lib directory is not ideal as it pollutes all
classloaders. We should add expression language support to the path
property if it isn't already as that makes variable registry access
available which makes use of the same template or flow in different
environments easier.
On Aug 19, 2016 7:27 AM, "Peter Wicks (pwicks)" <pw...@micron.com> wrote:
Sumanth,
If the driver is in your lib directory already then you should leave the
path empty. All jar’s in your lib directory are loaded in the classpath
for all NAR’s.
Personally I have three different JDBC drivers in my lib directory to make
them available for whoever needs them (MS SQL, SAP Hana, Teradata, and will
add Oracle soon).
--Peter
*From:* Sumanth Chinthagunta [mailto:xmlking@gmail.com]
*Sent:* Thursday, August 18, 2016 8:56 PM
*To:* users@nifi.apache.org
*Subject:* Re: adding dependencies like jdbc drivers to the build
It would be nice if we support relative paths for driver jar. E.g.,
./lib/mariadb-java-client-1.1.7.jar
This let flow templet portable (dev -> prod)
Sent from my iPhone
On Aug 18, 2016, at 2:25 PM, Bryan Bende <bb...@gmail.com> wrote:
For JDBC, if you are talking about the DBConnectionPool, you should be able
to reference a driver as an external file such as file:///var/tmp/mariadb-
java-client-1.1.7.jar'
If you are talking about something different besides the DBConnectionPool
then it depends what processor/component...
If you look in the lib directory you will see all the NAR files, each NAR
has one or more components along with all of the other JARs it needs, and
each NAR has isolated class loading so that they will not interfere with
each other.
You would need to figure out which NAR you are dealing with and add a
dependency to one of the poms related to that NAR.
-Bryan
On Thu, Aug 18, 2016 at 4:33 PM, Tom Gullo <to...@gmail.com> wrote:
If I want to add a jdbc driver or any third party dependency where should I
add that dependency in the Maven build for Nifi?
Thanks
-Tom
RE: adding dependencies like jdbc drivers to the build
Posted by "Peter Wicks (pwicks)" <pw...@micron.com>.
Sumanth,
If the driver is in your lib directory already then you should leave the path empty. All jar’s in your lib directory are loaded in the classpath for all NAR’s.
Personally I have three different JDBC drivers in my lib directory to make them available for whoever needs them (MS SQL, SAP Hana, Teradata, and will add Oracle soon).
--Peter
From: Sumanth Chinthagunta [mailto:xmlking@gmail.com]
Sent: Thursday, August 18, 2016 8:56 PM
To: users@nifi.apache.org
Subject: Re: adding dependencies like jdbc drivers to the build
It would be nice if we support relative paths for driver jar. E.g., ./lib/mariadb-java-client-1.1.7.jar
This let flow templet portable (dev -> prod)
Sent from my iPhone
On Aug 18, 2016, at 2:25 PM, Bryan Bende <bb...@gmail.com>> wrote:
For JDBC, if you are talking about the DBConnectionPool, you should be able to reference a driver as an external file such as file:///var/tmp/mariadb-java-client-1.1.7.jar<file:///\\var\tmp\mariadb-java-client-1.1.7.jar>'
If you are talking about something different besides the DBConnectionPool then it depends what processor/component...
If you look in the lib directory you will see all the NAR files, each NAR has one or more components along with all of the other JARs it needs, and each NAR has isolated class loading so that they will not interfere with each other.
You would need to figure out which NAR you are dealing with and add a dependency to one of the poms related to that NAR.
-Bryan
On Thu, Aug 18, 2016 at 4:33 PM, Tom Gullo <to...@gmail.com>> wrote:
If I want to add a jdbc driver or any third party dependency where should I add that dependency in the Maven build for Nifi?
Thanks
-Tom
Re: adding dependencies like jdbc drivers to the build
Posted by Sumanth Chinthagunta <xm...@gmail.com>.
It would be nice if we support relative paths for driver jar. E.g., ./lib/mariadb-java-client-1.1.7.jar
This let flow templet portable (dev -> prod)
Sent from my iPhone
> On Aug 18, 2016, at 2:25 PM, Bryan Bende <bb...@gmail.com> wrote:
>
> For JDBC, if you are talking about the DBConnectionPool, you should be able to reference a driver as an external file such as file:///var/tmp/mariadb-java-client-1.1.7.jar'
>
> If you are talking about something different besides the DBConnectionPool then it depends what processor/component...
> If you look in the lib directory you will see all the NAR files, each NAR has one or more components along with all of the other JARs it needs, and each NAR has isolated class loading so that they will not interfere with each other.
>
> You would need to figure out which NAR you are dealing with and add a dependency to one of the poms related to that NAR.
>
> -Bryan
>
>
>> On Thu, Aug 18, 2016 at 4:33 PM, Tom Gullo <to...@gmail.com> wrote:
>> If I want to add a jdbc driver or any third party dependency where should I add that dependency in the Maven build for Nifi?
>>
>> Thanks
>> -Tom
>
Re: adding dependencies like jdbc drivers to the build
Posted by Bryan Bende <bb...@gmail.com>.
For JDBC, if you are talking about the DBConnectionPool, you should be able
to reference a driver as an external file such
as file:///var/tmp/mariadb-java-client-1.1.7.jar'
If you are talking about something different besides the DBConnectionPool
then it depends what processor/component...
If you look in the lib directory you will see all the NAR files, each NAR
has one or more components along with all of the other JARs it needs, and
each NAR has isolated class loading so that they will not interfere with
each other.
You would need to figure out which NAR you are dealing with and add a
dependency to one of the poms related to that NAR.
-Bryan
On Thu, Aug 18, 2016 at 4:33 PM, Tom Gullo <to...@gmail.com> wrote:
> If I want to add a jdbc driver or any third party dependency where should
> I add that dependency in the Maven build for Nifi?
>
> Thanks
> -Tom
>