Creating a custom DatabaseAdapter

[Stanislav <eg...@yandex.com>]

Hi,

I would like to create a custom org.apache.nifi.processors.standard.db.DatabaseAdapter, is this possible?
I have tried creating new class that implements the interface and specified the full class name in META-INF\services\org.apache.nifi.processors.standard.db.DatabaseAdapter file, and created a nar.
But this does not appear to be working, the custom processor i have in that nar are picked-up but the DatabaseAdapter isn`t.
Any tips on how to make it work?

Best regards,
Stanislav.

Re: Creating a custom DatabaseAdapter

[Oleg Zhurakousky <oz...@hortonworks.com>]

Stanislav

Sorry to hear you’re having problems. 
Is ether any way we can look at the code (i.e., via GitHub)? Also, when you say you don’t see your DatabaseAdapter is not picked up, do you see any messages in the logs?

It is hard to determine the issue without more info, but usually symptoms like this point to miss-configuration of some type and are usually very easy to fix, once diagnosed. 
Let us know what you can do.

Also, a lot of times i helps to just copy another working bundle, then rename the artifact(s), build and make sure you can see it being deployed. Now you can modify the code any way you like knowing that at the very least you don’t have any configuration issues.

Let us know
Cheers
Oleg
> On Feb 17, 2017, at 5:05 AM, Stanislav <eg...@yandex.com> wrote:
> 
> Hi,
> 
> I would like to create a custom org.apache.nifi.processors.standard.db.DatabaseAdapter, is this possible?
> I have tried creating new class that implements the interface and specified the full class name in META-INF\services\org.apache.nifi.processors.standard.db.DatabaseAdapter file, and created a nar.
> But this does not appear to be working, the custom processor i have in that nar are picked-up but the DatabaseAdapter isn`t.
> Any tips on how to make it work?
> 
> Best regards,
> Stanislav.
> 

RE: Creating a custom DatabaseAdapter

["Peter Wicks (pwicks)" <pw...@micron.com>]

Matt/Stanislav,

As part of NIFI-3093 (HIVE support for QueryDatabaseTable) I had planned to move DatabaseAdapter to the DBCP Service Controller project; whether or not that makes sense I have no idea, and I haven't done it at this time. But I wanted to be able to rehome the new HIVE DatabaseAdapter to the HIVE NAR and this looked like a candidate project.

Stanislav, it sounds like you are trying to avoid customizing core code (very understandable), but if you are interested I have an open PR that shows everything I had to do to add a new DatabaseAdapter for MS SQL Server as part of the existing structure.: https://github.com/apache/nifi/pull/1510. 

Thanks,
 Peter

-----Original Message-----
From: Matt Burgess [mailto:mattyb149@apache.org] 
Sent: Friday, February 17, 2017 7:53 PM
To: dev@nifi.apache.org
Subject: Re: Creating a custom DatabaseAdapter

Stanislav,

This is not your fault, it is a current limitation due to the organization of the NiFi submodules with respect to the location of DatabaseAdapter.  In general, when a NiFi interface is to be extensible, there is a submodule for the API JAR to be available to other NARs/submodules/etc.  In this case, we'd likely want DatabaseAdapter (and any related classes) to be included in nifi-standard-services-api-nar as its own API NAR, yet at the moment it is a class inside nifi-standard-nar.

I agree that we should make this interface available, and refactor the submodules accordingly to make DatabaseAdapter extensible outside the nifi-standard NAR. Some care should be given to which "parent NAR" it belongs, in order to ensure that we don't upset the current mechanism of one parent NAR per child NAR, but hopefully "nifi-services-api-nar"
is the right spot.  Do you mind writing up a Jira case for this?  I think it will help the community in general as various DB integrations are desired.

Thanks in advance,
Matt

On Fri, Feb 17, 2017 at 5:05 AM, Stanislav <eg...@yandex.com> wrote:
> Hi,
>
> I would like to create a custom org.apache.nifi.processors.standard.db.DatabaseAdapter, is this possible?
> I have tried creating new class that implements the interface and specified the full class name in META-INF\services\org.apache.nifi.processors.standard.db.DatabaseAdapter file, and created a nar.
> But this does not appear to be working, the custom processor i have in that nar are picked-up but the DatabaseAdapter isn`t.
> Any tips on how to make it work?
>
> Best regards,
> Stanislav.

Re: Creating a custom DatabaseAdapter

[Matt Burgess <ma...@apache.org>]

Stanislav,

This is not your fault, it is a current limitation due to the
organization of the NiFi submodules with respect to the location of
DatabaseAdapter.  In general, when a NiFi interface is to be
extensible, there is a submodule for the API JAR to be available to
other NARs/submodules/etc.  In this case, we'd likely want
DatabaseAdapter (and any related classes) to be included in
nifi-standard-services-api-nar as its own API NAR, yet at the moment
it is a class inside nifi-standard-nar.

I agree that we should make this interface available, and refactor the
submodules accordingly to make DatabaseAdapter extensible outside the
nifi-standard NAR. Some care should be given to which "parent NAR" it
belongs, in order to ensure that we don't upset the current mechanism
of one parent NAR per child NAR, but hopefully "nifi-services-api-nar"
is the right spot.  Do you mind writing up a Jira case for this?  I
think it will help the community in general as various DB integrations
are desired.

Thanks in advance,
Matt

On Fri, Feb 17, 2017 at 5:05 AM, Stanislav <eg...@yandex.com> wrote:
> Hi,
>
> I would like to create a custom org.apache.nifi.processors.standard.db.DatabaseAdapter, is this possible?
> I have tried creating new class that implements the interface and specified the full class name in META-INF\services\org.apache.nifi.processors.standard.db.DatabaseAdapter file, and created a nar.
> But this does not appear to be working, the custom processor i have in that nar are picked-up but the DatabaseAdapter isn`t.
> Any tips on how to make it work?
>
> Best regards,
> Stanislav.