[jira] [Created] (NIFIREG-240) Add support for MariaDB database as configuration backend

["Michal Klempa (JIRA)" <ji...@apache.org>]

Michal Klempa created NIFIREG-240:
-------------------------------------

             Summary: Add support for MariaDB database as configuration backend
                 Key: NIFIREG-240
                 URL: https://issues.apache.org/jira/browse/NIFIREG-240
             Project: NiFi Registry
          Issue Type: Improvement
    Affects Versions: 0.3.0
            Reporter: Michal Klempa


I have tried to use MariaDB 10.3 with NiFi Registry, with no luck. Although there should be a way to enable MariaDB as a backend service instead of H2.

 

Environment:

NiFi Registry 0.3.0 (inside Docker, but this does not really affect the output)

MariaDB Java Client 2.3.0 - mariadb-java-client-2.3.0.jar (not 2.4.1 due to [https://github.com/flyway/flyway/issues/2289#issuecomment-459481080)]

Configuration:

nifi-registry.properties:
{code:java}
nifi.registry.db.url=jdbc:mariadb://database:3306/db
nifi.registry.db.driver.class=org.mariadb.jdbc.Driver
nifi.registry.db.driver.directory=/opt/nifi-registry/libs/
nifi.registry.db.username=root
nifi.registry.db.password=myPassword{code}
The {{mariadb-java-client-2.3.0.jar}} was placed to
{code:java}
/opt/nifi-registry/libs/{code}
 

Steps to reproduce:
 # unpack nifi registry to {{/opt/nifi-registry/nifi-registry-0.3.0}}
 # set the {{nifi-registry.properties}} as above
 # add {{mariadb-java-client-2.3.0.jar}} to {{/opt/nifi-registry/libs/}}
 # start

Or alternatively, use the Docker image I was developping and during which I found this issue.

The docker-compose file is [https://github.com/michalklempa/docker-nifi-registry/blob/master/docker-compose.mariadb.yml|https://github.com/michalklempa/docker-nifi-registry/blob/mariadb/docker-compose.mariadb.yml]

 

Expected behavior: working Registry

Actual behavior:
{code:java}
nifi-registry_1  | 2019-03-24 17:55:39,771 WARN [main] o.s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'accessPolicyResource' defined in file [/opt/nifi-registry/nifi-registry-0.3.0/work/jetty/nifi-registry-web-api-0.3.0.war/webapp/WEB-INF/classes/org/apache/nifi/registry/web/api/AccessPolicyResource.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'authorizationService' defined in URL [jar:file:/opt/nifi-registry/nifi-registry-0.3.0/work/jetty/nifi-registry-web-api-0.3.0.war/webapp/WEB-INF/lib/nifi-registry-framework-0.3.0.jar!/org/apache/nifi/registry/service/AuthorizationService.class]: Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'registryService' defined in URL [jar:file:/opt/nifi-registry/nifi-registry-0.3.0/work/jetty/nifi-registry-web-api-0.3.0.war/webapp/WEB-INF/lib/nifi-registry-framework-0.3.0.jar!/org/apache/nifi/registry/service/RegistryService.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'databaseMetadataService' defined in URL [jar:file:/opt/nifi-registry/nifi-registry-0.3.0/work/jetty/nifi-registry-web-api-0.3.0.war/webapp/WEB-INF/lib/nifi-registry-framework-0.3.0.jar!/org/apache/nifi/registry/db/DatabaseMetadataService.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateSqlException:
nifi-registry_1  | Migration V2__Initial.sql failed
nifi-registry_1  | --------------------------------
nifi-registry_1  | SQL State  : 42000
nifi-registry_1  | Error Code : 1071
nifi-registry_1  | Message    : (conn=8) Specified key was too long; max key length is 3072 bytes
nifi-registry_1  | Location   : db/migration/V2__Initial.sql (/opt/nifi-registry/nifi-registry-0.3.0/file:/opt/nifi-registry/nifi-registry-0.3.0/work/jetty/nifi-registry-web-api-0.3.0.war/webapp/WEB-INF/lib/nifi-registry-framework-0.3.0.jar!/db/migration/V2__Initial.sql)
nifi-registry_1  | Line       : 54
nifi-registry_1  | Statement  : CREATE TABLE SIGNING_KEY (
nifi-registry_1  |     ID VARCHAR(50) NOT NULL,
nifi-registry_1  |     TENANT_IDENTITY VARCHAR(4096) NOT NULL,
nifi-registry_1  |     KEY_VALUE VARCHAR(50) NOT NULL,
nifi-registry_1  |     CONSTRAINT PK__SIGNING_KEY_ID PRIMARY KEY (ID),
nifi-registry_1  |     CONSTRAINT UNIQUE__SIGNING_KEY_TENANT_IDENTITY UNIQUE (TENANT_IDENTITY)
nifi-registry_1  | )
nifi-registry_1  |
nifi-registry_1  | 2019-03-24 17:55:39,772 INFO [main] o.a.nifi.registry.event.EventService Shutting down event consumer...{code}
 

There are various SQL dialect differences.

I was able to shorten TENANT_IDENTITY varchar to 1096:
{code:java}
SIGNING_KEY.TENANT_IDENTITY VARCHAR(1096){code}
Then there was mariadb complaining about lenght of constraint name UNIQUE__BUNDLE_VERSION_DEPENDENCY_BUNDLE_ID_GROUP_ARTIFACT_VERSION

and finally the DROP CONSTRAINT for FOREIGN KEY syntax is different.

Reverse diff of my changes can be seen here:

[https://github.com/michalklempa/docker-nifi-registry/commit/2a78f5f6df0b00fa3011da9ee49c65a412012fcc]

 

I am unsure is such changes are compatible or not with the Registry codebase. Further work is required. And since I was creating the schema manually, NiFi Registry did not start up either - since it found existing schema without some required metadata.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)