You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2022/01/27 23:39:26 UTC
[GitHub] [nifi] mattyb149 opened a new pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
mattyb149 opened a new pull request #5724:
URL: https://github.com/apache/nifi/pull/5724
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
#### Description of PR
Upgrades NiFi projects to 2.x
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [x] Have you verified that the full build is successful on JDK 11?
- [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on a change in pull request #5724:
URL: https://github.com/apache/nifi/pull/5724#discussion_r813482003
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.h2.jdbc.JdbcSQLNonTransientException;
+import org.h2.jdbcx.JdbcDataSource;
+import org.h2.mvstore.MVStoreException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseUpdater {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseUpdater.class);
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String H2_URL_PREFIX = "jdbc:h2:";
+
+ private static final JdbcDataSource migrationDataSource = new JdbcDataSource();
+
+ public static void checkAndPerformMigration(final File dbFile, final String dbUrl, final String user, final String pass) throws Exception {
Review comment:
Does the `dbUrl` string contain the file path? In that case, it should be possible to parse the file path from the dbUrl string. Alternatively, passing `dbUrl` with a type of `java.net.URI` seems like it should support the JDBC URL syntax.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] bbende commented on a change in pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
bbende commented on a change in pull request #5724:
URL: https://github.com/apache/nifi/pull/5724#discussion_r798045638
##########
File path: minifi/pom.xml
##########
@@ -833,7 +833,7 @@ limitations under the License.
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
- <version>1.4.199</version>
+ <version>2.1.210</version>
Review comment:
Seems like we have a few places that directly use the version rather than referencing `${h2.version}`, I'm guessing the idea being that they may evolve independently, but would that ever really happen in practice? Seems like we'd always probably upgrade all usage at the same time since all in the same repo now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory closed pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
exceptionfactory closed pull request #5724:
URL: https://github.com/apache/nifi/pull/5724
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on a change in pull request #5724:
URL: https://github.com/apache/nifi/pull/5724#discussion_r811948491
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseMigrator.java
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.apache.nifi.org.h2.jdbcx.JdbcDataSource;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_POSTFIX;
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_PREFIX;
+
+public class H2DatabaseMigrator {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseMigrator.class);
+
+ public static final String BACKUP_FILE_POSTFIX = ".migration_backup";
+
+ public static void exportAndBackup(final String dbUrl, final String dbPath, final String user, final String pass) {
+
+ // Attempt to connect with the latest driver
+ JdbcDataSource migrationDataSource = new JdbcDataSource();
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+
+ final File dbPathFile = Paths.get(dbPath).toFile();
+
+ String exportFile = Paths.get(dbPathFile.getParent(), EXPORT_FILE_PREFIX + dbPathFile.getName() + EXPORT_FILE_POSTFIX).toString();
+
+ Connection conn = null;
+ Statement s = null;
+ try {
+ conn = migrationDataSource.getConnection();
+ s = conn.createStatement();
+ } catch (SQLException sqle) {
+ logger.error("Could not connect to the database with the legacy driver, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
+ }
+
+ try {
+ // Perform an export of the database to SQL statements
+ s.execute("SCRIPT TO '" + exportFile + "'");
+ } catch (SQLException sqle) {
+ try {
+ s.close();
+ } catch (SQLException se2) {
+ // Ignore, the error will be handled
+ }
+ logger.error("Export of the database with the legacy driver failed, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
Review comment:
As mentioned above, it would be helpful to include the URL and Path:
```suggestion
final String message = String.format("H2 1.4 export failed URL [%s] Path [%s] SQL State [%s]", dbUrl, dbPath, sqle.getSQLState());
throw new RuntimeException(message, sqle);
```
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseMigrator.java
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.apache.nifi.org.h2.jdbcx.JdbcDataSource;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_POSTFIX;
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_PREFIX;
+
+public class H2DatabaseMigrator {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseMigrator.class);
+
+ public static final String BACKUP_FILE_POSTFIX = ".migration_backup";
+
+ public static void exportAndBackup(final String dbUrl, final String dbPath, final String user, final String pass) {
+
+ // Attempt to connect with the latest driver
+ JdbcDataSource migrationDataSource = new JdbcDataSource();
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+
+ final File dbPathFile = Paths.get(dbPath).toFile();
+
+ String exportFile = Paths.get(dbPathFile.getParent(), EXPORT_FILE_PREFIX + dbPathFile.getName() + EXPORT_FILE_POSTFIX).toString();
+
+ Connection conn = null;
+ Statement s = null;
+ try {
+ conn = migrationDataSource.getConnection();
+ s = conn.createStatement();
+ } catch (SQLException sqle) {
+ logger.error("Could not connect to the database with the legacy driver, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
+ }
+
+ try {
+ // Perform an export of the database to SQL statements
+ s.execute("SCRIPT TO '" + exportFile + "'");
+ } catch (SQLException sqle) {
+ try {
+ s.close();
+ } catch (SQLException se2) {
+ // Ignore, the error will be handled
+ }
+ logger.error("Export of the database with the legacy driver failed, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
+ }
+
+ try {
+ s.close();
+ conn.close();
+ } catch (SQLException se2) {
+ // Ignore, all connections should be severed when the process exits
+ }
Review comment:
With the updates to avoid forking a separate process, it seems like adding a closeQuietly() method, and calling it separately for Statement and Connection would ensure that the method closes both resources.
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseMigrator.java
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.apache.nifi.org.h2.jdbcx.JdbcDataSource;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_POSTFIX;
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_PREFIX;
+
+public class H2DatabaseMigrator {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseMigrator.class);
+
+ public static final String BACKUP_FILE_POSTFIX = ".migration_backup";
+
+ public static void exportAndBackup(final String dbUrl, final String dbPath, final String user, final String pass) {
+
+ // Attempt to connect with the latest driver
+ JdbcDataSource migrationDataSource = new JdbcDataSource();
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+
+ final File dbPathFile = Paths.get(dbPath).toFile();
+
+ String exportFile = Paths.get(dbPathFile.getParent(), EXPORT_FILE_PREFIX + dbPathFile.getName() + EXPORT_FILE_POSTFIX).toString();
+
+ Connection conn = null;
+ Statement s = null;
+ try {
+ conn = migrationDataSource.getConnection();
+ s = conn.createStatement();
+ } catch (SQLException sqle) {
+ logger.error("Could not connect to the database with the legacy driver, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
+ }
+
+ try {
+ // Perform an export of the database to SQL statements
+ s.execute("SCRIPT TO '" + exportFile + "'");
+ } catch (SQLException sqle) {
+ try {
+ s.close();
+ } catch (SQLException se2) {
+ // Ignore, the error will be handled
+ }
+ logger.error("Export of the database with the legacy driver failed, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
+ }
+
+ try {
+ s.close();
+ conn.close();
+ } catch (SQLException se2) {
+ // Ignore, all connections should be severed when the process exits
+ }
+
+ // Verify the export file exists
+ if (!Files.exists(Paths.get(exportFile))) {
+ logger.error("Export of the database with the legacy driver failed, no export file was created");
+ throw new RuntimeException("Export of the database with the legacy driver failed, no export file was created");
+ }
+
+ // Now that the export file exists, backup (rename) the DB files so the main process with the newer H2 driver can create and import the previous database
+ File dbDir = new File(dbPath).getParentFile();
+ File[] dbFiles = dbDir.listFiles((dir, name) -> !name.endsWith(EXPORT_FILE_POSTFIX) && name.startsWith(dbPathFile.getName()));
+ if (dbFiles == null || dbFiles.length == 0) {
+ logger.error("Backing up the legacy database failed, no database files were found.");
+ throw new RuntimeException("Backing up the legacy database failed, no database files were found.");
Review comment:
```suggestion
throw new RuntimeException(String.format("H2 1.4 backup failed URL [%s] Path [%s] no database files found", dbUrl, dbPath));
```
##########
File path: nifi-h2/nifi-h2-database-migrator/src/test/java/org/apache/nifi/dbmigration/h2/TestH2DatabaseUpdater.java
##########
@@ -0,0 +1,58 @@
+package org.apache.nifi.dbmigration.h2;
+
+import org.junit.jupiter.api.Test;
+
+import java.io.File;
+import java.nio.file.Paths;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+public class TestH2DatabaseUpdater {
+
+ @Test
+ public void testMigration() throws Exception {
+ final String dbPathNoExtension = "./src/test/resources/nifi-flow-audit";
+ final String dbFilePath = dbPathNoExtension + ".mv.db";
+ final File dbFile = new File(dbPathNoExtension);
+ final String migrationDbUrl = H2DatabaseUpdater.H2_URL_PREFIX + dbFile + ";LOCK_MODE=3";
+
+ final File newDbFile = new File(dbFilePath);
+ H2DatabaseUpdater.checkAndPerformMigration(dbFile, migrationDbUrl, "nf", "nf");
+
+ // Verify the export, backup, and new database files were created
+ final File exportFile = Paths.get(dbFile.getParent(), H2DatabaseUpdater.EXPORT_FILE_PREFIX + dbFile.getName() + H2DatabaseUpdater.EXPORT_FILE_POSTFIX).toFile();
+ assertTrue(exportFile.exists());
+
+ File dbDir = dbFile.getParentFile();
+ File[] backupFiles = dbDir.listFiles((dir, name) -> name.endsWith(H2DatabaseMigrator.BACKUP_FILE_POSTFIX) && name.startsWith(dbFile.getName()));
+ assertNotNull(backupFiles);
+ // The database and its trace file should exist after the initial connection is made, so they both should be backed up
+ assertEquals(2, backupFiles.length);
+ assertTrue(newDbFile.exists());
+
+ // Remove the export and backup files
+ exportFile.delete();
+ for (File f : backupFiles) {
+ f.delete();
+ }
Review comment:
Should this be moved to a method annotated with `@After`?
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.h2.jdbc.JdbcSQLNonTransientException;
+import org.h2.jdbcx.JdbcDataSource;
+import org.h2.mvstore.MVStoreException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseUpdater {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseUpdater.class);
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String H2_URL_PREFIX = "jdbc:h2:";
+
+ private static final JdbcDataSource migrationDataSource = new JdbcDataSource();
+
+ public static void checkAndPerformMigration(final File dbFile, final String dbUrl, final String user, final String pass) throws Exception {
+ try {
+ // Attempt to connect with the latest driver
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+ migrationDataSource.getConnection();
+ return;
+ } catch (JdbcSQLNonTransientException jsqlnte) {
+ // Migration/version issues will be caused by an MVStoreException
+ final Throwable exceptionCause = jsqlnte.getCause();
+ if (exceptionCause instanceof MVStoreException) {
+ // Check for specific error message
+ final String errorMessage = exceptionCause.getMessage();
+ if (!errorMessage.contains("The write format")
+ && !errorMessage.contains("is smaller than the supported format")) {
+ throw jsqlnte;
+ }
+ }
+ } catch (SQLException sqle) {
+ throw new RuntimeException("Error getting connection to H2 database unrelated to migration issues", sqle);
+ }
+ // At this point it is known that migration should be attempted
+ logger.info("H2 database version 1 detected, the database will be migrated to version 2. Existing files will be backed up to the same directory.");
Review comment:
```suggestion
logger.info("H2 1.4 database detected [{}]: starting migration to H2 2.1", dbFile);
```
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.h2.jdbc.JdbcSQLNonTransientException;
+import org.h2.jdbcx.JdbcDataSource;
+import org.h2.mvstore.MVStoreException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseUpdater {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseUpdater.class);
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String H2_URL_PREFIX = "jdbc:h2:";
+
+ private static final JdbcDataSource migrationDataSource = new JdbcDataSource();
+
+ public static void checkAndPerformMigration(final File dbFile, final String dbUrl, final String user, final String pass) throws Exception {
+ try {
+ // Attempt to connect with the latest driver
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+ migrationDataSource.getConnection();
+ return;
+ } catch (JdbcSQLNonTransientException jsqlnte) {
+ // Migration/version issues will be caused by an MVStoreException
+ final Throwable exceptionCause = jsqlnte.getCause();
+ if (exceptionCause instanceof MVStoreException) {
+ // Check for specific error message
+ final String errorMessage = exceptionCause.getMessage();
+ if (!errorMessage.contains("The write format")
+ && !errorMessage.contains("is smaller than the supported format")) {
+ throw jsqlnte;
+ }
+ }
+ } catch (SQLException sqle) {
+ throw new RuntimeException("Error getting connection to H2 database unrelated to migration issues", sqle);
+ }
+ // At this point it is known that migration should be attempted
+ logger.info("H2 database version 1 detected, the database will be migrated to version 2. Existing files will be backed up to the same directory.");
+ H2DatabaseMigrator.exportAndBackup(dbUrl, dbFile.getAbsolutePath(), user, pass);
+
+ // The export file has been created and the DB has been backed up, now create a new one with the same name and run the SQL script to import the database
+ try (Connection migrationConnection = migrationDataSource.getConnection();
+ Statement s = migrationConnection.createStatement()) {
+ // use RUNSCRIPT to recreate the database
+ final String exportSqlLocation = dbFile.getParentFile().getAbsolutePath() + File.separator
+ + H2DatabaseUpdater.EXPORT_FILE_PREFIX + dbFile.getName() + H2DatabaseUpdater.EXPORT_FILE_POSTFIX;
+ s.execute("RUNSCRIPT FROM '" + exportSqlLocation + "'");
+
+ } catch (SQLException sqle) {
+ throw new IOException("Error creating database for import", sqle);
Review comment:
```suggestion
throw new IOException(String.format("H2 import database creation failed URL [%s]", dbUrl), sqle);
```
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseMigrator.java
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.apache.nifi.org.h2.jdbcx.JdbcDataSource;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_POSTFIX;
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_PREFIX;
+
+public class H2DatabaseMigrator {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseMigrator.class);
+
+ public static final String BACKUP_FILE_POSTFIX = ".migration_backup";
+
+ public static void exportAndBackup(final String dbUrl, final String dbPath, final String user, final String pass) {
+
+ // Attempt to connect with the latest driver
+ JdbcDataSource migrationDataSource = new JdbcDataSource();
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+
+ final File dbPathFile = Paths.get(dbPath).toFile();
+
+ String exportFile = Paths.get(dbPathFile.getParent(), EXPORT_FILE_PREFIX + dbPathFile.getName() + EXPORT_FILE_POSTFIX).toString();
+
+ Connection conn = null;
+ Statement s = null;
+ try {
+ conn = migrationDataSource.getConnection();
+ s = conn.createStatement();
+ } catch (SQLException sqle) {
+ logger.error("Could not connect to the database with the legacy driver, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
+ }
+
+ try {
+ // Perform an export of the database to SQL statements
+ s.execute("SCRIPT TO '" + exportFile + "'");
+ } catch (SQLException sqle) {
+ try {
+ s.close();
+ } catch (SQLException se2) {
+ // Ignore, the error will be handled
+ }
+ logger.error("Export of the database with the legacy driver failed, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
+ }
+
+ try {
+ s.close();
+ conn.close();
+ } catch (SQLException se2) {
+ // Ignore, all connections should be severed when the process exits
+ }
+
+ // Verify the export file exists
+ if (!Files.exists(Paths.get(exportFile))) {
+ logger.error("Export of the database with the legacy driver failed, no export file was created");
+ throw new RuntimeException("Export of the database with the legacy driver failed, no export file was created");
Review comment:
```suggestion
throw new RuntimeException(String.format("H2 1.4 export failed URL [%s] Path [%s] Export File not found [%s]", dbUrl, dbPath, exportFile));
```
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseMigrator.java
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.apache.nifi.org.h2.jdbcx.JdbcDataSource;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_POSTFIX;
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_PREFIX;
+
+public class H2DatabaseMigrator {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseMigrator.class);
+
+ public static final String BACKUP_FILE_POSTFIX = ".migration_backup";
+
+ public static void exportAndBackup(final String dbUrl, final String dbPath, final String user, final String pass) {
+
+ // Attempt to connect with the latest driver
+ JdbcDataSource migrationDataSource = new JdbcDataSource();
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+
+ final File dbPathFile = Paths.get(dbPath).toFile();
+
+ String exportFile = Paths.get(dbPathFile.getParent(), EXPORT_FILE_PREFIX + dbPathFile.getName() + EXPORT_FILE_POSTFIX).toString();
+
+ Connection conn = null;
+ Statement s = null;
+ try {
+ conn = migrationDataSource.getConnection();
+ s = conn.createStatement();
+ } catch (SQLException sqle) {
+ logger.error("Could not connect to the database with the legacy driver, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
+ }
+
+ try {
+ // Perform an export of the database to SQL statements
+ s.execute("SCRIPT TO '" + exportFile + "'");
+ } catch (SQLException sqle) {
+ try {
+ s.close();
+ } catch (SQLException se2) {
+ // Ignore, the error will be handled
+ }
+ logger.error("Export of the database with the legacy driver failed, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
+ }
+
+ try {
+ s.close();
+ conn.close();
+ } catch (SQLException se2) {
+ // Ignore, all connections should be severed when the process exits
+ }
+
+ // Verify the export file exists
+ if (!Files.exists(Paths.get(exportFile))) {
+ logger.error("Export of the database with the legacy driver failed, no export file was created");
+ throw new RuntimeException("Export of the database with the legacy driver failed, no export file was created");
+ }
+
+ // Now that the export file exists, backup (rename) the DB files so the main process with the newer H2 driver can create and import the previous database
+ File dbDir = new File(dbPath).getParentFile();
+ File[] dbFiles = dbDir.listFiles((dir, name) -> !name.endsWith(EXPORT_FILE_POSTFIX) && name.startsWith(dbPathFile.getName()));
+ if (dbFiles == null || dbFiles.length == 0) {
+ logger.error("Backing up the legacy database failed, no database files were found.");
+ throw new RuntimeException("Backing up the legacy database failed, no database files were found.");
+ }
+
+ for (File dbFile : dbFiles) {
+ File dbBackupFile = new File(dbFile.getAbsolutePath() + BACKUP_FILE_POSTFIX);
+ if (!dbFile.renameTo(dbBackupFile)) {
+ logger.error("Backing up the legacy database failed, " + dbFile.getName() + " could not be renamed.");
+ throw new RuntimeException("Backing up the legacy database failed, " + dbFile.getName() + " could not be renamed.");
+ }
+ }
+
+ // exit gracefully so the H2 migrator can proceed with rebuilding the database
Review comment:
Is this comment still applicable?
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.h2.jdbc.JdbcSQLNonTransientException;
+import org.h2.jdbcx.JdbcDataSource;
+import org.h2.mvstore.MVStoreException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseUpdater {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseUpdater.class);
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String H2_URL_PREFIX = "jdbc:h2:";
+
+ private static final JdbcDataSource migrationDataSource = new JdbcDataSource();
+
+ public static void checkAndPerformMigration(final File dbFile, final String dbUrl, final String user, final String pass) throws Exception {
+ try {
+ // Attempt to connect with the latest driver
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+ migrationDataSource.getConnection();
+ return;
+ } catch (JdbcSQLNonTransientException jsqlnte) {
+ // Migration/version issues will be caused by an MVStoreException
+ final Throwable exceptionCause = jsqlnte.getCause();
+ if (exceptionCause instanceof MVStoreException) {
+ // Check for specific error message
+ final String errorMessage = exceptionCause.getMessage();
+ if (!errorMessage.contains("The write format")
+ && !errorMessage.contains("is smaller than the supported format")) {
+ throw jsqlnte;
+ }
+ }
+ } catch (SQLException sqle) {
+ throw new RuntimeException("Error getting connection to H2 database unrelated to migration issues", sqle);
Review comment:
```suggestion
throw new RuntimeException(String.format("H2 connection failed URL [%s] File [%s]", dbUrl, dbFile), sqle);
```
##########
File path: nifi-h2/nifi-h2-database-migrator/src/test/java/org/apache/nifi/dbmigration/h2/TestH2DatabaseUpdater.java
##########
@@ -0,0 +1,58 @@
+package org.apache.nifi.dbmigration.h2;
+
+import org.junit.jupiter.api.Test;
+
+import java.io.File;
+import java.nio.file.Paths;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+public class TestH2DatabaseUpdater {
+
+ @Test
+ public void testMigration() throws Exception {
+ final String dbPathNoExtension = "./src/test/resources/nifi-flow-audit";
+ final String dbFilePath = dbPathNoExtension + ".mv.db";
+ final File dbFile = new File(dbPathNoExtension);
Review comment:
Does this migration test replace the original test file, or will it stay intact for subsequent test runs?
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseMigrator.java
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.apache.nifi.org.h2.jdbcx.JdbcDataSource;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_POSTFIX;
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_PREFIX;
+
+public class H2DatabaseMigrator {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseMigrator.class);
+
+ public static final String BACKUP_FILE_POSTFIX = ".migration_backup";
+
+ public static void exportAndBackup(final String dbUrl, final String dbPath, final String user, final String pass) {
+
+ // Attempt to connect with the latest driver
+ JdbcDataSource migrationDataSource = new JdbcDataSource();
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+
+ final File dbPathFile = Paths.get(dbPath).toFile();
+
+ String exportFile = Paths.get(dbPathFile.getParent(), EXPORT_FILE_PREFIX + dbPathFile.getName() + EXPORT_FILE_POSTFIX).toString();
+
+ Connection conn = null;
+ Statement s = null;
+ try {
+ conn = migrationDataSource.getConnection();
+ s = conn.createStatement();
+ } catch (SQLException sqle) {
+ logger.error("Could not connect to the database with the legacy driver, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
Review comment:
It would be helpful to include the Database URL and Database Path in the message for troubleshooting. Is it necessary to write an error log and throw the exception as opposed to just throwing the exception?
```suggestion
final String message = String.format("H2 1.4 connection failed URL [%s] Path [%s] SQL State [%s]", dbUrl, dbPath, sqle.getSQLState());
throw new RuntimeException(message, sqle);
```
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.h2.jdbc.JdbcSQLNonTransientException;
+import org.h2.jdbcx.JdbcDataSource;
+import org.h2.mvstore.MVStoreException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseUpdater {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseUpdater.class);
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String H2_URL_PREFIX = "jdbc:h2:";
+
+ private static final JdbcDataSource migrationDataSource = new JdbcDataSource();
+
+ public static void checkAndPerformMigration(final File dbFile, final String dbUrl, final String user, final String pass) throws Exception {
+ try {
+ // Attempt to connect with the latest driver
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+ migrationDataSource.getConnection();
Review comment:
It looks like this `getConnection()` call could leave the connection open on success, is that correct? What about the following approach?
```suggestion
migrationDataSource.setURL(dbUrl);
migrationDataSource.setUser(user);
migrationDataSource.setPassword(pass);
try (Connection connection = migrationDataSource.getConnection()) {
```
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.h2.jdbc.JdbcSQLNonTransientException;
+import org.h2.jdbcx.JdbcDataSource;
+import org.h2.mvstore.MVStoreException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseUpdater {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseUpdater.class);
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String H2_URL_PREFIX = "jdbc:h2:";
+
+ private static final JdbcDataSource migrationDataSource = new JdbcDataSource();
+
+ public static void checkAndPerformMigration(final File dbFile, final String dbUrl, final String user, final String pass) throws Exception {
+ try {
+ // Attempt to connect with the latest driver
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+ migrationDataSource.getConnection();
+ return;
+ } catch (JdbcSQLNonTransientException jsqlnte) {
+ // Migration/version issues will be caused by an MVStoreException
+ final Throwable exceptionCause = jsqlnte.getCause();
+ if (exceptionCause instanceof MVStoreException) {
+ // Check for specific error message
+ final String errorMessage = exceptionCause.getMessage();
+ if (!errorMessage.contains("The write format")
+ && !errorMessage.contains("is smaller than the supported format")) {
+ throw jsqlnte;
+ }
+ }
+ } catch (SQLException sqle) {
+ throw new RuntimeException("Error getting connection to H2 database unrelated to migration issues", sqle);
+ }
+ // At this point it is known that migration should be attempted
+ logger.info("H2 database version 1 detected, the database will be migrated to version 2. Existing files will be backed up to the same directory.");
+ H2DatabaseMigrator.exportAndBackup(dbUrl, dbFile.getAbsolutePath(), user, pass);
+
+ // The export file has been created and the DB has been backed up, now create a new one with the same name and run the SQL script to import the database
+ try (Connection migrationConnection = migrationDataSource.getConnection();
+ Statement s = migrationConnection.createStatement()) {
+ // use RUNSCRIPT to recreate the database
+ final String exportSqlLocation = dbFile.getParentFile().getAbsolutePath() + File.separator
+ + H2DatabaseUpdater.EXPORT_FILE_PREFIX + dbFile.getName() + H2DatabaseUpdater.EXPORT_FILE_POSTFIX;
+ s.execute("RUNSCRIPT FROM '" + exportSqlLocation + "'");
+
+ } catch (SQLException sqle) {
+ throw new IOException("Error creating database for import", sqle);
+ }
+
+ logger.info("H2 Database Migration process for " + dbFile.getName() + " completed successfully");
Review comment:
```suggestion
logger.info("H2 1.4 to 2.1 migration completed [{}]", dbFile);
```
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseMigrator.java
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.apache.nifi.org.h2.jdbcx.JdbcDataSource;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_POSTFIX;
+import static org.apache.nifi.dbmigration.h2.H2DatabaseUpdater.EXPORT_FILE_PREFIX;
+
+public class H2DatabaseMigrator {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseMigrator.class);
+
+ public static final String BACKUP_FILE_POSTFIX = ".migration_backup";
+
+ public static void exportAndBackup(final String dbUrl, final String dbPath, final String user, final String pass) {
+
+ // Attempt to connect with the latest driver
+ JdbcDataSource migrationDataSource = new JdbcDataSource();
+ migrationDataSource.setURL(dbUrl);
+ migrationDataSource.setUser(user);
+ migrationDataSource.setPassword(pass);
+
+ final File dbPathFile = Paths.get(dbPath).toFile();
+
+ String exportFile = Paths.get(dbPathFile.getParent(), EXPORT_FILE_PREFIX + dbPathFile.getName() + EXPORT_FILE_POSTFIX).toString();
+
+ Connection conn = null;
+ Statement s = null;
+ try {
+ conn = migrationDataSource.getConnection();
+ s = conn.createStatement();
+ } catch (SQLException sqle) {
+ logger.error("Could not connect to the database with the legacy driver, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
+ }
+
+ try {
+ // Perform an export of the database to SQL statements
+ s.execute("SCRIPT TO '" + exportFile + "'");
+ } catch (SQLException sqle) {
+ try {
+ s.close();
+ } catch (SQLException se2) {
+ // Ignore, the error will be handled
+ }
+ logger.error("Export of the database with the legacy driver failed, cause: " + sqle.getMessage() + ", SQL State = " + sqle.getSQLState());
+ throw new RuntimeException(sqle);
+ }
+
+ try {
+ s.close();
+ conn.close();
+ } catch (SQLException se2) {
+ // Ignore, all connections should be severed when the process exits
+ }
+
+ // Verify the export file exists
+ if (!Files.exists(Paths.get(exportFile))) {
+ logger.error("Export of the database with the legacy driver failed, no export file was created");
+ throw new RuntimeException("Export of the database with the legacy driver failed, no export file was created");
+ }
+
+ // Now that the export file exists, backup (rename) the DB files so the main process with the newer H2 driver can create and import the previous database
+ File dbDir = new File(dbPath).getParentFile();
+ File[] dbFiles = dbDir.listFiles((dir, name) -> !name.endsWith(EXPORT_FILE_POSTFIX) && name.startsWith(dbPathFile.getName()));
+ if (dbFiles == null || dbFiles.length == 0) {
+ logger.error("Backing up the legacy database failed, no database files were found.");
+ throw new RuntimeException("Backing up the legacy database failed, no database files were found.");
+ }
+
+ for (File dbFile : dbFiles) {
+ File dbBackupFile = new File(dbFile.getAbsolutePath() + BACKUP_FILE_POSTFIX);
+ if (!dbFile.renameTo(dbBackupFile)) {
+ logger.error("Backing up the legacy database failed, " + dbFile.getName() + " could not be renamed.");
+ throw new RuntimeException("Backing up the legacy database failed, " + dbFile.getName() + " could not be renamed.");
Review comment:
```suggestion
throw new RuntimeException(String.format("H2 1.4 backup failed URL [%s] Path [%s] rename failed [%s]", dbUrl, dbPath, dbFile));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a change in pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
mattyb149 commented on a change in pull request #5724:
URL: https://github.com/apache/nifi/pull/5724#discussion_r812020384
##########
File path: nifi-h2/nifi-h2-database-migrator/src/test/java/org/apache/nifi/dbmigration/h2/TestH2DatabaseUpdater.java
##########
@@ -0,0 +1,58 @@
+package org.apache.nifi.dbmigration.h2;
+
+import org.junit.jupiter.api.Test;
+
+import java.io.File;
+import java.nio.file.Paths;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+public class TestH2DatabaseUpdater {
+
+ @Test
+ public void testMigration() throws Exception {
+ final String dbPathNoExtension = "./src/test/resources/nifi-flow-audit";
+ final String dbFilePath = dbPathNoExtension + ".mv.db";
+ final File dbFile = new File(dbPathNoExtension);
Review comment:
It replaces it, I'll create a copy of the original and restore it at the end of the test, good catch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
joewitt commented on pull request #5724:
URL: https://github.com/apache/nifi/pull/5724#issuecomment-1024402198
So ... this is great because we needed NiFi to move to H2 version 2 anyway. And registry. However, we simply cannot have such a disruptive process. We're not truly exposed to the H2 vulnerabilities in question that we know of anyway so we can and should try to do more for the users here.
For NiFi H2 usage is for audit data and auth stuff. For auth it is fairly transient meaning users can just relogin in after the upgrade if we lost the data. For the audit data we can say they'll lose it. But lets try to do better. Lets try to have old h2 lib when we detect the old format. It would dump out the SQL statements/export of the old db. Then it is off/done closed out. Then we have new h2/v2 take over and import. This should keep the users stuff all happy/they lose nothing. And while technically the h2 1.x jars are still present we can easily declare we dont expose the vulnerability. Later we can delete it entirely such as at a major version change (nifi 2.x).
For the registry..well...we gotta make it seamless like the above.
So unfortunately I think we need to do a good bit more work here. This as is simply causes to much disruption for users. A toolkit option is also viable but introduces considerable additional complexity that we usually reserve for major upgrades.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a change in pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
mattyb149 commented on a change in pull request #5724:
URL: https://github.com/apache/nifi/pull/5724#discussion_r813405013
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.h2.jdbc.JdbcSQLNonTransientException;
+import org.h2.jdbcx.JdbcDataSource;
+import org.h2.mvstore.MVStoreException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseUpdater {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseUpdater.class);
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String H2_URL_PREFIX = "jdbc:h2:";
+
+ private static final JdbcDataSource migrationDataSource = new JdbcDataSource();
+
+ public static void checkAndPerformMigration(final File dbFile, final String dbUrl, final String user, final String pass) throws Exception {
Review comment:
`URL()` doesn't recognize `jdbc` as a protocol so I think we still need to pass in a String of some sort, but I can change it to String and name it more clearly since it's not really a file at all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on a change in pull request #5724:
URL: https://github.com/apache/nifi/pull/5724#discussion_r813317868
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.h2.jdbc.JdbcSQLNonTransientException;
+import org.h2.jdbcx.JdbcDataSource;
+import org.h2.mvstore.MVStoreException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseUpdater {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseUpdater.class);
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String H2_URL_PREFIX = "jdbc:h2:";
+
+ private static final JdbcDataSource migrationDataSource = new JdbcDataSource();
Review comment:
Although the `H2DatabaseUpdater` class does not appear to be used in a multi-threaded context, moving this declaration inside the `checkAndPerformMigration` method would avoid the potential for unexpected behavior with different connection parameters.
##########
File path: nifi-h2/nifi-h2-database-migrator/src/test/java/org/apache/nifi/dbmigration/h2/TestH2DatabaseUpdater.java
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+
+import java.io.File;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+
+import static java.nio.file.StandardCopyOption.REPLACE_EXISTING;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class TestH2DatabaseUpdater {
+
+ public static final String AUDIT_DB_PATH = "./src/test/resources/nifi-flow-audit";
+ public static final String AUDIT_DB_PATH_FILE = AUDIT_DB_PATH + ".mv.db";
+ public static final String AUDIT_DB_PATH_FILE_ORIGINAL = "./src/test/resources/orig.nifi-flow-audit";
Review comment:
Rather than adding two binary files to version control, it should be possible to create a `BeforeEach` annotated method that copies the `nifi-flow-audit` to a temporary directory. Using the `@TempDir` annotation on a member variable of type `java.io.File` will cause JUnit 5 to create a temporary directory that can be used for migration operations. This also should remove the need to manually removing backup files after test completion.
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
Review comment:
Based on the module name, it seems like the more natural package name would be as follows:
```suggestion
package org.apache.nifi.h2.database.migration;
```
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.h2.jdbc.JdbcSQLNonTransientException;
+import org.h2.jdbcx.JdbcDataSource;
+import org.h2.mvstore.MVStoreException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseUpdater {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseUpdater.class);
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String H2_URL_PREFIX = "jdbc:h2:";
+
+ private static final JdbcDataSource migrationDataSource = new JdbcDataSource();
+
+ public static void checkAndPerformMigration(final File dbFile, final String dbUrl, final String user, final String pass) throws Exception {
Review comment:
Although `dbFile` is of type `java.io.File`, it appears from the usage that the File does not actually exist. Since the H2 URL does not include `.mv.db`, this is somewhat confusing. Would it make sense to remove the `dbFile` parameter and have this method parse the file path from the `dbUrl`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on a change in pull request #5724:
URL: https://github.com/apache/nifi/pull/5724#discussion_r813482348
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.h2.jdbc.JdbcSQLNonTransientException;
+import org.h2.jdbcx.JdbcDataSource;
+import org.h2.mvstore.MVStoreException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseUpdater {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseUpdater.class);
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String H2_URL_PREFIX = "jdbc:h2:";
+
+ private static final JdbcDataSource migrationDataSource = new JdbcDataSource();
+
+ public static void checkAndPerformMigration(final File dbFile, final String dbUrl, final String user, final String pass) throws Exception {
Review comment:
Either way, if the two parameters could be collapsed into one, that might be helpful. Not a requirement, but might be worth considering since you are moving away from the `File` type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a change in pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
mattyb149 commented on a change in pull request #5724:
URL: https://github.com/apache/nifi/pull/5724#discussion_r813490046
##########
File path: nifi-h2/nifi-h2-database-migrator/src/main/java/org/apache/nifi/dbmigration/h2/H2DatabaseUpdater.java
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.dbmigration.h2;
+
+import org.h2.jdbc.JdbcSQLNonTransientException;
+import org.h2.jdbcx.JdbcDataSource;
+import org.h2.mvstore.MVStoreException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseUpdater {
+
+ private static final Logger logger = LoggerFactory.getLogger(H2DatabaseUpdater.class);
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String H2_URL_PREFIX = "jdbc:h2:";
+
+ private static final JdbcDataSource migrationDataSource = new JdbcDataSource();
+
+ public static void checkAndPerformMigration(final File dbFile, final String dbUrl, final String user, final String pass) throws Exception {
Review comment:
I can try to use URI instead of URL but I can't use the `URL()` constructor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a change in pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
mattyb149 commented on a change in pull request #5724:
URL: https://github.com/apache/nifi/pull/5724#discussion_r797122151
##########
File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-h2-database-migrator/src/main/java/org/apache/nifi/admin/dbmigration/H2DatabaseMigratorProcess.java
##########
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.admin.dbmigration;
+
+import org.h2.jdbcx.JdbcDataSource;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.sql.Statement;
+
+public class H2DatabaseMigratorProcess {
+
+ public static final String EXPORT_FILE_PREFIX = "export_";
+ public static final String EXPORT_FILE_POSTFIX = ".sql";
+ public static final String BACKUP_FILE_POSTFIX = ".migration_backup";
+
+ public static void main(String[] args) {
+ if (args.length < 4) {
+ // Write error to file
+ StringBuilder errorMsg = new StringBuilder("Migration tool requires 4 arguments, the JDBC URL to the H2 database, the path to the database file, "
+ + "and the username and password for the database.\n");
+ if (args.length == 0) {
+ errorMsg.append("No arguments given");
+ } else {
+ for (int i = 1; i < args.length; i++) {
+ errorMsg.append("arg[");
+ errorMsg.append(i);
+ errorMsg.append("] = ");
+ errorMsg.append(args[i]);
+ errorMsg.append("\n");
+ }
+ }
+ handleError(errorMsg.toString(), 10);
+ }
+
+ final String dbUrl = args[0];
+ final String dbPath = args[1];
+ final String user = args[2];
+ final String pass = args[3];
+
+ // Attempt to connect with the latest driver
Review comment:
Will change this to legacy driver along with any other review incorporations
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5724: NIFI-9585: Upgrade H2 to 2.1.210
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on a change in pull request #5724:
URL: https://github.com/apache/nifi/pull/5724#discussion_r798012090
##########
File path: pom.xml
##########
@@ -41,6 +41,8 @@
<module>nifi-toolkit</module>
<module>nifi-manifest</module>
<module>c2</module>
+ <module>nifi-h2-database-migrator</module>
+ <module>nifi-h2-database-updater</module>
Review comment:
What do you think about either moving these under their own module like `nifi-h2`, or moving them under `nifi-commons`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org