You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2022/03/07 14:54:22 UTC
[GitHub] [hive] pvary opened a new pull request #3081: HIVE-26002: Create db scripts for 4.0.0-alpha-1
pvary opened a new pull request #3081:
URL: https://github.com/apache/hive/pull/3081
### What changes were proposed in this pull request?
Create the 4.0.0-alpha-1 scripts for HMS db initialization
### Why are the changes needed?
So we can release the 4.0.0-alpha-1
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Added a new test case, and also manually added the new files when we want to upgrade from 4.0.0-alpha-1 to 4.0.0. And run the test with that too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary merged pull request #3081: HIVE-26002: Preparing for 4.0.0-alpha-1 development
Posted by GitBox <gi...@apache.org>.
pvary merged pull request #3081:
URL: https://github.com/apache/hive/pull/3081
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zabetak commented on a change in pull request #3081: HIVE-26002: Create db scripts for 4.0.0-alpha-1
Posted by GitBox <gi...@apache.org>.
zabetak commented on a change in pull request #3081:
URL: https://github.com/apache/hive/pull/3081#discussion_r820854060
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/tools/schematool/TestSchemaToolForMetastore.java
##########
@@ -370,6 +370,44 @@ public void testSchemaUpgrade() throws Exception {
schemaTool.verifySchemaVersion();
}
+ /**
+ * Test schema upgrade checking that the alpha version is called
+ */
+ @Test
+ public void testSchemaUpgradeFromAlphaVersion() throws Exception {
+ boolean foundException = false;
+ // Initialize 1.2.0 schema
+ execute(new SchemaToolTaskInit(), "-initSchemaTo 3.1.0");
+ // verify that driver fails due to older version schema
+ try {
+ schemaTool.verifySchemaVersion();
+ } catch (HiveMetaException e) {
+ // Expected to fail due to old schema
+ foundException = true;
+ }
+ if (!foundException) {
+ throw new Exception(
+ "Hive operations shouldn't pass with older version schema");
+ }
+
+ // Capture system out and err
+ schemaTool.setVerbose(true);
+ OutputStream stderr = new ByteArrayOutputStream();
+ PrintStream errPrintStream = new PrintStream(stderr);
+ System.setErr(errPrintStream);
+ OutputStream stdout = new ByteArrayOutputStream();
+ PrintStream outPrintStream = new PrintStream(stdout);
+ System.setOut(outPrintStream);
+
+ // Upgrade schema from 0.7.0 to latest
+ execute(new SchemaToolTaskUpgrade(), "-upgradeSchemaFrom 3.1.0");
Review comment:
nit: same comment as above
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/tools/schematool/TestSchemaToolForMetastore.java
##########
@@ -370,6 +370,44 @@ public void testSchemaUpgrade() throws Exception {
schemaTool.verifySchemaVersion();
}
+ /**
+ * Test schema upgrade checking that the alpha version is called
+ */
+ @Test
+ public void testSchemaUpgradeFromAlphaVersion() throws Exception {
Review comment:
Aren't tests under `DbInstallBase` sufficient to cover the upgrade? If they are not sufficient maybe it would make sense to add additional tests there so that we cover all supported backends.
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/tools/schematool/TestSchemaToolForMetastore.java
##########
@@ -370,6 +370,44 @@ public void testSchemaUpgrade() throws Exception {
schemaTool.verifySchemaVersion();
}
+ /**
+ * Test schema upgrade checking that the alpha version is called
+ */
+ @Test
+ public void testSchemaUpgradeFromAlphaVersion() throws Exception {
+ boolean foundException = false;
+ // Initialize 1.2.0 schema
+ execute(new SchemaToolTaskInit(), "-initSchemaTo 3.1.0");
+ // verify that driver fails due to older version schema
+ try {
+ schemaTool.verifySchemaVersion();
+ } catch (HiveMetaException e) {
+ // Expected to fail due to old schema
+ foundException = true;
+ }
+ if (!foundException) {
+ throw new Exception(
+ "Hive operations shouldn't pass with older version schema");
+ }
+
+ // Capture system out and err
+ schemaTool.setVerbose(true);
+ OutputStream stderr = new ByteArrayOutputStream();
+ PrintStream errPrintStream = new PrintStream(stderr);
+ System.setErr(errPrintStream);
+ OutputStream stdout = new ByteArrayOutputStream();
+ PrintStream outPrintStream = new PrintStream(stdout);
+ System.setOut(outPrintStream);
+
+ // Upgrade schema from 0.7.0 to latest
+ execute(new SchemaToolTaskUpgrade(), "-upgradeSchemaFrom 3.1.0");
+
+ Assert.assertTrue(stdout.toString().contains("upgrade-3.2.0-to-4.0.0-alpha-1.derby.sql"));
+
+ // Verify that driver works fine with latest schema
+ schemaTool.verifySchemaVersion();
Review comment:
nit: redundant comment
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/tools/schematool/TestSchemaToolForMetastore.java
##########
@@ -370,6 +370,44 @@ public void testSchemaUpgrade() throws Exception {
schemaTool.verifySchemaVersion();
}
+ /**
+ * Test schema upgrade checking that the alpha version is called
+ */
+ @Test
+ public void testSchemaUpgradeFromAlphaVersion() throws Exception {
+ boolean foundException = false;
+ // Initialize 1.2.0 schema
+ execute(new SchemaToolTaskInit(), "-initSchemaTo 3.1.0");
Review comment:
nit: comment does not add much to the code thus could be removed. Also the version in the comment is different than the one actually used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary commented on a change in pull request #3081: HIVE-26002: Create db scripts for 4.0.0-alpha-1
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #3081:
URL: https://github.com/apache/hive/pull/3081#discussion_r822714179
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/tools/schematool/TestSchemaToolForMetastore.java
##########
@@ -370,6 +370,44 @@ public void testSchemaUpgrade() throws Exception {
schemaTool.verifySchemaVersion();
}
+ /**
+ * Test schema upgrade checking that the alpha version is called
+ */
+ @Test
+ public void testSchemaUpgradeFromAlphaVersion() throws Exception {
Review comment:
With this test I was able to run them in unit tests and check the results.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zabetak commented on a change in pull request #3081: HIVE-26002: Create db scripts for 4.0.0-alpha-1
Posted by GitBox <gi...@apache.org>.
zabetak commented on a change in pull request #3081:
URL: https://github.com/apache/hive/pull/3081#discussion_r822796477
##########
File path: standalone-metastore/pom.xml
##########
@@ -31,7 +31,7 @@
</modules>
<properties>
<hive.version>4.0.0-SNAPSHOT</hive.version>
Review comment:
What about this? It doesn't require changing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary merged pull request #3081: HIVE-26002: Preparing for 4.0.0-alpha-1 development
Posted by GitBox <gi...@apache.org>.
pvary merged pull request #3081:
URL: https://github.com/apache/hive/pull/3081
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary commented on a change in pull request #3081: HIVE-26002: Update next version to 4.0.0-alpha-1
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #3081:
URL: https://github.com/apache/hive/pull/3081#discussion_r827750423
##########
File path: standalone-metastore/pom.xml
##########
@@ -31,7 +31,7 @@
</modules>
<properties>
<hive.version>4.0.0-SNAPSHOT</hive.version>
Review comment:
Yeah, I needed to fix it in several more places 😢
I hope I am finished for now 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zabetak commented on a change in pull request #3081: HIVE-26002: Create db scripts for 4.0.0-alpha-1
Posted by GitBox <gi...@apache.org>.
zabetak commented on a change in pull request #3081:
URL: https://github.com/apache/hive/pull/3081#discussion_r822746338
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/tools/schematool/TestSchemaToolForMetastore.java
##########
@@ -370,6 +370,44 @@ public void testSchemaUpgrade() throws Exception {
schemaTool.verifySchemaVersion();
}
+ /**
+ * Test schema upgrade checking that the alpha version is called
+ */
+ @Test
+ public void testSchemaUpgradeFromAlphaVersion() throws Exception {
Review comment:
It's true that `DbInstallBase` tests are a bit trickier to run but they should run everywhere. The following works for me locally:
`mvn -pl standalone-metastore/metastore-server/ verify -DskipITests=false -Dit.test=ITestDerby -Dtest=nosuch`
Moreover, I don't find a strong reason to have these `DbInstallBase` declared as integration tests. They just complicate our lives.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zabetak commented on a change in pull request #3081: HIVE-26002: Create db scripts for 4.0.0-alpha-1
Posted by GitBox <gi...@apache.org>.
zabetak commented on a change in pull request #3081:
URL: https://github.com/apache/hive/pull/3081#discussion_r822768769
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/tools/schematool/TestSchemaToolForMetastore.java
##########
@@ -370,6 +370,44 @@ public void testSchemaUpgrade() throws Exception {
schemaTool.verifySchemaVersion();
}
+ /**
+ * Test schema upgrade checking that the alpha version is called
+ */
+ @Test
+ public void testSchemaUpgradeFromAlphaVersion() throws Exception {
Review comment:
FYI: https://issues.apache.org/jira/browse/HIVE-26021 regarding my last comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] pvary commented on a change in pull request #3081: HIVE-26002: Create db scripts for 4.0.0-alpha-1
Posted by GitBox <gi...@apache.org>.
pvary commented on a change in pull request #3081:
URL: https://github.com/apache/hive/pull/3081#discussion_r822714418
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/tools/schematool/TestSchemaToolForMetastore.java
##########
@@ -370,6 +370,44 @@ public void testSchemaUpgrade() throws Exception {
schemaTool.verifySchemaVersion();
}
+ /**
+ * Test schema upgrade checking that the alpha version is called
+ */
+ @Test
+ public void testSchemaUpgradeFromAlphaVersion() throws Exception {
+ boolean foundException = false;
+ // Initialize 1.2.0 schema
+ execute(new SchemaToolTaskInit(), "-initSchemaTo 3.1.0");
Review comment:
removed the comment
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/tools/schematool/TestSchemaToolForMetastore.java
##########
@@ -370,6 +370,44 @@ public void testSchemaUpgrade() throws Exception {
schemaTool.verifySchemaVersion();
}
+ /**
+ * Test schema upgrade checking that the alpha version is called
+ */
+ @Test
+ public void testSchemaUpgradeFromAlphaVersion() throws Exception {
+ boolean foundException = false;
+ // Initialize 1.2.0 schema
+ execute(new SchemaToolTaskInit(), "-initSchemaTo 3.1.0");
+ // verify that driver fails due to older version schema
+ try {
+ schemaTool.verifySchemaVersion();
+ } catch (HiveMetaException e) {
+ // Expected to fail due to old schema
+ foundException = true;
+ }
+ if (!foundException) {
+ throw new Exception(
+ "Hive operations shouldn't pass with older version schema");
+ }
+
+ // Capture system out and err
+ schemaTool.setVerbose(true);
+ OutputStream stderr = new ByteArrayOutputStream();
+ PrintStream errPrintStream = new PrintStream(stderr);
+ System.setErr(errPrintStream);
+ OutputStream stdout = new ByteArrayOutputStream();
+ PrintStream outPrintStream = new PrintStream(stdout);
+ System.setOut(outPrintStream);
+
+ // Upgrade schema from 0.7.0 to latest
+ execute(new SchemaToolTaskUpgrade(), "-upgradeSchemaFrom 3.1.0");
+
+ Assert.assertTrue(stdout.toString().contains("upgrade-3.2.0-to-4.0.0-alpha-1.derby.sql"));
+
+ // Verify that driver works fine with latest schema
+ schemaTool.verifySchemaVersion();
Review comment:
removed the comment
##########
File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/tools/schematool/TestSchemaToolForMetastore.java
##########
@@ -370,6 +370,44 @@ public void testSchemaUpgrade() throws Exception {
schemaTool.verifySchemaVersion();
}
+ /**
+ * Test schema upgrade checking that the alpha version is called
+ */
+ @Test
+ public void testSchemaUpgradeFromAlphaVersion() throws Exception {
+ boolean foundException = false;
+ // Initialize 1.2.0 schema
+ execute(new SchemaToolTaskInit(), "-initSchemaTo 3.1.0");
+ // verify that driver fails due to older version schema
+ try {
+ schemaTool.verifySchemaVersion();
+ } catch (HiveMetaException e) {
+ // Expected to fail due to old schema
+ foundException = true;
+ }
+ if (!foundException) {
+ throw new Exception(
+ "Hive operations shouldn't pass with older version schema");
+ }
+
+ // Capture system out and err
+ schemaTool.setVerbose(true);
+ OutputStream stderr = new ByteArrayOutputStream();
+ PrintStream errPrintStream = new PrintStream(stderr);
+ System.setErr(errPrintStream);
+ OutputStream stdout = new ByteArrayOutputStream();
+ PrintStream outPrintStream = new PrintStream(stdout);
+ System.setOut(outPrintStream);
+
+ // Upgrade schema from 0.7.0 to latest
+ execute(new SchemaToolTaskUpgrade(), "-upgradeSchemaFrom 3.1.0");
Review comment:
removed the comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org