You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by GitBox <gi...@apache.org> on 2020/10/30 04:07:43 UTC
[GitHub] [phoenix] yanxinyi opened a new pull request #950: PHOENIX-5940 Pre-4.15 client cannot connect to 4.15+ server after SYS…
yanxinyi opened a new pull request #950:
URL: https://github.com/apache/phoenix/pull/950
…TEM.CATALOG region has split
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #950: PHOENIX-5940 Pre-4.15 client cannot connect to 4.15+ server after SYS…
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #950:
URL: https://github.com/apache/phoenix/pull/950#issuecomment-719324609
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 18s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ 4.x Compile Tests _ |
| +1 :green_heart: | mvninstall | 10m 45s | 4.x passed |
| +1 :green_heart: | compile | 0m 55s | 4.x passed |
| +1 :green_heart: | checkstyle | 1m 13s | 4.x passed |
| +1 :green_heart: | javadoc | 0m 42s | 4.x passed |
| +0 :ok: | spotbugs | 2m 53s | phoenix-core in 4.x has 953 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 18s | the patch passed |
| +1 :green_heart: | compile | 0m 55s | the patch passed |
| +1 :green_heart: | javac | 0m 55s | the patch passed |
| -1 :x: | checkstyle | 1m 12s | phoenix-core: The patch generated 2 new + 1723 unchanged - 2 fixed = 1725 total (was 1725) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 42s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 1s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 142m 25s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. |
| | | 177m 40s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | phoenix.end2end.index.SaltedIndexIT |
| | phoenix.end2end.join.WhereOptimizerForJoinFiltersIT |
| | phoenix.end2end.index.AsyncIndexDisabledIT |
| | phoenix.end2end.salted.SaltedTableIT |
| | phoenix.end2end.index.txn.MutableRollbackIT |
| | phoenix.end2end.index.AlterIndexIT |
| | phoenix.end2end.index.LocalImmutableNonTxIndexIT |
| | phoenix.end2end.OrphanViewToolIT |
| | phoenix.end2end.index.LocalImmutableTxIndexIT |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/950 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux ccb215169317 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | 4.x / 2aaf2e2 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/1/testReport/ |
| Max. process+thread count | 6859 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/1/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #950: PHOENIX-5940 Pre-4.15 client cannot connect to 4.15+ server after SYS…
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #950:
URL: https://github.com/apache/phoenix/pull/950#issuecomment-723402316
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ 4.x Compile Tests _ |
| +1 :green_heart: | mvninstall | 10m 38s | 4.x passed |
| +1 :green_heart: | compile | 0m 54s | 4.x passed |
| +1 :green_heart: | checkstyle | 1m 12s | 4.x passed |
| +1 :green_heart: | javadoc | 0m 45s | 4.x passed |
| +0 :ok: | spotbugs | 2m 52s | phoenix-core in 4.x has 946 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 13s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| -1 :x: | checkstyle | 1m 13s | phoenix-core: The patch generated 9 new + 1737 unchanged - 3 fixed = 1746 total (was 1740) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 41s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 4s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 223m 48s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 41s | The patch does not generate ASF License warnings. |
| | | 255m 14s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | phoenix.util.IndexScrutinyIT |
| | phoenix.rpc.PhoenixServerRpcIT |
| | phoenix.end2end.PointInTimeQueryIT |
| | phoenix.end2end.OrphanViewToolIT |
| | phoenix.end2end.index.GlobalMutableNonTxIndexIT |
| | phoenix.end2end.index.IndexWithTableSchemaChangeIT |
| | phoenix.end2end.BackwardCompatibilityForSplitableSyscatIT |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/7/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/950 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux 0aebbfcebc0a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | 4.x / 50edcc9 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/7/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/7/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/7/testReport/ |
| Max. process+thread count | 6907 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/7/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] yanxinyi commented on pull request #950: PHOENIX-5940 Pre-4.15 client cannot connect to 4.15+ server after SYS…
Posted by GitBox <gi...@apache.org>.
yanxinyi commented on pull request #950:
URL: https://github.com/apache/phoenix/pull/950#issuecomment-723357709
For additional testing on my local machine, I created more than 2k views, and the syscat split into 2 regions(confirmed by HBase UI page). Ran the select query from 4.14 and current 4.x clients without any issue.
@ChinmaySKulkarni
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #950: PHOENIX-5940 Pre-4.15 client cannot connect to 4.15+ server after SYS…
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #950:
URL: https://github.com/apache/phoenix/pull/950#issuecomment-723379116
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 23s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 1s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ 4.x Compile Tests _ |
| +1 :green_heart: | mvninstall | 11m 14s | 4.x passed |
| +1 :green_heart: | compile | 0m 55s | 4.x passed |
| +1 :green_heart: | checkstyle | 1m 14s | 4.x passed |
| +1 :green_heart: | javadoc | 0m 43s | 4.x passed |
| +0 :ok: | spotbugs | 2m 50s | phoenix-core in 4.x has 946 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 17s | the patch passed |
| +1 :green_heart: | compile | 0m 55s | the patch passed |
| +1 :green_heart: | javac | 0m 55s | the patch passed |
| -1 :x: | checkstyle | 1m 15s | phoenix-core: The patch generated 8 new + 1738 unchanged - 2 fixed = 1746 total (was 1740) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 44s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 4s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 122m 33s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 40s | The patch does not generate ASF License warnings. |
| | | 158m 31s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | phoenix.end2end.index.GlobalImmutableNonTxIndexIT |
| | phoenix.end2end.BackwardCompatibilityForSplitableSyscatIT |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/6/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/950 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux 6fc899b65a12 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | 4.x / 31c9912 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/6/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/6/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/6/testReport/ |
| Max. process+thread count | 6887 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/6/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] yanxinyi commented on a change in pull request #950: PHOENIX-5940 Pre-4.15 client cannot connect to 4.15+ server after SYS…
Posted by GitBox <gi...@apache.org>.
yanxinyi commented on a change in pull request #950:
URL: https://github.com/apache/phoenix/pull/950#discussion_r519060613
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/BackwardCompatibilityTestUtil.java
##########
@@ -0,0 +1,301 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import com.fasterxml.jackson.core.JsonParser;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.curator.shaded.com.google.common.collect.Lists;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.regionserver.ScanInfoUtil;
+import org.apache.hadoop.hbase.util.VersionInfo;
+import org.apache.phoenix.coprocessor.MetaDataProtocol;
+import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
+import org.apache.phoenix.query.QueryConstants;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.util.PropertiesUtil;
+
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.FileWriter;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.net.URL;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.util.List;
+import java.util.Properties;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assume.assumeFalse;
+
+public class BackwardCompatibilityTestUtil {
+ public static final String SQL_DIR = "sql_files/";
+ public static final String RESULTS_AND_GOLD_FILES_DIR = "gold_files/";
+ public static final String COMPATIBLE_CLIENTS_JSON =
+ "compatible_client_versions.json";
+ public static final String BASH = "/bin/bash";
+ public static final String EXECUTE_QUERY_SH = "scripts/execute_query.sh";
+ public static final String QUERY_PREFIX = "query_";
+ public static final String RESULT_PREFIX = "result_";
+ public static final String GOLD_PREFIX = "gold_";
+ public static final String SQL_EXTENSION = ".sql";
+ public static final String TEXT_EXTENSION = ".txt";
+ public static final String CREATE_ADD = "create_add";
+ public static final String CREATE_DIVERGED_VIEW = "create_diverged_view";
+ public static final String ADD_DATA = "add_data";
+ public static final String ADD_DELETE = "add_delete";
+ public static final String QUERY_CREATE_ADD = QUERY_PREFIX + CREATE_ADD;
+ public static final String QUERY_ADD_DATA = QUERY_PREFIX + ADD_DATA;
+ public static final String QUERY_ADD_DELETE = QUERY_PREFIX + ADD_DELETE;
+ public static final String QUERY_CREATE_DIVERGED_VIEW = QUERY_PREFIX + CREATE_DIVERGED_VIEW;
+ public static final String MVN_HOME = "maven.home";
+ public static final String JAVA_TMP_DIR = "java.io.tmpdir";
+
+ public static List<String> computeClientVersions() throws Exception {
+ String hbaseVersion = VersionInfo.getVersion();
+ Pattern p = Pattern.compile("\\d+\\.\\d+");
+ Matcher m = p.matcher(hbaseVersion);
+ String hbaseProfile = null;
+ if (m.find()) {
+ hbaseProfile = m.group();
+ }
+ List<String> clientVersions = Lists.newArrayList();
+ ObjectMapper mapper = new ObjectMapper();
+ mapper.configure(JsonParser.Feature.ALLOW_COMMENTS, true);
+ try (InputStream inputStream = BackwardCompatibilityIT.class
+ .getClassLoader().getResourceAsStream(COMPATIBLE_CLIENTS_JSON)) {
+ assertNotNull(inputStream);
+ JsonNode jsonNode = mapper.readTree(inputStream);
+ JsonNode HBaseProfile = jsonNode.get(hbaseProfile);
+ for (final JsonNode clientVersion : HBaseProfile) {
+ clientVersions.add(clientVersion.textValue() + "-HBase-" + hbaseProfile);
+ }
+ }
+ return clientVersions;
+ }
+
+ // Executes the queries listed in the operation file with a given client version
+ public static void executeQueryWithClientVersion(String clientVersion, String operation,
+ String zkQuorum) throws Exception {
+ List<String> cmdParams = Lists.newArrayList();
+ cmdParams.add(BASH);
+ // Note that auto-commit is true for queries executed via SQLline
+ URL fileUrl = BackwardCompatibilityIT.class.getClassLoader().getResource(EXECUTE_QUERY_SH);
+ assertNotNull(fileUrl);
+ cmdParams.add(new File(fileUrl.getFile()).getAbsolutePath());
+ cmdParams.add(zkQuorum);
+ cmdParams.add(clientVersion);
+
+ fileUrl = BackwardCompatibilityIT.class.getClassLoader()
+ .getResource(SQL_DIR + operation + SQL_EXTENSION);
+ assertNotNull(fileUrl);
+ cmdParams.add(new File(fileUrl.getFile()).getAbsolutePath());
+ fileUrl = BackwardCompatibilityIT.class.getClassLoader().getResource(
+ RESULTS_AND_GOLD_FILES_DIR);
+ assertNotNull(fileUrl);
+ String resultFilePath = new File(fileUrl.getFile()).getAbsolutePath() + "/" +
+ RESULT_PREFIX + operation + TEXT_EXTENSION;
+ cmdParams.add(resultFilePath);
+ cmdParams.add(System.getProperty(JAVA_TMP_DIR));
+
+ if (System.getProperty(MVN_HOME) != null) {
+ cmdParams.add(System.getProperty(MVN_HOME));
+ }
+
+ ProcessBuilder pb = new ProcessBuilder(cmdParams);
+ final Process p = pb.start();
+ final StringBuffer sb = new StringBuffer();
+ //Capture the output stream if any from the execution of the script
+ Thread outputStreamThread = new Thread() {
+ @Override
+ public void run() {
+ try (BufferedReader reader = new BufferedReader(
+ new InputStreamReader(p.getInputStream()))) {
+ String line;
+ while ((line = reader.readLine()) != null) {
+ sb.append(line);
+ }
+ } catch (final Exception e) {
+ sb.append(e.getMessage());
+ }
+ }
+ };
+ outputStreamThread.start();
+ //Capture the error stream if any from the execution of the script
+ Thread errorStreamThread = new Thread() {
+ @Override
+ public void run() {
+ try (BufferedReader reader = new BufferedReader(
+ new InputStreamReader(p.getErrorStream()))) {
+ String line;
+ while ((line = reader.readLine()) != null) {
+ sb.append(line);
+ }
+ } catch (final Exception e) {
+ sb.append(e.getMessage());
+ }
+ }
+ };
+ errorStreamThread.start();
+ p.waitFor();
+ assertEquals(String.format("Executing the query failed%s. Check the result file: %s",
+ sb.length() > 0 ? sb.append(" with : ").toString() : "", resultFilePath),
+ 0, p.exitValue());
+ }
+
+
+ public static void checkForPreConditions(String compatibleClientVersion, Configuration conf) throws Exception {
+ // For the first code cut of any major version, there wouldn't be any backward compatible
+ // clients. Hence the test wouldn't run and just return true when the client
+ // version to be tested is same as current version
+ assumeFalse(compatibleClientVersion.contains(MetaDataProtocol.CURRENT_CLIENT_VERSION));
+ // Make sure that cluster is clean before test execution with no system tables
+ try (org.apache.hadoop.hbase.client.Connection conn =
+ ConnectionFactory.createConnection(conf);
+ Admin admin = conn.getAdmin()) {
+ assertFalse(admin.tableExists(TableName.valueOf(QueryConstants.SYSTEM_SCHEMA_NAME,
+ PhoenixDatabaseMetaData.SYSTEM_CATALOG_TABLE)));
+ }
+ }
+
+ // Saves the result set to a text file to be compared against the gold file for difference
+ private static void saveResultSet(ResultSet rs, BufferedWriter br) throws Exception {
+ ResultSetMetaData rsm = rs.getMetaData();
+ int columnCount = rsm.getColumnCount();
+ StringBuilder row = new StringBuilder(formatStringWithQuotes(rsm.getColumnName(1)));
+ for (int i = 2; i <= columnCount; i++) {
+ row.append(",").append(formatStringWithQuotes(rsm.getColumnName(i)));
+ }
+ br.write(row.toString());
+ br.write("\n");
+ while (rs.next()) {
+ row = new StringBuilder(formatStringWithQuotes(rs.getString(1)));
+ for (int i = 2; i <= columnCount; i++) {
+ row.append(",").append(formatStringWithQuotes(rs.getString(i)));
+ }
+ br.write(row.toString());
+ br.write("\n");
+ }
+ }
+
+ private static String formatStringWithQuotes(String str) {
+ return (str != null) ? String.format("\'%s\'", str) : "\'\'";
+ }
+
+ private static BufferedReader getBufferedReaderForResource(String relativePath)
+ throws FileNotFoundException {
+ URL fileUrl = BackwardCompatibilityTestUtil.class.getClassLoader().getResource(relativePath);
+ assertNotNull(fileUrl);
+ return new BufferedReader(new FileReader(new File(fileUrl.getFile())));
+ }
+
+ // Compares the result file against the gold file to match for the expected output
+ // for the given operation
+ public static void assertExpectedOutput(String result) throws Exception {
+ List<String> resultFile = Lists.newArrayList();
+ List<String> goldFile = Lists.newArrayList();
+ String line;
+ try (BufferedReader resultFileReader = getBufferedReaderForResource(
+ RESULTS_AND_GOLD_FILES_DIR + RESULT_PREFIX + result + TEXT_EXTENSION)) {
+ while ((line = resultFileReader.readLine()) != null) {
+ resultFile.add(line.trim());
+ }
+ }
+ try (BufferedReader goldFileReader = getBufferedReaderForResource(
+ RESULTS_AND_GOLD_FILES_DIR + GOLD_PREFIX + result + TEXT_EXTENSION)) {
+ while ((line = goldFileReader.readLine()) != null) {
+ line = line.trim();
+ if ( !(line.isEmpty() || line.startsWith("*") || line.startsWith("/"))) {
+ goldFile.add(line);
+ }
+ }
+ }
+
+ // We take the first line in gold file and match against the result file to exclude any
+ // other WARNING messages that comes as a result of the query execution
+ int index = resultFile.indexOf(goldFile.get(0));
+ assertNotEquals("Mismatch found between gold file and result file", -1, index);
+ resultFile = resultFile.subList(index, resultFile.size());
+ assertEquals(goldFile, resultFile);
+ }
+
+ // Executes the SQL commands listed in the given operation file from the sql_files directory
+ public static void executeQueriesWithCurrentVersion(String operation, String url,
+ boolean setMaxLookBackAge) throws Exception {
+ Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+ if (setMaxLookBackAge) {
+ props.put(
+ QueryServices.GLOBAL_INDEX_ROW_AGE_THRESHOLD_TO_DELETE_MS_ATTRIB,
+ Long.toString(0));
+ props.put(ScanInfoUtil.PHOENIX_MAX_LOOKBACK_AGE_CONF_KEY,
+ Integer.toString(15));
+ }
+
+ try (Connection conn = DriverManager.getConnection(url, props)) {
+ StringBuilder sb = new StringBuilder();
+ try (BufferedReader reader =
+ getBufferedReaderForResource(SQL_DIR + operation + SQL_EXTENSION)) {
+ String sqlCommand;
+ while ((sqlCommand = reader.readLine()) != null) {
+ sqlCommand = sqlCommand.trim();
+ if (sqlCommand.length() == 0 || sqlCommand.startsWith("/")
+ || sqlCommand.startsWith("*")) {
+ continue;
+ }
+ sb.append(sqlCommand);
+ }
+ }
+ ResultSet rs;
+ String[] sqlCommands = sb.toString().split(";");
+
+ URL fileUrl = BackwardCompatibilityIT.class.getClassLoader().getResource(
+ RESULTS_AND_GOLD_FILES_DIR);
+ assertNotNull(fileUrl);
+ final String resultFile = new File(fileUrl.getFile()).getAbsolutePath() + "/" +
+ RESULT_PREFIX + operation + TEXT_EXTENSION;
+ try (BufferedWriter br = new BufferedWriter(new FileWriter(resultFile))) {
+ for (String command : sqlCommands) {
+ try (PreparedStatement stmt = conn.prepareStatement(command)) {
+ stmt.execute();
+ rs = stmt.getResultSet();
+ if (rs != null) {
+ saveResultSet(rs, br);
+ }
+ }
+ conn.commit();
+ }
+ }
+ }
+ }
+}
Review comment:
Yes, no logic changes. The only diff is adding extra param for zkQuorm or URL for the execute script.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] virajjasani commented on a change in pull request #950: PHOENIX-5940 Pre-4.15 client cannot connect to 4.15+ server after SYS…
Posted by GitBox <gi...@apache.org>.
virajjasani commented on a change in pull request #950:
URL: https://github.com/apache/phoenix/pull/950#discussion_r514890776
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java
##########
@@ -3295,11 +3295,14 @@ public void getVersion(RpcController controller, GetVersionRequest request, RpcC
PhoenixDatabaseMetaData.SYSTEM_CATALOG_TABLE_BYTES, HConstants.LATEST_TIMESTAMP, null,
request.getClientVersion());
} catch (Throwable t) {
- LOGGER.error("loading system catalog table inside getVersion failed", t);
- ProtobufUtil.setControllerException(controller,
- ServerUtil.createIOException(
- SchemaUtil.getPhysicalTableName(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES,
- isTablesMappingEnabled).toString(), t));
+ if (request.getClientVersion() >= MIN_SPLITTABLE_SYSTEM_CATALOG) {
+ LOGGER.error("loading system catalog table inside getVersion failed", t);
+ ProtobufUtil.setControllerException(controller,
+ ServerUtil.createIOException(
+ SchemaUtil.getPhysicalTableName(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES,
+ isTablesMappingEnabled).toString(), t));
+ }
Review comment:
For else part, would it be better to just provide INFO log with client version and exact Exception that we are ignoring?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #950: PHOENIX-5940 Pre-4.15 client cannot connect to 4.15+ server after SYS…
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #950:
URL: https://github.com/apache/phoenix/pull/950#issuecomment-723691221
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ 4.x Compile Tests _ |
| +1 :green_heart: | mvninstall | 10m 41s | 4.x passed |
| +1 :green_heart: | compile | 0m 56s | 4.x passed |
| +1 :green_heart: | checkstyle | 1m 12s | 4.x passed |
| +1 :green_heart: | javadoc | 0m 45s | 4.x passed |
| +0 :ok: | spotbugs | 2m 52s | phoenix-core in 4.x has 946 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 21s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| -1 :x: | checkstyle | 1m 15s | phoenix-core: The patch generated 8 new + 1738 unchanged - 2 fixed = 1746 total (was 1740) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 42s | the patch passed |
| +1 :green_heart: | spotbugs | 2m 59s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 124m 54s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. |
| | | 156m 28s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/9/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/950 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux 411fda1912db 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | 4.x / 50edcc9 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/9/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/9/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/9/testReport/ |
| Max. process+thread count | 6920 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-950/9/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] yanxinyi closed pull request #950: PHOENIX-5940 Pre-4.15 client cannot connect to 4.15+ server after SYS…
Posted by GitBox <gi...@apache.org>.
yanxinyi closed pull request #950:
URL: https://github.com/apache/phoenix/pull/950
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] yanxinyi commented on a change in pull request #950: PHOENIX-5940 Pre-4.15 client cannot connect to 4.15+ server after SYS…
Posted by GitBox <gi...@apache.org>.
yanxinyi commented on a change in pull request #950:
URL: https://github.com/apache/phoenix/pull/950#discussion_r519063248
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/BackwardCompatibilityTestUtil.java
##########
@@ -0,0 +1,301 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import com.fasterxml.jackson.core.JsonParser;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.curator.shaded.com.google.common.collect.Lists;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.regionserver.ScanInfoUtil;
+import org.apache.hadoop.hbase.util.VersionInfo;
+import org.apache.phoenix.coprocessor.MetaDataProtocol;
+import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
+import org.apache.phoenix.query.QueryConstants;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.util.PropertiesUtil;
+
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.FileWriter;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.net.URL;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.util.List;
+import java.util.Properties;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assume.assumeFalse;
+
+public class BackwardCompatibilityTestUtil {
+ public static final String SQL_DIR = "sql_files/";
+ public static final String RESULTS_AND_GOLD_FILES_DIR = "gold_files/";
+ public static final String COMPATIBLE_CLIENTS_JSON =
+ "compatible_client_versions.json";
+ public static final String BASH = "/bin/bash";
+ public static final String EXECUTE_QUERY_SH = "scripts/execute_query.sh";
+ public static final String QUERY_PREFIX = "query_";
+ public static final String RESULT_PREFIX = "result_";
+ public static final String GOLD_PREFIX = "gold_";
+ public static final String SQL_EXTENSION = ".sql";
+ public static final String TEXT_EXTENSION = ".txt";
+ public static final String CREATE_ADD = "create_add";
+ public static final String CREATE_DIVERGED_VIEW = "create_diverged_view";
+ public static final String ADD_DATA = "add_data";
+ public static final String ADD_DELETE = "add_delete";
+ public static final String QUERY_CREATE_ADD = QUERY_PREFIX + CREATE_ADD;
+ public static final String QUERY_ADD_DATA = QUERY_PREFIX + ADD_DATA;
+ public static final String QUERY_ADD_DELETE = QUERY_PREFIX + ADD_DELETE;
+ public static final String QUERY_CREATE_DIVERGED_VIEW = QUERY_PREFIX + CREATE_DIVERGED_VIEW;
+ public static final String MVN_HOME = "maven.home";
+ public static final String JAVA_TMP_DIR = "java.io.tmpdir";
+
Review comment:
added
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ChinmaySKulkarni commented on a change in pull request #950: PHOENIX-5940 Pre-4.15 client cannot connect to 4.15+ server after SYS…
Posted by GitBox <gi...@apache.org>.
ChinmaySKulkarni commented on a change in pull request #950:
URL: https://github.com/apache/phoenix/pull/950#discussion_r519039654
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/BackwardCompatibilityForSplitableSyscatIT.java
##########
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import com.google.common.collect.Maps;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.query.QueryServicesOptions;
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.ReadOnlyProps;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.util.Collection;
+import java.util.Map;
+
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.ADD_DELETE;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.CREATE_ADD;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.QUERY_ADD_DELETE;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.QUERY_CREATE_ADD;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.assertExpectedOutput;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.computeClientVersions;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.executeQueriesWithCurrentVersion;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.executeQueryWithClientVersion;
+
+/**
+ * This class is meant for testing all compatible client versions
+ * against the current server version. It runs SQL queries with given
+ * client versions and compares the output against gold files
+ */
+
+@RunWith(Parameterized.class)
+@Category(NeedsOwnMiniClusterTest.class)
+public class BackwardCompatibilityForSplitableSyscatIT extends SplitSystemCatalogIT {
+ protected final String compatibleClientVersion;
+ protected static String zkQuorum;
+
+ @Parameterized.Parameters(name = "BackwardCompatibilityIT_compatibleClientVersion={0}")
+ public static synchronized Collection<String> data() throws Exception {
+ return computeClientVersions();
+ }
+
+ public BackwardCompatibilityForSplitableSyscatIT(String compatibleClientVersion) {
+ this.compatibleClientVersion = compatibleClientVersion;
+ }
+
+ @BeforeClass
+ public static synchronized void doSetup() throws Exception {
+ NUM_SLAVES_BASE = 6;
Review comment:
Can't we just use the `doSetup()` from `SplitSystemCatalogIT` here to avoid code duplication?
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/BackwardCompatibilityForSplitableSyscatIT.java
##########
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import com.google.common.collect.Maps;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.query.QueryServicesOptions;
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.ReadOnlyProps;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.util.Collection;
+import java.util.Map;
+
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.ADD_DELETE;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.CREATE_ADD;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.QUERY_ADD_DELETE;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.QUERY_CREATE_ADD;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.assertExpectedOutput;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.computeClientVersions;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.executeQueriesWithCurrentVersion;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.executeQueryWithClientVersion;
+
+/**
+ * This class is meant for testing all compatible client versions
+ * against the current server version. It runs SQL queries with given
+ * client versions and compares the output against gold files
Review comment:
Modify comment to add that it is specifically for testing splittable syscat stuff
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java
##########
@@ -3357,11 +3357,28 @@ public void getVersion(RpcController controller, GetVersionRequest request, RpcC
PhoenixDatabaseMetaData.SYSTEM_CATALOG_TABLE_BYTES, HConstants.LATEST_TIMESTAMP, null,
request.getClientVersion());
} catch (Throwable t) {
- LOGGER.error("loading system catalog table inside getVersion failed", t);
- ProtobufUtil.setControllerException(controller,
- ServerUtil.createIOException(
- SchemaUtil.getPhysicalTableName(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES,
- isTablesMappingEnabled).toString(), t));
+ boolean isErrorSwallowed = false;
+ if (t instanceof SQLException &&
+ ((SQLException) t).getErrorCode() == SQLExceptionCode.GET_TABLE_ERROR.getErrorCode()) {
+ Region region = env.getRegion();
+ final byte[] key = SchemaUtil.getTableKey(
+ ByteUtil.EMPTY_BYTE_ARRAY,
+ PhoenixDatabaseMetaData.SYSTEM_CATALOG_SCHEMA_BYTES,
+ PhoenixDatabaseMetaData.SYSTEM_CATALOG_TABLE_BYTES);
+ if (!region.getRegionInfo().containsRow(key) &&
+ request.getClientVersion() < MIN_SPLITTABLE_SYSTEM_CATALOG) {
+ LOGGER.debug("The pre4.15 client is trying to get syscat region that contains head row");
Review comment:
nit: Change to "pre-4.15" and "SYSTEM.CATALOG" instead of "syscat"
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/BackwardCompatibilityTestUtil.java
##########
@@ -0,0 +1,301 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import com.fasterxml.jackson.core.JsonParser;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.curator.shaded.com.google.common.collect.Lists;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.regionserver.ScanInfoUtil;
+import org.apache.hadoop.hbase.util.VersionInfo;
+import org.apache.phoenix.coprocessor.MetaDataProtocol;
+import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
+import org.apache.phoenix.query.QueryConstants;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.util.PropertiesUtil;
+
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.FileWriter;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.net.URL;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.util.List;
+import java.util.Properties;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assume.assumeFalse;
+
+public class BackwardCompatibilityTestUtil {
+ public static final String SQL_DIR = "sql_files/";
+ public static final String RESULTS_AND_GOLD_FILES_DIR = "gold_files/";
+ public static final String COMPATIBLE_CLIENTS_JSON =
+ "compatible_client_versions.json";
+ public static final String BASH = "/bin/bash";
+ public static final String EXECUTE_QUERY_SH = "scripts/execute_query.sh";
+ public static final String QUERY_PREFIX = "query_";
+ public static final String RESULT_PREFIX = "result_";
+ public static final String GOLD_PREFIX = "gold_";
+ public static final String SQL_EXTENSION = ".sql";
+ public static final String TEXT_EXTENSION = ".txt";
+ public static final String CREATE_ADD = "create_add";
+ public static final String CREATE_DIVERGED_VIEW = "create_diverged_view";
+ public static final String ADD_DATA = "add_data";
+ public static final String ADD_DELETE = "add_delete";
+ public static final String QUERY_CREATE_ADD = QUERY_PREFIX + CREATE_ADD;
+ public static final String QUERY_ADD_DATA = QUERY_PREFIX + ADD_DATA;
+ public static final String QUERY_ADD_DELETE = QUERY_PREFIX + ADD_DELETE;
+ public static final String QUERY_CREATE_DIVERGED_VIEW = QUERY_PREFIX + CREATE_DIVERGED_VIEW;
+ public static final String MVN_HOME = "maven.home";
+ public static final String JAVA_TMP_DIR = "java.io.tmpdir";
+
+ public static List<String> computeClientVersions() throws Exception {
+ String hbaseVersion = VersionInfo.getVersion();
+ Pattern p = Pattern.compile("\\d+\\.\\d+");
+ Matcher m = p.matcher(hbaseVersion);
+ String hbaseProfile = null;
+ if (m.find()) {
+ hbaseProfile = m.group();
+ }
+ List<String> clientVersions = Lists.newArrayList();
+ ObjectMapper mapper = new ObjectMapper();
+ mapper.configure(JsonParser.Feature.ALLOW_COMMENTS, true);
+ try (InputStream inputStream = BackwardCompatibilityIT.class
+ .getClassLoader().getResourceAsStream(COMPATIBLE_CLIENTS_JSON)) {
+ assertNotNull(inputStream);
+ JsonNode jsonNode = mapper.readTree(inputStream);
+ JsonNode HBaseProfile = jsonNode.get(hbaseProfile);
+ for (final JsonNode clientVersion : HBaseProfile) {
+ clientVersions.add(clientVersion.textValue() + "-HBase-" + hbaseProfile);
+ }
+ }
+ return clientVersions;
+ }
+
+ // Executes the queries listed in the operation file with a given client version
+ public static void executeQueryWithClientVersion(String clientVersion, String operation,
+ String zkQuorum) throws Exception {
+ List<String> cmdParams = Lists.newArrayList();
+ cmdParams.add(BASH);
+ // Note that auto-commit is true for queries executed via SQLline
+ URL fileUrl = BackwardCompatibilityIT.class.getClassLoader().getResource(EXECUTE_QUERY_SH);
+ assertNotNull(fileUrl);
+ cmdParams.add(new File(fileUrl.getFile()).getAbsolutePath());
+ cmdParams.add(zkQuorum);
+ cmdParams.add(clientVersion);
+
+ fileUrl = BackwardCompatibilityIT.class.getClassLoader()
+ .getResource(SQL_DIR + operation + SQL_EXTENSION);
+ assertNotNull(fileUrl);
+ cmdParams.add(new File(fileUrl.getFile()).getAbsolutePath());
+ fileUrl = BackwardCompatibilityIT.class.getClassLoader().getResource(
+ RESULTS_AND_GOLD_FILES_DIR);
+ assertNotNull(fileUrl);
+ String resultFilePath = new File(fileUrl.getFile()).getAbsolutePath() + "/" +
+ RESULT_PREFIX + operation + TEXT_EXTENSION;
+ cmdParams.add(resultFilePath);
+ cmdParams.add(System.getProperty(JAVA_TMP_DIR));
+
+ if (System.getProperty(MVN_HOME) != null) {
+ cmdParams.add(System.getProperty(MVN_HOME));
+ }
+
+ ProcessBuilder pb = new ProcessBuilder(cmdParams);
+ final Process p = pb.start();
+ final StringBuffer sb = new StringBuffer();
+ //Capture the output stream if any from the execution of the script
+ Thread outputStreamThread = new Thread() {
+ @Override
+ public void run() {
+ try (BufferedReader reader = new BufferedReader(
+ new InputStreamReader(p.getInputStream()))) {
+ String line;
+ while ((line = reader.readLine()) != null) {
+ sb.append(line);
+ }
+ } catch (final Exception e) {
+ sb.append(e.getMessage());
+ }
+ }
+ };
+ outputStreamThread.start();
+ //Capture the error stream if any from the execution of the script
+ Thread errorStreamThread = new Thread() {
+ @Override
+ public void run() {
+ try (BufferedReader reader = new BufferedReader(
+ new InputStreamReader(p.getErrorStream()))) {
+ String line;
+ while ((line = reader.readLine()) != null) {
+ sb.append(line);
+ }
+ } catch (final Exception e) {
+ sb.append(e.getMessage());
+ }
+ }
+ };
+ errorStreamThread.start();
+ p.waitFor();
+ assertEquals(String.format("Executing the query failed%s. Check the result file: %s",
+ sb.length() > 0 ? sb.append(" with : ").toString() : "", resultFilePath),
+ 0, p.exitValue());
+ }
+
+
+ public static void checkForPreConditions(String compatibleClientVersion, Configuration conf) throws Exception {
+ // For the first code cut of any major version, there wouldn't be any backward compatible
+ // clients. Hence the test wouldn't run and just return true when the client
+ // version to be tested is same as current version
+ assumeFalse(compatibleClientVersion.contains(MetaDataProtocol.CURRENT_CLIENT_VERSION));
+ // Make sure that cluster is clean before test execution with no system tables
+ try (org.apache.hadoop.hbase.client.Connection conn =
+ ConnectionFactory.createConnection(conf);
+ Admin admin = conn.getAdmin()) {
+ assertFalse(admin.tableExists(TableName.valueOf(QueryConstants.SYSTEM_SCHEMA_NAME,
+ PhoenixDatabaseMetaData.SYSTEM_CATALOG_TABLE)));
+ }
+ }
+
+ // Saves the result set to a text file to be compared against the gold file for difference
+ private static void saveResultSet(ResultSet rs, BufferedWriter br) throws Exception {
+ ResultSetMetaData rsm = rs.getMetaData();
+ int columnCount = rsm.getColumnCount();
+ StringBuilder row = new StringBuilder(formatStringWithQuotes(rsm.getColumnName(1)));
+ for (int i = 2; i <= columnCount; i++) {
+ row.append(",").append(formatStringWithQuotes(rsm.getColumnName(i)));
+ }
+ br.write(row.toString());
+ br.write("\n");
+ while (rs.next()) {
+ row = new StringBuilder(formatStringWithQuotes(rs.getString(1)));
+ for (int i = 2; i <= columnCount; i++) {
+ row.append(",").append(formatStringWithQuotes(rs.getString(i)));
+ }
+ br.write(row.toString());
+ br.write("\n");
+ }
+ }
+
+ private static String formatStringWithQuotes(String str) {
+ return (str != null) ? String.format("\'%s\'", str) : "\'\'";
+ }
+
+ private static BufferedReader getBufferedReaderForResource(String relativePath)
+ throws FileNotFoundException {
+ URL fileUrl = BackwardCompatibilityTestUtil.class.getClassLoader().getResource(relativePath);
+ assertNotNull(fileUrl);
+ return new BufferedReader(new FileReader(new File(fileUrl.getFile())));
+ }
+
+ // Compares the result file against the gold file to match for the expected output
+ // for the given operation
+ public static void assertExpectedOutput(String result) throws Exception {
+ List<String> resultFile = Lists.newArrayList();
+ List<String> goldFile = Lists.newArrayList();
+ String line;
+ try (BufferedReader resultFileReader = getBufferedReaderForResource(
+ RESULTS_AND_GOLD_FILES_DIR + RESULT_PREFIX + result + TEXT_EXTENSION)) {
+ while ((line = resultFileReader.readLine()) != null) {
+ resultFile.add(line.trim());
+ }
+ }
+ try (BufferedReader goldFileReader = getBufferedReaderForResource(
+ RESULTS_AND_GOLD_FILES_DIR + GOLD_PREFIX + result + TEXT_EXTENSION)) {
+ while ((line = goldFileReader.readLine()) != null) {
+ line = line.trim();
+ if ( !(line.isEmpty() || line.startsWith("*") || line.startsWith("/"))) {
+ goldFile.add(line);
+ }
+ }
+ }
+
+ // We take the first line in gold file and match against the result file to exclude any
+ // other WARNING messages that comes as a result of the query execution
+ int index = resultFile.indexOf(goldFile.get(0));
+ assertNotEquals("Mismatch found between gold file and result file", -1, index);
+ resultFile = resultFile.subList(index, resultFile.size());
+ assertEquals(goldFile, resultFile);
+ }
+
+ // Executes the SQL commands listed in the given operation file from the sql_files directory
+ public static void executeQueriesWithCurrentVersion(String operation, String url,
+ boolean setMaxLookBackAge) throws Exception {
+ Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
+ if (setMaxLookBackAge) {
+ props.put(
+ QueryServices.GLOBAL_INDEX_ROW_AGE_THRESHOLD_TO_DELETE_MS_ATTRIB,
+ Long.toString(0));
+ props.put(ScanInfoUtil.PHOENIX_MAX_LOOKBACK_AGE_CONF_KEY,
+ Integer.toString(15));
+ }
+
+ try (Connection conn = DriverManager.getConnection(url, props)) {
+ StringBuilder sb = new StringBuilder();
+ try (BufferedReader reader =
+ getBufferedReaderForResource(SQL_DIR + operation + SQL_EXTENSION)) {
+ String sqlCommand;
+ while ((sqlCommand = reader.readLine()) != null) {
+ sqlCommand = sqlCommand.trim();
+ if (sqlCommand.length() == 0 || sqlCommand.startsWith("/")
+ || sqlCommand.startsWith("*")) {
+ continue;
+ }
+ sb.append(sqlCommand);
+ }
+ }
+ ResultSet rs;
+ String[] sqlCommands = sb.toString().split(";");
+
+ URL fileUrl = BackwardCompatibilityIT.class.getClassLoader().getResource(
+ RESULTS_AND_GOLD_FILES_DIR);
+ assertNotNull(fileUrl);
+ final String resultFile = new File(fileUrl.getFile()).getAbsolutePath() + "/" +
+ RESULT_PREFIX + operation + TEXT_EXTENSION;
+ try (BufferedWriter br = new BufferedWriter(new FileWriter(resultFile))) {
+ for (String command : sqlCommands) {
+ try (PreparedStatement stmt = conn.prepareStatement(command)) {
+ stmt.execute();
+ rs = stmt.getResultSet();
+ if (rs != null) {
+ saveResultSet(rs, br);
+ }
+ }
+ conn.commit();
+ }
+ }
+ }
+ }
+}
Review comment:
@yanxinyi my assumption is there is no _real_ diff in this class and is just a bunch of stuff copied from `BackwardCompatibilityIT` right?
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/BackwardCompatibilityTestUtil.java
##########
@@ -0,0 +1,301 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import com.fasterxml.jackson.core.JsonParser;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.curator.shaded.com.google.common.collect.Lists;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.regionserver.ScanInfoUtil;
+import org.apache.hadoop.hbase.util.VersionInfo;
+import org.apache.phoenix.coprocessor.MetaDataProtocol;
+import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
+import org.apache.phoenix.query.QueryConstants;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.util.PropertiesUtil;
+
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.FileWriter;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.net.URL;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.util.List;
+import java.util.Properties;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assume.assumeFalse;
+
+public class BackwardCompatibilityTestUtil {
Review comment:
nit: Make it `final` to prevent sub-classing the utility class
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/BackwardCompatibilityForSplitableSyscatIT.java
##########
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import com.google.common.collect.Maps;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.query.QueryServicesOptions;
+import org.apache.phoenix.util.PhoenixRuntime;
+import org.apache.phoenix.util.ReadOnlyProps;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.util.Collection;
+import java.util.Map;
+
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.ADD_DELETE;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.CREATE_ADD;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.QUERY_ADD_DELETE;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.QUERY_CREATE_ADD;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.assertExpectedOutput;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.computeClientVersions;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.executeQueriesWithCurrentVersion;
+import static org.apache.phoenix.end2end.BackwardCompatibilityTestUtil.executeQueryWithClientVersion;
+
+/**
+ * This class is meant for testing all compatible client versions
+ * against the current server version. It runs SQL queries with given
+ * client versions and compares the output against gold files
+ */
+
+@RunWith(Parameterized.class)
+@Category(NeedsOwnMiniClusterTest.class)
+public class BackwardCompatibilityForSplitableSyscatIT extends SplitSystemCatalogIT {
+ protected final String compatibleClientVersion;
+ protected static String zkQuorum;
+
+ @Parameterized.Parameters(name = "BackwardCompatibilityIT_compatibleClientVersion={0}")
+ public static synchronized Collection<String> data() throws Exception {
+ return computeClientVersions();
+ }
+
+ public BackwardCompatibilityForSplitableSyscatIT(String compatibleClientVersion) {
+ this.compatibleClientVersion = compatibleClientVersion;
+ }
+
+ @BeforeClass
+ public static synchronized void doSetup() throws Exception {
+ NUM_SLAVES_BASE = 6;
+ boolean splitSystemCatalog = (driver == null);
+ Map<String, String> serverProps = Maps.newHashMapWithExpectedSize(1);
+ config.set(QueryServices.EXTRA_JDBC_ARGUMENTS_ATTRIB, QueryServicesOptions.DEFAULT_EXTRA_JDBC_ARGUMENTS);
+ setUpTestDriver(new ReadOnlyProps(serverProps.entrySet().iterator()),
+ ReadOnlyProps.EMPTY_PROPS);
+ // Split SYSTEM.CATALOG once after the mini-cluster is started
+ if (splitSystemCatalog) {
+ // splitSystemCatalog is incompatible with the balancer chore
+ getUtility().getHBaseCluster().getMaster().balanceSwitch(false);
+ splitSystemCatalog();
+ }
+ zkQuorum = "localhost:" + getZKClientPort(config);
+ url = PhoenixRuntime.JDBC_PROTOCOL + PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR + zkQuorum;
Review comment:
Not sure you need to do this. Instead I think we can just use the `getUrl()` API in your tests
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/BackwardCompatibilityTestUtil.java
##########
@@ -0,0 +1,301 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import com.fasterxml.jackson.core.JsonParser;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.curator.shaded.com.google.common.collect.Lists;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.regionserver.ScanInfoUtil;
+import org.apache.hadoop.hbase.util.VersionInfo;
+import org.apache.phoenix.coprocessor.MetaDataProtocol;
+import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData;
+import org.apache.phoenix.query.QueryConstants;
+import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.util.PropertiesUtil;
+
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.FileWriter;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.net.URL;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.ResultSetMetaData;
+import java.util.List;
+import java.util.Properties;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assume.assumeFalse;
+
+public class BackwardCompatibilityTestUtil {
+ public static final String SQL_DIR = "sql_files/";
+ public static final String RESULTS_AND_GOLD_FILES_DIR = "gold_files/";
+ public static final String COMPATIBLE_CLIENTS_JSON =
+ "compatible_client_versions.json";
+ public static final String BASH = "/bin/bash";
+ public static final String EXECUTE_QUERY_SH = "scripts/execute_query.sh";
+ public static final String QUERY_PREFIX = "query_";
+ public static final String RESULT_PREFIX = "result_";
+ public static final String GOLD_PREFIX = "gold_";
+ public static final String SQL_EXTENSION = ".sql";
+ public static final String TEXT_EXTENSION = ".txt";
+ public static final String CREATE_ADD = "create_add";
+ public static final String CREATE_DIVERGED_VIEW = "create_diverged_view";
+ public static final String ADD_DATA = "add_data";
+ public static final String ADD_DELETE = "add_delete";
+ public static final String QUERY_CREATE_ADD = QUERY_PREFIX + CREATE_ADD;
+ public static final String QUERY_ADD_DATA = QUERY_PREFIX + ADD_DATA;
+ public static final String QUERY_ADD_DELETE = QUERY_PREFIX + ADD_DELETE;
+ public static final String QUERY_CREATE_DIVERGED_VIEW = QUERY_PREFIX + CREATE_DIVERGED_VIEW;
+ public static final String MVN_HOME = "maven.home";
+ public static final String JAVA_TMP_DIR = "java.io.tmpdir";
+
Review comment:
nit: Add a private constructor to prevent instantiating this class from outside.
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/BackwardCompatibilityIT.java
##########
@@ -86,33 +66,12 @@
@Category(NeedsOwnMiniClusterTest.class)
public class BackwardCompatibilityIT {
- private static final String SQL_DIR = "sql_files/";
- private static final String RESULTS_AND_GOLD_FILES_DIR = "gold_files/";
- private static final String COMPATIBLE_CLIENTS_JSON =
- "compatible_client_versions.json";
- private static final String BASH = "/bin/bash";
- private static final String EXECUTE_QUERY_SH = "scripts/execute_query.sh";
- private static final String QUERY_PREFIX = "query_";
- private static final String RESULT_PREFIX = "result_";
- private static final String GOLD_PREFIX = "gold_";
- private static final String SQL_EXTENSION = ".sql";
- private static final String TEXT_EXTENSION = ".txt";
- private static final String CREATE_ADD = "create_add";
- private static final String CREATE_DIVERGED_VIEW = "create_diverged_view";
- private static final String ADD_DATA = "add_data";
- private static final String ADD_DELETE = "add_delete";
- private static final String QUERY_CREATE_ADD = QUERY_PREFIX + CREATE_ADD;
- private static final String QUERY_ADD_DATA = QUERY_PREFIX + ADD_DATA;
- private static final String QUERY_ADD_DELETE = QUERY_PREFIX + ADD_DELETE;
- private static final String QUERY_CREATE_DIVERGED_VIEW = QUERY_PREFIX + CREATE_DIVERGED_VIEW;
- private static final String MVN_HOME = "maven.home";
- private static final String JAVA_TMP_DIR = "java.io.tmpdir";
+ protected final String compatibleClientVersion;
+ protected static Configuration conf;
+ protected static HBaseTestingUtility hbaseTestUtil;
+ protected static String zkQuorum;
+ protected static String url;
Review comment:
nit: Make package-protected instead. Also, some of these diffs may not be needed if you're just going to use the `doSetup()` of SplitSystemCatalogIT
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org