You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by GitBox <gi...@apache.org> on 2020/11/03 13:29:56 UTC
[GitHub] [phoenix] virajjasani opened a new pull request #952: PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them
virajjasani opened a new pull request #952:
URL: https://github.com/apache/phoenix/pull/952
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] virajjasani commented on pull request #952: PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #952:
URL: https://github.com/apache/phoenix/pull/952#issuecomment-722348410
Thanks for the review @yanxinyi
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #952: PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #952:
URL: https://github.com/apache/phoenix/pull/952#issuecomment-720378075
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 9s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 14m 9s | master passed |
| +1 :green_heart: | compile | 1m 2s | master passed |
| +1 :green_heart: | checkstyle | 1m 26s | master passed |
| +1 :green_heart: | javadoc | 0m 46s | master passed |
| +0 :ok: | spotbugs | 3m 6s | phoenix-core in master has 969 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 56s | the patch passed |
| +1 :green_heart: | compile | 0m 58s | the patch passed |
| +1 :green_heart: | javac | 0m 58s | the patch passed |
| -1 :x: | checkstyle | 1m 27s | phoenix-core: The patch generated 30 new + 3476 unchanged - 26 fixed = 3506 total (was 3502) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 46s | the patch passed |
| -1 :x: | spotbugs | 3m 23s | phoenix-core generated 1 new + 963 unchanged - 6 fixed = 964 total (was 969) |
||| _ Other Tests _ |
| -1 :x: | unit | 169m 46s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. |
| | | 209m 50s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:phoenix-core |
| | org.apache.phoenix.query.ConnectionQueryServicesImpl.upgradeSystemTask(PhoenixConnection, Map) passes a nonconstant String to an execute or addBatch method on an SQL statement At ConnectionQueryServicesImpl.java:String to an execute or addBatch method on an SQL statement At ConnectionQueryServicesImpl.java:[line 4146] |
| Failed junit tests | phoenix.schema.stats.NamespaceEnabledStatsCollectorIT |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/952 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux bb2d64d25e32 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / e828ef7 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| spotbugs | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/artifact/yetus-general-check/output/new-spotbugs-phoenix-core.html |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/testReport/ |
| Max. process+thread count | 6199 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/1/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #952: PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #952:
URL: https://github.com/apache/phoenix/pull/952#issuecomment-721670011
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 2s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 14m 39s | master passed |
| +1 :green_heart: | compile | 1m 1s | master passed |
| +1 :green_heart: | checkstyle | 1m 27s | master passed |
| +1 :green_heart: | javadoc | 0m 49s | master passed |
| +0 :ok: | spotbugs | 3m 7s | phoenix-core in master has 967 extant spotbugs warnings. |
| -0 :warning: | patch | 3m 14s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 55s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| -1 :x: | checkstyle | 1m 27s | phoenix-core: The patch generated 65 new + 3429 unchanged - 63 fixed = 3494 total (was 3492) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 50s | the patch passed |
| -1 :x: | spotbugs | 3m 31s | phoenix-core generated 1 new + 961 unchanged - 6 fixed = 962 total (was 967) |
||| _ Other Tests _ |
| -1 :x: | unit | 171m 24s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
| | | 217m 6s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:phoenix-core |
| | org.apache.phoenix.query.ConnectionQueryServicesImpl.upgradeSystemTask(PhoenixConnection, Map) passes a nonconstant String to an execute or addBatch method on an SQL statement At ConnectionQueryServicesImpl.java:String to an execute or addBatch method on an SQL statement At ConnectionQueryServicesImpl.java:[line 4134] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/952 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux 41412c74f0e5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / 337d795 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| spotbugs | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/artifact/yetus-general-check/output/new-spotbugs-phoenix-core.html |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/testReport/ |
| Max. process+thread count | 5259 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/3/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] virajjasani commented on pull request #952: PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #952:
URL: https://github.com/apache/phoenix/pull/952#issuecomment-721174388
`NamespaceEnabledStatsCollectorIT` seems flaky, the failure is not relevant to the changes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] yanxinyi commented on a change in pull request #952: PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them
Posted by GitBox <gi...@apache.org>.
yanxinyi commented on a change in pull request #952:
URL: https://github.com/apache/phoenix/pull/952#discussion_r517022108
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
##########
@@ -3821,17 +3823,37 @@ public void upgradeSystemTables(final String url, final Properties props) throws
}
sysCatalogTableName = SchemaUtil.getPhysicalName(SYSTEM_CATALOG_NAME_BYTES, this.getProps()).getNameAsString();
if (SchemaUtil.isNamespaceMappingEnabled(PTableType.SYSTEM, ConnectionQueryServicesImpl.this.getProps())) {
+ String snapshotName = null;
// Try acquiring a lock in SYSMUTEX table before migrating the tables since it involves disabling the table.
if (acquiredMutexLock = acquireUpgradeMutex(MetaDataProtocol.MIN_SYSTEM_TABLE_MIGRATION_TIMESTAMP)) {
LOGGER.debug("Acquired lock in SYSMUTEX table for migrating SYSTEM tables to SYSTEM namespace "
+ "and/or upgrading " + sysCatalogTableName);
+ snapshotName = getSysTableSnapshotName(
+ currentServerSideTableTimeStamp, SYSTEM_CATALOG_NAME);
+ createSnapshot(snapshotName, SYSTEM_CATALOG_NAME);
+ systemTableToSnapshotMap.put(SYSTEM_CATALOG_NAME,
+ snapshotName);
+ LOGGER.debug("Created snapshot for {}", SYSTEM_CATALOG_NAME);
Review comment:
nit: we should change the logging level from debug to info since this rarely happens, and we need this information for sure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] virajjasani commented on pull request #952: PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them
Posted by GitBox <gi...@apache.org>.
virajjasani commented on pull request #952:
URL: https://github.com/apache/phoenix/pull/952#issuecomment-720521681
spotbug warning `passes a nonconstant String to an execute or addBatch method on an SQL statement` almost seem quite difficult to avoid (also not related to main code changes, it was updated due to other spotbug reason) since we need to provide arguments to Alter Table query.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] virajjasani closed pull request #952: PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them
Posted by GitBox <gi...@apache.org>.
virajjasani closed pull request #952:
URL: https://github.com/apache/phoenix/pull/952
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #952: PHOENIX-6086 : Take snapshot of all SYSTEM tables before attempting to upgrade them
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #952:
URL: https://github.com/apache/phoenix/pull/952#issuecomment-721562503
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. |
| -1 :x: | patch | 0m 3s | https://github.com/apache/phoenix/pull/952 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/phoenix/pull/952 |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-952/2/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org