You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by GitBox <gi...@apache.org> on 2020/10/28 12:44:36 UTC
[GitHub] [phoenix] virajjasani opened a new pull request #943: PHOENIX-6087 : Resolved Phoenix Connection leak in UpgradeUtil.addViewIndexToParentLinks() and Admin leak in UpgradeUtil.syncTableAndIndexProperties()
virajjasani opened a new pull request #943:
URL: https://github.com/apache/phoenix/pull/943
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #943: PHOENIX-6087 : Resolve Phoenix Connection leak in UpgradeUtil.addViewIndexToParentLinks() and Admin leak in UpgradeUtil.syncTableAndIndexProperties()
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #943:
URL: https://github.com/apache/phoenix/pull/943#issuecomment-718161449
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 14m 32s | master passed |
| +1 :green_heart: | compile | 1m 2s | master passed |
| +1 :green_heart: | checkstyle | 1m 28s | master passed |
| +1 :green_heart: | javadoc | 0m 48s | master passed |
| +0 :ok: | spotbugs | 3m 12s | phoenix-core in master has 969 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 9m 10s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| -1 :x: | checkstyle | 1m 28s | phoenix-core: The patch generated 6 new + 3497 unchanged - 30 fixed = 3503 total (was 3527) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 45s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 19s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 156m 58s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. |
| | | 197m 41s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/943 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux b1169786098f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / 6d84d0f |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/2/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/2/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/2/testReport/ |
| Max. process+thread count | 6158 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/2/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #943: PHOENIX-6087 : Resolve Phoenix Connection leak in UpgradeUtil.addViewIndexToParentLinks() and Admin leak in UpgradeUtil.syncTableAndIndexProperties()
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #943:
URL: https://github.com/apache/phoenix/pull/943#issuecomment-719461621
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 47s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 14m 28s | master passed |
| +1 :green_heart: | compile | 1m 4s | master passed |
| +1 :green_heart: | checkstyle | 1m 28s | master passed |
| +1 :green_heart: | javadoc | 0m 48s | master passed |
| +0 :ok: | spotbugs | 3m 10s | phoenix-core in master has 969 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 9m 10s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| -1 :x: | checkstyle | 1m 28s | phoenix-core: The patch generated 43 new + 3460 unchanged - 67 fixed = 3503 total (was 3527) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 46s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 19s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 166m 12s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. |
| | | 212m 28s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | phoenix.end2end.join.SubqueryUsingSortMergeJoinIT |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/943 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux e0cb7cd78afa 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / ff46e35 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/3/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/3/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/3/testReport/ |
| Max. process+thread count | 6365 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/3/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] yanxinyi commented on a change in pull request #943: PHOENIX-6087 : Resolve Phoenix Connection leak in UpgradeUtil.addViewIndexToParentLinks() and Admin leak in UpgradeUtil.syncTableAndIndexProperties()
Posted by GitBox <gi...@apache.org>.
yanxinyi commented on a change in pull request #943:
URL: https://github.com/apache/phoenix/pull/943#discussion_r514726107
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/util/UpgradeUtil.java
##########
@@ -1211,36 +1213,56 @@ public static void addViewIndexToParentLinks(PhoenixConnection oldMetaConnection
String createViewIndexLink = "UPSERT INTO SYSTEM.CATALOG (TENANT_ID, TABLE_SCHEM, TABLE_NAME, COLUMN_FAMILY, LINK_TYPE) VALUES (?,?,?,?,?) ";
ResultSet rs = queryConn.createStatement().executeQuery(indexQuery);
String prevTenantId = null;
- PhoenixConnection metaConn = queryConn;
+ metaConn = queryConn;
Properties props = new Properties(queryConn.getClientInfo());
- props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(HConstants.LATEST_TIMESTAMP));
+ props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB,
+ Long.toString(HConstants.LATEST_TIMESTAMP));
while (rs.next()) {
- String tenantId = rs.getString("TENANT_ID");
- if (!java.util.Objects.equals(prevTenantId, tenantId)) {
- prevTenantId = tenantId;
- props.setProperty(PhoenixRuntime.TENANT_ID_ATTRIB, tenantId);
- metaConn = new PhoenixConnection(oldMetaConnection, props);
- }
- String schemaName = rs.getString("TABLE_SCHEM");
- String parentTableName = rs.getString("TABLE_NAME");
- String fullParentTableName = SchemaUtil.getTableName(schemaName, parentTableName);
- String indexName = rs.getString("COLUMN_FAMILY");
- PTable table = PhoenixRuntime.getTable(metaConn, fullParentTableName);
- if (table==null) {
- throw new TableNotFoundException(fullParentTableName);
- }
- if (table.getType().equals(PTableType.VIEW)) {
- PreparedStatement prepareStatement = upsertConn.prepareStatement(createViewIndexLink);
- prepareStatement.setString(1, tenantId);
- prepareStatement.setString(2, schemaName);
- prepareStatement.setString(3, indexName);
- prepareStatement.setString(4, parentTableName);
- prepareStatement.setByte(5, LinkType.VIEW_INDEX_PARENT_TABLE.getSerializedValue());
- prepareStatement.execute();
- upsertConn.commit();
- }
+ String tenantId = rs.getString("TENANT_ID");
+ if (!java.util.Objects.equals(prevTenantId, tenantId)) {
+ prevTenantId = tenantId;
+ props.setProperty(PhoenixRuntime.TENANT_ID_ATTRIB, tenantId);
+ // guard again queryConn because we don't want to close
+ // queryConn if metaConn was assigned queryConn at
+ // this point
+ if (!isMetaConnUsingQueryConn) {
+ metaConn.close();
+ }
+ metaConn = new PhoenixConnection(oldMetaConnection, props);
+ // now that we have reassigned metaConn, make
+ // isMetaConnUsingQueryConn false so that we make
+ // metaConn eligible for closure
+ isMetaConnUsingQueryConn = false;
+ }
+ String schemaName = rs.getString("TABLE_SCHEM");
Review comment:
please using `PhoenixDatabaseMetaData.TABLE_SCHEM` constant
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #943: PHOENIX-6087 : Resolve Phoenix Connection leak in UpgradeUtil.addViewIndexToParentLinks() and Admin leak in UpgradeUtil.syncTableAndIndexProperties()
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #943:
URL: https://github.com/apache/phoenix/pull/943#issuecomment-718036418
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 14m 12s | master passed |
| +1 :green_heart: | compile | 1m 3s | master passed |
| +1 :green_heart: | checkstyle | 1m 27s | master passed |
| +1 :green_heart: | javadoc | 0m 47s | master passed |
| +0 :ok: | spotbugs | 3m 11s | phoenix-core in master has 969 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 52s | the patch passed |
| +1 :green_heart: | compile | 0m 58s | the patch passed |
| +1 :green_heart: | javac | 0m 58s | the patch passed |
| -1 :x: | checkstyle | 1m 29s | phoenix-core: The patch generated 44 new + 3460 unchanged - 67 fixed = 3504 total (was 3527) |
| -1 :x: | whitespace | 0m 0s | The patch 1 line(s) with tabs. |
| +1 :green_heart: | javadoc | 0m 45s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 18s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 159m 42s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. |
| | | 199m 45s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | phoenix.end2end.salted.SaltedTableVarLengthRowKeyIT |
| | phoenix.end2end.UpsertSelectIT |
| | phoenix.end2end.index.IndexMetadataIT |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/943 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux c5ecfed6ca82 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / 6d84d0f |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/1/artifact/yetus-general-check/output/whitespace-tabs.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/1/testReport/ |
| Max. process+thread count | 6288 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-943/1/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org