You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2021/09/07 09:20:12 UTC

[GitHub] [shardingsphere] sandynz opened a new pull request #12267: Drop PostgreSQL replication slot when scaling task finished

sandynz opened a new pull request #12267:
URL: https://github.com/apache/shardingsphere/pull/12267


   
   Changes proposed in this pull request:
   - Add destroy method for PositionInitializer
   - Drop slot when scaling task finished
   - Stop replication before drop slot
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] codecov-commenter commented on pull request #12267: Drop PostgreSQL replication slot when scaling task finished

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #12267:
URL: https://github.com/apache/shardingsphere/pull/12267#issuecomment-914223935


   # [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/12267?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#12267](https://codecov.io/gh/apache/shardingsphere/pull/12267?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (00c10b2) into [master](https://codecov.io/gh/apache/shardingsphere/commit/865d138d9b5b5168857ecd5e5633497a2d3d8d4f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (865d138) will **decrease** coverage by `0.00%`.
   > The diff coverage is `44.64%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/shardingsphere/pull/12267/graphs/tree.svg?width=650&height=150&src=pr&token=ZvlXpWa7so&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/shardingsphere/pull/12267?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #12267      +/-   ##
   ============================================
   - Coverage     63.72%   63.72%   -0.01%     
   - Complexity     1260     1265       +5     
   ============================================
     Files          2351     2358       +7     
     Lines         35762    35838      +76     
     Branches       6211     6230      +19     
   ============================================
   + Hits          22790    22837      +47     
   - Misses        11147    11158      +11     
   - Partials       1825     1843      +18     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/shardingsphere/pull/12267?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...he/shardingsphere/scaling/core/job/JobContext.java](https://codecov.io/gh/apache/shardingsphere/pull/12267/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvY29yZS9qb2IvSm9iQ29udGV4dC5qYXZh) | `100.00% <ø> (ø)` | |
   | [...scaling/core/job/position/PositionInitializer.java](https://codecov.io/gh/apache/shardingsphere/pull/12267/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvY29yZS9qb2IvcG9zaXRpb24vUG9zaXRpb25Jbml0aWFsaXplci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [.../scaling/core/job/preparer/ScalingJobPreparer.java](https://codecov.io/gh/apache/shardingsphere/pull/12267/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvY29yZS9qb2IvcHJlcGFyZXIvU2NhbGluZ0pvYlByZXBhcmVyLmphdmE=) | `68.08% <0.00%> (-16.13%)` | :arrow_down: |
   | [...sphere/scaling/core/job/schedule/JobScheduler.java](https://codecov.io/gh/apache/shardingsphere/pull/12267/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvY29yZS9qb2Ivc2NoZWR1bGUvSm9iU2NoZWR1bGVyLmphdmE=) | `46.66% <0.00%> (-5.19%)` | :arrow_down: |
   | [.../scaling/core/job/schedule/JobSchedulerCenter.java](https://codecov.io/gh/apache/shardingsphere/pull/12267/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvY29yZS9qb2Ivc2NoZWR1bGUvSm9iU2NoZWR1bGVyQ2VudGVyLmphdmE=) | `46.42% <37.50%> (+6.42%)` | :arrow_up: |
   | [...resql/component/PostgreSQLPositionInitializer.java](https://codecov.io/gh/apache/shardingsphere/pull/12267/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWRpYWxlY3Qvc2hhcmRpbmdzcGhlcmUtc2NhbGluZy1wb3N0Z3Jlc3FsL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9zY2FsaW5nL3Bvc3RncmVzcWwvY29tcG9uZW50L1Bvc3RncmVTUUxQb3NpdGlvbkluaXRpYWxpemVyLmphdmE=) | `67.34% <61.53%> (-6.57%)` | :arrow_down: |
   | [...rdingsphere/scaling/core/job/FinishedCheckJob.java](https://codecov.io/gh/apache/shardingsphere/pull/12267/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvY29yZS9qb2IvRmluaXNoZWRDaGVja0pvYi5qYXZh) | `52.50% <100.00%> (+1.21%)` | :arrow_up: |
   | [...che/shardingsphere/scaling/core/job/JobStatus.java](https://codecov.io/gh/apache/shardingsphere/pull/12267/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvY29yZS9qb2IvSm9iU3RhdHVzLmphdmE=) | `100.00% <100.00%> (ø)` | |
   | [...he/shardingsphere/scaling/core/job/ScalingJob.java](https://codecov.io/gh/apache/shardingsphere/pull/12267/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvY29yZS9qb2IvU2NhbGluZ0pvYi5qYXZh) | `100.00% <100.00%> (ø)` | |
   | [...ling/postgresql/component/PostgreSQLWalDumper.java](https://codecov.io/gh/apache/shardingsphere/pull/12267/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWRpYWxlY3Qvc2hhcmRpbmdzcGhlcmUtc2NhbGluZy1wb3N0Z3Jlc3FsL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9zY2FsaW5nL3Bvc3RncmVzcWwvY29tcG9uZW50L1Bvc3RncmVTUUxXYWxEdW1wZXIuamF2YQ==) | `70.58% <100.00%> (-2.95%)` | :arrow_down: |
   | ... and [43 more](https://codecov.io/gh/apache/shardingsphere/pull/12267/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/shardingsphere/pull/12267?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/12267?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [865d138...00c10b2](https://codecov.io/gh/apache/shardingsphere/pull/12267?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] sandynz commented on pull request #12267: Drop PostgreSQL replication slot when scaling task finished

Posted by GitBox <gi...@apache.org>.
sandynz commented on pull request #12267:
URL: https://github.com/apache/shardingsphere/pull/12267#issuecomment-914279259


   Consumer (`PGReplicationStream`) must be closed before drop replication slot, else exception thrown:
   ```
   o.a.s.s.c.j.p.ScalingJobPreparer - Scaling job destroying failed
   org.postgresql.util.PSQLException: ERROR: replication slot "sharding_scaling" is active for PID 76289
   	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
   	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
   	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
   	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
   	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
   	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
   	at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:132)
   	at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
   	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
   	at org.apache.shardingsphere.scaling.postgresql.component.PostgreSQLPositionInitializer.dropSlotIfExists(PostgreSQLPositionInitializer.java:116)
   	at org.apache.shardingsphere.scaling.postgresql.component.PostgreSQLPositionInitializer.destroy(PostgreSQLPositionInitializer.java:103)
   	at org.apache.shardingsphere.scaling.core.job.preparer.ScalingJobPreparer.cleanup(ScalingJobPreparer.java:113)
   	at org.apache.shardingsphere.scaling.core.job.schedule.JobScheduler.stop(JobScheduler.java:57)
   	at org.apache.shardingsphere.scaling.core.job.schedule.JobSchedulerCenter.stop(JobSchedulerCenter.java:76)
   	at org.apache.shardingsphere.scaling.core.executor.job.ScalingJobExecutor.lambda$watchGovernanceRepositoryConfiguration$0(ScalingJobExecutor.java:62)
   	at org.apache.shardingsphere.mode.repository.cluster.zookeeper.CuratorZookeeperRepository.lambda$watch$1(CuratorZookeeperRepository.java:247)
   	at org.apache.curator.framework.recipes.cache.TreeCacheListenerWrapper.sendEvent(TreeCacheListenerWrapper.java:71)
   	at org.apache.curator.framework.recipes.cache.TreeCacheListenerWrapper.event(TreeCacheListenerWrapper.java:48)
   	at org.apache.curator.framework.recipes.cache.CuratorCacheListenerBuilderImpl$2.lambda$event$0(CuratorCacheListenerBuilderImpl.java:149)
   	at java.util.ArrayList.forEach(ArrayList.java:1259)
   	at org.apache.curator.framework.recipes.cache.CuratorCacheListenerBuilderImpl$2.event(CuratorCacheListenerBuilderImpl.java:149)
   	at org.apache.curator.framework.recipes.cache.CuratorCacheImpl.lambda$putStorage$6(CuratorCacheImpl.java:274)
   	at org.apache.curator.framework.listen.MappingListenerManager.lambda$forEach$0(MappingListenerManager.java:92)
   	at org.apache.curator.framework.listen.MappingListenerManager.forEach(MappingListenerManager.java:89)
   	at org.apache.curator.framework.listen.StandardListenerManager.forEach(StandardListenerManager.java:89)
   	at org.apache.curator.framework.recipes.cache.CuratorCacheImpl.lambda$callListeners$10(CuratorCacheImpl.java:293)
   	at java.util.concurrent.CompletableFuture$AsyncRun.run$$$capture(CompletableFuture.java:1640)
   	at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:748)
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] sandynz commented on pull request #12267: Drop PostgreSQL replication slot when scaling task finished

Posted by GitBox <gi...@apache.org>.
sandynz commented on pull request #12267:
URL: https://github.com/apache/shardingsphere/pull/12267#issuecomment-914260198


   Why we change it:
   
   1, From openGauss testing response, execute `SELECT * FROM pg_create_logical_replication_slot('slot'_name, 'test_decoding');` repeatedly will cause exception, and error code is different as PostgreSQL, so exception thrown will cause scaling task broken.
   
   We'll check whether slot exists before create it, and drop slot when scaling task finished.
   
   2, Keep slot after task finished, it will cost system resources unnecessaryly.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] tristaZero merged pull request #12267: Drop PostgreSQL replication slot when scaling task finished

Posted by GitBox <gi...@apache.org>.
tristaZero merged pull request #12267:
URL: https://github.com/apache/shardingsphere/pull/12267


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org