You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by "Hadoop QA (Jira)" <ji...@apache.org> on 2020/11/05 06:38:00 UTC
[jira] [Commented] (PHOENIX-5860) Throw exception which region is
closing or splitting when delete data
[ https://issues.apache.org/jira/browse/PHOENIX-5860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17226514#comment-17226514 ]
Hadoop QA commented on PHOENIX-5860:
------------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 38s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m 0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} |
|| || || || {color:brown} 4.x Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 48s{color} | {color:green} 4.x passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s{color} | {color:green} 4.x passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 42s{color} | {color:green} 4.x passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s{color} | {color:green} 4.x passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m 45s{color} | {color:blue} phoenix-core in 4.x has 946 extant spotbugs warnings. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 42s{color} | {color:red} phoenix-core: The patch generated 9 new + 521 unchanged - 6 fixed = 530 total (was 527) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 3m 3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}125m 33s{color} | {color:red} phoenix-core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 35s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}148m 35s{color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | phoenix.end2end.OrphanViewToolIT |
| | phoenix.end2end.index.LocalIndexIT |
| | phoenix.query.MaxConcurrentConnectionsIT |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/PreCommit-PHOENIX-Build/161/artifact/patchprocess/Dockerfile |
| JIRA Issue | PHOENIX-5860 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/13014745/PHOENIX-5860-4.x.001.patch |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux 4c4f05b33728 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | 4.x / 20d2a6d |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/PreCommit-PHOENIX-Build/161/artifact/patchprocess/diff-checkstyle-phoenix-core.txt |
| unit | https://ci-hadoop.apache.org/job/PreCommit-PHOENIX-Build/161/artifact/patchprocess/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/PreCommit-PHOENIX-Build/161/testReport/ |
| Max. process+thread count | 6979 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/PreCommit-PHOENIX-Build/161/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
> Throw exception which region is closing or splitting when delete data
> ---------------------------------------------------------------------
>
> Key: PHOENIX-5860
> URL: https://issues.apache.org/jira/browse/PHOENIX-5860
> Project: Phoenix
> Issue Type: Bug
> Components: core
> Affects Versions: 4.13.1, 4.x
> Reporter: Chao Wang
> Assignee: Chao Wang
> Priority: Blocker
> Fix For: 4.x
>
> Attachments: PHOENIX-5860-4.x.001.patch, PHOENIX-5860-4.x.patch, PHOENIX-5860.4.13.x-HBASE.1.3.x.002.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently delete data is UngroupedAggregateRegionObserver class on server side, this class check if isRegionClosingOrSplitting is true. when isRegionClosingOrSplitting is true, will throw new IOException("Temporarily unable to write from scan because region is closing or splitting").
> when region online , which initialize phoenix CP that isRegionClosingOrSplitting is false.before region split, region change isRegionClosingOrSplitting to true.but if region split failed,split will roll back where not change isRegionClosingOrSplitting to false. after that all write opration will always throw exception which is Temporarily unable to write from scan because region is closing or splitting。
> so we should change isRegionClosingOrSplitting to false when region preRollBackSplit in UngroupedAggregateRegionObserver class。
> A simple test where a data table split failed, then roll back success.but delete data always throw exception.
> # create data table
> # bulkload data for this table
> # alter hbase-server code, which region split will throw exception , then rollback.
> # use hbase shell , split region
> # view regionserver log, where region split failed, and then rollback success.
> # user phoenix sqlline.py for delete data, which will throw exption
> Caused by: java.io.IOException: Temporarily unable to write from scan because region is closing or splitting Caused by: java.io.IOException: Temporarily unable to write from scan because region is closing or splitting at org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:516) at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.overrideDelegate(BaseScannerRegionObserver.java:245) at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:293) at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2881) at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3082) ... 5 more
> at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108) at org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:548) at org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50) at org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97) at org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117) at org.apache.phoenix.iterate.BaseGroupedAggregatingResultIterator.next(BaseGroupedAggregatingResultIterator.java:64) at org.apache.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:39) at org.apache.phoenix.compile.DeleteCompiler$2.execute(DeleteCompiler.java:498) at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:303) at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:295) at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:293) at org.apache.phoenix.jdbc.PhoenixPreparedStatement.executeUpdate(PhoenixPreparedStatement.java:200) at com.huawei.mds.apps.ecidRepeatProcess.EcidProcessCommon$$anonfun$40$$anonfun$apply$19.apply(EcidProcessCommon.scala:2253) at com.huawei.mds.apps.ecidRepeatProcess.EcidProcessCommon$$anonfun$40$$anonfun$apply$19.apply(EcidProcessCommon.scala:2249) at scala.collection.Iterator$class.foreach(Iterator.scala:893) at org.apache.spark.InterruptibleIterator.foreach(InterruptibleIterator.scala:28) at com.huawei.mds.apps.ecidRepeatProcess.EcidProcessCommon$$anonfun$40.apply(EcidProcessCommon.scala:2249) at com.huawei.mds.apps.ecidRepeatProcess.EcidProcessCommon$$anonfun$40.apply(EcidProcessCommon.scala:2243) at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:798) at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:798) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) at org.apache.spark.scheduler.Task.run(Task.scala:99) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:325) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)