You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Li Gao <ga...@marinsoftware.com> on 2016/01/14 21:14:28 UTC
StaleRegionBoundaryCacheException with Phoenix 4.5.1-HBase 1.0 and
Spark 1.4.1
Hi Phoenix users,
We are seeing occasionally (maybe 30 ~ 50% of time) this
StaleRegionBoundaryCacheException when running Spark 1.4.1 and Phoenix
4.5.1-HBase 1.0.
Not sure how to troubleshoot such issue. Any hints and insights are greatly
appreciated!
Thanks,
Li
PS: The following are the exception stack trace:
16/01/14 19:40:16 ERROR yarn.ApplicationMaster: User class threw exception:
org.apache.spark.SparkException: Job aborted due to stage failure: Task 5
in stage 110.0 failed 4 times, most recent failure: Lost task 5.3 in stage
110.0 (TID 35526, datanode-123.somewhere): java.lang.RuntimeException:
org.apache.phoenix.schema.StaleRegionBoundaryCacheException: ERROR 1108
(XCL08): Cache of region boundaries are out of date.
at com.google.common.base.Throwables.propagate(Throwables.java:156)
at
org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:126)
at org.apache.spark.rdd.NewHadoopRDD$$anon$1.<init>(NewHadoopRDD.scala:133)
at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:104)
at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:66)
at org.apache.phoenix.spark.PhoenixRDD.compute(PhoenixRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
at org.apache.spark.scheduler.Task.run(Task.scala:70)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.phoenix.schema.StaleRegionBoundaryCacheException:
ERROR 1108 (XCL08): Cache of region boundaries are out of date.
at
org.apache.phoenix.exception.SQLExceptionCode$13.newException(SQLExceptionCode.java:304)
at
org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
at
org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:131)
at
org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:115)
at
org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:104)
at
org.apache.phoenix.iterate.TableResultIterator.getDelegate(TableResultIterator.java:70)
at
org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:88)
at
org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:79)
at
org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:111)
... 18 more
Re: StaleRegionBoundaryCacheException with Phoenix 4.5.1-HBase 1.0
and Spark 1.4.1
Posted by Josh Mahonin <jm...@gmail.com>.
Hi Li,
This should be resolved in Phoenix 4.7.0. You can also apply the patch
yourself from here:
https://issues.apache.org/jira/browse/PHOENIX-2599
Please let us know if it fixes the problem for you.
Thanks!
Josh
On Thu, Jan 14, 2016 at 6:49 PM, James Taylor <ja...@apache.org>
wrote:
> That was my first thought too, Alicia. However, our MR and Spark
> integration uses a different code path. See my comment on PHOENIX-2599 for
> a potential fix.
>
> Thanks,
> James
>
> On Thu, Jan 14, 2016 at 2:55 PM, Alicia Shu <as...@hortonworks.com> wrote:
>
>> With the fix of PHOENIX-2447
>> <https://issues.apache.org/jira/browse/PHOENIX-2447> in 4.7.0, now
>> Phoenix will try multiple times for StaleRegionBoundaryCacheException until
>> query timeout.
>>
>> Alicia
>>
>> From: Josh Mahonin <jm...@gmail.com>
>> Reply-To: "user@phoenix.apache.org" <us...@phoenix.apache.org>
>> Date: Thursday, January 14, 2016 at 12:33 PM
>> To: "user@phoenix.apache.org" <us...@phoenix.apache.org>
>> Subject: Re: StaleRegionBoundaryCacheException with Phoenix 4.5.1-HBase
>> 1.0 and Spark 1.4.1
>>
>> Hi Li,
>>
>> I've not seen this error myself, though some searching returns a possible
>> root cause:
>>
>> http://mail-archives.apache.org/mod_mbox/incubator-phoenix-user/201507.mbox/%3CCAAF1JdjNW98dAnxf3kx=nDkSWyorpt1redB9DQwjBHVVLfnTqw@mail.gmail.com%3E
>>
>> Could you file a JIRA ticket for this please? It's possible the MapReduce
>> (and by extension, Spark) integration isn't handling this gracefully.
>>
>> Also, if possible, any information you have about your environment, table
>> sizes, read/write workload, etc. would be helpful as well.
>>
>> Thanks!
>>
>> Josh
>>
>> On Thu, Jan 14, 2016 at 3:14 PM, Li Gao <ga...@marinsoftware.com> wrote:
>>
>>> Hi Phoenix users,
>>>
>>>
>>> We are seeing occasionally (maybe 30 ~ 50% of time) this
>>> StaleRegionBoundaryCacheException when running Spark 1.4.1 and Phoenix
>>> 4.5.1-HBase 1.0.
>>>
>>> Not sure how to troubleshoot such issue. Any hints and insights are
>>> greatly appreciated!
>>>
>>> Thanks,
>>>
>>> Li
>>>
>>> PS: The following are the exception stack trace:
>>>
>>> 16/01/14 19:40:16 ERROR yarn.ApplicationMaster: User class threw
>>> exception: org.apache.spark.SparkException: Job aborted due to stage
>>> failure: Task 5 in stage 110.0 failed 4 times, most recent failure: Lost
>>> task 5.3 in stage 110.0 (TID 35526, datanode-123.somewhere):
>>> java.lang.RuntimeException:
>>> org.apache.phoenix.schema.StaleRegionBoundaryCacheException: ERROR 1108
>>> (XCL08): Cache of region boundaries are out of date.
>>>
>>> at com.google.common.base.Throwables.propagate(Throwables.java:156)
>>>
>>> at
>>> org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:126)
>>>
>>> at
>>> org.apache.spark.rdd.NewHadoopRDD$$anon$1.<init>(NewHadoopRDD.scala:133)
>>>
>>> at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:104)
>>>
>>> at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:66)
>>>
>>> at org.apache.phoenix.spark.PhoenixRDD.compute(PhoenixRDD.scala:52)
>>>
>>> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>>>
>>> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>>>
>>> at
>>> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>>>
>>> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>>>
>>> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>>>
>>> at
>>> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>>>
>>> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>>>
>>> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>>>
>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>>
>>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>>
>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>> Caused by: org.apache.phoenix.schema.StaleRegionBoundaryCacheException:
>>> ERROR 1108 (XCL08): Cache of region boundaries are out of date.
>>>
>>> at
>>> org.apache.phoenix.exception.SQLExceptionCode$13.newException(SQLExceptionCode.java:304)
>>>
>>> at
>>> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
>>>
>>> at
>>> org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:131)
>>>
>>> at
>>> org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:115)
>>>
>>> at
>>> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:104)
>>>
>>> at
>>> org.apache.phoenix.iterate.TableResultIterator.getDelegate(TableResultIterator.java:70)
>>>
>>> at
>>> org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:88)
>>>
>>> at
>>> org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:79)
>>>
>>> at
>>> org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:111)
>>>
>>> ... 18 more
>>>
>>
>>
>
Re: StaleRegionBoundaryCacheException with Phoenix 4.5.1-HBase 1.0
and Spark 1.4.1
Posted by James Taylor <ja...@apache.org>.
That was my first thought too, Alicia. However, our MR and Spark
integration uses a different code path. See my comment on PHOENIX-2599 for
a potential fix.
Thanks,
James
On Thu, Jan 14, 2016 at 2:55 PM, Alicia Shu <as...@hortonworks.com> wrote:
> With the fix of PHOENIX-2447
> <https://issues.apache.org/jira/browse/PHOENIX-2447> in 4.7.0, now
> Phoenix will try multiple times for StaleRegionBoundaryCacheException until
> query timeout.
>
> Alicia
>
> From: Josh Mahonin <jm...@gmail.com>
> Reply-To: "user@phoenix.apache.org" <us...@phoenix.apache.org>
> Date: Thursday, January 14, 2016 at 12:33 PM
> To: "user@phoenix.apache.org" <us...@phoenix.apache.org>
> Subject: Re: StaleRegionBoundaryCacheException with Phoenix 4.5.1-HBase
> 1.0 and Spark 1.4.1
>
> Hi Li,
>
> I've not seen this error myself, though some searching returns a possible
> root cause:
>
> http://mail-archives.apache.org/mod_mbox/incubator-phoenix-user/201507.mbox/%3CCAAF1JdjNW98dAnxf3kx=nDkSWyorpt1redB9DQwjBHVVLfnTqw@mail.gmail.com%3E
>
> Could you file a JIRA ticket for this please? It's possible the MapReduce
> (and by extension, Spark) integration isn't handling this gracefully.
>
> Also, if possible, any information you have about your environment, table
> sizes, read/write workload, etc. would be helpful as well.
>
> Thanks!
>
> Josh
>
> On Thu, Jan 14, 2016 at 3:14 PM, Li Gao <ga...@marinsoftware.com> wrote:
>
>> Hi Phoenix users,
>>
>>
>> We are seeing occasionally (maybe 30 ~ 50% of time) this
>> StaleRegionBoundaryCacheException when running Spark 1.4.1 and Phoenix
>> 4.5.1-HBase 1.0.
>>
>> Not sure how to troubleshoot such issue. Any hints and insights are
>> greatly appreciated!
>>
>> Thanks,
>>
>> Li
>>
>> PS: The following are the exception stack trace:
>>
>> 16/01/14 19:40:16 ERROR yarn.ApplicationMaster: User class threw
>> exception: org.apache.spark.SparkException: Job aborted due to stage
>> failure: Task 5 in stage 110.0 failed 4 times, most recent failure: Lost
>> task 5.3 in stage 110.0 (TID 35526, datanode-123.somewhere):
>> java.lang.RuntimeException:
>> org.apache.phoenix.schema.StaleRegionBoundaryCacheException: ERROR 1108
>> (XCL08): Cache of region boundaries are out of date.
>>
>> at com.google.common.base.Throwables.propagate(Throwables.java:156)
>>
>> at
>> org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:126)
>>
>> at
>> org.apache.spark.rdd.NewHadoopRDD$$anon$1.<init>(NewHadoopRDD.scala:133)
>>
>> at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:104)
>>
>> at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:66)
>>
>> at org.apache.phoenix.spark.PhoenixRDD.compute(PhoenixRDD.scala:52)
>>
>> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>>
>> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>>
>> at
>> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>>
>> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>>
>> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>>
>> at
>> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>>
>> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>>
>> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>>
>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>>
>> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>>
>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>
>> at java.lang.Thread.run(Thread.java:745)
>>
>> Caused by: org.apache.phoenix.schema.StaleRegionBoundaryCacheException:
>> ERROR 1108 (XCL08): Cache of region boundaries are out of date.
>>
>> at
>> org.apache.phoenix.exception.SQLExceptionCode$13.newException(SQLExceptionCode.java:304)
>>
>> at
>> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
>>
>> at
>> org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:131)
>>
>> at
>> org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:115)
>>
>> at
>> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:104)
>>
>> at
>> org.apache.phoenix.iterate.TableResultIterator.getDelegate(TableResultIterator.java:70)
>>
>> at
>> org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:88)
>>
>> at
>> org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:79)
>>
>> at
>> org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:111)
>>
>> ... 18 more
>>
>
>
Re: StaleRegionBoundaryCacheException with Phoenix 4.5.1-HBase 1.0
and Spark 1.4.1
Posted by Alicia Shu <as...@hortonworks.com>.
With the fix of PHOENIX-2447<https://issues.apache.org/jira/browse/PHOENIX-2447> in 4.7.0, now Phoenix will try multiple times for StaleRegionBoundaryCacheException until query timeout.
Alicia
From: Josh Mahonin <jm...@gmail.com>>
Reply-To: "user@phoenix.apache.org<ma...@phoenix.apache.org>" <us...@phoenix.apache.org>>
Date: Thursday, January 14, 2016 at 12:33 PM
To: "user@phoenix.apache.org<ma...@phoenix.apache.org>" <us...@phoenix.apache.org>>
Subject: Re: StaleRegionBoundaryCacheException with Phoenix 4.5.1-HBase 1.0 and Spark 1.4.1
Hi Li,
I've not seen this error myself, though some searching returns a possible root cause:
http://mail-archives.apache.org/mod_mbox/incubator-phoenix-user/201507.mbox/%3CCAAF1JdjNW98dAnxf3kx=nDkSWyorpt1redB9DQwjBHVVLfnTqw@mail.gmail.com%3E
Could you file a JIRA ticket for this please? It's possible the MapReduce (and by extension, Spark) integration isn't handling this gracefully.
Also, if possible, any information you have about your environment, table sizes, read/write workload, etc. would be helpful as well.
Thanks!
Josh
On Thu, Jan 14, 2016 at 3:14 PM, Li Gao <ga...@marinsoftware.com>> wrote:
Hi Phoenix users,
We are seeing occasionally (maybe 30 ~ 50% of time) this StaleRegionBoundaryCacheException when running Spark 1.4.1 and Phoenix 4.5.1-HBase 1.0.
Not sure how to troubleshoot such issue. Any hints and insights are greatly appreciated!
Thanks,
Li
PS: The following are the exception stack trace:
16/01/14 19:40:16 ERROR yarn.ApplicationMaster: User class threw exception: org.apache.spark.SparkException: Job aborted due to stage failure: Task 5 in stage 110.0 failed 4 times, most recent failure: Lost task 5.3 in stage 110.0 (TID 35526, datanode-123.somewhere): java.lang.RuntimeException: org.apache.phoenix.schema.StaleRegionBoundaryCacheException: ERROR 1108 (XCL08): Cache of region boundaries are out of date.
at com.google.common.base.Throwables.propagate(Throwables.java:156)
at org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:126)
at org.apache.spark.rdd.NewHadoopRDD$$anon$1.<init>(NewHadoopRDD.scala:133)
at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:104)
at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:66)
at org.apache.phoenix.spark.PhoenixRDD.compute(PhoenixRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
at org.apache.spark.scheduler.Task.run(Task.scala:70)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.phoenix.schema.StaleRegionBoundaryCacheException: ERROR 1108 (XCL08): Cache of region boundaries are out of date.
at org.apache.phoenix.exception.SQLExceptionCode$13.newException(SQLExceptionCode.java:304)
at org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
at org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:131)
at org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:115)
at org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:104)
at org.apache.phoenix.iterate.TableResultIterator.getDelegate(TableResultIterator.java:70)
at org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:88)
at org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:79)
at org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:111)
... 18 more
Re: StaleRegionBoundaryCacheException with Phoenix 4.5.1-HBase 1.0
and Spark 1.4.1
Posted by Josh Mahonin <jm...@gmail.com>.
Hi Li,
I've not seen this error myself, though some searching returns a possible
root cause:
http://mail-archives.apache.org/mod_mbox/incubator-phoenix-user/201507.mbox/%3CCAAF1JdjNW98dAnxf3kx=nDkSWyorpt1redB9DQwjBHVVLfnTqw@mail.gmail.com%3E
Could you file a JIRA ticket for this please? It's possible the MapReduce
(and by extension, Spark) integration isn't handling this gracefully.
Also, if possible, any information you have about your environment, table
sizes, read/write workload, etc. would be helpful as well.
Thanks!
Josh
On Thu, Jan 14, 2016 at 3:14 PM, Li Gao <ga...@marinsoftware.com> wrote:
> Hi Phoenix users,
>
>
> We are seeing occasionally (maybe 30 ~ 50% of time) this
> StaleRegionBoundaryCacheException when running Spark 1.4.1 and Phoenix
> 4.5.1-HBase 1.0.
>
> Not sure how to troubleshoot such issue. Any hints and insights are
> greatly appreciated!
>
> Thanks,
>
> Li
>
> PS: The following are the exception stack trace:
>
> 16/01/14 19:40:16 ERROR yarn.ApplicationMaster: User class threw
> exception: org.apache.spark.SparkException: Job aborted due to stage
> failure: Task 5 in stage 110.0 failed 4 times, most recent failure: Lost
> task 5.3 in stage 110.0 (TID 35526, datanode-123.somewhere):
> java.lang.RuntimeException:
> org.apache.phoenix.schema.StaleRegionBoundaryCacheException: ERROR 1108
> (XCL08): Cache of region boundaries are out of date.
>
> at com.google.common.base.Throwables.propagate(Throwables.java:156)
>
> at
> org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:126)
>
> at org.apache.spark.rdd.NewHadoopRDD$$anon$1.<init>(NewHadoopRDD.scala:133)
>
> at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:104)
>
> at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:66)
>
> at org.apache.phoenix.spark.PhoenixRDD.compute(PhoenixRDD.scala:52)
>
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
>
> at org.apache.spark.scheduler.Task.run(Task.scala:70)
>
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by: org.apache.phoenix.schema.StaleRegionBoundaryCacheException:
> ERROR 1108 (XCL08): Cache of region boundaries are out of date.
>
> at
> org.apache.phoenix.exception.SQLExceptionCode$13.newException(SQLExceptionCode.java:304)
>
> at
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
>
> at
> org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:131)
>
> at
> org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:115)
>
> at
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:104)
>
> at
> org.apache.phoenix.iterate.TableResultIterator.getDelegate(TableResultIterator.java:70)
>
> at
> org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:88)
>
> at
> org.apache.phoenix.iterate.TableResultIterator.<init>(TableResultIterator.java:79)
>
> at
> org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:111)
>
> ... 18 more
>