You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by scottmf <sc...@gmail.com> on 2020/04/28 05:37:28 UTC
Random2LruPageEvictionTracker causing hanging in our integration
tests
I'm seeing our integration tests hang. The cause is something in
Random2LruPageEvictionTracker, i'm able to reproduce this 100% of time on
many different laptops and in our ci/cd. I was able to work around this by
disabling PageEviction in our Default Data Region during the tests. I don't
see this in our production clusters.
Does anyone have an idea of how i can fix it without disabling eviction?
I'm using Ignite 2.8.0.
One thing to note is that I'm not sure what to make of the line
Random2LruPageEvictionTracker.evictDataPage(Random2LruPageEvictionTracker.java:152),
but it seemed to simply loop forever when stepping through it in my IDE.
Stack Trace:
"Test worker" #22 prio=5 os_prio=31 cpu=299703.41ms elapsed=317.18s
tid=0x00007ff3cfc8c800 nid=0x7203 runnable [0x0000700005b38000]
java.lang.Thread.State: RUNNABLE at
org.apache.ignite.internal.processors.cache.persistence.evict.Random2LruPageEvictionTracker.evictDataPage(Random2LruPageEvictionTracker.java:152)
at
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.ensureFreeSpace(IgniteCacheDatabaseSharedManager.java:1086)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.ensureFreeSpace(GridCacheMapEntry.java:4513)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerSet(GridCacheMapEntry.java:1461)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:745)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.localFinish(GridNearTxLocal.java:3850)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.doFinish(GridNearTxFinishFuture.java:440)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:390)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:4129)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:4118)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:399)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:354)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commitNearTxLocalAsync(GridNearTxLocal.java:4118)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commit(GridNearTxLocal.java:4086)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:587)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:556)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.retryTopologySafe(DataStructuresProcessor.java:1664)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.getAtomic(DataStructuresProcessor.java:556)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.reentrantLock(DataStructuresProcessor.java:1361)
at
org.apache.ignite.internal.IgniteKernal.reentrantLock(IgniteKernal.java:4136)
at jdk.internal.reflect.GeneratedMethodAccessor713.invoke(Unknown Source)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
at com.example.ignite.IgniteInitializer$1.invoke(IgniteInitializer.java:158)
at com.sun.proxy.$Proxy205.reentrantLock(Unknown Source) at
com.example.data.store.jdbc.cache.CacheService.getCount(CacheService.java:47)
at
com.example.data.store.jdbc.cache.CacheService$$FastClassBySpringCGLIB$$7efa9131.invoke()......
Configuration: I tried different permutations, the only thing that worked
was disabling eviction. I also tried LRU instead of LRU_2 but that didn't
work either.
@Bean public IgniteConfiguration igniteConfiguration(
@Value("#{'${IGNITE_HOME:}' ?: systemProperties['java.io.tmpdir']}") String
igniteHome, SymphonyIgniteProperties symphonyIgniteProperties, Optional
dataStorageConfiguration, TcpDiscoverySpi tcpDiscoverySpi) {
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setIgniteInstanceName(symphonyIgniteProperties.getIgniteInstanceName());
dataStorageConfiguration.ifPresent(dss ->
cfg.setDataStorageConfiguration(dss)); cfg.setIgniteHome(igniteHome);
cfg.setPeerClassLoadingEnabled(true); cfg.setIncludeEventTypes(new int[] {
EVT_CACHE_STARTED });
System.setProperty(IgniteSystemProperties.IGNITE_QUIET, "true");
System.setProperty(IgniteSystemProperties.IGNITE_UPDATE_NOTIFIER, "false");
cfg.setGridLogger(new Slf4jLogger()); cfg.setDiscoverySpi(tcpDiscoverySpi);
cfg.setCommunicationSpi(new StandaloneNoopCommunicationSpi());
cfg.setClientMode(false); cfg.setDataStreamerThreadPoolSize(2);
cfg.setPublicThreadPoolSize(2); cfg.setSystemThreadPoolSize(2);
cfg.setStripedPoolSize(2); cfg.setIgfsThreadPoolSize(2);
cfg.setManagementThreadPoolSize(2); cfg.setServiceThreadPoolSize(2);
cfg.setQueryThreadPoolSize(2); cfg.setPeerClassLoadingThreadPoolSize(2);
// setting ConnectorConfiguration to null removes these thread pools: //
"grid-nio-worker-tcp-rest-x", "nio-acceptor-tcp-rest-#18%%" // and
"session-timeout-worker-#13%%" cfg.setConnectorConfiguration(null);
cfg.setClientConnectorConfiguration(null);
cfg.setRebalanceThreadPoolSize(1); return cfg; }
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Random2LruPageEvictionTracker causing hanging in our integration tests
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
If you have a lot of caches of uniform structure and low entry count, you
should use cache groups.
For unrelated caches you should not use cache groups.
That's the rule of thumb.
Regards,
--
Ilya Kasnacheev
чт, 4 июн. 2020 г. в 08:41, scottmf <sc...@gmail.com>:
> Thanks Ilya. I tried all your suggestions and they work as expected. I'll
> close the bug.
>
> WRT cache groups, what's the rule of thumb with using cache groups? I read
> https://apacheignite.readme.io/docs/cache-groups, but I'm not sure on the
> approach I should take. Should I simply stay away from cache groups until
> I
> need them? Or should I use them from the start.
>
> What is your suggestion?
>
> thanks.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Random2LruPageEvictionTracker causing hanging in our
integration tests
Posted by scottmf <sc...@gmail.com>.
Thanks Ilya. I tried all your suggestions and they work as expected. I'll
close the bug.
WRT cache groups, what's the rule of thumb with using cache groups? I read
https://apacheignite.readme.io/docs/cache-groups, but I'm not sure on the
approach I should take. Should I simply stay away from cache groups until I
need them? Or should I use them from the start.
What is your suggestion?
thanks.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Random2LruPageEvictionTracker causing hanging in our integration tests
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
This is expected :)
I recommend decreasing the number of caches, grouping them into
cacheGroup's so that they share metadata, or decreasing number of
partitions in their affinity functions.
Regards,
--
Ilya Kasnacheev
сб, 30 мая 2020 г. в 03:19, scottmf <sc...@gmail.com>:
> hi Ilya, I have reproduced the problem and logged a bug
>
> bug -> https://issues.apache.org/jira/browse/IGNITE-13097
> repro -> https://github.com/scottmf/ignite-oom
>
> As I stated on the bug:
>
> My observation is that each IgniteCache object takes up almost 8MB. I'm
> pretty sure this is taking up all the off-heap memory.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Random2LruPageEvictionTracker causing hanging in our
integration tests
Posted by scottmf <sc...@gmail.com>.
hi Ilya, I have reproduced the problem and logged a bug
bug -> https://issues.apache.org/jira/browse/IGNITE-13097
repro -> https://github.com/scottmf/ignite-oom
As I stated on the bug:
My observation is that each IgniteCache object takes up almost 8MB. I'm
pretty sure this is taking up all the off-heap memory.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Random2LruPageEvictionTracker causing hanging in our
integration tests
Posted by scottmf <sc...@gmail.com>.
thanks Ilya,
I really have no idea why it is hanging like this. I was even more
surprised when I looked at the code because I saw that there is throttling
in the code that should prevent this like you said. I do see an OOM when I
turn off page eviction all together. But when eviction is turned on it just
hangs. Ultimately what I did was to reset my ignite cluster on random tests
after they completed so that I don't run into this anymore.
Given your response I think the next thing to do is to put in the work to
give you an environment that reproduces this.
I'll add that to the ticket when I am able to make it available for you all.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Random2LruPageEvictionTracker causing hanging in our integration tests
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I can see that it did see one of this messages, but why do you think it is
stuck? For what I'm seeing, it has promptly un-stuck in one second:
2020-05-01 12:27:23.386 PDT priority='WARN' thread='Test worker'
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager@127
- Page-based evictions started. Consider increasing 'maxSize' on Data
Region configuration: DefaultDataRegion
2020-05-01 12:27:23.387 PDT priority='WARN' thread='Test worker'
org.apache.ignite.internal.processors.cache.persistence.evict.Random2LruPageEvictionTracker@127
- Too many attempts to choose data page: 5000
2020-05-01 12:27:24.374 PDT priority='TRACE' thread='Test worker'
org.apache.ignite.internal.processors.cache.GridCacheMapEntry@91 -
markObsolete0 [key=KeyCacheObjectImpl [part=-1, val=reentrant-select
count(t."id") from "sample" t where 1=1 AND t."other"=? AND
t."id"=?|bbb291c45a1-5c55-4bf1-86b3-dd8e0c410695, hasValBytes=true],
entry=29960847, clear=false]
I would maybe expect a IgniteOOM error after this, but it does not seem to
materialize. Why do you think it is stuck for good?
However, this message is throttled so it can be spending any amount of time
in that code.
I can see that you have partition map exchange at the same time, so that's
where you might be waiting.
I have also found https://issues.apache.org/jira/browse/IGNITE-12510, is
there a chance it is relevant? How large are your entries? I also think
512M is a tiny size for data region, try increasing it.
Regards,
--
Ilya Kasnacheev
пт, 1 мая 2020 г. в 23:08, scottmf <sc...@gmail.com>:
> out.multipart-aa
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-aa>
>
> out.multipart-ab
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-ab>
>
> out.multipart-ac
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-ac>
>
> out.multipart-ad
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-ad>
>
> out.multipart-ae
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-ae>
>
> out.multipart-af
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-af>
>
> out.multipart-ag
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-ag>
>
>
> hi Ilya, I turned on debugging for ignite and dumped the output into a
> multipart set of files that i've attached. Let me know if you need anymore
> info. If needed I can try to reproduce this in a generic setting but that
> will take time.
>
> since 5MB is the limit, i had to upload the files in 5MB chunks. To
> assemble them, put them into a directory then run 'cat * > file.gz'
>
> Answers to your questions:
>
> > Do you see any "Too many attempts to choose data page" or "Too many
> failed
> > attempts to evict page" messages in your logs?
>
> See output file
>
> > How large are your data regions
>
> we only use the default data region with default settings - 512MB
>
> > how many caches do they have?
>
> maybe 20ish?
>
> > I would expect that behavior if eviction can't find any page to evict, if
> > all data pages are evicted already and only metadata pages remain, ones
> > that cannot be evicted.
>
> Could you elaborate on this or point me to any docs?
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Random2LruPageEvictionTracker causing hanging in our
integration tests
Posted by scottmf <sc...@gmail.com>.
out.multipart-aa
<http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-aa>
out.multipart-ab
<http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-ab>
out.multipart-ac
<http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-ac>
out.multipart-ad
<http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-ad>
out.multipart-ae
<http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-ae>
out.multipart-af
<http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-af>
out.multipart-ag
<http://apache-ignite-users.70518.x6.nabble.com/file/t1632/out.multipart-ag>
hi Ilya, I turned on debugging for ignite and dumped the output into a
multipart set of files that i've attached. Let me know if you need anymore
info. If needed I can try to reproduce this in a generic setting but that
will take time.
since 5MB is the limit, i had to upload the files in 5MB chunks. To
assemble them, put them into a directory then run 'cat * > file.gz'
Answers to your questions:
> Do you see any "Too many attempts to choose data page" or "Too many failed
> attempts to evict page" messages in your logs?
See output file
> How large are your data regions
we only use the default data region with default settings - 512MB
> how many caches do they have?
maybe 20ish?
> I would expect that behavior if eviction can't find any page to evict, if
> all data pages are evicted already and only metadata pages remain, ones
> that cannot be evicted.
Could you elaborate on this or point me to any docs?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Random2LruPageEvictionTracker causing hanging in our integration tests
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Yes, you are right, I happened to miss the relevant portion.
Do you see any "Too many attempts to choose data page" or "Too many failed
attempts to evict page" messages in your logs?
How large are your data regions, how many caches do they have? I would
expect that behavior if eviction can't find any page to evict, if all data
pages are evicted already and only metadata pages remain, ones that cannot
be evicted.
Regards,
--
Ilya Kasnacheev
чт, 30 апр. 2020 г. в 22:14, scottmf <sc...@gmail.com>:
> Hi Ilya, I'm confused. What do you see? I am posting a stack that ends
> with several apache ignite calls.
>
> "Test worker" #22 prio=5 os_prio=31 cpu=299703.41ms elapsed=317.18s tid=0x00007ff3cfc8c800 nid=0x7203 runnable [0x0000700005b38000]
> java.lang.Thread.State: RUNNABLE
> at org.apache.ignite.internal.processors.cache.persistence.evict.Random2LruPageEvictionTracker.evictDataPage(Random2LruPageEvictionTracker.java:152)
> at org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.ensureFreeSpace(IgniteCacheDatabaseSharedManager.java:1086)
> at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.ensureFreeSpace(GridCacheMapEntry.java:4513)
> at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerSet(GridCacheMapEntry.java:1461)
> at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:745)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.localFinish(GridNearTxLocal.java:3850)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.doFinish(GridNearTxFinishFuture.java:440)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:390)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:4129)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:4118)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:399)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:354)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commitNearTxLocalAsync(GridNearTxLocal.java:4118)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commit(GridNearTxLocal.java:4086)
> at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:587)
> at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:556)
> at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.retryTopologySafe(DataStructuresProcessor.java:1664)
> at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.getAtomic(DataStructuresProcessor.java:556)
> at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.reentrantLock(DataStructuresProcessor.java:1361)
> at org.apache.ignite.internal.IgniteKernal.reentrantLock(IgniteKernal.java:4136)
> at jdk.internal.reflect.GeneratedMethodAccessor713.invoke(Unknown Source)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at com.example.symphony.cmf.ignite.IgniteInitializer$1.invoke(IgniteInitializer.java:158)
> at com.sun.proxy.$Proxy205.reentrantLock(Unknown Source)
> at com.example.data.store.jdbc.cache.CacheService.getCount(CacheService.java:47)
> at com.example.data.store.jdbc.cache.CacheService$$FastClassBySpringCGLIB$$7efa9131.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)
> at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:105)
> at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:94)
> at jdk.internal.reflect.GeneratedMethodAccessor712.invoke(Unknown Source)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
> at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
> at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91)
> at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287)
> at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:164)
> at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:118)
> at org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:153)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
> at com.example.data.store.jdbc.cache.CacheService$$EnhancerBySpringCGLIB$$18de2c44.getCount()
> at com.example.data.store.jdbc.crud.Search.execute(Search.java:65)
> at com.example.data.store.jdbc.crud.Get.execute(Get.java:38)
> at com.example.data.store.jdbc.BaseDataChannel.lambda$get$6(BaseDataChannel.java:154)
> at com.example.data.store.jdbc.BaseDataChannel$$Lambda$2394/0x0000000801429040.apply(Unknown Source)
> at com.example.data.store.jdbc.DatabaseOperation.execute(DatabaseOperation.java:33)
> at com.example.data.store.jdbc.BaseDataChannel.execute(BaseDataChannel.java:236)
> at com.example.data.store.jdbc.BaseDataChannel.get(BaseDataChannel.java:153)
> at com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.lambda$get$6(MeteringStorageEngine.java:86)
> at com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData$$Lambda$2393/0x0000000801428c40.get(Unknown Source)
> at com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.execute(MeteringStorageEngine.java:156)
> at com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.get(MeteringStorageEngine.java:86)
> at com.example.data.core.store.data.DataAccess.get(DataAccess.java:319)
> at com.example.data.core.store.data.DataAccess$$FastClassBySpringCGLIB$$9d07c86a.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
> at com.example.data.core.store.data.DataAccess$$EnhancerBySpringCGLIB$$e3df238d_3.get()
> at com.example.data.core.impl.DataServiceImpl.get(DataServiceImpl.java:69)
> at com.example.data.web.v1.endpoint.DataController.lambda$get$2(DataController.java:171)
> at com.example.data.web.v1.endpoint.DataController$$Lambda$2310/0x000000080139b440.apply(Unknown Source)
> at com.example.data.web.v1.endpoint.DataController.getOrNotFound(DataController.java:202)
> at com.example.data.web.v1.endpoint.DataController.get(DataController.java:170)
> at com.example.data.web.v1.endpoint.DataController$$FastClassBySpringCGLIB$$4ee654da.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
> at com.example.data.web.v1.endpoint.DataController$$EnhancerBySpringCGLIB$$b6fe0edd_3.get()
> at jdk.internal.reflect.GeneratedMethodAccessor719.invoke(Unknown Source)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
> at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
> at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
> at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
> at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
> at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
> at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
> at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
> at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
> at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
> at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
> at org.springframework.test.web.servlet.TestDispatcherServlet.service(TestDispatcherServlet.java:72)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
> at org.springframework.mock.web.MockFilterChain$ServletFilterProxy.doFilter(MockFilterChain.java:167)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at com.example.library.authz.spring.UrlAccessFilter.doFilterInternal(UrlAccessFilter.java:67)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at com.example.symphony.csp.auth.filters.SymphonyOAuth2Filter.doFilter(SymphonyOAuth2Filter.java:87)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:155)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:114)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
> at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
> at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
> at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
> at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at io.opentracing.contrib.web.servlet.filter.TracingFilter.doFilter(TracingFilter.java:189)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.test.web.servlet.MockMvc.perform(MockMvc.java:183)
> at com.example.data.web.v1.RenameFieldIT.addObject(RenameFieldIT.java:381)
> at com.example.data.web.v1.RenameFieldIT.testSchemaUpdate(RenameFieldIT.java:138)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
> at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
> at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:132)
> at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:124)
> at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:74)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$152/0x00000008001a4440.apply(Unknown Source)
> at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
> at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall$$Lambda$153/0x00000008001a3840.apply(Unknown Source)
> at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
> at org.junit.jupiter.engine.execution.ExecutableInvoker$$Lambda$364/0x00000008002dc840.apply(Unknown Source)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
> at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
> at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$1876/0x00000008011ee840.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService$$Lambda$222/0x00000008001e4040.accept(Unknown Source)
> at java.util.ArrayList.forEach(java.base@11.0.5/ArrayList.java:1540)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService$$Lambda$222/0x00000008001e4040.accept(Unknown Source)
> at java.util.ArrayList.forEach(java.base@11.0.5/ArrayList.java:1540)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
> at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
> at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
> at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:220)
> at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:188)
> at org.junit.platform.launcher.core.DefaultLauncher$$Lambda$175/0x00000008001c6c40.accept(Unknown Source)
> at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202)
> at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181)
> at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
> at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:102)
> at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:82)
> at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:78)
> at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
> at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
> at com.sun.proxy.$Proxy5.stop(Unknown Source)
> at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:132)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
> at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
> at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
> at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
> at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.5/ThreadPoolExecutor.java:1128)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.5/ThreadPoolExecutor.java:628)
> at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
> at java.lang.Thread.run(java.base@11.0.5/Thread.java:834)
>
>
> ------------------------------
> Sent from the Apache Ignite Users mailing list archive
> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>
Re: Random2LruPageEvictionTracker causing hanging in our
integration tests
Posted by scottmf <sc...@gmail.com>.
Hi Ilya, I'm confused. What do you see?I am posting a stack that ends with
several apache ignite calls.
"Test worker" #22 prio=5 os_prio=31 cpu=299703.41ms elapsed=317.18s
tid=0x00007ff3cfc8c800 nid=0x7203 runnable [0x0000700005b38000]
java.lang.Thread.State: RUNNABLE at
org.apache.ignite.internal.processors.cache.persistence.evict.Random2LruPageEvictionTracker.evictDataPage(Random2LruPageEvictionTracker.java:152)
at
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.ensureFreeSpace(IgniteCacheDatabaseSharedManager.java:1086)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.ensureFreeSpace(GridCacheMapEntry.java:4513)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerSet(GridCacheMapEntry.java:1461)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:745)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.localFinish(GridNearTxLocal.java:3850)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.doFinish(GridNearTxFinishFuture.java:440)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:390)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:4129)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:4118)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:399)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:354)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commitNearTxLocalAsync(GridNearTxLocal.java:4118)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commit(GridNearTxLocal.java:4086)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:587)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:556)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.retryTopologySafe(DataStructuresProcessor.java:1664)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.getAtomic(DataStructuresProcessor.java:556)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.reentrantLock(DataStructuresProcessor.java:1361)
at
org.apache.ignite.internal.IgniteKernal.reentrantLock(IgniteKernal.java:4136)
at jdk.internal.reflect.GeneratedMethodAccessor713.invoke(Unknown Source) at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
com.example.symphony.cmf.ignite.IgniteInitializer$1.invoke(IgniteInitializer.java:158)
at com.sun.proxy.$Proxy205.reentrantLock(Unknown Source) at
com.example.data.store.jdbc.cache.CacheService.getCount(CacheService.java:47)
at
com.example.data.store.jdbc.cache.CacheService$$FastClassBySpringCGLIB$$7efa9131.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)
at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:105)
at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:94) at
jdk.internal.reflect.GeneratedMethodAccessor712.invoke(Unknown Source) at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
at
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91)
at
org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287)
at
org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:164)
at
org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:118)
at
org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:153)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
at
com.example.data.store.jdbc.cache.CacheService$$EnhancerBySpringCGLIB$$18de2c44.getCount()
at com.example.data.store.jdbc.crud.Search.execute(Search.java:65) at
com.example.data.store.jdbc.crud.Get.execute(Get.java:38) at
com.example.data.store.jdbc.BaseDataChannel.lambda$get$6(BaseDataChannel.java:154)
at
com.example.data.store.jdbc.BaseDataChannel$$Lambda$2394/0x0000000801429040.apply(Unknown
Source) at
com.example.data.store.jdbc.DatabaseOperation.execute(DatabaseOperation.java:33)
at
com.example.data.store.jdbc.BaseDataChannel.execute(BaseDataChannel.java:236)
at com.example.data.store.jdbc.BaseDataChannel.get(BaseDataChannel.java:153)
at
com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.lambda$get$6(MeteringStorageEngine.java:86)
at
com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData$$Lambda$2393/0x0000000801428c40.get(Unknown
Source) at
com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.execute(MeteringStorageEngine.java:156)
at
com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.get(MeteringStorageEngine.java:86)
at com.example.data.core.store.data.DataAccess.get(DataAccess.java:319) at
com.example.data.core.store.data.DataAccess$$FastClassBySpringCGLIB$$9d07c86a.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
at
com.example.data.core.store.data.DataAccess$$EnhancerBySpringCGLIB$$e3df238d_3.get()
at com.example.data.core.impl.DataServiceImpl.get(DataServiceImpl.java:69)
at
com.example.data.web.v1.endpoint.DataController.lambda$get$2(DataController.java:171)
at
com.example.data.web.v1.endpoint.DataController$$Lambda$2310/0x000000080139b440.apply(Unknown
Source) at
com.example.data.web.v1.endpoint.DataController.getOrNotFound(DataController.java:202)
at
com.example.data.web.v1.endpoint.DataController.get(DataController.java:170)
at
com.example.data.web.v1.endpoint.DataController$$FastClassBySpringCGLIB$$4ee654da.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
at
com.example.data.web.v1.endpoint.DataController$$EnhancerBySpringCGLIB$$b6fe0edd_3.get()
at jdk.internal.reflect.GeneratedMethodAccessor719.invoke(Unknown Source) at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at
org.springframework.test.web.servlet.TestDispatcherServlet.service(TestDispatcherServlet.java:72)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at
org.springframework.mock.web.MockFilterChain$ServletFilterProxy.doFilter(MockFilterChain.java:167)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
com.example.library.authz.spring.UrlAccessFilter.doFilterInternal(UrlAccessFilter.java:67)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
com.example.symphony.csp.auth.filters.SymphonyOAuth2Filter.doFilter(SymphonyOAuth2Filter.java:87)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:155)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:114)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
at
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
io.opentracing.contrib.web.servlet.filter.TracingFilter.doFilter(TracingFilter.java:189)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at org.springframework.test.web.servlet.MockMvc.perform(MockMvc.java:183) at
com.example.data.web.v1.RenameFieldIT.addObject(RenameFieldIT.java:381) at
com.example.data.web.v1.RenameFieldIT.testSchemaUpdate(RenameFieldIT.java:138)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native
Method) at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:132)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:124)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:74)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$152/0x00000008001a4440.apply(Unknown
Source) at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall$$Lambda$153/0x00000008001a3840.apply(Unknown
Source) at
org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at
org.junit.jupiter.engine.execution.ExecutableInvoker$$Lambda$364/0x00000008002dc840.apply(Unknown
Source) at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$1876/0x00000008011ee840.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService$$Lambda$222/0x00000008001e4040.accept(Unknown
Source) at java.util.ArrayList.forEach(java.base@11.0.5/ArrayList.java:1540)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService$$Lambda$222/0x00000008001e4040.accept(Unknown
Source) at java.util.ArrayList.forEach(java.base@11.0.5/ArrayList.java:1540)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at
org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at
org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:220)
at
org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:188)
at
org.junit.platform.launcher.core.DefaultLauncher$$Lambda$175/0x00000008001c6c40.accept(Unknown
Source) at
org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
at
org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:102)
at
org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:82)
at
org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:78)
at
org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native
Method) at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
at
org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
at com.sun.proxy.$Proxy5.stop(Unknown Source) at
org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:132)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native
Method) at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
at
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
at
org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
at
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.5/ThreadPoolExecutor.java:1128)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.5/ThreadPoolExecutor.java:628)
at
org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(java.base@11.0.5/Thread.java:834)
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Random2LruPageEvictionTracker causing hanging in our integration tests
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
In your stack trace I don't see a single 'org.apache.ignite' line. Why do
you think Apache Ignite is to blame here?
Regards,
--
Ilya Kasnacheev
ср, 29 апр. 2020 г. в 22:26, scottmf <sc...@gmail.com>:
> Hi Anton, Just to be clear, the stack trace is from a thread dump that I
> took while the process was hanging indefinitely.
>
> Although I can reproduce this easily in my service, I can't share the code
> with you. I'll attempt to get a generic use case to hang in this manner and
> post it to github.
>
> The full stack is below.
>
> No use-case in this particular scenario for eviction. Like I said, it is
> just for integration testing. My only concern is to ensure that there is no
> bug that would hit us in production.
>
> It is eviction for an in-memory cluster, no persistence, on-heap or near
> cache.
>
> "Test worker" #22 prio=5 os_prio=31 cpu=299703.41ms elapsed=317.18s tid=0x00007ff3cfc8c800 nid=0x7203 runnable [0x0000700005b38000]
> java.lang.Thread.State: RUNNABLE
> at org.apache.ignite.internal.processors.cache.persistence.evict.Random2LruPageEvictionTracker.evictDataPage(Random2LruPageEvictionTracker.java:152)
> at org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.ensureFreeSpace(IgniteCacheDatabaseSharedManager.java:1086)
> at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.ensureFreeSpace(GridCacheMapEntry.java:4513)
> at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerSet(GridCacheMapEntry.java:1461)
> at org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:745)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.localFinish(GridNearTxLocal.java:3850)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.doFinish(GridNearTxFinishFuture.java:440)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:390)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:4129)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:4118)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:399)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:354)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commitNearTxLocalAsync(GridNearTxLocal.java:4118)
> at org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commit(GridNearTxLocal.java:4086)
> at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:587)
> at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:556)
> at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.retryTopologySafe(DataStructuresProcessor.java:1664)
> at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.getAtomic(DataStructuresProcessor.java:556)
> at org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.reentrantLock(DataStructuresProcessor.java:1361)
> at org.apache.ignite.internal.IgniteKernal.reentrantLock(IgniteKernal.java:4136)
> at jdk.internal.reflect.GeneratedMethodAccessor713.invoke(Unknown Source)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at com.example.symphony.cmf.ignite.IgniteInitializer$1.invoke(IgniteInitializer.java:158)
> at com.sun.proxy.$Proxy205.reentrantLock(Unknown Source)
> at com.example.data.store.jdbc.cache.CacheService.getCount(CacheService.java:47)
> at com.example.data.store.jdbc.cache.CacheService$$FastClassBySpringCGLIB$$7efa9131.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)
> at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:105)
> at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:94)
> at jdk.internal.reflect.GeneratedMethodAccessor712.invoke(Unknown Source)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
> at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
> at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91)
> at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287)
> at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:164)
> at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:118)
> at org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:153)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
> at com.example.data.store.jdbc.cache.CacheService$$EnhancerBySpringCGLIB$$18de2c44.getCount()
> at com.example.data.store.jdbc.crud.Search.execute(Search.java:65)
> at com.example.data.store.jdbc.crud.Get.execute(Get.java:38)
> at com.example.data.store.jdbc.BaseDataChannel.lambda$get$6(BaseDataChannel.java:154)
> at com.example.data.store.jdbc.BaseDataChannel$$Lambda$2394/0x0000000801429040.apply(Unknown Source)
> at com.example.data.store.jdbc.DatabaseOperation.execute(DatabaseOperation.java:33)
> at com.example.data.store.jdbc.BaseDataChannel.execute(BaseDataChannel.java:236)
> at com.example.data.store.jdbc.BaseDataChannel.get(BaseDataChannel.java:153)
> at com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.lambda$get$6(MeteringStorageEngine.java:86)
> at com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData$$Lambda$2393/0x0000000801428c40.get(Unknown Source)
> at com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.execute(MeteringStorageEngine.java:156)
> at com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.get(MeteringStorageEngine.java:86)
> at com.example.data.core.store.data.DataAccess.get(DataAccess.java:319)
> at com.example.data.core.store.data.DataAccess$$FastClassBySpringCGLIB$$9d07c86a.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
> at com.example.data.core.store.data.DataAccess$$EnhancerBySpringCGLIB$$e3df238d_3.get()
> at com.example.data.core.impl.DataServiceImpl.get(DataServiceImpl.java:69)
> at com.example.data.web.v1.endpoint.DataController.lambda$get$2(DataController.java:171)
> at com.example.data.web.v1.endpoint.DataController$$Lambda$2310/0x000000080139b440.apply(Unknown Source)
> at com.example.data.web.v1.endpoint.DataController.getOrNotFound(DataController.java:202)
> at com.example.data.web.v1.endpoint.DataController.get(DataController.java:170)
> at com.example.data.web.v1.endpoint.DataController$$FastClassBySpringCGLIB$$4ee654da.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
> at com.example.data.web.v1.endpoint.DataController$$EnhancerBySpringCGLIB$$b6fe0edd_3.get()
> at jdk.internal.reflect.GeneratedMethodAccessor719.invoke(Unknown Source)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
> at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
> at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
> at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
> at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
> at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
> at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
> at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
> at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
> at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
> at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
> at org.springframework.test.web.servlet.TestDispatcherServlet.service(TestDispatcherServlet.java:72)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
> at org.springframework.mock.web.MockFilterChain$ServletFilterProxy.doFilter(MockFilterChain.java:167)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at com.example.library.authz.spring.UrlAccessFilter.doFilterInternal(UrlAccessFilter.java:67)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
> at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at com.example.symphony.csp.auth.filters.SymphonyOAuth2Filter.doFilter(SymphonyOAuth2Filter.java:87)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:155)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:114)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
> at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
> at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
> at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
> at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
> at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at io.opentracing.contrib.web.servlet.filter.TracingFilter.doFilter(TracingFilter.java:189)
> at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
> at org.springframework.test.web.servlet.MockMvc.perform(MockMvc.java:183)
> at com.example.data.web.v1.RenameFieldIT.addObject(RenameFieldIT.java:381)
> at com.example.data.web.v1.RenameFieldIT.testSchemaUpdate(RenameFieldIT.java:138)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
> at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
> at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:132)
> at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:124)
> at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:74)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$152/0x00000008001a4440.apply(Unknown Source)
> at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
> at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall$$Lambda$153/0x00000008001a3840.apply(Unknown Source)
> at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
> at org.junit.jupiter.engine.execution.ExecutableInvoker$$Lambda$364/0x00000008002dc840.apply(Unknown Source)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
> at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
> at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
> at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$1876/0x00000008011ee840.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
> at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService$$Lambda$222/0x00000008001e4040.accept(Unknown Source)
> at java.util.ArrayList.forEach(java.base@11.0.5/ArrayList.java:1540)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService$$Lambda$222/0x00000008001e4040.accept(Unknown Source)
> at java.util.ArrayList.forEach(java.base@11.0.5/ArrayList.java:1540)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown Source)
> at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
> at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
> at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
> at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
> at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
> at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:220)
> at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:188)
> at org.junit.platform.launcher.core.DefaultLauncher$$Lambda$175/0x00000008001c6c40.accept(Unknown Source)
> at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202)
> at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181)
> at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
> at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:102)
> at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:82)
> at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:78)
> at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
> at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
> at com.sun.proxy.$Proxy5.stop(Unknown Source)
> at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:132)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native Method)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
> at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
> at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
> at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
> at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.5/ThreadPoolExecutor.java:1128)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.5/ThreadPoolExecutor.java:628)
> at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
> at java.lang.Thread.run(java.base@11.0.5/Thread.java:834)
>
>
> ------------------------------
> Sent from the Apache Ignite Users mailing list archive
> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>
Re: Random2LruPageEvictionTracker causing hanging in our
integration tests
Posted by scottmf <sc...@gmail.com>.
Hi Anton,Just to be clear, the stack trace is from a thread dump that I took
while the process was hanging indefinitely.
Although I can reproduce this easily in my service, I can't share the code
with you. I'll attempt to get a generic use case to hang in this manner and
post it to github.
The full stack is below.
No use-case in this particular scenario for eviction. Like I said, it is
just for integration testing. My only concern is to ensure that there is no
bug that would hit us in production.
It is eviction for an in-memory cluster, no persistence, on-heap or near
cache.
"Test worker" #22 prio=5 os_prio=31 cpu=299703.41ms elapsed=317.18s
tid=0x00007ff3cfc8c800 nid=0x7203 runnable [0x0000700005b38000]
java.lang.Thread.State: RUNNABLE at
org.apache.ignite.internal.processors.cache.persistence.evict.Random2LruPageEvictionTracker.evictDataPage(Random2LruPageEvictionTracker.java:152)
at
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.ensureFreeSpace(IgniteCacheDatabaseSharedManager.java:1086)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.ensureFreeSpace(GridCacheMapEntry.java:4513)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerSet(GridCacheMapEntry.java:1461)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:745)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.localFinish(GridNearTxLocal.java:3850)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.doFinish(GridNearTxFinishFuture.java:440)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxFinishFuture.finish(GridNearTxFinishFuture.java:390)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:4129)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal$25.apply(GridNearTxLocal.java:4118)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:399)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:354)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commitNearTxLocalAsync(GridNearTxLocal.java:4118)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.commit(GridNearTxLocal.java:4086)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:587)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:556)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.retryTopologySafe(DataStructuresProcessor.java:1664)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.getAtomic(DataStructuresProcessor.java:556)
at
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.reentrantLock(DataStructuresProcessor.java:1361)
at
org.apache.ignite.internal.IgniteKernal.reentrantLock(IgniteKernal.java:4136)
at jdk.internal.reflect.GeneratedMethodAccessor713.invoke(Unknown Source) at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
com.example.symphony.cmf.ignite.IgniteInitializer$1.invoke(IgniteInitializer.java:158)
at com.sun.proxy.$Proxy205.reentrantLock(Unknown Source) at
com.example.data.store.jdbc.cache.CacheService.getCount(CacheService.java:47)
at
com.example.data.store.jdbc.cache.CacheService$$FastClassBySpringCGLIB$$7efa9131.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)
at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:105)
at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:94) at
jdk.internal.reflect.GeneratedMethodAccessor712.invoke(Unknown Source) at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
at
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91)
at
org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287)
at
org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:164)
at
org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:118)
at
org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:153)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
at
com.example.data.store.jdbc.cache.CacheService$$EnhancerBySpringCGLIB$$18de2c44.getCount()
at com.example.data.store.jdbc.crud.Search.execute(Search.java:65) at
com.example.data.store.jdbc.crud.Get.execute(Get.java:38) at
com.example.data.store.jdbc.BaseDataChannel.lambda$get$6(BaseDataChannel.java:154)
at
com.example.data.store.jdbc.BaseDataChannel$$Lambda$2394/0x0000000801429040.apply(Unknown
Source) at
com.example.data.store.jdbc.DatabaseOperation.execute(DatabaseOperation.java:33)
at
com.example.data.store.jdbc.BaseDataChannel.execute(BaseDataChannel.java:236)
at com.example.data.store.jdbc.BaseDataChannel.get(BaseDataChannel.java:153)
at
com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.lambda$get$6(MeteringStorageEngine.java:86)
at
com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData$$Lambda$2393/0x0000000801428c40.get(Unknown
Source) at
com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.execute(MeteringStorageEngine.java:156)
at
com.example.data.core.store.metrics.MeteringStorageEngine$MeteringData.get(MeteringStorageEngine.java:86)
at com.example.data.core.store.data.DataAccess.get(DataAccess.java:319) at
com.example.data.core.store.data.DataAccess$$FastClassBySpringCGLIB$$9d07c86a.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
at
com.example.data.core.store.data.DataAccess$$EnhancerBySpringCGLIB$$e3df238d_3.get()
at com.example.data.core.impl.DataServiceImpl.get(DataServiceImpl.java:69)
at
com.example.data.web.v1.endpoint.DataController.lambda$get$2(DataController.java:171)
at
com.example.data.web.v1.endpoint.DataController$$Lambda$2310/0x000000080139b440.apply(Unknown
Source) at
com.example.data.web.v1.endpoint.DataController.getOrNotFound(DataController.java:202)
at
com.example.data.web.v1.endpoint.DataController.get(DataController.java:170)
at
com.example.data.web.v1.endpoint.DataController$$FastClassBySpringCGLIB$$4ee654da.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:69)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
at
com.example.data.web.v1.endpoint.DataController$$EnhancerBySpringCGLIB$$b6fe0edd_3.get()
at jdk.internal.reflect.GeneratedMethodAccessor719.invoke(Unknown Source) at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) at
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at
org.springframework.test.web.servlet.TestDispatcherServlet.service(TestDispatcherServlet.java:72)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at
org.springframework.mock.web.MockFilterChain$ServletFilterProxy.doFilter(MockFilterChain.java:167)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
com.example.library.authz.spring.UrlAccessFilter.doFilterInternal(UrlAccessFilter.java:67)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
com.example.symphony.csp.auth.filters.SymphonyOAuth2Filter.doFilter(SymphonyOAuth2Filter.java:87)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:155)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:114)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
at
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at
io.opentracing.contrib.web.servlet.filter.TracingFilter.doFilter(TracingFilter.java:189)
at
org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at org.springframework.test.web.servlet.MockMvc.perform(MockMvc.java:183) at
com.example.data.web.v1.RenameFieldIT.addObject(RenameFieldIT.java:381) at
com.example.data.web.v1.RenameFieldIT.testSchemaUpdate(RenameFieldIT.java:138)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native
Method) at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:125)
at
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:132)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:124)
at
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:74)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$152/0x00000008001a4440.apply(Unknown
Source) at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at
org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall$$Lambda$153/0x00000008001a3840.apply(Unknown
Source) at
org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at
org.junit.jupiter.engine.execution.ExecutableInvoker$$Lambda$364/0x00000008002dc840.apply(Unknown
Source) at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:104)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:62)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:43)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:35)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at
org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:202)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$1876/0x00000008011ee840.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:198)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
at
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService$$Lambda$222/0x00000008001e4040.accept(Unknown
Source) at java.util.ArrayList.forEach(java.base@11.0.5/ArrayList.java:1540)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService$$Lambda$222/0x00000008001e4040.accept(Unknown
Source) at java.util.ArrayList.forEach(java.base@11.0.5/ArrayList.java:1540)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$218/0x00000008001e3040.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$217/0x00000008001e2c40.invoke(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$216/0x00000008001e2840.execute(Unknown
Source) at
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
at
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
at
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at
org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at
org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:220)
at
org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:188)
at
org.junit.platform.launcher.core.DefaultLauncher$$Lambda$175/0x00000008001c6c40.accept(Unknown
Source) at
org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
at
org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:102)
at
org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:82)
at
org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:78)
at
org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native
Method) at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
at
org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
at com.sun.proxy.$Proxy5.stop(Unknown Source) at
org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:132)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.5/Native
Method) at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.5/NativeMethodAccessorImpl.java:62)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.5/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.5/Method.java:566) at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
at
org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
at
org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
at
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.5/ThreadPoolExecutor.java:1128)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.5/ThreadPoolExecutor.java:628)
at
org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(java.base@11.0.5/Thread.java:834)
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Random2LruPageEvictionTracker causing hanging in our
integration tests
Posted by akurbanov <an...@gmail.com>.
Hello,
I have not seen this exact stacktrace before, is it possible to provide a
minimal reproducer to this, since it seems you are able to reproduce this
easily?
Also please make sure to provide the full thread dump from the JVM that
hangs and the data region configuration as well.
What is your use-case for using eviction policy?
Is it eviction for in-memory cluster, eviction for cluster with 3rd party
persistence, on-heap or near cache?
Best regards,
Anton
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/