You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by yangjiajun <13...@qq.com> on 2019/05/07 08:52:08 UTC
[data-streamer-stripe-2-#35][] Critical system error detected.
Hello.
My ignite crash with following errors.
[16:39:31,163][SEVERE][data-streamer-stripe-2-#35][] Critical system error
detected. Will be handled accordingly to configured handler
[hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext
[type=CRITICAL_ERROR, err=class
o.a.i.i.processors.cache.persistence.StorageException: Failed to initialize
partition file:
/XXXX/data/persistence/node00-51465194-9ac4-4c65-b156-9d61d3cd3299/cache-SQL_PUBLIC_TEMP_TABLE_94_1_5_20190507163914_94/part-439.bin]]
class
org.apache.ignite.internal.processors.cache.persistence.StorageException:
Failed to initialize partition file:
/XXXX/ignite/data/persistence/node00-51465194-9ac4-4c65-b156-9d61d3cd3299/cache-SQL_PUBLIC_TEMP_TABLE_94_1_5_20190507163914_94/part-439.bin
at
org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.init(FilePageStore.java:466)
at
org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:355)
at
org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:490)
at
org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:474)
at
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:857)
at
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:698)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.getOrAllocatePartitionMetas(GridCacheOffheapManager.java:1657)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.init0(GridCacheOffheapManager.java:1494)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:2136)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:457)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2357)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2584)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2041)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1870)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1690)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:317)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:504)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:464)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:266)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1175)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:633)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2517)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2494)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1306)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:835)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerCacheUpdaters$Individual.receive(DataStreamerCacheUpdaters.java:138)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:157)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6836)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:984)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:137)
at
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:522)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:137)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.file.FileSystemException:
/data01/eippoc/ignite/data/persistence/node00-51465194-9ac4-4c65-b156-9d61d3cd3299/cache-SQL_PUBLIC_TEMP_TABLE_94_1_5_20190507163914_94/part-439.bin:
Too many open files
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at
sun.nio.fs.UnixFileSystemProvider.newAsynchronousFileChannel(UnixFileSystemProvider.java:196)
at
java.nio.channels.AsynchronousFileChannel.open(AsynchronousFileChannel.java:248)
at
java.nio.channels.AsynchronousFileChannel.open(AsynchronousFileChannel.java:301)
at
org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIO.<init>(AsyncFileIO.java:74)
at
org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory.create(AsyncFileIOFactory.java:61)
at
org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.init(FilePageStore.java:438)
... 32 more
My ignite version is 2.7.I use 'SET STREAMING ON ORDERED ALLOW_OVERWRITE
ON;' to write data to ignite.
Here is the thread dump:
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: [data-streamer-stripe-2-#35][] Critical system error detected.
Posted by Evgenii Zhuravlev <e....@gmail.com>.
Hi,
Here is the documentation regarding this message:
https://apacheignite.readme.io/v2.7/docs/jvm-and-system-tuning#section-file-descriptors
Evgenii
вт, 7 мая 2019 г. в 12:09, yangjiajun <13...@qq.com>:
> thread_dump.rar
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2059/thread_dump.rar>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: [data-streamer-stripe-2-#35][] Critical system error detected.
Posted by yangjiajun <13...@qq.com>.
thread_dump.rar
<http://apache-ignite-users.70518.x6.nabble.com/file/t2059/thread_dump.rar>
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/