You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Alexey Kukushkin (Jira)" <ji...@apache.org> on 2021/11/19 08:28:00 UTC
[jira] [Created] (IGNITE-15959) Remove operation fetches entry value into heap
Alexey Kukushkin created IGNITE-15959:
-----------------------------------------
Summary: Remove operation fetches entry value into heap
Key: IGNITE-15959
URL: https://issues.apache.org/jira/browse/IGNITE-15959
Project: Ignite
Issue Type: Improvement
Affects Versions: 2.11
Reporter: Alexey Kukushkin
{{IgniteCache#remove(key)}} operation fetches full entry into heap memory. This makes Ignite inefficient with handling large objects: our application running with limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap space}} when trying to remove an entry by key.
It seems wrong that Ignite needs to fetch the full entry on heap to remove the entry. Please enhance Ignite to not be doing that or explain why Ignite must do that.
h2. Reproducer
h3. Steps
Create a Gradle project with the below class and run it as
{{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
{{build.gradle}}
{code:groovy}
test {
minHeapSize = "512m"
maxHeapSize = "512m"
}
{code}
{{RemoveOperationHeapUsage.java}}
{code:java}
public class RemoveOperationHeapUsage {
/** Run the test with -Xmx512m -Xms512m */
@Test
public void removeOperationFetchesValueOnHeap() {
var igniteCfg = new IgniteConfiguration()
.setDiscoverySpi(
new TcpDiscoverySpi()
.setIpFinder(new TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
)
.setCacheConfiguration(new CacheConfiguration<>("blobs"));
try (var ignite = Ignition.start(igniteCfg)) {
Cache<Integer, byte[]> cache = ignite.cache("blobs");
// Put a BLOB having size of 40% of free memory to the cache
Runtime.getRuntime().gc();
var freeMemory = Runtime.getRuntime().freeMemory();
var blobSize = (int)(freeMemory * 0.4);
putBlob(cache, blobSize);
// Use 80% of the free heap
Runtime.getRuntime().gc();
var blob1 = new byte[2 * blobSize];
// Remove the blob from the cache.
// This throws OutOfMemoryError: Java heap space since Ignite retrieves full entry to the heap.
// Why does Ignite retrieve entry value to delete the entry?
cache.remove(1);
}
}
private static void putBlob(Cache<Integer, byte[]> cache, int blobSize) {
var blob = new byte[blobSize];
cache.put(1, blob);
}
}
{code}
h3. Expected
The test passes
h3. Actual
The {{cache.remove}} operatoin fails with:
{noformat}
java.lang.OutOfMemoryError: Java heap space
at org.apache.ignite.internal.processors.cache.IncompleteCacheObject.<init>(IncompleteCacheObject.java:44)
at org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
at org.apache.ignite.internal.processors.cache.tree.DataRow.<init>(DataRow.java:55)
at org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5921)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:290)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5907)
at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:174)
at org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:397)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.read(BPlusTree.java:6108)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1991)
at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1920)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1765)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1748)
at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:441)
at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2342)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2589)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2049)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1866)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1725)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:306)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:487)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)