You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/02/02 20:29:34 UTC
[jira] [Commented] (FLINK-1320) Add an off-heap variant of the
managed memory
[ https://issues.apache.org/jira/browse/FLINK-1320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14301746#comment-14301746 ]
ASF GitHub Bot commented on FLINK-1320:
---------------------------------------
Github user hsaputra commented on a diff in the pull request:
https://github.com/apache/flink/pull/290#discussion_r23950666
--- Diff: flink-core/src/main/java/org/apache/flink/core/memory/DirectMemorySegment.java ---
@@ -0,0 +1,595 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.core.memory;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.lang.reflect.Field;
+import java.nio.BufferOverflowException;
+import java.nio.BufferUnderflowException;
+import java.nio.ByteBuffer;
+
+/**
+ * This class uses in parts code from Java's direct byte buffer API.
+ *
+ * The use in this class two crucial additions:
+ * - It uses collapsed checks for range check and memory segment disposal.
+ * - It offers absolute positioning methods for byte array put/get methods, to guarantee thread safe use.
+ *
+ * In addition, the code that uses this class should make sure that only one implementation class is ever loaded -
+ * Either the {@link HeapMemorySegment}, or this DirectMemorySegment. That way, all the abstract methods in the
+ * MemorySegment base class have only one loaded actual implementation. This is easy for the JIT to recognize through
+ * class hierarchy analysis, or by identifying that the invocations are monomorphic (all go to the same concrete
+ * method implementation). Under this precondition, the JIT can perfectly inline methods.
+ *
+ * This is harder to do and control with byte buffers, where different code paths use different versions of the class
+ * (heap, direct, mapped) and thus virtual method invocations are polymorphic and are not as easily inlined.
+ */
+public class DirectMemorySegment extends MemorySegment {
+
+ /** The direct byte buffer that allocated the memory */
+ protected final ByteBuffer buffer;
+
+ /** The address to the off-heap data */
+ private long address;
+
+ /** The address one byte after the last addressable byte.
+ * This is address + size while the segment is not disposed */
+ private final long addressLimit;
+
+ /** The size in bytes of the memory segment */
+ private final int size;
+
+ // -------------------------------------------------------------------------
+ // Constructors
+ // -------------------------------------------------------------------------
+
+ public DirectMemorySegment(int size) {
+ this(ByteBuffer.allocateDirect(size));
+ }
+
+ public DirectMemorySegment(ByteBuffer buffer) {
+ if (buffer == null || !buffer.isDirect()) {
+ throw new IllegalArgumentException();
--- End diff --
Put message to the exception to help diagnose problem.
> Add an off-heap variant of the managed memory
> ---------------------------------------------
>
> Key: FLINK-1320
> URL: https://issues.apache.org/jira/browse/FLINK-1320
> Project: Flink
> Issue Type: Improvement
> Components: Local Runtime
> Reporter: Stephan Ewen
> Assignee: Max Michels
> Priority: Minor
>
> For (nearly) all memory that Flink accumulates (in the form of sort buffers, hash tables, caching), we use a special way of representing data serialized across a set of memory pages. The big work lies in the way the algorithms are implemented to operate on pages, rather than on objects.
> The core class for the memory is the {{MemorySegment}}, which has all methods to set and get primitives values efficiently. It is a somewhat simpler (and faster) variant of a HeapByteBuffer.
> As such, it should be straightforward to create a version where the memory segment is not backed by a heap byte[], but by memory allocated outside the JVM, in a similar way as the NIO DirectByteBuffers, or the Netty direct buffers do it.
> This may have multiple advantages:
> - We reduce the size of the JVM heap (garbage collected) and the number and size of long living alive objects. For large JVM sizes, this may improve performance quite a bit. Utilmately, we would in many cases reduce JVM size to 1/3 to 1/2 and keep the remaining memory outside the JVM.
> - We save copies when we move memory pages to disk (spilling) or through the network (shuffling / broadcasting / forward piping)
> The changes required to implement this are
> - Add a {{UnmanagedMemorySegment}} that only stores the memory adress as a long, and the segment size. It is initialized from a DirectByteBuffer.
> - Allow the MemoryManager to allocate these MemorySegments, instead of the current ones.
> - Make sure that the startup script pick up the mode and configure the heap size and the max direct memory properly.
> Since the MemorySegment is probably the most performance critical class in Flink, we must take care that we do this right. The following are critical considerations:
> - If we want both solutions (heap and off-heap) to exist side-by-side (configurable), we must make the base MemorySegment abstract and implement two versions (heap and off-heap).
> - To get the best performance, we need to make sure that only one class gets loaded (or at least ever used), to ensure optimal JIT de-virtualization and inlining.
> - We should carefully measure the performance of both variants. From previous micro benchmarks, I remember that individual byte accesses in DirectByteBuffers (off-heap) were slightly slower than on-heap, any larger accesses were equally good or slightly better.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)