You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datasketches.apache.org by GitBox <gi...@apache.org> on 2021/09/02 22:42:27 UTC
[GitHub] [datasketches-memory] leerho commented on a change in pull request #152: Update readme
leerho commented on a change in pull request #152:
URL: https://github.com/apache/datasketches-memory/pull/152#discussion_r701469039
##########
File path: README.md
##########
@@ -26,28 +26,104 @@
=================
# DataSketches Java Memory Component
- The goal of this component of the library is to provide high performance access to heap memory or
- native memory for primitives, primitive arrays, ByteBuffers and memory mapped files.
- This package is general purpose, has minimal external runtime dependencies and can be used in any
- application that needs to manage data structures outside the Java heap.
+The goal of this component of the DataSketches library is to provide high performance access to heap memory or native memory for primitives, primitive arrays, ByteBuffers and memory mapped files.
+This package is general purpose, has minimal external runtime dependencies and can be used in any
+application that needs to manage data structures outside the Java heap.
Please visit the main [DataSketches website](https://datasketches.apache.org) for more information.
-If you are interested in making contributions to this site please see our
-[Community](https://datasketches.apache.org/docs/Community/) page for how to contact us.
+If you are interested in making contributions to this Memory component please see our
+[Community](https://datasketches.apache.org/docs/Community/) page.
----
+## Release 2.0.0+
+With releases starting with 2.0.0, this Memory component supports Java 8 through Java 13.
-## Java Support
-Datasketches Memory currently supports Java 8 up to and including Java 13.
+Starting with Java 9, the Java language is built using the Java Platform Module System (JPMS).
+Because this Memory component leverages several JVM internal classes for improved performance in Java 8, we had to redesign the build system to allow the user to be able to use as well as develop with this component.
-__NOTE:__ You may have to provide additional JPMS arguments in order to use the library in Java 9 and above,
-see the [usage instructions](docs/usage-instructions.md) for details.
+__NOTE:__ You may have to provide additional JPMS arguments to the JVM in order to use the library in Java 9 and above as described in the following use cases and environments. Also see the [usage instructions](docs/usage-instructions.md) for more information.
----
-## Build Instructions
-__NOTE:__
+## USE AS A LIBRARY (using jars from Maven Central)
+In this environment, the user is using the Jars from Maven Central as a library and not attempting to build the source code or run the Memory component tests. Depending on how the user intends to use the Memory API, the Java version used to run the user's application and whether the user's application is a JPMS application or not, will determine if the user will need to supply arguments to the JVM running their application and what those arguments need to be.
+
+* API USE CASES
+ * Restricted API #1 - All On-heap
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * No off-heap ByteBuffers
+ * No allocations of off-heap memory
+ * No use of memory-mapped files
+ * Restricted API #2 - Add off-heap ByteBuffers
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * wrapped off-heap ByteBuffers
+ * No allocations of off-heap memory
+ * No use of memory-mapped files
+ * Full API
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * wrapped off-heap ByteBuffers
+ * allocations of off-heap memory
+ * use of memory-mapped files
+
+* CALLING APPLICATION JAVA VERSION & JPMS APPLICATION
+ * Running Java 8
+ * Running Java 9 - 13, non-JPMS application
+ * Running Java 9 - 13, JPMS application
+
+
+### Table of API Use Cases Compared to Application Java Version and JPMS Application, and Possible JVM Arguments Required
+
+| Java Version & JPMS | Restricted API 1 | Restricted API 2 API | Full API |
+|:------------------------:|:----------------:|:--------------------:|:--------:|
+| Java 8 | Args 1 | Args 1 | Args 1 |
+| Java 9-13, non-JPMS | Args 1 | Args 1 | Args 3 |
+| Java 9-13, JPMS | Args 1 | Args 2 | Args 4 |
+
+### Args 1
+No JVM arguments required
+
+### Args 2
+User must supply the following argument to the JVM:
+
+* --add-opens java.base/sun.nio.ch=org.apache.datasketches.memory
+
+### Args 3
+User must supply the following arguments to the JVM:
+
+* --add-exports java.base/jdk.internal.misc=ALL-UNNAMED
+* --add-exports java.base/jdk.internal.ref=ALL-UNNAMED
+* --add-opens java.base/java.nio=ALL-UNNAMED
+* --add-opens java.base/sun.nio.ch=ALL-UNNAMED
+
+### Args 4
+User must supply the following arguments to the JVM:
+
+* --add-exports java.base/jdk.internal.misc=org.apache.datasketches.memory
+* --add-exports java.base/jdk.internal.ref=org.apache.datasketches.memory
+* --add-opens java.base/java.nio=org.apache.datasketches.memory
+* --add-opens java.base/sun.nio.ch=org.apache.datasketches.memory
Review comment:
`sun.nio.ch` is also used for direct ByteBuffers. But memory-mapped files I think also requires more than just `sun.nio.ch`. I'd have to test it.
Nonetheless, this would create yet another restricted API subset combination for the user to understand. The reason I pulled out the ByteBuffer use case is because that is a major one that Druid uses and may likely be used by other platforms. Is there a use case for just memory-mapped files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org