You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datasketches.apache.org by GitBox <gi...@apache.org> on 2021/09/02 20:31:20 UTC
[GitHub] [datasketches-memory] davecromberge commented on a change in pull request #152: Update readme
davecromberge commented on a change in pull request #152:
URL: https://github.com/apache/datasketches-memory/pull/152#discussion_r701398543
##########
File path: README.md
##########
@@ -26,28 +26,104 @@
=================
# DataSketches Java Memory Component
- The goal of this component of the library is to provide high performance access to heap memory or
- native memory for primitives, primitive arrays, ByteBuffers and memory mapped files.
- This package is general purpose, has minimal external runtime dependencies and can be used in any
- application that needs to manage data structures outside the Java heap.
+The goal of this component of the DataSketches library is to provide high performance access to heap memory or native memory for primitives, primitive arrays, ByteBuffers and memory mapped files.
+This package is general purpose, has minimal external runtime dependencies and can be used in any
+application that needs to manage data structures outside the Java heap.
Review comment:
I do not know of any runtime dependencies - we removed the dependency on `slf4j`, and the module declaration does not require any modules outside of those provided by Java. Should this change to `no external runtime dependencies`?
##########
File path: README.md
##########
@@ -26,28 +26,104 @@
=================
# DataSketches Java Memory Component
- The goal of this component of the library is to provide high performance access to heap memory or
- native memory for primitives, primitive arrays, ByteBuffers and memory mapped files.
- This package is general purpose, has minimal external runtime dependencies and can be used in any
- application that needs to manage data structures outside the Java heap.
+The goal of this component of the DataSketches library is to provide high performance access to heap memory or native memory for primitives, primitive arrays, ByteBuffers and memory mapped files.
+This package is general purpose, has minimal external runtime dependencies and can be used in any
+application that needs to manage data structures outside the Java heap.
Please visit the main [DataSketches website](https://datasketches.apache.org) for more information.
-If you are interested in making contributions to this site please see our
-[Community](https://datasketches.apache.org/docs/Community/) page for how to contact us.
+If you are interested in making contributions to this Memory component please see our
+[Community](https://datasketches.apache.org/docs/Community/) page.
----
+## Release 2.0.0+
+With releases starting with 2.0.0, this Memory component supports Java 8 through Java 13.
-## Java Support
-Datasketches Memory currently supports Java 8 up to and including Java 13.
+Starting with Java 9, the Java language is built using the Java Platform Module System (JPMS).
+Because this Memory component leverages several JVM internal classes for improved performance in Java 8, we had to redesign the build system to allow the user to be able to use as well as develop with this component.
-__NOTE:__ You may have to provide additional JPMS arguments in order to use the library in Java 9 and above,
-see the [usage instructions](docs/usage-instructions.md) for details.
+__NOTE:__ You may have to provide additional JPMS arguments to the JVM in order to use the library in Java 9 and above as described in the following use cases and environments. Also see the [usage instructions](docs/usage-instructions.md) for more information.
----
-## Build Instructions
-__NOTE:__
+## USE AS A LIBRARY (using jars from Maven Central)
+In this environment, the user is using the Jars from Maven Central as a library and not attempting to build the source code or run the Memory component tests. Depending on how the user intends to use the Memory API, the Java version used to run the user's application and whether the user's application is a JPMS application or not, will determine if the user will need to supply arguments to the JVM running their application and what those arguments need to be.
+
+* API USE CASES
+ * Restricted API #1 - All On-heap
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * No off-heap ByteBuffers
+ * No allocations of off-heap memory
+ * No use of memory-mapped files
+ * Restricted API #2 - Add off-heap ByteBuffers
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * wrapped off-heap ByteBuffers
+ * No allocations of off-heap memory
+ * No use of memory-mapped files
+ * Full API
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * wrapped off-heap ByteBuffers
+ * allocations of off-heap memory
+ * use of memory-mapped files
+
+* CALLING APPLICATION JAVA VERSION & JPMS APPLICATION
+ * Running Java 8
+ * Running Java 9 - 13, non-JPMS application
+ * Running Java 9 - 13, JPMS application
+
+
+### Table of API Use Cases Compared to Application Java Version and JPMS Application, and Possible JVM Arguments Required
+
+| Java Version & JPMS | Restricted API 1 | Restricted API 2 API | Full API |
+|:------------------------:|:----------------:|:--------------------:|:--------:|
+| Java 8 | Args 1 | Args 1 | Args 1 |
+| Java 9-13, non-JPMS | Args 1 | Args 1 | Args 3 |
+| Java 9-13, JPMS | Args 1 | Args 2 | Args 4 |
+
+### Args 1
+No JVM arguments required
+
+### Args 2
+User must supply the following argument to the JVM:
+
+* --add-opens java.base/sun.nio.ch=org.apache.datasketches.memory
+
+### Args 3
+User must supply the following arguments to the JVM:
+
+* --add-exports java.base/jdk.internal.misc=ALL-UNNAMED
+* --add-exports java.base/jdk.internal.ref=ALL-UNNAMED
+* --add-opens java.base/java.nio=ALL-UNNAMED
+* --add-opens java.base/sun.nio.ch=ALL-UNNAMED
+
+### Args 4
+User must supply the following arguments to the JVM:
+
+* --add-exports java.base/jdk.internal.misc=org.apache.datasketches.memory
+* --add-exports java.base/jdk.internal.ref=org.apache.datasketches.memory
+* --add-opens java.base/java.nio=org.apache.datasketches.memory
+* --add-opens java.base/sun.nio.ch=org.apache.datasketches.memory
+
+
+## DEVELOPER USAGE
+In this environent the developer needs to build the Memory component from source and run the Memory Component tests. There are two use cases. The first is for a *System Developer* that needs to build and test their own Jar from source for a specific Java version. The second use case is for a *Memory Component Developer and Contributor*.
+
+* System Developer
+ * Compile, test and create a Jar for a specific Java version
+ * use the privided script for this purpose
Review comment:
I'll correct the small typo to `provided`.
##########
File path: README.md
##########
@@ -26,28 +26,104 @@
=================
# DataSketches Java Memory Component
- The goal of this component of the library is to provide high performance access to heap memory or
- native memory for primitives, primitive arrays, ByteBuffers and memory mapped files.
- This package is general purpose, has minimal external runtime dependencies and can be used in any
- application that needs to manage data structures outside the Java heap.
+The goal of this component of the DataSketches library is to provide high performance access to heap memory or native memory for primitives, primitive arrays, ByteBuffers and memory mapped files.
+This package is general purpose, has minimal external runtime dependencies and can be used in any
+application that needs to manage data structures outside the Java heap.
Please visit the main [DataSketches website](https://datasketches.apache.org) for more information.
-If you are interested in making contributions to this site please see our
-[Community](https://datasketches.apache.org/docs/Community/) page for how to contact us.
+If you are interested in making contributions to this Memory component please see our
+[Community](https://datasketches.apache.org/docs/Community/) page.
----
+## Release 2.0.0+
+With releases starting with 2.0.0, this Memory component supports Java 8 through Java 13.
-## Java Support
-Datasketches Memory currently supports Java 8 up to and including Java 13.
+Starting with Java 9, the Java language is built using the Java Platform Module System (JPMS).
+Because this Memory component leverages several JVM internal classes for improved performance in Java 8, we had to redesign the build system to allow the user to be able to use as well as develop with this component.
-__NOTE:__ You may have to provide additional JPMS arguments in order to use the library in Java 9 and above,
-see the [usage instructions](docs/usage-instructions.md) for details.
+__NOTE:__ You may have to provide additional JPMS arguments to the JVM in order to use the library in Java 9 and above as described in the following use cases and environments. Also see the [usage instructions](docs/usage-instructions.md) for more information.
----
-## Build Instructions
-__NOTE:__
+## USE AS A LIBRARY (using jars from Maven Central)
+In this environment, the user is using the Jars from Maven Central as a library and not attempting to build the source code or run the Memory component tests. Depending on how the user intends to use the Memory API, the Java version used to run the user's application and whether the user's application is a JPMS application or not, will determine if the user will need to supply arguments to the JVM running their application and what those arguments need to be.
+
+* API USE CASES
+ * Restricted API #1 - All On-heap
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * No off-heap ByteBuffers
+ * No allocations of off-heap memory
+ * No use of memory-mapped files
+ * Restricted API #2 - Add off-heap ByteBuffers
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * wrapped off-heap ByteBuffers
+ * No allocations of off-heap memory
+ * No use of memory-mapped files
+ * Full API
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * wrapped off-heap ByteBuffers
+ * allocations of off-heap memory
+ * use of memory-mapped files
+
+* CALLING APPLICATION JAVA VERSION & JPMS APPLICATION
+ * Running Java 8
+ * Running Java 9 - 13, non-JPMS application
+ * Running Java 9 - 13, JPMS application
+
+
+### Table of API Use Cases Compared to Application Java Version and JPMS Application, and Possible JVM Arguments Required
+
+| Java Version & JPMS | Restricted API 1 | Restricted API 2 API | Full API |
+|:------------------------:|:----------------:|:--------------------:|:--------:|
+| Java 8 | Args 1 | Args 1 | Args 1 |
+| Java 9-13, non-JPMS | Args 1 | Args 1 | Args 3 |
+| Java 9-13, JPMS | Args 1 | Args 2 | Args 4 |
+
+### Args 1
+No JVM arguments required
+
+### Args 2
+User must supply the following argument to the JVM:
+
+* --add-opens java.base/sun.nio.ch=org.apache.datasketches.memory
+
+### Args 3
+User must supply the following arguments to the JVM:
+
+* --add-exports java.base/jdk.internal.misc=ALL-UNNAMED
+* --add-exports java.base/jdk.internal.ref=ALL-UNNAMED
+* --add-opens java.base/java.nio=ALL-UNNAMED
+* --add-opens java.base/sun.nio.ch=ALL-UNNAMED
+
+### Args 4
+User must supply the following arguments to the JVM:
+
+* --add-exports java.base/jdk.internal.misc=org.apache.datasketches.memory
+* --add-exports java.base/jdk.internal.ref=org.apache.datasketches.memory
+* --add-opens java.base/java.nio=org.apache.datasketches.memory
+* --add-opens java.base/sun.nio.ch=org.apache.datasketches.memory
Review comment:
I seem to recall that the opens to `sun.nio.ch` is not strictly required for allocations of off-heap memory, unless memory-mapped files are used.
##########
File path: README.md
##########
@@ -26,28 +26,104 @@
=================
# DataSketches Java Memory Component
- The goal of this component of the library is to provide high performance access to heap memory or
- native memory for primitives, primitive arrays, ByteBuffers and memory mapped files.
- This package is general purpose, has minimal external runtime dependencies and can be used in any
- application that needs to manage data structures outside the Java heap.
+The goal of this component of the DataSketches library is to provide high performance access to heap memory or native memory for primitives, primitive arrays, ByteBuffers and memory mapped files.
+This package is general purpose, has minimal external runtime dependencies and can be used in any
+application that needs to manage data structures outside the Java heap.
Please visit the main [DataSketches website](https://datasketches.apache.org) for more information.
-If you are interested in making contributions to this site please see our
-[Community](https://datasketches.apache.org/docs/Community/) page for how to contact us.
+If you are interested in making contributions to this Memory component please see our
+[Community](https://datasketches.apache.org/docs/Community/) page.
----
+## Release 2.0.0+
+With releases starting with 2.0.0, this Memory component supports Java 8 through Java 13.
-## Java Support
-Datasketches Memory currently supports Java 8 up to and including Java 13.
+Starting with Java 9, the Java language is built using the Java Platform Module System (JPMS).
+Because this Memory component leverages several JVM internal classes for improved performance in Java 8, we had to redesign the build system to allow the user to be able to use as well as develop with this component.
-__NOTE:__ You may have to provide additional JPMS arguments in order to use the library in Java 9 and above,
-see the [usage instructions](docs/usage-instructions.md) for details.
+__NOTE:__ You may have to provide additional JPMS arguments to the JVM in order to use the library in Java 9 and above as described in the following use cases and environments. Also see the [usage instructions](docs/usage-instructions.md) for more information.
----
-## Build Instructions
-__NOTE:__
+## USE AS A LIBRARY (using jars from Maven Central)
+In this environment, the user is using the Jars from Maven Central as a library and not attempting to build the source code or run the Memory component tests. Depending on how the user intends to use the Memory API, the Java version used to run the user's application and whether the user's application is a JPMS application or not, will determine if the user will need to supply arguments to the JVM running their application and what those arguments need to be.
+
+* API USE CASES
+ * Restricted API #1 - All On-heap
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * No off-heap ByteBuffers
+ * No allocations of off-heap memory
+ * No use of memory-mapped files
+ * Restricted API #2 - Add off-heap ByteBuffers
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * wrapped off-heap ByteBuffers
+ * No allocations of off-heap memory
+ * No use of memory-mapped files
+ * Full API
+ * wrapped heap arrays
+ * wrapped on-heap ByteBuffers
+ * wrapped off-heap ByteBuffers
+ * allocations of off-heap memory
+ * use of memory-mapped files
+
+* CALLING APPLICATION JAVA VERSION & JPMS APPLICATION
+ * Running Java 8
+ * Running Java 9 - 13, non-JPMS application
+ * Running Java 9 - 13, JPMS application
+
+
+### Table of API Use Cases Compared to Application Java Version and JPMS Application, and Possible JVM Arguments Required
+
+| Java Version & JPMS | Restricted API 1 | Restricted API 2 API | Full API |
+|:------------------------:|:----------------:|:--------------------:|:--------:|
+| Java 8 | Args 1 | Args 1 | Args 1 |
+| Java 9-13, non-JPMS | Args 1 | Args 1 | Args 3 |
+| Java 9-13, JPMS | Args 1 | Args 2 | Args 4 |
+
+### Args 1
+No JVM arguments required
+
+### Args 2
+User must supply the following argument to the JVM:
+
+* --add-opens java.base/sun.nio.ch=org.apache.datasketches.memory
+
+### Args 3
+User must supply the following arguments to the JVM:
+
+* --add-exports java.base/jdk.internal.misc=ALL-UNNAMED
+* --add-exports java.base/jdk.internal.ref=ALL-UNNAMED
+* --add-opens java.base/java.nio=ALL-UNNAMED
+* --add-opens java.base/sun.nio.ch=ALL-UNNAMED
+
+### Args 4
+User must supply the following arguments to the JVM:
+
+* --add-exports java.base/jdk.internal.misc=org.apache.datasketches.memory
+* --add-exports java.base/jdk.internal.ref=org.apache.datasketches.memory
+* --add-opens java.base/java.nio=org.apache.datasketches.memory
+* --add-opens java.base/sun.nio.ch=org.apache.datasketches.memory
+
+
+## DEVELOPER USAGE
+In this environent the developer needs to build the Memory component from source and run the Memory Component tests. There are two use cases. The first is for a *System Developer* that needs to build and test their own Jar from source for a specific Java version. The second use case is for a *Memory Component Developer and Contributor*.
+
+* System Developer
+ * Compile, test and create a Jar for a specific Java version
+ * use the privided script for this purpose
+
+* Memory Component Developer / Contributor
+ * Compile & test the library from source code using:
+ * Eclipse (version)
+ * IntelliJ (version)
+ * Maven (version)
+ * Command-line
Review comment:
Does command-line here refer to the provided script?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org