You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by sh...@apache.org on 2024/02/12 18:17:46 UTC
(daffodil-extra) branch main updated (cc7f52e -> 5b593f5)
This is an automated email from the ASF dual-hosted git repository.
shanedell pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/daffodil-extra.git
discard cc7f52e remove StdinSeek and instead loop over reading of stdin
discard 6237b5f remove CustomSeek code, rename read_seek.rs to stdin_seek.rs
discard 1730581 remove Cargo.lock from git, add to gitignore
discard 6154804 Add lsbfdump example using Scala and Rust
omit 3eb1acc First version (0.1.0) of lsbfdump tool
omit a026e2e Add content to README
omit 66b055f Merge pull request #1 from stevedlawrence/daffodil-extra-license
new 62e7b9f Add content to README
new 2eea695 First version (0.1.0) of lsbfdump tool
new 5b593f5 Add lsbfdump example using Scala and Rust
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (cc7f52e)
\
N -- N -- N refs/heads/main (5b593f5)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
(daffodil-extra) 01/03: Add content to README
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shanedell pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/daffodil-extra.git
commit 62e7b9f673adf0c3bc81d5cb32818eace38dcd96
Author: Michael Beckerle <mb...@apache.org>
AuthorDate: Thu Dec 15 17:47:01 2022 -0500
Add content to README
---
README.md | 20 +++++++++++++++++++-
1 file changed, 19 insertions(+), 1 deletion(-)
diff --git a/README.md b/README.md
index 43df87d..a161c34 100644
--- a/README.md
+++ b/README.md
@@ -1 +1,19 @@
-# daffodil-extra
+# Daffodil Extra
+
+This is a repository for Apache Daffodil related things such as:
+- examples
+- utilities/tools
+
+These are not included in releases of Apache Daffodil.
+
+DFDL Schema projects would *not* normally be found here.
+They should have separate git repositories.
+
+This repository is intended to supercede the github openDFDL project
+for most things.
+
+
+
+
+
+
(daffodil-extra) 02/03: First version (0.1.0) of lsbfdump tool
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shanedell pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/daffodil-extra.git
commit 2eea695d6e67eb9193de3e1d792b396801351584
Author: Michael Beckerle <mb...@apache.org>
AuthorDate: Wed Dec 13 12:22:58 2023 -0500
First version (0.1.0) of lsbfdump tool
This version contains code initially generated by Mike Beckerle (mbeckerle)
using: ChatGPT 4.0 on 2023-12-13 but subsequently further modified by hand.
ChatGPT 4.0 provided this statement with respect to originality of its contribution:
> There are no direct citations or sources for this code, as **it was not taken from
> copyrighted material or external sources.**
---
lsbfdump/.gitignore | 27 +++
lsbfdump/LICENSE | 202 ++++++++++++++++++++
lsbfdump/README.md | 75 ++++++++
lsbfdump/build.sbt | 20 ++
lsbfdump/project/build.properties | 1 +
lsbfdump/project/plugins.sbt | 18 ++
.../org/apache/daffodil/lsbfDump/LSBFDump.scala | 203 +++++++++++++++++++++
lsbfdump/src/test/resources/testData1.txt | 1 +
.../apache/daffodil/lsbfDump/LSBFDumpTest.scala | 126 +++++++++++++
9 files changed, 673 insertions(+)
diff --git a/lsbfdump/.gitignore b/lsbfdump/.gitignore
new file mode 100644
index 0000000..ae03a39
--- /dev/null
+++ b/lsbfdump/.gitignore
@@ -0,0 +1,27 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# NOTE: This file should not list entries for operating system or IDE related
+# files. To ignore those files, entries should be added to one of the following
+# locations, as described in the "gitignore" man page:
+#
+# $GIT_DIR/info/exclude
+# $XDG_CONFIG_HOME/git/ignore
+# $HOME/.config/git/ignore
+
+target
+
+# Directory auto-generated by the sbt console for build server protocol
+.bsp
diff --git a/lsbfdump/LICENSE b/lsbfdump/LICENSE
new file mode 100644
index 0000000..d645695
--- /dev/null
+++ b/lsbfdump/LICENSE
@@ -0,0 +1,202 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
diff --git a/lsbfdump/README.md b/lsbfdump/README.md
new file mode 100644
index 0000000..3d31aa2
--- /dev/null
+++ b/lsbfdump/README.md
@@ -0,0 +1,75 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+# lsbfDump - Least Significant Bit First Bit Dump Utility
+
+Creates a data dump at bits level for data that has dfdl:bitOrder="leastSignificantBitFirst".
+
+For example:
+
+```
+01000110 01001100 01000101 01111111 | 0x00000000
+00000000 00000001 00000001 00000010 | 0x00000004
+00000000 00000000 00000000 00000000 | 0x00000008
+00000000 00000000 00000000 00000000 | 0x0000000C
+00000000 00111110 00000000 00000011 | 0x00000010
+00000000 00000000 00000000 00000001 | 0x00000014
+```
+The address (in hex) is on the right. The bytes start on the right and increase moving left and downward.
+The least significant bit of each byte is on the right (as people usually write numbers).
+
+The purpose of this is for use with data where the bit positions are numbered from right to left. I.e.,
+the first bit (position 0, or position 1 if using 1-based indexing) in each byte is the rightmost bit.
+
+# License
+
+This is open-source software under the Apache Software License (V2.0). See the LICENSE file for the full text.
+
+# Running
+
+After building, the native executable will be in the target/native-image directory, named lsbfdump.
+It can be copied/moved to wherever you want it, likely to some directory on the PATH so that it can be used
+conveniently.
+
+Instructions on how to use can be obtained by running `target/native-image/lsbfdump --help`.
+
+# Building
+
+This is written in Scala (2.13) and compiled and deployed using GraalVM to create a fast-starting executable intended for command line use.
+
+The command
+
+```
+sbt nativeImage
+```
+
+Creates a native image under target/native, named lsbfdump.
+
+Note that a huge amount of stuff including an entire GraalVM SDK downloads on first build using 'sbt nativeImage'.
+
+# Release Notes
+
+## Version 0.1.0
+
+This version contains code initially generated by Mike Beckerle (mbeckerle)
+using: ChatGPT 4.0 on 2023-12-13 but subsequently further modified by hand.
+
+ChatGPT 4.0 provided this statement with respect to originality of its contribution:
+
+> There are no direct citations or sources for this code, as **it was not taken from
+> copyrighted material or external sources.**
+
+
\ No newline at end of file
diff --git a/lsbfdump/build.sbt b/lsbfdump/build.sbt
new file mode 100644
index 0000000..2699d1a
--- /dev/null
+++ b/lsbfdump/build.sbt
@@ -0,0 +1,20 @@
+version := "0.1.0"
+
+scalaVersion := "2.13.12"
+
+enablePlugins(NativeImagePlugin)
+
+// Define the main class of your application
+Compile / mainClass := Some("org.apache.daffodil.lsbfDump.LSBFDump")
+
+// Settings for the native image
+nativeImageOptions ++= Seq(
+ "--no-fallback", // does not create nativeImage if it will require a JVM.
+// "--enable-all-security-services"
+// Add other native-image options as needed
+)
+
+// Library dependencies
+libraryDependencies ++= Seq(
+ "junit" % "junit" % "4.13.2" % Test,
+)
diff --git a/lsbfdump/project/build.properties b/lsbfdump/project/build.properties
new file mode 100644
index 0000000..b19d4e1
--- /dev/null
+++ b/lsbfdump/project/build.properties
@@ -0,0 +1 @@
+sbt.version = 1.9.7
diff --git a/lsbfdump/project/plugins.sbt b/lsbfdump/project/plugins.sbt
new file mode 100644
index 0000000..e9ec367
--- /dev/null
+++ b/lsbfdump/project/plugins.sbt
@@ -0,0 +1,18 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+addSbtPlugin("org.scalameta" % "sbt-native-image" % "0.3.4")
diff --git a/lsbfdump/src/main/scala/org/apache/daffodil/lsbfDump/LSBFDump.scala b/lsbfdump/src/main/scala/org/apache/daffodil/lsbfDump/LSBFDump.scala
new file mode 100644
index 0000000..8208e6f
--- /dev/null
+++ b/lsbfdump/src/main/scala/org/apache/daffodil/lsbfDump/LSBFDump.scala
@@ -0,0 +1,203 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.daffodil.lsbfDump
+
+import java.io.IOException
+import java.io.InputStream
+import java.nio.channels.Channels
+import java.nio.channels.FileChannel
+import java.nio.file.NoSuchFileException
+import java.nio.file.Paths
+
+object LSBFDump {
+
+ private val helpText: String =
+ """Usage: lsbfdump [--file <filename>] [--offset <offset>] [--length <numBytes>] [--noAddress] [--help]
+ |
+ |<filename> : The file to read bytes from or '-' for standard input or if not provided standard input is used.
+ |[offset] : The starting offset in the file (default is 0). If the offset is past the length of the data, no output is produced.
+ |[length] : The number of bytes to display (default is entire file). If 0 no output is produced.
+ |--noAddress : Do not display the address of each byte line.
+ |--help : Display this help information.
+ |
+ |Examples:
+ | Default usage (128 bytes from standard input, starting at offset 0, with addresses):
+ | lsbfdump --file - --length 128
+ |
+ | With specific file, offset and byte count:
+ | lsbfdump --file filename --offset 10 --length 64
+ |
+ | With --noAddress to hide addresses:
+ | lsbfdump --file filename --offset 10 --length 64 --noAddress
+ |""".stripMargin
+
+ private def usageError(): Unit = {
+ System.err.println("lsbfdump: Invalid arguments provided.")
+ System.err.println(helpText)
+ sys.exit(1)
+ }
+
+ def main(args: Array[String]): Unit = {
+ val parsedArgs = parseArgs(args)
+
+ if (parsedArgs.contains("help")) {
+ println(helpText)
+ sys.exit(0)
+ }
+
+ if (parsedArgs.contains("invalid")) usageError()
+
+ val filename =
+ parsedArgs.getOrElse("file", "-") // if not provided at all, also uses std-in.
+ val offset = parsedArgs.get("offset").map(_.toLong).getOrElse(0L)
+ val length = parsedArgs.get("length").map(_.toLong).getOrElse(Long.MaxValue)
+ val showAddress = !parsedArgs.contains("noAddress")
+
+ assert(offset >= 0)
+ assert(length >= 0)
+
+ val lines = lsbfDumpFile(filename, offset, length, showAddress)
+ lines.foreach(println(_))
+ }
+
+ /**
+ * Suitable for use on large data files, as it streams the data as it displays it.
+ */
+ private[lsbfDump] def lsbfDumpFile(
+ filename: String,
+ offset: Long,
+ length: Long,
+ showAddress: Boolean,
+ ): Iterator[String] = {
+ try {
+ val input = openAndSeekInputStream(filename, offset)
+ val bytes = inputStreamToIterator(input, length)
+ val lines = lsbfDump(bytes, offset, showAddress)
+ lines
+ } catch {
+ case e: NoSuchFileException =>
+ System.err.println(s"lsbfdump: File not found - ${e.getMessage}")
+ sys.exit(1)
+ case e: IOException =>
+ System.err.println(s"lsbfdump: An I/O error occurred - ${e.getMessage}")
+ sys.exit(1)
+ }
+ }
+
+ private[lsbfDump] def lsbfDump(
+ bytes: Iterator[Byte],
+ offset: Long,
+ showAddress: Boolean,
+ ): Iterator[String] = {
+ val lines = bytes
+ .grouped(4)
+ .zipWithIndex
+ .map { case (group, index) =>
+ val byteStrings = group.reverse.map(byteToBinaryString).mkString(" ")
+ val extraSpaces =
+ if (group.length == 4) ""
+ else {
+ val nBytesMissing = 4 - group.length
+ ((" " * 9) * nBytesMissing) // 9 to account for space between bytes
+ }
+ val addressString = if (showAddress) f" | 0x${index * 4 + offset}%08X" else ""
+ extraSpaces + byteStrings + addressString
+ }
+ lines
+ }
+
+ private def openAndSeekInputStream(filename: String, offset: Long) = {
+ if (filename == "-") {
+ if (skip(System.in, offset) != offset) {
+ // throw new IOException("lsbfdump: Unable to skip to offset.")
+ }
+ System.in
+ } else
+ Channels.newInputStream(FileChannel.open(Paths.get(filename)).position(offset))
+ }
+
+ /**
+ * java.io.InputStream.skip on standard input doesn't work - I get illegal seek errors.
+ * (at least on GraalVM nativeImage). So we use our own skip.
+ * @param is
+ * @param count
+ * @return
+ */
+ private def skip(is: InputStream, count: Long): Long = {
+ assert(count >= 0)
+ var n: Long = 0
+ while (is.read() != -1 && n < count) { n += 1 }
+ n
+ }
+
+ private def inputStreamToIterator(inputStream: InputStream, len: Long): Iterator[Byte] =
+ new Iterator[Byte] {
+
+ private var count: Long = 0
+
+ def hasNext: Boolean = inputStream.available() > 0 && count < len
+
+ def next(): Byte = {
+ val nextByte = inputStream.read()
+ if (nextByte == -1) {
+ throw new NoSuchElementException("End of stream reached")
+ }
+ count += 1
+ nextByte.toByte
+ }
+ }
+
+ private[lsbfDump] def byteToBinaryString(b: Byte): String = {
+ val unsignedByte = b & 0xff
+ String.format("%8s", Integer.toBinaryString(unsignedByte)).replace(' ', '0')
+ }
+
+ def parseArgs(args: Array[String]): Map[String, String] = {
+ var argMap = Map[String, String]()
+ var i = 0
+ while (i < args.length) {
+ args(i) match {
+ case "--file" if i + 1 < args.length =>
+ argMap += ("file" -> args(i + 1))
+ i += 2
+ case "--offset" if i + 1 < args.length =>
+ argMap += ("offset" -> args(i + 1))
+ if (!isNonNegativeInteger(args(i + 1))) usageError()
+ i += 2
+ case "--length" if i + 1 < args.length =>
+ argMap += ("length" -> args(i + 1))
+ if (!isNonNegativeInteger(args(i + 1))) usageError()
+ i += 2
+ case "--noAddress" =>
+ argMap += ("noAddress" -> "")
+ i += 1
+ case "--help" =>
+ argMap += ("help" -> "")
+ i = args.length // Break the loop
+ case _ =>
+ argMap += ("invalid" -> "")
+ i = args.length // Break the loop
+ }
+ }
+ argMap
+ }
+
+ private def isNonNegativeInteger(s: String): Boolean =
+ try { s.toLong >= 0 }
+ catch { case _: NumberFormatException => false }
+
+}
diff --git a/lsbfdump/src/test/resources/testData1.txt b/lsbfdump/src/test/resources/testData1.txt
new file mode 100644
index 0000000..ad47100
--- /dev/null
+++ b/lsbfdump/src/test/resources/testData1.txt
@@ -0,0 +1 @@
+0123456789
\ No newline at end of file
diff --git a/lsbfdump/src/test/scala/org/apache/daffodil/lsbfDump/LSBFDumpTest.scala b/lsbfdump/src/test/scala/org/apache/daffodil/lsbfDump/LSBFDumpTest.scala
new file mode 100644
index 0000000..4878b59
--- /dev/null
+++ b/lsbfdump/src/test/scala/org/apache/daffodil/lsbfDump/LSBFDumpTest.scala
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.daffodil.lsbfDump
+
+import java.io.File
+import java.nio.file.Paths
+
+import org.junit.Assert._
+import org.junit.Test
+
+class LSBFDumpTest {
+
+ @Test
+ def testDump1(): Unit = {
+ val data = List(0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08).map { _.toByte }.iterator
+ val lines = LSBFDump.lsbfDump(data, 0, showAddress = true).toSeq
+ assertEquals(2, lines.length)
+ assertEquals("00000100 00000011 00000010 00000001 | 0x00000000", lines.head)
+ assertEquals("00001000 00000111 00000110 00000101 | 0x00000004", lines(1))
+ }
+
+ @Test
+ def testDump2WithOffset4(): Unit = {
+ val data = List(0x05, 0x06, 0x07, 0x08).map { _.toByte }.iterator
+ val lines = LSBFDump.lsbfDump(data, 4, showAddress = true).toSeq
+ assertEquals(1, lines.length)
+ assertEquals("00001000 00000111 00000110 00000101 | 0x00000004", lines.head)
+ }
+
+ @Test
+ def testDump2WithOffset4_shortBy1(): Unit = {
+ val data = List(0x05, 0x06, 0x07).map { _.toByte }.iterator
+ val lines = LSBFDump.lsbfDump(data, 4, showAddress = true).toSeq
+ assertEquals(1, lines.length)
+ assertEquals(" 00000111 00000110 00000101 | 0x00000004", lines.head)
+ }
+
+ @Test
+ def testDump2WithOffset4_noData(): Unit = {
+ val data = List[Byte]().iterator
+ val lines = LSBFDump.lsbfDump(data, 4, showAddress = true).toSeq
+ assertEquals(0, lines.length)
+ }
+
+ @Test
+ def testDump2WithOffset4NoAddress(): Unit = {
+ val data = List(0x05, 0x06, 0x07, 0x08).map { _.toByte }.iterator
+ val lines = LSBFDump.lsbfDump(data, 4, showAddress = false).toSeq
+ assertEquals(1, lines.length)
+ assertEquals("00001000 00000111 00000110 00000101", lines.head)
+ }
+
+ @Test
+ def testDumpFileWithOffset4(): Unit = {
+ val resource =
+ new File(
+ "src/test/resources/testData1.txt",
+ ).toURI // getClass.getResource("testData1.txt").toURI
+ val file = Paths.get(resource).toFile
+ val fn = file.toString
+ val lines = LSBFDump
+ .lsbfDumpFile(
+ filename = fn,
+ offset = 4,
+ length = file.length().toInt,
+ showAddress = true,
+ )
+ .toSeq
+ assertEquals(2, lines.length)
+ assertEquals("00110111 00110110 00110101 00110100 | 0x00000004", lines.head)
+ }
+
+ @Test
+ def testByteToBinaryString(): Unit = {
+ assertEquals("00000000", LSBFDump.byteToBinaryString(0.toByte))
+ assertEquals("00000001", LSBFDump.byteToBinaryString(1.toByte))
+ assertEquals("11111111", LSBFDump.byteToBinaryString(255.toByte))
+ assertEquals("10000000", LSBFDump.byteToBinaryString(128.toByte))
+ assertEquals("01111111", LSBFDump.byteToBinaryString(127.toByte))
+ assertEquals("11111110", LSBFDump.byteToBinaryString(-2.toByte))
+ assertEquals("11111111", LSBFDump.byteToBinaryString(-1.toByte))
+ }
+
+ @Test
+ def testParseArgsWithFile(): Unit = {
+ val args = Array("--file", "testFile.txt")
+ val parsedArgs = LSBFDump.parseArgs(args)
+ assertEquals("testFile.txt", parsedArgs("file"))
+ assertFalse(parsedArgs.contains("offset"))
+ assertFalse(parsedArgs.contains("length"))
+ assertFalse(parsedArgs.contains("noAddress"))
+ }
+
+ @Test
+ def testParseArgsWithAllOptions(): Unit = {
+ val args =
+ Array("--file", "testFile.txt", "--offset", "10", "--length", "100", "--noAddress")
+ val parsedArgs = LSBFDump.parseArgs(args)
+ assertEquals("testFile.txt", parsedArgs("file"))
+ assertEquals("10", parsedArgs("offset"))
+ assertEquals("100", parsedArgs("length"))
+ assertTrue(parsedArgs.contains("noAddress"))
+ }
+
+ @Test
+ def testParseArgsWithInvalidOption(): Unit = {
+ val args = Array("--unknownOption")
+ val parsedArgs = LSBFDump.parseArgs(args)
+ assertTrue(parsedArgs.contains("invalid"))
+ }
+
+}
(daffodil-extra) 03/03: Add lsbfdump example using Scala and Rust
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shanedell pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/daffodil-extra.git
commit 5b593f5538a034ce5b31e3a4536c4b2eb046075d
Author: Shane Dell <sh...@gmail.com>
AuthorDate: Mon Jan 29 12:24:03 2024 -0500
Add lsbfdump example using Scala and Rust
---
.gitignore | 7 +
lsbfdump-jni-rs/LICENSE | 202 +++++++++++++++++++++
lsbfdump-jni-rs/README.md | 76 ++++++++
lsbfdump-jni-rs/build.sbt | 35 ++++
lsbfdump-jni-rs/native/Cargo.toml | 11 ++
lsbfdump-jni-rs/native/src/helper_functions.rs | 69 +++++++
lsbfdump-jni-rs/native/src/lib.rs | 97 ++++++++++
lsbfdump-jni-rs/project/build.properties | 1 +
lsbfdump-jni-rs/project/plugins.sbt | 2 +
.../scala/org/apache/daffodil/lsbfDump/FFI.scala | 29 +++
.../org/apache/daffodil/lsbfDump/LSBFDump.scala | 102 +++++++++++
.../org/apache/daffodil/lsbfDump/FFITest.scala | 38 ++++
lsbfdump/build.sbt | 2 +
13 files changed, 671 insertions(+)
diff --git a/.gitignore b/.gitignore
index df70556..18ed0e0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -41,3 +41,10 @@ resource_managed
# Project local cache of managed dependencies
lib_managed
+
+# vscode/metals generated files
+.vscode
+.metals
+.bloop
+
+Cargo.lock
diff --git a/lsbfdump-jni-rs/LICENSE b/lsbfdump-jni-rs/LICENSE
new file mode 100644
index 0000000..d645695
--- /dev/null
+++ b/lsbfdump-jni-rs/LICENSE
@@ -0,0 +1,202 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
diff --git a/lsbfdump-jni-rs/README.md b/lsbfdump-jni-rs/README.md
new file mode 100644
index 0000000..04064e3
--- /dev/null
+++ b/lsbfdump-jni-rs/README.md
@@ -0,0 +1,76 @@
+# Example Scala calling Rust using JNI
+
+## Rust naming conventions
+
+This url, https://rust-lang.github.io/api-guidelines/naming.html, provides helpful information on Rust naming conventions.
+
+## Reference
+
+This repos implementation is based on the library [jni-rs](https://github.com/jni-rs/jni-rs). The example provided is very helpful to understand how things work.
+
+## License
+
+This is open-source software under the Apache Software License (V2.0). See the [LICENSE](./LICENSE) file for the full text.
+
+## Testing
+
+No prior commands should need ran.
+
+The command:
+
+```bash
+sbt test
+```
+
+Should provide output similar to the below (note the `[...]` in the paths is so the full path wasn't shown):
+
+```bash
+[info] welcome to sbt 1.9.7 (Eclipse Adoptium Java 17.0.5)
+[info] loading settings for project lsbfdump-jni-rs-build from plugins.sbt ...
+[info] loading project definition from [...]/daffodil-extra/lsbfdump-jni-rs/project
+[info] loading settings for project root from build.sbt ...
+[info] set current project to lsbfdump-jni-rs (in build file:[...]/daffodil-extra/lsbfdump-jni-rs/)
+[info] Building library with native build tool Cargo
+[info] compiling 2 Scala sources to [...]/daffodil-extra/lsbfdump-jni-rs/target/scala-2.13/classes ...
+[warn] Finished release [optimized] target(s) in 0.02s
+[warn] More than one file was created during compilation, only the first one ([...]/daffodil-extra/lsbfdump-jni-rs/native/target/native/arm64-darwin/bin/release/liblsbfdump.dylib) will be used.
+[success] Library built in [...]/daffodil-extra/lsbfdump-jni-rs/native/target/native/arm64-darwin/bin/release/liblsbfdump.dylib
+[info] compiling 1 Scala source to [...]/daffodil-extra/lsbfdump-jni-rs/target/scala-2.13/test-classes ...
+01111000 01000101 00100000 00100011 | 0x00000000
+01101100 01110000 01101101 01100001 | 0x00000004
+01100011 01010011 00100000 01100101 | 0x00000008
+00100000 01100001 01101100 01100001 | 0x0000000C
+01101100 01101100 01100001 01100011 | 0x00000010
+00100000 01100111 01101110 01101001 | 0x00000014
+01110100 01110011 01110101 01010010 | 0x00000018
+01101001 01110011 01110101 00100000 | 0x0000001C
+01001010 00100000 01100111 01101110 | 0x00000020
+00001010 00001010 01001001 01001110 | 0x00000024
+01010010 00100000 00100011 00100011 | 0x00000028
+00100000 01110100 01110011 01110101 | 0x0000002C
+01101001 01101101 01100001 01101110 | 0x00000030
+01100011 00100000 01100111 01101110 | 0x00000034
+01100101 01110110 01101110 01101111 | 0x00000038
+01101111 01101001 01110100 01101110 | 0x0000003C
+00001010 00001010 01110011 01101110 | 0x00000040
+01110011 01101001 01101000 01010100 | 0x00000044
+01101100 01110010 01110101 00100000 | 0x00000048
+01110100 01101000 00100000 00101100 | 0x0000004C
+00111010 01110011 01110000 01110100 | 0x00000050
+01110101 01110010 00101111 00101111 | 0x00000054
+01101100 00101101 01110100 01110011 | 0x00000058
+00101110 01100111 01101110 01100001 | 0x0000005C
+01101000 01110100 01101001 01100111 | 0x00000060
+01101001 00101110 01100010 01110101 | 0x00000064
+01110000 01100001 00101111 01101111 | 0x00000068
+01110101 01100111 00101101 01101001 | 0x0000006C
+01101100 01100101 01100100 01101001 | 0x00000070
+01110011 01100101 01101110 01101001 | 0x00000074
+01101101 01100001 01101110 00101111 | 0x00000078
+00101110 01100111 01101110 01101001 | 0x0000007C
+lsbfdump: An I/O error occurred - No such file or directory (os error 2)
+[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
+...
+```
+
+Don't worry about the output that comes out as it comes from the Rust code. The Rust code then also returns either true or false based on if everything ran okay.
diff --git a/lsbfdump-jni-rs/build.sbt b/lsbfdump-jni-rs/build.sbt
new file mode 100644
index 0000000..b095e0d
--- /dev/null
+++ b/lsbfdump-jni-rs/build.sbt
@@ -0,0 +1,35 @@
+lazy val commonSettings = Seq(
+ scalaVersion := "2.13.12"
+)
+
+lazy val native = project
+ .in(file("native"))
+ .settings(commonSettings)
+ .settings(
+ name := "lsbfdump-jni-rs-native",
+ nativeCompile / sourceDirectory := baseDirectory.value,
+ )
+ .enablePlugins(JniNative)
+
+lazy val root = project
+ .in(file("."))
+ .settings(commonSettings)
+ .settings(
+ name := "lsbfdump-jni-rs",
+ Compile / mainClass := Some("org.apache.daffodil.lsbfDump.LSBFDump"),
+ libraryDependencies ++= List(
+ "junit" % "junit" % "4.13.2" % Test,
+ // needed below package to successfully run junit tests
+ "com.github.sbt" % "junit-interface" % "0.13.3" % Test exclude("junit", "junit-dep"),
+ ),
+ sbtJniCoreScope := Compile, // because we use `NativeLoader`, not the `@nativeLoader` macro
+ )
+ .dependsOn(native % Runtime)
+
+// enablePlugins(NativeImagePlugin)
+// // Settings for the native image
+// nativeImageOptions ++= Seq(
+// "--no-fallback", // does not create nativeImage if it will require a JVM.
+// // "--enable-all-security-services"
+// // Add other native-image options as needed
+// )
diff --git a/lsbfdump-jni-rs/native/Cargo.toml b/lsbfdump-jni-rs/native/Cargo.toml
new file mode 100644
index 0000000..464a9b1
--- /dev/null
+++ b/lsbfdump-jni-rs/native/Cargo.toml
@@ -0,0 +1,11 @@
+[package]
+name = "lsbfdump"
+version = "0.1.0"
+authors = ["Shane Dell <sh...@gmail.com>"]
+edition = "2021"
+
+[dependencies]
+jni = "0.21.1"
+
+[lib]
+crate_type = ["cdylib"]
diff --git a/lsbfdump-jni-rs/native/src/helper_functions.rs b/lsbfdump-jni-rs/native/src/helper_functions.rs
new file mode 100644
index 0000000..7cd45a6
--- /dev/null
+++ b/lsbfdump-jni-rs/native/src/helper_functions.rs
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// View https://rust-lang.github.io/api-guidelines/naming.html for naming conventions for functions
+
+use std::fs::File;
+use std::io::{self, Read, Seek, SeekFrom};
+
+pub fn lsbf_dump(reader: &mut dyn Read, offset: u64, length: u64, show_address: bool) -> io::Result<Vec<String>> {
+ let mut lines = Vec::new();
+ let mut count = 0;
+ let mut buffer = [0; 4];
+
+ while count < length {
+ match reader.read_exact(&mut buffer) {
+ Ok(_) => {
+ let byte_strings = buffer.iter().copied().rev().map(byte_to_binary_string).collect::<Vec<String>>().join(" ");
+ let extra_spaces = if buffer.len() == 4 { "".to_string() } else { " ".repeat(4 - buffer.len()) };
+ let address_string = if show_address { format!(" | 0x{:08X}", count + offset) } else { "".to_string() };
+ lines.push(format!("{}{}{}", extra_spaces, byte_strings, address_string));
+ count += 4;
+ }
+ Err(_) => break,
+ }
+ }
+
+ Ok(lines)
+}
+
+pub fn open_and_seek_input_stream(filename: &str, offset: u64) -> io::Result<Box<dyn Read>> {
+ if filename == "-" {
+ let stdin = io::stdin();
+ let mut reader = stdin.lock();
+ let mut remaining_bytes_to_seek = offset;
+
+ while remaining_bytes_to_seek > 0 {
+ let mut temp_buffer = [0; 1];
+ match reader.read_exact(&mut temp_buffer) {
+ Ok(_) => remaining_bytes_to_seek -= 1,
+ Err(e) => return Err(e),
+ }
+ }
+
+ Ok(Box::new(reader) as Box<dyn Read>)
+ } else {
+ let file = File::open(filename)?;
+ let mut file_seek = io::BufReader::new(file);
+ file_seek.seek(SeekFrom::Start(offset))?;
+ Ok(Box::new(file_seek) as Box<dyn Read>)
+ }
+}
+
+fn byte_to_binary_string(b: u8) -> String {
+ format!("{:08b}", b)
+}
diff --git a/lsbfdump-jni-rs/native/src/lib.rs b/lsbfdump-jni-rs/native/src/lib.rs
new file mode 100644
index 0000000..2638024
--- /dev/null
+++ b/lsbfdump-jni-rs/native/src/lib.rs
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+mod helper_functions;
+
+use helper_functions::{lsbf_dump, open_and_seek_input_stream};
+
+use jni::JNIEnv;
+use jni::objects::{JClass, JString};
+use jni::sys::{jboolean,jlong};
+
+/**
+ * For naming conventions for functions being exported to be callable from Scala
+ * see https://github.com/jni-rs/jni-rs. But it basically is this pattern:
+ *
+ * 1. Java_
+ * 2. Organization folder structure, after src/main/scala, or package name by
+ * replacing "." or "/" with "_" folowed by a trailing "_"
+ * eg:
+ * package name: org.apache.daffodil.lsbfDump
+ * rust function part: org_apache_daffodil_lsbfDump
+ * 3. Scala class name followed by "_"
+ * eg:
+ * scala class: class FFI() extends NativeLoader("lsbfdump")
+ * rust function part: FFI_
+ * 4. Scala @native function name.
+ * eg:
+ * scala class @native function: @native def lsbfDumpFile
+ * rust function part: lsbfDumpFile
+ *
+ * If you follow these numbered items for this example you should get:
+ *
+ * 1. Java_
+ * 2. org_apache_daffodil_lsbfDump (package_name = org.apache.daffodil.lsbfDump)
+ * 3. FFI_ (scala_class_name = FFI)
+ * 4. lsbfDumpFile (scala_class_function_name = lsbfDumpFile)
+ *
+ * Adding all four parts together gives us our function name:
+ * Java_org_apache_daffodil_lsbfDump_FFI_lsbfDumpFile
+ *
+ */
+
+#[no_mangle]
+pub extern "system" fn Java_org_apache_daffodil_lsbfDump_FFI_lsbfDumpFile(
+ mut env: JNIEnv,
+ _class: JClass,
+ filename_in: JString,
+ offset_in: jlong,
+ length_in: jlong,
+ show_address_in: jboolean,
+) -> jboolean {
+ let filename: String = env
+ .get_string(&filename_in)
+ .expect("Couldn't get scala string")
+ .into();
+
+ let offset: u64 = offset_in as u64;
+ let length: u64 = length_in as u64;
+ let show_address: bool = show_address_in != 0;
+
+ match crate::open_and_seek_input_stream(&filename, offset) {
+ Ok(mut reader) => {
+ match crate::lsbf_dump(&mut reader, offset, length, show_address) {
+ Ok(lines) => {
+ for line in &lines {
+ println!("{}", line);
+ }
+ return true as jboolean
+ }
+ Err(err) => {
+ eprintln!("lsbfdump: An I/O error occurred - {}", err);
+ // std::process::exit(1);
+ return false as jboolean
+ }
+ }
+ }
+ Err(err) => {
+ eprintln!("lsbfdump: An I/O error occurred - {}", err);
+ // std::process::exit(1);
+ return false as jboolean
+ }
+ }
+}
diff --git a/lsbfdump-jni-rs/project/build.properties b/lsbfdump-jni-rs/project/build.properties
new file mode 100644
index 0000000..b19d4e1
--- /dev/null
+++ b/lsbfdump-jni-rs/project/build.properties
@@ -0,0 +1 @@
+sbt.version = 1.9.7
diff --git a/lsbfdump-jni-rs/project/plugins.sbt b/lsbfdump-jni-rs/project/plugins.sbt
new file mode 100644
index 0000000..a39c9a2
--- /dev/null
+++ b/lsbfdump-jni-rs/project/plugins.sbt
@@ -0,0 +1,2 @@
+addSbtPlugin("com.github.sbt" % "sbt-jni" % "1.5.3")
+// addSbtPlugin("org.scalameta" % "sbt-native-image" % "0.3.4")
diff --git a/lsbfdump-jni-rs/src/main/scala/org/apache/daffodil/lsbfDump/FFI.scala b/lsbfdump-jni-rs/src/main/scala/org/apache/daffodil/lsbfDump/FFI.scala
new file mode 100644
index 0000000..306ca72
--- /dev/null
+++ b/lsbfdump-jni-rs/src/main/scala/org/apache/daffodil/lsbfDump/FFI.scala
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.lsbfDump
+
+import com.github.sbt.jni.syntax.NativeLoader
+
+class FFI() extends NativeLoader("lsbfdump") {
+ @native def lsbfDumpFile(
+ filename: String,
+ offset: Long,
+ length: Long,
+ showAddress: Boolean,
+ ): Boolean // implemented in liblsbfdump.so
+}
diff --git a/lsbfdump-jni-rs/src/main/scala/org/apache/daffodil/lsbfDump/LSBFDump.scala b/lsbfdump-jni-rs/src/main/scala/org/apache/daffodil/lsbfDump/LSBFDump.scala
new file mode 100644
index 0000000..0860a99
--- /dev/null
+++ b/lsbfdump-jni-rs/src/main/scala/org/apache/daffodil/lsbfDump/LSBFDump.scala
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.lsbfDump
+
+object LSBFDump {
+ private val helpText: String =
+ """Usage: lsbfdump [--file <filename>] [--offset <offset>] [--length <numBytes>] [--noAddress] [--help]
+ |
+ |<filename> : The file to read bytes from or '-' for standard input or if not provided standard input is used.
+ |[offset] : The starting offset in the file (default is 0). If the offset is past the length of the data, no output is produced.
+ |[length] : The number of bytes to display (default is entire file). If 0 no output is produced.
+ |--noAddress : Do not display the address of each byte line.
+ |--help : Display this help information.
+ |
+ |Examples:
+ | Default usage (128 bytes from standard input, starting at offset 0, with addresses):
+ | lsbfdump --file - --length 128
+ |
+ | With specific file, offset and byte count:
+ | lsbfdump --file filename --offset 10 --length 64
+ |
+ | With --noAddress to hide addresses:
+ | lsbfdump --file filename --offset 10 --length 64 --noAddress
+ |""".stripMargin
+
+ private def usageError(): Unit = {
+ System.err.println("lsbfdump: Invalid arguments provided.")
+ System.err.println(helpText)
+ sys.exit(1)
+ }
+
+ def main(args: Array[String]): Unit = {
+ val parsedArgs = parseArgs(args)
+
+ if (parsedArgs.contains("help")) {
+ println(helpText)
+ sys.exit(0)
+ }
+
+ if (parsedArgs.contains("invalid")) usageError()
+
+ val filename =
+ parsedArgs.getOrElse("file", "-") // if not provided at all, also uses std-in.
+ val offset = parsedArgs.get("offset").map(_.toLong).getOrElse(0L)
+ val length = parsedArgs.get("length").map(_.toLong).getOrElse(Long.MaxValue)
+ val showAddress = !parsedArgs.contains("noAddress")
+
+ assert(offset >= 0)
+ assert(length >= 0)
+
+ new FFI().lsbfDumpFile(filename, offset, length, showAddress)
+ }
+
+ def parseArgs(args: Array[String]): Map[String, String] = {
+ var argMap = Map[String, String]()
+ var i = 0
+ while (i < args.length) {
+ args(i) match {
+ case "--file" if i + 1 < args.length =>
+ argMap += ("file" -> args(i + 1))
+ i += 2
+ case "--offset" if i + 1 < args.length =>
+ argMap += ("offset" -> args(i + 1))
+ if (!isNonNegativeInteger(args(i + 1))) usageError()
+ i += 2
+ case "--length" if i + 1 < args.length =>
+ argMap += ("length" -> args(i + 1))
+ if (!isNonNegativeInteger(args(i + 1))) usageError()
+ i += 2
+ case "--noAddress" =>
+ argMap += ("noAddress" -> "")
+ i += 1
+ case "--help" =>
+ argMap += ("help" -> "")
+ i = args.length // Break the loop
+ case _ =>
+ argMap += ("invalid" -> "")
+ i = args.length // Break the loop
+ }
+ }
+ argMap
+ }
+
+ private def isNonNegativeInteger(s: String): Boolean =
+ try { s.toLong >= 0 }
+ catch { case _: NumberFormatException => false }
+}
diff --git a/lsbfdump-jni-rs/src/test/scala/org/apache/daffodil/lsbfDump/FFITest.scala b/lsbfdump-jni-rs/src/test/scala/org/apache/daffodil/lsbfDump/FFITest.scala
new file mode 100644
index 0000000..bb3f551
--- /dev/null
+++ b/lsbfdump-jni-rs/src/test/scala/org/apache/daffodil/lsbfDump/FFITest.scala
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.lsbfDump
+
+import java.io.File
+import java.nio.file.Paths
+
+import org.junit.Assert._
+import org.junit.Test
+
+class FFITest {
+ @Test
+ def testBadFile(): Unit = {
+ val actual = new FFI().lsbfDumpFile("no_such/file", 0, 0, false)
+ assertEquals(actual, false)
+ }
+
+ @Test
+ def testGoodFile(): Unit = {
+ val actual = new FFI().lsbfDumpFile(s"${System.getProperty("user.dir")}/README.md", 0, 128, true)
+ assertEquals(actual, true)
+ }
+}
diff --git a/lsbfdump/build.sbt b/lsbfdump/build.sbt
index 2699d1a..fd739c5 100644
--- a/lsbfdump/build.sbt
+++ b/lsbfdump/build.sbt
@@ -16,5 +16,7 @@ nativeImageOptions ++= Seq(
// Library dependencies
libraryDependencies ++= Seq(
+ // needed below package to successfully run junit tests
+ "com.github.sbt" % "junit-interface" % "0.13.3" % Test exclude("junit", "junit-dep"),
"junit" % "junit" % "4.13.2" % Test,
)