You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datasketches.apache.org by le...@apache.org on 2020/01/15 00:19:42 UTC
[incubator-datasketches-website] branch clean_asf_site created (now
e443537)
This is an automated email from the ASF dual-hosted git repository.
leerho pushed a change to branch clean_asf_site
in repository https://gitbox.apache.org/repos/asf/incubator-datasketches-website.git.
at e443537 Cleanup asf-site
This branch includes the following new commits:
new e443537 Cleanup asf-site
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[incubator-datasketches-website] 01/01: Cleanup asf-site
Posted by le...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
leerho pushed a commit to branch clean_asf_site
in repository https://gitbox.apache.org/repos/asf/incubator-datasketches-website.git
commit e443537c3dc550ec4d64a8ae4b1a7829b23a8479
Author: Lee Rhodes <le...@users.noreply.github.com>
AuthorDate: Tue Jan 14 16:19:14 2020 -0800
Cleanup asf-site
---
.gitignore | 4 +
404.html | 24 -
DISCLAIMER-WIP | 26 -
LICENSE | 210 ----
NOTICE | 11 -
README.html | 573 -----------
pom.xml | 630 ------------
script/cibuild | 6 -
.../org/apache/datasketches/ByteArrayBuilder.java | 210 ----
src/main/java/org/apache/datasketches/Files.java | 1040 --------------------
.../apache/datasketches/docgen/ErrorTables.java | 86 --
.../apache/datasketches/docgen/MarkDownTable.java | 133 ---
.../apache/datasketches/docgen/TocGenerator.java | 263 -----
src/main/resources/docgen/toc.json | 232 -----
src/main/resources/docgen/tocScript.html | 64 --
15 files changed, 4 insertions(+), 3508 deletions(-)
diff --git a/.gitignore b/.gitignore
index 9afbe71..f329bfb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,9 +6,13 @@ about.md
Gemfile
Gemfile.lock
_posts/
+.DS_store
+
# Eclipse
.project
+.classpath
+.settings/
# yml files
diff --git a/404.html b/404.html
deleted file mode 100644
index c472b4e..0000000
--- a/404.html
+++ /dev/null
@@ -1,24 +0,0 @@
----
-layout: default
----
-
-<style type="text/css" media="screen">
- .container {
- margin: 10px auto;
- max-width: 600px;
- text-align: center;
- }
- h1 {
- margin: 30px 0;
- font-size: 4em;
- line-height: 1;
- letter-spacing: -1px;
- }
-</style>
-
-<div class="container">
- <h1>404</h1>
-
- <p><strong>Page not found :(</strong></p>
- <p>The requested page could not be found.</p>
-</div>
diff --git a/DISCLAIMER-WIP b/DISCLAIMER-WIP
deleted file mode 100644
index c90f3f7..0000000
--- a/DISCLAIMER-WIP
+++ /dev/null
@@ -1,26 +0,0 @@
-Apache DataSketches (incubating) is an effort undergoing incubation
-at The Apache Software Foundation (ASF), sponsored by the Apache Incubator.
-
-Incubation is required of all newly accepted projects until a further review
-indicates that the infrastructure, communications, and decision making process
-have stabilized in a manner consistent with other successful ASF projects.
-
-While incubation status is not necessarily a reflection of the
-completeness or stability of the code, it does indicate that the
-project has yet to be fully endorsed by the ASF.
-
-Some of the incubating project's releases may not be fully compliant
-with ASF policy. For example, releases may have incomplete or
-un-reviewed licensing conditions. What follows is a list of known
-issues the project is currently aware of (note that this list, by
-definition, is likely to be incomplete):
-
- * The LICENSE and NOTICE files may not be complete and will be fixed with the next release.
-
-If you are planning to incorporate this work into your
-product or project, please be aware that you will need to conduct a
-thorough licensing review to determine the overall implications of
-including this work. For the current status of this project through the Apache
-Incubator visit:
-
-http://incubator.apache.org/projects/datasketches.html
diff --git a/LICENSE b/LICENSE
deleted file mode 100644
index 5667780..0000000
--- a/LICENSE
+++ /dev/null
@@ -1,210 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
-
-
-APPENDIX A: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
- -------------------------------------------------------------
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
- -------------------------------------------------------------
-
-
-
-APPENDIX B: Additional licenses relevant to this Apache DataSketches (incubating) component:
- (none)
diff --git a/NOTICE b/NOTICE
deleted file mode 100644
index bd2e463..0000000
--- a/NOTICE
+++ /dev/null
@@ -1,11 +0,0 @@
-Apache DataSketches Website (incubating)
-Copyright 2019 - The Apache Software Foundation
-
-Copyright 2015-2018 Yahoo
-Copyright 2018 - Verizon Media
-
-This product includes software developed at
-The Apache Software Foundation (http://www.apache.org/).
-
-Prior to moving to ASF, the software for this project was developed at
-Yahoo (now Verizon Media) (https://developer.yahoo.com).
diff --git a/README.html b/README.html
deleted file mode 100644
index 8665490..0000000
--- a/README.html
+++ /dev/null
@@ -1,573 +0,0 @@
-<!DOCTYPE html>
-<!-- Start _layouts/doc_page.html-->
-<html lang="en">
-
-<head>
- <!-- Start site_head.html include-->
-<meta charset="UTF-8" />
-<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="description" content="">
-<meta name="author" content="datasketches">
-
-<title>DataSketches | </title>
-
-<link rel="shortcut icon" href="/img/favicon.png">
-
-<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.1.0/css/font-awesome.min.css">
-<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css">
-
-<link href='https://fonts.googleapis.com/css?family=Open+Sans+Condensed:300,700,300italic|Open+Sans:300italic,400italic,600italic,400,300,600'
- rel='stylesheet' type='text/css'>
-
-<link rel="stylesheet" href="/css/main.css">
-<link rel="stylesheet" href="/css/header.css">
-<link rel="stylesheet" href="/css/footer.css">
-<link rel="stylesheet" href="/css/syntax.css">
-<link rel="stylesheet" href="/css/docs.css">
-
-
-<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML-full">
-</script>
-<script src="https://code.jquery.com/jquery.min.js"></script>
-<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/js/bootstrap.min.js"></script>
-<!-- End site_head.html include-->
-
-</head>
-
-<body>
- <!-- Start page_header.html include -->
-<div class="navbar navbar-inverse navbar-static-top ds-nav">
- <div class="container">
- <div class="navbar-header">
- <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
- <span class="sr-only">Toggle navigation</span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
- <!-- navbar-brand: https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css -->
- <!-- /css/main.css:57:.logo -->
- <a class="navbar-brand logo" href="/">DataSketches</a>
- </div>
- <div class="navbar-collapse collapse">
- <ul class="nav navbar-nav navbar-right">
- <li>
- <a href="/overview.html">
- <span class="fa fa-info-circle"></span> OVERVIEW</a>
- </li>
- <li>
- <a href="/docs/downloads.html">
- <span class="fa fa-download"></span> DOWNLOAD</a>
- </li>
- <li>
- <a href="https://github.com/apache?utf8=%E2%9C%93&q=datasketches&type=&language=">
- <span class="fa fa-github"></span> GITHUB</a>
- </li>
- <li>
- <a href="/docs/Research.html">
- <span class="fa fa-paper-plane"></span> RESEARCH</a>
- </li>
- <li>
- <a href="https://groups.google.com/forum/#!forum/sketches-user">
- <span class="fa fa-comment"></span> FORUM</a>
- </li>
-
- </ul>
- </div>
- </div>
-</div>
-<!-- End page_header.html include -->
-
-
- <div class="ds-header">
- <div class="container">
- <h4>API Snapshots:
- <a href="/api/java/snapshot/apidocs/index.html">Java Core</a>,
- <a href="/api/memory/snapshot/apidocs/index.html">Memory</a>,
- <a href="/api/pig/snapshot/apidocs/index.html">Pig</a>,
- <a href="/api/hive/snapshot/apidocs/index.html">Hive</a>,
- </h4>
- </div>
- </div>
-
- <div class="container">
- <div class="row">
- <div class="col-md-3">
- <div class="searchbox" style="position:relative">
- <gcse:searchbox-only></gcse:searchbox-only>
- </div>
- <!-- Computer Generated File, Do Not Edit! -->
-<link rel="stylesheet" href="/css/toc.css">
-<div id="toc" class="nav toc hidden-print">
-
- <p id="overview">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_overview">Overview</a>
- </p>
- <div class="collapse" id="collapse_overview">
- <li><a href="/docs/TheChallenge.html">The Challenge</a></li>
- <li><a href="/docs/MajorSketchFamilies.html">The Major Sketch Families</a></li>
- <li><a href="/docs/SketchOrigins.html">Sketch Origins</a></li>
- <li><a href="/docs/SketchElements.html">Sketch Elements</a></li>
- <li><a href="/docs/KeyFeatures.html">Key Features</a></li>
- <li><a href="/docs/LargeScale.html">Large Scale Computing</a></li>
- <li><a href="/docs/Architecture.html">Architecture</a></li>
- <li><a href="/docs/OrderSensitivity.html">Notes on Order Sensitivity</a></li>
- <li><a href="/docs/Concurrency.html">Notes on Concurrency</a></li>
- <li><a href="https://github.com/DataSketches/DataSketches.github.io/blob/master/docs/pdf/DataSketches_deck.pdf">Overview Slide Deck</a></li>
- </div>
-
- <p id="research">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_research">Research</a>
- </p>
- <div class="collapse" id="collapse_research">
- <li><a href="/docs/Research.html">Research</a></li>
- </div>
-
- <p id="distinct-counting">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_distinct_counting">Distinct Counting</a>
- </p>
- <div class="collapse" id="collapse_distinct_counting">
-
- <p id="hyperloglog-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_hyperloglog_sketches">HyperLogLog Sketches</a>
- </p>
- <div class="collapse" id="collapse_hyperloglog_sketches">
- <li><a href="/docs/HLL/HLL.html">HLL Sketch</a></li>
- <li><a href="/docs/HLL/HllMap.html">HLL Map Sketch</a></li>
-
- <p id="hll-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_hll_examples">HLL Examples</a>
- </p>
- <div class="collapse" id="collapse_hll_examples">
- <li><a href="/docs/HLL/HllJavaExample.html">HLL Sketch Java Example</a></li>
- <li><a href="/docs/HLL/HllPigUDFs.html">HLL Sketch Pig UDFs</a></li>
- <li><a href="/docs/HLL/HllHiveUDFs.html">HLL Sketch Hive UDFs</a></li>
- <li><a href="/docs/DruidIntegration.html">Using Sketches in Druid</a></li>
- </div>
-
- <p id="hll-studies">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_hll_studies">HLL Studies</a>
- </p>
- <div class="collapse" id="collapse_hll_studies">
- <li><a href="/docs/HLL/Hll_vs_Hllpp.html">HLL vs HLL++</a></li>
- <li><a href="/docs/HLL/HllSketchVsDruidHyperLogLogCollector.html">HLL Sketch vs Druid HyperLogLogCollector</a></li>
- </div>
- </div>
-
- <p id="theta-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_sketches">Theta Sketches</a>
- </p>
- <div class="collapse" id="collapse_theta_sketches">
- <li><a href="/docs/Theta/ThetaSketchFramework.html">Theta Sketch Framework</a></li>
-
- <p id="theta-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_examples">Theta Examples</a>
- </p>
- <div class="collapse" id="collapse_theta_examples">
- <li><a href="/docs/Theta/ConcurrentThetaSketch.html">Concurrent Theta Sketch</a></li>
- <li><a href="/docs/Theta/ThetaJavaExample.html">Theta Sketch Java Example</a></li>
- <li><a href="/docs/Theta/ThetaSparkExample.html">Theta Sketch Spark Example</a></li>
- <li><a href="/docs/Theta/ThetaPigUDFs.html">Theta Sketch Pig UDFs</a></li>
- <li><a href="/docs/Theta/ThetaHiveUDFs.html">Theta Sketch Hive UDFs</a></li>
- <li><a href="/docs/DruidIntegration.html">Using Sketches in Druid</a></li>
- </div>
-
- <p id="kmv-tutorial">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_kmv_tutorial">KMV Tutorial</a>
- </p>
- <div class="collapse" id="collapse_kmv_tutorial">
- <li><a href="/docs/Theta/InverseEstimate.html">The Inverse Estimate</a></li>
- <li><a href="/docs/Theta/KMVempty.html">Empty Sketch</a></li>
- <li><a href="/docs/Theta/KMVfirstEst.html">First Estimator</a></li>
- <li><a href="/docs/Theta/KMVbetterEst.html">Better Estimator</a></li>
- <li><a href="/docs/Theta/KMVrejection.html">Rejection Rules</a></li>
- <li><a href="/docs/Theta/KMVupdateVkth.html">Update V(kth) Rule</a></li>
- </div>
-
- <p id="set-operations-and-p-sampling">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_set_operations_and_p-sampling">Set Operations and P-sampling</a>
- </p>
- <div class="collapse" id="collapse_set_operations_and_p-sampling">
- <li><a href="/docs/Theta/ThetaSketchSetOps.html">Set Operations</a></li>
- <li><a href="/docs/Theta/ThetaPSampling.html"><i>p</i>-Sampling</a></li>
- </div>
-
- <p id="accuracy">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_accuracy">Accuracy</a>
- </p>
- <div class="collapse" id="collapse_accuracy">
- <li><a href="/docs/Theta/ThetaAccuracy.html">Basic Accuracy</a></li>
- <li><a href="/docs/Theta/ThetaAccuracyPlots.html">Accuracy Plots</a></li>
- <li><a href="/docs/Theta/ThetaErrorTable.html">Relative Error Table</a></li>
- <li><a href="/docs/Theta/ThetaSketchSetOpsAccuracy.html">SetOp Accuracy</a></li>
- <li><a href="/docs/Theta/AccuracyOfDifferentKUnions.html">Unions With Different k</a></li>
- </div>
-
- <p id="size">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_size">Size</a>
- </p>
- <div class="collapse" id="collapse_size">
- <li><a href="/docs/Theta/ThetaSize.html">Theta Sketch Size</a></li>
- </div>
-
- <p id="speed">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_speed">Speed</a>
- </p>
- <div class="collapse" id="collapse_speed">
- <li><a href="/docs/Theta/ThetaUpdateSpeed.html">Update Speed</a></li>
- <li><a href="/docs/Theta/ThetaMergeSpeed.html">Merge Speed</a></li>
- </div>
-
- <p id="theta-sketch-theory">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_sketch_theory">Theta Sketch Theory</a>
- </p>
- <div class="collapse" id="collapse_theta_sketch_theory">
- <li><a href="https://github.com/DataSketches/DataSketches.github.io/blob/master/docs/pdf/ThetaSketchFramework.pdf">Theta Sketch Framework (PDF)</a></li>
- <li><a href="https://github.com/DataSketches/DataSketches.github.io/blob/master/docs/pdf/SketchEquations.pdf">Sketch Equations (PDF)</a></li>
- <li><a href="https://github.com/DataSketches/DataSketches.github.io/blob/master/docs/pdf/DataSketches.pdf">DataSketches (PDF)</a></li>
- <li><a href="/docs/Theta/ThetaConfidenceIntervals.html">Confidence Intervals Notes</a></li>
- <li><a href="/docs/Theta/ThetaMergingAlgorithm.html">Merging Algorithm Notes</a></li>
- <li><a href="/docs/Theta/ThetaReferences.html">Theta References</a></li>
- </div>
- </div>
-
- <p id="tuple-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_tuple_sketches">Tuple Sketches</a>
- </p>
- <div class="collapse" id="collapse_tuple_sketches">
- <li><a href="/docs/Tuple/TupleOverview.html">Tuple Overview</a></li>
-
- <p id="tuple-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_tuple_examples">Tuple Examples</a>
- </p>
- <div class="collapse" id="collapse_tuple_examples">
- <li><a href="/docs/Tuple/TupleJavaExample.html">Tuple Java Example</a></li>
- <li><a href="/docs/Tuple/TupleEngagementExample.html">Tuple Engagement Example</a></li>
- <li><a href="/docs/Frequency/FrequentDistinctTuplesSketch.html">Frequent Distinct Tuples Sketch</a></li>
- <li><a href="/docs/Tuple/TuplePigUDFs.html">Tuple Pig UDFs</a></li>
- <li><a href="/docs/Tuple/TupleHiveUDFs.html">Tuple Hive UDFs</a></li>
- <li><a href="/docs/DruidIntegration.html">Using Sketches in Druid</a></li>
- </div>
- </div>
- </div>
-
- <p id="most-frequent">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_most_frequent">Most Frequent</a>
- </p>
- <div class="collapse" id="collapse_most_frequent">
- <li><a href="/docs/Frequency/FrequencySketchesOverview.html">Frequency Sketches Overview</a></li>
-
- <p id="frequent-item-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_frequent_item_sketches">Frequent Item Sketches</a>
- </p>
- <div class="collapse" id="collapse_frequent_item_sketches">
- <li><a href="/docs/Frequency/FrequentItemsOverview.html">Frequent Items Overview</a></li>
- <li><a href="/docs/Frequency/FrequentItemsErrorTable.html">Frequent Items Error Table</a></li>
- <li><a href="/docs/Frequency/FrequentItemsReferences.html">Frequent Items References</a></li>
-
- <p id="most-frequent-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_most_frequent_examples">Most Frequent Examples</a>
- </p>
- <div class="collapse" id="collapse_most_frequent_examples">
- <li><a href="/docs/Frequency/FrequentItemsJavaExample.html">Frequent Items Java Example</a></li>
- <li><a href="/docs/Frequency/FrequentItemsPigUDFs.html">Frequent Items Pig UDFs</a></li>
- <li><a href="/docs/Frequency/FrequentItemsHiveUDFs.html">Frequent Items Hive UDFs</a></li>
- <li><a href="/docs/DruidIntegration.html">Using Sketches in Druid</a></li>
- </div>
- </div>
-
- <p id="frequent-distinct-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_frequent_distinct_sketches">Frequent Distinct Sketches</a>
- </p>
- <div class="collapse" id="collapse_frequent_distinct_sketches">
- <li><a href="/docs/Frequency/FrequentDistinctTuplesSketch.html">Frequent Distinct Tuples Sketch</a></li>
- </div>
- </div>
-
- <p id="quantiles-and-histograms">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_and_histograms">Quantiles And Histograms</a>
- </p>
- <div class="collapse" id="collapse_quantiles_and_histograms">
- <li><a href="/docs/Quantiles/QuantilesOverview.html">Quantiles Overview</a></li>
- <li><a href="/docs/Quantiles/QuantilesAccuracy.html">Quantiles Accuracy and Size</a></li>
-
- <p id="quantiles-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_examples">Quantiles Examples</a>
- </p>
- <div class="collapse" id="collapse_quantiles_examples">
- <li><a href="/docs/Quantiles/QuantilesJavaExample.html">Quantiles Sketch Java Example</a></li>
- <li><a href="/docs/Quantiles/QuantilesPigUDFs.html">Quantiles Sketch Pig UDFs</a></li>
- <li><a href="/docs/Quantiles/QuantilesHiveUDFs.html">Quantiles Sketch Hive UDFs</a></li>
- <li><a href="/docs/DruidIntegration.html">Using Sketches in Druid</a></li>
- </div>
-
- <p id="quantiles-studies">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_studies">Quantiles Studies</a>
- </p>
- <div class="collapse" id="collapse_quantiles_studies">
- <li><a href="/docs/Quantiles/KLLSketch.html">New KLL sketch and comparison with DoublesSketch</a></li>
- <li><a href="/docs/Quantiles/KllSketchVsTDigest.html">KLL sketch vs t-digest</a></li>
- <li><a href="/docs/Quantiles/DruidApproxHistogramStudy.html">Druid Approximate Histogram</a></li>
- <li><a href="/docs/Quantiles/MomentsSketchStudy.html">Moments Sketch Study</a></li>
- <li><a href="/docs/Quantiles/QuantilesStreamAStudy.html">Quantiles StreamA Study</a></li>
- </div>
-
- <p id="quantiles-sketch-theory">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_sketch_theory">Quantiles Sketch Theory</a>
- </p>
- <div class="collapse" id="collapse_quantiles_sketch_theory">
- <li><a href="https://github.com/DataSketches/DataSketches.github.io/blob/master/docs/pdf/Quantiles_KLL.pdf">Optimal Quantile Approximation in Streams</a></li>
- <li><a href="/docs/Quantiles/QuantilesReferences.html">Quantiles References</a></li>
- </div>
- </div>
-
- <p id="sampling">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_sampling">Sampling</a>
- </p>
- <div class="collapse" id="collapse_sampling">
- <li><a href="/docs/Sampling/ReservoirSampling.html">Reservoir Sampling</a></li>
- <li><a href="/docs/Sampling/ReservoirSamplingPerformance.html">Reservoir Sampling Performance</a></li>
- <li><a href="/docs/Sampling/VarOptSampling.html">VarOpt Sampling</a></li>
-
- <p id="sampling-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_sampling_examples">Sampling Examples</a>
- </p>
- <div class="collapse" id="collapse_sampling_examples">
- <li><a href="/docs/Sampling/ReservoirSamplingJava.html">Reservoir Sampling Java Example</a></li>
- <li><a href="/docs/Sampling/ReservoirSamplingPigUDFs.html">Reservoir Sampling Pig UDFs</a></li>
- <li><a href="/docs/Sampling/VarOptSamplingJava.html">VarOpt Sampling Java Example</a></li>
- <li><a href="/docs/Sampling/VarOptPigUDFs.html">VarOpt Sampling Pig UDFs</a></li>
- </div>
- </div>
-
- <p id="memory-package">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_memory_package">Memory Package</a>
- </p>
- <div class="collapse" id="collapse_memory_package">
- <li><a href="/docs/Memory/MemoryPackage.html">Memory Package</a></li>
- <li><a href="/docs/Memory/MemoryPerformance.html">Memory Package Performance</a></li>
- </div>
-
- <p id="command-line">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_command_line">Command Line</a>
- </p>
- <div class="collapse" id="collapse_command_line">
- <li><a href="/docs/CommandLine/CommandLine.html">Creating Command Line Executables</a></li>
- </div>
-
- <p id="who-uses">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_who_uses">Who Uses</a>
- </p>
- <div class="collapse" id="collapse_who_uses">
- <li><a href="/docs/WhoUses.html">Who Uses</a></li>
- </div>
-</div>
-<script>
- (function () {
-
- var findLineItem = function (path) {
- return document.querySelector(`[href="${path}"]`);
- };
-
- var highlighLineItem = function (element) {
- element.classList.add('highlight');
- };
-
- var checkHasClass = function (element, className) {
- return element.className.split(' ').find(function (item) { return item === className || '' })
- }
-
- var findAllCollapseParents = function (element) {
- var collapseMenus = [];
- var elementPointer = element;
- while (elementPointer !== document.body) {
- if (checkHasClass(elementPointer, 'collapse')) {
- collapseMenus.push(elementPointer);
- }
- elementPointer = elementPointer.parentElement
- }
- return collapseMenus
- };
-
- var findMatchingCollapseMenu = function (collapseItem) {
- return document.querySelector(`[href="#${collapseItem.id}"]`);
- };
-
- var findAllParentMenus = function (element) {
- var parentMenus = []
- var elementPointer = element;
- while (elementPointer !== document.body) {
- if (elementPointer.className.split(' ').indexOf('menu') > -1) {
- parentMenus.push(elementPointer);
- }
- elementPointer = elementPointer.parentElement;
- }
- return parentMenus;
- }
-
- var openMenuItem = function (element) {
- element.click();
- };
-
- var openAllFromList = function (elementList) {
- elementList.forEach(function (element) {
- console.log(element)
- openMenuItem(findMatchingCollapseMenu(element))
- });
- };
-
- var highlightAndOpenMenu = function () {
- var currentLineItem = findLineItem(document.location.pathname);
- highlighLineItem(currentLineItem)
- openAllFromList(findAllCollapseParents(currentLineItem));
- };
-
- $(highlightAndOpenMenu);
-
- }());
-</script>
-
-
- </div>
- <div class="col-md-9 doc-content">
- <a class="btn btn-default btn-xs visible-xs-inline-block visible-sm-inline-block" href="/docs/toc.html">Table of Contents</a>
- <!--
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
--->
-
-<h1 id="apache-datasketches-incubating">Apache DataSketches (incubating)</h1>
-<h2 id="a-software-library-of-stochastic-streaming-algorithms">A software library of stochastic streaming algorithms</h2>
-
-<p>Apache DataSketches is an open source, high-performance library of stochastic streaming algorithms commonly called “sketches” in the data sciences. Sketches are small, stateful programs that process massive data as a stream and provide approximate answers, with mathematical guarantees, to computationally difficult queries orders-of-magnitude faster than traditional, exact methods.</p>
-
-<p>After 8 years of development and 5 years as in Open Source, we have begun the important migration from a stand-alone GitHub site to being a member of the Apache Software Foundation community. While we undergo this migration, we beg your patience.</p>
-
-<ul>
- <li>Ways to contact us:
- <ul>
- <li>We have two ASF <a href="http://the-ASF.slack.com">the-ASF.slack.com</a> slack channels:
- <ul>
- <li>datasketches – general user questions</li>
- <li>datasketches-dev – similar to our Apache <a href="mailto:dev@datasketches.apache.org">dev@datasketches.apache.org</a>, except more interactive, but not as easily searchable.</li>
- </ul>
- </li>
- <li>For discussions about problems with any of our sketch repository components (core-java, core-cpp, Hive, Pig, Vector, Android, etc.)
- <ul>
- <li><a href="https://groups.google.com/forum/#!forum/sketches-user">Google-groups forum</a></li>
- </ul>
- </li>
- <li>For issues about a specific component, please open issues on the appropriate GitHub repository (listed below).</li>
- <li>If you wish to contribute to our sketch development please contact us on our development email list
- <ul>
- <li><a href="mailto:dev@datasketches.apache.org">dev@datasketches.apache.org</a></li>
- </ul>
- </li>
- </ul>
- </li>
- <li>Finding our code releases:
- <ul>
- <li>Apache releases (versions 1.X.X):
- <ul>
- <li>Zip source: <a href="https://dist.apache.org/repos/dist/release/incubator/datasketches/">dist.apache.org/repos/dist/release/incubator/datasketches/</a></li>
- <li>Jar Artifacts (Java only): <a href="https://repository.apache.org/#nexus-search;quick~datasketches">repository.apache.org/#nexus-search;quick~datasketches</a></li>
- </ul>
- </li>
- <li>Pre-Apache releases (versions 0.X.X)
- <ul>
- <li>Jar Artifacts (Java only): <a href="https://search.maven.org/classic/#search%7Cga%7C1%7Ccom.yahoo.datasketches">search.maven.org/classic/#search%7Cga%7C1%7Ccom.yahoo.datasketches</a></li>
- </ul>
- </li>
- </ul>
- </li>
- <li>
- <p>As the repositories under GitHub.io/DataSketches migrate they will disapear from the the GitHub.com/DataSketches organization page. Please refer to this list be directed to the proper locations.</p>
- </li>
- <li>Web Sites
- <ul>
- <li><a href="http://DataSketches.GitHub.io">Original Website</a></li>
- <li><a href="https://datasketches.apache.org">New Website</a></li>
- <li><a href="https://github.com/DataSketches/DataSketches.github.io">Original Website Source</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-website">New Website Source</a></li>
- </ul>
- </li>
- <li>Java
- <ul>
- <li><a href="https://github.com/apache/incubator-datasketches-memory">memory now incubator-datasketches-memory</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-java">sketches-core now incubator-datasketches-java</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-hive">sketches-hive now incubator-datasketches-hive</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-pig">sketches-pig now incubator-datasketches-pig</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-vector">sketches-vector now incubator-datasketches-vector</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-characterization">characterization now incubator-datasketches-characterization</a></li>
- </ul>
- </li>
- <li>C++
- <ul>
- <li><a href="https://github.com/apache/incubator-datasketches-cpp">sketches-core-cpp now incubator-datasketches-cpp</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-postgresql">sketches-postgres now incubator-datasketches-postgresql</a></li>
- </ul>
- </li>
- <li>Java/C++/Python (the following may be moved to Apache later)
- <ul>
- <li><a href="https://github.com/DataSketches/sketches-misc">sketches-misc</a></li>
- <li><a href="https://github.com/DataSketches/sketches-android">sketches-android</a></li>
- <li><a href="https://github.com/DataSketches/experimental">experimental</a></li>
- <li><a href="https://github.com/DataSketches/characterization-cpp">characterization-cpp</a></li>
- <li><a href="https://github.com/DataSketches/experimental-cpp">experimental-cpp</a></li>
- </ul>
- </li>
- <li>Command Line (These may move to Apache later or replaced by Python)
- <ul>
- <li><a href="https://github.com/DataSketches/homebrew-sketches-cmd">homebrew-sketches-cmd</a></li>
- <li><a href="https://github.com/DataSketches/sketches-cmd">sketches-cmd</a></li>
- <li><a href="https://github.com/DataSketches/homebrew-sketches">homebrew-sketches</a></li>
- </ul>
- </li>
-</ul>
-
-<hr />
-
-<p>Disclaimer: Apache DataSketches is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indic [...]
-
- </div>
- </div>
- </div>
-
- <!-- Start page_footer.html include -->
-<footer class="ds-footer">
- <div class="container">
- <div class="text-center">
- <p>
- <a href="/overview.html">
- <span class="fa fa-info-circle"></span> Overview</a> · 
- <a href="/docs/downloads.html">
- <span class="fa fa-download"></span> Download</a> · 
- <a href="https://github.com/apache?utf8=%E2%9C%93&q=datasketches">
- <span class="fa fa-github"></span> GitHub</a> · 
- <a href="https://groups.google.com/forum/#!forum/sketches-user">
- <span class="fa fa-comment"></span> Comments</a> · 
- <a href="https://github.com/apache/incubator-datasketches-website/blob/asf-site/LICENSE">Licensing</a>
- </p>
- </div>
- </div>
-</footer>
-<!-- End page_footer.html include -->
-
-</body>
-
-</html>
-<!-- End _layouts/doc_page.html-->
diff --git a/pom.xml b/pom.xml
deleted file mode 100644
index 9c1939c..0000000
--- a/pom.xml
+++ /dev/null
@@ -1,630 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-<project xmlns="http://maven.apache.org/POM/4.0.0"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
- <modelVersion>4.0.0</modelVersion>
- <parent>
- <groupId>org.apache</groupId>
- <artifactId>apache</artifactId>
- <version>21</version>
- </parent>
- <groupId>org.apache.datasketches</groupId>
-
- <!-- UNIQUE FOR THIS JAVA COMPONENT -->
- <artifactId>datasketches-website</artifactId>
- <version>1.0.0-incubating-SNAPSHOT</version>
- <description>Website source for the DataSketches library.</description>
- <!-- END: UNIQUE FOR THIS JAVA COMPONENT -->
-
- <url>https://datasketches.apache.org/</url>
- <name>${project.artifactId}</name>
- <inceptionYear>2015</inceptionYear>
- <packaging>jar</packaging>
- <!-- jar is the default -->
- <!-- With the 3.6.1 version of Maven this causes a warning to obsolete this in
- favor of using the enforcer plugin (which we do). But removing this
- causes the command "mvn versions:display-plugin-updates" to issue an error
- and not work properly. So this will stay for the time being.
- -->
- <prerequisites>
- <maven>3.2.0</maven>
- </prerequisites>
- <licenses>
- <license>
- <name>Apache License, Version 2.0</name>
- <url>https://www.apache.org/licenses/LICENSE-2.0</url>
- <distribution>repo</distribution>
- </license>
- </licenses>
- <scm>
- <connection>scm:git:ssh://git@github.com/apache/incubator-${project.artifactId}.git</connection>
- <developerConnection>scm:git:ssh://git@github.com/apache/incubator-${project.artifactId}.git</developerConnection>
- <url>https://github.com/apache/incubator-${project.artifactId}</url>
- <tag>HEAD</tag>
- </scm>
- <issueManagement>
- <!-- <system>jira</system>
- <url>https://issues.apache.org/jira/browse/DATASKETCHES</url> -->
- <system>GitHub</system>
- <url>https://github.com/apache/incubator-${project.artifactId}/issues</url>
- </issueManagement>
- <mailingLists>
- <mailingList>
- <name>DataSketches Developers</name>
- <subscribe>dev-subscribe@datasketches.apache.org</subscribe>
- <unsubscribe>dev-unsubscribe@datasketches.apache.org</unsubscribe>
- <post>dev@datasketches.apache.org</post>
- <archive>https://mail-archives.apache.org/mod_mbox/datasketches-dev</archive>
- </mailingList>
- <mailingList>
- <name>sketches-user</name>
- <archive>https://groups.google.com/forum/#!forum/sketches-user</archive>
- <subscribe>mailto:sketches-user%2Bsubscribe@googlegroups.com</subscribe>
- <unsubscribe>mailto:sketches-user%2Bunsubscribe@googlegroups.com</unsubscribe>
- <post>mailto:sketches-user@googlegroups.com</post>
- </mailingList>
- </mailingLists>
- <developers>
- <developer>
- <name>The Apache DataSketches Team</name>
- <email>dev@datasketches.apache.org</email>
- <url>https://datasketches.apache.org</url>
- <organization>Apache Software Foundation</organization>
- <organizationUrl>http://www.apache.org</organizationUrl>
- </developer>
- </developers>
- <properties>
-
- <!-- UNIQUE FOR THIS JAVA COMPONENT -->
- <org-json.version>20180813</org-json.version>
- <!-- END:UNIQUE FOR THIS JAVA COMPONENT -->
-
- <!-- Test -->
- <testng.version>6.14.3</testng.version>
- <!-- System-wide properties -->
- <argLine>-Xmx4g -Duser.language=en -Duser.country=US -Dfile.encoding=UTF-8</argLine>
- <charset.encoding>UTF-8</charset.encoding>
- <project.build.sourceEncoding>${charset.encoding}</project.build.sourceEncoding>
- <project.build.resourceEncoding>${charset.encoding}</project.build.resourceEncoding>
- <project.reporting.outputEncoding>${charset.encoding}</project.reporting.outputEncoding>
- <java.version>1.8</java.version>
- <maven.compiler.source>${java.version}</maven.compiler.source>
- <maven.compiler.target>${java.version}</maven.compiler.target>
- <!-- org.codehaus.plexus used for strict profile testing-->
- <plexus-compiler-javac-errorprone.version>2.8.5</plexus-compiler-javac-errorprone.version>
- <!-- Maven Plugins -->
- <maven-assembly-plugin.version>3.1.1</maven-assembly-plugin.version>
- <maven-compiler-plugin.version>3.8.1</maven-compiler-plugin.version>
- <maven-deploy-plugin.version>3.0.0-M1</maven-deploy-plugin.version>
- <maven-enforcer-plugin.version>3.0.0-M2</maven-enforcer-plugin.version>
- <maven-gpg-plugin.version>1.6</maven-gpg-plugin.version>
- <maven-jar-plugin.version>3.1.2</maven-jar-plugin.version>
- <maven-javadoc-plugin.version>3.1.1</maven-javadoc-plugin.version>
- <maven-release-plugin.version>2.5.3</maven-release-plugin.version>
- <maven-remote-resources-plugin.version>[1.0,)</maven-remote-resources-plugin.version>
- <maven-source-plugin.version>3.1.0</maven-source-plugin.version>
- <maven-surefire-plugin.version>3.0.0-M3</maven-surefire-plugin.version>
- <!-- Apache Plugins -->
- <apache-rat-plugin.version>0.13</apache-rat-plugin.version>
- <!-- org.jacoco Maven Plugins -->
- <jacoco-maven-plugin.version>0.8.4</jacoco-maven-plugin.version>
- <!-- org.eluder Maven Plugins -->
- <coveralls-maven-plugin.version>4.3.0</coveralls-maven-plugin.version>
- <!-- other -->
- <lifecycle-mapping.version>1.0.0</lifecycle-mapping.version>
- <git-commit-id-plugin.version>3.0.0</git-commit-id-plugin.version>
- </properties>
- <repositories>
- <repository>
- <id>apache.snapshots</id>
- <name>Apache Snapshot Repository</name>
- <url>https://repository.apache.org/content/groups/snapshots/org/apache/datasketches/</url>
- <releases>
- <enabled>false</enabled>
- </releases>
- <snapshots>
- <enabled>true</enabled>
- </snapshots>
- </repository>
- <repository>
- <id>apache</id>
- <name>Apache Releases Repository</name>
- <url>https://repository.apache.org/content/repositories/releases/org/apache/datasketches/</url>
- <releases>
- <enabled>true</enabled>
- </releases>
- <snapshots>
- <enabled>false</enabled>
- </snapshots>
- </repository>
- </repositories>
- <dependencies>
-
- <!-- UNIQUE FOR THIS JAVA COMPONENT -->
- <!-- org.json -->
- <dependency>
- <groupId>org.json</groupId>
- <artifactId>json</artifactId>
- <version>${org-json.version}</version>
- </dependency>
- <!-- END: UNIQUE FOR THIS JAVA COMPONENT -->
-
- <!-- Test Scope -->
- <dependency>
- <groupId>org.testng</groupId>
- <artifactId>testng</artifactId>
- <version>${testng.version}</version>
- <scope>test</scope>
- </dependency>
- </dependencies>
- <build>
- <pluginManagement>
- <plugins>
- <plugin>
- <!-- We want to deploy the artifacts to a staging location for perusal -->
- <!-- Apache Parent pom: apache-release profile -->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-deploy-plugin</artifactId>
- <version>${maven-deploy-plugin.version}</version>
- <configuration>
- <updateReleaseInfo>true</updateReleaseInfo>
- <!-- see maven-install-plugin -->
- </configuration>
- </plugin>
- <plugin>
- <!-- Apache Parent pom, pluginManagement-->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-release-plugin</artifactId>
- <version>${maven-release-plugin.version}</version>
- </plugin>
- <plugin>
- <!-- Extends Apache Parent pom, pluginManagement-->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- <version>${maven-jar-plugin.version}</version>
- <executions>
- <execution>
- <id>default-jar</id>
- <goals>
- <goal>jar</goal>
- </goals>
- </execution>
- <execution>
- <id>default-test-jar</id>
- <goals>
- <goal>test-jar</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <!-- Extends Apache Parent pom, apache-release profile -->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-javadoc-plugin</artifactId>
- <version>${maven-javadoc-plugin.version}</version>
- <configuration>
- <docfilessubdirs>true</docfilessubdirs>
- </configuration>
- <executions>
- <execution>
- <id>attach-javadocs</id>
- <goals>
- <goal>jar</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-enforcer-plugin</artifactId>
- <version>${maven-enforcer-plugin.version}</version>
- <executions>
- <execution>
- <id>enforce-banned-dependencies</id>
- <goals>
- <goal>enforce</goal>
- </goals>
- <configuration>
- <rules>
- <requireJavaVersion>
- <version>1.8.0</version>
- </requireJavaVersion>
- <requireMavenVersion>
- <version>3.2,</version>
- </requireMavenVersion>
- <bannedDependencies>
- <excludes>
- <!--LGPL licenced library-->
- <exclude>com.google.code.findbugs:annotations</exclude>
- </excludes>
- </bannedDependencies>
- </rules>
- <fail>true</fail>
- </configuration>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <!-- Apache Parent pom, pluginManagement-->
- <groupId>org.apache.rat</groupId>
- <artifactId>apache-rat-plugin</artifactId>
- <version>${apache-rat-plugin.version}</version>
- <executions>
- <execution>
- <phase>verify</phase>
- <goals>
- <goal>check</goal>
- </goals>
- </execution>
- </executions>
- <configuration>
- <outputDirectory>${project.basedir}/rat</outputDirectory>
- <consoleOutput>true</consoleOutput>
- <useDefaultExcludes>true</useDefaultExcludes>
- <excludes>
- <!-- rat uses .gitignore for excludes by default -->
- <exclude>**/test/resources/**/*.txt</exclude>
- <!-- UNIQUE FOR THIS JAVA COMPONENT -->
- <exclude>**/*.txt</exclude>
- <exclude>**/*.html</exclude>
- <exclude>**/*.css</exclude>
- <exclude>**/*.js</exclude>
- <exclude>**/*.lock</exclude>
- <exclude>**/*.json</exclude>
- <exclude>**/package-list</exclude>
- <!-- END: UNIQUE FOR THIS JAVA COMPONENT -->
- <exclude>LICENSE</exclude>
- <exclude>NOTICE</exclude>
- <exclude>DISCLAIMER-WIP</exclude>
- </excludes>
- </configuration>
- </plugin>
- <plugin>
- <!-- Extends Apache Parent pom, apache-release profile -->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-source-plugin</artifactId>
- <version>${maven-source-plugin.version}</version>
- <executions>
- <execution>
- <id>attach-sources</id>
- <phase>package</phase>
- <goals>
- <goal>jar-no-fork</goal>
- </goals>
- </execution>
- <execution>
- <id>attach-test-sources</id>
- <phase>package</phase>
- <goals>
- <goal>test-jar-no-fork</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <!-- Apache Parent pom, pluginManagement-->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
- <configuration>
- <trimStackTrace>false</trimStackTrace>
- <useManifestOnlyJar>false</useManifestOnlyJar>
- <redirectTestOutputToFile>true</redirectTestOutputToFile>
- </configuration>
- </plugin>
- <plugin>
- <!-- Generates code coverage report from website. -->
- <groupId>org.jacoco</groupId>
- <artifactId>jacoco-maven-plugin</artifactId>
- <version>${jacoco-maven-plugin.version}</version>
- <executions>
- <execution>
- <id>prepare-agent</id>
- <goals>
- <goal>prepare-agent</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <!-- Submit code coverage report to Coveralls.io. -->
- <groupId>org.eluder.coveralls</groupId>
- <artifactId>coveralls-maven-plugin</artifactId>
- <version>${coveralls-maven-plugin.version}</version>
- <configuration>
- <!-- Since we use Travis CI we do not have to put a Coveralls token here. -->
- </configuration>
- </plugin>
- </plugins>
- </pluginManagement>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-deploy-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-release-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-javadoc-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-enforcer-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.rat</groupId>
- <artifactId>apache-rat-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-source-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-surefire-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.jacoco</groupId>
- <artifactId>jacoco-maven-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.eluder.coveralls</groupId>
- <artifactId>coveralls-maven-plugin</artifactId>
- </plugin>
- </plugins>
- </build>
- <profiles>
- <!-- Ignore nuisance warning from Apache parent plugin:
- "maven-remote-resources-plugin (goal "process") is ignored by m2e".
- This also should fix the Maven warning that it can't find the lifecycle-mapping jar.
- This profile is only active when the property "m2e.version" is set,
- which is the case when building in Eclipse with m2e.
- The ignore below tells m2eclipse to skip the execution.
- -->
- <profile>
- <id>m2e</id>
- <activation>
- <property>
- <name>m2e.version</name>
- </property>
- </activation>
- <build>
- <pluginManagement>
- <plugins>
- <plugin>
- <groupId>org.eclipse.m2e</groupId>
- <artifactId>lifecycle-mapping</artifactId>
- <version>${lifecycle-mapping.version}</version>
- <configuration>
- <lifecycleMappingMetadata>
- <pluginExecutions>
- <pluginExecution>
- <pluginExecutionFilter>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-remote-resources-plugin</artifactId>
- <versionRange>${maven-remote-resources-plugin.version}</versionRange>
- <goals>
- <goal>process</goal>
- </goals>
- </pluginExecutionFilter>
- <action>
- <ignore/>
- </action>
- </pluginExecution>
- </pluginExecutions>
- </lifecycleMappingMetadata>
- </configuration>
- </plugin>
- </plugins>
- </pluginManagement>
- </build>
- </profile>
- <profile>
- <id>strict</id>
- <build>
- <pluginManagement>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
- <dependencies>
- <dependency>
- <groupId>org.codehaus.plexus</groupId>
- <artifactId>plexus-compiler-javac-errorprone</artifactId>
- <version>${plexus-compiler-javac-errorprone.version}</version>
- </dependency>
- </dependencies>
- <configuration>
- <source>${maven.compiler.source}</source>
- <target>${maven.compiler.target}</target>
- <compilerId>javac-with-errorprone</compilerId>
- <forceJavacCompilerUse>true</forceJavacCompilerUse>
- </configuration>
- </plugin>
- </plugins>
- </pluginManagement>
- </build>
- </profile>
- <!-- This profile is used to release signed jars to the Apache Nexus repository.
- This must be executed from a git repository set at the proper Release branch (e.g., 1.1.X-incubating)
- and at a Release Candidate tag (e.g., 1.1.0-incubating-RC1).
- The pom version in the release branch must be properly set to something like: "1.1.0-incubating".
- The pom version in the master would be set to something like: "1.2.0-incubating-SNAPSHOT".
- Test Command: mvn clean verify -Pnexus-jars -DskipTests=true
- Command: mvn clean deploy -Dnexus-jars
- Verify Command (from terminal): gpg -v --verify $ASC $FILE # dashdashverify
- -->
- <profile>
- <id>nexus-jars</id>
- <build>
- <pluginManagement>
- <plugins>
- <plugin>
- <groupId>pl.project13.maven</groupId>
- <artifactId>git-commit-id-plugin</artifactId>
- <version>${git-commit-id-plugin.version}</version>
- <executions>
- <execution>
- <goals>
- <goal>revision</goal>
- </goals>
- <phase>initialize</phase>
- </execution>
- </executions>
- <configuration>
- <dotGitDirectory>${project.basedir}/.git</dotGitDirectory>
- <dateFormatTimeZone>UTC</dateFormatTimeZone>
- <verbose>false</verbose>
- <skipPoms>false</skipPoms>
- <format>json</format>
- <generateGitPropertiesFile>true</generateGitPropertiesFile>
- <generateGitPropertiesFilename>${project.build.directory}/git.properties</generateGitPropertiesFilename>
- <failOnNoGitDirectory>true</failOnNoGitDirectory>
- <failOnUnableToExtractRepoInfo>true</failOnUnableToExtractRepoInfo>
- <commitIdGenerationMode>full</commitIdGenerationMode>
- <includeOnlyProperties>
- <includeProperty>git.branch</includeProperty>
- <includeProperty>git.commit.id.full</includeProperty>
- <includeProperty>git.commit.time</includeProperty>
- <includeProperty>git.commit.user.email</includeProperty>
- <includeProperty>git.tags</includeProperty>
- </includeOnlyProperties>
- <gitDescribe>
- <skip>false</skip>
- <always>true</always>
- <abbrev>7</abbrev>
- <dirty>-dirty</dirty>
- <tags>true</tags>
- <forceLongFormat>true</forceLongFormat>
- </gitDescribe>
- </configuration>
- </plugin>
- <!-- Extends Apache Parent pom, pluginManagement-->
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- <version>${maven-jar-plugin.version}</version>
- <executions>
- <execution>
- <id>default-jar</id>
- <goals>
- <goal>jar</goal>
- </goals>
- </execution>
- <execution>
- <id>default-test-jar</id>
- <goals>
- <goal>test-jar</goal>
- </goals>
- </execution>
- </executions>
- <configuration>
- <injectAllReactorProjects>true</injectAllReactorProjects>
- <archive>
- <manifest>
- <addDefaultEntries>false</addDefaultEntries>
- <addDefaultSpecificationEntries>false</addDefaultSpecificationEntries>
- <addDefaultImplementationEntries>false</addDefaultImplementationEntries>
- </manifest>
- <manifestEntries>
- <Build-Jdk>${java.version} (${java.vendor} ${java.vm.version})</Build-Jdk>
- <Build-OS>${os.name} ${os.arch} ${os.version}</Build-OS>
- <Implementation-Vendor>The Apache Software Foundation</Implementation-Vendor>
- <GroupId-ArtifactId>${project.groupId}:${project.artifactId}</GroupId-ArtifactId>
- <git-branch>${git.branch}</git-branch>
- <git-commit-id>${git.commit.id.full}</git-commit-id>
- <git-commit-time>${git.commit.time}</git-commit-time>
- <git-commit-user-email>${git.commit.user.email}</git-commit-user-email>
- <git-commit-tag>${git.tags}</git-commit-tag>
- </manifestEntries>
- </archive>
- </configuration>
- </plugin>
- <!-- We want to sign the artifacts, POM, and all attached artifacts -->
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-gpg-plugin</artifactId>
- <version>${maven-gpg-plugin.version}</version>
- <executions>
- <execution>
- <id>sign-artifacts</id>
- <phase>verify</phase>
- <goals>
- <goal>sign</goal>
- </goals>
- </execution>
- </executions>
- <configuration>
- <gpgArguments>
- <arg>--verbose</arg>
- <!-- prints the algorithm used -->
- <arg>--personal-digest-preferences=SHA512</arg>
- </gpgArguments>
- </configuration>
- </plugin>
- </plugins>
- </pluginManagement>
- <plugins>
- <plugin>
- <groupId>pl.project13.maven</groupId>
- <artifactId>git-commit-id-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-gpg-plugin</artifactId>
- </plugin>
- </plugins>
- </build>
- </profile>
- <!-- Disable source release assembly for 'apache-release' profile.
- This is performed from a script outside Maven
- -->
- <profile>
- <id>apache-release</id>
- <build>
- <plugins>
- <plugin>
- <artifactId>maven-assembly-plugin</artifactId>
- <version>${maven-assembly-plugin.version}</version>
- <executions>
- <execution>
- <id>source-release-assembly</id>
- <phase>none</phase>
- </execution>
- </executions>
- </plugin>
- </plugins>
- </build>
- </profile>
- </profiles>
-</project>
diff --git a/script/cibuild b/script/cibuild
deleted file mode 100644
index 540ffd6..0000000
--- a/script/cibuild
+++ /dev/null
@@ -1,6 +0,0 @@
-#!/usr/bin/env bash
-set -e # halt script on error
-
-bundle exec jekyll build
-#bundle exec htmlproofer ./_site --disable-external
-
diff --git a/src/main/java/org/apache/datasketches/ByteArrayBuilder.java b/src/main/java/org/apache/datasketches/ByteArrayBuilder.java
deleted file mode 100644
index 9935d61..0000000
--- a/src/main/java/org/apache/datasketches/ByteArrayBuilder.java
+++ /dev/null
@@ -1,210 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.datasketches;
-
-import java.util.Arrays;
-
-/**
- * A mutable sequence of bytes.
- *
- * <p>Implements a modifiable byte array. At any point in time it contains some
- * particular sequence of bytes but the length and content of the sequence can
- * be changed through certain method calls.
- *
- * @author Lee Rhodes
- */
-@SuppressWarnings("javadoc")
-public class ByteArrayBuilder {
- private byte[] arr_;
- private int count_ = 0;
- private int capacity_;
-
- public ByteArrayBuilder() {
- this(1024);
- }
-
- /**
- * Constructs an empty ByteArrayBuilder with an initial capacity specified by
- * the <code>capacity</code> argument.
- *
- * @param capacity the initial capacity.
- * @throws NegativeArraySizeException if the <code>capacity</code> argument is
- * less than <code>0</code>.
- */
- public ByteArrayBuilder(final int capacity) {
- arr_ = new byte[capacity];
- }
-
- /**
- * Returns the current capacity. The capacity is the amount of storage
- * available for newly inserted bytes, beyond which an allocation will occur.
- *
- * @return the current capacity
- */
- public int capacity() {
- return capacity_;
- }
-
- /**
- * Returns the length (byte count).
- *
- * @return the length of the sequence of bytes currently represented by this
- * object
- */
- public int length() {
- return count_;
- }
-
- /**
- * Ensures that the capacity is at least equal to the specified minimum. If
- * the current capacity is less than the argument, then a new internal array
- * is allocated with greater capacity. The new capacity is the larger of:
- * <ul>
- * <li>The <code>minimumCapacity</code> argument.
- * <li>Twice the old capacity, plus <code>2</code>.
- * </ul>
- * If the <code>minimumCapacity</code> argument is nonpositive, this method
- * takes no action and simply returns.
- *
- * @param minimumCapacity the minimum desired capacity.
- */
- public void ensureCapacity(final int minimumCapacity) {
- if (minimumCapacity > arr_.length) {
- expandCapacity(minimumCapacity);
- }
- }
-
- /**
- * This implements the expansion semantics of ensureCapacity with no size
- * check or synchronization.
- */
- void expandCapacity(final int minimumCapacity) {
- int newCapacity = (arr_.length + 1) * 2;
- if (newCapacity < 0) {
- newCapacity = Integer.MAX_VALUE;
- } else if (minimumCapacity > newCapacity) {
- newCapacity = minimumCapacity;
- }
- arr_ = Arrays.copyOf(arr_, newCapacity);
- }
-
- /**
- * Ensures that the space remaining (capacity - count) is at least as large as
- * the given space.
- *
- * @param space the given space in bytes
- */
- public void ensureSpace(final int space) {
- final int newCount = count_ + space;
- if (newCount > arr_.length) {
- expandCapacity(newCount);
- }
- }
-
- /**
- * Appends the given byte to the end of the current byte sequence.
- *
- * @param b byte
- * @return this ByteArrayBuilder
- */
- public ByteArrayBuilder append(final byte b) {
- ensureSpace(1);
- arr_[count_++] = b;
- return this;
- }
-
- /**
- * Appends the given byte array to the end of the current byte sequence.
- *
- * @param bArr byte array
- * @return this ByteArrayBuilder
- */
- public ByteArrayBuilder append(final byte[] bArr) {
- final int len = bArr.length;
- ensureSpace(len);
- System.arraycopy(bArr, 0, arr_, count_, len);
- count_ += len;
- return this;
- }
-
- /**
- * Sets the length of the byte sequence. The sequence is changed to a new byte
- * sequence whose length is specified by the argument. For every nonnegative
- * index <i>k</i> less than <code>newLength</code>, the byte at index <i>k</i>
- * in the new byte sequence is the same as the byte at index <i>k</i> in the
- * old sequence if <i>k</i> is less than the length of the old byte sequence;
- * otherwise, it is the null byte <code>0</code>.
- *
- * <p>In other words, if the <code>newLength</code> argument is less than the
- * current length, the length is changed to the specified length.
- *
- * <p>If the <code>newLength</code> argument is greater than or equal to the
- * current length, sufficient null byte (<code>0</code>) are appended so that
- * length becomes the <code>newLength</code> argument.
- *
- * <p>The <code>newLength</code> argument must be greater than or equal to
- * <code>0</code>.
- *
- * @param newLength the new length
- * @throws IndexOutOfBoundsException if the <code>newLength</code> argument is
- * negative.
- */
- public void setLength(final int newLength) {
- if (newLength < 0) {
- throw new ArrayIndexOutOfBoundsException(newLength);
- }
- if (newLength > arr_.length) {
- expandCapacity(newLength);
- }
- if (count_ < newLength) {
- for (; count_ < newLength; count_++) {
- arr_[count_] = 0;
- }
- } else {
- count_ = newLength;
- }
- }
-
- /**
- * Returns a byte array representing the data in this sequence. A new
- * <code>byte[]</code> object is allocated and initialized to contain the byte
- * sequence currently represented by this object. This <code>byte[]</code> is
- * then returned. Subsequent changes to this sequence do not affect the
- * contents of the <code>byte[]</code>.
- *
- * @return a byte array representation of this byte sequence.
- */
- public byte[] toByteArray() {
- return Arrays.copyOf(arr_, count_);
- }
-
- /**
- * Attempts to reduce storage used for the byte sequence. If the buffer is
- * larger than necessary to hold its current sequence of bytes, then it may be
- * resized to become more space efficient. Calling this method may, but is not
- * required to, affect the value returned by a subsequent call to the
- * {@link #capacity()} method.
- */
- public void trimToSize() {
- if (count_ < arr_.length) {
- arr_ = Arrays.copyOf(arr_, count_);
- }
- }
-}
diff --git a/src/main/java/org/apache/datasketches/Files.java b/src/main/java/org/apache/datasketches/Files.java
deleted file mode 100644
index cb94dcd..0000000
--- a/src/main/java/org/apache/datasketches/Files.java
+++ /dev/null
@@ -1,1040 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.datasketches;
-
-import static java.nio.channels.FileChannel.MapMode.READ_ONLY;
-
-import java.io.BufferedReader;
-import java.io.BufferedWriter;
-import java.io.File;
-import java.io.FileDescriptor;
-import java.io.FileInputStream;
-import java.io.FileNotFoundException;
-import java.io.FileOutputStream;
-import java.io.IOException;
-import java.io.InputStreamReader;
-import java.io.OutputStreamWriter;
-import java.io.PrintWriter;
-import java.io.RandomAccessFile;
-import java.io.Reader;
-import java.nio.BufferUnderflowException;
-import java.nio.ByteBuffer;
-import java.nio.MappedByteBuffer;
-import java.nio.channels.FileChannel;
-import java.nio.charset.Charset;
-import java.util.Arrays;
-
-/**
- * A collection of useful static file handlers that conveniently convert the
- * java.io checked exceptions into runtime exceptions.
- *
- * @author Lee Rhodes
- */
-@SuppressWarnings("javadoc")
-public final class Files {
- private static final String LS = System.getProperty("line.separator");
- private static final byte CR = 0xD;
- private static final byte LF = 0xA;
- public static final int DEFAULT_BUFSIZE = 8192;
-
- // Common IO & NIO file methods
-
- /**
- * If the fileName string is null or empty, this method throws a
- * RuntimeException.
- *
- * @param fileName the given fileName
- * @throws RuntimeException if fileName is null or empty.
- */
- public static void checkFileName(final String fileName) {
- if (fileName == null) {
- throw new RuntimeException(LS + "FileName is null.");
- }
- if (fileName.length() == 0) {
- throw new RuntimeException(LS + "FileName is empty.");
- }
- return;
- }
-
- /**
- * Gets an existing normal file as a File. If fileName is null, or empty, or
- * if the file is actually a directory, or doesn't exist this will throw a
- * Runtime Exception; otherwise it will return the fileName as a File object.
- *
- * @param fileName the given fileName
- * @return a File object
- * @throws RuntimeException if fileName cannot resolve to a existing normal
- * file.
- */
- public static File getExistingFile(final String fileName) {
- checkFileName(fileName);
- final File file = new File(fileName);
- if (file.isFile()) {
- return file;
- }
- if (file.isDirectory()) {
- throw new RuntimeException(LS + "FileName is a Directory: " + fileName);
- }
-
- throw new RuntimeException(LS + "FileName does not exist: " + fileName);
- }
-
- /**
- * Returns true if file is a normal file and not a directory.
- *
- * @param fileName the given fileName
- * @return true if file is a normal file and not a directory.
- * @throws RuntimeException if fileName is null or empty.
- */
- public static boolean isFileValid(final String fileName) {
- checkFileName(fileName);
- final File file = new File(fileName);
- return file.isFile();
- }
-
- /**
- * Gets the System.getProperty("user.dir"), which is the expected location of
- * the user root directory.
- *
- * @return location of user root directory
- */
- public static String getUserDir() {
- return System.getProperty("user.dir");
- }
-
- /**
- * Opens a RandomAccessFile given a File object and String mode: "r", "rw",
- * "rws" or "rwd". The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @param mode the given mode
- * @return RandomAccessFile
- * @throws RuntimeException if an IOException occurs.
- */
- public static RandomAccessFile openRandomAccessFile(final File file, final String mode) {
- final RandomAccessFile raf;
- try {
- raf = new RandomAccessFile(file, mode);
- } catch (final FileNotFoundException e) {
- throw new RuntimeException(LS + "Failed to open RandomAccessFile " + LS + e);
- }
- return raf;
- }
-
- /**
- * Gets the FileDescriptor from the given RandomAccessFile.
- *
- * @param raf RandomAccessFile
- * @return the FileDescriptor
- * @throws RuntimeException if an IOException occurs.
- */
- public static FileDescriptor getFD(final RandomAccessFile raf) {
- FileDescriptor fd = null;
- try {
- fd = raf.getFD();
- } catch (final IOException e) {
- throw new RuntimeException(LS + "RandomAccessFile.getFD() failure" + LS + e);
- }
- return fd;
- }
-
- /**
- * Sets the position of the given RandomAccessFile to the given position.
- *
- * @param raf RandomAccessFile
- * @param position the given position
- * @throws RuntimeException if an IOException occurs.
- */
- public static void seek(final RandomAccessFile raf, final long position) {
- try {
- raf.seek(position);
- } catch (final IOException e) {
- throw new RuntimeException(LS + "RandomAccessFile seek failure" + LS + e);
- }
- }
-
- /**
- * Reads buf.length bytes into the given buf.
- *
- * @param raf RandomAccessFile
- * @param buf the size of this buffer is the number of bytes requested.
- * @return the number of bytes actually read.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int readBytes(final RandomAccessFile raf, final byte[] buf) {
- final int len = buf.length;
- int read = 0;
- try {
- read = raf.read(buf);
- } catch (final IOException e) {
- throw new RuntimeException(LS + "RandomAccessFile read failure" + LS + e);
- }
- if (read < len) {
- Arrays.fill(buf, read, len, (byte) 0);
- }
- return read;
- }
-
- // ************************
- // NIO OPERATIONS
- // ************************
- // ByteBuffer methods
- /**
- * Gets a MappedByteBuffer from the given FileChannel, mode, position and
- * size.
- *
- * @param fChan the given FileChannel
- * @param mmode the given MapMode
- * @param position the given position
- * @param size the given size
- * @return a MappedByteBuffer
- * @throws RuntimeException if an IOException occurs.
- */
- public static MappedByteBuffer getMappedByteBuffer(final FileChannel fChan,
- final FileChannel.MapMode mmode, final long position, final long size) {
- final MappedByteBuffer mbBuf;
- try {
- mbBuf = fChan.map(mmode, position, size);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return mbBuf;
- }
-
- /**
- * Gets a MappedByteBuffer from the given FileChannel and mode. Assumes a
- * start position of zero and size of the length of the file (via the
- * FileChannel. Equivalent to:<br>
- * getMappedByteBuffer(FileChannel, FileChanel.MapMode, 0L, size(fChan));
- *
- * @param fChan the given FileChannel
- * @param mmode the given MapMode
- * @return a MappedByteBuffer
- * @throws RuntimeException if an IOException occurs.
- */
- public static MappedByteBuffer getMappedByteBuffer(final FileChannel fChan,
- final FileChannel.MapMode mmode) {
- return getMappedByteBuffer(fChan, mmode, 0L, size(fChan));
- }
-
- /**
- * Reads bytes from the given (Mapped)ByteBuffer until either a CR, LF or CRLF
- * is detected in the byte stream and then converts the captured bytes,
- * excluding the CR and LF characters into a string. This method will work
- * with US-ASCII, ISO-8859 families, Windows 1252, and UTF-8. encodings. In
- * general any character encoding that is isomorphic with respect to the
- * exclusive use of the CR (0xD) and the LF (0xA) codes. Equivalent to:<br>
- * readLine(ByteBuffer, ByteArrayBuilder, Charset.defaultCharset());
- *
- * @param mbBuf Given ByteBuffer or MappedByteBuffer
- * @param bab an optional ByteArrayBuilder for internal reuse, which will
- * improve multi-line reading performance. The result of the read as an array
- * of bytes is available from the bab.
- * @return the line as a string
- *
- */
- public static String readLine(final ByteBuffer mbBuf, final ByteArrayBuilder bab) {
- return readLine(mbBuf, bab, Charset.defaultCharset());
- }
-
- /**
- * Reads bytes from the given (Mapped)ByteBuffer until either a CR, LF or CRLF
- * is detected in the byte stream and then converts the captured bytes,
- * excluding the CR and LF characters into a string. This method will work
- * with US-ASCII, ISO-8859 families, Windows 1252, and UTF-8. encodings. In
- * general any character encoding that is isomorphic with respect to the
- * exclusive use of the CR (0xD) and the LF (0xA) codes.
- *
- * @param mbBuf Given ByteBuffer or MappedByteBuffer
- * @param bab an optional ByteArrayBuilder for internal reuse, which will
- * improve multiline reading performance. The result of the read as an array
- * of bytes is available from the bab.
- * @param charset The Charset to use when converting arrays of bytes from the
- * source to a Unicode String (UTF-16).
- * @return The characters of a line, or NULL if End-of-File, or "" if line was
- * empty.
- */
- public static String readLine(final ByteBuffer mbBuf, final ByteArrayBuilder bab,
- final Charset charset) {
- if (!mbBuf.hasRemaining()) {
- return null;
- }
- final ByteArrayBuilder bab1;
- if (bab == null) {
- bab1 = new ByteArrayBuilder();
- } else {
- bab1 = bab;
- bab1.setLength(0);
- }
- while (mbBuf.hasRemaining()) {
- final byte b = mbBuf.get();
- if (b == LF) {
- break; // EOL
- }
- if (b == CR) {
- if (mbBuf.hasRemaining()) {
- // peek next byte without moving position
- if (mbBuf.get(mbBuf.position()) == LF) {
- mbBuf.get(); // consume it
- }
- }
- break; // EOL
- }
- bab1.append(b); // transfer the byte
- }
- if (bab1.length() == 0) {
- if (!mbBuf.hasRemaining()) {
- return null;
- }
- return "";
- }
- final byte[] out = bab1.toByteArray();
- final String s = new String(out, charset);
- return s;
- }
-
- /**
- * Reads a ByteBuffer (or subclass) with a request for numBytes. The result is
- * stuffed into the provided byte[] array (required), which must be larger or
- * equal to numBytes.
- *
- * @param bb The ByteBuffer to read from
- * @param numBytes The requested number of bytes to read.
- * @param out The target array for the bytes.
- * @return the actual number of bytes read.
- * @throws BufferUnderflowException if numBytes is greater than bytes
- * available in the buffer.
- */
- public static int readByteBuffer(final ByteBuffer bb, final int numBytes, final byte[] out) {
- final int rem = bb.remaining();
- if (rem == 0) {
- return 0;
- }
- final int nBytes = (rem < numBytes) ? rem : numBytes;
- bb.get(out);
- return nBytes;
- }
-
- // FileChannel methods
- /**
- * Sets the FileChannel position.
- *
- * @param fc FileChannel
- * @param position the position
- * @throws RuntimeException if an IOException occurs.
- */
- public static void position(final FileChannel fc, final long position) {
- try {
- fc.position(position);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- }
-
- /**
- * Gets the current size of the FileChannel.
- *
- * @param fc FileChannel
- * @return the size in bytes.
- * @throws RuntimeException if an IOException occurs.
- */
- public static long size(final FileChannel fc) {
- final long sz;
- try {
- sz = fc.size();
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return sz;
- }
-
- /**
- * Appends the given string to the end of the file specified via the given
- * FileChannel. Equivalent to:<br>
- * append(String, FileChannel, Charset.defaultCharset());
- *
- * @param out the string to append
- * @param fc the given FileChannel
- * @return the number of bytes actually appended.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int append(final String out, final FileChannel fc) {
- return append(out, fc, Charset.defaultCharset());
- }
-
- /**
- * Appends the given string to the end of the file specified via the given
- * FileChannel.
- *
- * @param out the string to append
- * @param fc the given FileChannel
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return the number of bytes actually appended.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int append(final String out, final FileChannel fc, final Charset charset) {
- final int bytes;
- final ByteBuffer bb = ByteBuffer.wrap(out.getBytes(charset));
- try {
- bytes = fc.write(bb, fc.size());
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return bytes;
- }
-
- /**
- * Appends the given byteArr to the end of the file specified via the given
- * FileChannel.
- *
- * @param byteArr the byte[] to append
- * @param fc the given FileChannel
- * @return the number of bytes actually appended.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int append(final byte[] byteArr, final FileChannel fc) {
- final int bytes;
- final ByteBuffer bb = ByteBuffer.wrap(byteArr);
- try {
- bytes = fc.write(bb, fc.size());
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return bytes;
- }
-
- /**
- * Writes the given string out to the file specified via the given FileChannel
- * starting at the given file position. Equivalent to:<br>
- * write(String, FileChannel, long, Charset.defaultCharset());
- *
- * @param out the given string
- * @param fc FileChannel
- * @param position the position
- * @return the total number of bytes written.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int write(final String out, final FileChannel fc, final long position) {
- return write(out, fc, position, Charset.defaultCharset());
- }
-
- /**
- * Writes the given string out to the file specified via the given FileChannel
- * starting at the given file position.
- *
- * @param out the given string
- * @param fc FileChannel
- * @param position the given position
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return the total number of bytes written.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int write(final String out, final FileChannel fc, final long position,
- final Charset charset) {
- final int bytes;
- final ByteBuffer bb = ByteBuffer.wrap(out.getBytes(charset));
- try {
- bytes = fc.write(bb, position);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return bytes;
- }
-
- /**
- * Writes the given byteArr to the file specified via the given FileChannel at
- * the given position.
- *
- * @param byteArr the byte[] to append
- * @param fc the given FileChannel
- * @param position the given position
- * @return the number of bytes actually appended.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int write(final byte[] byteArr, final FileChannel fc, final long position) {
- final int bytes;
- final ByteBuffer bb = ByteBuffer.wrap(byteArr);
- try {
- bytes = fc.write(bb, position);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return bytes;
- }
-
- // Complete NIO BASED FILE WRITE OPERATIONS
- /**
- * Writes the given String text to the fileName using NIO FileChannel.
- *
- * @param text Source string to place in a file. Equivalent to: <br>
- * stringToFileNIO(String, String, Charset.defaultCharset());
- * @param fileName name of target file
- * @return the total number of bytes written.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int stringToFileNIO(final String text, final String fileName) {
- return stringToFileNIO(text, fileName, Charset.defaultCharset());
- }
-
- /**
- * Writes the given String text to the fileName using NIO FileChannel
- *
- * @param text Source string to place in a file.
- * @param fileName name of target file
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return the total number of bytes written.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int stringToFileNIO(final String text, final String fileName,
- final Charset charset) {
- checkFileName(fileName);
- final File file = new File(fileName);
- int bytes = 0;
- if (!file.isFile()) {
- try {
- file.createNewFile();
- } catch (final IOException e) {
- throw new RuntimeException("Cannot create new file: " + fileName + LS + e);
- }
- try (FileChannel fc = openRandomAccessFile(file, "rw").getChannel()) {
- bytes = write(text, fc, 0L, charset);
- } catch (final IOException e) {
- throw new RuntimeException("Cannot create File Channel: " + fileName + LS + e);
- }
- }
-
- return bytes;
- }
-
- /**
- * Appends a String to a file using NIO. If fileName does not exist, this
- * creates a new empty file of that name. This closes the file after
- * appending.
- *
- * @param text is the source String. Equivalent to: <br>
- * appendStringToFileNIO(String, String, Charset.defautCharset());
- * @param fileName the given fileName
- * @return the total number of bytes written
- * @throws RuntimeException if IOException or SecurityException occurs, or if
- * fileName is null or empty.
- */
- public static int appendStringToFileNIO(final String text, final String fileName) {
- return appendStringToFileNIO(text, fileName, Charset.defaultCharset());
- }
-
- /**
- * Appends a String to a file using NIO and a Charset. If fileName does not
- * exist, this creates a new empty file of that name. This closes the file
- * after appending.
- *
- * @param text is the source String.
- * @param fileName the given fileName
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return the total number of bytes written
- * @throws RuntimeException if IOException or SecurityException occurs, or if
- * fileName is null or empty.
- */
- public static int appendStringToFileNIO(final String text, final String fileName,
- final Charset charset) {
- checkFileName(fileName);
- final File file = new File(fileName);
- if (!file.isFile()) { // does not exist
- try {
- file.createNewFile();
- } catch (final Exception e) {
- throw new RuntimeException("Cannot create file: " + fileName + LS + e);
- }
- }
- int bytes = 0;
- try (FileChannel fc = openRandomAccessFile(file, "rw").getChannel()) {
- bytes = append(text, fc, charset);
- } catch (final IOException e) {
- throw new RuntimeException("Cannot create File Channel: " + fileName + LS + e);
- }
- return bytes;
- }
-
- // Complete NIO BASED FILE READ OPERATIONS
- /**
- * Reads a file into a char array using NIO FileChannel, then closes the file. Useful when
- * special handling of Line-Separation characters is required. Equivalent to:
- * <br>
- * fileToCharArrayNIO(String, Charset.defaultCharset());
- *
- * @param fileName the given fileName
- * @return a char[]
- * @throws RuntimeException if IOException occurs.
- * @throws IllegalArgumentException if File size is greater than
- * Integer.MAX_VALUE.
- */
- public static char[] fileToCharArrayNIO(final String fileName) {
- return fileToCharArrayNIO(fileName, Charset.defaultCharset());
- }
-
- /**
- * Reads a file into a char array using NIO FileChannel, then closes the file. Useful when
- * special handling of Line-Separation characters is required.
- *
- * @param fileName the given fileName
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return a char[]
- * @throws RuntimeException if IOException occurs.
- * @throws IllegalArgumentException if File size is greater than
- * Integer.MAX_VALUE.
- */
- public static char[] fileToCharArrayNIO(final String fileName, final Charset charset) {
- final File file = getExistingFile(fileName);
- char[] chArr = null;
- try (RandomAccessFile raf = openRandomAccessFile(file, "r");
- FileChannel fc = raf.getChannel()) {
- final MappedByteBuffer mbBuf = getMappedByteBuffer(fc, READ_ONLY);
- final long len = size(fc);
- if (len > Integer.MAX_VALUE) {
- fc.close();
- throw new IllegalArgumentException("File size cannot exceed Integer.MAX_VALUE.");
- }
- final byte[] in = new byte[(int) len];
- mbBuf.get(in); // fill the buffer
- final String out = new String(in, charset);
- chArr = out.toCharArray();
- } catch (final IOException e) {
- throw new RuntimeException("Could not create or close File Channel.");
- }
- return chArr;
- }
-
- /**
- * Reads a file into a String using NIO FileChannel. Each line of the file is delimited by
- * the current operating systems's "line.separator" characters. Closes the
- * file. This method is equivalent to:<br>
- * fileToStringNIO(String fileName, Charset.defaultCharset())
- *
- * @param fileName given fileName
- * @return a String
- * @throws RuntimeException if IOException occurs.
- */
- public static String fileToStringNIO(final String fileName) {
- return fileToStringNIO(fileName, Charset.defaultCharset());
- }
-
- /**
- * Reads a file into a String using NIO FileChannel. Each line of the file is delimited by
- * the current operating systems's "line.separator" characters. Closes the
- * file.
- *
- * @param fileName given fileName
- * @param charset The Charset to use when converting arrays of bytes from the
- * source to a Unicode String (UTF-16).
- * @return a String
- * @throws RuntimeException if IOException occurs.
- */
- public static String fileToStringNIO(final String fileName, final Charset charset) {
- final File file = getExistingFile(fileName);
- final StringBuilder sb = new StringBuilder(1024);
- try (RandomAccessFile raf = openRandomAccessFile(file, "r");
- FileChannel fChan = raf.getChannel();) {
- final MappedByteBuffer mbBuf = getMappedByteBuffer(fChan, READ_ONLY);
- final ByteArrayBuilder bab = new ByteArrayBuilder();
-
- String s;
- while ((s = readLine(mbBuf, bab, charset)) != null) {
- sb.append(s);
- sb.append(LS);
- }
- } catch (final IOException e) {
- throw new RuntimeException("Cannot create File Channel.");
- }
- return sb.toString();
- }
-
- // *******************************
- // STANDARD IO READER OPERATIONS
- // *******************************
- /**
- * Opens a BufferedReader wrapped around a File. Equivalent to the call<br>
- * openBufferedReader(file, DEFAULT_BUFSIZE) Rethrows any IOException as a
- * RuntimeException. The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @return BufferedReader object
- * @throws RuntimeException if File Not Found.
- */
- public static BufferedReader openBufferedReader(final File file) {
- return openBufferedReader(file, DEFAULT_BUFSIZE, Charset.defaultCharset());
- }
-
- /**
- * Opens a BufferedReader wrapped around a FileReader with a specified file
- * and buffer size. If bufSize is less than the default (8192) the default
- * will be used. The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @param bufSize the given buffer size
- * @return a BufferedReader object
- * @throws RuntimeException if File Not Found.
- */
- public static BufferedReader openBufferedReader(final File file, final int bufSize) {
- return openBufferedReader(file, bufSize, Charset.defaultCharset());
- }
-
- /**
- * Opens a BufferedReader, which specifies a bufSize, wrapped around an
- * InputStreamReader, which specifies a Charset. The InputStreamReader wraps a
- * FileInputStream. If bufSize is less than the default (8192) the default
- * will be used. If the charset is null, Charset.defaultCharset() will be
- * used. The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @param bufSize the given buffer size
- * @param charset the given Charset
- * @return a BufferedReader object
- * @throws RuntimeException if FileNotFoundException occurs.
- */
- public static BufferedReader openBufferedReader(final File file, final int bufSize,
- final Charset charset) {
- final int bufSz = (bufSize < DEFAULT_BUFSIZE) ? DEFAULT_BUFSIZE : bufSize;
- final Charset cSet = (charset == null) ? Charset.defaultCharset() : charset;
- BufferedReader in = null; // default bufsize is 8192.
- try {
- final FileInputStream fis = new FileInputStream(file);
- final InputStreamReader isr = new InputStreamReader(fis, cSet);
- in = new BufferedReader(isr, bufSz);
- } catch (final FileNotFoundException e) { // from FileInputStream
- // never opened, so don't close it.
- throw new RuntimeException(LS + "File Not Found: " + file.getPath() + LS + e);
- }
- return in;
- }
-
- /**
- * Configures a file for string writing. If a file by the same name exists, it will be
- * deleted. If fileName is not fully qualified, it will be relative to the root of this
- * package. The returned object must be closed by the calling program.
- * @param fileName the name of the file to configure
- * @return a PrintWriter
- */
- public static final PrintWriter openPrintWriter(final String fileName) {
- File file = null;
- PrintWriter pw = null;
- if ((fileName != null) && !fileName.isEmpty()) {
- file = new File(fileName);
- if (file.isFile()) {
- file.delete(); //remove old file if it exists
- } else {
- try {
- file.createNewFile();
- } catch (final Exception e) {
- throw new RuntimeException("Cannot create file: " + fileName + LS + e);
- }
- }
- final BufferedWriter bw;
- try {
- final FileOutputStream fos = new FileOutputStream(file, true);
- final OutputStreamWriter osw = new OutputStreamWriter(fos, Charset.defaultCharset());
- bw = new BufferedWriter(osw, 8192);
- } catch (final IOException e) {
- // never opened, so don't close it.
- throw new RuntimeException("Could not create: " + file.getPath() + LS + e);
- }
- pw = new PrintWriter(bw);
- }
- return pw;
- }
-
- /**
- * Tests a Reader object if it is ready.
- *
- * @param in the given Reader
- * @return boolean true if ready.
- * @throws RuntimeException if IOException occurs.
- */
- public static boolean ready(final Reader in) {
- boolean out = false;
- try {
- out = in.ready();
- } catch (final IOException e) {
- throw new RuntimeException(LS + "Reader.ready() unsuccessful: " + LS + e);
- }
- return out;
- }
-
- /**
- * Skips bytes in the given Reader object.
- *
- * @param in the given Reader
- * @param skipLen in bytes.
- * @throws RuntimeException if IOException occurs.
- */
- public static void skip(final Reader in, final long skipLen) {
- try {
- in.skip(skipLen);
- } catch (final IOException e) {
- try {
- in.close();
- } catch (final IOException f) {
- throw new RuntimeException(LS + "Close Unsuccessful" + LS + f);
- }
- throw new RuntimeException(LS + "Reader.skip(len) unsuccessful: " + LS + e);
- }
- }
-
- /**
- * Reads characters from the given Reader into the given character array.
- *
- * @param in the given Reader
- * @param length number of characters to read
- * @param cArr Array must be equal to or larger than length
- * @return number of characters actually read from Reader
- * @throws RuntimeException if IOException occurs.
- */
- public static int readCharacters(final Reader in, final int length, final char[] cArr) {
- int readLen = 0;
- try {
- readLen = in.read(cArr, 0, length);
- } catch (final IOException e) {
- try {
- in.close();
- } catch (final IOException f) {
- throw new RuntimeException(LS + "Close Unsuccessful" + LS + f);
- }
- throw new RuntimeException(LS + "Reader.read(char[],0,len) unsuccessful: " + LS + e);
- }
- return readLen;
- }
-
- /**
- * Reads a file into a char array, then closes the file. Useful when special
- * handling of Line-Separation characters is required. Equivalent to: <br>
- * fileToCharArray(String, int, Charset.defaultCharset());
- *
- * @param fileName the given fileName
- * @return a char[]
- * @throws RuntimeException if IOException occurs.
- * @throws IllegalArgumentException if File size is greater than
- * Integer.MAX_VALUE.
- */
- public static char[] fileToCharArray(final String fileName) {
- return fileToCharArray(fileName, DEFAULT_BUFSIZE, Charset.defaultCharset());
- }
-
- /**
- * Reads a file into a char array, then closes the file. Useful when special
- * handling of Line-Separation characters is required.
- *
- * @param fileName the given fileName
- * @param bufSize if less than 8192 it defaults to 8192
- * @param charset The Charset to use when converting arrays of bytes from the
- * source to a Unicode String (UTF-16).
- * @return a char[]
- * @throws RuntimeException if IOException occurs.
- * @throws IllegalArgumentException if File size is greater than
- * Integer.MAX_VALUE.
- */
- public static char[] fileToCharArray(final String fileName, final int bufSize,
- final Charset charset) {
- final File file = getExistingFile(fileName);
- char[] cArr = null;
- final long fileLen = (long) (file.length() * 1.1); // 10% headroom
- if (fileLen > Integer.MAX_VALUE) {
- throw new IllegalArgumentException(
- LS + "File Size is too large: " + fileLen + " >" + " Max: " + Integer.MAX_VALUE);
- }
- final int len = (int) fileLen;
- try (BufferedReader in = openBufferedReader(file, bufSize, charset)) {
- cArr = new char[len];
- in.read(cArr, 0, len);
- } catch (final IOException e) { // thrown by read()
- throw new RuntimeException(LS + "BufferedReader.read(char[],0,len) unsuccessful: " + LS + e);
- }
- return cArr;
- }
-
- /**
- * Reads a file into a String. Each line of the file is delimited by the
- * current operating systems's "line.separator" characters. Closes the file.
- * Equivalent to: <br>
- * fileToString(String, 8192, Charset.defaultCharset());
- *
- * @param fileName the given fileName
- * @return a String
- * @throws RuntimeException if IOException occurs.
- */
- public static String fileToString(final String fileName) {
- return fileToString(fileName, DEFAULT_BUFSIZE, Charset.defaultCharset());
- }
-
- /**
- * Reads a file into a String. Each line of the file is delimited by the
- * current operating systems's "line.separator" characters. Closes the file.
- *
- * @param fileName the given fileName
- * @param bufSize if less than 8192 it defaults to 8192
- * @param charset The Charset to use when converting arrays of bytes from the
- * source to a Unicode String (UTF-16).
- * @return String
- * @throws RuntimeException if IOException occurs.
- */
- public static String fileToString(final String fileName, final int bufSize,
- final Charset charset) {
- final StringBuilder sb = new StringBuilder();
- final File file = getExistingFile(fileName);
- try (BufferedReader in = openBufferedReader(file, bufSize, charset)) {
- String s;
- while ((s = in.readLine()) != null) {
- sb.append(s);
- sb.append(LS);
- }
- } catch (final IOException e) { // thrown by readLine()
- throw new RuntimeException(LS + "BufferedReader.readLine() unsuccessful: " + LS + e);
- }
- return sb.toString();
- }
-
- // STANDARD IO WRITE OPERATIONS
- /**
- * Opens a BufferedWriter wrapped around a FileWriter with a specified file
- * and buffer size. If bufSize is less than the default (8192) the default
- * will be used. The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @param bufSize the given buffer size
- * @param append to existing file if true.
- * @return BufferedWriter object
- * @throws RuntimeException if IOException occurs.
- */
- public static BufferedWriter openBufferedWriter(final File file, final int bufSize,
- final boolean append) {
- return openBufferedWriter(file, bufSize, append, Charset.defaultCharset());
- }
-
- /**
- * Opens a BufferedWriter wrapped around a FileWriter with a specified file
- * and buffer size. If bufSize is less than the default (8192) the default
- * will be used. The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @param bufSize if less than 8192 it defaults to 8192.
- * @param append to existing file if true.
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return BufferedWriter object
- * @throws RuntimeException if IOException occurs.
- */
- public static BufferedWriter openBufferedWriter(final File file, final int bufSize,
- final boolean append, final Charset charset) {
- final int bufSz = (bufSize < DEFAULT_BUFSIZE) ? DEFAULT_BUFSIZE : bufSize;
- BufferedWriter out = null; // default bufsize is 8192.
- try {
- final FileOutputStream fos = new FileOutputStream(file, append);
- final OutputStreamWriter osw = new OutputStreamWriter(fos, charset);
- out = new BufferedWriter(osw, bufSz);
- } catch (final IOException e) {
- // never opened, so don't close it.
- throw new RuntimeException(LS + "Could not create: " + file.getPath() + LS + e);
- }
- return out;
- }
-
- /**
- * Writes a String to a file using a BufferedWriter. Closes the file.
- *
- * @param text is the source String.
- * @param fileName the given fileName
- * @throws RuntimeException if IOException occurs or if fileName is null or
- * empty.
- */
- public static void stringToFile(final String text, final String fileName) {
- stringToFile(text, fileName, DEFAULT_BUFSIZE, Charset.defaultCharset());
- }
-
- /**
- * Writes a String to a file using a BufferedWriter. Closes the file.
- *
- * @param text is the source String.
- * @param fileName the given fileName
- * @param bufSize if less than 8192 it defaults to 8192.
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @throws RuntimeException if IOException occurs or if fileName is null or
- * empty.
- */
- public static void stringToFile(final String text, final String fileName, final int bufSize,
- final Charset charset) {
- checkFileName(fileName);
- final File file = new File(fileName);
- try (BufferedWriter bw = openBufferedWriter(file, bufSize, false, charset);
- PrintWriter out = new PrintWriter(bw);) {
- out.print(text);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- }
-
- /**
- * Appends a String to a file using a BufferedWriter. If fileName does not
- * exist, this creates a new empty file of that name. This closes the file
- * after appending.
- *
- * @param text is the source String.
- * @param fileName the given fileName
- * @throws RuntimeException if IOException or SecurityException occurs, or if
- * fileName is null or empty.
- */
- public static void appendStringToFile(final String text, final String fileName) {
- appendStringToFile(text, fileName, DEFAULT_BUFSIZE, Charset.defaultCharset());
- }
-
- /**
- * Appends a String to a file using a BufferedWriter, bufSize and Charset. If
- * fileName does not exist, this creates a new empty file of that name. This
- * closes the file after appending.
- *
- * @param text is the source String.
- * @param fileName the given fileName
- * @param bufSize if less than 8192 it defaults to 8192.
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @throws RuntimeException if IOException or SecurityException occurs, or if
- * fileName is null or empty.
- */
- public static void appendStringToFile(final String text, final String fileName,
- final int bufSize, final Charset charset) {
- checkFileName(fileName);
- final File file = new File(fileName);
- if (!file.isFile()) { // does not exist
- try {
- file.createNewFile();
- } catch (final Exception e) {
- throw new RuntimeException("Cannot create file: " + fileName + LS + e);
- }
- }
- try (BufferedWriter bw = openBufferedWriter(file, bufSize, true, charset);
- PrintWriter out = new PrintWriter(bw);) {
- out.print(text);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- }
-}
diff --git a/src/main/java/org/apache/datasketches/docgen/ErrorTables.java b/src/main/java/org/apache/datasketches/docgen/ErrorTables.java
deleted file mode 100644
index 33ea61f..0000000
--- a/src/main/java/org/apache/datasketches/docgen/ErrorTables.java
+++ /dev/null
@@ -1,86 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.datasketches.docgen;
-
-import static org.apache.datasketches.docgen.MarkDownTable.emitAlignLine;
-import static org.apache.datasketches.docgen.MarkDownTable.emitBoldRow;
-import static org.apache.datasketches.docgen.MarkDownTable.emitRow;
-
-import org.testng.annotations.Test;
-
-/**
- * Builds the error tables for Theta QuickSelect and Alpha sketches.
- * @author Lee Rhodes
- */
-
-public class ErrorTables {
- static final String LS = System.getProperty("line.separator");
-
- /**
- * Builds the QuickSelect error table
- */
- @Test
- public void buildQuickSelectTable() {
- buildTable(1.0, 4);
- }
-
- /**
- * Builds the Alpha error table
- */
- @Test
- public void buildAlphaTable() {
- buildTable(.708, 9);
- }
-
- private static void buildTable(double factor, int lgKlow) {
- StringBuilder sb = new StringBuilder();
- int[] colWidths = {10, 12, 7, 7, 7 };
- double f = factor; //.708 for Alpha, 1.0 for QuickSelect
- //the is required on the first line when Col#1 is blank for GitHub to render properly.
- //MackDown doesn't care.
- sb.append(emitRow(colWidths, " ","#Std Dev:", "1","2","3"));
- sb.append(emitAlignLine(colWidths, "CCCCC"));
- sb.append(emitBoldRow(colWidths,"","Conf:", "68.27%","95.45%","99.73%"));
- sb.append(emitBoldRow(colWidths,"LgK","K", "1 RSE","2 RSE","3 RSE"));
- for (int lgK = lgKlow; lgK <= 26; lgK++) { //for Alpha start with 9, QuickSelect start with 4
- String lgKStr = Integer.toString(lgK);
- sb.append(emitRow(colWidths,lgKStr, k(lgK), rse(lgK, 1, f), rse(lgK,2,f), rse(lgK,3,f)));
- }
- println(sb.toString());
- }
-
- static String k(int lgK) {
- int lgk = 1 << lgK;
- return String.format("%,d", lgk);
- }
-
- static String rse(int lgK, int stdDev, double factor) {
- double k = 1 << lgK;
- double rse = (stdDev * factor * 100.0) / Math.sqrt(k - 1.0);
- return String.format("%6.3f%%", rse);
- }
-
- static void println(Object o) { System.out.println(o.toString()); }
-
- static void printf(String fmt, Object ... args) {
- System.out.printf(fmt, args);
- }
-
-}
diff --git a/src/main/java/org/apache/datasketches/docgen/MarkDownTable.java b/src/main/java/org/apache/datasketches/docgen/MarkDownTable.java
deleted file mode 100644
index c570fad..0000000
--- a/src/main/java/org/apache/datasketches/docgen/MarkDownTable.java
+++ /dev/null
@@ -1,133 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.datasketches.docgen;
-
-import static java.lang.Math.max;
-
-/**
- * General purpose MarkDown table generator for GitHub Flavored Markdown.
- * This produces readable source and particularly useful when the data of the
- * table is machine generated.
- * @author Lee Rhodes
- */
-public class MarkDownTable {
- private static final String LS = System.getProperty("line.separator");
- private static final int mcw = 7; //min column width
-
- /**
- * Emit a table row
- * @param colWidths integers representing the column widths of each column.
- * This only makes the markdown source look nice.
- * @param strings the content of each column
- * @return the source table row
- */
- public static String emitRow(int[] colWidths, String ... strings) {
- int cols = colWidths.length;
- if (cols != strings.length) {
- throw new IllegalArgumentException("Unequal # of columns");
- }
- StringBuilder sb = new StringBuilder();
- for (int i = 0; i < cols; i++) {
- String s = strings[i];
- int w = colWidths[i];
- if (s.isEmpty()) {
- sb.append(pad(s, max(w, mcw))).append("|");
- } else {
- sb.append(pad(s, max(w, mcw))).append("|");
- }
- }
- sb.append(LS);
- return sb.toString();
- }
-
- /**
- * Emit a table row where all columns are bold face type
- * @param colWidths integers representing the column widths of each column.
- * This only makes the markdown source look nice.
- * @param strings the content of each column
- * @return the source table row in bold
- */
- public static String emitBoldRow(int[] colWidths, String ... strings) {
- int cols = colWidths.length;
- if (cols != strings.length) {
- throw new IllegalArgumentException("Unequal # of columns");
- }
- StringBuilder sb = new StringBuilder();
- for (int i = 0; i < cols; i++) {
- String s = strings[i];
- int w = colWidths[i];
- if (s.isEmpty()) {
- sb.append(pad(s, max(w, mcw))).append("|");
- } else {
- String s2 = "<b>" + s + "</b>";
- sb.append(pad(s2, max(w, mcw))).append("|");
- }
- }
- sb.append(LS);
- return sb.toString();
- }
-
- /**
- * Emit the alignment line for GitHub MarkDown tables
- * @param colWidths the maximum width for each column
- * @param align a sequence of #columns characters in column order:
- * "C"=center, "L"=left, "R"=right
- * @return the alignment line
- */
- public static String emitAlignLine(int[] colWidths, String align) {
- int cols = colWidths.length;
- StringBuilder sb = new StringBuilder();
- for (int i = 0; i < cols; i++) {
- char c = align.charAt(i);
- int w = colWidths[i];
- if (c == 'C') {
- int numDash = max(w - 2, mcw - 2);
- sb.append(":").append(dupDash(numDash)).append(":|");
- }
- else if (c == 'L') {
- int numDash = max(w - 1, mcw - 1);
- sb.append(":").append(dupDash(numDash)).append("|");
- }
- else {
- int numDash = max(w - 1, mcw - 1);
- sb.append(dupDash(numDash)).append(":|");
- }
- }
- sb.append(LS);
- return sb.toString();
- }
-
- private static String dupDash(int num) {
- StringBuilder sb = new StringBuilder();
- for (int i = 0; i < num; i++) { sb.append("-"); }
- return sb.toString();
- }
-
- private static String pad(String s, int num) {
- StringBuilder sb = new StringBuilder();
- int strLen = s.length();
- if (strLen >= num) { return s; }
- sb.append(s);
- for (int i = 0; i < (num - strLen); i++) {
- sb.append(" ");
- }
- return sb.toString();
- }
-}
diff --git a/src/main/java/org/apache/datasketches/docgen/TocGenerator.java b/src/main/java/org/apache/datasketches/docgen/TocGenerator.java
deleted file mode 100644
index 84aa626..0000000
--- a/src/main/java/org/apache/datasketches/docgen/TocGenerator.java
+++ /dev/null
@@ -1,263 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.datasketches.docgen;
-
-import static org.apache.datasketches.Files.openPrintWriter;
-
-import java.io.File;
-import java.io.PrintWriter;
-import java.util.Iterator;
-
-import org.json.JSONArray;
-import org.json.JSONObject;
-import org.testng.annotations.Test;
-import org.apache.datasketches.Files;
-
-/**
- * @author Lee Rhodes
- */
-public class TocGenerator {
- private static final String LS = System.getProperty("line.separator");
- private int level = 0;
- private PrintWriter pw = null;
- private String jsonScrFile;
- private String htmlScriptFile;
-
- TocGenerator() {} //needed for TestNG
-
- /**
- * To create the Table of Contents for the website:
- * <ol>
- * <li>Edit the JSON source file (refereced below) for the structure you want.</li>
- * <li>Execute this test. The result will be placed in the proper location as part of the web
- * source.</li>
- * <li>Stage the changes and push the web site source to origin.</li>
- * <li>Refresh your browser and confirm that the TOC is correct.
- * </ol>
- * @author Lee Rhodes
- */
- @Test
- public static void runTocGenerator() {
- final String jsonSrcFile = "src/main/resources/docgen/toc.json";
- final String htmlScriptFile = "src/main/resources/docgen/tocScript.html";
- final String tgtTocFile = "_includes/toc.html";
- TocGenerator tocgen = new TocGenerator(jsonSrcFile, htmlScriptFile, tgtTocFile);
- tocgen.readJson();
- }
-
- /**
- * Execute the runTocGenerator above.
- * @param jsonSrcFile The JSON source file
- * @param htmlScriptFile The javascript source file
- * @param tgtTocFile the target toc.html file
- */
- public TocGenerator(final String jsonSrcFile, final String htmlScriptFile, final String tgtTocFile) {
- jsonScrFile = jsonSrcFile;
- this.htmlScriptFile = htmlScriptFile;
- if ((tgtTocFile != null) && (!tgtTocFile.isEmpty())) {
- final File file = new File(tgtTocFile);
- if (file.exists()) { file.delete(); }
- pw = openPrintWriter(tgtTocFile);
- }
- }
-
- /**
- * Reads the JSON source file and the html script file and generates the target toc.html file.
- */
- public void readJson() {
- final StringBuilder sb = new StringBuilder();
- final String jin = Files.fileToString(jsonScrFile);
- final JSONObject jo = new JSONObject(jin);
- final String clazz = jo.getString("class");
- if (clazz.equals("TOC")) { emitToc(jo, sb); }
- else if (clazz.equals("Dropdown")) { emitDropdown(jo, sb); }
- else { emitDoc(jo, sb); }
- if ((htmlScriptFile != null) && (!htmlScriptFile.isEmpty())) {
- final String script = Files.fileToString(htmlScriptFile);
- sb.append(script);
- }
- println(sb.toString());
- }
-
- /**
- * Generates an entire toc.html in a StringBuilder
- * @param toc the input JSON object
- * @param sb the target StringBuilder
- */
- void emitToc(final JSONObject toc, final StringBuilder sb) {
- sb.append("<!-- Computer Generated File, Do Not Edit! -->").append(LS);
- sb.append("<link rel=\"stylesheet\" href=\"/css/toc.css\">").append(LS);
- sb.append("<div id=\"toc\" class=\"nav toc hidden-print\">").append(LS);
-
- //JSONArray
- level++;
- final JSONArray jarr = toc.getJSONArray("array");
- final Iterator<Object> itr = jarr.iterator();
- while (itr.hasNext()) {
- final JSONObject jo = (JSONObject) itr.next();
- final String clazz = jo.getString("class");
- if (clazz.equals("Dropdown")) { emitDropdown(jo, sb); }
- else { emitDoc(jo, sb); }
- }
- level--;
-
- sb.append("</div>").append(LS);
- }
-
- /**
- * Generates a Dropdown object from the input JSON source.
- * This is a recursive co-routine with the emitDoc() function.
- * @param dropdn the input JSON
- * @param sb the target StringBuilder
- */
- void emitDropdown(final JSONObject dropdn, final StringBuilder sb) {
- final String desc = dropdn.getString("desc");
- final String lowercaseDesc = desc.toLowerCase();
- final String pId = lowercaseDesc.replace(' ', '-');
- final String divId = "collapse_" + lowercaseDesc.replace(' ', '_');
- final String href = "#" + divId;
- final String indent = indent(level);
- //paragraph with desc
- sb.append(LS);
- sb.append(indent).append("<p id=").append(quotes(pId)).append(">").append(LS);
- sb.append(indent).append(" ").append("<a data-toggle=\"collapse\" ")
- .append("class=\"menu collapsed\" href=").append(quotes(href)).append(">")
- .append(desc).append("</a>").append(LS);
- sb.append(indent).append("</p>").append(LS);
- //start dropdown array
- sb.append(indent).append("<div class=\"collapse\" ").append("id=").append(quotes(divId))
- .append(">").append(LS);
-
- //JSONArray
- level++;
- final JSONArray jarr = dropdn.getJSONArray("array");
- final Iterator<Object> itr = jarr.iterator();
- while (itr.hasNext()) {
- final JSONObject jo = (JSONObject) itr.next();
- final String clazz = jo.getString("class");
- if (clazz.equals("Dropdown")) { emitDropdown(jo, sb); }
- else { emitDoc(jo, sb); }
- }
- level--;
-
- sb.append(indent).append("</div>").append(LS);
- }
-
- /**
- * Generates a Document object from the input JSON source.
- * This is a recursive co-routine with the emitDropdown() function.
- * @param doc the input JSON
- * @param sb the target StringBuilder
- */
- void emitDoc(final JSONObject doc, final StringBuilder sb) {
- final String dir = doc.getString("dir");
- final String file = doc.getString("file");
- final String desc = doc.getString("desc");
- final boolean pdf = doc.optBoolean("pdf");
- final String indent = indent(level);
- sb.append(indent).append("<li><a href=\"");
- if (dir.equals("ROOT")) { sb.append("/"); }
- else {
- final String baseDir = pdf ? "{{site.docs_pdf_dir}}/" : "{{site.docs_dir}}/";
- sb.append(baseDir);
- if (!dir.isEmpty()) {
- sb.append(dir + "/");
- }
- }
- sb.append(file);
- final String sfx = pdf ? ".pdf" : ".html";
- sb.append(sfx + "\">");
- sb.append(desc);
- sb.append("</a></li>").append(LS);
- }
-
- /**
- * Encases the given string in quotes.
- * @param s the given string
- * @return the quoted string
- */
- public static String quotes(final String s) {
- return '"' + s + '"';
- }
-
- /**
- * @param level indention level
- * @return the indention spaces
- */
- public static String indent(final int level) {
- assert level >= 0;
- final StringBuilder sb = new StringBuilder();
- for (int i = 0; i < level; i++) {
- sb.append(" ");
- }
- return sb.toString();
- }
-
-
- /**
- * The JVM may call this method to close the PrintWriter resource.
- */
- @Override
- protected void finalize() throws Throwable {
- try {
- if (pw != null) {
- pw.close(); // close open files
- }
- } finally {
- super.finalize();
- }
- }
-
- /**
- * Outputs a line to the configured PrintWriter and stdOut.
- * @param s The String to print
- */
- public final void println(final String s) {
- System.out.println(s);
- if (pw != null) {
- pw.println(s);
- pw.flush();
- }
- }
-
- /**
- * Flush any buffered output to the configured PrintWriter.
- */
- public final void flush() {
- if (pw != null) { pw.flush(); }
- }
-
- /**
- * Command line access.
- * @param args three arguments are required:
- * <ol><li>The JSON source file</li>
- * <li>The html script file that is appended to the end.</li>
- * <li>The target toc.html file</li>
- * </ol>
- */
- public static void main(final String[] args) {
- final String jsonSrcFile = args[0];
- final String htmlScriptFile = args[1];
- final String tgtTocFile = args[2];
- final TocGenerator tocgen = new TocGenerator(jsonSrcFile, htmlScriptFile, tgtTocFile);
- tocgen.readJson();
- }
-
-}
diff --git a/src/main/resources/docgen/toc.json b/src/main/resources/docgen/toc.json
deleted file mode 100644
index 7ba42ee..0000000
--- a/src/main/resources/docgen/toc.json
+++ /dev/null
@@ -1,232 +0,0 @@
-{ "class":"TOC", "array":
- [
- { "class":"Dropdown", "desc" : "Overview", "array":
- [
- {"class":"Doc", "desc" : "The Challenge", "dir" : "", "file": "TheChallenge" },
- {"class":"Doc", "desc" : "The Major Sketch Families", "dir" : "", "file": "MajorSketchFamilies" },
- {"class":"Doc", "desc" : "Sketch Origins", "dir" : "", "file": "SketchOrigins" },
- {"class":"Doc", "desc" : "Sketch Elements", "dir" : "", "file": "SketchElements" },
- {"class":"Doc", "desc" : "Key Features", "dir" : "", "file": "KeyFeatures" },
- {"class":"Doc", "desc" : "Large Scale Computing", "dir" : "", "file": "LargeScale" },
- {"class":"Doc", "desc" : "Architecture", "dir" : "", "file": "Architecture" },
- {"class":"Doc", "desc" : "Notes on Order Sensitivity", "dir" : "", "file": "OrderSensitivity" },
- {"class":"Doc", "desc" : "Notes on Concurrency", "dir" : "", "file": "Concurrency" },
- {"class":"Doc", "desc" : "Overview Slide Deck", "dir" : "", "file": "DataSketches_deck", "pdf":"true" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Research", "array":
- [
- {"class":"Doc", "desc" : "Research", "dir" : "", "file": "Research" }
- ]
- },
-
- { "class":"Dropdown", "desc" : "Distinct Counting", "array":
- [
-
- { "class":"Dropdown", "desc" : "HyperLogLog Sketches", "array":
- [
- {"class":"Doc", "desc" : "HLL Sketch", "dir" : "HLL", "file": "HLL" },
- {"class":"Doc", "desc" : "HLL Map Sketch", "dir" : "HLL", "file": "HllMap" },
- { "class":"Dropdown", "desc" : "HLL Examples", "array":
- [
- {"class":"Doc", "desc" : "HLL Sketch Java Example", "dir" : "HLL", "file": "HllJavaExample" },
- {"class":"Doc", "desc" : "HLL Sketch Pig UDFs", "dir" : "HLL", "file": "HllPigUDFs" },
- {"class":"Doc", "desc" : "HLL Sketch Hive UDFs", "dir" : "HLL", "file": "HllHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
- { "class":"Dropdown", "desc" : "HLL Studies", "array":
- [
- {"class":"Doc", "desc" : "HLL vs HLL++", "dir" : "HLL", "file": "Hll_vs_Hllpp" },
- {"class":"Doc", "desc" : "HLL Sketch vs Druid HyperLogLogCollector", "dir" : "HLL", "file": "HllSketchVsDruidHyperLogLogCollector" },
- ]
- },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Theta Sketches", "array":
- [
- { "class":"Doc", "desc" : "Theta Sketch Framework", "dir" : "Theta", "file": "ThetaSketchFramework" },
-
- { "class":"Dropdown", "desc" : "Theta Examples", "array":
- [
- {"class":"Doc", "desc" : "Concurrent Theta Sketch", "dir" : "Theta", "file": "ConcurrentThetaSketch" },
- {"class":"Doc", "desc" : "Theta Sketch Java Example", "dir" : "Theta", "file": "ThetaJavaExample" },
- {"class":"Doc", "desc" : "Theta Sketch Spark Example", "dir" : "Theta", "file": "ThetaSparkExample" },
- {"class":"Doc", "desc" : "Theta Sketch Pig UDFs", "dir" : "Theta", "file": "ThetaPigUDFs" },
- {"class":"Doc", "desc" : "Theta Sketch Hive UDFs", "dir" : "Theta", "file": "ThetaHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "KMV Tutorial", "array":
- [
- {"class":"Doc", "desc" : "The Inverse Estimate", "dir" : "Theta", "file": "InverseEstimate" },
- {"class":"Doc", "desc" : "Empty Sketch", "dir" : "Theta", "file": "KMVempty" },
- {"class":"Doc", "desc" : "First Estimator", "dir" : "Theta", "file": "KMVfirstEst" },
- {"class":"Doc", "desc" : "Better Estimator", "dir" : "Theta", "file": "KMVbetterEst" },
- {"class":"Doc", "desc" : "Rejection Rules", "dir" : "Theta", "file": "KMVrejection" },
- {"class":"Doc", "desc" : "Update V(kth) Rule", "dir" : "Theta", "file": "KMVupdateVkth" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Set Operations and P-sampling", "array":
- [
- {"class":"Doc", "desc" : "Set Operations", "dir" : "Theta", "file": "ThetaSketchSetOps" },
- {"class":"Doc", "desc" : "<i>p</i>-Sampling", "dir" : "Theta", "file": "ThetaPSampling" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Accuracy", "array":
- [
- {"class":"Doc", "desc" : "Basic Accuracy", "dir" : "Theta", "file": "ThetaAccuracy" },
- {"class":"Doc", "desc" : "Accuracy Plots", "dir" : "Theta", "file": "ThetaAccuracyPlots" },
- {"class":"Doc", "desc" : "Relative Error Table", "dir" : "Theta", "file": "ThetaErrorTable" },
- {"class":"Doc", "desc" : "SetOp Accuracy", "dir" : "Theta", "file": "ThetaSketchSetOpsAccuracy" },
- {"class":"Doc", "desc" : "Unions With Different k", "dir" : "Theta", "file": "AccuracyOfDifferentKUnions" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Size", "array":
- [
- {"class":"Doc", "desc" : "Theta Sketch Size", "dir" : "Theta", "file": "ThetaSize" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Speed", "array":
- [
- {"class":"Doc", "desc" : "Update Speed", "dir" : "Theta", "file": "ThetaUpdateSpeed" },
- {"class":"Doc", "desc" : "Merge Speed", "dir" : "Theta", "file": "ThetaMergeSpeed" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Theta Sketch Theory", "array":
- [
- {"class":"Doc", "desc" : "Theta Sketch Framework (PDF)", "dir" : "", "file": "ThetaSketchFramework", "pdf":"true" },
- {"class":"Doc", "desc" : "Sketch Equations (PDF)", "dir" : "", "file": "SketchEquations", "pdf":"true" },
- {"class":"Doc", "desc" : "DataSketches (PDF)", "dir" : "", "file": "DataSketches", "pdf":"true" },
- {"class":"Doc", "desc" : "Confidence Intervals Notes", "dir" : "Theta", "file": "ThetaConfidenceIntervals" },
- {"class":"Doc", "desc" : "Merging Algorithm Notes", "dir" : "Theta", "file": "ThetaMergingAlgorithm" },
- {"class":"Doc", "desc" : "Theta References", "dir" : "Theta", "file": "ThetaReferences" },
- ]
- }
- ]
- },
-
- { "class":"Dropdown", "desc" : "Tuple Sketches", "array":
- [
- {"class":"Doc", "desc" : "Tuple Overview", "dir" : "Tuple", "file": "TupleOverview" },
- { "class":"Dropdown", "desc" : "Tuple Examples", "array":
- [
- {"class":"Doc", "desc" : "Tuple Java Example", "dir" : "Tuple", "file": "TupleJavaExample" },
- {"class":"Doc", "desc" : "Tuple Engagement Example", "dir" : "Tuple", "file": "TupleEngagementExample" },
- {"class":"Doc", "desc" : "Frequent Distinct Tuples Sketch", "dir" : "Frequency", "file": "FrequentDistinctTuplesSketch" },
- {"class":"Doc", "desc" : "Tuple Pig UDFs", "dir" : "Tuple", "file": "TuplePigUDFs" },
- {"class":"Doc", "desc" : "Tuple Hive UDFs", "dir" : "Tuple", "file": "TupleHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
- ]
- },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Most Frequent", "array":
- [
- {"class":"Doc", "desc" : "Frequency Sketches Overview", "dir" : "Frequency", "file": "FrequencySketchesOverview" },
-
- { "class":"Dropdown", "desc" : "Frequent Item Sketches", "array":
- [
- {"class":"Doc", "desc" : "Frequent Items Overview", "dir" : "Frequency", "file": "FrequentItemsOverview" },
- {"class":"Doc", "desc" : "Frequent Items Error Table", "dir" : "Frequency", "file": "FrequentItemsErrorTable" },
- {"class":"Doc", "desc" : "Frequent Items References", "dir" : "Frequency", "file": "FrequentItemsReferences" },
-
- { "class":"Dropdown", "desc" : "Most Frequent Examples", "array":
- [
- {"class":"Doc", "desc" : "Frequent Items Java Example", "dir" : "Frequency", "file": "FrequentItemsJavaExample" },
- {"class":"Doc", "desc" : "Frequent Items Pig UDFs", "dir" : "Frequency", "file": "FrequentItemsPigUDFs" },
- {"class":"Doc", "desc" : "Frequent Items Hive UDFs", "dir" : "Frequency", "file": "FrequentItemsHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Frequent Distinct Sketches", "array":
- [
- {"class":"Doc", "desc" : "Frequent Distinct Tuples Sketch", "dir" : "Frequency", "file": "FrequentDistinctTuplesSketch" },
- ]
- }
- ]
- },
-
- { "class":"Dropdown", "desc" : "Quantiles And Histograms", "array":
- [
- {"class":"Doc", "desc" : "Quantiles Overview", "dir" : "Quantiles", "file": "QuantilesOverview" },
- {"class":"Doc", "desc" : "Quantiles Accuracy and Size", "dir" : "Quantiles", "file": "QuantilesAccuracy" },
-
- { "class":"Dropdown", "desc" : "Quantiles Examples", "array":
- [
- {"class":"Doc", "desc" : "Quantiles Sketch Java Example", "dir" : "Quantiles", "file": "QuantilesJavaExample" },
- {"class":"Doc", "desc" : "Quantiles Sketch Pig UDFs", "dir" : "Quantiles", "file": "QuantilesPigUDFs" },
- {"class":"Doc", "desc" : "Quantiles Sketch Hive UDFs", "dir" : "Quantiles", "file": "QuantilesHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
- { "class":"Dropdown", "desc" : "Quantiles Studies", "array":
- [
- {"class":"Doc", "desc" : "New KLL sketch and comparison with DoublesSketch", "dir" : "Quantiles", "file": "KLLSketch" },
- {"class":"Doc", "desc" : "KLL sketch vs t-digest", "dir" : "Quantiles", "file": "KllSketchVsTDigest" },
- {"class":"Doc", "desc" : "Druid Approximate Histogram", "dir" : "Quantiles", "file": "DruidApproxHistogramStudy" },
- {"class":"Doc", "desc" : "Moments Sketch Study", "dir" : "Quantiles", "file": "MomentsSketchStudy" },
- {"class":"Doc", "desc" : "Quantiles StreamA Study", "dir" : "Quantiles", "file": "QuantilesStreamAStudy" },
- ]
- },
- { "class":"Dropdown", "desc" : "Quantiles Sketch Theory", "array":
- [
- {"class":"Doc", "desc" : "Optimal Quantile Approximation in Streams", "dir" : "", "file": "Quantiles_KLL", "pdf":"true" },
- {"class":"Doc", "desc" : "Quantiles References", "dir" : "Quantiles", "file": "QuantilesReferences" },
- ]
- }
- ]
- },
-
- { "class":"Dropdown", "desc" : "Sampling", "array":
- [
- {"class":"Doc", "desc" : "Reservoir Sampling", "dir" : "Sampling", "file": "ReservoirSampling" },
- {"class":"Doc", "desc" : "Reservoir Sampling Performance", "dir" : "Sampling", "file": "ReservoirSamplingPerformance" },
- {"class":"Doc", "desc" : "VarOpt Sampling", "dir" : "Sampling", "file": "VarOptSampling" },
-
- { "class":"Dropdown", "desc" : "Sampling Examples", "array":
- [
- {"class":"Doc", "desc" : "Reservoir Sampling Java Example", "dir" : "Sampling", "file": "ReservoirSamplingJava" },
- {"class":"Doc", "desc" : "Reservoir Sampling Pig UDFs", "dir" : "Sampling", "file": "ReservoirSamplingPigUDFs" },
- {"class":"Doc", "desc" : "VarOpt Sampling Java Example", "dir" : "Sampling", "file": "VarOptSamplingJava" },
- {"class":"Doc", "desc" : "VarOpt Sampling Pig UDFs", "dir" : "Sampling", "file": "VarOptPigUDFs" },
- ]
- },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Memory Package", "array":
- [
- {"class":"Doc", "desc" : "Memory Package", "dir" : "Memory", "file": "MemoryPackage" },
- {"class":"Doc", "desc" : "Memory Package Performance", "dir" : "Memory", "file": "MemoryPerformance" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Command Line", "array":
- [
- {"class":"Doc", "desc" : "Creating Command Line Executables", "dir" : "CommandLine", "file": "CommandLine" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Who Uses", "array":
- [
- {"class":"Doc", "desc" : "Who Uses", "dir" : "", "file": "WhoUses" },
- ]
- },
- ]
-}
-
diff --git a/src/main/resources/docgen/tocScript.html b/src/main/resources/docgen/tocScript.html
deleted file mode 100644
index 8a5ec35..0000000
--- a/src/main/resources/docgen/tocScript.html
+++ /dev/null
@@ -1,64 +0,0 @@
-<script>
- (function () {
-
- var findLineItem = function (path) {
- return document.querySelector(`[href="${path}"]`);
- };
-
- var highlighLineItem = function (element) {
- element.classList.add('highlight');
- };
-
- var checkHasClass = function (element, className) {
- return element.className.split(' ').find(function (item) { return item === className || '' })
- }
-
- var findAllCollapseParents = function (element) {
- var collapseMenus = [];
- var elementPointer = element;
- while (elementPointer !== document.body) {
- if (checkHasClass(elementPointer, 'collapse')) {
- collapseMenus.push(elementPointer);
- }
- elementPointer = elementPointer.parentElement
- }
- return collapseMenus
- };
-
- var findMatchingCollapseMenu = function (collapseItem) {
- return document.querySelector(`[href="#${collapseItem.id}"]`);
- };
-
- var findAllParentMenus = function (element) {
- var parentMenus = []
- var elementPointer = element;
- while (elementPointer !== document.body) {
- if (elementPointer.className.split(' ').indexOf('menu') > -1) {
- parentMenus.push(elementPointer);
- }
- elementPointer = elementPointer.parentElement;
- }
- return parentMenus;
- }
-
- var openMenuItem = function (element) {
- element.click();
- };
-
- var openAllFromList = function (elementList) {
- elementList.forEach(function (element) {
- console.log(element)
- openMenuItem(findMatchingCollapseMenu(element))
- });
- };
-
- var highlightAndOpenMenu = function () {
- var currentLineItem = findLineItem(document.location.pathname);
- highlighLineItem(currentLineItem)
- openAllFromList(findAllCollapseParents(currentLineItem));
- };
-
- $(highlightAndOpenMenu);
-
- }());
-</script>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org