You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datasketches.apache.org by le...@apache.org on 2020/01/15 00:19:43 UTC
[incubator-datasketches-website] 01/01: Cleanup asf-site
This is an automated email from the ASF dual-hosted git repository.
leerho pushed a commit to branch clean_asf_site
in repository https://gitbox.apache.org/repos/asf/incubator-datasketches-website.git
commit e443537c3dc550ec4d64a8ae4b1a7829b23a8479
Author: Lee Rhodes <le...@users.noreply.github.com>
AuthorDate: Tue Jan 14 16:19:14 2020 -0800
Cleanup asf-site
---
.gitignore | 4 +
404.html | 24 -
DISCLAIMER-WIP | 26 -
LICENSE | 210 ----
NOTICE | 11 -
README.html | 573 -----------
pom.xml | 630 ------------
script/cibuild | 6 -
.../org/apache/datasketches/ByteArrayBuilder.java | 210 ----
src/main/java/org/apache/datasketches/Files.java | 1040 --------------------
.../apache/datasketches/docgen/ErrorTables.java | 86 --
.../apache/datasketches/docgen/MarkDownTable.java | 133 ---
.../apache/datasketches/docgen/TocGenerator.java | 263 -----
src/main/resources/docgen/toc.json | 232 -----
src/main/resources/docgen/tocScript.html | 64 --
15 files changed, 4 insertions(+), 3508 deletions(-)
diff --git a/.gitignore b/.gitignore
index 9afbe71..f329bfb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,9 +6,13 @@ about.md
Gemfile
Gemfile.lock
_posts/
+.DS_store
+
# Eclipse
.project
+.classpath
+.settings/
# yml files
diff --git a/404.html b/404.html
deleted file mode 100644
index c472b4e..0000000
--- a/404.html
+++ /dev/null
@@ -1,24 +0,0 @@
----
-layout: default
----
-
-<style type="text/css" media="screen">
- .container {
- margin: 10px auto;
- max-width: 600px;
- text-align: center;
- }
- h1 {
- margin: 30px 0;
- font-size: 4em;
- line-height: 1;
- letter-spacing: -1px;
- }
-</style>
-
-<div class="container">
- <h1>404</h1>
-
- <p><strong>Page not found :(</strong></p>
- <p>The requested page could not be found.</p>
-</div>
diff --git a/DISCLAIMER-WIP b/DISCLAIMER-WIP
deleted file mode 100644
index c90f3f7..0000000
--- a/DISCLAIMER-WIP
+++ /dev/null
@@ -1,26 +0,0 @@
-Apache DataSketches (incubating) is an effort undergoing incubation
-at The Apache Software Foundation (ASF), sponsored by the Apache Incubator.
-
-Incubation is required of all newly accepted projects until a further review
-indicates that the infrastructure, communications, and decision making process
-have stabilized in a manner consistent with other successful ASF projects.
-
-While incubation status is not necessarily a reflection of the
-completeness or stability of the code, it does indicate that the
-project has yet to be fully endorsed by the ASF.
-
-Some of the incubating project's releases may not be fully compliant
-with ASF policy. For example, releases may have incomplete or
-un-reviewed licensing conditions. What follows is a list of known
-issues the project is currently aware of (note that this list, by
-definition, is likely to be incomplete):
-
- * The LICENSE and NOTICE files may not be complete and will be fixed with the next release.
-
-If you are planning to incorporate this work into your
-product or project, please be aware that you will need to conduct a
-thorough licensing review to determine the overall implications of
-including this work. For the current status of this project through the Apache
-Incubator visit:
-
-http://incubator.apache.org/projects/datasketches.html
diff --git a/LICENSE b/LICENSE
deleted file mode 100644
index 5667780..0000000
--- a/LICENSE
+++ /dev/null
@@ -1,210 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
-
-
-APPENDIX A: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
- -------------------------------------------------------------
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
- -------------------------------------------------------------
-
-
-
-APPENDIX B: Additional licenses relevant to this Apache DataSketches (incubating) component:
- (none)
diff --git a/NOTICE b/NOTICE
deleted file mode 100644
index bd2e463..0000000
--- a/NOTICE
+++ /dev/null
@@ -1,11 +0,0 @@
-Apache DataSketches Website (incubating)
-Copyright 2019 - The Apache Software Foundation
-
-Copyright 2015-2018 Yahoo
-Copyright 2018 - Verizon Media
-
-This product includes software developed at
-The Apache Software Foundation (http://www.apache.org/).
-
-Prior to moving to ASF, the software for this project was developed at
-Yahoo (now Verizon Media) (https://developer.yahoo.com).
diff --git a/README.html b/README.html
deleted file mode 100644
index 8665490..0000000
--- a/README.html
+++ /dev/null
@@ -1,573 +0,0 @@
-<!DOCTYPE html>
-<!-- Start _layouts/doc_page.html-->
-<html lang="en">
-
-<head>
- <!-- Start site_head.html include-->
-<meta charset="UTF-8" />
-<meta name="viewport" content="width=device-width, initial-scale=1.0">
-<meta name="description" content="">
-<meta name="author" content="datasketches">
-
-<title>DataSketches | </title>
-
-<link rel="shortcut icon" href="/img/favicon.png">
-
-<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.1.0/css/font-awesome.min.css">
-<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css">
-
-<link href='https://fonts.googleapis.com/css?family=Open+Sans+Condensed:300,700,300italic|Open+Sans:300italic,400italic,600italic,400,300,600'
- rel='stylesheet' type='text/css'>
-
-<link rel="stylesheet" href="/css/main.css">
-<link rel="stylesheet" href="/css/header.css">
-<link rel="stylesheet" href="/css/footer.css">
-<link rel="stylesheet" href="/css/syntax.css">
-<link rel="stylesheet" href="/css/docs.css">
-
-
-<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML-full">
-</script>
-<script src="https://code.jquery.com/jquery.min.js"></script>
-<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/js/bootstrap.min.js"></script>
-<!-- End site_head.html include-->
-
-</head>
-
-<body>
- <!-- Start page_header.html include -->
-<div class="navbar navbar-inverse navbar-static-top ds-nav">
- <div class="container">
- <div class="navbar-header">
- <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
- <span class="sr-only">Toggle navigation</span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- <span class="icon-bar"></span>
- </button>
- <!-- navbar-brand: https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css -->
- <!-- /css/main.css:57:.logo -->
- <a class="navbar-brand logo" href="/">DataSketches</a>
- </div>
- <div class="navbar-collapse collapse">
- <ul class="nav navbar-nav navbar-right">
- <li>
- <a href="/overview.html">
- <span class="fa fa-info-circle"></span> OVERVIEW</a>
- </li>
- <li>
- <a href="/docs/downloads.html">
- <span class="fa fa-download"></span> DOWNLOAD</a>
- </li>
- <li>
- <a href="https://github.com/apache?utf8=%E2%9C%93&q=datasketches&type=&language=">
- <span class="fa fa-github"></span> GITHUB</a>
- </li>
- <li>
- <a href="/docs/Research.html">
- <span class="fa fa-paper-plane"></span> RESEARCH</a>
- </li>
- <li>
- <a href="https://groups.google.com/forum/#!forum/sketches-user">
- <span class="fa fa-comment"></span> FORUM</a>
- </li>
-
- </ul>
- </div>
- </div>
-</div>
-<!-- End page_header.html include -->
-
-
- <div class="ds-header">
- <div class="container">
- <h4>API Snapshots:
- <a href="/api/java/snapshot/apidocs/index.html">Java Core</a>,
- <a href="/api/memory/snapshot/apidocs/index.html">Memory</a>,
- <a href="/api/pig/snapshot/apidocs/index.html">Pig</a>,
- <a href="/api/hive/snapshot/apidocs/index.html">Hive</a>,
- </h4>
- </div>
- </div>
-
- <div class="container">
- <div class="row">
- <div class="col-md-3">
- <div class="searchbox" style="position:relative">
- <gcse:searchbox-only></gcse:searchbox-only>
- </div>
- <!-- Computer Generated File, Do Not Edit! -->
-<link rel="stylesheet" href="/css/toc.css">
-<div id="toc" class="nav toc hidden-print">
-
- <p id="overview">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_overview">Overview</a>
- </p>
- <div class="collapse" id="collapse_overview">
- <li><a href="/docs/TheChallenge.html">The Challenge</a></li>
- <li><a href="/docs/MajorSketchFamilies.html">The Major Sketch Families</a></li>
- <li><a href="/docs/SketchOrigins.html">Sketch Origins</a></li>
- <li><a href="/docs/SketchElements.html">Sketch Elements</a></li>
- <li><a href="/docs/KeyFeatures.html">Key Features</a></li>
- <li><a href="/docs/LargeScale.html">Large Scale Computing</a></li>
- <li><a href="/docs/Architecture.html">Architecture</a></li>
- <li><a href="/docs/OrderSensitivity.html">Notes on Order Sensitivity</a></li>
- <li><a href="/docs/Concurrency.html">Notes on Concurrency</a></li>
- <li><a href="https://github.com/DataSketches/DataSketches.github.io/blob/master/docs/pdf/DataSketches_deck.pdf">Overview Slide Deck</a></li>
- </div>
-
- <p id="research">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_research">Research</a>
- </p>
- <div class="collapse" id="collapse_research">
- <li><a href="/docs/Research.html">Research</a></li>
- </div>
-
- <p id="distinct-counting">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_distinct_counting">Distinct Counting</a>
- </p>
- <div class="collapse" id="collapse_distinct_counting">
-
- <p id="hyperloglog-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_hyperloglog_sketches">HyperLogLog Sketches</a>
- </p>
- <div class="collapse" id="collapse_hyperloglog_sketches">
- <li><a href="/docs/HLL/HLL.html">HLL Sketch</a></li>
- <li><a href="/docs/HLL/HllMap.html">HLL Map Sketch</a></li>
-
- <p id="hll-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_hll_examples">HLL Examples</a>
- </p>
- <div class="collapse" id="collapse_hll_examples">
- <li><a href="/docs/HLL/HllJavaExample.html">HLL Sketch Java Example</a></li>
- <li><a href="/docs/HLL/HllPigUDFs.html">HLL Sketch Pig UDFs</a></li>
- <li><a href="/docs/HLL/HllHiveUDFs.html">HLL Sketch Hive UDFs</a></li>
- <li><a href="/docs/DruidIntegration.html">Using Sketches in Druid</a></li>
- </div>
-
- <p id="hll-studies">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_hll_studies">HLL Studies</a>
- </p>
- <div class="collapse" id="collapse_hll_studies">
- <li><a href="/docs/HLL/Hll_vs_Hllpp.html">HLL vs HLL++</a></li>
- <li><a href="/docs/HLL/HllSketchVsDruidHyperLogLogCollector.html">HLL Sketch vs Druid HyperLogLogCollector</a></li>
- </div>
- </div>
-
- <p id="theta-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_sketches">Theta Sketches</a>
- </p>
- <div class="collapse" id="collapse_theta_sketches">
- <li><a href="/docs/Theta/ThetaSketchFramework.html">Theta Sketch Framework</a></li>
-
- <p id="theta-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_examples">Theta Examples</a>
- </p>
- <div class="collapse" id="collapse_theta_examples">
- <li><a href="/docs/Theta/ConcurrentThetaSketch.html">Concurrent Theta Sketch</a></li>
- <li><a href="/docs/Theta/ThetaJavaExample.html">Theta Sketch Java Example</a></li>
- <li><a href="/docs/Theta/ThetaSparkExample.html">Theta Sketch Spark Example</a></li>
- <li><a href="/docs/Theta/ThetaPigUDFs.html">Theta Sketch Pig UDFs</a></li>
- <li><a href="/docs/Theta/ThetaHiveUDFs.html">Theta Sketch Hive UDFs</a></li>
- <li><a href="/docs/DruidIntegration.html">Using Sketches in Druid</a></li>
- </div>
-
- <p id="kmv-tutorial">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_kmv_tutorial">KMV Tutorial</a>
- </p>
- <div class="collapse" id="collapse_kmv_tutorial">
- <li><a href="/docs/Theta/InverseEstimate.html">The Inverse Estimate</a></li>
- <li><a href="/docs/Theta/KMVempty.html">Empty Sketch</a></li>
- <li><a href="/docs/Theta/KMVfirstEst.html">First Estimator</a></li>
- <li><a href="/docs/Theta/KMVbetterEst.html">Better Estimator</a></li>
- <li><a href="/docs/Theta/KMVrejection.html">Rejection Rules</a></li>
- <li><a href="/docs/Theta/KMVupdateVkth.html">Update V(kth) Rule</a></li>
- </div>
-
- <p id="set-operations-and-p-sampling">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_set_operations_and_p-sampling">Set Operations and P-sampling</a>
- </p>
- <div class="collapse" id="collapse_set_operations_and_p-sampling">
- <li><a href="/docs/Theta/ThetaSketchSetOps.html">Set Operations</a></li>
- <li><a href="/docs/Theta/ThetaPSampling.html"><i>p</i>-Sampling</a></li>
- </div>
-
- <p id="accuracy">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_accuracy">Accuracy</a>
- </p>
- <div class="collapse" id="collapse_accuracy">
- <li><a href="/docs/Theta/ThetaAccuracy.html">Basic Accuracy</a></li>
- <li><a href="/docs/Theta/ThetaAccuracyPlots.html">Accuracy Plots</a></li>
- <li><a href="/docs/Theta/ThetaErrorTable.html">Relative Error Table</a></li>
- <li><a href="/docs/Theta/ThetaSketchSetOpsAccuracy.html">SetOp Accuracy</a></li>
- <li><a href="/docs/Theta/AccuracyOfDifferentKUnions.html">Unions With Different k</a></li>
- </div>
-
- <p id="size">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_size">Size</a>
- </p>
- <div class="collapse" id="collapse_size">
- <li><a href="/docs/Theta/ThetaSize.html">Theta Sketch Size</a></li>
- </div>
-
- <p id="speed">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_speed">Speed</a>
- </p>
- <div class="collapse" id="collapse_speed">
- <li><a href="/docs/Theta/ThetaUpdateSpeed.html">Update Speed</a></li>
- <li><a href="/docs/Theta/ThetaMergeSpeed.html">Merge Speed</a></li>
- </div>
-
- <p id="theta-sketch-theory">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_sketch_theory">Theta Sketch Theory</a>
- </p>
- <div class="collapse" id="collapse_theta_sketch_theory">
- <li><a href="https://github.com/DataSketches/DataSketches.github.io/blob/master/docs/pdf/ThetaSketchFramework.pdf">Theta Sketch Framework (PDF)</a></li>
- <li><a href="https://github.com/DataSketches/DataSketches.github.io/blob/master/docs/pdf/SketchEquations.pdf">Sketch Equations (PDF)</a></li>
- <li><a href="https://github.com/DataSketches/DataSketches.github.io/blob/master/docs/pdf/DataSketches.pdf">DataSketches (PDF)</a></li>
- <li><a href="/docs/Theta/ThetaConfidenceIntervals.html">Confidence Intervals Notes</a></li>
- <li><a href="/docs/Theta/ThetaMergingAlgorithm.html">Merging Algorithm Notes</a></li>
- <li><a href="/docs/Theta/ThetaReferences.html">Theta References</a></li>
- </div>
- </div>
-
- <p id="tuple-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_tuple_sketches">Tuple Sketches</a>
- </p>
- <div class="collapse" id="collapse_tuple_sketches">
- <li><a href="/docs/Tuple/TupleOverview.html">Tuple Overview</a></li>
-
- <p id="tuple-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_tuple_examples">Tuple Examples</a>
- </p>
- <div class="collapse" id="collapse_tuple_examples">
- <li><a href="/docs/Tuple/TupleJavaExample.html">Tuple Java Example</a></li>
- <li><a href="/docs/Tuple/TupleEngagementExample.html">Tuple Engagement Example</a></li>
- <li><a href="/docs/Frequency/FrequentDistinctTuplesSketch.html">Frequent Distinct Tuples Sketch</a></li>
- <li><a href="/docs/Tuple/TuplePigUDFs.html">Tuple Pig UDFs</a></li>
- <li><a href="/docs/Tuple/TupleHiveUDFs.html">Tuple Hive UDFs</a></li>
- <li><a href="/docs/DruidIntegration.html">Using Sketches in Druid</a></li>
- </div>
- </div>
- </div>
-
- <p id="most-frequent">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_most_frequent">Most Frequent</a>
- </p>
- <div class="collapse" id="collapse_most_frequent">
- <li><a href="/docs/Frequency/FrequencySketchesOverview.html">Frequency Sketches Overview</a></li>
-
- <p id="frequent-item-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_frequent_item_sketches">Frequent Item Sketches</a>
- </p>
- <div class="collapse" id="collapse_frequent_item_sketches">
- <li><a href="/docs/Frequency/FrequentItemsOverview.html">Frequent Items Overview</a></li>
- <li><a href="/docs/Frequency/FrequentItemsErrorTable.html">Frequent Items Error Table</a></li>
- <li><a href="/docs/Frequency/FrequentItemsReferences.html">Frequent Items References</a></li>
-
- <p id="most-frequent-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_most_frequent_examples">Most Frequent Examples</a>
- </p>
- <div class="collapse" id="collapse_most_frequent_examples">
- <li><a href="/docs/Frequency/FrequentItemsJavaExample.html">Frequent Items Java Example</a></li>
- <li><a href="/docs/Frequency/FrequentItemsPigUDFs.html">Frequent Items Pig UDFs</a></li>
- <li><a href="/docs/Frequency/FrequentItemsHiveUDFs.html">Frequent Items Hive UDFs</a></li>
- <li><a href="/docs/DruidIntegration.html">Using Sketches in Druid</a></li>
- </div>
- </div>
-
- <p id="frequent-distinct-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_frequent_distinct_sketches">Frequent Distinct Sketches</a>
- </p>
- <div class="collapse" id="collapse_frequent_distinct_sketches">
- <li><a href="/docs/Frequency/FrequentDistinctTuplesSketch.html">Frequent Distinct Tuples Sketch</a></li>
- </div>
- </div>
-
- <p id="quantiles-and-histograms">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_and_histograms">Quantiles And Histograms</a>
- </p>
- <div class="collapse" id="collapse_quantiles_and_histograms">
- <li><a href="/docs/Quantiles/QuantilesOverview.html">Quantiles Overview</a></li>
- <li><a href="/docs/Quantiles/QuantilesAccuracy.html">Quantiles Accuracy and Size</a></li>
-
- <p id="quantiles-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_examples">Quantiles Examples</a>
- </p>
- <div class="collapse" id="collapse_quantiles_examples">
- <li><a href="/docs/Quantiles/QuantilesJavaExample.html">Quantiles Sketch Java Example</a></li>
- <li><a href="/docs/Quantiles/QuantilesPigUDFs.html">Quantiles Sketch Pig UDFs</a></li>
- <li><a href="/docs/Quantiles/QuantilesHiveUDFs.html">Quantiles Sketch Hive UDFs</a></li>
- <li><a href="/docs/DruidIntegration.html">Using Sketches in Druid</a></li>
- </div>
-
- <p id="quantiles-studies">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_studies">Quantiles Studies</a>
- </p>
- <div class="collapse" id="collapse_quantiles_studies">
- <li><a href="/docs/Quantiles/KLLSketch.html">New KLL sketch and comparison with DoublesSketch</a></li>
- <li><a href="/docs/Quantiles/KllSketchVsTDigest.html">KLL sketch vs t-digest</a></li>
- <li><a href="/docs/Quantiles/DruidApproxHistogramStudy.html">Druid Approximate Histogram</a></li>
- <li><a href="/docs/Quantiles/MomentsSketchStudy.html">Moments Sketch Study</a></li>
- <li><a href="/docs/Quantiles/QuantilesStreamAStudy.html">Quantiles StreamA Study</a></li>
- </div>
-
- <p id="quantiles-sketch-theory">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_sketch_theory">Quantiles Sketch Theory</a>
- </p>
- <div class="collapse" id="collapse_quantiles_sketch_theory">
- <li><a href="https://github.com/DataSketches/DataSketches.github.io/blob/master/docs/pdf/Quantiles_KLL.pdf">Optimal Quantile Approximation in Streams</a></li>
- <li><a href="/docs/Quantiles/QuantilesReferences.html">Quantiles References</a></li>
- </div>
- </div>
-
- <p id="sampling">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_sampling">Sampling</a>
- </p>
- <div class="collapse" id="collapse_sampling">
- <li><a href="/docs/Sampling/ReservoirSampling.html">Reservoir Sampling</a></li>
- <li><a href="/docs/Sampling/ReservoirSamplingPerformance.html">Reservoir Sampling Performance</a></li>
- <li><a href="/docs/Sampling/VarOptSampling.html">VarOpt Sampling</a></li>
-
- <p id="sampling-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_sampling_examples">Sampling Examples</a>
- </p>
- <div class="collapse" id="collapse_sampling_examples">
- <li><a href="/docs/Sampling/ReservoirSamplingJava.html">Reservoir Sampling Java Example</a></li>
- <li><a href="/docs/Sampling/ReservoirSamplingPigUDFs.html">Reservoir Sampling Pig UDFs</a></li>
- <li><a href="/docs/Sampling/VarOptSamplingJava.html">VarOpt Sampling Java Example</a></li>
- <li><a href="/docs/Sampling/VarOptPigUDFs.html">VarOpt Sampling Pig UDFs</a></li>
- </div>
- </div>
-
- <p id="memory-package">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_memory_package">Memory Package</a>
- </p>
- <div class="collapse" id="collapse_memory_package">
- <li><a href="/docs/Memory/MemoryPackage.html">Memory Package</a></li>
- <li><a href="/docs/Memory/MemoryPerformance.html">Memory Package Performance</a></li>
- </div>
-
- <p id="command-line">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_command_line">Command Line</a>
- </p>
- <div class="collapse" id="collapse_command_line">
- <li><a href="/docs/CommandLine/CommandLine.html">Creating Command Line Executables</a></li>
- </div>
-
- <p id="who-uses">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_who_uses">Who Uses</a>
- </p>
- <div class="collapse" id="collapse_who_uses">
- <li><a href="/docs/WhoUses.html">Who Uses</a></li>
- </div>
-</div>
-<script>
- (function () {
-
- var findLineItem = function (path) {
- return document.querySelector(`[href="${path}"]`);
- };
-
- var highlighLineItem = function (element) {
- element.classList.add('highlight');
- };
-
- var checkHasClass = function (element, className) {
- return element.className.split(' ').find(function (item) { return item === className || '' })
- }
-
- var findAllCollapseParents = function (element) {
- var collapseMenus = [];
- var elementPointer = element;
- while (elementPointer !== document.body) {
- if (checkHasClass(elementPointer, 'collapse')) {
- collapseMenus.push(elementPointer);
- }
- elementPointer = elementPointer.parentElement
- }
- return collapseMenus
- };
-
- var findMatchingCollapseMenu = function (collapseItem) {
- return document.querySelector(`[href="#${collapseItem.id}"]`);
- };
-
- var findAllParentMenus = function (element) {
- var parentMenus = []
- var elementPointer = element;
- while (elementPointer !== document.body) {
- if (elementPointer.className.split(' ').indexOf('menu') > -1) {
- parentMenus.push(elementPointer);
- }
- elementPointer = elementPointer.parentElement;
- }
- return parentMenus;
- }
-
- var openMenuItem = function (element) {
- element.click();
- };
-
- var openAllFromList = function (elementList) {
- elementList.forEach(function (element) {
- console.log(element)
- openMenuItem(findMatchingCollapseMenu(element))
- });
- };
-
- var highlightAndOpenMenu = function () {
- var currentLineItem = findLineItem(document.location.pathname);
- highlighLineItem(currentLineItem)
- openAllFromList(findAllCollapseParents(currentLineItem));
- };
-
- $(highlightAndOpenMenu);
-
- }());
-</script>
-
-
- </div>
- <div class="col-md-9 doc-content">
- <a class="btn btn-default btn-xs visible-xs-inline-block visible-sm-inline-block" href="/docs/toc.html">Table of Contents</a>
- <!--
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
--->
-
-<h1 id="apache-datasketches-incubating">Apache DataSketches (incubating)</h1>
-<h2 id="a-software-library-of-stochastic-streaming-algorithms">A software library of stochastic streaming algorithms</h2>
-
-<p>Apache DataSketches is an open source, high-performance library of stochastic streaming algorithms commonly called “sketches” in the data sciences. Sketches are small, stateful programs that process massive data as a stream and provide approximate answers, with mathematical guarantees, to computationally difficult queries orders-of-magnitude faster than traditional, exact methods.</p>
-
-<p>After 8 years of development and 5 years as in Open Source, we have begun the important migration from a stand-alone GitHub site to being a member of the Apache Software Foundation community. While we undergo this migration, we beg your patience.</p>
-
-<ul>
- <li>Ways to contact us:
- <ul>
- <li>We have two ASF <a href="http://the-ASF.slack.com">the-ASF.slack.com</a> slack channels:
- <ul>
- <li>datasketches – general user questions</li>
- <li>datasketches-dev – similar to our Apache <a href="mailto:dev@datasketches.apache.org">dev@datasketches.apache.org</a>, except more interactive, but not as easily searchable.</li>
- </ul>
- </li>
- <li>For discussions about problems with any of our sketch repository components (core-java, core-cpp, Hive, Pig, Vector, Android, etc.)
- <ul>
- <li><a href="https://groups.google.com/forum/#!forum/sketches-user">Google-groups forum</a></li>
- </ul>
- </li>
- <li>For issues about a specific component, please open issues on the appropriate GitHub repository (listed below).</li>
- <li>If you wish to contribute to our sketch development please contact us on our development email list
- <ul>
- <li><a href="mailto:dev@datasketches.apache.org">dev@datasketches.apache.org</a></li>
- </ul>
- </li>
- </ul>
- </li>
- <li>Finding our code releases:
- <ul>
- <li>Apache releases (versions 1.X.X):
- <ul>
- <li>Zip source: <a href="https://dist.apache.org/repos/dist/release/incubator/datasketches/">dist.apache.org/repos/dist/release/incubator/datasketches/</a></li>
- <li>Jar Artifacts (Java only): <a href="https://repository.apache.org/#nexus-search;quick~datasketches">repository.apache.org/#nexus-search;quick~datasketches</a></li>
- </ul>
- </li>
- <li>Pre-Apache releases (versions 0.X.X)
- <ul>
- <li>Jar Artifacts (Java only): <a href="https://search.maven.org/classic/#search%7Cga%7C1%7Ccom.yahoo.datasketches">search.maven.org/classic/#search%7Cga%7C1%7Ccom.yahoo.datasketches</a></li>
- </ul>
- </li>
- </ul>
- </li>
- <li>
- <p>As the repositories under GitHub.io/DataSketches migrate they will disapear from the the GitHub.com/DataSketches organization page. Please refer to this list be directed to the proper locations.</p>
- </li>
- <li>Web Sites
- <ul>
- <li><a href="http://DataSketches.GitHub.io">Original Website</a></li>
- <li><a href="https://datasketches.apache.org">New Website</a></li>
- <li><a href="https://github.com/DataSketches/DataSketches.github.io">Original Website Source</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-website">New Website Source</a></li>
- </ul>
- </li>
- <li>Java
- <ul>
- <li><a href="https://github.com/apache/incubator-datasketches-memory">memory now incubator-datasketches-memory</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-java">sketches-core now incubator-datasketches-java</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-hive">sketches-hive now incubator-datasketches-hive</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-pig">sketches-pig now incubator-datasketches-pig</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-vector">sketches-vector now incubator-datasketches-vector</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-characterization">characterization now incubator-datasketches-characterization</a></li>
- </ul>
- </li>
- <li>C++
- <ul>
- <li><a href="https://github.com/apache/incubator-datasketches-cpp">sketches-core-cpp now incubator-datasketches-cpp</a></li>
- <li><a href="https://github.com/apache/incubator-datasketches-postgresql">sketches-postgres now incubator-datasketches-postgresql</a></li>
- </ul>
- </li>
- <li>Java/C++/Python (the following may be moved to Apache later)
- <ul>
- <li><a href="https://github.com/DataSketches/sketches-misc">sketches-misc</a></li>
- <li><a href="https://github.com/DataSketches/sketches-android">sketches-android</a></li>
- <li><a href="https://github.com/DataSketches/experimental">experimental</a></li>
- <li><a href="https://github.com/DataSketches/characterization-cpp">characterization-cpp</a></li>
- <li><a href="https://github.com/DataSketches/experimental-cpp">experimental-cpp</a></li>
- </ul>
- </li>
- <li>Command Line (These may move to Apache later or replaced by Python)
- <ul>
- <li><a href="https://github.com/DataSketches/homebrew-sketches-cmd">homebrew-sketches-cmd</a></li>
- <li><a href="https://github.com/DataSketches/sketches-cmd">sketches-cmd</a></li>
- <li><a href="https://github.com/DataSketches/homebrew-sketches">homebrew-sketches</a></li>
- </ul>
- </li>
-</ul>
-
-<hr />
-
-<p>Disclaimer: Apache DataSketches is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indic [...]
-
- </div>
- </div>
- </div>
-
- <!-- Start page_footer.html include -->
-<footer class="ds-footer">
- <div class="container">
- <div class="text-center">
- <p>
- <a href="/overview.html">
- <span class="fa fa-info-circle"></span> Overview</a> · 
- <a href="/docs/downloads.html">
- <span class="fa fa-download"></span> Download</a> · 
- <a href="https://github.com/apache?utf8=%E2%9C%93&q=datasketches">
- <span class="fa fa-github"></span> GitHub</a> · 
- <a href="https://groups.google.com/forum/#!forum/sketches-user">
- <span class="fa fa-comment"></span> Comments</a> · 
- <a href="https://github.com/apache/incubator-datasketches-website/blob/asf-site/LICENSE">Licensing</a>
- </p>
- </div>
- </div>
-</footer>
-<!-- End page_footer.html include -->
-
-</body>
-
-</html>
-<!-- End _layouts/doc_page.html-->
diff --git a/pom.xml b/pom.xml
deleted file mode 100644
index 9c1939c..0000000
--- a/pom.xml
+++ /dev/null
@@ -1,630 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-<project xmlns="http://maven.apache.org/POM/4.0.0"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
- <modelVersion>4.0.0</modelVersion>
- <parent>
- <groupId>org.apache</groupId>
- <artifactId>apache</artifactId>
- <version>21</version>
- </parent>
- <groupId>org.apache.datasketches</groupId>
-
- <!-- UNIQUE FOR THIS JAVA COMPONENT -->
- <artifactId>datasketches-website</artifactId>
- <version>1.0.0-incubating-SNAPSHOT</version>
- <description>Website source for the DataSketches library.</description>
- <!-- END: UNIQUE FOR THIS JAVA COMPONENT -->
-
- <url>https://datasketches.apache.org/</url>
- <name>${project.artifactId}</name>
- <inceptionYear>2015</inceptionYear>
- <packaging>jar</packaging>
- <!-- jar is the default -->
- <!-- With the 3.6.1 version of Maven this causes a warning to obsolete this in
- favor of using the enforcer plugin (which we do). But removing this
- causes the command "mvn versions:display-plugin-updates" to issue an error
- and not work properly. So this will stay for the time being.
- -->
- <prerequisites>
- <maven>3.2.0</maven>
- </prerequisites>
- <licenses>
- <license>
- <name>Apache License, Version 2.0</name>
- <url>https://www.apache.org/licenses/LICENSE-2.0</url>
- <distribution>repo</distribution>
- </license>
- </licenses>
- <scm>
- <connection>scm:git:ssh://git@github.com/apache/incubator-${project.artifactId}.git</connection>
- <developerConnection>scm:git:ssh://git@github.com/apache/incubator-${project.artifactId}.git</developerConnection>
- <url>https://github.com/apache/incubator-${project.artifactId}</url>
- <tag>HEAD</tag>
- </scm>
- <issueManagement>
- <!-- <system>jira</system>
- <url>https://issues.apache.org/jira/browse/DATASKETCHES</url> -->
- <system>GitHub</system>
- <url>https://github.com/apache/incubator-${project.artifactId}/issues</url>
- </issueManagement>
- <mailingLists>
- <mailingList>
- <name>DataSketches Developers</name>
- <subscribe>dev-subscribe@datasketches.apache.org</subscribe>
- <unsubscribe>dev-unsubscribe@datasketches.apache.org</unsubscribe>
- <post>dev@datasketches.apache.org</post>
- <archive>https://mail-archives.apache.org/mod_mbox/datasketches-dev</archive>
- </mailingList>
- <mailingList>
- <name>sketches-user</name>
- <archive>https://groups.google.com/forum/#!forum/sketches-user</archive>
- <subscribe>mailto:sketches-user%2Bsubscribe@googlegroups.com</subscribe>
- <unsubscribe>mailto:sketches-user%2Bunsubscribe@googlegroups.com</unsubscribe>
- <post>mailto:sketches-user@googlegroups.com</post>
- </mailingList>
- </mailingLists>
- <developers>
- <developer>
- <name>The Apache DataSketches Team</name>
- <email>dev@datasketches.apache.org</email>
- <url>https://datasketches.apache.org</url>
- <organization>Apache Software Foundation</organization>
- <organizationUrl>http://www.apache.org</organizationUrl>
- </developer>
- </developers>
- <properties>
-
- <!-- UNIQUE FOR THIS JAVA COMPONENT -->
- <org-json.version>20180813</org-json.version>
- <!-- END:UNIQUE FOR THIS JAVA COMPONENT -->
-
- <!-- Test -->
- <testng.version>6.14.3</testng.version>
- <!-- System-wide properties -->
- <argLine>-Xmx4g -Duser.language=en -Duser.country=US -Dfile.encoding=UTF-8</argLine>
- <charset.encoding>UTF-8</charset.encoding>
- <project.build.sourceEncoding>${charset.encoding}</project.build.sourceEncoding>
- <project.build.resourceEncoding>${charset.encoding}</project.build.resourceEncoding>
- <project.reporting.outputEncoding>${charset.encoding}</project.reporting.outputEncoding>
- <java.version>1.8</java.version>
- <maven.compiler.source>${java.version}</maven.compiler.source>
- <maven.compiler.target>${java.version}</maven.compiler.target>
- <!-- org.codehaus.plexus used for strict profile testing-->
- <plexus-compiler-javac-errorprone.version>2.8.5</plexus-compiler-javac-errorprone.version>
- <!-- Maven Plugins -->
- <maven-assembly-plugin.version>3.1.1</maven-assembly-plugin.version>
- <maven-compiler-plugin.version>3.8.1</maven-compiler-plugin.version>
- <maven-deploy-plugin.version>3.0.0-M1</maven-deploy-plugin.version>
- <maven-enforcer-plugin.version>3.0.0-M2</maven-enforcer-plugin.version>
- <maven-gpg-plugin.version>1.6</maven-gpg-plugin.version>
- <maven-jar-plugin.version>3.1.2</maven-jar-plugin.version>
- <maven-javadoc-plugin.version>3.1.1</maven-javadoc-plugin.version>
- <maven-release-plugin.version>2.5.3</maven-release-plugin.version>
- <maven-remote-resources-plugin.version>[1.0,)</maven-remote-resources-plugin.version>
- <maven-source-plugin.version>3.1.0</maven-source-plugin.version>
- <maven-surefire-plugin.version>3.0.0-M3</maven-surefire-plugin.version>
- <!-- Apache Plugins -->
- <apache-rat-plugin.version>0.13</apache-rat-plugin.version>
- <!-- org.jacoco Maven Plugins -->
- <jacoco-maven-plugin.version>0.8.4</jacoco-maven-plugin.version>
- <!-- org.eluder Maven Plugins -->
- <coveralls-maven-plugin.version>4.3.0</coveralls-maven-plugin.version>
- <!-- other -->
- <lifecycle-mapping.version>1.0.0</lifecycle-mapping.version>
- <git-commit-id-plugin.version>3.0.0</git-commit-id-plugin.version>
- </properties>
- <repositories>
- <repository>
- <id>apache.snapshots</id>
- <name>Apache Snapshot Repository</name>
- <url>https://repository.apache.org/content/groups/snapshots/org/apache/datasketches/</url>
- <releases>
- <enabled>false</enabled>
- </releases>
- <snapshots>
- <enabled>true</enabled>
- </snapshots>
- </repository>
- <repository>
- <id>apache</id>
- <name>Apache Releases Repository</name>
- <url>https://repository.apache.org/content/repositories/releases/org/apache/datasketches/</url>
- <releases>
- <enabled>true</enabled>
- </releases>
- <snapshots>
- <enabled>false</enabled>
- </snapshots>
- </repository>
- </repositories>
- <dependencies>
-
- <!-- UNIQUE FOR THIS JAVA COMPONENT -->
- <!-- org.json -->
- <dependency>
- <groupId>org.json</groupId>
- <artifactId>json</artifactId>
- <version>${org-json.version}</version>
- </dependency>
- <!-- END: UNIQUE FOR THIS JAVA COMPONENT -->
-
- <!-- Test Scope -->
- <dependency>
- <groupId>org.testng</groupId>
- <artifactId>testng</artifactId>
- <version>${testng.version}</version>
- <scope>test</scope>
- </dependency>
- </dependencies>
- <build>
- <pluginManagement>
- <plugins>
- <plugin>
- <!-- We want to deploy the artifacts to a staging location for perusal -->
- <!-- Apache Parent pom: apache-release profile -->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-deploy-plugin</artifactId>
- <version>${maven-deploy-plugin.version}</version>
- <configuration>
- <updateReleaseInfo>true</updateReleaseInfo>
- <!-- see maven-install-plugin -->
- </configuration>
- </plugin>
- <plugin>
- <!-- Apache Parent pom, pluginManagement-->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-release-plugin</artifactId>
- <version>${maven-release-plugin.version}</version>
- </plugin>
- <plugin>
- <!-- Extends Apache Parent pom, pluginManagement-->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- <version>${maven-jar-plugin.version}</version>
- <executions>
- <execution>
- <id>default-jar</id>
- <goals>
- <goal>jar</goal>
- </goals>
- </execution>
- <execution>
- <id>default-test-jar</id>
- <goals>
- <goal>test-jar</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <!-- Extends Apache Parent pom, apache-release profile -->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-javadoc-plugin</artifactId>
- <version>${maven-javadoc-plugin.version}</version>
- <configuration>
- <docfilessubdirs>true</docfilessubdirs>
- </configuration>
- <executions>
- <execution>
- <id>attach-javadocs</id>
- <goals>
- <goal>jar</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-enforcer-plugin</artifactId>
- <version>${maven-enforcer-plugin.version}</version>
- <executions>
- <execution>
- <id>enforce-banned-dependencies</id>
- <goals>
- <goal>enforce</goal>
- </goals>
- <configuration>
- <rules>
- <requireJavaVersion>
- <version>1.8.0</version>
- </requireJavaVersion>
- <requireMavenVersion>
- <version>3.2,</version>
- </requireMavenVersion>
- <bannedDependencies>
- <excludes>
- <!--LGPL licenced library-->
- <exclude>com.google.code.findbugs:annotations</exclude>
- </excludes>
- </bannedDependencies>
- </rules>
- <fail>true</fail>
- </configuration>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <!-- Apache Parent pom, pluginManagement-->
- <groupId>org.apache.rat</groupId>
- <artifactId>apache-rat-plugin</artifactId>
- <version>${apache-rat-plugin.version}</version>
- <executions>
- <execution>
- <phase>verify</phase>
- <goals>
- <goal>check</goal>
- </goals>
- </execution>
- </executions>
- <configuration>
- <outputDirectory>${project.basedir}/rat</outputDirectory>
- <consoleOutput>true</consoleOutput>
- <useDefaultExcludes>true</useDefaultExcludes>
- <excludes>
- <!-- rat uses .gitignore for excludes by default -->
- <exclude>**/test/resources/**/*.txt</exclude>
- <!-- UNIQUE FOR THIS JAVA COMPONENT -->
- <exclude>**/*.txt</exclude>
- <exclude>**/*.html</exclude>
- <exclude>**/*.css</exclude>
- <exclude>**/*.js</exclude>
- <exclude>**/*.lock</exclude>
- <exclude>**/*.json</exclude>
- <exclude>**/package-list</exclude>
- <!-- END: UNIQUE FOR THIS JAVA COMPONENT -->
- <exclude>LICENSE</exclude>
- <exclude>NOTICE</exclude>
- <exclude>DISCLAIMER-WIP</exclude>
- </excludes>
- </configuration>
- </plugin>
- <plugin>
- <!-- Extends Apache Parent pom, apache-release profile -->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-source-plugin</artifactId>
- <version>${maven-source-plugin.version}</version>
- <executions>
- <execution>
- <id>attach-sources</id>
- <phase>package</phase>
- <goals>
- <goal>jar-no-fork</goal>
- </goals>
- </execution>
- <execution>
- <id>attach-test-sources</id>
- <phase>package</phase>
- <goals>
- <goal>test-jar-no-fork</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <!-- Apache Parent pom, pluginManagement-->
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
- <configuration>
- <trimStackTrace>false</trimStackTrace>
- <useManifestOnlyJar>false</useManifestOnlyJar>
- <redirectTestOutputToFile>true</redirectTestOutputToFile>
- </configuration>
- </plugin>
- <plugin>
- <!-- Generates code coverage report from website. -->
- <groupId>org.jacoco</groupId>
- <artifactId>jacoco-maven-plugin</artifactId>
- <version>${jacoco-maven-plugin.version}</version>
- <executions>
- <execution>
- <id>prepare-agent</id>
- <goals>
- <goal>prepare-agent</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <!-- Submit code coverage report to Coveralls.io. -->
- <groupId>org.eluder.coveralls</groupId>
- <artifactId>coveralls-maven-plugin</artifactId>
- <version>${coveralls-maven-plugin.version}</version>
- <configuration>
- <!-- Since we use Travis CI we do not have to put a Coveralls token here. -->
- </configuration>
- </plugin>
- </plugins>
- </pluginManagement>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-deploy-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-release-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-javadoc-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-enforcer-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.rat</groupId>
- <artifactId>apache-rat-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-source-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-surefire-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.jacoco</groupId>
- <artifactId>jacoco-maven-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.eluder.coveralls</groupId>
- <artifactId>coveralls-maven-plugin</artifactId>
- </plugin>
- </plugins>
- </build>
- <profiles>
- <!-- Ignore nuisance warning from Apache parent plugin:
- "maven-remote-resources-plugin (goal "process") is ignored by m2e".
- This also should fix the Maven warning that it can't find the lifecycle-mapping jar.
- This profile is only active when the property "m2e.version" is set,
- which is the case when building in Eclipse with m2e.
- The ignore below tells m2eclipse to skip the execution.
- -->
- <profile>
- <id>m2e</id>
- <activation>
- <property>
- <name>m2e.version</name>
- </property>
- </activation>
- <build>
- <pluginManagement>
- <plugins>
- <plugin>
- <groupId>org.eclipse.m2e</groupId>
- <artifactId>lifecycle-mapping</artifactId>
- <version>${lifecycle-mapping.version}</version>
- <configuration>
- <lifecycleMappingMetadata>
- <pluginExecutions>
- <pluginExecution>
- <pluginExecutionFilter>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-remote-resources-plugin</artifactId>
- <versionRange>${maven-remote-resources-plugin.version}</versionRange>
- <goals>
- <goal>process</goal>
- </goals>
- </pluginExecutionFilter>
- <action>
- <ignore/>
- </action>
- </pluginExecution>
- </pluginExecutions>
- </lifecycleMappingMetadata>
- </configuration>
- </plugin>
- </plugins>
- </pluginManagement>
- </build>
- </profile>
- <profile>
- <id>strict</id>
- <build>
- <pluginManagement>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
- <dependencies>
- <dependency>
- <groupId>org.codehaus.plexus</groupId>
- <artifactId>plexus-compiler-javac-errorprone</artifactId>
- <version>${plexus-compiler-javac-errorprone.version}</version>
- </dependency>
- </dependencies>
- <configuration>
- <source>${maven.compiler.source}</source>
- <target>${maven.compiler.target}</target>
- <compilerId>javac-with-errorprone</compilerId>
- <forceJavacCompilerUse>true</forceJavacCompilerUse>
- </configuration>
- </plugin>
- </plugins>
- </pluginManagement>
- </build>
- </profile>
- <!-- This profile is used to release signed jars to the Apache Nexus repository.
- This must be executed from a git repository set at the proper Release branch (e.g., 1.1.X-incubating)
- and at a Release Candidate tag (e.g., 1.1.0-incubating-RC1).
- The pom version in the release branch must be properly set to something like: "1.1.0-incubating".
- The pom version in the master would be set to something like: "1.2.0-incubating-SNAPSHOT".
- Test Command: mvn clean verify -Pnexus-jars -DskipTests=true
- Command: mvn clean deploy -Dnexus-jars
- Verify Command (from terminal): gpg -v --verify $ASC $FILE # dashdashverify
- -->
- <profile>
- <id>nexus-jars</id>
- <build>
- <pluginManagement>
- <plugins>
- <plugin>
- <groupId>pl.project13.maven</groupId>
- <artifactId>git-commit-id-plugin</artifactId>
- <version>${git-commit-id-plugin.version}</version>
- <executions>
- <execution>
- <goals>
- <goal>revision</goal>
- </goals>
- <phase>initialize</phase>
- </execution>
- </executions>
- <configuration>
- <dotGitDirectory>${project.basedir}/.git</dotGitDirectory>
- <dateFormatTimeZone>UTC</dateFormatTimeZone>
- <verbose>false</verbose>
- <skipPoms>false</skipPoms>
- <format>json</format>
- <generateGitPropertiesFile>true</generateGitPropertiesFile>
- <generateGitPropertiesFilename>${project.build.directory}/git.properties</generateGitPropertiesFilename>
- <failOnNoGitDirectory>true</failOnNoGitDirectory>
- <failOnUnableToExtractRepoInfo>true</failOnUnableToExtractRepoInfo>
- <commitIdGenerationMode>full</commitIdGenerationMode>
- <includeOnlyProperties>
- <includeProperty>git.branch</includeProperty>
- <includeProperty>git.commit.id.full</includeProperty>
- <includeProperty>git.commit.time</includeProperty>
- <includeProperty>git.commit.user.email</includeProperty>
- <includeProperty>git.tags</includeProperty>
- </includeOnlyProperties>
- <gitDescribe>
- <skip>false</skip>
- <always>true</always>
- <abbrev>7</abbrev>
- <dirty>-dirty</dirty>
- <tags>true</tags>
- <forceLongFormat>true</forceLongFormat>
- </gitDescribe>
- </configuration>
- </plugin>
- <!-- Extends Apache Parent pom, pluginManagement-->
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- <version>${maven-jar-plugin.version}</version>
- <executions>
- <execution>
- <id>default-jar</id>
- <goals>
- <goal>jar</goal>
- </goals>
- </execution>
- <execution>
- <id>default-test-jar</id>
- <goals>
- <goal>test-jar</goal>
- </goals>
- </execution>
- </executions>
- <configuration>
- <injectAllReactorProjects>true</injectAllReactorProjects>
- <archive>
- <manifest>
- <addDefaultEntries>false</addDefaultEntries>
- <addDefaultSpecificationEntries>false</addDefaultSpecificationEntries>
- <addDefaultImplementationEntries>false</addDefaultImplementationEntries>
- </manifest>
- <manifestEntries>
- <Build-Jdk>${java.version} (${java.vendor} ${java.vm.version})</Build-Jdk>
- <Build-OS>${os.name} ${os.arch} ${os.version}</Build-OS>
- <Implementation-Vendor>The Apache Software Foundation</Implementation-Vendor>
- <GroupId-ArtifactId>${project.groupId}:${project.artifactId}</GroupId-ArtifactId>
- <git-branch>${git.branch}</git-branch>
- <git-commit-id>${git.commit.id.full}</git-commit-id>
- <git-commit-time>${git.commit.time}</git-commit-time>
- <git-commit-user-email>${git.commit.user.email}</git-commit-user-email>
- <git-commit-tag>${git.tags}</git-commit-tag>
- </manifestEntries>
- </archive>
- </configuration>
- </plugin>
- <!-- We want to sign the artifacts, POM, and all attached artifacts -->
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-gpg-plugin</artifactId>
- <version>${maven-gpg-plugin.version}</version>
- <executions>
- <execution>
- <id>sign-artifacts</id>
- <phase>verify</phase>
- <goals>
- <goal>sign</goal>
- </goals>
- </execution>
- </executions>
- <configuration>
- <gpgArguments>
- <arg>--verbose</arg>
- <!-- prints the algorithm used -->
- <arg>--personal-digest-preferences=SHA512</arg>
- </gpgArguments>
- </configuration>
- </plugin>
- </plugins>
- </pluginManagement>
- <plugins>
- <plugin>
- <groupId>pl.project13.maven</groupId>
- <artifactId>git-commit-id-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-gpg-plugin</artifactId>
- </plugin>
- </plugins>
- </build>
- </profile>
- <!-- Disable source release assembly for 'apache-release' profile.
- This is performed from a script outside Maven
- -->
- <profile>
- <id>apache-release</id>
- <build>
- <plugins>
- <plugin>
- <artifactId>maven-assembly-plugin</artifactId>
- <version>${maven-assembly-plugin.version}</version>
- <executions>
- <execution>
- <id>source-release-assembly</id>
- <phase>none</phase>
- </execution>
- </executions>
- </plugin>
- </plugins>
- </build>
- </profile>
- </profiles>
-</project>
diff --git a/script/cibuild b/script/cibuild
deleted file mode 100644
index 540ffd6..0000000
--- a/script/cibuild
+++ /dev/null
@@ -1,6 +0,0 @@
-#!/usr/bin/env bash
-set -e # halt script on error
-
-bundle exec jekyll build
-#bundle exec htmlproofer ./_site --disable-external
-
diff --git a/src/main/java/org/apache/datasketches/ByteArrayBuilder.java b/src/main/java/org/apache/datasketches/ByteArrayBuilder.java
deleted file mode 100644
index 9935d61..0000000
--- a/src/main/java/org/apache/datasketches/ByteArrayBuilder.java
+++ /dev/null
@@ -1,210 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.datasketches;
-
-import java.util.Arrays;
-
-/**
- * A mutable sequence of bytes.
- *
- * <p>Implements a modifiable byte array. At any point in time it contains some
- * particular sequence of bytes but the length and content of the sequence can
- * be changed through certain method calls.
- *
- * @author Lee Rhodes
- */
-@SuppressWarnings("javadoc")
-public class ByteArrayBuilder {
- private byte[] arr_;
- private int count_ = 0;
- private int capacity_;
-
- public ByteArrayBuilder() {
- this(1024);
- }
-
- /**
- * Constructs an empty ByteArrayBuilder with an initial capacity specified by
- * the <code>capacity</code> argument.
- *
- * @param capacity the initial capacity.
- * @throws NegativeArraySizeException if the <code>capacity</code> argument is
- * less than <code>0</code>.
- */
- public ByteArrayBuilder(final int capacity) {
- arr_ = new byte[capacity];
- }
-
- /**
- * Returns the current capacity. The capacity is the amount of storage
- * available for newly inserted bytes, beyond which an allocation will occur.
- *
- * @return the current capacity
- */
- public int capacity() {
- return capacity_;
- }
-
- /**
- * Returns the length (byte count).
- *
- * @return the length of the sequence of bytes currently represented by this
- * object
- */
- public int length() {
- return count_;
- }
-
- /**
- * Ensures that the capacity is at least equal to the specified minimum. If
- * the current capacity is less than the argument, then a new internal array
- * is allocated with greater capacity. The new capacity is the larger of:
- * <ul>
- * <li>The <code>minimumCapacity</code> argument.
- * <li>Twice the old capacity, plus <code>2</code>.
- * </ul>
- * If the <code>minimumCapacity</code> argument is nonpositive, this method
- * takes no action and simply returns.
- *
- * @param minimumCapacity the minimum desired capacity.
- */
- public void ensureCapacity(final int minimumCapacity) {
- if (minimumCapacity > arr_.length) {
- expandCapacity(minimumCapacity);
- }
- }
-
- /**
- * This implements the expansion semantics of ensureCapacity with no size
- * check or synchronization.
- */
- void expandCapacity(final int minimumCapacity) {
- int newCapacity = (arr_.length + 1) * 2;
- if (newCapacity < 0) {
- newCapacity = Integer.MAX_VALUE;
- } else if (minimumCapacity > newCapacity) {
- newCapacity = minimumCapacity;
- }
- arr_ = Arrays.copyOf(arr_, newCapacity);
- }
-
- /**
- * Ensures that the space remaining (capacity - count) is at least as large as
- * the given space.
- *
- * @param space the given space in bytes
- */
- public void ensureSpace(final int space) {
- final int newCount = count_ + space;
- if (newCount > arr_.length) {
- expandCapacity(newCount);
- }
- }
-
- /**
- * Appends the given byte to the end of the current byte sequence.
- *
- * @param b byte
- * @return this ByteArrayBuilder
- */
- public ByteArrayBuilder append(final byte b) {
- ensureSpace(1);
- arr_[count_++] = b;
- return this;
- }
-
- /**
- * Appends the given byte array to the end of the current byte sequence.
- *
- * @param bArr byte array
- * @return this ByteArrayBuilder
- */
- public ByteArrayBuilder append(final byte[] bArr) {
- final int len = bArr.length;
- ensureSpace(len);
- System.arraycopy(bArr, 0, arr_, count_, len);
- count_ += len;
- return this;
- }
-
- /**
- * Sets the length of the byte sequence. The sequence is changed to a new byte
- * sequence whose length is specified by the argument. For every nonnegative
- * index <i>k</i> less than <code>newLength</code>, the byte at index <i>k</i>
- * in the new byte sequence is the same as the byte at index <i>k</i> in the
- * old sequence if <i>k</i> is less than the length of the old byte sequence;
- * otherwise, it is the null byte <code>0</code>.
- *
- * <p>In other words, if the <code>newLength</code> argument is less than the
- * current length, the length is changed to the specified length.
- *
- * <p>If the <code>newLength</code> argument is greater than or equal to the
- * current length, sufficient null byte (<code>0</code>) are appended so that
- * length becomes the <code>newLength</code> argument.
- *
- * <p>The <code>newLength</code> argument must be greater than or equal to
- * <code>0</code>.
- *
- * @param newLength the new length
- * @throws IndexOutOfBoundsException if the <code>newLength</code> argument is
- * negative.
- */
- public void setLength(final int newLength) {
- if (newLength < 0) {
- throw new ArrayIndexOutOfBoundsException(newLength);
- }
- if (newLength > arr_.length) {
- expandCapacity(newLength);
- }
- if (count_ < newLength) {
- for (; count_ < newLength; count_++) {
- arr_[count_] = 0;
- }
- } else {
- count_ = newLength;
- }
- }
-
- /**
- * Returns a byte array representing the data in this sequence. A new
- * <code>byte[]</code> object is allocated and initialized to contain the byte
- * sequence currently represented by this object. This <code>byte[]</code> is
- * then returned. Subsequent changes to this sequence do not affect the
- * contents of the <code>byte[]</code>.
- *
- * @return a byte array representation of this byte sequence.
- */
- public byte[] toByteArray() {
- return Arrays.copyOf(arr_, count_);
- }
-
- /**
- * Attempts to reduce storage used for the byte sequence. If the buffer is
- * larger than necessary to hold its current sequence of bytes, then it may be
- * resized to become more space efficient. Calling this method may, but is not
- * required to, affect the value returned by a subsequent call to the
- * {@link #capacity()} method.
- */
- public void trimToSize() {
- if (count_ < arr_.length) {
- arr_ = Arrays.copyOf(arr_, count_);
- }
- }
-}
diff --git a/src/main/java/org/apache/datasketches/Files.java b/src/main/java/org/apache/datasketches/Files.java
deleted file mode 100644
index cb94dcd..0000000
--- a/src/main/java/org/apache/datasketches/Files.java
+++ /dev/null
@@ -1,1040 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.datasketches;
-
-import static java.nio.channels.FileChannel.MapMode.READ_ONLY;
-
-import java.io.BufferedReader;
-import java.io.BufferedWriter;
-import java.io.File;
-import java.io.FileDescriptor;
-import java.io.FileInputStream;
-import java.io.FileNotFoundException;
-import java.io.FileOutputStream;
-import java.io.IOException;
-import java.io.InputStreamReader;
-import java.io.OutputStreamWriter;
-import java.io.PrintWriter;
-import java.io.RandomAccessFile;
-import java.io.Reader;
-import java.nio.BufferUnderflowException;
-import java.nio.ByteBuffer;
-import java.nio.MappedByteBuffer;
-import java.nio.channels.FileChannel;
-import java.nio.charset.Charset;
-import java.util.Arrays;
-
-/**
- * A collection of useful static file handlers that conveniently convert the
- * java.io checked exceptions into runtime exceptions.
- *
- * @author Lee Rhodes
- */
-@SuppressWarnings("javadoc")
-public final class Files {
- private static final String LS = System.getProperty("line.separator");
- private static final byte CR = 0xD;
- private static final byte LF = 0xA;
- public static final int DEFAULT_BUFSIZE = 8192;
-
- // Common IO & NIO file methods
-
- /**
- * If the fileName string is null or empty, this method throws a
- * RuntimeException.
- *
- * @param fileName the given fileName
- * @throws RuntimeException if fileName is null or empty.
- */
- public static void checkFileName(final String fileName) {
- if (fileName == null) {
- throw new RuntimeException(LS + "FileName is null.");
- }
- if (fileName.length() == 0) {
- throw new RuntimeException(LS + "FileName is empty.");
- }
- return;
- }
-
- /**
- * Gets an existing normal file as a File. If fileName is null, or empty, or
- * if the file is actually a directory, or doesn't exist this will throw a
- * Runtime Exception; otherwise it will return the fileName as a File object.
- *
- * @param fileName the given fileName
- * @return a File object
- * @throws RuntimeException if fileName cannot resolve to a existing normal
- * file.
- */
- public static File getExistingFile(final String fileName) {
- checkFileName(fileName);
- final File file = new File(fileName);
- if (file.isFile()) {
- return file;
- }
- if (file.isDirectory()) {
- throw new RuntimeException(LS + "FileName is a Directory: " + fileName);
- }
-
- throw new RuntimeException(LS + "FileName does not exist: " + fileName);
- }
-
- /**
- * Returns true if file is a normal file and not a directory.
- *
- * @param fileName the given fileName
- * @return true if file is a normal file and not a directory.
- * @throws RuntimeException if fileName is null or empty.
- */
- public static boolean isFileValid(final String fileName) {
- checkFileName(fileName);
- final File file = new File(fileName);
- return file.isFile();
- }
-
- /**
- * Gets the System.getProperty("user.dir"), which is the expected location of
- * the user root directory.
- *
- * @return location of user root directory
- */
- public static String getUserDir() {
- return System.getProperty("user.dir");
- }
-
- /**
- * Opens a RandomAccessFile given a File object and String mode: "r", "rw",
- * "rws" or "rwd". The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @param mode the given mode
- * @return RandomAccessFile
- * @throws RuntimeException if an IOException occurs.
- */
- public static RandomAccessFile openRandomAccessFile(final File file, final String mode) {
- final RandomAccessFile raf;
- try {
- raf = new RandomAccessFile(file, mode);
- } catch (final FileNotFoundException e) {
- throw new RuntimeException(LS + "Failed to open RandomAccessFile " + LS + e);
- }
- return raf;
- }
-
- /**
- * Gets the FileDescriptor from the given RandomAccessFile.
- *
- * @param raf RandomAccessFile
- * @return the FileDescriptor
- * @throws RuntimeException if an IOException occurs.
- */
- public static FileDescriptor getFD(final RandomAccessFile raf) {
- FileDescriptor fd = null;
- try {
- fd = raf.getFD();
- } catch (final IOException e) {
- throw new RuntimeException(LS + "RandomAccessFile.getFD() failure" + LS + e);
- }
- return fd;
- }
-
- /**
- * Sets the position of the given RandomAccessFile to the given position.
- *
- * @param raf RandomAccessFile
- * @param position the given position
- * @throws RuntimeException if an IOException occurs.
- */
- public static void seek(final RandomAccessFile raf, final long position) {
- try {
- raf.seek(position);
- } catch (final IOException e) {
- throw new RuntimeException(LS + "RandomAccessFile seek failure" + LS + e);
- }
- }
-
- /**
- * Reads buf.length bytes into the given buf.
- *
- * @param raf RandomAccessFile
- * @param buf the size of this buffer is the number of bytes requested.
- * @return the number of bytes actually read.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int readBytes(final RandomAccessFile raf, final byte[] buf) {
- final int len = buf.length;
- int read = 0;
- try {
- read = raf.read(buf);
- } catch (final IOException e) {
- throw new RuntimeException(LS + "RandomAccessFile read failure" + LS + e);
- }
- if (read < len) {
- Arrays.fill(buf, read, len, (byte) 0);
- }
- return read;
- }
-
- // ************************
- // NIO OPERATIONS
- // ************************
- // ByteBuffer methods
- /**
- * Gets a MappedByteBuffer from the given FileChannel, mode, position and
- * size.
- *
- * @param fChan the given FileChannel
- * @param mmode the given MapMode
- * @param position the given position
- * @param size the given size
- * @return a MappedByteBuffer
- * @throws RuntimeException if an IOException occurs.
- */
- public static MappedByteBuffer getMappedByteBuffer(final FileChannel fChan,
- final FileChannel.MapMode mmode, final long position, final long size) {
- final MappedByteBuffer mbBuf;
- try {
- mbBuf = fChan.map(mmode, position, size);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return mbBuf;
- }
-
- /**
- * Gets a MappedByteBuffer from the given FileChannel and mode. Assumes a
- * start position of zero and size of the length of the file (via the
- * FileChannel. Equivalent to:<br>
- * getMappedByteBuffer(FileChannel, FileChanel.MapMode, 0L, size(fChan));
- *
- * @param fChan the given FileChannel
- * @param mmode the given MapMode
- * @return a MappedByteBuffer
- * @throws RuntimeException if an IOException occurs.
- */
- public static MappedByteBuffer getMappedByteBuffer(final FileChannel fChan,
- final FileChannel.MapMode mmode) {
- return getMappedByteBuffer(fChan, mmode, 0L, size(fChan));
- }
-
- /**
- * Reads bytes from the given (Mapped)ByteBuffer until either a CR, LF or CRLF
- * is detected in the byte stream and then converts the captured bytes,
- * excluding the CR and LF characters into a string. This method will work
- * with US-ASCII, ISO-8859 families, Windows 1252, and UTF-8. encodings. In
- * general any character encoding that is isomorphic with respect to the
- * exclusive use of the CR (0xD) and the LF (0xA) codes. Equivalent to:<br>
- * readLine(ByteBuffer, ByteArrayBuilder, Charset.defaultCharset());
- *
- * @param mbBuf Given ByteBuffer or MappedByteBuffer
- * @param bab an optional ByteArrayBuilder for internal reuse, which will
- * improve multi-line reading performance. The result of the read as an array
- * of bytes is available from the bab.
- * @return the line as a string
- *
- */
- public static String readLine(final ByteBuffer mbBuf, final ByteArrayBuilder bab) {
- return readLine(mbBuf, bab, Charset.defaultCharset());
- }
-
- /**
- * Reads bytes from the given (Mapped)ByteBuffer until either a CR, LF or CRLF
- * is detected in the byte stream and then converts the captured bytes,
- * excluding the CR and LF characters into a string. This method will work
- * with US-ASCII, ISO-8859 families, Windows 1252, and UTF-8. encodings. In
- * general any character encoding that is isomorphic with respect to the
- * exclusive use of the CR (0xD) and the LF (0xA) codes.
- *
- * @param mbBuf Given ByteBuffer or MappedByteBuffer
- * @param bab an optional ByteArrayBuilder for internal reuse, which will
- * improve multiline reading performance. The result of the read as an array
- * of bytes is available from the bab.
- * @param charset The Charset to use when converting arrays of bytes from the
- * source to a Unicode String (UTF-16).
- * @return The characters of a line, or NULL if End-of-File, or "" if line was
- * empty.
- */
- public static String readLine(final ByteBuffer mbBuf, final ByteArrayBuilder bab,
- final Charset charset) {
- if (!mbBuf.hasRemaining()) {
- return null;
- }
- final ByteArrayBuilder bab1;
- if (bab == null) {
- bab1 = new ByteArrayBuilder();
- } else {
- bab1 = bab;
- bab1.setLength(0);
- }
- while (mbBuf.hasRemaining()) {
- final byte b = mbBuf.get();
- if (b == LF) {
- break; // EOL
- }
- if (b == CR) {
- if (mbBuf.hasRemaining()) {
- // peek next byte without moving position
- if (mbBuf.get(mbBuf.position()) == LF) {
- mbBuf.get(); // consume it
- }
- }
- break; // EOL
- }
- bab1.append(b); // transfer the byte
- }
- if (bab1.length() == 0) {
- if (!mbBuf.hasRemaining()) {
- return null;
- }
- return "";
- }
- final byte[] out = bab1.toByteArray();
- final String s = new String(out, charset);
- return s;
- }
-
- /**
- * Reads a ByteBuffer (or subclass) with a request for numBytes. The result is
- * stuffed into the provided byte[] array (required), which must be larger or
- * equal to numBytes.
- *
- * @param bb The ByteBuffer to read from
- * @param numBytes The requested number of bytes to read.
- * @param out The target array for the bytes.
- * @return the actual number of bytes read.
- * @throws BufferUnderflowException if numBytes is greater than bytes
- * available in the buffer.
- */
- public static int readByteBuffer(final ByteBuffer bb, final int numBytes, final byte[] out) {
- final int rem = bb.remaining();
- if (rem == 0) {
- return 0;
- }
- final int nBytes = (rem < numBytes) ? rem : numBytes;
- bb.get(out);
- return nBytes;
- }
-
- // FileChannel methods
- /**
- * Sets the FileChannel position.
- *
- * @param fc FileChannel
- * @param position the position
- * @throws RuntimeException if an IOException occurs.
- */
- public static void position(final FileChannel fc, final long position) {
- try {
- fc.position(position);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- }
-
- /**
- * Gets the current size of the FileChannel.
- *
- * @param fc FileChannel
- * @return the size in bytes.
- * @throws RuntimeException if an IOException occurs.
- */
- public static long size(final FileChannel fc) {
- final long sz;
- try {
- sz = fc.size();
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return sz;
- }
-
- /**
- * Appends the given string to the end of the file specified via the given
- * FileChannel. Equivalent to:<br>
- * append(String, FileChannel, Charset.defaultCharset());
- *
- * @param out the string to append
- * @param fc the given FileChannel
- * @return the number of bytes actually appended.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int append(final String out, final FileChannel fc) {
- return append(out, fc, Charset.defaultCharset());
- }
-
- /**
- * Appends the given string to the end of the file specified via the given
- * FileChannel.
- *
- * @param out the string to append
- * @param fc the given FileChannel
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return the number of bytes actually appended.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int append(final String out, final FileChannel fc, final Charset charset) {
- final int bytes;
- final ByteBuffer bb = ByteBuffer.wrap(out.getBytes(charset));
- try {
- bytes = fc.write(bb, fc.size());
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return bytes;
- }
-
- /**
- * Appends the given byteArr to the end of the file specified via the given
- * FileChannel.
- *
- * @param byteArr the byte[] to append
- * @param fc the given FileChannel
- * @return the number of bytes actually appended.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int append(final byte[] byteArr, final FileChannel fc) {
- final int bytes;
- final ByteBuffer bb = ByteBuffer.wrap(byteArr);
- try {
- bytes = fc.write(bb, fc.size());
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return bytes;
- }
-
- /**
- * Writes the given string out to the file specified via the given FileChannel
- * starting at the given file position. Equivalent to:<br>
- * write(String, FileChannel, long, Charset.defaultCharset());
- *
- * @param out the given string
- * @param fc FileChannel
- * @param position the position
- * @return the total number of bytes written.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int write(final String out, final FileChannel fc, final long position) {
- return write(out, fc, position, Charset.defaultCharset());
- }
-
- /**
- * Writes the given string out to the file specified via the given FileChannel
- * starting at the given file position.
- *
- * @param out the given string
- * @param fc FileChannel
- * @param position the given position
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return the total number of bytes written.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int write(final String out, final FileChannel fc, final long position,
- final Charset charset) {
- final int bytes;
- final ByteBuffer bb = ByteBuffer.wrap(out.getBytes(charset));
- try {
- bytes = fc.write(bb, position);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return bytes;
- }
-
- /**
- * Writes the given byteArr to the file specified via the given FileChannel at
- * the given position.
- *
- * @param byteArr the byte[] to append
- * @param fc the given FileChannel
- * @param position the given position
- * @return the number of bytes actually appended.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int write(final byte[] byteArr, final FileChannel fc, final long position) {
- final int bytes;
- final ByteBuffer bb = ByteBuffer.wrap(byteArr);
- try {
- bytes = fc.write(bb, position);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- return bytes;
- }
-
- // Complete NIO BASED FILE WRITE OPERATIONS
- /**
- * Writes the given String text to the fileName using NIO FileChannel.
- *
- * @param text Source string to place in a file. Equivalent to: <br>
- * stringToFileNIO(String, String, Charset.defaultCharset());
- * @param fileName name of target file
- * @return the total number of bytes written.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int stringToFileNIO(final String text, final String fileName) {
- return stringToFileNIO(text, fileName, Charset.defaultCharset());
- }
-
- /**
- * Writes the given String text to the fileName using NIO FileChannel
- *
- * @param text Source string to place in a file.
- * @param fileName name of target file
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return the total number of bytes written.
- * @throws RuntimeException if an IOException occurs.
- */
- public static int stringToFileNIO(final String text, final String fileName,
- final Charset charset) {
- checkFileName(fileName);
- final File file = new File(fileName);
- int bytes = 0;
- if (!file.isFile()) {
- try {
- file.createNewFile();
- } catch (final IOException e) {
- throw new RuntimeException("Cannot create new file: " + fileName + LS + e);
- }
- try (FileChannel fc = openRandomAccessFile(file, "rw").getChannel()) {
- bytes = write(text, fc, 0L, charset);
- } catch (final IOException e) {
- throw new RuntimeException("Cannot create File Channel: " + fileName + LS + e);
- }
- }
-
- return bytes;
- }
-
- /**
- * Appends a String to a file using NIO. If fileName does not exist, this
- * creates a new empty file of that name. This closes the file after
- * appending.
- *
- * @param text is the source String. Equivalent to: <br>
- * appendStringToFileNIO(String, String, Charset.defautCharset());
- * @param fileName the given fileName
- * @return the total number of bytes written
- * @throws RuntimeException if IOException or SecurityException occurs, or if
- * fileName is null or empty.
- */
- public static int appendStringToFileNIO(final String text, final String fileName) {
- return appendStringToFileNIO(text, fileName, Charset.defaultCharset());
- }
-
- /**
- * Appends a String to a file using NIO and a Charset. If fileName does not
- * exist, this creates a new empty file of that name. This closes the file
- * after appending.
- *
- * @param text is the source String.
- * @param fileName the given fileName
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return the total number of bytes written
- * @throws RuntimeException if IOException or SecurityException occurs, or if
- * fileName is null or empty.
- */
- public static int appendStringToFileNIO(final String text, final String fileName,
- final Charset charset) {
- checkFileName(fileName);
- final File file = new File(fileName);
- if (!file.isFile()) { // does not exist
- try {
- file.createNewFile();
- } catch (final Exception e) {
- throw new RuntimeException("Cannot create file: " + fileName + LS + e);
- }
- }
- int bytes = 0;
- try (FileChannel fc = openRandomAccessFile(file, "rw").getChannel()) {
- bytes = append(text, fc, charset);
- } catch (final IOException e) {
- throw new RuntimeException("Cannot create File Channel: " + fileName + LS + e);
- }
- return bytes;
- }
-
- // Complete NIO BASED FILE READ OPERATIONS
- /**
- * Reads a file into a char array using NIO FileChannel, then closes the file. Useful when
- * special handling of Line-Separation characters is required. Equivalent to:
- * <br>
- * fileToCharArrayNIO(String, Charset.defaultCharset());
- *
- * @param fileName the given fileName
- * @return a char[]
- * @throws RuntimeException if IOException occurs.
- * @throws IllegalArgumentException if File size is greater than
- * Integer.MAX_VALUE.
- */
- public static char[] fileToCharArrayNIO(final String fileName) {
- return fileToCharArrayNIO(fileName, Charset.defaultCharset());
- }
-
- /**
- * Reads a file into a char array using NIO FileChannel, then closes the file. Useful when
- * special handling of Line-Separation characters is required.
- *
- * @param fileName the given fileName
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return a char[]
- * @throws RuntimeException if IOException occurs.
- * @throws IllegalArgumentException if File size is greater than
- * Integer.MAX_VALUE.
- */
- public static char[] fileToCharArrayNIO(final String fileName, final Charset charset) {
- final File file = getExistingFile(fileName);
- char[] chArr = null;
- try (RandomAccessFile raf = openRandomAccessFile(file, "r");
- FileChannel fc = raf.getChannel()) {
- final MappedByteBuffer mbBuf = getMappedByteBuffer(fc, READ_ONLY);
- final long len = size(fc);
- if (len > Integer.MAX_VALUE) {
- fc.close();
- throw new IllegalArgumentException("File size cannot exceed Integer.MAX_VALUE.");
- }
- final byte[] in = new byte[(int) len];
- mbBuf.get(in); // fill the buffer
- final String out = new String(in, charset);
- chArr = out.toCharArray();
- } catch (final IOException e) {
- throw new RuntimeException("Could not create or close File Channel.");
- }
- return chArr;
- }
-
- /**
- * Reads a file into a String using NIO FileChannel. Each line of the file is delimited by
- * the current operating systems's "line.separator" characters. Closes the
- * file. This method is equivalent to:<br>
- * fileToStringNIO(String fileName, Charset.defaultCharset())
- *
- * @param fileName given fileName
- * @return a String
- * @throws RuntimeException if IOException occurs.
- */
- public static String fileToStringNIO(final String fileName) {
- return fileToStringNIO(fileName, Charset.defaultCharset());
- }
-
- /**
- * Reads a file into a String using NIO FileChannel. Each line of the file is delimited by
- * the current operating systems's "line.separator" characters. Closes the
- * file.
- *
- * @param fileName given fileName
- * @param charset The Charset to use when converting arrays of bytes from the
- * source to a Unicode String (UTF-16).
- * @return a String
- * @throws RuntimeException if IOException occurs.
- */
- public static String fileToStringNIO(final String fileName, final Charset charset) {
- final File file = getExistingFile(fileName);
- final StringBuilder sb = new StringBuilder(1024);
- try (RandomAccessFile raf = openRandomAccessFile(file, "r");
- FileChannel fChan = raf.getChannel();) {
- final MappedByteBuffer mbBuf = getMappedByteBuffer(fChan, READ_ONLY);
- final ByteArrayBuilder bab = new ByteArrayBuilder();
-
- String s;
- while ((s = readLine(mbBuf, bab, charset)) != null) {
- sb.append(s);
- sb.append(LS);
- }
- } catch (final IOException e) {
- throw new RuntimeException("Cannot create File Channel.");
- }
- return sb.toString();
- }
-
- // *******************************
- // STANDARD IO READER OPERATIONS
- // *******************************
- /**
- * Opens a BufferedReader wrapped around a File. Equivalent to the call<br>
- * openBufferedReader(file, DEFAULT_BUFSIZE) Rethrows any IOException as a
- * RuntimeException. The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @return BufferedReader object
- * @throws RuntimeException if File Not Found.
- */
- public static BufferedReader openBufferedReader(final File file) {
- return openBufferedReader(file, DEFAULT_BUFSIZE, Charset.defaultCharset());
- }
-
- /**
- * Opens a BufferedReader wrapped around a FileReader with a specified file
- * and buffer size. If bufSize is less than the default (8192) the default
- * will be used. The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @param bufSize the given buffer size
- * @return a BufferedReader object
- * @throws RuntimeException if File Not Found.
- */
- public static BufferedReader openBufferedReader(final File file, final int bufSize) {
- return openBufferedReader(file, bufSize, Charset.defaultCharset());
- }
-
- /**
- * Opens a BufferedReader, which specifies a bufSize, wrapped around an
- * InputStreamReader, which specifies a Charset. The InputStreamReader wraps a
- * FileInputStream. If bufSize is less than the default (8192) the default
- * will be used. If the charset is null, Charset.defaultCharset() will be
- * used. The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @param bufSize the given buffer size
- * @param charset the given Charset
- * @return a BufferedReader object
- * @throws RuntimeException if FileNotFoundException occurs.
- */
- public static BufferedReader openBufferedReader(final File file, final int bufSize,
- final Charset charset) {
- final int bufSz = (bufSize < DEFAULT_BUFSIZE) ? DEFAULT_BUFSIZE : bufSize;
- final Charset cSet = (charset == null) ? Charset.defaultCharset() : charset;
- BufferedReader in = null; // default bufsize is 8192.
- try {
- final FileInputStream fis = new FileInputStream(file);
- final InputStreamReader isr = new InputStreamReader(fis, cSet);
- in = new BufferedReader(isr, bufSz);
- } catch (final FileNotFoundException e) { // from FileInputStream
- // never opened, so don't close it.
- throw new RuntimeException(LS + "File Not Found: " + file.getPath() + LS + e);
- }
- return in;
- }
-
- /**
- * Configures a file for string writing. If a file by the same name exists, it will be
- * deleted. If fileName is not fully qualified, it will be relative to the root of this
- * package. The returned object must be closed by the calling program.
- * @param fileName the name of the file to configure
- * @return a PrintWriter
- */
- public static final PrintWriter openPrintWriter(final String fileName) {
- File file = null;
- PrintWriter pw = null;
- if ((fileName != null) && !fileName.isEmpty()) {
- file = new File(fileName);
- if (file.isFile()) {
- file.delete(); //remove old file if it exists
- } else {
- try {
- file.createNewFile();
- } catch (final Exception e) {
- throw new RuntimeException("Cannot create file: " + fileName + LS + e);
- }
- }
- final BufferedWriter bw;
- try {
- final FileOutputStream fos = new FileOutputStream(file, true);
- final OutputStreamWriter osw = new OutputStreamWriter(fos, Charset.defaultCharset());
- bw = new BufferedWriter(osw, 8192);
- } catch (final IOException e) {
- // never opened, so don't close it.
- throw new RuntimeException("Could not create: " + file.getPath() + LS + e);
- }
- pw = new PrintWriter(bw);
- }
- return pw;
- }
-
- /**
- * Tests a Reader object if it is ready.
- *
- * @param in the given Reader
- * @return boolean true if ready.
- * @throws RuntimeException if IOException occurs.
- */
- public static boolean ready(final Reader in) {
- boolean out = false;
- try {
- out = in.ready();
- } catch (final IOException e) {
- throw new RuntimeException(LS + "Reader.ready() unsuccessful: " + LS + e);
- }
- return out;
- }
-
- /**
- * Skips bytes in the given Reader object.
- *
- * @param in the given Reader
- * @param skipLen in bytes.
- * @throws RuntimeException if IOException occurs.
- */
- public static void skip(final Reader in, final long skipLen) {
- try {
- in.skip(skipLen);
- } catch (final IOException e) {
- try {
- in.close();
- } catch (final IOException f) {
- throw new RuntimeException(LS + "Close Unsuccessful" + LS + f);
- }
- throw new RuntimeException(LS + "Reader.skip(len) unsuccessful: " + LS + e);
- }
- }
-
- /**
- * Reads characters from the given Reader into the given character array.
- *
- * @param in the given Reader
- * @param length number of characters to read
- * @param cArr Array must be equal to or larger than length
- * @return number of characters actually read from Reader
- * @throws RuntimeException if IOException occurs.
- */
- public static int readCharacters(final Reader in, final int length, final char[] cArr) {
- int readLen = 0;
- try {
- readLen = in.read(cArr, 0, length);
- } catch (final IOException e) {
- try {
- in.close();
- } catch (final IOException f) {
- throw new RuntimeException(LS + "Close Unsuccessful" + LS + f);
- }
- throw new RuntimeException(LS + "Reader.read(char[],0,len) unsuccessful: " + LS + e);
- }
- return readLen;
- }
-
- /**
- * Reads a file into a char array, then closes the file. Useful when special
- * handling of Line-Separation characters is required. Equivalent to: <br>
- * fileToCharArray(String, int, Charset.defaultCharset());
- *
- * @param fileName the given fileName
- * @return a char[]
- * @throws RuntimeException if IOException occurs.
- * @throws IllegalArgumentException if File size is greater than
- * Integer.MAX_VALUE.
- */
- public static char[] fileToCharArray(final String fileName) {
- return fileToCharArray(fileName, DEFAULT_BUFSIZE, Charset.defaultCharset());
- }
-
- /**
- * Reads a file into a char array, then closes the file. Useful when special
- * handling of Line-Separation characters is required.
- *
- * @param fileName the given fileName
- * @param bufSize if less than 8192 it defaults to 8192
- * @param charset The Charset to use when converting arrays of bytes from the
- * source to a Unicode String (UTF-16).
- * @return a char[]
- * @throws RuntimeException if IOException occurs.
- * @throws IllegalArgumentException if File size is greater than
- * Integer.MAX_VALUE.
- */
- public static char[] fileToCharArray(final String fileName, final int bufSize,
- final Charset charset) {
- final File file = getExistingFile(fileName);
- char[] cArr = null;
- final long fileLen = (long) (file.length() * 1.1); // 10% headroom
- if (fileLen > Integer.MAX_VALUE) {
- throw new IllegalArgumentException(
- LS + "File Size is too large: " + fileLen + " >" + " Max: " + Integer.MAX_VALUE);
- }
- final int len = (int) fileLen;
- try (BufferedReader in = openBufferedReader(file, bufSize, charset)) {
- cArr = new char[len];
- in.read(cArr, 0, len);
- } catch (final IOException e) { // thrown by read()
- throw new RuntimeException(LS + "BufferedReader.read(char[],0,len) unsuccessful: " + LS + e);
- }
- return cArr;
- }
-
- /**
- * Reads a file into a String. Each line of the file is delimited by the
- * current operating systems's "line.separator" characters. Closes the file.
- * Equivalent to: <br>
- * fileToString(String, 8192, Charset.defaultCharset());
- *
- * @param fileName the given fileName
- * @return a String
- * @throws RuntimeException if IOException occurs.
- */
- public static String fileToString(final String fileName) {
- return fileToString(fileName, DEFAULT_BUFSIZE, Charset.defaultCharset());
- }
-
- /**
- * Reads a file into a String. Each line of the file is delimited by the
- * current operating systems's "line.separator" characters. Closes the file.
- *
- * @param fileName the given fileName
- * @param bufSize if less than 8192 it defaults to 8192
- * @param charset The Charset to use when converting arrays of bytes from the
- * source to a Unicode String (UTF-16).
- * @return String
- * @throws RuntimeException if IOException occurs.
- */
- public static String fileToString(final String fileName, final int bufSize,
- final Charset charset) {
- final StringBuilder sb = new StringBuilder();
- final File file = getExistingFile(fileName);
- try (BufferedReader in = openBufferedReader(file, bufSize, charset)) {
- String s;
- while ((s = in.readLine()) != null) {
- sb.append(s);
- sb.append(LS);
- }
- } catch (final IOException e) { // thrown by readLine()
- throw new RuntimeException(LS + "BufferedReader.readLine() unsuccessful: " + LS + e);
- }
- return sb.toString();
- }
-
- // STANDARD IO WRITE OPERATIONS
- /**
- * Opens a BufferedWriter wrapped around a FileWriter with a specified file
- * and buffer size. If bufSize is less than the default (8192) the default
- * will be used. The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @param bufSize the given buffer size
- * @param append to existing file if true.
- * @return BufferedWriter object
- * @throws RuntimeException if IOException occurs.
- */
- public static BufferedWriter openBufferedWriter(final File file, final int bufSize,
- final boolean append) {
- return openBufferedWriter(file, bufSize, append, Charset.defaultCharset());
- }
-
- /**
- * Opens a BufferedWriter wrapped around a FileWriter with a specified file
- * and buffer size. If bufSize is less than the default (8192) the default
- * will be used. The returned object must be closed by the calling program.
- *
- * @param file the given file
- * @param bufSize if less than 8192 it defaults to 8192.
- * @param append to existing file if true.
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @return BufferedWriter object
- * @throws RuntimeException if IOException occurs.
- */
- public static BufferedWriter openBufferedWriter(final File file, final int bufSize,
- final boolean append, final Charset charset) {
- final int bufSz = (bufSize < DEFAULT_BUFSIZE) ? DEFAULT_BUFSIZE : bufSize;
- BufferedWriter out = null; // default bufsize is 8192.
- try {
- final FileOutputStream fos = new FileOutputStream(file, append);
- final OutputStreamWriter osw = new OutputStreamWriter(fos, charset);
- out = new BufferedWriter(osw, bufSz);
- } catch (final IOException e) {
- // never opened, so don't close it.
- throw new RuntimeException(LS + "Could not create: " + file.getPath() + LS + e);
- }
- return out;
- }
-
- /**
- * Writes a String to a file using a BufferedWriter. Closes the file.
- *
- * @param text is the source String.
- * @param fileName the given fileName
- * @throws RuntimeException if IOException occurs or if fileName is null or
- * empty.
- */
- public static void stringToFile(final String text, final String fileName) {
- stringToFile(text, fileName, DEFAULT_BUFSIZE, Charset.defaultCharset());
- }
-
- /**
- * Writes a String to a file using a BufferedWriter. Closes the file.
- *
- * @param text is the source String.
- * @param fileName the given fileName
- * @param bufSize if less than 8192 it defaults to 8192.
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @throws RuntimeException if IOException occurs or if fileName is null or
- * empty.
- */
- public static void stringToFile(final String text, final String fileName, final int bufSize,
- final Charset charset) {
- checkFileName(fileName);
- final File file = new File(fileName);
- try (BufferedWriter bw = openBufferedWriter(file, bufSize, false, charset);
- PrintWriter out = new PrintWriter(bw);) {
- out.print(text);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- }
-
- /**
- * Appends a String to a file using a BufferedWriter. If fileName does not
- * exist, this creates a new empty file of that name. This closes the file
- * after appending.
- *
- * @param text is the source String.
- * @param fileName the given fileName
- * @throws RuntimeException if IOException or SecurityException occurs, or if
- * fileName is null or empty.
- */
- public static void appendStringToFile(final String text, final String fileName) {
- appendStringToFile(text, fileName, DEFAULT_BUFSIZE, Charset.defaultCharset());
- }
-
- /**
- * Appends a String to a file using a BufferedWriter, bufSize and Charset. If
- * fileName does not exist, this creates a new empty file of that name. This
- * closes the file after appending.
- *
- * @param text is the source String.
- * @param fileName the given fileName
- * @param bufSize if less than 8192 it defaults to 8192.
- * @param charset The Charset to use when converting the source string
- * (UTF-16) to a sequence of encoded bytes of the Charset.
- * @throws RuntimeException if IOException or SecurityException occurs, or if
- * fileName is null or empty.
- */
- public static void appendStringToFile(final String text, final String fileName,
- final int bufSize, final Charset charset) {
- checkFileName(fileName);
- final File file = new File(fileName);
- if (!file.isFile()) { // does not exist
- try {
- file.createNewFile();
- } catch (final Exception e) {
- throw new RuntimeException("Cannot create file: " + fileName + LS + e);
- }
- }
- try (BufferedWriter bw = openBufferedWriter(file, bufSize, true, charset);
- PrintWriter out = new PrintWriter(bw);) {
- out.print(text);
- } catch (final IOException e) {
- throw new RuntimeException(e);
- }
- }
-}
diff --git a/src/main/java/org/apache/datasketches/docgen/ErrorTables.java b/src/main/java/org/apache/datasketches/docgen/ErrorTables.java
deleted file mode 100644
index 33ea61f..0000000
--- a/src/main/java/org/apache/datasketches/docgen/ErrorTables.java
+++ /dev/null
@@ -1,86 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.datasketches.docgen;
-
-import static org.apache.datasketches.docgen.MarkDownTable.emitAlignLine;
-import static org.apache.datasketches.docgen.MarkDownTable.emitBoldRow;
-import static org.apache.datasketches.docgen.MarkDownTable.emitRow;
-
-import org.testng.annotations.Test;
-
-/**
- * Builds the error tables for Theta QuickSelect and Alpha sketches.
- * @author Lee Rhodes
- */
-
-public class ErrorTables {
- static final String LS = System.getProperty("line.separator");
-
- /**
- * Builds the QuickSelect error table
- */
- @Test
- public void buildQuickSelectTable() {
- buildTable(1.0, 4);
- }
-
- /**
- * Builds the Alpha error table
- */
- @Test
- public void buildAlphaTable() {
- buildTable(.708, 9);
- }
-
- private static void buildTable(double factor, int lgKlow) {
- StringBuilder sb = new StringBuilder();
- int[] colWidths = {10, 12, 7, 7, 7 };
- double f = factor; //.708 for Alpha, 1.0 for QuickSelect
- //the is required on the first line when Col#1 is blank for GitHub to render properly.
- //MackDown doesn't care.
- sb.append(emitRow(colWidths, " ","#Std Dev:", "1","2","3"));
- sb.append(emitAlignLine(colWidths, "CCCCC"));
- sb.append(emitBoldRow(colWidths,"","Conf:", "68.27%","95.45%","99.73%"));
- sb.append(emitBoldRow(colWidths,"LgK","K", "1 RSE","2 RSE","3 RSE"));
- for (int lgK = lgKlow; lgK <= 26; lgK++) { //for Alpha start with 9, QuickSelect start with 4
- String lgKStr = Integer.toString(lgK);
- sb.append(emitRow(colWidths,lgKStr, k(lgK), rse(lgK, 1, f), rse(lgK,2,f), rse(lgK,3,f)));
- }
- println(sb.toString());
- }
-
- static String k(int lgK) {
- int lgk = 1 << lgK;
- return String.format("%,d", lgk);
- }
-
- static String rse(int lgK, int stdDev, double factor) {
- double k = 1 << lgK;
- double rse = (stdDev * factor * 100.0) / Math.sqrt(k - 1.0);
- return String.format("%6.3f%%", rse);
- }
-
- static void println(Object o) { System.out.println(o.toString()); }
-
- static void printf(String fmt, Object ... args) {
- System.out.printf(fmt, args);
- }
-
-}
diff --git a/src/main/java/org/apache/datasketches/docgen/MarkDownTable.java b/src/main/java/org/apache/datasketches/docgen/MarkDownTable.java
deleted file mode 100644
index c570fad..0000000
--- a/src/main/java/org/apache/datasketches/docgen/MarkDownTable.java
+++ /dev/null
@@ -1,133 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.datasketches.docgen;
-
-import static java.lang.Math.max;
-
-/**
- * General purpose MarkDown table generator for GitHub Flavored Markdown.
- * This produces readable source and particularly useful when the data of the
- * table is machine generated.
- * @author Lee Rhodes
- */
-public class MarkDownTable {
- private static final String LS = System.getProperty("line.separator");
- private static final int mcw = 7; //min column width
-
- /**
- * Emit a table row
- * @param colWidths integers representing the column widths of each column.
- * This only makes the markdown source look nice.
- * @param strings the content of each column
- * @return the source table row
- */
- public static String emitRow(int[] colWidths, String ... strings) {
- int cols = colWidths.length;
- if (cols != strings.length) {
- throw new IllegalArgumentException("Unequal # of columns");
- }
- StringBuilder sb = new StringBuilder();
- for (int i = 0; i < cols; i++) {
- String s = strings[i];
- int w = colWidths[i];
- if (s.isEmpty()) {
- sb.append(pad(s, max(w, mcw))).append("|");
- } else {
- sb.append(pad(s, max(w, mcw))).append("|");
- }
- }
- sb.append(LS);
- return sb.toString();
- }
-
- /**
- * Emit a table row where all columns are bold face type
- * @param colWidths integers representing the column widths of each column.
- * This only makes the markdown source look nice.
- * @param strings the content of each column
- * @return the source table row in bold
- */
- public static String emitBoldRow(int[] colWidths, String ... strings) {
- int cols = colWidths.length;
- if (cols != strings.length) {
- throw new IllegalArgumentException("Unequal # of columns");
- }
- StringBuilder sb = new StringBuilder();
- for (int i = 0; i < cols; i++) {
- String s = strings[i];
- int w = colWidths[i];
- if (s.isEmpty()) {
- sb.append(pad(s, max(w, mcw))).append("|");
- } else {
- String s2 = "<b>" + s + "</b>";
- sb.append(pad(s2, max(w, mcw))).append("|");
- }
- }
- sb.append(LS);
- return sb.toString();
- }
-
- /**
- * Emit the alignment line for GitHub MarkDown tables
- * @param colWidths the maximum width for each column
- * @param align a sequence of #columns characters in column order:
- * "C"=center, "L"=left, "R"=right
- * @return the alignment line
- */
- public static String emitAlignLine(int[] colWidths, String align) {
- int cols = colWidths.length;
- StringBuilder sb = new StringBuilder();
- for (int i = 0; i < cols; i++) {
- char c = align.charAt(i);
- int w = colWidths[i];
- if (c == 'C') {
- int numDash = max(w - 2, mcw - 2);
- sb.append(":").append(dupDash(numDash)).append(":|");
- }
- else if (c == 'L') {
- int numDash = max(w - 1, mcw - 1);
- sb.append(":").append(dupDash(numDash)).append("|");
- }
- else {
- int numDash = max(w - 1, mcw - 1);
- sb.append(dupDash(numDash)).append(":|");
- }
- }
- sb.append(LS);
- return sb.toString();
- }
-
- private static String dupDash(int num) {
- StringBuilder sb = new StringBuilder();
- for (int i = 0; i < num; i++) { sb.append("-"); }
- return sb.toString();
- }
-
- private static String pad(String s, int num) {
- StringBuilder sb = new StringBuilder();
- int strLen = s.length();
- if (strLen >= num) { return s; }
- sb.append(s);
- for (int i = 0; i < (num - strLen); i++) {
- sb.append(" ");
- }
- return sb.toString();
- }
-}
diff --git a/src/main/java/org/apache/datasketches/docgen/TocGenerator.java b/src/main/java/org/apache/datasketches/docgen/TocGenerator.java
deleted file mode 100644
index 84aa626..0000000
--- a/src/main/java/org/apache/datasketches/docgen/TocGenerator.java
+++ /dev/null
@@ -1,263 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.datasketches.docgen;
-
-import static org.apache.datasketches.Files.openPrintWriter;
-
-import java.io.File;
-import java.io.PrintWriter;
-import java.util.Iterator;
-
-import org.json.JSONArray;
-import org.json.JSONObject;
-import org.testng.annotations.Test;
-import org.apache.datasketches.Files;
-
-/**
- * @author Lee Rhodes
- */
-public class TocGenerator {
- private static final String LS = System.getProperty("line.separator");
- private int level = 0;
- private PrintWriter pw = null;
- private String jsonScrFile;
- private String htmlScriptFile;
-
- TocGenerator() {} //needed for TestNG
-
- /**
- * To create the Table of Contents for the website:
- * <ol>
- * <li>Edit the JSON source file (refereced below) for the structure you want.</li>
- * <li>Execute this test. The result will be placed in the proper location as part of the web
- * source.</li>
- * <li>Stage the changes and push the web site source to origin.</li>
- * <li>Refresh your browser and confirm that the TOC is correct.
- * </ol>
- * @author Lee Rhodes
- */
- @Test
- public static void runTocGenerator() {
- final String jsonSrcFile = "src/main/resources/docgen/toc.json";
- final String htmlScriptFile = "src/main/resources/docgen/tocScript.html";
- final String tgtTocFile = "_includes/toc.html";
- TocGenerator tocgen = new TocGenerator(jsonSrcFile, htmlScriptFile, tgtTocFile);
- tocgen.readJson();
- }
-
- /**
- * Execute the runTocGenerator above.
- * @param jsonSrcFile The JSON source file
- * @param htmlScriptFile The javascript source file
- * @param tgtTocFile the target toc.html file
- */
- public TocGenerator(final String jsonSrcFile, final String htmlScriptFile, final String tgtTocFile) {
- jsonScrFile = jsonSrcFile;
- this.htmlScriptFile = htmlScriptFile;
- if ((tgtTocFile != null) && (!tgtTocFile.isEmpty())) {
- final File file = new File(tgtTocFile);
- if (file.exists()) { file.delete(); }
- pw = openPrintWriter(tgtTocFile);
- }
- }
-
- /**
- * Reads the JSON source file and the html script file and generates the target toc.html file.
- */
- public void readJson() {
- final StringBuilder sb = new StringBuilder();
- final String jin = Files.fileToString(jsonScrFile);
- final JSONObject jo = new JSONObject(jin);
- final String clazz = jo.getString("class");
- if (clazz.equals("TOC")) { emitToc(jo, sb); }
- else if (clazz.equals("Dropdown")) { emitDropdown(jo, sb); }
- else { emitDoc(jo, sb); }
- if ((htmlScriptFile != null) && (!htmlScriptFile.isEmpty())) {
- final String script = Files.fileToString(htmlScriptFile);
- sb.append(script);
- }
- println(sb.toString());
- }
-
- /**
- * Generates an entire toc.html in a StringBuilder
- * @param toc the input JSON object
- * @param sb the target StringBuilder
- */
- void emitToc(final JSONObject toc, final StringBuilder sb) {
- sb.append("<!-- Computer Generated File, Do Not Edit! -->").append(LS);
- sb.append("<link rel=\"stylesheet\" href=\"/css/toc.css\">").append(LS);
- sb.append("<div id=\"toc\" class=\"nav toc hidden-print\">").append(LS);
-
- //JSONArray
- level++;
- final JSONArray jarr = toc.getJSONArray("array");
- final Iterator<Object> itr = jarr.iterator();
- while (itr.hasNext()) {
- final JSONObject jo = (JSONObject) itr.next();
- final String clazz = jo.getString("class");
- if (clazz.equals("Dropdown")) { emitDropdown(jo, sb); }
- else { emitDoc(jo, sb); }
- }
- level--;
-
- sb.append("</div>").append(LS);
- }
-
- /**
- * Generates a Dropdown object from the input JSON source.
- * This is a recursive co-routine with the emitDoc() function.
- * @param dropdn the input JSON
- * @param sb the target StringBuilder
- */
- void emitDropdown(final JSONObject dropdn, final StringBuilder sb) {
- final String desc = dropdn.getString("desc");
- final String lowercaseDesc = desc.toLowerCase();
- final String pId = lowercaseDesc.replace(' ', '-');
- final String divId = "collapse_" + lowercaseDesc.replace(' ', '_');
- final String href = "#" + divId;
- final String indent = indent(level);
- //paragraph with desc
- sb.append(LS);
- sb.append(indent).append("<p id=").append(quotes(pId)).append(">").append(LS);
- sb.append(indent).append(" ").append("<a data-toggle=\"collapse\" ")
- .append("class=\"menu collapsed\" href=").append(quotes(href)).append(">")
- .append(desc).append("</a>").append(LS);
- sb.append(indent).append("</p>").append(LS);
- //start dropdown array
- sb.append(indent).append("<div class=\"collapse\" ").append("id=").append(quotes(divId))
- .append(">").append(LS);
-
- //JSONArray
- level++;
- final JSONArray jarr = dropdn.getJSONArray("array");
- final Iterator<Object> itr = jarr.iterator();
- while (itr.hasNext()) {
- final JSONObject jo = (JSONObject) itr.next();
- final String clazz = jo.getString("class");
- if (clazz.equals("Dropdown")) { emitDropdown(jo, sb); }
- else { emitDoc(jo, sb); }
- }
- level--;
-
- sb.append(indent).append("</div>").append(LS);
- }
-
- /**
- * Generates a Document object from the input JSON source.
- * This is a recursive co-routine with the emitDropdown() function.
- * @param doc the input JSON
- * @param sb the target StringBuilder
- */
- void emitDoc(final JSONObject doc, final StringBuilder sb) {
- final String dir = doc.getString("dir");
- final String file = doc.getString("file");
- final String desc = doc.getString("desc");
- final boolean pdf = doc.optBoolean("pdf");
- final String indent = indent(level);
- sb.append(indent).append("<li><a href=\"");
- if (dir.equals("ROOT")) { sb.append("/"); }
- else {
- final String baseDir = pdf ? "{{site.docs_pdf_dir}}/" : "{{site.docs_dir}}/";
- sb.append(baseDir);
- if (!dir.isEmpty()) {
- sb.append(dir + "/");
- }
- }
- sb.append(file);
- final String sfx = pdf ? ".pdf" : ".html";
- sb.append(sfx + "\">");
- sb.append(desc);
- sb.append("</a></li>").append(LS);
- }
-
- /**
- * Encases the given string in quotes.
- * @param s the given string
- * @return the quoted string
- */
- public static String quotes(final String s) {
- return '"' + s + '"';
- }
-
- /**
- * @param level indention level
- * @return the indention spaces
- */
- public static String indent(final int level) {
- assert level >= 0;
- final StringBuilder sb = new StringBuilder();
- for (int i = 0; i < level; i++) {
- sb.append(" ");
- }
- return sb.toString();
- }
-
-
- /**
- * The JVM may call this method to close the PrintWriter resource.
- */
- @Override
- protected void finalize() throws Throwable {
- try {
- if (pw != null) {
- pw.close(); // close open files
- }
- } finally {
- super.finalize();
- }
- }
-
- /**
- * Outputs a line to the configured PrintWriter and stdOut.
- * @param s The String to print
- */
- public final void println(final String s) {
- System.out.println(s);
- if (pw != null) {
- pw.println(s);
- pw.flush();
- }
- }
-
- /**
- * Flush any buffered output to the configured PrintWriter.
- */
- public final void flush() {
- if (pw != null) { pw.flush(); }
- }
-
- /**
- * Command line access.
- * @param args three arguments are required:
- * <ol><li>The JSON source file</li>
- * <li>The html script file that is appended to the end.</li>
- * <li>The target toc.html file</li>
- * </ol>
- */
- public static void main(final String[] args) {
- final String jsonSrcFile = args[0];
- final String htmlScriptFile = args[1];
- final String tgtTocFile = args[2];
- final TocGenerator tocgen = new TocGenerator(jsonSrcFile, htmlScriptFile, tgtTocFile);
- tocgen.readJson();
- }
-
-}
diff --git a/src/main/resources/docgen/toc.json b/src/main/resources/docgen/toc.json
deleted file mode 100644
index 7ba42ee..0000000
--- a/src/main/resources/docgen/toc.json
+++ /dev/null
@@ -1,232 +0,0 @@
-{ "class":"TOC", "array":
- [
- { "class":"Dropdown", "desc" : "Overview", "array":
- [
- {"class":"Doc", "desc" : "The Challenge", "dir" : "", "file": "TheChallenge" },
- {"class":"Doc", "desc" : "The Major Sketch Families", "dir" : "", "file": "MajorSketchFamilies" },
- {"class":"Doc", "desc" : "Sketch Origins", "dir" : "", "file": "SketchOrigins" },
- {"class":"Doc", "desc" : "Sketch Elements", "dir" : "", "file": "SketchElements" },
- {"class":"Doc", "desc" : "Key Features", "dir" : "", "file": "KeyFeatures" },
- {"class":"Doc", "desc" : "Large Scale Computing", "dir" : "", "file": "LargeScale" },
- {"class":"Doc", "desc" : "Architecture", "dir" : "", "file": "Architecture" },
- {"class":"Doc", "desc" : "Notes on Order Sensitivity", "dir" : "", "file": "OrderSensitivity" },
- {"class":"Doc", "desc" : "Notes on Concurrency", "dir" : "", "file": "Concurrency" },
- {"class":"Doc", "desc" : "Overview Slide Deck", "dir" : "", "file": "DataSketches_deck", "pdf":"true" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Research", "array":
- [
- {"class":"Doc", "desc" : "Research", "dir" : "", "file": "Research" }
- ]
- },
-
- { "class":"Dropdown", "desc" : "Distinct Counting", "array":
- [
-
- { "class":"Dropdown", "desc" : "HyperLogLog Sketches", "array":
- [
- {"class":"Doc", "desc" : "HLL Sketch", "dir" : "HLL", "file": "HLL" },
- {"class":"Doc", "desc" : "HLL Map Sketch", "dir" : "HLL", "file": "HllMap" },
- { "class":"Dropdown", "desc" : "HLL Examples", "array":
- [
- {"class":"Doc", "desc" : "HLL Sketch Java Example", "dir" : "HLL", "file": "HllJavaExample" },
- {"class":"Doc", "desc" : "HLL Sketch Pig UDFs", "dir" : "HLL", "file": "HllPigUDFs" },
- {"class":"Doc", "desc" : "HLL Sketch Hive UDFs", "dir" : "HLL", "file": "HllHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
- { "class":"Dropdown", "desc" : "HLL Studies", "array":
- [
- {"class":"Doc", "desc" : "HLL vs HLL++", "dir" : "HLL", "file": "Hll_vs_Hllpp" },
- {"class":"Doc", "desc" : "HLL Sketch vs Druid HyperLogLogCollector", "dir" : "HLL", "file": "HllSketchVsDruidHyperLogLogCollector" },
- ]
- },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Theta Sketches", "array":
- [
- { "class":"Doc", "desc" : "Theta Sketch Framework", "dir" : "Theta", "file": "ThetaSketchFramework" },
-
- { "class":"Dropdown", "desc" : "Theta Examples", "array":
- [
- {"class":"Doc", "desc" : "Concurrent Theta Sketch", "dir" : "Theta", "file": "ConcurrentThetaSketch" },
- {"class":"Doc", "desc" : "Theta Sketch Java Example", "dir" : "Theta", "file": "ThetaJavaExample" },
- {"class":"Doc", "desc" : "Theta Sketch Spark Example", "dir" : "Theta", "file": "ThetaSparkExample" },
- {"class":"Doc", "desc" : "Theta Sketch Pig UDFs", "dir" : "Theta", "file": "ThetaPigUDFs" },
- {"class":"Doc", "desc" : "Theta Sketch Hive UDFs", "dir" : "Theta", "file": "ThetaHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "KMV Tutorial", "array":
- [
- {"class":"Doc", "desc" : "The Inverse Estimate", "dir" : "Theta", "file": "InverseEstimate" },
- {"class":"Doc", "desc" : "Empty Sketch", "dir" : "Theta", "file": "KMVempty" },
- {"class":"Doc", "desc" : "First Estimator", "dir" : "Theta", "file": "KMVfirstEst" },
- {"class":"Doc", "desc" : "Better Estimator", "dir" : "Theta", "file": "KMVbetterEst" },
- {"class":"Doc", "desc" : "Rejection Rules", "dir" : "Theta", "file": "KMVrejection" },
- {"class":"Doc", "desc" : "Update V(kth) Rule", "dir" : "Theta", "file": "KMVupdateVkth" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Set Operations and P-sampling", "array":
- [
- {"class":"Doc", "desc" : "Set Operations", "dir" : "Theta", "file": "ThetaSketchSetOps" },
- {"class":"Doc", "desc" : "<i>p</i>-Sampling", "dir" : "Theta", "file": "ThetaPSampling" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Accuracy", "array":
- [
- {"class":"Doc", "desc" : "Basic Accuracy", "dir" : "Theta", "file": "ThetaAccuracy" },
- {"class":"Doc", "desc" : "Accuracy Plots", "dir" : "Theta", "file": "ThetaAccuracyPlots" },
- {"class":"Doc", "desc" : "Relative Error Table", "dir" : "Theta", "file": "ThetaErrorTable" },
- {"class":"Doc", "desc" : "SetOp Accuracy", "dir" : "Theta", "file": "ThetaSketchSetOpsAccuracy" },
- {"class":"Doc", "desc" : "Unions With Different k", "dir" : "Theta", "file": "AccuracyOfDifferentKUnions" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Size", "array":
- [
- {"class":"Doc", "desc" : "Theta Sketch Size", "dir" : "Theta", "file": "ThetaSize" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Speed", "array":
- [
- {"class":"Doc", "desc" : "Update Speed", "dir" : "Theta", "file": "ThetaUpdateSpeed" },
- {"class":"Doc", "desc" : "Merge Speed", "dir" : "Theta", "file": "ThetaMergeSpeed" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Theta Sketch Theory", "array":
- [
- {"class":"Doc", "desc" : "Theta Sketch Framework (PDF)", "dir" : "", "file": "ThetaSketchFramework", "pdf":"true" },
- {"class":"Doc", "desc" : "Sketch Equations (PDF)", "dir" : "", "file": "SketchEquations", "pdf":"true" },
- {"class":"Doc", "desc" : "DataSketches (PDF)", "dir" : "", "file": "DataSketches", "pdf":"true" },
- {"class":"Doc", "desc" : "Confidence Intervals Notes", "dir" : "Theta", "file": "ThetaConfidenceIntervals" },
- {"class":"Doc", "desc" : "Merging Algorithm Notes", "dir" : "Theta", "file": "ThetaMergingAlgorithm" },
- {"class":"Doc", "desc" : "Theta References", "dir" : "Theta", "file": "ThetaReferences" },
- ]
- }
- ]
- },
-
- { "class":"Dropdown", "desc" : "Tuple Sketches", "array":
- [
- {"class":"Doc", "desc" : "Tuple Overview", "dir" : "Tuple", "file": "TupleOverview" },
- { "class":"Dropdown", "desc" : "Tuple Examples", "array":
- [
- {"class":"Doc", "desc" : "Tuple Java Example", "dir" : "Tuple", "file": "TupleJavaExample" },
- {"class":"Doc", "desc" : "Tuple Engagement Example", "dir" : "Tuple", "file": "TupleEngagementExample" },
- {"class":"Doc", "desc" : "Frequent Distinct Tuples Sketch", "dir" : "Frequency", "file": "FrequentDistinctTuplesSketch" },
- {"class":"Doc", "desc" : "Tuple Pig UDFs", "dir" : "Tuple", "file": "TuplePigUDFs" },
- {"class":"Doc", "desc" : "Tuple Hive UDFs", "dir" : "Tuple", "file": "TupleHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
- ]
- },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Most Frequent", "array":
- [
- {"class":"Doc", "desc" : "Frequency Sketches Overview", "dir" : "Frequency", "file": "FrequencySketchesOverview" },
-
- { "class":"Dropdown", "desc" : "Frequent Item Sketches", "array":
- [
- {"class":"Doc", "desc" : "Frequent Items Overview", "dir" : "Frequency", "file": "FrequentItemsOverview" },
- {"class":"Doc", "desc" : "Frequent Items Error Table", "dir" : "Frequency", "file": "FrequentItemsErrorTable" },
- {"class":"Doc", "desc" : "Frequent Items References", "dir" : "Frequency", "file": "FrequentItemsReferences" },
-
- { "class":"Dropdown", "desc" : "Most Frequent Examples", "array":
- [
- {"class":"Doc", "desc" : "Frequent Items Java Example", "dir" : "Frequency", "file": "FrequentItemsJavaExample" },
- {"class":"Doc", "desc" : "Frequent Items Pig UDFs", "dir" : "Frequency", "file": "FrequentItemsPigUDFs" },
- {"class":"Doc", "desc" : "Frequent Items Hive UDFs", "dir" : "Frequency", "file": "FrequentItemsHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Frequent Distinct Sketches", "array":
- [
- {"class":"Doc", "desc" : "Frequent Distinct Tuples Sketch", "dir" : "Frequency", "file": "FrequentDistinctTuplesSketch" },
- ]
- }
- ]
- },
-
- { "class":"Dropdown", "desc" : "Quantiles And Histograms", "array":
- [
- {"class":"Doc", "desc" : "Quantiles Overview", "dir" : "Quantiles", "file": "QuantilesOverview" },
- {"class":"Doc", "desc" : "Quantiles Accuracy and Size", "dir" : "Quantiles", "file": "QuantilesAccuracy" },
-
- { "class":"Dropdown", "desc" : "Quantiles Examples", "array":
- [
- {"class":"Doc", "desc" : "Quantiles Sketch Java Example", "dir" : "Quantiles", "file": "QuantilesJavaExample" },
- {"class":"Doc", "desc" : "Quantiles Sketch Pig UDFs", "dir" : "Quantiles", "file": "QuantilesPigUDFs" },
- {"class":"Doc", "desc" : "Quantiles Sketch Hive UDFs", "dir" : "Quantiles", "file": "QuantilesHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
- { "class":"Dropdown", "desc" : "Quantiles Studies", "array":
- [
- {"class":"Doc", "desc" : "New KLL sketch and comparison with DoublesSketch", "dir" : "Quantiles", "file": "KLLSketch" },
- {"class":"Doc", "desc" : "KLL sketch vs t-digest", "dir" : "Quantiles", "file": "KllSketchVsTDigest" },
- {"class":"Doc", "desc" : "Druid Approximate Histogram", "dir" : "Quantiles", "file": "DruidApproxHistogramStudy" },
- {"class":"Doc", "desc" : "Moments Sketch Study", "dir" : "Quantiles", "file": "MomentsSketchStudy" },
- {"class":"Doc", "desc" : "Quantiles StreamA Study", "dir" : "Quantiles", "file": "QuantilesStreamAStudy" },
- ]
- },
- { "class":"Dropdown", "desc" : "Quantiles Sketch Theory", "array":
- [
- {"class":"Doc", "desc" : "Optimal Quantile Approximation in Streams", "dir" : "", "file": "Quantiles_KLL", "pdf":"true" },
- {"class":"Doc", "desc" : "Quantiles References", "dir" : "Quantiles", "file": "QuantilesReferences" },
- ]
- }
- ]
- },
-
- { "class":"Dropdown", "desc" : "Sampling", "array":
- [
- {"class":"Doc", "desc" : "Reservoir Sampling", "dir" : "Sampling", "file": "ReservoirSampling" },
- {"class":"Doc", "desc" : "Reservoir Sampling Performance", "dir" : "Sampling", "file": "ReservoirSamplingPerformance" },
- {"class":"Doc", "desc" : "VarOpt Sampling", "dir" : "Sampling", "file": "VarOptSampling" },
-
- { "class":"Dropdown", "desc" : "Sampling Examples", "array":
- [
- {"class":"Doc", "desc" : "Reservoir Sampling Java Example", "dir" : "Sampling", "file": "ReservoirSamplingJava" },
- {"class":"Doc", "desc" : "Reservoir Sampling Pig UDFs", "dir" : "Sampling", "file": "ReservoirSamplingPigUDFs" },
- {"class":"Doc", "desc" : "VarOpt Sampling Java Example", "dir" : "Sampling", "file": "VarOptSamplingJava" },
- {"class":"Doc", "desc" : "VarOpt Sampling Pig UDFs", "dir" : "Sampling", "file": "VarOptPigUDFs" },
- ]
- },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Memory Package", "array":
- [
- {"class":"Doc", "desc" : "Memory Package", "dir" : "Memory", "file": "MemoryPackage" },
- {"class":"Doc", "desc" : "Memory Package Performance", "dir" : "Memory", "file": "MemoryPerformance" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Command Line", "array":
- [
- {"class":"Doc", "desc" : "Creating Command Line Executables", "dir" : "CommandLine", "file": "CommandLine" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Who Uses", "array":
- [
- {"class":"Doc", "desc" : "Who Uses", "dir" : "", "file": "WhoUses" },
- ]
- },
- ]
-}
-
diff --git a/src/main/resources/docgen/tocScript.html b/src/main/resources/docgen/tocScript.html
deleted file mode 100644
index 8a5ec35..0000000
--- a/src/main/resources/docgen/tocScript.html
+++ /dev/null
@@ -1,64 +0,0 @@
-<script>
- (function () {
-
- var findLineItem = function (path) {
- return document.querySelector(`[href="${path}"]`);
- };
-
- var highlighLineItem = function (element) {
- element.classList.add('highlight');
- };
-
- var checkHasClass = function (element, className) {
- return element.className.split(' ').find(function (item) { return item === className || '' })
- }
-
- var findAllCollapseParents = function (element) {
- var collapseMenus = [];
- var elementPointer = element;
- while (elementPointer !== document.body) {
- if (checkHasClass(elementPointer, 'collapse')) {
- collapseMenus.push(elementPointer);
- }
- elementPointer = elementPointer.parentElement
- }
- return collapseMenus
- };
-
- var findMatchingCollapseMenu = function (collapseItem) {
- return document.querySelector(`[href="#${collapseItem.id}"]`);
- };
-
- var findAllParentMenus = function (element) {
- var parentMenus = []
- var elementPointer = element;
- while (elementPointer !== document.body) {
- if (elementPointer.className.split(' ').indexOf('menu') > -1) {
- parentMenus.push(elementPointer);
- }
- elementPointer = elementPointer.parentElement;
- }
- return parentMenus;
- }
-
- var openMenuItem = function (element) {
- element.click();
- };
-
- var openAllFromList = function (elementList) {
- elementList.forEach(function (element) {
- console.log(element)
- openMenuItem(findMatchingCollapseMenu(element))
- });
- };
-
- var highlightAndOpenMenu = function () {
- var currentLineItem = findLineItem(document.location.pathname);
- highlighLineItem(currentLineItem)
- openAllFromList(findAllCollapseParents(currentLineItem));
- };
-
- $(highlightAndOpenMenu);
-
- }());
-</script>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org