You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datasketches.apache.org by pa...@apache.org on 2021/01/07 16:07:40 UTC
[datasketches-java] branch master updated (3501b0d -> d768b80)
This is an automated email from the ASF dual-hosted git repository.
pavelvesely pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/datasketches-java.git.
from 3501b0d Add HRA check to merge
new 1138560 minor tweaks of the sketch description
new 7dd55fc superfluous calls of buf.sort() -- calling sort() in FloatBuffer.getEvensOrOdds is sufficient
new a869181 miniscule speedup
new d768b80 removing laziness of compression
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
src/main/java/org/apache/datasketches/req/ReqCompactor.java | 2 +-
src/main/java/org/apache/datasketches/req/ReqSketch.java | 11 ++++++-----
2 files changed, 7 insertions(+), 6 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-java] 04/04: removing laziness of compression
Posted by pa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
pavelvesely pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/datasketches-java.git
commit d768b80374db91d13925a780d3ebde637f110de7
Author: Pavel Vesely <ve...@iuuk.mff.cuni.cz>
AuthorDate: Thu Jan 7 14:01:02 2021 +0100
removing laziness of compression
---
src/main/java/org/apache/datasketches/req/ReqSketch.java | 1 -
1 file changed, 1 deletion(-)
diff --git a/src/main/java/org/apache/datasketches/req/ReqSketch.java b/src/main/java/org/apache/datasketches/req/ReqSketch.java
index d7d6f9d..9b99717 100644
--- a/src/main/java/org/apache/datasketches/req/ReqSketch.java
+++ b/src/main/java/org/apache/datasketches/req/ReqSketch.java
@@ -185,7 +185,6 @@ public class ReqSketch extends BaseReqSketch {
compactors.get(h + 1).getBuffer().mergeSortIn(promoted);
retItems += cReturn.deltaRetItems;
maxNomSize += cReturn.deltaNomSize;
- if (retItems < maxNomSize) { break; }
}
}
aux = null;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-java] 01/04: minor tweaks of the sketch description
Posted by pa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
pavelvesely pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/datasketches-java.git
commit 1138560c5ddbeb4843ea45828ed56889a850c21b
Author: Pavel Vesely <ve...@iuuk.mff.cuni.cz>
AuthorDate: Thu Jan 7 12:08:06 2021 +0100
minor tweaks of the sketch description
---
src/main/java/org/apache/datasketches/req/ReqSketch.java | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/src/main/java/org/apache/datasketches/req/ReqSketch.java b/src/main/java/org/apache/datasketches/req/ReqSketch.java
index aca257a..8ff6829 100644
--- a/src/main/java/org/apache/datasketches/req/ReqSketch.java
+++ b/src/main/java/org/apache/datasketches/req/ReqSketch.java
@@ -39,14 +39,17 @@ import org.apache.datasketches.memory.Memory;
* <ul>
* <li>The algorithm requires no upper bound on the stream length.
* Instead, each relative-compactor counts the number of compaction operations performed
- * so far (variable numCompactions). Initially, the relative-compactor starts with 3 sections.
- * Each time the numCompactions exceeds 2^{numSections - 1}, we double numSections.</li>
+ * so far (via variable state). Initially, the relative-compactor starts with 3 sections.
+ * Each time the number of compactions (variable state) exceeds 2^{numSections - 1}, we double numSections.
+ * Note that after merging the sketch with another one variable state may not correspond to the number of
+ * compactions performed at a particular level, however, since the state variable never exceeds
+ * the number of compactions, the guarantees of the sketch remain valid.</li>
*
* <li>The size of each section (variable k and sectionSize in the code and parameter k in
* the paper) is initialized with a value set by the user via variable k.
* When the number of sections doubles, we decrease sectionSize by a factor of sqrt(2).
* This is applied at each level separately. Thus, when we double the number of sections, the
- * nominal compactor size increases by a factor of sqrt(2) (up to +-1 after rounding).</li>
+ * nominal compactor size increases by a factor of approx. sqrt(2) (up to rounding issues).</li>
*
* <li>The merge operation here does not perform "special compactions", which are used in the paper
* to allow for a tight mathematical analysis of the sketch.</li>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-java] 03/04: miniscule speedup
Posted by pa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
pavelvesely pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/datasketches-java.git
commit a8691817d3c98928c8ca33a66f47bd0b3c54bcbf
Author: Pavel Vesely <ve...@iuuk.mff.cuni.cz>
AuthorDate: Thu Jan 7 13:04:21 2021 +0100
miniscule speedup
---
src/main/java/org/apache/datasketches/req/ReqCompactor.java | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/main/java/org/apache/datasketches/req/ReqCompactor.java b/src/main/java/org/apache/datasketches/req/ReqCompactor.java
index d89b9e2..5c88410 100644
--- a/src/main/java/org/apache/datasketches/req/ReqCompactor.java
+++ b/src/main/java/org/apache/datasketches/req/ReqCompactor.java
@@ -232,6 +232,7 @@ class ReqCompactor {
final float szf;
final int ne;
if (state >= 1L << numSections - 1
+ && sectionSize > MIN_K
&& (ne = nearestEven(szf = (float)(sectionSizeFlt / SQRT2))) >= MIN_K)
{
sectionSizeFlt = szf;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[datasketches-java] 02/04: superfluous calls of buf.sort() --
calling sort() in FloatBuffer.getEvensOrOdds is sufficient
Posted by pa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
pavelvesely pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/datasketches-java.git
commit 7dd55fcc94e5f8d092cb842e02547125a48aa9e2
Author: Pavel Vesely <ve...@iuuk.mff.cuni.cz>
AuthorDate: Thu Jan 7 12:58:25 2021 +0100
superfluous calls of buf.sort() -- calling sort() in FloatBuffer.getEvensOrOdds is sufficient
---
src/main/java/org/apache/datasketches/req/ReqCompactor.java | 1 -
src/main/java/org/apache/datasketches/req/ReqSketch.java | 1 -
2 files changed, 2 deletions(-)
diff --git a/src/main/java/org/apache/datasketches/req/ReqCompactor.java b/src/main/java/org/apache/datasketches/req/ReqCompactor.java
index 172882a..d89b9e2 100644
--- a/src/main/java/org/apache/datasketches/req/ReqCompactor.java
+++ b/src/main/java/org/apache/datasketches/req/ReqCompactor.java
@@ -119,7 +119,6 @@ class ReqCompactor {
*/
FloatBuffer compact(final CompactorReturn cReturn) {
if (reqDebug != null) { reqDebug.emitCompactingStart(lgWeight); }
- buf.sort();
final int startRetItems = buf.getCount();
final int startNomCap = getNomCapacity();
// choose a part of the buffer to compact
diff --git a/src/main/java/org/apache/datasketches/req/ReqSketch.java b/src/main/java/org/apache/datasketches/req/ReqSketch.java
index 8ff6829..d7d6f9d 100644
--- a/src/main/java/org/apache/datasketches/req/ReqSketch.java
+++ b/src/main/java/org/apache/datasketches/req/ReqSketch.java
@@ -515,7 +515,6 @@ public class ReqSketch extends BaseReqSketch {
retItems++;
totalN++;
if (retItems >= maxNomSize) {
- buf.sort();
compress();
}
aux = null;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org