You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by kt...@apache.org on 2019/07/18 19:40:32 UTC

[accumulo-testing] 01/01: Update ci bulk ingest docs

This is an automated email from the ASF dual-hosted git repository.

kturner pushed a commit to branch keith-turner-patch-1
in repository https://gitbox.apache.org/repos/asf/accumulo-testing.git

commit 6f059376d6644b24a0aa836cd6dba96c879dd8ff
Author: Keith Turner <kt...@apache.org>
AuthorDate: Thu Jul 18 15:40:27 2019 -0400

    Update ci bulk ingest docs
---
 docs/bulk-test.md | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/docs/bulk-test.md b/docs/bulk-test.md
index df5dc45..da8f163 100644
--- a/docs/bulk-test.md
+++ b/docs/bulk-test.md
@@ -8,6 +8,12 @@ in a loop like the following to continually bulk import data.
 # create the ci table if necessary
 ./bin/cingest createtable
 
+# Optionally, consider lowering the split threshold to make splits happen more 
+# frequently while the test runs.  Choose a threshold base on the amount of data
+# being imported and the desired number of splits.
+# 
+#   accumulo shell -u root -p secret -e 'config -t ci -s table.split.threshold=32M'
+
 for i in $(seq 1 10); do
    # run map reduce job to generate data for bulk import
    ./bin/cingest bulk /tmp/bt/$i
@@ -47,3 +53,13 @@ scan -t accumulo.metadata -b ~blip -e ~blip~
 scan -t accumulo.metadata -c loaded
 ```
 
+The counts (add referenced and unrefrenced) output by `cingest verify` should equal :
+
+```
+test.ci.bulk.map.task * test.ci.bulk.map.nodes * num_bulk_generate_jobs
+``` 
+
+Its possible the counts could be slightly smaller because of collisions. However collisions 
+are unlikely with the default settings given there are 63 bits of randomness in the row and 
+30 bits in the column.  This gives a total of 93 bits of randomness per key.
+