[GitHub] [accumulo-website] dlmarion commented on pull request #282: Documentation for external compactions

[GitBox <gi...@apache.org>]

dlmarion commented on pull request #282:
URL: https://github.com/apache/accumulo-website/pull/282#issuecomment-843179335


   Right, so what's missing here is that you have to start the compactor and the coordinator. Here is an example that I gave to @milleruntime using uno:
   
   ```
   ./bin/uno fetch accumulo
   ./bin/uno setup accumulo
   ./install/accumulo-2.1.0-SNAPSHOT/bin/accumulo compaction-coordinator >coordinator.out 2> coordinator.err &
   ./install/accumulo-2.1.0-SNAPSHOT/bin/accumulo compactor -q DCQ1 >compactor.out 2>compactor.err &
   
   ./install/accumulo-2.1.0-SNAPSHOT/bin/accumulo shell -u root -p secret <<EOF
   
   config -s tserver.compaction.major.service.cs1.planner=org.apache.accumulo.core.spi.compaction.DefaultCompactionPlanner
   config -s 'tserver.compaction.major.service.cs1.planner.opts.executors=[{"name":"all","externalQueue":"DCQ1"}]'
   createtable testTable
   config -t testTable -s table.compaction.dispatcher=org.apache.accumulo.core.spi.compaction.SimpleCompactionDispatcher
   config -t testTable -s table.compaction.dispatcher.opts.service=cs1
   EOF
   ```
   
   Then, you need to insert some data into the table. You could do it in the Accumulo shell using commands like:
   ```
   insert r1 q1 f1 v1
   insert r1 q1 f2 v2
   insert r1 q1 f3 v3
   insert r1 q1 f4 v4
   insert r1 q1 f5 v5
   insert r1 q1 f6 v6
   flush -t testTable -w
   insert r2 q1 f1 v7
   insert r2 q1 f2 v8
   insert r2 q1 f3 v9
   flush -t testTable -w
   compact -t testTable
   ```
   
   Or, you could do it with a script. For example, drop the following content into /tmp/insert.js:
   ```
   function insertData(tableName, numRows) {
     var bwConfig = new org.apache.accumulo.core.client.BatchWriterConfig();
     bwConfig.setMaxMemory(1024);
     var bw = connection.createBatchWriter(tableName, bwConfig);
     for (var x = 0; x < numRows; x++) {
       var mut = new org.apache.accumulo.core.data.Mutation(new org.apache.hadoop.io.Text(x));
       mut.put("cf", "cq", new org.apache.accumulo.core.data.Value(new java.lang.String("value").getBytes()));
       bw.addMutation(mut);
       // println("Adding " + x);
       if ((x % 1000) == 0) {
         connection.tableOperations().compact(tableName, null, null, java.util.Collections.emptyList(), true, false);
       }
     }
     bw.flush();
     bw.close();
   }
   ```
   
   Then, run the following in the Accumulo shell:
   ```
   script -e nashorn -f /tmp/insert.js -fx insertData -a tableName=testTable,numRows=100000
   ```
   
   The output will be in the coordinator and compactor logs. You can also start multiple compactors.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org