You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Revan1988 <an...@gmail.com> on 2015/05/05 11:35:53 UTC

Ingest speed

Hi,
I'm an Italian student and i'm going to be graduated having  thesis about
using accumulo.
I've developed a little java application that reads LOGS in a .json file and
insert them into accumulo.
In my virtual machine my app's insert/sec score is about 6'000. 
I see that there are some bench test that scores about 50'000 in my VM.
(It's in $ACCUMULO_HOME/test/system/* folder.)
Those test use the shell call to insert a large amount of data.
So I think that i could write all my insert using an accumulo-shell call in
my app.
I've seen that i can execute that command:

./bin/accumulo shell -u username -p passw -e "insert row fam qual val [vis]
[timestamp]"

but there is a problem: i need to set up the table with the command table 
 (infact insert command has not -t 
 option).

So there is any way to execute two commands in a row into accumulo shell? i
tried with && ; and other separator but with no success.

The other option that i have is to write a big command file with all
commands that i need and send it to accumulo shell using the command 

./bin/accumulo shell -f <file> 

it may be a better solution cause i'll connect to accumulo just one time
(and not for every insert)...

Any suggest?

Thank you everybody and sorry for my bad english.

Revan



--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Ingest-speed-tp14005.html
Sent from the Developers mailing list archive at Nabble.com.

Re: Ingest speed

Posted by Christopher <ct...@apache.org>.
I was thinking about deprecating this behavior, and just using the
ServiceLoader to load all shell commands, rather than have a separate
one just for user-provided commands. It'd make the documentation
simpler. Something to keep in mind. Doesn't change the need for docs
on current behavior.

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii


On Tue, May 5, 2015 at 11:51 AM, Josh Elser <jo...@gmail.com> wrote:
> Well, let this serve as a big "please" to write _something_. JIRA is not a
> form of documentation for users.
>
> It's a neat feature, but if there's no documentation about it, it's going to
> die.
>
>
> dlmarion@comcast.net wrote:
>>
>> Only what is in the ticket. Not sure there is enough information to fill a
>> chapter.
>>
>> ----- Original Message -----
>>
>> From: "Josh Elser"<jo...@gmail.com>
>> To: dev@accumulo.apache.org
>> Sent: Tuesday, May 5, 2015 11:38:47 AM
>> Subject: Re: Ingest speed
>>
>> Dave -- is there any documentation on this? Seems like it would be very
>> well served by a user-manual chapter.
>>
>> dlmarion@comcast.net wrote:
>>>
>>> Revan,
>>>
>>> You could use the scripting[1] feature in the shell for this purpose. The
>>> feature allows you to use JSR-223 compatible languages for your scripts. You
>>> will see javascript and python as examples in the comments on the JIRA
>>> ticket.
>>>
>>> [1] https://issues.apache.org/jira/browse/ACCUMULO-1399
>>>
>>> ----- Original Message -----
>>>
>>> From: "Revan1988"<an...@gmail.com>
>>> To: dev@accumulo.apache.org
>>> Sent: Tuesday, May 5, 2015 5:35:53 AM
>>> Subject: Ingest speed
>>>
>>> Hi,
>>> I'm an Italian student and i'm going to be graduated having thesis about
>>> using accumulo.
>>> I've developed a little java application that reads LOGS in a .json file
>>> and
>>> insert them into accumulo.
>>> In my virtual machine my app's insert/sec score is about 6'000.
>>> I see that there are some bench test that scores about 50'000 in my VM.
>>> (It's in $ACCUMULO_HOME/test/system/* folder.)
>>> Those test use the shell call to insert a large amount of data.
>>> So I think that i could write all my insert using an accumulo-shell call
>>> in
>>> my app.
>>> I've seen that i can execute that command:
>>>
>>> ./bin/accumulo shell -u username -p passw -e "insert row fam qual val
>>> [vis]
>>> [timestamp]"
>>>
>>> but there is a problem: i need to set up the table with the command table
>>> (infact insert command has not -t
>>> option).
>>>
>>> So there is any way to execute two commands in a row into accumulo shell?
>>> i
>>> tried with&&  ; and other separator but with no success.
>>>
>>> The other option that i have is to write a big command file with all
>>> commands that i need and send it to accumulo shell using the command
>>>
>>> ./bin/accumulo shell -f<file>
>>>
>>> it may be a better solution cause i'll connect to accumulo just one time
>>> (and not for every insert)...
>>>
>>> Any suggest?
>>>
>>> Thank you everybody and sorry for my bad english.
>>>
>>> Revan
>>>
>>>
>>>
>>
>>
>

Re: Ingest speed

Posted by dl...@comcast.net.
Fair enough. Point taken. 

----- Original Message -----

From: "Josh Elser" <jo...@gmail.com> 
To: dev@accumulo.apache.org 
Sent: Tuesday, May 5, 2015 11:51:35 AM 
Subject: Re: Ingest speed 

Well, let this serve as a big "please" to write _something_. JIRA is not 
a form of documentation for users. 

It's a neat feature, but if there's no documentation about it, it's 
going to die. 

dlmarion@comcast.net wrote: 
> Only what is in the ticket. Not sure there is enough information to fill a chapter. 
> 
> ----- Original Message ----- 
> 
> From: "Josh Elser"<jo...@gmail.com> 
> To: dev@accumulo.apache.org 
> Sent: Tuesday, May 5, 2015 11:38:47 AM 
> Subject: Re: Ingest speed 
> 
> Dave -- is there any documentation on this? Seems like it would be very 
> well served by a user-manual chapter. 
> 
> dlmarion@comcast.net wrote: 
>> Revan, 
>> 
>> You could use the scripting[1] feature in the shell for this purpose. The feature allows you to use JSR-223 compatible languages for your scripts. You will see javascript and python as examples in the comments on the JIRA ticket. 
>> 
>> [1] https://issues.apache.org/jira/browse/ACCUMULO-1399 
>> 
>> ----- Original Message ----- 
>> 
>> From: "Revan1988"<an...@gmail.com> 
>> To: dev@accumulo.apache.org 
>> Sent: Tuesday, May 5, 2015 5:35:53 AM 
>> Subject: Ingest speed 
>> 
>> Hi, 
>> I'm an Italian student and i'm going to be graduated having thesis about 
>> using accumulo. 
>> I've developed a little java application that reads LOGS in a .json file and 
>> insert them into accumulo. 
>> In my virtual machine my app's insert/sec score is about 6'000. 
>> I see that there are some bench test that scores about 50'000 in my VM. 
>> (It's in $ACCUMULO_HOME/test/system/* folder.) 
>> Those test use the shell call to insert a large amount of data. 
>> So I think that i could write all my insert using an accumulo-shell call in 
>> my app. 
>> I've seen that i can execute that command: 
>> 
>> ./bin/accumulo shell -u username -p passw -e "insert row fam qual val [vis] 
>> [timestamp]" 
>> 
>> but there is a problem: i need to set up the table with the command table 
>> (infact insert command has not -t 
>> option). 
>> 
>> So there is any way to execute two commands in a row into accumulo shell? i 
>> tried with&& ; and other separator but with no success. 
>> 
>> The other option that i have is to write a big command file with all 
>> commands that i need and send it to accumulo shell using the command 
>> 
>> ./bin/accumulo shell -f<file> 
>> 
>> it may be a better solution cause i'll connect to accumulo just one time 
>> (and not for every insert)... 
>> 
>> Any suggest? 
>> 
>> Thank you everybody and sorry for my bad english. 
>> 
>> Revan 
>> 
>> 
>> 
> 
> 


Re: Ingest speed

Posted by Josh Elser <jo...@gmail.com>.
Well, let this serve as a big "please" to write _something_. JIRA is not 
a form of documentation for users.

It's a neat feature, but if there's no documentation about it, it's 
going to die.

dlmarion@comcast.net wrote:
> Only what is in the ticket. Not sure there is enough information to fill a chapter.
>
> ----- Original Message -----
>
> From: "Josh Elser"<jo...@gmail.com>
> To: dev@accumulo.apache.org
> Sent: Tuesday, May 5, 2015 11:38:47 AM
> Subject: Re: Ingest speed
>
> Dave -- is there any documentation on this? Seems like it would be very
> well served by a user-manual chapter.
>
> dlmarion@comcast.net wrote:
>> Revan,
>>
>> You could use the scripting[1] feature in the shell for this purpose. The feature allows you to use JSR-223 compatible languages for your scripts. You will see javascript and python as examples in the comments on the JIRA ticket.
>>
>> [1] https://issues.apache.org/jira/browse/ACCUMULO-1399
>>
>> ----- Original Message -----
>>
>> From: "Revan1988"<an...@gmail.com>
>> To: dev@accumulo.apache.org
>> Sent: Tuesday, May 5, 2015 5:35:53 AM
>> Subject: Ingest speed
>>
>> Hi,
>> I'm an Italian student and i'm going to be graduated having thesis about
>> using accumulo.
>> I've developed a little java application that reads LOGS in a .json file and
>> insert them into accumulo.
>> In my virtual machine my app's insert/sec score is about 6'000.
>> I see that there are some bench test that scores about 50'000 in my VM.
>> (It's in $ACCUMULO_HOME/test/system/* folder.)
>> Those test use the shell call to insert a large amount of data.
>> So I think that i could write all my insert using an accumulo-shell call in
>> my app.
>> I've seen that i can execute that command:
>>
>> ./bin/accumulo shell -u username -p passw -e "insert row fam qual val [vis]
>> [timestamp]"
>>
>> but there is a problem: i need to set up the table with the command table
>> (infact insert command has not -t
>> option).
>>
>> So there is any way to execute two commands in a row into accumulo shell? i
>> tried with&&  ; and other separator but with no success.
>>
>> The other option that i have is to write a big command file with all
>> commands that i need and send it to accumulo shell using the command
>>
>> ./bin/accumulo shell -f<file>
>>
>> it may be a better solution cause i'll connect to accumulo just one time
>> (and not for every insert)...
>>
>> Any suggest?
>>
>> Thank you everybody and sorry for my bad english.
>>
>> Revan
>>
>>
>>
>
>

Re: Ingest speed

Posted by dl...@comcast.net.
Only what is in the ticket. Not sure there is enough information to fill a chapter. 

----- Original Message -----

From: "Josh Elser" <jo...@gmail.com> 
To: dev@accumulo.apache.org 
Sent: Tuesday, May 5, 2015 11:38:47 AM 
Subject: Re: Ingest speed 

Dave -- is there any documentation on this? Seems like it would be very 
well served by a user-manual chapter. 

dlmarion@comcast.net wrote: 
> Revan, 
> 
> You could use the scripting[1] feature in the shell for this purpose. The feature allows you to use JSR-223 compatible languages for your scripts. You will see javascript and python as examples in the comments on the JIRA ticket. 
> 
> [1] https://issues.apache.org/jira/browse/ACCUMULO-1399 
> 
> ----- Original Message ----- 
> 
> From: "Revan1988"<an...@gmail.com> 
> To: dev@accumulo.apache.org 
> Sent: Tuesday, May 5, 2015 5:35:53 AM 
> Subject: Ingest speed 
> 
> Hi, 
> I'm an Italian student and i'm going to be graduated having thesis about 
> using accumulo. 
> I've developed a little java application that reads LOGS in a .json file and 
> insert them into accumulo. 
> In my virtual machine my app's insert/sec score is about 6'000. 
> I see that there are some bench test that scores about 50'000 in my VM. 
> (It's in $ACCUMULO_HOME/test/system/* folder.) 
> Those test use the shell call to insert a large amount of data. 
> So I think that i could write all my insert using an accumulo-shell call in 
> my app. 
> I've seen that i can execute that command: 
> 
> ./bin/accumulo shell -u username -p passw -e "insert row fam qual val [vis] 
> [timestamp]" 
> 
> but there is a problem: i need to set up the table with the command table 
> (infact insert command has not -t 
> option). 
> 
> So there is any way to execute two commands in a row into accumulo shell? i 
> tried with&& ; and other separator but with no success. 
> 
> The other option that i have is to write a big command file with all 
> commands that i need and send it to accumulo shell using the command 
> 
> ./bin/accumulo shell -f<file> 
> 
> it may be a better solution cause i'll connect to accumulo just one time 
> (and not for every insert)... 
> 
> Any suggest? 
> 
> Thank you everybody and sorry for my bad english. 
> 
> Revan 
> 
> 
> 


Re: Ingest speed

Posted by Josh Elser <jo...@gmail.com>.
Dave -- is there any documentation on this? Seems like it would be very 
well served by a user-manual chapter.

dlmarion@comcast.net wrote:
> Revan,
>
> You could use the scripting[1] feature in the shell for this purpose. The feature allows you to use JSR-223 compatible languages for your scripts. You will see javascript and python as examples in the comments on the JIRA ticket.
>
> [1] https://issues.apache.org/jira/browse/ACCUMULO-1399
>
> ----- Original Message -----
>
> From: "Revan1988"<an...@gmail.com>
> To: dev@accumulo.apache.org
> Sent: Tuesday, May 5, 2015 5:35:53 AM
> Subject: Ingest speed
>
> Hi,
> I'm an Italian student and i'm going to be graduated having thesis about
> using accumulo.
> I've developed a little java application that reads LOGS in a .json file and
> insert them into accumulo.
> In my virtual machine my app's insert/sec score is about 6'000.
> I see that there are some bench test that scores about 50'000 in my VM.
> (It's in $ACCUMULO_HOME/test/system/* folder.)
> Those test use the shell call to insert a large amount of data.
> So I think that i could write all my insert using an accumulo-shell call in
> my app.
> I've seen that i can execute that command:
>
> ./bin/accumulo shell -u username -p passw -e "insert row fam qual val [vis]
> [timestamp]"
>
> but there is a problem: i need to set up the table with the command table
> (infact insert command has not -t
> option).
>
> So there is any way to execute two commands in a row into accumulo shell? i
> tried with&&  ; and other separator but with no success.
>
> The other option that i have is to write a big command file with all
> commands that i need and send it to accumulo shell using the command
>
> ./bin/accumulo shell -f<file>
>
> it may be a better solution cause i'll connect to accumulo just one time
> (and not for every insert)...
>
> Any suggest?
>
> Thank you everybody and sorry for my bad english.
>
> Revan
>
>
>

Re: Ingest speed

Posted by dl...@comcast.net.
Revan, 

You could use the scripting[1] feature in the shell for this purpose. The feature allows you to use JSR-223 compatible languages for your scripts. You will see javascript and python as examples in the comments on the JIRA ticket. 

[1] https://issues.apache.org/jira/browse/ACCUMULO-1399 

----- Original Message -----

From: "Revan1988" <an...@gmail.com> 
To: dev@accumulo.apache.org 
Sent: Tuesday, May 5, 2015 5:35:53 AM 
Subject: Ingest speed 

Hi, 
I'm an Italian student and i'm going to be graduated having thesis about 
using accumulo. 
I've developed a little java application that reads LOGS in a .json file and 
insert them into accumulo. 
In my virtual machine my app's insert/sec score is about 6'000. 
I see that there are some bench test that scores about 50'000 in my VM. 
(It's in $ACCUMULO_HOME/test/system/* folder.) 
Those test use the shell call to insert a large amount of data. 
So I think that i could write all my insert using an accumulo-shell call in 
my app. 
I've seen that i can execute that command: 

./bin/accumulo shell -u username -p passw -e "insert row fam qual val [vis] 
[timestamp]" 

but there is a problem: i need to set up the table with the command table 
(infact insert command has not -t 
option). 

So there is any way to execute two commands in a row into accumulo shell? i 
tried with && ; and other separator but with no success. 

The other option that i have is to write a big command file with all 
commands that i need and send it to accumulo shell using the command 

./bin/accumulo shell -f <file> 

it may be a better solution cause i'll connect to accumulo just one time 
(and not for every insert)... 

Any suggest? 

Thank you everybody and sorry for my bad english. 

Revan 



-- 
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Ingest-speed-tp14005.html 
Sent from the Developers mailing list archive at Nabble.com. 


Re: Ingest speed

Posted by Revan1988 <an...@gmail.com>.
Fantastic! with the multiple batch writer i can have about 80k insert /
sec!!!
Now i'm going to study MapReduce!
Thank you very (VERY) much!




-----
Andrea Leoni
Italy
Computer Engineering
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Ingest-speed-tp14005p14029.html
Sent from the Developers mailing list archive at Nabble.com.

Re: Ingest speed

Posted by Josh Elser <jo...@gmail.com>.
Yes, a BatchWriter is for one table only. If you're writing to multiple 
tables, the MultiTableBatchWriter might be helpful. The 
MultiTableBatchWriter does the same thing that managing multiple 
BatchWriters would do but shares the memory usage.

Are you familiar with Hadoop's MapReduce framework?

http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html

MapReduce jobs accept data from InputFormats and write data to 
OutputFormats. Specifically, the FileInputFormat allows your MapReduce 
jobs to read data from HDFS and the AccumuloOutputFormat will write 
Mutations to an Accumulo table. Unless you have many nodes with lots and 
lots of data constantly flowing in, MapReduce might be overkill. I just 
thought I'd mention it though.

http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/client/mapreduce/AccumuloOutputFormat.html

Keep in touch -- wouldn't want to keep you from being able to graduate :)

Revan1988 wrote:
> Every one batchWriter is for only one table (isn't it?).
> I need to separate my json record in 3 tables (my record came from an IDS so
> i have to divide ALERT, DNS and HTTP record type).
> So maybe i can use 3 batchWriter... I'll try!!
>
> And what about FileInputFormat and the AccumuloOutputFormat? I'm sorry but i
> don't know it very well... do you have any website, pdf or sample that i can
> study about this?
>
> Thank you again!
> I want to do a good work because it is the project for my graduation of
> MSc... but here in my university no one know so much about accumulo.
>
>
>
> -----
> Andrea Leoni
> Italy
> Computer Engineering
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Ingest-speed-tp14005p14024.html
> Sent from the Developers mailing list archive at Nabble.com.

Re: Ingest speed

Posted by Revan1988 <an...@gmail.com>.
Every one batchWriter is for only one table (isn't it?). 
I need to separate my json record in 3 tables (my record came from an IDS so
i have to divide ALERT, DNS and HTTP record type).
So maybe i can use 3 batchWriter... I'll try!!

And what about FileInputFormat and the AccumuloOutputFormat? I'm sorry but i
don't know it very well... do you have any website, pdf or sample that i can
study about this?

Thank you again! 
I want to do a good work because it is the project for my graduation of
MSc... but here in my university no one know so much about accumulo.



-----
Andrea Leoni
Italy
Computer Engineering
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Ingest-speed-tp14005p14024.html
Sent from the Developers mailing list archive at Nabble.com.

Re: Ingest speed

Posted by Josh Elser <jo...@gmail.com>.
On a single node, you can easily achieve 10s of thousands of key-value 
inserts per second. Depending on how many columns are in each row, 600 a 
second is rather slow :)

Your loop looks good. Using a single BatchWriter and letting it amortize 
sending data from your client to the servers will be the most efficient.

If the JSON parsing is the slowest part, you could consider a single 
thread reading the file and provide the line to a thread pool, parse the 
line and add it to some concurrent data structure. You could have a 
consumer on that data structure reading each parsed object and sending 
it to Accumulo.

Alternatively, this is where MapReduce is a clear win as it's very good 
at parallelizing these types of problems. You could use the 
FileInputFormat and the AccumuloOutputFormat to accomplish this task.

Andrea Leoni wrote:
> Thank you for your answer.
> Today i tried to create a big command file and push it to shell (about 300k
> insert per file). As you said it is too slow for me (about 600 inserted
> row/sec)
>
> I'm on Accumulo by just one week. I'm a noob but i'm learning.
>
> Actually my app has to store a large number of data.
>
> The row is the timestamp and the family/qualif are the column... I catch my
> data from a JSON file, so my app scan it for new records, parse it and once
> for record create a mutation and push it on Accumulo with batchWriter...
>
> Maybe I wrong something that can increase the speed of my inserts.
>
> Actually I:
>
> LOOP
> 1) read a json line
> 2) parse it
> 3) create a mutation
> 4) put in this mutation the line's information
> 5) use batchWriter to insert mutation in Accumulo
> END LOOP
>
> Is it all right? I now that point 1) and 2) are slow but it's necessary and
> i use the fastest json parser i've found online.
>
> Thank you so much again!
> (and sorry again for my bad english!)
>
>
>
> -----
> Andrea Leoni
> Italy
> Computer Engineering
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Ingest-speed-tp14005p14013.html
> Sent from the Developers mailing list archive at Nabble.com.

Re: Ingest speed

Posted by dl...@comcast.net.
Your process seems sound, it's likely that you just need to scale it up. If you are not seeing wait times on the Accumulo monitor, then you have the ability to push more data. Are you processing multiple JSON files concurrently? 

----- Original Message -----

From: "Andrea Leoni" <an...@gmail.com> 
To: dev@accumulo.apache.org 
Sent: Tuesday, May 5, 2015 11:32:54 AM 
Subject: Re: Ingest speed 

Thank you for your answer. 
Today i tried to create a big command file and push it to shell (about 300k 
insert per file). As you said it is too slow for me (about 600 inserted 
row/sec) 

I'm on Accumulo by just one week. I'm a noob but i'm learning. 

Actually my app has to store a large number of data. 

The row is the timestamp and the family/qualif are the column... I catch my 
data from a JSON file, so my app scan it for new records, parse it and once 
for record create a mutation and push it on Accumulo with batchWriter... 

Maybe I wrong something that can increase the speed of my inserts. 

Actually I: 

LOOP 
1) read a json line 
2) parse it 
3) create a mutation 
4) put in this mutation the line's information 
5) use batchWriter to insert mutation in Accumulo 
END LOOP 

Is it all right? I now that point 1) and 2) are slow but it's necessary and 
i use the fastest json parser i've found online. 

Thank you so much again! 
(and sorry again for my bad english!) 



----- 
Andrea Leoni 
Italy 
Computer Engineering 
-- 
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Ingest-speed-tp14005p14013.html 
Sent from the Developers mailing list archive at Nabble.com. 


Re: Ingest speed

Posted by Andrea Leoni <an...@gmail.com>.
Thank you for your answer.
Today i tried to create a big command file and push it to shell (about 300k
insert per file). As you said it is too slow for me (about 600 inserted
row/sec)

I'm on Accumulo by just one week. I'm a noob but i'm learning.

Actually my app has to store a large number of data.

The row is the timestamp and the family/qualif are the column... I catch my
data from a JSON file, so my app scan it for new records, parse it and once
for record create a mutation and push it on Accumulo with batchWriter... 

Maybe I wrong something that can increase the speed of my inserts.

Actually I:

LOOP
1) read a json line
2) parse it
3) create a mutation
4) put in this mutation the line's information
5) use batchWriter to insert mutation in Accumulo
END LOOP

Is it all right? I now that point 1) and 2) are slow but it's necessary and
i use the fastest json parser i've found online.

Thank you so much again!
(and sorry again for my bad english!)



-----
Andrea Leoni
Italy
Computer Engineering
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Ingest-speed-tp14005p14013.html
Sent from the Developers mailing list archive at Nabble.com.

Re: Ingest speed

Posted by Josh Elser <jo...@gmail.com>.
Hi Revan,

You likely don't want to use the shell as a means to ingest as you will 
get abysmal performance (each record you insert will create a 
batchwriter, write one record and close the batchwriter).

But, if bad performance is ok, using a large file of shell commands with 
the `shell -f ..` option you noted would work. This would be slightly 
more efficient as you said (not re-authenticating with Accumulo for 
every insert. Before you start insert records, you can switch to a table 
context using the `table <tablename>` command.

Feel free to open an issue on JIRA to add a "-t" option to the insert 
command as this would be a good addition 
(https://issues.apache.org/jira/secure/CreateIssue!default.jspa)


Revan1988 wrote:
> Hi,
> I'm an Italian student and i'm going to be graduated having  thesis about
> using accumulo.
> I've developed a little java application that reads LOGS in a .json file and
> insert them into accumulo.
> In my virtual machine my app's insert/sec score is about 6'000.
> I see that there are some bench test that scores about 50'000 in my VM.
> (It's in $ACCUMULO_HOME/test/system/* folder.)
> Those test use the shell call to insert a large amount of data.
> So I think that i could write all my insert using an accumulo-shell call in
> my app.
> I've seen that i can execute that command:
>
> ./bin/accumulo shell -u username -p passw -e "insert row fam qual val [vis]
> [timestamp]"
>
> but there is a problem: i need to set up the table with the command table
>   (infact insert command has not -t
>   option).
>
> So there is any way to execute two commands in a row into accumulo shell? i
> tried with&&  ; and other separator but with no success.
>
> The other option that i have is to write a big command file with all
> commands that i need and send it to accumulo shell using the command
>
> ./bin/accumulo shell -f<file>
>
> it may be a better solution cause i'll connect to accumulo just one time
> (and not for every insert)...
>
> Any suggest?
>
> Thank you everybody and sorry for my bad english.
>
> Revan
>
>
>
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Ingest-speed-tp14005.html
> Sent from the Developers mailing list archive at Nabble.com.