You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by Vicky Kak <vi...@gmail.com> on 2014/07/10 07:52:43 UTC
Re: How does Accumulo compare to HBase
Have you got started with the blog, in case not I can spare some time
writing about it?
High ingestion speed and scan wrt to HBase is eye opener for me, I would be
interested to dig deeper into it my self.
Wondering if anyone had some information about the ingestion/scan speed
with the Cassandra?
On Tue, Jun 24, 2014 at 1:58 AM, Josh Elser <jo...@gmail.com> wrote:
> Noted: I'll add it to the top of my "to blog" queue. If anyone else wants
> to do a write-up, I'm happy to help.
>
>
> On 6/23/14, 4:23 PM, Donald Miner wrote:
>
>> This needs to be documented on the official blog.
>>
>>
>> On Mon, Jun 23, 2014 at 3:31 PM, Josh Elser <josh.elser@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> Sent too quickly..
>>
>> - The BatchScanner is communicating to tservers in *parallel* which
>> is where this really shows it strength.
>>
>> - A "default" locality group. You don't have to define the locality
>> groups for a table at creation time in Accumulo (or have to modify
>> the table if you want to insert a new column family). Because of
>> this, you have a lot more flexibility in how you structure your
>> tables while also being able to take advantage of the efficient
>> filtering you get having locality groups you have configured. Adding
>> a new locality group does still require a compaction to re-write the
>> data in separate files.
>>
>>
>> On 6/23/14, 3:24 PM, Josh Elser wrote:
>>
>> A few observations I can make from watching both communities
>> (although
>> only really participating in Accumulo's).
>>
>> - HBase undeniably has a much larger public community of both
>> users and
>> developers; however, we are seeing broader adoption across
>> different
>> vertical markets with Accumulo. IMO, I think we have a rather
>> responsive
>> community built up here. Lots of smart people are working that are
>> available and happy to help with problems.
>>
>> - BatchScanner: The BatchScanner is a query construct which will
>> automatically fetch data from a collection of Ranges on a table
>> and
>> return the results in the form of a Java Iterator. This makes
>> for a very
>> natural way to read lots of data from Accumulo, automatically
>> performing
>> some reduction in the data server-side (using Accumulo
>> Iterators), and
>> getting a wonderfully simple Iterator<Entry<Key,Value>> in your
>> client
>> code. It really helps to encourage a state-less and
>> functional-like
>> style to your code.
>>
>> I really like it, and, when combined with the ability to push a
>> bunch of
>> work server-side, it has often kept me from having to write
>> MapReduce
>> jobs (which is always a win to me).
>>
>> - Accumulo Iterators are a common thing you might hear as a
>> difference.
>> AFAICT, they're a bit more powerful than what you can do with
>> HBase
>> filters because you are presented with a stream of Key-Value pairs
>> inside of the TServer. Again, it's a bit functional programming
>> inspired. You have the ability to combine, consume, seek within
>> the
>> stream and do what you please (more context would be helpful in
>> giving
>> specific examples)
>>
>> That being said, Iterators do come with a learning curve, but
>> that's to
>> be expected with the amount of flexibility they provide. It's
>> just like
>> anything else :)
>>
>> - <disclaimer>I can't comment about running HBase in production
>> environments, but I tend to hear a lot of "war stories" about
>> it. I also
>> don't know how much of this is from running old version of HBase
>> which
>> don't have known issues patched. </disclaimer>
>>
>> In my experience, Accumulo just works. It doesn't require much
>> day-to-day interaction, processes stay running and if some node
>> goes
>> haywire, I have absolutely no qualms against `kill -9`'ing it and
>> knowing that everything will come back fine.
>>
>> My $0.02.
>>
>> - Josh
>>
>> On 6/23/14, 2:49 PM, Josh Elser wrote:
>>
>> Another way you could word this is that Accumulo has a very
>> "mature"
>> security implementation, whereas, like you pointed out,
>> HBase has only
>> recently added this in 0.98.
>>
>> The note about how visibility being in the Key as opposed to
>> the Value
>> also has impact when writing Iterators. Because the
>> visibility is a
>> "first class citizen" instead of an afterthought, having it
>> uniquely
>> define some pair makes aggregations much easier to think
>> about, IMO.
>> This is especially prevalent when doing this server-side with
>> an
>> Accumulo Iterator.
>>
>> There are also other differences between the implementations
>> visibility
>> filtering, the most common being the support of a "NOT"
>> operator in
>> HBase whereas Accumulo explicitly chose not to implement
>> this. By
>> allowing "NOT" into the syntax, it becomes much more
>> possible that data
>> is inadvertently leaked. Marking data correctly is more
>> difficult than
>> it seems and introducing the ability to negate certain
>> branches makes it
>> even more difficult. Auditors are scary :)
>>
>> - Josh
>>
>> On 6/23/14, 2:34 PM, Aaron wrote:
>>
>> I'm not sure of all the differences, but, wrt HBase Cell
>> Level security
>> (CLS)..while similar..not 100% the same. If I
>> understand how the HBase
>> CLS works it's extension to ACL system. And that ACL is
>> "applied" to a
>> cell. In Accumulo's case, it is part of the key. So
>> the ramification
>> is that in Accumulo, you can have:
>>
>> RowID, CF, CQ, VIS1, TS --> Value1
>> RowID, CF, CQ, VIS2, TS --> Value2
>>
>> If everything is the same, including the timestamp, the
>> visibility can
>> actually determine which value to return. So, a more
>> concrete example
>> would be:
>>
>> XXX, METADATA, NAME, everyone, 100--> Bruce Wayne
>> XXX, METADATA, NAME, alfred-only, 100--> Batman
>>
>> Where Alfred could/would see both "values"...but,
>> everyone else would
>> only see "Bruce"
>>
>> Hope that helps.
>>
>> Cheers,
>> Aaron
>>
>> PS: this is my understanding of how HBase CLS
>> works...based on what I
>> have read/interpreted.
>>
>>
>>
>> On Mon, Jun 23, 2014 at 1:55 PM, Jianshi Huang
>> <jianshi.huang@gmail.com <ma...@gmail.com>
>> <mailto:jianshi.huang@gmail.__com
>>
>> <ma...@gmail.com>>> wrote:
>>
>> Er... basically I need to explain to my manager why
>> choosing
>> Accumulo, instead of HBase.
>>
>> So what are the pros and cons of Accumulo vs.
>> HBase? (btw HBase 0.98
>> also got cell-level security, modeled after Accumulo)
>>
>> --
>> Jianshi Huang
>>
>> LinkedIn: jianshi
>> Twitter: @jshuang
>> Github & Blog: http://huangjs.github.com/
>>
>>
>>
>>
>>
>> --
>> *
>> *Donald Miner
>>
>> Chief Technology Officer
>> ClearEdge IT Solutions, LLC
>> Cell: 443 799 7807
>> www.clearedgeit.com <http://www.clearedgeit.com>
>>
>
Re: How does Accumulo compare to HBase
Posted by David Medinets <da...@gmail.com>.
>On Fri, Jul 11, 2014 at 7:25 AM, <ch...@oracle.com> wrote:
>
>The entirety of both data corpuses were re-loaded every night?
Yes.
>What did the users do while the data was dropped and reloaded?
The technique of 'dropping and reloading' was not used.
Users were not impacted. For the original system, we used
a combination of Sqoop and the Fair Scheduler in Hadoop to
throttle the export. For Accumulo, we created new tables using
a date-based naming convention. Accumulo queries used a lookup
process to find the current table. When the new table was
ready it was automatically used.
>What happened in the middle of night if the job failed?
Why has this conversation topic changed to "Is David
competent to design an ingest system"?
>Couldn’t you identify the incremental updates to the two sources
>and incrementally load the new data into the combined target?
Yes, we could. But, for reasons not germane to this conversation,
we pulled the whole corpus.
>This brute force implementation is only applicable to a few use
>cases with lax SLAs.
Ok.
>>From: David Medinets [mailto:david.medinets@gmail.com]
>>Last year, I used Accumulo's rapid ingest ability to join two data
>>silos into one dataset. Every field was fully indexed. Having all
>>of the data in one place allowed cross-referencing queries to be
>>executed. For various reason, this kind of query was not possible
>>using the existing technology. The rapid ingest was important
>>because a new copy of the data silos was pulled every night.
RE: How does Accumulo compare to HBase
Posted by Chuck Adams <ch...@oracle.com>.
David –
The entirety of both data corpuses were re-loaded every night? What did the users do while the data was dropped and reloaded? What happened in the middle of night if the job failed? Couldn’t you identify the incremental updates to the two sources and incrementally load the new data into the combined target?
This brute force implementation is only applicable to a few use cases with lax SLAs.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HYPERLINK "http://www.oracle.com/"Oracle
Innovative technologies enabling the worlds best intelligence
Chuck Adams
Vice President Technical Leadership Team
National Security Group
1910 Oracle Way
Reston, VA 20190
Cell: 301.529.9396
Email: HYPERLINK "mailto:dave.rose@oracle.com"chuck.adams@oracle.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
From: David Medinets [mailto:david.medinets@gmail.com]
Sent: Thursday, July 10, 2014 2:56 PM
To: accumulo-user
Subject: Re: How does Accumulo compare to HBase
Last year, I used Accumulo's rapid ingest ability to join two data silos into one dataset. Every field was fully indexed. Having all of the data in one place allowed cross-referencing queries to be executed. For various reason, this kind of query was not possible using the existing technology. The rapid ingest was important because a new copy of the data silos was pulled every night.
On Thu, Jul 10, 2014 at 1:55 PM, Sean Busbey <HYPERLINK "mailto:busbey@cloudera.com" \nbusbey@cloudera.com> wrote:
Chuck,
It would help the community and my own benchmarking efforts if you could describe how you think a benchmark might incorporate representations of real-world bottlenecks.
Do you think YCSB sufficiently covers the kind of testing you'd prefer?
Marc,
Similarly, it would help if you could describe the use case(s) behind your statement of interest.
-Sean
On Thu, Jul 10, 2014 at 12:11 PM, Marc Parisi <HYPERLINK "mailto:marc@accumulo.net" \nmarc@accumulo.net> wrote:
I care
On Thu, Jul 10, 2014 at 11:33 AM, Chuck Adams <HYPERLINK "mailto:chuck.adams@oracle.com" \nchuck.adams@oracle.com> wrote:
Dr. Kepner,
Who cares how fast you can load data into a non-indexed HBase or Accumulo database? What is the strategy to handle user queries against this corpus? Run some real world tests which include simultaneous queries, index maintenance, information life cycle management, during the initial and incremental data loads.
The test you are running do not appear to have any of the real world bottlenecks that occur for production systems that users rely on for their business.
Respectfully,
Chuck Adams
Vice President Technical Leadership Team
Oracle National Security Group
1910 Oracle Way
Reston, VA 20190
Cell: HYPERLINK "tel:301.529.9396" \n301.529.9396
Email: HYPERLINK "mailto:chuck.adams@oracle.com" \nchuck.adams@oracle.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----Original Message-----
From: Jeremy Kepner [mailto:HYPERLINK "mailto:kepner@ll.mit.edu" \nkepner@ll.mit.edu]
Sent: Thursday, July 10, 2014 11:08 AM
To: HYPERLINK "mailto:user@accumulo.apache.org" \nuser@accumulo.apache.org
Subject: Re: How does Accumulo compare to HBase
If you repeated the experiments you did for the Accumulo only portion with HBase what would you expect the results to be?
On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
> At the risk of derailing the original thread, I'll take a moment to
> explain my methodology. Since each entry can be thought of as a 6
> dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
> fiddling with the specifics of it. YCSB gives you several knobs, but
> unfortunately, not absolutely everything was tunable.
>
> The things that are configurable:
> - # of rows
> - # of column qualifiers
> - length of value
> - number of operations per client
> - number of clients
>
> Things that are not configurable:
> - row length (it's a long int)
> - # of column families (constant at one per row)
> - length of column qualifier (basically a one-up counter per row)
> - visibilities
>
> In all of my experiments, the goal was to keep data size constant.
> This can be approximated by (number of entries * entry size). Number
> of entries is intuitively rows (configurable) * column families (1) *
> columns qualifiers per family (configurable), while entry size is key
> overhead (about 40
> bytes) + configured length of value. So to keep total size constant,
> we have three easy knobs. However, tweaking three values at a time
> produces really messy data where you're not always going to be sure
> where the causality arrow lies. Even doing two at a time can cause
> issues but then the choice is between tweaking two properties of the
> data, or one property of the data and the total size (which is also a relevant attribute).
>
> Whew. So why did I use two different independent variables between the
> two halves?
>
> Partly, because I'm not comparing the two tests to each other, so they
> don't have to be duplicative. I ran them on different hardware from
> each other, with different number of clients, disks, cores, etc.
> There's no meaningful comparisons to be drawn, so I wanted to remove
> the temptation to compare results against each other. I'll admit that
> I might be wrong in this regard.
>
> The graphs are not my complete data sets. For the Accumulo v Accumulo
> tests, we have about ten more data points varying rows and data size
> as well. Trying to show three independent variables on a graph was
> pretty painful, so they didn't make it into the presentation. The
> short version of the story is that nothing scaled linearly (some
> things were better, some things were worse) but the general trend
> lines were approximately what you would expect.
>
> Let me know if you have more questions, but we can probably start a
> new thread for future search posterity! (This applies to everybody).
>
> Mike
>
>
> On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
> HYPERLINK "mailto:kepner@ll.mit.edu" \nkepner@ll.mit.edu> wrote:
>
> > Mike Drob put together a great talk at the Accumulo Summit (
> > http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
> > Accumulo performance and HBase performance. This exactly the kind
> > of work the entire Hadoop community needs to continue to move forward.
> >
> > I had one question about the talk which I was wondering if someone
> > might be able to shed light on. In the Accumulo part of the talk
> > the experiments varied #rows while keeping the #cols fixed, while in
> > the Accumulo/HBase part of the the experiments varied #cols while keeping #rows fixed?
--
Sean
Re: How does Accumulo compare to HBase
Posted by David Medinets <da...@gmail.com>.
Last year, I used Accumulo's rapid ingest ability to join two data silos
into one dataset. Every field was fully indexed. Having all of the data in
one place allowed cross-referencing queries to be executed. For various
reason, this kind of query was not possible using the existing technology.
The rapid ingest was important because a new copy of the data silos was
pulled every night.
On Thu, Jul 10, 2014 at 1:55 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
> Chuck,
>
> It would help the community and my own benchmarking efforts if you could
> describe how you think a benchmark might incorporate representations of
> real-world bottlenecks.
>
> Do you think YCSB sufficiently covers the kind of testing you'd prefer?
>
>
> Marc,
>
> Similarly, it would help if you could describe the use case(s) behind your
> statement of interest.
>
> -Sean
>
>
> On Thu, Jul 10, 2014 at 12:11 PM, Marc Parisi <ma...@accumulo.net> wrote:
>
>> I care
>>
>>
>> On Thu, Jul 10, 2014 at 11:33 AM, Chuck Adams <ch...@oracle.com>
>> wrote:
>>
>>> Dr. Kepner,
>>>
>>> Who cares how fast you can load data into a non-indexed HBase or
>>> Accumulo database? What is the strategy to handle user queries against
>>> this corpus? Run some real world tests which include simultaneous queries,
>>> index maintenance, information life cycle management, during the initial
>>> and incremental data loads.
>>>
>>> The test you are running do not appear to have any of the real world
>>> bottlenecks that occur for production systems that users rely on for their
>>> business.
>>>
>>> Respectfully,
>>> Chuck Adams
>>>
>>> Vice President Technical Leadership Team
>>> Oracle National Security Group
>>> 1910 Oracle Way
>>> Reston, VA 20190
>>>
>>> Cell: 301.529.9396
>>> Email: chuck.adams@oracle.com
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>
>>> -----Original Message-----
>>> From: Jeremy Kepner [mailto:kepner@ll.mit.edu]
>>> Sent: Thursday, July 10, 2014 11:08 AM
>>> To: user@accumulo.apache.org
>>> Subject: Re: How does Accumulo compare to HBase
>>>
>>> If you repeated the experiments you did for the Accumulo only portion
>>> with HBase what would you expect the results to be?
>>>
>>> On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
>>> > At the risk of derailing the original thread, I'll take a moment to
>>> > explain my methodology. Since each entry can be thought of as a 6
>>> > dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
>>> > fiddling with the specifics of it. YCSB gives you several knobs, but
>>> > unfortunately, not absolutely everything was tunable.
>>> >
>>> > The things that are configurable:
>>> > - # of rows
>>> > - # of column qualifiers
>>> > - length of value
>>> > - number of operations per client
>>> > - number of clients
>>> >
>>> > Things that are not configurable:
>>> > - row length (it's a long int)
>>> > - # of column families (constant at one per row)
>>> > - length of column qualifier (basically a one-up counter per row)
>>> > - visibilities
>>> >
>>> > In all of my experiments, the goal was to keep data size constant.
>>> > This can be approximated by (number of entries * entry size). Number
>>> > of entries is intuitively rows (configurable) * column families (1) *
>>> > columns qualifiers per family (configurable), while entry size is key
>>> > overhead (about 40
>>> > bytes) + configured length of value. So to keep total size constant,
>>> > we have three easy knobs. However, tweaking three values at a time
>>> > produces really messy data where you're not always going to be sure
>>> > where the causality arrow lies. Even doing two at a time can cause
>>> > issues but then the choice is between tweaking two properties of the
>>> > data, or one property of the data and the total size (which is also a
>>> relevant attribute).
>>> >
>>> > Whew. So why did I use two different independent variables between the
>>> > two halves?
>>> >
>>> > Partly, because I'm not comparing the two tests to each other, so they
>>> > don't have to be duplicative. I ran them on different hardware from
>>> > each other, with different number of clients, disks, cores, etc.
>>> > There's no meaningful comparisons to be drawn, so I wanted to remove
>>> > the temptation to compare results against each other. I'll admit that
>>> > I might be wrong in this regard.
>>> >
>>> > The graphs are not my complete data sets. For the Accumulo v Accumulo
>>> > tests, we have about ten more data points varying rows and data size
>>> > as well. Trying to show three independent variables on a graph was
>>> > pretty painful, so they didn't make it into the presentation. The
>>> > short version of the story is that nothing scaled linearly (some
>>> > things were better, some things were worse) but the general trend
>>> > lines were approximately what you would expect.
>>> >
>>> > Let me know if you have more questions, but we can probably start a
>>> > new thread for future search posterity! (This applies to everybody).
>>> >
>>> > Mike
>>> >
>>> >
>>> > On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
>>> > kepner@ll.mit.edu> wrote:
>>> >
>>> > > Mike Drob put together a great talk at the Accumulo Summit (
>>> > > http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
>>> > > Accumulo performance and HBase performance. This exactly the kind
>>> > > of work the entire Hadoop community needs to continue to move
>>> forward.
>>> > >
>>> > > I had one question about the talk which I was wondering if someone
>>> > > might be able to shed light on. In the Accumulo part of the talk
>>> > > the experiments varied #rows while keeping the #cols fixed, while in
>>> > > the Accumulo/HBase part of the the experiments varied #cols while
>>> keeping #rows fixed?
>>>
>>
>>
>
>
> --
> Sean
>
RE: How does Accumulo compare to HBase
Posted by Chuck Adams <ch...@oracle.com>.
Hi Sean –
Sorry I told Dr. Kepner that no one cares, because obviously some do!
I understand that there is a need to perform initial data loads and all products have the ability to bulk load data for this purpose. In this case, CPU is the limiting factor due to parsing and compressing of the data during the load. Add more CPU(nodes) and data segments to scale.
Once the data is loaded and indexed to meet the system query SLAs, incremental data load and aging data off while supporting simultaneous queries, incremental indexing, query concurrency, ILM, and meeting the query performance SLAs introduce NON-CPU bottlenecks. Here are couple of real world use cases I see everywhere, everyday:
1. 24x7 user and system to system query access to the corpus
2. 24x7 data stream loading from multiple sources into the corpus
3. 24x7 data roll off into a well defined ILM strategy
4. SLA for immediate access to new data. For example, no data staging and bulk load.
5. SLA for query performance. For example, no queuing.
Say hi to Rob Morrow for me. We miss him!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HYPERLINK "http://www.oracle.com/"Oracle
Innovative technologies enabling the worlds best intelligence
Chuck Adams
Vice President Technical Leadership Team
National Security Group
1910 Oracle Way
Reston, VA 20190
Cell: 301.529.9396
Email: HYPERLINK "mailto:dave.rose@oracle.com"chuck.adams@oracle.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
From: Sean Busbey [mailto:busbey@cloudera.com]
Sent: Thursday, July 10, 2014 1:55 PM
To: Accumulo User List
Subject: Re: How does Accumulo compare to HBase
Chuck,
It would help the community and my own benchmarking efforts if you could describe how you think a benchmark might incorporate representations of real-world bottlenecks.
Do you think YCSB sufficiently covers the kind of testing you'd prefer?
Marc,
Similarly, it would help if you could describe the use case(s) behind your statement of interest.
-Sean
On Thu, Jul 10, 2014 at 12:11 PM, Marc Parisi <HYPERLINK "mailto:marc@accumulo.net" \nmarc@accumulo.net> wrote:
I care
On Thu, Jul 10, 2014 at 11:33 AM, Chuck Adams <HYPERLINK "mailto:chuck.adams@oracle.com" \nchuck.adams@oracle.com> wrote:
Dr. Kepner,
Who cares how fast you can load data into a non-indexed HBase or Accumulo database? What is the strategy to handle user queries against this corpus? Run some real world tests which include simultaneous queries, index maintenance, information life cycle management, during the initial and incremental data loads.
The test you are running do not appear to have any of the real world bottlenecks that occur for production systems that users rely on for their business.
Respectfully,
Chuck Adams
Vice President Technical Leadership Team
Oracle National Security Group
1910 Oracle Way
Reston, VA 20190
Cell: HYPERLINK "tel:301.529.9396" \n301.529.9396
Email: HYPERLINK "mailto:chuck.adams@oracle.com" \nchuck.adams@oracle.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----Original Message-----
From: Jeremy Kepner [mailto:HYPERLINK "mailto:kepner@ll.mit.edu" \nkepner@ll.mit.edu]
Sent: Thursday, July 10, 2014 11:08 AM
To: HYPERLINK "mailto:user@accumulo.apache.org" \nuser@accumulo.apache.org
Subject: Re: How does Accumulo compare to HBase
If you repeated the experiments you did for the Accumulo only portion with HBase what would you expect the results to be?
On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
> At the risk of derailing the original thread, I'll take a moment to
> explain my methodology. Since each entry can be thought of as a 6
> dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
> fiddling with the specifics of it. YCSB gives you several knobs, but
> unfortunately, not absolutely everything was tunable.
>
> The things that are configurable:
> - # of rows
> - # of column qualifiers
> - length of value
> - number of operations per client
> - number of clients
>
> Things that are not configurable:
> - row length (it's a long int)
> - # of column families (constant at one per row)
> - length of column qualifier (basically a one-up counter per row)
> - visibilities
>
> In all of my experiments, the goal was to keep data size constant.
> This can be approximated by (number of entries * entry size). Number
> of entries is intuitively rows (configurable) * column families (1) *
> columns qualifiers per family (configurable), while entry size is key
> overhead (about 40
> bytes) + configured length of value. So to keep total size constant,
> we have three easy knobs. However, tweaking three values at a time
> produces really messy data where you're not always going to be sure
> where the causality arrow lies. Even doing two at a time can cause
> issues but then the choice is between tweaking two properties of the
> data, or one property of the data and the total size (which is also a relevant attribute).
>
> Whew. So why did I use two different independent variables between the
> two halves?
>
> Partly, because I'm not comparing the two tests to each other, so they
> don't have to be duplicative. I ran them on different hardware from
> each other, with different number of clients, disks, cores, etc.
> There's no meaningful comparisons to be drawn, so I wanted to remove
> the temptation to compare results against each other. I'll admit that
> I might be wrong in this regard.
>
> The graphs are not my complete data sets. For the Accumulo v Accumulo
> tests, we have about ten more data points varying rows and data size
> as well. Trying to show three independent variables on a graph was
> pretty painful, so they didn't make it into the presentation. The
> short version of the story is that nothing scaled linearly (some
> things were better, some things were worse) but the general trend
> lines were approximately what you would expect.
>
> Let me know if you have more questions, but we can probably start a
> new thread for future search posterity! (This applies to everybody).
>
> Mike
>
>
> On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
> HYPERLINK "mailto:kepner@ll.mit.edu" \nkepner@ll.mit.edu> wrote:
>
> > Mike Drob put together a great talk at the Accumulo Summit (
> > http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
> > Accumulo performance and HBase performance. This exactly the kind
> > of work the entire Hadoop community needs to continue to move forward.
> >
> > I had one question about the talk which I was wondering if someone
> > might be able to shed light on. In the Accumulo part of the talk
> > the experiments varied #rows while keeping the #cols fixed, while in
> > the Accumulo/HBase part of the the experiments varied #cols while keeping #rows fixed?
--
Sean
Re: How does Accumulo compare to HBase
Posted by Sean Busbey <bu...@cloudera.com>.
Chuck,
It would help the community and my own benchmarking efforts if you could
describe how you think a benchmark might incorporate representations of
real-world bottlenecks.
Do you think YCSB sufficiently covers the kind of testing you'd prefer?
Marc,
Similarly, it would help if you could describe the use case(s) behind your
statement of interest.
-Sean
On Thu, Jul 10, 2014 at 12:11 PM, Marc Parisi <ma...@accumulo.net> wrote:
> I care
>
>
> On Thu, Jul 10, 2014 at 11:33 AM, Chuck Adams <ch...@oracle.com>
> wrote:
>
>> Dr. Kepner,
>>
>> Who cares how fast you can load data into a non-indexed HBase or Accumulo
>> database? What is the strategy to handle user queries against this corpus?
>> Run some real world tests which include simultaneous queries, index
>> maintenance, information life cycle management, during the initial and
>> incremental data loads.
>>
>> The test you are running do not appear to have any of the real world
>> bottlenecks that occur for production systems that users rely on for their
>> business.
>>
>> Respectfully,
>> Chuck Adams
>>
>> Vice President Technical Leadership Team
>> Oracle National Security Group
>> 1910 Oracle Way
>> Reston, VA 20190
>>
>> Cell: 301.529.9396
>> Email: chuck.adams@oracle.com
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>
>> -----Original Message-----
>> From: Jeremy Kepner [mailto:kepner@ll.mit.edu]
>> Sent: Thursday, July 10, 2014 11:08 AM
>> To: user@accumulo.apache.org
>> Subject: Re: How does Accumulo compare to HBase
>>
>> If you repeated the experiments you did for the Accumulo only portion
>> with HBase what would you expect the results to be?
>>
>> On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
>> > At the risk of derailing the original thread, I'll take a moment to
>> > explain my methodology. Since each entry can be thought of as a 6
>> > dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
>> > fiddling with the specifics of it. YCSB gives you several knobs, but
>> > unfortunately, not absolutely everything was tunable.
>> >
>> > The things that are configurable:
>> > - # of rows
>> > - # of column qualifiers
>> > - length of value
>> > - number of operations per client
>> > - number of clients
>> >
>> > Things that are not configurable:
>> > - row length (it's a long int)
>> > - # of column families (constant at one per row)
>> > - length of column qualifier (basically a one-up counter per row)
>> > - visibilities
>> >
>> > In all of my experiments, the goal was to keep data size constant.
>> > This can be approximated by (number of entries * entry size). Number
>> > of entries is intuitively rows (configurable) * column families (1) *
>> > columns qualifiers per family (configurable), while entry size is key
>> > overhead (about 40
>> > bytes) + configured length of value. So to keep total size constant,
>> > we have three easy knobs. However, tweaking three values at a time
>> > produces really messy data where you're not always going to be sure
>> > where the causality arrow lies. Even doing two at a time can cause
>> > issues but then the choice is between tweaking two properties of the
>> > data, or one property of the data and the total size (which is also a
>> relevant attribute).
>> >
>> > Whew. So why did I use two different independent variables between the
>> > two halves?
>> >
>> > Partly, because I'm not comparing the two tests to each other, so they
>> > don't have to be duplicative. I ran them on different hardware from
>> > each other, with different number of clients, disks, cores, etc.
>> > There's no meaningful comparisons to be drawn, so I wanted to remove
>> > the temptation to compare results against each other. I'll admit that
>> > I might be wrong in this regard.
>> >
>> > The graphs are not my complete data sets. For the Accumulo v Accumulo
>> > tests, we have about ten more data points varying rows and data size
>> > as well. Trying to show three independent variables on a graph was
>> > pretty painful, so they didn't make it into the presentation. The
>> > short version of the story is that nothing scaled linearly (some
>> > things were better, some things were worse) but the general trend
>> > lines were approximately what you would expect.
>> >
>> > Let me know if you have more questions, but we can probably start a
>> > new thread for future search posterity! (This applies to everybody).
>> >
>> > Mike
>> >
>> >
>> > On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
>> > kepner@ll.mit.edu> wrote:
>> >
>> > > Mike Drob put together a great talk at the Accumulo Summit (
>> > > http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
>> > > Accumulo performance and HBase performance. This exactly the kind
>> > > of work the entire Hadoop community needs to continue to move forward.
>> > >
>> > > I had one question about the talk which I was wondering if someone
>> > > might be able to shed light on. In the Accumulo part of the talk
>> > > the experiments varied #rows while keeping the #cols fixed, while in
>> > > the Accumulo/HBase part of the the experiments varied #cols while
>> keeping #rows fixed?
>>
>
>
--
Sean
Re: How does Accumulo compare to HBase
Posted by Marc Parisi <ma...@accumulo.net>.
I care
On Thu, Jul 10, 2014 at 11:33 AM, Chuck Adams <ch...@oracle.com>
wrote:
> Dr. Kepner,
>
> Who cares how fast you can load data into a non-indexed HBase or Accumulo
> database? What is the strategy to handle user queries against this corpus?
> Run some real world tests which include simultaneous queries, index
> maintenance, information life cycle management, during the initial and
> incremental data loads.
>
> The test you are running do not appear to have any of the real world
> bottlenecks that occur for production systems that users rely on for their
> business.
>
> Respectfully,
> Chuck Adams
>
> Vice President Technical Leadership Team
> Oracle National Security Group
> 1910 Oracle Way
> Reston, VA 20190
>
> Cell: 301.529.9396
> Email: chuck.adams@oracle.com
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
> -----Original Message-----
> From: Jeremy Kepner [mailto:kepner@ll.mit.edu]
> Sent: Thursday, July 10, 2014 11:08 AM
> To: user@accumulo.apache.org
> Subject: Re: How does Accumulo compare to HBase
>
> If you repeated the experiments you did for the Accumulo only portion with
> HBase what would you expect the results to be?
>
> On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
> > At the risk of derailing the original thread, I'll take a moment to
> > explain my methodology. Since each entry can be thought of as a 6
> > dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
> > fiddling with the specifics of it. YCSB gives you several knobs, but
> > unfortunately, not absolutely everything was tunable.
> >
> > The things that are configurable:
> > - # of rows
> > - # of column qualifiers
> > - length of value
> > - number of operations per client
> > - number of clients
> >
> > Things that are not configurable:
> > - row length (it's a long int)
> > - # of column families (constant at one per row)
> > - length of column qualifier (basically a one-up counter per row)
> > - visibilities
> >
> > In all of my experiments, the goal was to keep data size constant.
> > This can be approximated by (number of entries * entry size). Number
> > of entries is intuitively rows (configurable) * column families (1) *
> > columns qualifiers per family (configurable), while entry size is key
> > overhead (about 40
> > bytes) + configured length of value. So to keep total size constant,
> > we have three easy knobs. However, tweaking three values at a time
> > produces really messy data where you're not always going to be sure
> > where the causality arrow lies. Even doing two at a time can cause
> > issues but then the choice is between tweaking two properties of the
> > data, or one property of the data and the total size (which is also a
> relevant attribute).
> >
> > Whew. So why did I use two different independent variables between the
> > two halves?
> >
> > Partly, because I'm not comparing the two tests to each other, so they
> > don't have to be duplicative. I ran them on different hardware from
> > each other, with different number of clients, disks, cores, etc.
> > There's no meaningful comparisons to be drawn, so I wanted to remove
> > the temptation to compare results against each other. I'll admit that
> > I might be wrong in this regard.
> >
> > The graphs are not my complete data sets. For the Accumulo v Accumulo
> > tests, we have about ten more data points varying rows and data size
> > as well. Trying to show three independent variables on a graph was
> > pretty painful, so they didn't make it into the presentation. The
> > short version of the story is that nothing scaled linearly (some
> > things were better, some things were worse) but the general trend
> > lines were approximately what you would expect.
> >
> > Let me know if you have more questions, but we can probably start a
> > new thread for future search posterity! (This applies to everybody).
> >
> > Mike
> >
> >
> > On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
> > kepner@ll.mit.edu> wrote:
> >
> > > Mike Drob put together a great talk at the Accumulo Summit (
> > > http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
> > > Accumulo performance and HBase performance. This exactly the kind
> > > of work the entire Hadoop community needs to continue to move forward.
> > >
> > > I had one question about the talk which I was wondering if someone
> > > might be able to shed light on. In the Accumulo part of the talk
> > > the experiments varied #rows while keeping the #cols fixed, while in
> > > the Accumulo/HBase part of the the experiments varied #cols while
> keeping #rows fixed?
>
Re: How does Accumulo compare to HBase
Posted by Jianshi Huang <ji...@gmail.com>.
Hey Ted,
You're right, specialized filters such as ColumnRangeFilter and
ColumnPrefixFilter are/should be as efficient as row scan. Here's the
source of information:
http://hadoop-hbase.blogspot.com/2012/01/hbase-intra-row-scanning.html
So the API is not limited, and we can do both range scans on RK and CQ.
More general filters such as QualifierFilter will need to scan the whole
row, here's the source of info:
http://stackoverflow.com/questions/20837458/hbase-qualifier-filter-vs-value-filter-performance
Now it all makes sense to me, thanks for the challenge! :)
Jianshi
On Thu, Jul 17, 2014 at 4:09 PM, Ted Yu <yu...@gmail.com> wrote:
> W.r.t. HBase filter's performance, can you let us know the source of the
> information ?
> Is performance bad for all filters or some types of filters ?
>
> Cheers
>
> On Jul 16, 2014, at 11:48 PM, Jianshi Huang <ji...@gmail.com>
> wrote:
>
> I finally decided to make the Storage part agnostic and wrote a HBase
> adapter as well.
>
> I'd like to say that Accumulo's API makes more sense than HBase'.
>
> One example is scan. In Accumulo, scan can take a Range which can take a
> startKey and endKey; a Key is a combination of RK+CF+CQ+CV+TS. This worked
> perfect and really free the CQ to a magic field in schema design.
>
> In HBase, scan can only take a startRow and an endRow... which both
> indicates the RK part. Scan through a range of Columns will require a
> ColumnRangeFilter AFAIK. (I heard HBase filter's performance sucks, not
> sure why).
>
> Any HBase expert disagrees? :)
>
> Jianshi
>
>
>
>
> On Fri, Jul 11, 2014 at 9:56 PM, Kepner, Jeremy - 0553 - MITLL <
> kepner@ll.mit.edu> wrote:
>
>> In many communities real applications can be hard to come by. These
>> communities often sponsor the development of application/technology
>> agnostic benchmarks as a surrogate. YCBS is one example. Graph500 is
>> another example. This allows a technology developer to focus on their own
>> technology and not have to be in the awkward position of having to also
>> benchmark multiple technologies simultaneously. It's not a perfect system,
>> but in general, a few mediocre benchmarks tend to be a lot better than no
>> benchmarks (too many benchmarks is also a problem). The benchmarks also
>> help with communicating the work because you can just reference the
>> benchmark.
>>
>> Now the tricky part is educating the customer base in how to interpret
>> the benchmarks. To a certain degree this is simply the burden of being an
>> informed consumer, but we can do as much as we can to help them. Using
>> standard benchmarks and showing how they correlate with some applications
>> and don't correlate with other applications is our obligation.
>>
>> On Jul 11, 2014, at 1:05 AM, Josh Elser <jo...@gmail.com> wrote:
>>
>> > It's important to remember that YCSB is a benchmark designed to test a
>> specific workload across database systems.
>> >
>> > No benchmark is going to be representative of every "real-life"
>> workload that can be thought of. Understand what the benchmark is showing
>> and draw realistic conclusions about what the results are showing you WRT
>> the problem(s) you care about.
>> >
>> > On 7/10/14, 11:33 AM, Chuck Adams wrote:
>> >> Dr. Kepner,
>> >>
>> >> Who cares how fast you can load data into a non-indexed HBase or
>> Accumulo database? What is the strategy to handle user queries against
>> this corpus? Run some real world tests which include simultaneous queries,
>> index maintenance, information life cycle management, during the initial
>> and incremental data loads.
>> >>
>> >> The test you are running do not appear to have any of the real world
>> bottlenecks that occur for production systems that users rely on for their
>> business.
>> >>
>> >> Respectfully,
>> >> Chuck Adams
>> >>
>> >> Vice President Technical Leadership Team
>> >> Oracle National Security Group
>> >> 1910 Oracle Way
>> >> Reston, VA 20190
>> >>
>> >> Cell: 301.529.9396
>> >> Email: chuck.adams@oracle.com
>> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Jeremy Kepner [mailto:kepner@ll.mit.edu]
>> >> Sent: Thursday, July 10, 2014 11:08 AM
>> >> To: user@accumulo.apache.org
>> >> Subject: Re: How does Accumulo compare to HBase
>> >>
>> >> If you repeated the experiments you did for the Accumulo only portion
>> with HBase what would you expect the results to be?
>> >>
>> >> On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
>> >>> At the risk of derailing the original thread, I'll take a moment to
>> >>> explain my methodology. Since each entry can be thought of as a 6
>> >>> dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
>> >>> fiddling with the specifics of it. YCSB gives you several knobs, but
>> >>> unfortunately, not absolutely everything was tunable.
>> >>>
>> >>> The things that are configurable:
>> >>> - # of rows
>> >>> - # of column qualifiers
>> >>> - length of value
>> >>> - number of operations per client
>> >>> - number of clients
>> >>>
>> >>> Things that are not configurable:
>> >>> - row length (it's a long int)
>> >>> - # of column families (constant at one per row)
>> >>> - length of column qualifier (basically a one-up counter per row)
>> >>> - visibilities
>> >>>
>> >>> In all of my experiments, the goal was to keep data size constant.
>> >>> This can be approximated by (number of entries * entry size). Number
>> >>> of entries is intuitively rows (configurable) * column families (1) *
>> >>> columns qualifiers per family (configurable), while entry size is key
>> >>> overhead (about 40
>> >>> bytes) + configured length of value. So to keep total size constant,
>> >>> we have three easy knobs. However, tweaking three values at a time
>> >>> produces really messy data where you're not always going to be sure
>> >>> where the causality arrow lies. Even doing two at a time can cause
>> >>> issues but then the choice is between tweaking two properties of the
>> >>> data, or one property of the data and the total size (which is also a
>> relevant attribute).
>> >>>
>> >>> Whew. So why did I use two different independent variables between the
>> >>> two halves?
>> >>>
>> >>> Partly, because I'm not comparing the two tests to each other, so they
>> >>> don't have to be duplicative. I ran them on different hardware from
>> >>> each other, with different number of clients, disks, cores, etc.
>> >>> There's no meaningful comparisons to be drawn, so I wanted to remove
>> >>> the temptation to compare results against each other. I'll admit that
>> >>> I might be wrong in this regard.
>> >>>
>> >>> The graphs are not my complete data sets. For the Accumulo v Accumulo
>> >>> tests, we have about ten more data points varying rows and data size
>> >>> as well. Trying to show three independent variables on a graph was
>> >>> pretty painful, so they didn't make it into the presentation. The
>> >>> short version of the story is that nothing scaled linearly (some
>> >>> things were better, some things were worse) but the general trend
>> >>> lines were approximately what you would expect.
>> >>>
>> >>> Let me know if you have more questions, but we can probably start a
>> >>> new thread for future search posterity! (This applies to everybody).
>> >>>
>> >>> Mike
>> >>>
>> >>>
>> >>> On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
>> >>> kepner@ll.mit.edu> wrote:
>> >>>
>> >>>> Mike Drob put together a great talk at the Accumulo Summit (
>> >>>> http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
>> >>>> Accumulo performance and HBase performance. This exactly the kind
>> >>>> of work the entire Hadoop community needs to continue to move
>> forward.
>> >>>>
>> >>>> I had one question about the talk which I was wondering if someone
>> >>>> might be able to shed light on. In the Accumulo part of the talk
>> >>>> the experiments varied #rows while keeping the #cols fixed, while in
>> >>>> the Accumulo/HBase part of the the experiments varied #cols while
>> keeping #rows fixed?
>>
>>
>
>
> --
> Jianshi Huang
>
> LinkedIn: jianshi
> Twitter: @jshuang
> Github & Blog: http://huangjs.github.com/
>
>
--
Jianshi Huang
LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/
Re: How does Accumulo compare to HBase
Posted by Ted Yu <yu...@gmail.com>.
W.r.t. HBase filter's performance, can you let us know the source of the information ?
Is performance bad for all filters or some types of filters ?
Cheers
On Jul 16, 2014, at 11:48 PM, Jianshi Huang <ji...@gmail.com> wrote:
> I finally decided to make the Storage part agnostic and wrote a HBase adapter as well.
>
> I'd like to say that Accumulo's API makes more sense than HBase'.
>
> One example is scan. In Accumulo, scan can take a Range which can take a startKey and endKey; a Key is a combination of RK+CF+CQ+CV+TS. This worked perfect and really free the CQ to a magic field in schema design.
>
> In HBase, scan can only take a startRow and an endRow... which both indicates the RK part. Scan through a range of Columns will require a ColumnRangeFilter AFAIK. (I heard HBase filter's performance sucks, not sure why).
>
> Any HBase expert disagrees? :)
>
> Jianshi
>
>
>
>
> On Fri, Jul 11, 2014 at 9:56 PM, Kepner, Jeremy - 0553 - MITLL <ke...@ll.mit.edu> wrote:
>> In many communities real applications can be hard to come by. These communities often sponsor the development of application/technology agnostic benchmarks as a surrogate. YCBS is one example. Graph500 is another example. This allows a technology developer to focus on their own technology and not have to be in the awkward position of having to also benchmark multiple technologies simultaneously. It's not a perfect system, but in general, a few mediocre benchmarks tend to be a lot better than no benchmarks (too many benchmarks is also a problem). The benchmarks also help with communicating the work because you can just reference the benchmark.
>>
>> Now the tricky part is educating the customer base in how to interpret the benchmarks. To a certain degree this is simply the burden of being an informed consumer, but we can do as much as we can to help them. Using standard benchmarks and showing how they correlate with some applications and don't correlate with other applications is our obligation.
>>
>> On Jul 11, 2014, at 1:05 AM, Josh Elser <jo...@gmail.com> wrote:
>>
>> > It's important to remember that YCSB is a benchmark designed to test a specific workload across database systems.
>> >
>> > No benchmark is going to be representative of every "real-life" workload that can be thought of. Understand what the benchmark is showing and draw realistic conclusions about what the results are showing you WRT the problem(s) you care about.
>> >
>> > On 7/10/14, 11:33 AM, Chuck Adams wrote:
>> >> Dr. Kepner,
>> >>
>> >> Who cares how fast you can load data into a non-indexed HBase or Accumulo database? What is the strategy to handle user queries against this corpus? Run some real world tests which include simultaneous queries, index maintenance, information life cycle management, during the initial and incremental data loads.
>> >>
>> >> The test you are running do not appear to have any of the real world bottlenecks that occur for production systems that users rely on for their business.
>> >>
>> >> Respectfully,
>> >> Chuck Adams
>> >>
>> >> Vice President Technical Leadership Team
>> >> Oracle National Security Group
>> >> 1910 Oracle Way
>> >> Reston, VA 20190
>> >>
>> >> Cell: 301.529.9396
>> >> Email: chuck.adams@oracle.com
>> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Jeremy Kepner [mailto:kepner@ll.mit.edu]
>> >> Sent: Thursday, July 10, 2014 11:08 AM
>> >> To: user@accumulo.apache.org
>> >> Subject: Re: How does Accumulo compare to HBase
>> >>
>> >> If you repeated the experiments you did for the Accumulo only portion with HBase what would you expect the results to be?
>> >>
>> >> On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
>> >>> At the risk of derailing the original thread, I'll take a moment to
>> >>> explain my methodology. Since each entry can be thought of as a 6
>> >>> dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
>> >>> fiddling with the specifics of it. YCSB gives you several knobs, but
>> >>> unfortunately, not absolutely everything was tunable.
>> >>>
>> >>> The things that are configurable:
>> >>> - # of rows
>> >>> - # of column qualifiers
>> >>> - length of value
>> >>> - number of operations per client
>> >>> - number of clients
>> >>>
>> >>> Things that are not configurable:
>> >>> - row length (it's a long int)
>> >>> - # of column families (constant at one per row)
>> >>> - length of column qualifier (basically a one-up counter per row)
>> >>> - visibilities
>> >>>
>> >>> In all of my experiments, the goal was to keep data size constant.
>> >>> This can be approximated by (number of entries * entry size). Number
>> >>> of entries is intuitively rows (configurable) * column families (1) *
>> >>> columns qualifiers per family (configurable), while entry size is key
>> >>> overhead (about 40
>> >>> bytes) + configured length of value. So to keep total size constant,
>> >>> we have three easy knobs. However, tweaking three values at a time
>> >>> produces really messy data where you're not always going to be sure
>> >>> where the causality arrow lies. Even doing two at a time can cause
>> >>> issues but then the choice is between tweaking two properties of the
>> >>> data, or one property of the data and the total size (which is also a relevant attribute).
>> >>>
>> >>> Whew. So why did I use two different independent variables between the
>> >>> two halves?
>> >>>
>> >>> Partly, because I'm not comparing the two tests to each other, so they
>> >>> don't have to be duplicative. I ran them on different hardware from
>> >>> each other, with different number of clients, disks, cores, etc.
>> >>> There's no meaningful comparisons to be drawn, so I wanted to remove
>> >>> the temptation to compare results against each other. I'll admit that
>> >>> I might be wrong in this regard.
>> >>>
>> >>> The graphs are not my complete data sets. For the Accumulo v Accumulo
>> >>> tests, we have about ten more data points varying rows and data size
>> >>> as well. Trying to show three independent variables on a graph was
>> >>> pretty painful, so they didn't make it into the presentation. The
>> >>> short version of the story is that nothing scaled linearly (some
>> >>> things were better, some things were worse) but the general trend
>> >>> lines were approximately what you would expect.
>> >>>
>> >>> Let me know if you have more questions, but we can probably start a
>> >>> new thread for future search posterity! (This applies to everybody).
>> >>>
>> >>> Mike
>> >>>
>> >>>
>> >>> On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
>> >>> kepner@ll.mit.edu> wrote:
>> >>>
>> >>>> Mike Drob put together a great talk at the Accumulo Summit (
>> >>>> http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
>> >>>> Accumulo performance and HBase performance. This exactly the kind
>> >>>> of work the entire Hadoop community needs to continue to move forward.
>> >>>>
>> >>>> I had one question about the talk which I was wondering if someone
>> >>>> might be able to shed light on. In the Accumulo part of the talk
>> >>>> the experiments varied #rows while keeping the #cols fixed, while in
>> >>>> the Accumulo/HBase part of the the experiments varied #cols while keeping #rows fixed?
>
>
>
> --
> Jianshi Huang
>
> LinkedIn: jianshi
> Twitter: @jshuang
> Github & Blog: http://huangjs.github.com/
Re: How does Accumulo compare to HBase
Posted by Keith Turner <ke...@deenlo.com>.
Something else to be aware of, some of the Accumulo iterators will
optionally seek. For example the system iterator that filters column
families will read a few key/values when trying to find the next column
familiy and if it does not find the CF soon, it will eventually seek. This
can give dramatic performance improvements when there is a row with
millions of column families between the two you are interested in.
On Thu, Jul 17, 2014 at 2:48 AM, Jianshi Huang <ji...@gmail.com>
wrote:
> I finally decided to make the Storage part agnostic and wrote a HBase
> adapter as well.
>
> I'd like to say that Accumulo's API makes more sense than HBase'.
>
> One example is scan. In Accumulo, scan can take a Range which can take a
> startKey and endKey; a Key is a combination of RK+CF+CQ+CV+TS. This worked
> perfect and really free the CQ to a magic field in schema design.
>
> In HBase, scan can only take a startRow and an endRow... which both
> indicates the RK part. Scan through a range of Columns will require a
> ColumnRangeFilter AFAIK. (I heard HBase filter's performance sucks, not
> sure why).
>
> Any HBase expert disagrees? :)
>
> Jianshi
>
>
>
>
> On Fri, Jul 11, 2014 at 9:56 PM, Kepner, Jeremy - 0553 - MITLL <
> kepner@ll.mit.edu> wrote:
>
>> In many communities real applications can be hard to come by. These
>> communities often sponsor the development of application/technology
>> agnostic benchmarks as a surrogate. YCBS is one example. Graph500 is
>> another example. This allows a technology developer to focus on their own
>> technology and not have to be in the awkward position of having to also
>> benchmark multiple technologies simultaneously. It's not a perfect system,
>> but in general, a few mediocre benchmarks tend to be a lot better than no
>> benchmarks (too many benchmarks is also a problem). The benchmarks also
>> help with communicating the work because you can just reference the
>> benchmark.
>>
>> Now the tricky part is educating the customer base in how to interpret
>> the benchmarks. To a certain degree this is simply the burden of being an
>> informed consumer, but we can do as much as we can to help them. Using
>> standard benchmarks and showing how they correlate with some applications
>> and don't correlate with other applications is our obligation.
>>
>> On Jul 11, 2014, at 1:05 AM, Josh Elser <jo...@gmail.com> wrote:
>>
>> > It's important to remember that YCSB is a benchmark designed to test a
>> specific workload across database systems.
>> >
>> > No benchmark is going to be representative of every "real-life"
>> workload that can be thought of. Understand what the benchmark is showing
>> and draw realistic conclusions about what the results are showing you WRT
>> the problem(s) you care about.
>> >
>> > On 7/10/14, 11:33 AM, Chuck Adams wrote:
>> >> Dr. Kepner,
>> >>
>> >> Who cares how fast you can load data into a non-indexed HBase or
>> Accumulo database? What is the strategy to handle user queries against
>> this corpus? Run some real world tests which include simultaneous queries,
>> index maintenance, information life cycle management, during the initial
>> and incremental data loads.
>> >>
>> >> The test you are running do not appear to have any of the real world
>> bottlenecks that occur for production systems that users rely on for their
>> business.
>> >>
>> >> Respectfully,
>> >> Chuck Adams
>> >>
>> >> Vice President Technical Leadership Team
>> >> Oracle National Security Group
>> >> 1910 Oracle Way
>> >> Reston, VA 20190
>> >>
>> >> Cell: 301.529.9396
>> >> Email: chuck.adams@oracle.com
>> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Jeremy Kepner [mailto:kepner@ll.mit.edu]
>> >> Sent: Thursday, July 10, 2014 11:08 AM
>> >> To: user@accumulo.apache.org
>> >> Subject: Re: How does Accumulo compare to HBase
>> >>
>> >> If you repeated the experiments you did for the Accumulo only portion
>> with HBase what would you expect the results to be?
>> >>
>> >> On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
>> >>> At the risk of derailing the original thread, I'll take a moment to
>> >>> explain my methodology. Since each entry can be thought of as a 6
>> >>> dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
>> >>> fiddling with the specifics of it. YCSB gives you several knobs, but
>> >>> unfortunately, not absolutely everything was tunable.
>> >>>
>> >>> The things that are configurable:
>> >>> - # of rows
>> >>> - # of column qualifiers
>> >>> - length of value
>> >>> - number of operations per client
>> >>> - number of clients
>> >>>
>> >>> Things that are not configurable:
>> >>> - row length (it's a long int)
>> >>> - # of column families (constant at one per row)
>> >>> - length of column qualifier (basically a one-up counter per row)
>> >>> - visibilities
>> >>>
>> >>> In all of my experiments, the goal was to keep data size constant.
>> >>> This can be approximated by (number of entries * entry size). Number
>> >>> of entries is intuitively rows (configurable) * column families (1) *
>> >>> columns qualifiers per family (configurable), while entry size is key
>> >>> overhead (about 40
>> >>> bytes) + configured length of value. So to keep total size constant,
>> >>> we have three easy knobs. However, tweaking three values at a time
>> >>> produces really messy data where you're not always going to be sure
>> >>> where the causality arrow lies. Even doing two at a time can cause
>> >>> issues but then the choice is between tweaking two properties of the
>> >>> data, or one property of the data and the total size (which is also a
>> relevant attribute).
>> >>>
>> >>> Whew. So why did I use two different independent variables between the
>> >>> two halves?
>> >>>
>> >>> Partly, because I'm not comparing the two tests to each other, so they
>> >>> don't have to be duplicative. I ran them on different hardware from
>> >>> each other, with different number of clients, disks, cores, etc.
>> >>> There's no meaningful comparisons to be drawn, so I wanted to remove
>> >>> the temptation to compare results against each other. I'll admit that
>> >>> I might be wrong in this regard.
>> >>>
>> >>> The graphs are not my complete data sets. For the Accumulo v Accumulo
>> >>> tests, we have about ten more data points varying rows and data size
>> >>> as well. Trying to show three independent variables on a graph was
>> >>> pretty painful, so they didn't make it into the presentation. The
>> >>> short version of the story is that nothing scaled linearly (some
>> >>> things were better, some things were worse) but the general trend
>> >>> lines were approximately what you would expect.
>> >>>
>> >>> Let me know if you have more questions, but we can probably start a
>> >>> new thread for future search posterity! (This applies to everybody).
>> >>>
>> >>> Mike
>> >>>
>> >>>
>> >>> On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
>> >>> kepner@ll.mit.edu> wrote:
>> >>>
>> >>>> Mike Drob put together a great talk at the Accumulo Summit (
>> >>>> http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
>> >>>> Accumulo performance and HBase performance. This exactly the kind
>> >>>> of work the entire Hadoop community needs to continue to move
>> forward.
>> >>>>
>> >>>> I had one question about the talk which I was wondering if someone
>> >>>> might be able to shed light on. In the Accumulo part of the talk
>> >>>> the experiments varied #rows while keeping the #cols fixed, while in
>> >>>> the Accumulo/HBase part of the the experiments varied #cols while
>> keeping #rows fixed?
>>
>>
>
>
> --
> Jianshi Huang
>
> LinkedIn: jianshi
> Twitter: @jshuang
> Github & Blog: http://huangjs.github.com/
>
Re: How does Accumulo compare to HBase
Posted by Jianshi Huang <ji...@gmail.com>.
I finally decided to make the Storage part agnostic and wrote a HBase
adapter as well.
I'd like to say that Accumulo's API makes more sense than HBase'.
One example is scan. In Accumulo, scan can take a Range which can take a
startKey and endKey; a Key is a combination of RK+CF+CQ+CV+TS. This worked
perfect and really free the CQ to a magic field in schema design.
In HBase, scan can only take a startRow and an endRow... which both
indicates the RK part. Scan through a range of Columns will require a
ColumnRangeFilter AFAIK. (I heard HBase filter's performance sucks, not
sure why).
Any HBase expert disagrees? :)
Jianshi
On Fri, Jul 11, 2014 at 9:56 PM, Kepner, Jeremy - 0553 - MITLL <
kepner@ll.mit.edu> wrote:
> In many communities real applications can be hard to come by. These
> communities often sponsor the development of application/technology
> agnostic benchmarks as a surrogate. YCBS is one example. Graph500 is
> another example. This allows a technology developer to focus on their own
> technology and not have to be in the awkward position of having to also
> benchmark multiple technologies simultaneously. It's not a perfect system,
> but in general, a few mediocre benchmarks tend to be a lot better than no
> benchmarks (too many benchmarks is also a problem). The benchmarks also
> help with communicating the work because you can just reference the
> benchmark.
>
> Now the tricky part is educating the customer base in how to interpret the
> benchmarks. To a certain degree this is simply the burden of being an
> informed consumer, but we can do as much as we can to help them. Using
> standard benchmarks and showing how they correlate with some applications
> and don't correlate with other applications is our obligation.
>
> On Jul 11, 2014, at 1:05 AM, Josh Elser <jo...@gmail.com> wrote:
>
> > It's important to remember that YCSB is a benchmark designed to test a
> specific workload across database systems.
> >
> > No benchmark is going to be representative of every "real-life" workload
> that can be thought of. Understand what the benchmark is showing and draw
> realistic conclusions about what the results are showing you WRT the
> problem(s) you care about.
> >
> > On 7/10/14, 11:33 AM, Chuck Adams wrote:
> >> Dr. Kepner,
> >>
> >> Who cares how fast you can load data into a non-indexed HBase or
> Accumulo database? What is the strategy to handle user queries against
> this corpus? Run some real world tests which include simultaneous queries,
> index maintenance, information life cycle management, during the initial
> and incremental data loads.
> >>
> >> The test you are running do not appear to have any of the real world
> bottlenecks that occur for production systems that users rely on for their
> business.
> >>
> >> Respectfully,
> >> Chuck Adams
> >>
> >> Vice President Technical Leadership Team
> >> Oracle National Security Group
> >> 1910 Oracle Way
> >> Reston, VA 20190
> >>
> >> Cell: 301.529.9396
> >> Email: chuck.adams@oracle.com
> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >>
> >>
> >> -----Original Message-----
> >> From: Jeremy Kepner [mailto:kepner@ll.mit.edu]
> >> Sent: Thursday, July 10, 2014 11:08 AM
> >> To: user@accumulo.apache.org
> >> Subject: Re: How does Accumulo compare to HBase
> >>
> >> If you repeated the experiments you did for the Accumulo only portion
> with HBase what would you expect the results to be?
> >>
> >> On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
> >>> At the risk of derailing the original thread, I'll take a moment to
> >>> explain my methodology. Since each entry can be thought of as a 6
> >>> dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
> >>> fiddling with the specifics of it. YCSB gives you several knobs, but
> >>> unfortunately, not absolutely everything was tunable.
> >>>
> >>> The things that are configurable:
> >>> - # of rows
> >>> - # of column qualifiers
> >>> - length of value
> >>> - number of operations per client
> >>> - number of clients
> >>>
> >>> Things that are not configurable:
> >>> - row length (it's a long int)
> >>> - # of column families (constant at one per row)
> >>> - length of column qualifier (basically a one-up counter per row)
> >>> - visibilities
> >>>
> >>> In all of my experiments, the goal was to keep data size constant.
> >>> This can be approximated by (number of entries * entry size). Number
> >>> of entries is intuitively rows (configurable) * column families (1) *
> >>> columns qualifiers per family (configurable), while entry size is key
> >>> overhead (about 40
> >>> bytes) + configured length of value. So to keep total size constant,
> >>> we have three easy knobs. However, tweaking three values at a time
> >>> produces really messy data where you're not always going to be sure
> >>> where the causality arrow lies. Even doing two at a time can cause
> >>> issues but then the choice is between tweaking two properties of the
> >>> data, or one property of the data and the total size (which is also a
> relevant attribute).
> >>>
> >>> Whew. So why did I use two different independent variables between the
> >>> two halves?
> >>>
> >>> Partly, because I'm not comparing the two tests to each other, so they
> >>> don't have to be duplicative. I ran them on different hardware from
> >>> each other, with different number of clients, disks, cores, etc.
> >>> There's no meaningful comparisons to be drawn, so I wanted to remove
> >>> the temptation to compare results against each other. I'll admit that
> >>> I might be wrong in this regard.
> >>>
> >>> The graphs are not my complete data sets. For the Accumulo v Accumulo
> >>> tests, we have about ten more data points varying rows and data size
> >>> as well. Trying to show three independent variables on a graph was
> >>> pretty painful, so they didn't make it into the presentation. The
> >>> short version of the story is that nothing scaled linearly (some
> >>> things were better, some things were worse) but the general trend
> >>> lines were approximately what you would expect.
> >>>
> >>> Let me know if you have more questions, but we can probably start a
> >>> new thread for future search posterity! (This applies to everybody).
> >>>
> >>> Mike
> >>>
> >>>
> >>> On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
> >>> kepner@ll.mit.edu> wrote:
> >>>
> >>>> Mike Drob put together a great talk at the Accumulo Summit (
> >>>> http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
> >>>> Accumulo performance and HBase performance. This exactly the kind
> >>>> of work the entire Hadoop community needs to continue to move forward.
> >>>>
> >>>> I had one question about the talk which I was wondering if someone
> >>>> might be able to shed light on. In the Accumulo part of the talk
> >>>> the experiments varied #rows while keeping the #cols fixed, while in
> >>>> the Accumulo/HBase part of the the experiments varied #cols while
> keeping #rows fixed?
>
>
--
Jianshi Huang
LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/
Re: How does Accumulo compare to HBase
Posted by "Kepner, Jeremy - 0553 - MITLL" <ke...@ll.mit.edu>.
In many communities real applications can be hard to come by. These communities often sponsor the development of application/technology agnostic benchmarks as a surrogate. YCBS is one example. Graph500 is another example. This allows a technology developer to focus on their own technology and not have to be in the awkward position of having to also benchmark multiple technologies simultaneously. It's not a perfect system, but in general, a few mediocre benchmarks tend to be a lot better than no benchmarks (too many benchmarks is also a problem). The benchmarks also help with communicating the work because you can just reference the benchmark.
Now the tricky part is educating the customer base in how to interpret the benchmarks. To a certain degree this is simply the burden of being an informed consumer, but we can do as much as we can to help them. Using standard benchmarks and showing how they correlate with some applications and don't correlate with other applications is our obligation.
On Jul 11, 2014, at 1:05 AM, Josh Elser <jo...@gmail.com> wrote:
> It's important to remember that YCSB is a benchmark designed to test a specific workload across database systems.
>
> No benchmark is going to be representative of every "real-life" workload that can be thought of. Understand what the benchmark is showing and draw realistic conclusions about what the results are showing you WRT the problem(s) you care about.
>
> On 7/10/14, 11:33 AM, Chuck Adams wrote:
>> Dr. Kepner,
>>
>> Who cares how fast you can load data into a non-indexed HBase or Accumulo database? What is the strategy to handle user queries against this corpus? Run some real world tests which include simultaneous queries, index maintenance, information life cycle management, during the initial and incremental data loads.
>>
>> The test you are running do not appear to have any of the real world bottlenecks that occur for production systems that users rely on for their business.
>>
>> Respectfully,
>> Chuck Adams
>>
>> Vice President Technical Leadership Team
>> Oracle National Security Group
>> 1910 Oracle Way
>> Reston, VA 20190
>>
>> Cell: 301.529.9396
>> Email: chuck.adams@oracle.com
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>
>> -----Original Message-----
>> From: Jeremy Kepner [mailto:kepner@ll.mit.edu]
>> Sent: Thursday, July 10, 2014 11:08 AM
>> To: user@accumulo.apache.org
>> Subject: Re: How does Accumulo compare to HBase
>>
>> If you repeated the experiments you did for the Accumulo only portion with HBase what would you expect the results to be?
>>
>> On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
>>> At the risk of derailing the original thread, I'll take a moment to
>>> explain my methodology. Since each entry can be thought of as a 6
>>> dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
>>> fiddling with the specifics of it. YCSB gives you several knobs, but
>>> unfortunately, not absolutely everything was tunable.
>>>
>>> The things that are configurable:
>>> - # of rows
>>> - # of column qualifiers
>>> - length of value
>>> - number of operations per client
>>> - number of clients
>>>
>>> Things that are not configurable:
>>> - row length (it's a long int)
>>> - # of column families (constant at one per row)
>>> - length of column qualifier (basically a one-up counter per row)
>>> - visibilities
>>>
>>> In all of my experiments, the goal was to keep data size constant.
>>> This can be approximated by (number of entries * entry size). Number
>>> of entries is intuitively rows (configurable) * column families (1) *
>>> columns qualifiers per family (configurable), while entry size is key
>>> overhead (about 40
>>> bytes) + configured length of value. So to keep total size constant,
>>> we have three easy knobs. However, tweaking three values at a time
>>> produces really messy data where you're not always going to be sure
>>> where the causality arrow lies. Even doing two at a time can cause
>>> issues but then the choice is between tweaking two properties of the
>>> data, or one property of the data and the total size (which is also a relevant attribute).
>>>
>>> Whew. So why did I use two different independent variables between the
>>> two halves?
>>>
>>> Partly, because I'm not comparing the two tests to each other, so they
>>> don't have to be duplicative. I ran them on different hardware from
>>> each other, with different number of clients, disks, cores, etc.
>>> There's no meaningful comparisons to be drawn, so I wanted to remove
>>> the temptation to compare results against each other. I'll admit that
>>> I might be wrong in this regard.
>>>
>>> The graphs are not my complete data sets. For the Accumulo v Accumulo
>>> tests, we have about ten more data points varying rows and data size
>>> as well. Trying to show three independent variables on a graph was
>>> pretty painful, so they didn't make it into the presentation. The
>>> short version of the story is that nothing scaled linearly (some
>>> things were better, some things were worse) but the general trend
>>> lines were approximately what you would expect.
>>>
>>> Let me know if you have more questions, but we can probably start a
>>> new thread for future search posterity! (This applies to everybody).
>>>
>>> Mike
>>>
>>>
>>> On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
>>> kepner@ll.mit.edu> wrote:
>>>
>>>> Mike Drob put together a great talk at the Accumulo Summit (
>>>> http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
>>>> Accumulo performance and HBase performance. This exactly the kind
>>>> of work the entire Hadoop community needs to continue to move forward.
>>>>
>>>> I had one question about the talk which I was wondering if someone
>>>> might be able to shed light on. In the Accumulo part of the talk
>>>> the experiments varied #rows while keeping the #cols fixed, while in
>>>> the Accumulo/HBase part of the the experiments varied #cols while keeping #rows fixed?
Re: How does Accumulo compare to HBase
Posted by Josh Elser <jo...@gmail.com>.
It's important to remember that YCSB is a benchmark designed to test a
specific workload across database systems.
No benchmark is going to be representative of every "real-life" workload
that can be thought of. Understand what the benchmark is showing and
draw realistic conclusions about what the results are showing you WRT
the problem(s) you care about.
On 7/10/14, 11:33 AM, Chuck Adams wrote:
> Dr. Kepner,
>
> Who cares how fast you can load data into a non-indexed HBase or Accumulo database? What is the strategy to handle user queries against this corpus? Run some real world tests which include simultaneous queries, index maintenance, information life cycle management, during the initial and incremental data loads.
>
> The test you are running do not appear to have any of the real world bottlenecks that occur for production systems that users rely on for their business.
>
> Respectfully,
> Chuck Adams
>
> Vice President Technical Leadership Team
> Oracle National Security Group
> 1910 Oracle Way
> Reston, VA 20190
>
> Cell: 301.529.9396
> Email: chuck.adams@oracle.com
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
> -----Original Message-----
> From: Jeremy Kepner [mailto:kepner@ll.mit.edu]
> Sent: Thursday, July 10, 2014 11:08 AM
> To: user@accumulo.apache.org
> Subject: Re: How does Accumulo compare to HBase
>
> If you repeated the experiments you did for the Accumulo only portion with HBase what would you expect the results to be?
>
> On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
>> At the risk of derailing the original thread, I'll take a moment to
>> explain my methodology. Since each entry can be thought of as a 6
>> dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
>> fiddling with the specifics of it. YCSB gives you several knobs, but
>> unfortunately, not absolutely everything was tunable.
>>
>> The things that are configurable:
>> - # of rows
>> - # of column qualifiers
>> - length of value
>> - number of operations per client
>> - number of clients
>>
>> Things that are not configurable:
>> - row length (it's a long int)
>> - # of column families (constant at one per row)
>> - length of column qualifier (basically a one-up counter per row)
>> - visibilities
>>
>> In all of my experiments, the goal was to keep data size constant.
>> This can be approximated by (number of entries * entry size). Number
>> of entries is intuitively rows (configurable) * column families (1) *
>> columns qualifiers per family (configurable), while entry size is key
>> overhead (about 40
>> bytes) + configured length of value. So to keep total size constant,
>> we have three easy knobs. However, tweaking three values at a time
>> produces really messy data where you're not always going to be sure
>> where the causality arrow lies. Even doing two at a time can cause
>> issues but then the choice is between tweaking two properties of the
>> data, or one property of the data and the total size (which is also a relevant attribute).
>>
>> Whew. So why did I use two different independent variables between the
>> two halves?
>>
>> Partly, because I'm not comparing the two tests to each other, so they
>> don't have to be duplicative. I ran them on different hardware from
>> each other, with different number of clients, disks, cores, etc.
>> There's no meaningful comparisons to be drawn, so I wanted to remove
>> the temptation to compare results against each other. I'll admit that
>> I might be wrong in this regard.
>>
>> The graphs are not my complete data sets. For the Accumulo v Accumulo
>> tests, we have about ten more data points varying rows and data size
>> as well. Trying to show three independent variables on a graph was
>> pretty painful, so they didn't make it into the presentation. The
>> short version of the story is that nothing scaled linearly (some
>> things were better, some things were worse) but the general trend
>> lines were approximately what you would expect.
>>
>> Let me know if you have more questions, but we can probably start a
>> new thread for future search posterity! (This applies to everybody).
>>
>> Mike
>>
>>
>> On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
>> kepner@ll.mit.edu> wrote:
>>
>>> Mike Drob put together a great talk at the Accumulo Summit (
>>> http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
>>> Accumulo performance and HBase performance. This exactly the kind
>>> of work the entire Hadoop community needs to continue to move forward.
>>>
>>> I had one question about the talk which I was wondering if someone
>>> might be able to shed light on. In the Accumulo part of the talk
>>> the experiments varied #rows while keeping the #cols fixed, while in
>>> the Accumulo/HBase part of the the experiments varied #cols while keeping #rows fixed?
Re: How does Accumulo compare to HBase
Posted by Arshak Navruzyan <ar...@gmail.com>.
Chuck,
With regard to "the strategy to handle user queries against the corpus," we
have implementing the secondary indexing concepts of D4M
<http://www.mit.edu/~kepner/pubs/D4Mschema_HPEC2013_Paper.pdf> and have
introduced distributed SQL queries on top of Accumulo. Here is a
presentation for the recent Accumulo summit:
http://www.slideshare.net/AccumuloSummit/14-25-navruzyan
Jeremy's research and benchmarks were instrumental to our efforts.
Arshak Navruzyan | VP Product | Argyle Data, Inc.
On Thu, Jul 10, 2014 at 8:33 AM, Chuck Adams <ch...@oracle.com> wrote:
> Dr. Kepner,
>
> Who cares how fast you can load data into a non-indexed HBase or Accumulo
> database? What is the strategy to handle user queries against this corpus?
> Run some real world tests which include simultaneous queries, index
> maintenance, information life cycle management, during the initial and
> incremental data loads.
>
> The test you are running do not appear to have any of the real world
> bottlenecks that occur for production systems that users rely on for their
> business.
>
> Respectfully,
> Chuck Adams
>
> Vice President Technical Leadership Team
> Oracle National Security Group
> 1910 Oracle Way
> Reston, VA 20190
>
> Cell: 301.529.9396
> Email: chuck.adams@oracle.com
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
> -----Original Message-----
> From: Jeremy Kepner [mailto:kepner@ll.mit.edu]
> Sent: Thursday, July 10, 2014 11:08 AM
> To: user@accumulo.apache.org
> Subject: Re: How does Accumulo compare to HBase
>
> If you repeated the experiments you did for the Accumulo only portion with
> HBase what would you expect the results to be?
>
> On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
> > At the risk of derailing the original thread, I'll take a moment to
> > explain my methodology. Since each entry can be thought of as a 6
> > dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
> > fiddling with the specifics of it. YCSB gives you several knobs, but
> > unfortunately, not absolutely everything was tunable.
> >
> > The things that are configurable:
> > - # of rows
> > - # of column qualifiers
> > - length of value
> > - number of operations per client
> > - number of clients
> >
> > Things that are not configurable:
> > - row length (it's a long int)
> > - # of column families (constant at one per row)
> > - length of column qualifier (basically a one-up counter per row)
> > - visibilities
> >
> > In all of my experiments, the goal was to keep data size constant.
> > This can be approximated by (number of entries * entry size). Number
> > of entries is intuitively rows (configurable) * column families (1) *
> > columns qualifiers per family (configurable), while entry size is key
> > overhead (about 40
> > bytes) + configured length of value. So to keep total size constant,
> > we have three easy knobs. However, tweaking three values at a time
> > produces really messy data where you're not always going to be sure
> > where the causality arrow lies. Even doing two at a time can cause
> > issues but then the choice is between tweaking two properties of the
> > data, or one property of the data and the total size (which is also a
> relevant attribute).
> >
> > Whew. So why did I use two different independent variables between the
> > two halves?
> >
> > Partly, because I'm not comparing the two tests to each other, so they
> > don't have to be duplicative. I ran them on different hardware from
> > each other, with different number of clients, disks, cores, etc.
> > There's no meaningful comparisons to be drawn, so I wanted to remove
> > the temptation to compare results against each other. I'll admit that
> > I might be wrong in this regard.
> >
> > The graphs are not my complete data sets. For the Accumulo v Accumulo
> > tests, we have about ten more data points varying rows and data size
> > as well. Trying to show three independent variables on a graph was
> > pretty painful, so they didn't make it into the presentation. The
> > short version of the story is that nothing scaled linearly (some
> > things were better, some things were worse) but the general trend
> > lines were approximately what you would expect.
> >
> > Let me know if you have more questions, but we can probably start a
> > new thread for future search posterity! (This applies to everybody).
> >
> > Mike
> >
> >
> > On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
> > kepner@ll.mit.edu> wrote:
> >
> > > Mike Drob put together a great talk at the Accumulo Summit (
> > > http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
> > > Accumulo performance and HBase performance. This exactly the kind
> > > of work the entire Hadoop community needs to continue to move forward.
> > >
> > > I had one question about the talk which I was wondering if someone
> > > might be able to shed light on. In the Accumulo part of the talk
> > > the experiments varied #rows while keeping the #cols fixed, while in
> > > the Accumulo/HBase part of the the experiments varied #cols while
> keeping #rows fixed?
>
RE: How does Accumulo compare to HBase
Posted by Chuck Adams <ch...@oracle.com>.
Dr. Kepner,
Who cares how fast you can load data into a non-indexed HBase or Accumulo database? What is the strategy to handle user queries against this corpus? Run some real world tests which include simultaneous queries, index maintenance, information life cycle management, during the initial and incremental data loads.
The test you are running do not appear to have any of the real world bottlenecks that occur for production systems that users rely on for their business.
Respectfully,
Chuck Adams
Vice President Technical Leadership Team
Oracle National Security Group
1910 Oracle Way
Reston, VA 20190
Cell: 301.529.9396
Email: chuck.adams@oracle.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----Original Message-----
From: Jeremy Kepner [mailto:kepner@ll.mit.edu]
Sent: Thursday, July 10, 2014 11:08 AM
To: user@accumulo.apache.org
Subject: Re: How does Accumulo compare to HBase
If you repeated the experiments you did for the Accumulo only portion with HBase what would you expect the results to be?
On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
> At the risk of derailing the original thread, I'll take a moment to
> explain my methodology. Since each entry can be thought of as a 6
> dimensional vector (r, cf, cq, vis, ts, val) there's a lot of room for
> fiddling with the specifics of it. YCSB gives you several knobs, but
> unfortunately, not absolutely everything was tunable.
>
> The things that are configurable:
> - # of rows
> - # of column qualifiers
> - length of value
> - number of operations per client
> - number of clients
>
> Things that are not configurable:
> - row length (it's a long int)
> - # of column families (constant at one per row)
> - length of column qualifier (basically a one-up counter per row)
> - visibilities
>
> In all of my experiments, the goal was to keep data size constant.
> This can be approximated by (number of entries * entry size). Number
> of entries is intuitively rows (configurable) * column families (1) *
> columns qualifiers per family (configurable), while entry size is key
> overhead (about 40
> bytes) + configured length of value. So to keep total size constant,
> we have three easy knobs. However, tweaking three values at a time
> produces really messy data where you're not always going to be sure
> where the causality arrow lies. Even doing two at a time can cause
> issues but then the choice is between tweaking two properties of the
> data, or one property of the data and the total size (which is also a relevant attribute).
>
> Whew. So why did I use two different independent variables between the
> two halves?
>
> Partly, because I'm not comparing the two tests to each other, so they
> don't have to be duplicative. I ran them on different hardware from
> each other, with different number of clients, disks, cores, etc.
> There's no meaningful comparisons to be drawn, so I wanted to remove
> the temptation to compare results against each other. I'll admit that
> I might be wrong in this regard.
>
> The graphs are not my complete data sets. For the Accumulo v Accumulo
> tests, we have about ten more data points varying rows and data size
> as well. Trying to show three independent variables on a graph was
> pretty painful, so they didn't make it into the presentation. The
> short version of the story is that nothing scaled linearly (some
> things were better, some things were worse) but the general trend
> lines were approximately what you would expect.
>
> Let me know if you have more questions, but we can probably start a
> new thread for future search posterity! (This applies to everybody).
>
> Mike
>
>
> On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
> kepner@ll.mit.edu> wrote:
>
> > Mike Drob put together a great talk at the Accumulo Summit (
> > http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing
> > Accumulo performance and HBase performance. This exactly the kind
> > of work the entire Hadoop community needs to continue to move forward.
> >
> > I had one question about the talk which I was wondering if someone
> > might be able to shed light on. In the Accumulo part of the talk
> > the experiments varied #rows while keeping the #cols fixed, while in
> > the Accumulo/HBase part of the the experiments varied #cols while keeping #rows fixed?
Re: How does Accumulo compare to HBase
Posted by Jeremy Kepner <ke...@ll.mit.edu>.
If you repeated the experiments you did for the Accumulo only portion with HBase what would you expect the results to be?
On Thu, Jul 10, 2014 at 10:02:00AM -0500, Mike Drob wrote:
> At the risk of derailing the original thread, I'll take a moment to explain
> my methodology. Since each entry can be thought of as a 6 dimensional
> vector (r, cf, cq, vis, ts, val) there's a lot of room for fiddling with
> the specifics of it. YCSB gives you several knobs, but unfortunately, not
> absolutely everything was tunable.
>
> The things that are configurable:
> - # of rows
> - # of column qualifiers
> - length of value
> - number of operations per client
> - number of clients
>
> Things that are not configurable:
> - row length (it's a long int)
> - # of column families (constant at one per row)
> - length of column qualifier (basically a one-up counter per row)
> - visibilities
>
> In all of my experiments, the goal was to keep data size constant. This can
> be approximated by (number of entries * entry size). Number of entries is
> intuitively rows (configurable) * column families (1) * columns qualifiers
> per family (configurable), while entry size is key overhead (about 40
> bytes) + configured length of value. So to keep total size constant, we
> have three easy knobs. However, tweaking three values at a time produces
> really messy data where you're not always going to be sure where the
> causality arrow lies. Even doing two at a time can cause issues but then
> the choice is between tweaking two properties of the data, or one property
> of the data and the total size (which is also a relevant attribute).
>
> Whew. So why did I use two different independent variables between the two
> halves?
>
> Partly, because I'm not comparing the two tests to each other, so they
> don't have to be duplicative. I ran them on different hardware from each
> other, with different number of clients, disks, cores, etc. There's no
> meaningful comparisons to be drawn, so I wanted to remove the temptation to
> compare results against each other. I'll admit that I might be wrong in
> this regard.
>
> The graphs are not my complete data sets. For the Accumulo v Accumulo
> tests, we have about ten more data points varying rows and data size as
> well. Trying to show three independent variables on a graph was pretty
> painful, so they didn't make it into the presentation. The short version of
> the story is that nothing scaled linearly (some things were better, some
> things were worse) but the general trend lines were approximately what you
> would expect.
>
> Let me know if you have more questions, but we can probably start a new
> thread for future search posterity! (This applies to everybody).
>
> Mike
>
>
> On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
> kepner@ll.mit.edu> wrote:
>
> > Mike Drob put together a great talk at the Accumulo Summit (
> > http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing Accumulo
> > performance and HBase performance. This exactly the kind of work the
> > entire Hadoop community needs to continue to move forward.
> >
> > I had one question about the talk which I was wondering if someone might
> > be able to shed light on. In the Accumulo part of the talk the experiments
> > varied #rows while keeping the #cols fixed, while in the Accumulo/HBase
> > part of the the experiments varied #cols while keeping #rows fixed?
Re: How does Accumulo compare to HBase
Posted by Mike Drob <md...@apache.org>.
At the risk of derailing the original thread, I'll take a moment to explain
my methodology. Since each entry can be thought of as a 6 dimensional
vector (r, cf, cq, vis, ts, val) there's a lot of room for fiddling with
the specifics of it. YCSB gives you several knobs, but unfortunately, not
absolutely everything was tunable.
The things that are configurable:
- # of rows
- # of column qualifiers
- length of value
- number of operations per client
- number of clients
Things that are not configurable:
- row length (it's a long int)
- # of column families (constant at one per row)
- length of column qualifier (basically a one-up counter per row)
- visibilities
In all of my experiments, the goal was to keep data size constant. This can
be approximated by (number of entries * entry size). Number of entries is
intuitively rows (configurable) * column families (1) * columns qualifiers
per family (configurable), while entry size is key overhead (about 40
bytes) + configured length of value. So to keep total size constant, we
have three easy knobs. However, tweaking three values at a time produces
really messy data where you're not always going to be sure where the
causality arrow lies. Even doing two at a time can cause issues but then
the choice is between tweaking two properties of the data, or one property
of the data and the total size (which is also a relevant attribute).
Whew. So why did I use two different independent variables between the two
halves?
Partly, because I'm not comparing the two tests to each other, so they
don't have to be duplicative. I ran them on different hardware from each
other, with different number of clients, disks, cores, etc. There's no
meaningful comparisons to be drawn, so I wanted to remove the temptation to
compare results against each other. I'll admit that I might be wrong in
this regard.
The graphs are not my complete data sets. For the Accumulo v Accumulo
tests, we have about ten more data points varying rows and data size as
well. Trying to show three independent variables on a graph was pretty
painful, so they didn't make it into the presentation. The short version of
the story is that nothing scaled linearly (some things were better, some
things were worse) but the general trend lines were approximately what you
would expect.
Let me know if you have more questions, but we can probably start a new
thread for future search posterity! (This applies to everybody).
Mike
On Thu, Jul 10, 2014 at 9:26 AM, Kepner, Jeremy - 0553 - MITLL <
kepner@ll.mit.edu> wrote:
> Mike Drob put together a great talk at the Accumulo Summit (
> http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing Accumulo
> performance and HBase performance. This exactly the kind of work the
> entire Hadoop community needs to continue to move forward.
>
> I had one question about the talk which I was wondering if someone might
> be able to shed light on. In the Accumulo part of the talk the experiments
> varied #rows while keeping the #cols fixed, while in the Accumulo/HBase
> part of the the experiments varied #cols while keeping #rows fixed?
Re: How does Accumulo compare to HBase
Posted by "Kepner, Jeremy - 0553 - MITLL" <ke...@ll.mit.edu>.
Mike Drob put together a great talk at the Accumulo Summit (http://www.slideshare.net/AccumuloSummit/10-30-drob) discussing Accumulo performance and HBase performance. This exactly the kind of work the entire Hadoop community needs to continue to move forward.
I had one question about the talk which I was wondering if someone might be able to shed light on. In the Accumulo part of the talk the experiments varied #rows while keeping the #cols fixed, while in the Accumulo/HBase part of the the experiments varied #cols while keeping #rows fixed?