You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Tanakorn Leesatapornwongsa <ta...@cs.uchicago.edu> on 2017/02/28 04:25:54 UTC
Scalability Bugs in HBase
Dear HBase team,
We are CS researchers at the University of Chicago. In our recent
research, we are building a framework to find and replay "scalability
bugs" with just one machine (scalability bugs are bugs that only surface
in large-scale deployments such as in hundreds of nodes, but do not appear
in small/medium-scale deployments).
We attach our paper about the problem, if you are interested in reading
more.
In this email, we would like to ask if any of you would like to
collaborate by pointing us to any new scalability bugs (JIRA entry) that
you are aware of.
We ourselves had manually scan each bug entry in JIRA, but we might miss
some of the new ones.
Thank you in advance for your time!
Sincerely,
Tanakorn Leesatapornwongsa and Prof. Haryadi Gunawi
http://ucare.cs.uchicago.edu
Re: Scalability Bugs in HBase
Posted by Ted Yu <yu...@gmail.com>.
I noticed that a few of the bugs below are still open.
If you clone hbase git repo, you can find fixed bugs w.r.t.
StochasticLoadBalancer by examining git history of
./hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
FYI
On Mon, Feb 27, 2017 at 8:52 PM, Tanakorn Leesatapornwongsa <
tanakorn@cs.uchicago.edu> wrote:
> Sure, here is the list of HBase bugs,
>
> https://issues.apache.org/jira/browse/HBASE-3620
> https://issues.apache.org/jira/browse/HBASE-8778
> https://issues.apache.org/jira/browse/HBASE-7190
> https://issues.apache.org/jira/browse/HBASE-10209
> https://issues.apache.org/jira/browse/HBASE-12139
> https://issues.apache.org/jira/browse/HBASE-9208
> https://issues.apache.org/jira/browse/HBASE-9377
> https://issues.apache.org/jira/browse/HBASE-9775
> https://issues.apache.org/jira/browse/HBASE-7060
>
> Best,
> Tanakorn L.
>
>
>
> On Feb 27, 2017, at 10:43 PM, Ted Yu <yu...@gmail.com> wrote:
>
> The references only listed C* bugs.
>
> Can you tell us which 9 hbase bugs you have analyzed ?
>
> Thanks
>
> On Mon, Feb 27, 2017 at 8:25 PM, Tanakorn Leesatapornwongsa <
> tanakorn@cs.uchicago.edu> wrote:
>
>> Dear HBase team,
>>
>> We are CS researchers at the University of Chicago. In our recent
>> research, we are building a framework to find and replay "scalability
>> bugs" with just one machine (scalability bugs are bugs that only surface
>> in large-scale deployments such as in hundreds of nodes, but do not appear
>> in small/medium-scale deployments).
>>
>> We attach our paper about the problem, if you are interested in reading
>> more.
>>
>> In this email, we would like to ask if any of you would like to
>> collaborate by pointing us to any new scalability bugs (JIRA entry) that
>> you are aware of.
>>
>> We ourselves had manually scan each bug entry in JIRA, but we might miss
>> some of the new ones.
>>
>> Thank you in advance for your time!
>>
>> Sincerely,
>> Tanakorn Leesatapornwongsa and Prof. Haryadi Gunawi
>> http://ucare.cs.uchicago.edu
>>
>>
>>
>
>
Re: Scalability Bugs in HBase
Posted by Tanakorn Leesatapornwongsa <ta...@cs.uchicago.edu>.
Sure, here is the list of HBase bugs,
https://issues.apache.org/jira/browse/HBASE-3620 <https://issues.apache.org/jira/browse/HBASE-3620>
https://issues.apache.org/jira/browse/HBASE-8778 <https://issues.apache.org/jira/browse/HBASE-8778>
https://issues.apache.org/jira/browse/HBASE-7190 <https://issues.apache.org/jira/browse/HBASE-7190>
https://issues.apache.org/jira/browse/HBASE-10209 <https://issues.apache.org/jira/browse/HBASE-10209>
https://issues.apache.org/jira/browse/HBASE-12139 <https://issues.apache.org/jira/browse/HBASE-12139>
https://issues.apache.org/jira/browse/HBASE-9208 <https://issues.apache.org/jira/browse/HBASE-9208>
https://issues.apache.org/jira/browse/HBASE-9377 <https://issues.apache.org/jira/browse/HBASE-9377>
https://issues.apache.org/jira/browse/HBASE-9775 <https://issues.apache.org/jira/browse/HBASE-9775>
https://issues.apache.org/jira/browse/HBASE-7060 <https://issues.apache.org/jira/browse/HBASE-7060>
Best,
Tanakorn L.
> On Feb 27, 2017, at 10:43 PM, Ted Yu <yu...@gmail.com> wrote:
>
> The references only listed C* bugs.
>
> Can you tell us which 9 hbase bugs you have analyzed ?
>
> Thanks
>
> On Mon, Feb 27, 2017 at 8:25 PM, Tanakorn Leesatapornwongsa <tanakorn@cs.uchicago.edu <ma...@cs.uchicago.edu>> wrote:
> Dear HBase team,
>
> We are CS researchers at the University of Chicago. In our recent
> research, we are building a framework to find and replay "scalability
> bugs" with just one machine (scalability bugs are bugs that only surface
> in large-scale deployments such as in hundreds of nodes, but do not appear
> in small/medium-scale deployments).
>
> We attach our paper about the problem, if you are interested in reading
> more.
>
> In this email, we would like to ask if any of you would like to
> collaborate by pointing us to any new scalability bugs (JIRA entry) that
> you are aware of.
>
> We ourselves had manually scan each bug entry in JIRA, but we might miss
> some of the new ones.
>
> Thank you in advance for your time!
>
> Sincerely,
> Tanakorn Leesatapornwongsa and Prof. Haryadi Gunawi
> http://ucare.cs.uchicago.edu <http://ucare.cs.uchicago.edu/>
>
>
>
Re: Scalability Bugs in HBase
Posted by Ted Yu <yu...@gmail.com>.
The references only listed C* bugs.
Can you tell us which 9 hbase bugs you have analyzed ?
Thanks
On Mon, Feb 27, 2017 at 8:25 PM, Tanakorn Leesatapornwongsa <
tanakorn@cs.uchicago.edu> wrote:
> Dear HBase team,
>
> We are CS researchers at the University of Chicago. In our recent
> research, we are building a framework to find and replay "scalability
> bugs" with just one machine (scalability bugs are bugs that only surface
> in large-scale deployments such as in hundreds of nodes, but do not appear
> in small/medium-scale deployments).
>
> We attach our paper about the problem, if you are interested in reading
> more.
>
> In this email, we would like to ask if any of you would like to
> collaborate by pointing us to any new scalability bugs (JIRA entry) that
> you are aware of.
>
> We ourselves had manually scan each bug entry in JIRA, but we might miss
> some of the new ones.
>
> Thank you in advance for your time!
>
> Sincerely,
> Tanakorn Leesatapornwongsa and Prof. Haryadi Gunawi
> http://ucare.cs.uchicago.edu
>
>
>
Re: Scalability Bugs in HBase
Posted by Tanakorn Leesatapornwongsa <ta...@cs.uchicago.edu>.
I think the mailing list does not allow attachment, so I attach it here as a link to the paper.
http://people.cs.uchicago.edu/~tanakorn/scale.pdf <http://people.cs.uchicago.edu/~tanakorn/scale.pdf>
Best,
Tanakorn L.
> On Feb 27, 2017, at 10:25 PM, Tanakorn Leesatapornwongsa <ta...@cs.uchicago.edu> wrote:
>
> Dear HBase team,
>
> We are CS researchers at the University of Chicago. In our recent
> research, we are building a framework to find and replay "scalability
> bugs" with just one machine (scalability bugs are bugs that only surface
> in large-scale deployments such as in hundreds of nodes, but do not appear
> in small/medium-scale deployments).
>
> We attach our paper about the problem, if you are interested in reading
> more.
>
> In this email, we would like to ask if any of you would like to
> collaborate by pointing us to any new scalability bugs (JIRA entry) that
> you are aware of.
>
> We ourselves had manually scan each bug entry in JIRA, but we might miss
> some of the new ones.
>
> Thank you in advance for your time!
>
> Sincerely,
> Tanakorn Leesatapornwongsa and Prof. Haryadi Gunawi
> http://ucare.cs.uchicago.edu
>
>
Re: Scalability Bugs in HBase
Posted by Tanakorn Leesatapornwongsa <ta...@cs.uchicago.edu>.
Hi Ted,
Thank you for the pointer to the bugs. I will look into it.
For the feature that you ask, we estimate that it will take 2 - 3 months to finish.
We will keep you posted.
Thank you again!
Best,
Tanakorn L.
> On Feb 28, 2017, at 10:42 AM, Ted Yu <yu...@gmail.com> wrote:
>
> On page 4:
> The PIL-replaced functions are still manually picked and
> replaced. The next major task is to make the whole process
> automated and find more bugs.
>
> Do you have estimate when the task of automation would be completed ?
>
> Thanks
>
> On Mon, Feb 27, 2017 at 8:25 PM, Tanakorn Leesatapornwongsa <tanakorn@cs.uchicago.edu <ma...@cs.uchicago.edu>> wrote:
> Dear HBase team,
>
> We are CS researchers at the University of Chicago. In our recent
> research, we are building a framework to find and replay "scalability
> bugs" with just one machine (scalability bugs are bugs that only surface
> in large-scale deployments such as in hundreds of nodes, but do not appear
> in small/medium-scale deployments).
>
> We attach our paper about the problem, if you are interested in reading
> more.
>
> In this email, we would like to ask if any of you would like to
> collaborate by pointing us to any new scalability bugs (JIRA entry) that
> you are aware of.
>
> We ourselves had manually scan each bug entry in JIRA, but we might miss
> some of the new ones.
>
> Thank you in advance for your time!
>
> Sincerely,
> Tanakorn Leesatapornwongsa and Prof. Haryadi Gunawi
> http://ucare.cs.uchicago.edu <http://ucare.cs.uchicago.edu/>
>
>
>
Re: Scalability Bugs in HBase
Posted by Ted Yu <yu...@gmail.com>.
On page 4:
The PIL-replaced functions are still manually picked and
replaced. The next major task is to make the whole process
automated and find more bugs.
Do you have estimate when the task of automation would be completed ?
Thanks
On Mon, Feb 27, 2017 at 8:25 PM, Tanakorn Leesatapornwongsa <
tanakorn@cs.uchicago.edu> wrote:
> Dear HBase team,
>
> We are CS researchers at the University of Chicago. In our recent
> research, we are building a framework to find and replay "scalability
> bugs" with just one machine (scalability bugs are bugs that only surface
> in large-scale deployments such as in hundreds of nodes, but do not appear
> in small/medium-scale deployments).
>
> We attach our paper about the problem, if you are interested in reading
> more.
>
> In this email, we would like to ask if any of you would like to
> collaborate by pointing us to any new scalability bugs (JIRA entry) that
> you are aware of.
>
> We ourselves had manually scan each bug entry in JIRA, but we might miss
> some of the new ones.
>
> Thank you in advance for your time!
>
> Sincerely,
> Tanakorn Leesatapornwongsa and Prof. Haryadi Gunawi
> http://ucare.cs.uchicago.edu
>
>
>