You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Sean Busbey <se...@manvsbeard.com> on 2014/05/17 23:25:29 UTC
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/
-----------------------------------------------------------
(Updated May 17, 2014, 9:25 p.m.)
Review request for accumulo, Eric Newton and Mike Drob.
Changes
-------
Rebased to 1.5 dev line. Incorporated feedback from Christopher and Mike.
Bugs: ACCUMULO-2694
https://issues.apache.org/jira/browse/ACCUMULO-2694
Repository: accumulo
Description
-------
ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
* Adds a funtional test that fails due to not rebalancing
* Fix master to clear migrations when it learns that a table has gone offline
* Update master to periodically clean up migrations for offline tables
* Fix balancers to make sure they log if they can't balance.
Diffs (updated)
-----
server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
Diff: https://reviews.apache.org/r/20525/diff/
Testing
-------
Ran functional test without other changes -> failed. After full patch functional test passes.
Thanks,
Sean Busbey
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Sean Busbey <se...@manvsbeard.com>.
> On June 2, 2014, 3:53 a.m., Josh Elser wrote:
> > server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java, line 146
> > <https://reviews.apache.org/r/20525/diff/4/?file=583985#file583985line146>
> >
> > Logging the KeyExtents has the potential to leak sensitive data (think inverted index storing terms in the row). Maybe you can log just the tableId for the first 10 migrations instead.
Good point. I'll include a note about why we're limiting the data as well.
- Sean
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review44473
-----------------------------------------------------------
On May 17, 2014, 9:25 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated May 17, 2014, 9:25 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Josh Elser <jo...@gmail.com>.
> On June 2, 2014, 3:53 a.m., Josh Elser wrote:
> > server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java, line 146
> > <https://reviews.apache.org/r/20525/diff/4/?file=583985#file583985line146>
> >
> > Logging the KeyExtents has the potential to leak sensitive data (think inverted index storing terms in the row). Maybe you can log just the tableId for the first 10 migrations instead.
>
> Sean Busbey wrote:
> Good point. I'll include a note about why we're limiting the data as well.
Excellent.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review44473
-----------------------------------------------------------
On May 17, 2014, 9:25 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated May 17, 2014, 9:25 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Josh Elser <jo...@gmail.com>.
> On June 2, 2014, 3:53 a.m., Josh Elser wrote:
> > server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java, line 146
> > <https://reviews.apache.org/r/20525/diff/4/?file=583985#file583985line146>
> >
> > Logging the KeyExtents has the potential to leak sensitive data (think inverted index storing terms in the row). Maybe you can log just the tableId for the first 10 migrations instead.
>
> Sean Busbey wrote:
> Good point. I'll include a note about why we're limiting the data as well.
>
> Josh Elser wrote:
> Excellent.
>
> Sean Busbey wrote:
> FYI, we currently output key extents in tons of other places :/
Blech. I'll file a ticket to investigate this.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review44473
-----------------------------------------------------------
On May 17, 2014, 9:25 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated May 17, 2014, 9:25 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Sean Busbey <se...@manvsbeard.com>.
> On June 2, 2014, 3:53 a.m., Josh Elser wrote:
> > server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java, line 146
> > <https://reviews.apache.org/r/20525/diff/4/?file=583985#file583985line146>
> >
> > Logging the KeyExtents has the potential to leak sensitive data (think inverted index storing terms in the row). Maybe you can log just the tableId for the first 10 migrations instead.
>
> Sean Busbey wrote:
> Good point. I'll include a note about why we're limiting the data as well.
>
> Josh Elser wrote:
> Excellent.
>
> Sean Busbey wrote:
> FYI, we currently output key extents in tons of other places :/
>
> Josh Elser wrote:
> Blech. I'll file a ticket to investigate this.
Does that mean I can leave this log message in place and have it dealt with as a part of the follow on? :)
- Sean
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review44473
-----------------------------------------------------------
On May 17, 2014, 9:25 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated May 17, 2014, 9:25 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Josh Elser <jo...@gmail.com>.
> On June 2, 2014, 3:53 a.m., Josh Elser wrote:
> > server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java, line 146
> > <https://reviews.apache.org/r/20525/diff/4/?file=583985#file583985line146>
> >
> > Logging the KeyExtents has the potential to leak sensitive data (think inverted index storing terms in the row). Maybe you can log just the tableId for the first 10 migrations instead.
>
> Sean Busbey wrote:
> Good point. I'll include a note about why we're limiting the data as well.
>
> Josh Elser wrote:
> Excellent.
>
> Sean Busbey wrote:
> FYI, we currently output key extents in tons of other places :/
>
> Josh Elser wrote:
> Blech. I'll file a ticket to investigate this.
>
> Sean Busbey wrote:
> Does that mean I can leave this log message in place and have it dealt with as a part of the follow on? :)
I'd rather it just be trounced now, but I won't -1 you if you don't.
- Josh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review44473
-----------------------------------------------------------
On May 17, 2014, 9:25 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated May 17, 2014, 9:25 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Sean Busbey <se...@manvsbeard.com>.
> On June 2, 2014, 3:53 a.m., Josh Elser wrote:
> > server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java, line 146
> > <https://reviews.apache.org/r/20525/diff/4/?file=583985#file583985line146>
> >
> > Logging the KeyExtents has the potential to leak sensitive data (think inverted index storing terms in the row). Maybe you can log just the tableId for the first 10 migrations instead.
>
> Sean Busbey wrote:
> Good point. I'll include a note about why we're limiting the data as well.
>
> Josh Elser wrote:
> Excellent.
>
> Sean Busbey wrote:
> FYI, we currently output key extents in tons of other places :/
>
> Josh Elser wrote:
> Blech. I'll file a ticket to investigate this.
>
> Sean Busbey wrote:
> Does that mean I can leave this log message in place and have it dealt with as a part of the follow on? :)
>
> Josh Elser wrote:
> I'd rather it just be trounced now, but I won't -1 you if you don't.
I'd rather increase the priority and fixVersion for ACCUMULO-2938 than hack something in for just this one location and then undo it for that ticket.
- Sean
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review44473
-----------------------------------------------------------
On May 17, 2014, 9:25 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated May 17, 2014, 9:25 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Sean Busbey <se...@manvsbeard.com>.
> On June 2, 2014, 3:53 a.m., Josh Elser wrote:
> > server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java, line 146
> > <https://reviews.apache.org/r/20525/diff/4/?file=583985#file583985line146>
> >
> > Logging the KeyExtents has the potential to leak sensitive data (think inverted index storing terms in the row). Maybe you can log just the tableId for the first 10 migrations instead.
>
> Sean Busbey wrote:
> Good point. I'll include a note about why we're limiting the data as well.
>
> Josh Elser wrote:
> Excellent.
FYI, we currently output key extents in tons of other places :/
- Sean
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review44473
-----------------------------------------------------------
On May 17, 2014, 9:25 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated May 17, 2014, 9:25 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Josh Elser <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review44473
-----------------------------------------------------------
server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java
<https://reviews.apache.org/r/20525/#comment78890>
Logging the KeyExtents has the potential to leak sensitive data (think inverted index storing terms in the row). Maybe you can log just the tableId for the first 10 migrations instead.
test/system/auto/stress/migrations.py
<https://reviews.apache.org/r/20525/#comment78891>
Just noting (to make sure it doesn't go missing in the merge) that this will need to be migrated to an IT.
- Josh Elser
On May 17, 2014, 9:25 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated May 17, 2014, 9:25 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Sean Busbey <se...@manvsbeard.com>.
> On June 2, 2014, 2:01 p.m., Eric Newton wrote:
> > server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java, line 98
> > <https://reviews.apache.org/r/20525/diff/4/?file=583985#file583985line98>
> >
> > Nit: spelling: deferred, can't
> >
fixed.
- Sean
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review44494
-----------------------------------------------------------
On May 17, 2014, 9:25 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated May 17, 2014, 9:25 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Eric Newton <er...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review44494
-----------------------------------------------------------
server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java
<https://reviews.apache.org/r/20525/#comment78896>
Nit: spelling: deferred, can't
- Eric Newton
On May 17, 2014, 9:25 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated May 17, 2014, 9:25 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Sean Busbey <se...@manvsbeard.com>.
> On June 23, 2014, 9:43 p.m., Bill Havanki wrote:
> > server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java, lines 157-158
> > <https://reviews.apache.org/r/20525/diff/6/?file=615223#file615223line157>
> >
> > When constraintNotMet() transitions to stuck = true, it won't run the cause to log it; i.e., the first cause in a series won't get logged.
Correct, it's not supposed to log until the problem has persisted for the threshold period.
- Sean
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review46455
-----------------------------------------------------------
On June 23, 2014, 9:18 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated June 23, 2014, 9:18 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Bill Havanki <bh...@clouderagovt.com>.
> On June 23, 2014, 5:43 p.m., Bill Havanki wrote:
> > server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java, lines 157-158
> > <https://reviews.apache.org/r/20525/diff/6/?file=615223#file615223line157>
> >
> > When constraintNotMet() transitions to stuck = true, it won't run the cause to log it; i.e., the first cause in a series won't get logged.
>
> Sean Busbey wrote:
> Correct, it's not supposed to log until the problem has persisted for the threshold period.
OK. I was concerned that this would lead to under-reporting - for example, a 59-second long problem would never be logged. But, if the nature of this is that it's fine to not mention more transient problems (say, a 30-second long problem that flip-flops), I'm good.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review46455
-----------------------------------------------------------
On June 23, 2014, 5:18 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated June 23, 2014, 5:18 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Bill Havanki <bh...@clouderagovt.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review46455
-----------------------------------------------------------
server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java
<https://reviews.apache.org/r/20525/#comment81847>
When constraintNotMet() transitions to stuck = true, it won't run the cause to log it; i.e., the first cause in a series won't get logged.
- Bill Havanki
On June 23, 2014, 5:18 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated June 23, 2014, 5:18 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Josh Elser <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review46799
-----------------------------------------------------------
Ship it!
Ship it w/ a friendly reminder to convert the python test into an IT after the merge.
- Josh Elser
On June 23, 2014, 9:18 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated June 23, 2014, 9:18 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Sean Busbey <se...@manvsbeard.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/
-----------------------------------------------------------
(Updated June 23, 2014, 9:18 p.m.)
Review request for accumulo, Eric Newton and Mike Drob.
Changes
-------
more typos
Bugs: ACCUMULO-2694
https://issues.apache.org/jira/browse/ACCUMULO-2694
Repository: accumulo
Description
-------
ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
* Adds a funtional test that fails due to not rebalancing
* Fix master to clear migrations when it learns that a table has gone offline
* Update master to periodically clean up migrations for offline tables
* Fix balancers to make sure they log if they can't balance.
Diffs (updated)
-----
server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
Diff: https://reviews.apache.org/r/20525/diff/
Testing
-------
Ran functional test without other changes -> failed. After full patch functional test passes.
Thanks,
Sean Busbey
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Mike Drob <md...@mdrob.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/#review46448
-----------------------------------------------------------
server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java
<https://reviews.apache.org/r/20525/#comment81842>
spelling: there
- Mike Drob
On June 23, 2014, 9:02 p.m., Sean Busbey wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20525/
> -----------------------------------------------------------
>
> (Updated June 23, 2014, 9:02 p.m.)
>
>
> Review request for accumulo, Eric Newton and Mike Drob.
>
>
> Bugs: ACCUMULO-2694
> https://issues.apache.org/jira/browse/ACCUMULO-2694
>
>
> Repository: accumulo
>
>
> Description
> -------
>
> ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
>
> * Adds a funtional test that fails due to not rebalancing
> * Fix master to clear migrations when it learns that a table has gone offline
> * Update master to periodically clean up migrations for offline tables
> * Fix balancers to make sure they log if they can't balance.
>
>
> Diffs
> -----
>
> server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
> server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
> server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
> server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
> server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
> test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
>
> Diff: https://reviews.apache.org/r/20525/diff/
>
>
> Testing
> -------
>
> Ran functional test without other changes -> failed. After full patch functional test passes.
>
>
> Thanks,
>
> Sean Busbey
>
>
Re: Review Request 20525: ACCUMULO-2694 Fix handling of tablet migrations
for offline tables.
Posted by Sean Busbey <se...@manvsbeard.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20525/
-----------------------------------------------------------
(Updated June 23, 2014, 9:02 p.m.)
Review request for accumulo, Eric Newton and Mike Drob.
Changes
-------
rebased to current. fixed typos. added TODO for ACCUMULO-2938.
Bugs: ACCUMULO-2694
https://issues.apache.org/jira/browse/ACCUMULO-2694
Repository: accumulo
Description
-------
ACCUMULO-2694 Fix handling of tablet migrations for offline tables.
* Adds a funtional test that fails due to not rebalancing
* Fix master to clear migrations when it learns that a table has gone offline
* Update master to periodically clean up migrations for offline tables
* Fix balancers to make sure they log if they can't balance.
Diffs (updated)
-----
server/pom.xml bd61fe6d870449247cde10ae6ed88c98a31657a2
server/src/main/java/org/apache/accumulo/server/master/Master.java a2ad2e65e5766d5760199a2c477679f5616a0710
server/src/main/java/org/apache/accumulo/server/master/balancer/ChaoticLoadBalancer.java e14008a8e4caf72133dd9a08b9aab566bb03d862
server/src/main/java/org/apache/accumulo/server/master/balancer/DefaultLoadBalancer.java 1fcab4681c32b20d04d8ab9e72d6a61eb6bdddf0
server/src/main/java/org/apache/accumulo/server/master/balancer/TabletBalancer.java 69387d365e7207b2faaa91805ca2fe8b432e0d49
test/system/auto/stress/migrations.py d07d7a89ddc930f538f736883d1a3ec43d020d00
Diff: https://reviews.apache.org/r/20525/diff/
Testing
-------
Ran functional test without other changes -> failed. After full patch functional test passes.
Thanks,
Sean Busbey