You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Chris Bennight <ch...@slowcar.net> on 2014/11/06 00:58:53 UTC
TabletLocator API stability / alternates?
So we have some code (a custom input format for data persisted in accumulo
with a custom indexing scheme (geospatial/n-dimensional)):
https://github.com/ngageoint/geowave/blob/GEOWAVE-84-squash/geowave-accumulo/src/main/java/mil/nga/giat/geowave/accumulo/mapreduce/input/GeoWaveInputFormat.java#L355
The intent behind this was to provide better locality and split information
since we have a bit more application specific knowledge available than the
general use case.
I'm pretty sure there's no other way to get this locality information other
than using the TableLocator class.
The arguments + ordering change for TCredentials to Credentials and the
method signature from getInstance() to getLocator() are the two things
breaking our 1.5.1 -> 1.6.x compatibility.
(specifically:
https://github.com/apache/accumulo/commit/99da5641c28784c7b717cce6749673863c2ec8cf#diff-c45768534f53d5455cc05c75676fb871R49
https://github.com/apache/accumulo/commit/446a37a9795f2df7adc841154ca05add79cf286e#diff-c45768534f53d5455cc05c75676fb871R95
)
It's pretty obvious from the diff these were intentional - so no joy there
in accidental changes that could be fixed.
Are we just to far down in the weeds, and are going to have to deal with
supporting multiple versions/breaking changes (via refactoring, dropping
support, or maven-munge maybe), or is this class/methods/signatures
expected to be pretty stable now?
(Or is there a better/more supported way of getting tablet locality
information?)
Re: TabletLocator API stability / alternates?
Posted by Chris Bennight <ch...@slowcar.net>.
Thanks for sitrep and tickets. Definitely willing to pitch in on the use
cases - we can add some comments/use cases/opinions to the ticket
Christopher T. referenced (2883), and take it from there.
Chris Bennight
On Wed, Nov 5, 2014 at 6:46 PM, Christopher <ct...@apache.org> wrote:
> We definitely don't have any public API for this, but I can imagine, and
> would be in favor of, an API improvement in 2.0.0 that exposes tablets.
> There's already an open issue for it:
> https://issues.apache.org/jira/browse/ACCUMULO-2883.
>
>
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii
>
> On Wed, Nov 5, 2014 at 7:31 PM, Josh Elser <jo...@gmail.com> wrote:
>
> > I don't think there is any class/method that we consider to be in our
> > "Public API" that does what you want.
> >
> > You're also not the first one that has tried to do something using
> > TabletLocator and has been bitten by it (https://issues.apache.org/
> > jira/browse/ACCUMULO-2594).
> >
> > At this point, I think we should look at bringing tablet locality into
> the
> > public API for 1.7.0 or 2.0.0. Would you be interested in filing an issue
> > on JIRA and helping to flesh out the functionality requirements?
> >
> >
> > Chris Bennight wrote:
> >
> >> So we have some code (a custom input format for data persisted in
> accumulo
> >> with a custom indexing scheme (geospatial/n-dimensional)):
> >>
> >> https://github.com/ngageoint/geowave/blob/GEOWAVE-84-
> >> squash/geowave-accumulo/src/main/java/mil/nga/giat/
> >> geowave/accumulo/mapreduce/input/GeoWaveInputFormat.java#L355
> >>
> >> The intent behind this was to provide better locality and split
> >> information
> >> since we have a bit more application specific knowledge available than
> the
> >> general use case.
> >>
> >> I'm pretty sure there's no other way to get this locality information
> >> other
> >> than using the TableLocator class.
> >>
> >> The arguments + ordering change for TCredentials to Credentials and the
> >> method signature from getInstance() to getLocator() are the two things
> >> breaking our 1.5.1 -> 1.6.x compatibility.
> >> (specifically:
> >>
> https://github.com/apache/accumulo/commit/99da5641c28784c7b717cce6749673
> >> 863c2ec8cf#diff-c45768534f53d5455cc05c75676fb871R49
> >>
> https://github.com/apache/accumulo/commit/446a37a9795f2df7adc841154ca05a
> >> dd79cf286e#diff-c45768534f53d5455cc05c75676fb871R95
> >> )
> >>
> >> It's pretty obvious from the diff these were intentional - so no joy
> there
> >> in accidental changes that could be fixed.
> >>
> >> Are we just to far down in the weeds, and are going to have to deal with
> >> supporting multiple versions/breaking changes (via refactoring, dropping
> >> support, or maven-munge maybe), or is this class/methods/signatures
> >> expected to be pretty stable now?
> >>
> >> (Or is there a better/more supported way of getting tablet locality
> >> information?)
> >>
> >>
>
Re: TabletLocator API stability / alternates?
Posted by Christopher <ct...@apache.org>.
We definitely don't have any public API for this, but I can imagine, and
would be in favor of, an API improvement in 2.0.0 that exposes tablets.
There's already an open issue for it:
https://issues.apache.org/jira/browse/ACCUMULO-2883.
--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Wed, Nov 5, 2014 at 7:31 PM, Josh Elser <jo...@gmail.com> wrote:
> I don't think there is any class/method that we consider to be in our
> "Public API" that does what you want.
>
> You're also not the first one that has tried to do something using
> TabletLocator and has been bitten by it (https://issues.apache.org/
> jira/browse/ACCUMULO-2594).
>
> At this point, I think we should look at bringing tablet locality into the
> public API for 1.7.0 or 2.0.0. Would you be interested in filing an issue
> on JIRA and helping to flesh out the functionality requirements?
>
>
> Chris Bennight wrote:
>
>> So we have some code (a custom input format for data persisted in accumulo
>> with a custom indexing scheme (geospatial/n-dimensional)):
>>
>> https://github.com/ngageoint/geowave/blob/GEOWAVE-84-
>> squash/geowave-accumulo/src/main/java/mil/nga/giat/
>> geowave/accumulo/mapreduce/input/GeoWaveInputFormat.java#L355
>>
>> The intent behind this was to provide better locality and split
>> information
>> since we have a bit more application specific knowledge available than the
>> general use case.
>>
>> I'm pretty sure there's no other way to get this locality information
>> other
>> than using the TableLocator class.
>>
>> The arguments + ordering change for TCredentials to Credentials and the
>> method signature from getInstance() to getLocator() are the two things
>> breaking our 1.5.1 -> 1.6.x compatibility.
>> (specifically:
>> https://github.com/apache/accumulo/commit/99da5641c28784c7b717cce6749673
>> 863c2ec8cf#diff-c45768534f53d5455cc05c75676fb871R49
>> https://github.com/apache/accumulo/commit/446a37a9795f2df7adc841154ca05a
>> dd79cf286e#diff-c45768534f53d5455cc05c75676fb871R95
>> )
>>
>> It's pretty obvious from the diff these were intentional - so no joy there
>> in accidental changes that could be fixed.
>>
>> Are we just to far down in the weeds, and are going to have to deal with
>> supporting multiple versions/breaking changes (via refactoring, dropping
>> support, or maven-munge maybe), or is this class/methods/signatures
>> expected to be pretty stable now?
>>
>> (Or is there a better/more supported way of getting tablet locality
>> information?)
>>
>>
Re: TabletLocator API stability / alternates?
Posted by Josh Elser <jo...@gmail.com>.
I don't think there is any class/method that we consider to be in our
"Public API" that does what you want.
You're also not the first one that has tried to do something using
TabletLocator and has been bitten by it
(https://issues.apache.org/jira/browse/ACCUMULO-2594).
At this point, I think we should look at bringing tablet locality into
the public API for 1.7.0 or 2.0.0. Would you be interested in filing an
issue on JIRA and helping to flesh out the functionality requirements?
Chris Bennight wrote:
> So we have some code (a custom input format for data persisted in accumulo
> with a custom indexing scheme (geospatial/n-dimensional)):
>
> https://github.com/ngageoint/geowave/blob/GEOWAVE-84-squash/geowave-accumulo/src/main/java/mil/nga/giat/geowave/accumulo/mapreduce/input/GeoWaveInputFormat.java#L355
>
> The intent behind this was to provide better locality and split information
> since we have a bit more application specific knowledge available than the
> general use case.
>
> I'm pretty sure there's no other way to get this locality information other
> than using the TableLocator class.
>
> The arguments + ordering change for TCredentials to Credentials and the
> method signature from getInstance() to getLocator() are the two things
> breaking our 1.5.1 -> 1.6.x compatibility.
> (specifically:
> https://github.com/apache/accumulo/commit/99da5641c28784c7b717cce6749673863c2ec8cf#diff-c45768534f53d5455cc05c75676fb871R49
> https://github.com/apache/accumulo/commit/446a37a9795f2df7adc841154ca05add79cf286e#diff-c45768534f53d5455cc05c75676fb871R95
> )
>
> It's pretty obvious from the diff these were intentional - so no joy there
> in accidental changes that could be fixed.
>
> Are we just to far down in the weeds, and are going to have to deal with
> supporting multiple versions/breaking changes (via refactoring, dropping
> support, or maven-munge maybe), or is this class/methods/signatures
> expected to be pretty stable now?
>
> (Or is there a better/more supported way of getting tablet locality
> information?)
>