You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by z11373 <z1...@outlook.com> on 2015/09/01 01:25:30 UTC
Specify Range for data in specific column family
Hi,
AccumuloInputFormat has setRanges method which takes collection of Range.
I want it to process all rows with specific column family name.
However, I couldn't find a way to do so from Range.exact or Range.prefix,
all of them require to specify row id. Do you know if there is a way to
achieve what I am looking for? I haven't tried specifying empty Text for the
row, the doc also doesn't say about it.
Thanks,
zainal
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Specify-Range-for-data-in-specific-column-family-tp15012.html
Sent from the Developers mailing list archive at Nabble.com.
Re: Specify Range for data in specific column family
Posted by z11373 <z1...@outlook.com>.
Thanks Josh! What you said makes sense indeed :-)
Earlier I was too lazy to make additional change in existing code (which is
already using setRanges), and made me think why not it allows us to specify
all row ids with specific column family). But now after you explained very
well, it clearly doesn't make sense to mash them together :-)
Thanks,
Z
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Specify-Range-for-data-in-specific-column-family-tp15012p15036.html
Sent from the Developers mailing list archive at Nabble.com.
Re: Specify Range for data in specific column family
Posted by Josh Elser <jo...@gmail.com>.
That's very likely not ever going to happen.
Ranges are used for identifying portions of a table to scan. These
ranges are identified by start and end keys. Thus, the granularity of
Key itself defines what is valid in a Range (e.g. you can start at a
row+cf and end at a row+cf+cq).
A second way to limit the data you receive is by requesting a limited
collection of columns via the fetchColumns API methods. These methods
are agnostic of the range of data being consumed, only concerned with
the columns being fetched.
I don't think it makes any sense to try to mash them together :)
z11373 wrote:
> Thanks Josh! I wish future version of Accumulo API would allow us to specify
> only column family in the Range, so that AccumuloInputFormat.setRanges will
> work for my case. This is low pri though, as it has fetchColumns as
> alternative :-)
>
>
> Thanks,
> Z
>
>
>
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Specify-Range-for-data-in-specific-column-family-tp15012p15031.html
> Sent from the Developers mailing list archive at Nabble.com.
Re: Specify Range for data in specific column family
Posted by z11373 <z1...@outlook.com>.
Thanks Josh! I wish future version of Accumulo API would allow us to specify
only column family in the Range, so that AccumuloInputFormat.setRanges will
work for my case. This is low pri though, as it has fetchColumns as
alternative :-)
Thanks,
Z
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Specify-Range-for-data-in-specific-column-family-tp15012p15031.html
Sent from the Developers mailing list archive at Nabble.com.
Re: Specify Range for data in specific column family
Posted by Josh Elser <jo...@gmail.com>.
Keith Turner wrote:
> On Mon, Aug 31, 2015 at 8:27 PM, z11373<z1...@outlook.com> wrote:
>
>> > Hi Josh,
>> > Apologize that I still don't get it.
>> > How to pass it to the AccumuloInputFormat?
>> > If there is no way to do so (means I have to change the code), then I'd
>> > think about it.
>> > Right now the existing code already using AIF, and I just want to tweak the
>> > Range condition.
>> >
>
> Look not in Range but in AccumuloInputFormat methods and you will find the
> method you seek.
>
>
Ack, yeah, sorry for the run-around. I spaced on the fact that you said
your were using the InputFormat.
http://accumulo.apache.org/1.6/apidocs/org/apache/accumulo/core/client/mapreduce/InputFormatBase.html#fetchColumns-org.apache.hadoop.mapreduce.Job-java.util.Collection-
In your case, you'd want to do something like:
`AccumuloInputFormat.fetchColumns(job, Arrays.asList(new
Pair<Text,Text>(new Text(myColfam), null));`
(sorry if you're using the mapred libraries instead of mapreduce
libraries -- note the java package. Both will have the same method, one
just accepts a Job and the other a JobConf to match both mapreduce APIs
from Hadoop)
Re: Specify Range for data in specific column family
Posted by Keith Turner <ke...@deenlo.com>.
On Mon, Aug 31, 2015 at 8:27 PM, z11373 <z1...@outlook.com> wrote:
> Hi Josh,
> Apologize that I still don't get it.
> How to pass it to the AccumuloInputFormat?
> If there is no way to do so (means I have to change the code), then I'd
> think about it.
> Right now the existing code already using AIF, and I just want to tweak the
> Range condition.
>
Look not in Range but in AccumuloInputFormat methods and you will find the
method you seek.
>
>
> Thanks,
> Z
>
>
>
> --
> View this message in context:
> http://apache-accumulo.1065345.n5.nabble.com/Specify-Range-for-data-in-specific-column-family-tp15012p15016.html
> Sent from the Developers mailing list archive at Nabble.com.
>
Re: Specify Range for data in specific column family
Posted by z11373 <z1...@outlook.com>.
Hi Josh,
Apologize that I still don't get it.
How to pass it to the AccumuloInputFormat?
If there is no way to do so (means I have to change the code), then I'd
think about it.
Right now the existing code already using AIF, and I just want to tweak the
Range condition.
Thanks,
Z
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Specify-Range-for-data-in-specific-column-family-tp15012p15016.html
Sent from the Developers mailing list archive at Nabble.com.
Re: Specify Range for data in specific column family
Posted by Josh Elser <jo...@gmail.com>.
Use `[batch]scanner.fetchColumnFamily(Text)` to specify the column
family. All rows would just be specified by `Collections.singleton(new
Range())`.
- Josh
z11373 wrote:
> Hi,
> AccumuloInputFormat has setRanges method which takes collection of Range.
> I want it to process all rows with specific column family name.
> However, I couldn't find a way to do so from Range.exact or Range.prefix,
> all of them require to specify row id. Do you know if there is a way to
> achieve what I am looking for? I haven't tried specifying empty Text for the
> row, the doc also doesn't say about it.
>
> Thanks,
> zainal
>
>
>
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/Specify-Range-for-data-in-specific-column-family-tp15012.html
> Sent from the Developers mailing list archive at Nabble.com.