You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by Victoria Bare <vb...@gmail.com> on 2012/10/23 15:54:00 UTC
Filter Implementation - Accumulo 1.3
Hello,
I am currently using Accumulo 1.3 to implement a Filter. Since I'm using
1.3, I realize that the Filter class is not an iterator so I have created a
MyFilter class that implements Filter to use when I initialize my Scanner.
When I run my code, I am getting an AccumuloServerException.
I was referencing the posts from December 2011 on "Filter Use" to
initialize my scanner with MyFilter.
My scanner initialization currently appears as so:
Instance zooInstance = new ZooKeeperInstance(*instanceName*, *zooServers*);
Connector connector = zooInstance.getConnector(*userName*, *password*);
Authorizations authorizations = new Authorizations();
Scanner scanner = connector.createScanner(*tableName*, authorizations);
scanner.setRange(*range*);
scanner.setScanIterators(1,
"org.apache.accumulo.core.iterators.FilteringIterator", "myFilter");
scanner.setScanIteratorOption("myFilter", "0", "test.offsets.MyFilter");
scanner.setScanIteratorOption("myFilter", "0.start", *start*);
Iterator<Entry<Key,Value>> iterator = scanner.iterator();
while(iterator.hasNext()) { <--- Exception here
...
}
-------------------------------------------------------------------------------------------------------------------------
public class MyFilter implements Filter{
long startOfRange = 0;
@Override
public boolean accept(Key key, Value value) {
String colqual = key.getColumnQualifier().toString();
long end = Long.parseLong(colqual.substring(20, 39));
if(end < startOfRange){
return false;
}
return true;
}
@Override
public void init(Map<String, String> options) {
if(options == null){
throw new IllegalArgumentException("'start' must be set for filter");
}
String start = options.get("start");
if(start == null){
throw new IllegalArgumentException("'start' must be set for filter");
}
startOfRange = Long.parseLong(start);
}
}
-------------------------------------------------------------------------------------------------------------------------
The Exception that I'm receiving is:
Exception in thread "main" java.lang.RuntimeException:
org.apache.accumulo.core.client.impl.AccumuloServerException:
at
org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java)
at test.offsets.TestFilter.getFilterEntrySetRange(TestFilter.java)
at
test.offsets.TestFilter.getAnalysisProductsByClassFilteredOffset(TestFilter.java)
at test.offsets.TestFilter.main(TestFilter.java)
-------------------------------------------------------------------------------------------------------------------------
I was thinking that maybe the server couldn't find the MyFilter class, or
maybe it was a permissions error, but I wasn't sure. When I initialize my
Scanner to use MyFilter, is it looking on the server for the file or in my
project?
Any assistance you can provide would be greatly appreciated, thanks!
Tori
Re: Filter Implementation - Accumulo 1.3
Posted by Victoria Bare <vb...@gmail.com>.
Thank you guys for you input, I really appreciate it! I just had to put
the jar into the $ACCUMULO_HOME/lib/ext directory and everything seems to
be working.
On Tue, Oct 23, 2012 at 2:41 PM, William Slacum <
wilhelm.von.cloud@accumulo.net> wrote:
> Make sure that the class is available to the the tserver process. This is
> done by putting the jar containing your class on all nodes under the
> $ACCUMULO_HOME/lib/ext directory. If you put it under lib/ext, then you
> won't need to stop and restart the process for the tserver to pick it up.
>
>
> On Tue, Oct 23, 2012 at 10:15 AM, Eric Newton <er...@gmail.com>wrote:
>
>> Check the tablet server logs... you'll see the real problem using the
>> filter in there.
>>
>> -Eric
>>
>>
>> On Tue, Oct 23, 2012 at 9:54 AM, Victoria Bare <vb...@gmail.com>wrote:
>>
>>> Hello,
>>>
>>> I am currently using Accumulo 1.3 to implement a Filter. Since I'm
>>> using 1.3, I realize that the Filter class is not an iterator so I have
>>> created a MyFilter class that implements Filter to use when I initialize my
>>> Scanner. When I run my code, I am getting an AccumuloServerException.
>>>
>>> I was referencing the posts from December 2011 on "Filter Use" to
>>> initialize my scanner with MyFilter.
>>> My scanner initialization currently appears as so:
>>>
>>> Instance zooInstance = new ZooKeeperInstance(*instanceName*, *zooServers
>>> *);
>>> Connector connector = zooInstance.getConnector(*userName*, *password*);
>>> Authorizations authorizations = new Authorizations();
>>> Scanner scanner = connector.createScanner(*tableName*, authorizations);
>>>
>>> scanner.setRange(*range*);
>>>
>>> scanner.setScanIterators(1,
>>> "org.apache.accumulo.core.iterators.FilteringIterator", "myFilter");
>>> scanner.setScanIteratorOption("myFilter", "0", "test.offsets.MyFilter");
>>> scanner.setScanIteratorOption("myFilter", "0.start", *start*);
>>>
>>> Iterator<Entry<Key,Value>> iterator = scanner.iterator();
>>>
>>> while(iterator.hasNext()) { <--- Exception here
>>>
>>> ...
>>>
>>> }
>>>
>>>
>>> -------------------------------------------------------------------------------------------------------------------------
>>>
>>> public class MyFilter implements Filter{
>>> long startOfRange = 0;
>>> @Override
>>> public boolean accept(Key key, Value value) {
>>> String colqual = key.getColumnQualifier().toString();
>>> long end = Long.parseLong(colqual.substring(20, 39));
>>> if(end < startOfRange){
>>> return false;
>>> }
>>> return true;
>>> }
>>>
>>> @Override
>>> public void init(Map<String, String> options) {
>>>
>>> if(options == null){
>>> throw new IllegalArgumentException("'start' must be set for filter");
>>> }
>>> String start = options.get("start");
>>> if(start == null){
>>> throw new IllegalArgumentException("'start' must be set for filter");
>>> }
>>> startOfRange = Long.parseLong(start);
>>> }
>>>
>>> }
>>>
>>>
>>> -------------------------------------------------------------------------------------------------------------------------
>>>
>>> The Exception that I'm receiving is:
>>>
>>> Exception in thread "main" java.lang.RuntimeException:
>>> org.apache.accumulo.core.client.impl.AccumuloServerException:
>>> at
>>> org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java)
>>> at test.offsets.TestFilter.getFilterEntrySetRange(TestFilter.java)
>>> at
>>> test.offsets.TestFilter.getAnalysisProductsByClassFilteredOffset(TestFilter.java)
>>> at test.offsets.TestFilter.main(TestFilter.java)
>>>
>>>
>>> -------------------------------------------------------------------------------------------------------------------------
>>>
>>> I was thinking that maybe the server couldn't find the MyFilter class,
>>> or maybe it was a permissions error, but I wasn't sure. When I initialize
>>> my Scanner to use MyFilter, is it looking on the server for the file or in
>>> my project?
>>>
>>> Any assistance you can provide would be greatly appreciated, thanks!
>>> Tori
>>>
>>
>>
>
Re: Filter Implementation - Accumulo 1.3
Posted by William Slacum <wi...@accumulo.net>.
Make sure that the class is available to the the tserver process. This is
done by putting the jar containing your class on all nodes under the
$ACCUMULO_HOME/lib/ext directory. If you put it under lib/ext, then you
won't need to stop and restart the process for the tserver to pick it up.
On Tue, Oct 23, 2012 at 10:15 AM, Eric Newton <er...@gmail.com> wrote:
> Check the tablet server logs... you'll see the real problem using the
> filter in there.
>
> -Eric
>
>
> On Tue, Oct 23, 2012 at 9:54 AM, Victoria Bare <vb...@gmail.com>wrote:
>
>> Hello,
>>
>> I am currently using Accumulo 1.3 to implement a Filter. Since I'm using
>> 1.3, I realize that the Filter class is not an iterator so I have created a
>> MyFilter class that implements Filter to use when I initialize my Scanner.
>> When I run my code, I am getting an AccumuloServerException.
>>
>> I was referencing the posts from December 2011 on "Filter Use" to
>> initialize my scanner with MyFilter.
>> My scanner initialization currently appears as so:
>>
>> Instance zooInstance = new ZooKeeperInstance(*instanceName*, *zooServers*
>> );
>> Connector connector = zooInstance.getConnector(*userName*, *password*);
>> Authorizations authorizations = new Authorizations();
>> Scanner scanner = connector.createScanner(*tableName*, authorizations);
>>
>> scanner.setRange(*range*);
>>
>> scanner.setScanIterators(1,
>> "org.apache.accumulo.core.iterators.FilteringIterator", "myFilter");
>> scanner.setScanIteratorOption("myFilter", "0", "test.offsets.MyFilter");
>> scanner.setScanIteratorOption("myFilter", "0.start", *start*);
>>
>> Iterator<Entry<Key,Value>> iterator = scanner.iterator();
>>
>> while(iterator.hasNext()) { <--- Exception here
>>
>> ...
>>
>> }
>>
>>
>> -------------------------------------------------------------------------------------------------------------------------
>>
>> public class MyFilter implements Filter{
>> long startOfRange = 0;
>> @Override
>> public boolean accept(Key key, Value value) {
>> String colqual = key.getColumnQualifier().toString();
>> long end = Long.parseLong(colqual.substring(20, 39));
>> if(end < startOfRange){
>> return false;
>> }
>> return true;
>> }
>>
>> @Override
>> public void init(Map<String, String> options) {
>>
>> if(options == null){
>> throw new IllegalArgumentException("'start' must be set for filter");
>> }
>> String start = options.get("start");
>> if(start == null){
>> throw new IllegalArgumentException("'start' must be set for filter");
>> }
>> startOfRange = Long.parseLong(start);
>> }
>>
>> }
>>
>>
>> -------------------------------------------------------------------------------------------------------------------------
>>
>> The Exception that I'm receiving is:
>>
>> Exception in thread "main" java.lang.RuntimeException:
>> org.apache.accumulo.core.client.impl.AccumuloServerException:
>> at
>> org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java)
>> at test.offsets.TestFilter.getFilterEntrySetRange(TestFilter.java)
>> at
>> test.offsets.TestFilter.getAnalysisProductsByClassFilteredOffset(TestFilter.java)
>> at test.offsets.TestFilter.main(TestFilter.java)
>>
>>
>> -------------------------------------------------------------------------------------------------------------------------
>>
>> I was thinking that maybe the server couldn't find the MyFilter class, or
>> maybe it was a permissions error, but I wasn't sure. When I initialize my
>> Scanner to use MyFilter, is it looking on the server for the file or in my
>> project?
>>
>> Any assistance you can provide would be greatly appreciated, thanks!
>> Tori
>>
>
>
Re: Filter Implementation - Accumulo 1.3
Posted by Eric Newton <er...@gmail.com>.
Check the tablet server logs... you'll see the real problem using the
filter in there.
-Eric
On Tue, Oct 23, 2012 at 9:54 AM, Victoria Bare <vb...@gmail.com>wrote:
> Hello,
>
> I am currently using Accumulo 1.3 to implement a Filter. Since I'm using
> 1.3, I realize that the Filter class is not an iterator so I have created a
> MyFilter class that implements Filter to use when I initialize my Scanner.
> When I run my code, I am getting an AccumuloServerException.
>
> I was referencing the posts from December 2011 on "Filter Use" to
> initialize my scanner with MyFilter.
> My scanner initialization currently appears as so:
>
> Instance zooInstance = new ZooKeeperInstance(*instanceName*, *zooServers*
> );
> Connector connector = zooInstance.getConnector(*userName*, *password*);
> Authorizations authorizations = new Authorizations();
> Scanner scanner = connector.createScanner(*tableName*, authorizations);
>
> scanner.setRange(*range*);
>
> scanner.setScanIterators(1,
> "org.apache.accumulo.core.iterators.FilteringIterator", "myFilter");
> scanner.setScanIteratorOption("myFilter", "0", "test.offsets.MyFilter");
> scanner.setScanIteratorOption("myFilter", "0.start", *start*);
>
> Iterator<Entry<Key,Value>> iterator = scanner.iterator();
>
> while(iterator.hasNext()) { <--- Exception here
>
> ...
>
> }
>
>
> -------------------------------------------------------------------------------------------------------------------------
>
> public class MyFilter implements Filter{
> long startOfRange = 0;
> @Override
> public boolean accept(Key key, Value value) {
> String colqual = key.getColumnQualifier().toString();
> long end = Long.parseLong(colqual.substring(20, 39));
> if(end < startOfRange){
> return false;
> }
> return true;
> }
>
> @Override
> public void init(Map<String, String> options) {
>
> if(options == null){
> throw new IllegalArgumentException("'start' must be set for filter");
> }
> String start = options.get("start");
> if(start == null){
> throw new IllegalArgumentException("'start' must be set for filter");
> }
> startOfRange = Long.parseLong(start);
> }
>
> }
>
>
> -------------------------------------------------------------------------------------------------------------------------
>
> The Exception that I'm receiving is:
>
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.accumulo.core.client.impl.AccumuloServerException:
> at
> org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java)
> at test.offsets.TestFilter.getFilterEntrySetRange(TestFilter.java)
> at
> test.offsets.TestFilter.getAnalysisProductsByClassFilteredOffset(TestFilter.java)
> at test.offsets.TestFilter.main(TestFilter.java)
>
>
> -------------------------------------------------------------------------------------------------------------------------
>
> I was thinking that maybe the server couldn't find the MyFilter class, or
> maybe it was a permissions error, but I wasn't sure. When I initialize my
> Scanner to use MyFilter, is it looking on the server for the file or in my
> project?
>
> Any assistance you can provide would be greatly appreciated, thanks!
> Tori
>