You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@phoenix.apache.org by "Biju G.S Nair" <gs...@gmail.com> on 2014/03/09 04:49:39 UTC

Re: SkipScanFilter

Hello there,
   I am trying to understand how to use the Phoenix skipscan filter with
native HBase JAVA scan call. Following is a simple code snippet. The table
is 't1' with column family 'f1' and column 'c1'. There are rows with keys
'r1' to 'r10'. The idea is to only retrieve the rows 'r3' to 'r8' using the
filter but is not successful. Any help on this is much appreciated.

Code:

public static void main(String[] args) throws IOException {

Configuration conf = HBaseConfiguration.create();

HTable table = new HTable(conf, "t1");

RowKeySchemaBuilder rksBuilder = new RowKeySchema.RowKeySchemaBuilder(1);

PDatum pDatum = new PColumnImpl(PNameFactory.newName("c1"),

PNameFactory.newName("f1"), PDataType.VARCHAR, 3, 0, false, 0,

SortOrder.getDefault(),0,null);

rksBuilder.addField(pDatum, false, SortOrder.getDefault());

RowKeySchema rks = rksBuilder.build();

KeyRange kr = KeyRange.getKeyRange(Bytes.toBytes("r3"), true,

Bytes.toBytes("r8"), true);

List<KeyRange> krList1 = new ArrayList<KeyRange>();

krList1.add(kr);

List<List<KeyRange>> krList = new ArrayList<List<KeyRange>>();

krList.add(krList1);

Filter filter = new SkipScanFilter(krList, rks);

int totalRows = 0;

byte[] lastRow = null;

Scan scan = new Scan();

scan.setFilter(filter);

ResultScanner scanner = table.getScanner(scan);

int localRows = 0;

Result result;

while ((result = scanner.next()) != null) {

System.out.println(localRows++ + ": " + result);

totalRows++;

lastRow = result.getRow();

System.out.println(lastRow.toString());

}

scanner.close();

table.close();

System.out.println("total rows: " + totalRows);

}
The following message is written to the console while the code functions
fine without the filter.

2014-03-08 22:18:05,221 DEBUG [main]
client.HConnectionManager$HConnectionImplementation: Removed
192.168.0.102:60020 as a location of
t1,,1394218174403.2a743b9968964bac636325ae6ff40422. for tableName=t1 from
cache


Thanks,
Biju

Re: SkipScanFilter

Posted by Biju Nair <gs...@gmail.com>.

Thanks James for helping with this query. You are correct. This is not a good case but wanted to try a simple case to prove the process. Regarding the key value, it is just a string since wanted to keep it simple.

Thanks,
Biju

> On Mar 9, 2014, at 3:24 PM, James Taylor <ja...@apache.org> wrote:
> 
> I don't see a flaw with your code. Does the row key only have the single string in it? Note that it doesn't really make sense to use a skip scan in this case, as you're just doing a range scan. Try adding a singleton list that contains the keys for "k1", "k5", and "k8" instead, as this is a better usage.
> 
> Thanks,
> James
> 
> 
>> On Sat, Mar 8, 2014 at 7:49 PM, Biju G.S Nair <gs...@gmail.com> wrote:
>> Hello there,
>>    I am trying to understand how to use the Phoenix skipscan filter with
>> native HBase JAVA scan call. Following is a simple code snippet. The table
>> is 't1' with column family 'f1' and column 'c1'. There are rows with keys
>> 'r1' to 'r10'. The idea is to only retrieve the rows 'r3' to 'r8' using the
>> filter but is not successful. Any help on this is much appreciated.
>> 
>> Code:
>> 
>> public static void main(String[] args) throws IOException {
>> 
>> Configuration conf = HBaseConfiguration.create();
>> 
>> HTable table = new HTable(conf, "t1");
>> 
>> RowKeySchemaBuilder rksBuilder = new RowKeySchema.RowKeySchemaBuilder(1);
>> 
>> PDatum pDatum = new PColumnImpl(PNameFactory.newName("c1"),
>> 
>> PNameFactory.newName("f1"), PDataType.VARCHAR, 3, 0, false, 0,
>> 
>> SortOrder.getDefault(),0,null);
>> 
>> rksBuilder.addField(pDatum, false, SortOrder.getDefault());
>> 
>> RowKeySchema rks = rksBuilder.build();
>> 
>> KeyRange kr = KeyRange.getKeyRange(Bytes.toBytes("r3"), true,
>> 
>> Bytes.toBytes("r8"), true);
>> 
>> List<KeyRange> krList1 = new ArrayList<KeyRange>();
>> 
>> krList1.add(kr);
>> 
>> List<List<KeyRange>> krList = new ArrayList<List<KeyRange>>();
>> 
>> krList.add(krList1);
>> 
>> Filter filter = new SkipScanFilter(krList, rks);
>> 
>> int totalRows = 0;
>> 
>> byte[] lastRow = null;
>> 
>> Scan scan = new Scan();
>> 
>> scan.setFilter(filter);
>> 
>> ResultScanner scanner = table.getScanner(scan);
>> 
>> int localRows = 0;
>> 
>> Result result;
>> 
>> while ((result = scanner.next()) != null) {
>> 
>> System.out.println(localRows++ + ": " + result);
>> 
>> totalRows++;
>> 
>> lastRow = result.getRow();
>> 
>> System.out.println(lastRow.toString());
>> 
>> }
>> 
>> scanner.close();
>> 
>> table.close();
>> 
>> System.out.println("total rows: " + totalRows);
>> 
>> }
>> The following message is written to the console while the code functions
>> fine without the filter.
>> 
>> 2014-03-08 22:18:05,221 DEBUG [main]
>> client.HConnectionManager$HConnectionImplementation: Removed
>> 192.168.0.102:60020 as a location of
>> t1,,1394218174403.2a743b9968964bac636325ae6ff40422. for tableName=t1 from
>> cache
>> 
>> 
>> Thanks,
>> Biju
>

Re: SkipScanFilter

Posted by Biju Nair <gs...@gmail.com>.

Thanks James for helping with this query. You are correct. This is not a good case but wanted to try a simple case to prove the process. Regarding the key value, it is just a string since wanted to keep it simple.

Thanks,
Biju

> On Mar 9, 2014, at 3:24 PM, James Taylor <ja...@apache.org> wrote:
> 
> I don't see a flaw with your code. Does the row key only have the single string in it? Note that it doesn't really make sense to use a skip scan in this case, as you're just doing a range scan. Try adding a singleton list that contains the keys for "k1", "k5", and "k8" instead, as this is a better usage.
> 
> Thanks,
> James
> 
> 
>> On Sat, Mar 8, 2014 at 7:49 PM, Biju G.S Nair <gs...@gmail.com> wrote:
>> Hello there,
>>    I am trying to understand how to use the Phoenix skipscan filter with
>> native HBase JAVA scan call. Following is a simple code snippet. The table
>> is 't1' with column family 'f1' and column 'c1'. There are rows with keys
>> 'r1' to 'r10'. The idea is to only retrieve the rows 'r3' to 'r8' using the
>> filter but is not successful. Any help on this is much appreciated.
>> 
>> Code:
>> 
>> public static void main(String[] args) throws IOException {
>> 
>> Configuration conf = HBaseConfiguration.create();
>> 
>> HTable table = new HTable(conf, "t1");
>> 
>> RowKeySchemaBuilder rksBuilder = new RowKeySchema.RowKeySchemaBuilder(1);
>> 
>> PDatum pDatum = new PColumnImpl(PNameFactory.newName("c1"),
>> 
>> PNameFactory.newName("f1"), PDataType.VARCHAR, 3, 0, false, 0,
>> 
>> SortOrder.getDefault(),0,null);
>> 
>> rksBuilder.addField(pDatum, false, SortOrder.getDefault());
>> 
>> RowKeySchema rks = rksBuilder.build();
>> 
>> KeyRange kr = KeyRange.getKeyRange(Bytes.toBytes("r3"), true,
>> 
>> Bytes.toBytes("r8"), true);
>> 
>> List<KeyRange> krList1 = new ArrayList<KeyRange>();
>> 
>> krList1.add(kr);
>> 
>> List<List<KeyRange>> krList = new ArrayList<List<KeyRange>>();
>> 
>> krList.add(krList1);
>> 
>> Filter filter = new SkipScanFilter(krList, rks);
>> 
>> int totalRows = 0;
>> 
>> byte[] lastRow = null;
>> 
>> Scan scan = new Scan();
>> 
>> scan.setFilter(filter);
>> 
>> ResultScanner scanner = table.getScanner(scan);
>> 
>> int localRows = 0;
>> 
>> Result result;
>> 
>> while ((result = scanner.next()) != null) {
>> 
>> System.out.println(localRows++ + ": " + result);
>> 
>> totalRows++;
>> 
>> lastRow = result.getRow();
>> 
>> System.out.println(lastRow.toString());
>> 
>> }
>> 
>> scanner.close();
>> 
>> table.close();
>> 
>> System.out.println("total rows: " + totalRows);
>> 
>> }
>> The following message is written to the console while the code functions
>> fine without the filter.
>> 
>> 2014-03-08 22:18:05,221 DEBUG [main]
>> client.HConnectionManager$HConnectionImplementation: Removed
>> 192.168.0.102:60020 as a location of
>> t1,,1394218174403.2a743b9968964bac636325ae6ff40422. for tableName=t1 from
>> cache
>> 
>> 
>> Thanks,
>> Biju
>

Re: SkipScanFilter

Posted by James Taylor <ja...@apache.org>.

I don't see a flaw with your code. Does the row key only have the single
string in it? Note that it doesn't really make sense to use a skip scan in
this case, as you're just doing a range scan. Try adding a singleton list
that contains the keys for "k1", "k5", and "k8" instead, as this is a
better usage.

Thanks,
James


On Sat, Mar 8, 2014 at 7:49 PM, Biju G.S Nair <gs...@gmail.com> wrote:

> Hello there,
>    I am trying to understand how to use the Phoenix skipscan filter with
> native HBase JAVA scan call. Following is a simple code snippet. The table
> is 't1' with column family 'f1' and column 'c1'. There are rows with keys
> 'r1' to 'r10'. The idea is to only retrieve the rows 'r3' to 'r8' using the
> filter but is not successful. Any help on this is much appreciated.
>
> Code:
>
> public static void main(String[] args) throws IOException {
>
> Configuration conf = HBaseConfiguration.create();
>
> HTable table = new HTable(conf, "t1");
>
> RowKeySchemaBuilder rksBuilder = new RowKeySchema.RowKeySchemaBuilder(1);
>
> PDatum pDatum = new PColumnImpl(PNameFactory.newName("c1"),
>
> PNameFactory.newName("f1"), PDataType.VARCHAR, 3, 0, false, 0,
>
> SortOrder.getDefault(),0,null);
>
> rksBuilder.addField(pDatum, false, SortOrder.getDefault());
>
> RowKeySchema rks = rksBuilder.build();
>
> KeyRange kr = KeyRange.getKeyRange(Bytes.toBytes("r3"), true,
>
> Bytes.toBytes("r8"), true);
>
> List<KeyRange> krList1 = new ArrayList<KeyRange>();
>
> krList1.add(kr);
>
> List<List<KeyRange>> krList = new ArrayList<List<KeyRange>>();
>
> krList.add(krList1);
>
> Filter filter = new SkipScanFilter(krList, rks);
>
> int totalRows = 0;
>
> byte[] lastRow = null;
>
> Scan scan = new Scan();
>
> scan.setFilter(filter);
>
> ResultScanner scanner = table.getScanner(scan);
>
> int localRows = 0;
>
> Result result;
>
> while ((result = scanner.next()) != null) {
>
> System.out.println(localRows++ + ": " + result);
>
> totalRows++;
>
> lastRow = result.getRow();
>
> System.out.println(lastRow.toString());
>
> }
>
> scanner.close();
>
> table.close();
>
> System.out.println("total rows: " + totalRows);
>
> }
> The following message is written to the console while the code functions
> fine without the filter.
>
> 2014-03-08 22:18:05,221 DEBUG [main]
> client.HConnectionManager$HConnectionImplementation: Removed
> 192.168.0.102:60020 as a location of
> t1,,1394218174403.2a743b9968964bac636325ae6ff40422. for tableName=t1 from
> cache
>
>
> Thanks,
> Biju
>

Re: SkipScanFilter

Posted by James Taylor <ja...@apache.org>.

I don't see a flaw with your code. Does the row key only have the single
string in it? Note that it doesn't really make sense to use a skip scan in
this case, as you're just doing a range scan. Try adding a singleton list
that contains the keys for "k1", "k5", and "k8" instead, as this is a
better usage.

Thanks,
James


On Sat, Mar 8, 2014 at 7:49 PM, Biju G.S Nair <gs...@gmail.com> wrote:

> Hello there,
>    I am trying to understand how to use the Phoenix skipscan filter with
> native HBase JAVA scan call. Following is a simple code snippet. The table
> is 't1' with column family 'f1' and column 'c1'. There are rows with keys
> 'r1' to 'r10'. The idea is to only retrieve the rows 'r3' to 'r8' using the
> filter but is not successful. Any help on this is much appreciated.
>
> Code:
>
> public static void main(String[] args) throws IOException {
>
> Configuration conf = HBaseConfiguration.create();
>
> HTable table = new HTable(conf, "t1");
>
> RowKeySchemaBuilder rksBuilder = new RowKeySchema.RowKeySchemaBuilder(1);
>
> PDatum pDatum = new PColumnImpl(PNameFactory.newName("c1"),
>
> PNameFactory.newName("f1"), PDataType.VARCHAR, 3, 0, false, 0,
>
> SortOrder.getDefault(),0,null);
>
> rksBuilder.addField(pDatum, false, SortOrder.getDefault());
>
> RowKeySchema rks = rksBuilder.build();
>
> KeyRange kr = KeyRange.getKeyRange(Bytes.toBytes("r3"), true,
>
> Bytes.toBytes("r8"), true);
>
> List<KeyRange> krList1 = new ArrayList<KeyRange>();
>
> krList1.add(kr);
>
> List<List<KeyRange>> krList = new ArrayList<List<KeyRange>>();
>
> krList.add(krList1);
>
> Filter filter = new SkipScanFilter(krList, rks);
>
> int totalRows = 0;
>
> byte[] lastRow = null;
>
> Scan scan = new Scan();
>
> scan.setFilter(filter);
>
> ResultScanner scanner = table.getScanner(scan);
>
> int localRows = 0;
>
> Result result;
>
> while ((result = scanner.next()) != null) {
>
> System.out.println(localRows++ + ": " + result);
>
> totalRows++;
>
> lastRow = result.getRow();
>
> System.out.println(lastRow.toString());
>
> }
>
> scanner.close();
>
> table.close();
>
> System.out.println("total rows: " + totalRows);
>
> }
> The following message is written to the console while the code functions
> fine without the filter.
>
> 2014-03-08 22:18:05,221 DEBUG [main]
> client.HConnectionManager$HConnectionImplementation: Removed
> 192.168.0.102:60020 as a location of
> t1,,1394218174403.2a743b9968964bac636325ae6ff40422. for tableName=t1 from
> cache
>
>
> Thanks,
> Biju
>