how to issue command inside command.

[chethan <ch...@gmail.com>]

Hi,

I want to issue command inside command in PIG Script.

command 1 : result = load 'hbase://sample_names' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey true
-gt 1 -lt 3') as (id:chararray);
command 2 : user_links = load 'hbase://sample_names' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey true
-gt result') as (name:chararray);

user_links = load 'hbase://sample_names' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey true
-gt command 1') as (name:chararray);

Thanks & Regards
Chethan Prakash.

Re: how to issue command inside command.

[Dmitriy Ryaboy <dv...@gmail.com>]

Things that come back from a load statement are relations. A column of a relation is also not a single value, but a collection of values. Result.id is a column. 

In any case, even if that worked (and you can in fact promise pig that a relation only has one row and treat it as a scalar) -- you are trying to take something only known at runtime (result of a load) and turn it into something needed at compile time (arguments to a loader). That means you need to wait for that first part to run before even attempting to parse the second part. That's something you can do via python embedding. Check out the docs and the hortonworks blog post for details. 

On Feb 19, 2012, at 11:42 PM, chethan <ch...@gmail.com> wrote:

> If i want to give  variable value i.e.., 'result.id' in the command then
> how to append the variable value 'result.id' in to the string.
> 
> result = load 'hbase://sample_names' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey true
> -gt 1 -lt 3') as (id:chararray);
> user_links = load 'hbase://sample_names' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey true
> -gt' +result.id+' ) as (name:chararray);
> 
> 
> On Fri, Feb 17, 2012 at 8:56 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:
> 
>> This is not currently possible. It's also ill-defined since 'result' is a
>> relation, not a scalar. What you really want is to call an hbase lookup
>> udf, to ensure you get a single record back, not to use the result of a
>> table scan in another table scan.
>> 
>> The way to get this done 'properly' is to implement partition push down in
>> hbase storage. This is nontrivial and valuable. If someone has cycles to do
>> it, that would be a great patch.
>> 
>> On Feb 17, 2012, at 12:19 AM, chethan <ch...@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> I want to issue command inside command in PIG Script.
>>> 
>>> command 1 : result = load 'hbase://sample_names' using
>>> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey
>> true
>>> -gt 1 -lt 3') as (id:chararray);
>>> command 2 : user_links = load 'hbase://sample_names' using
>>> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey
>> true
>>> -gt result') as (name:chararray);
>>> 
>>> user_links = load 'hbase://sample_names' using
>>> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey
>> true
>>> -gt command 1') as (name:chararray);
>>> 
>>> Thanks & Regards
>>> Chethan Prakash.
>> 

Re: how to issue command inside command.

[chethan <ch...@gmail.com>]

If i want to give  variable value i.e.., 'result.id' in the command then
how to append the variable value 'result.id' in to the string.

result = load 'hbase://sample_names' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey true
-gt 1 -lt 3') as (id:chararray);
user_links = load 'hbase://sample_names' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey true
-gt' +result.id+' ) as (name:chararray);


On Fri, Feb 17, 2012 at 8:56 PM, Dmitriy Ryaboy <dv...@gmail.com> wrote:

> This is not currently possible. It's also ill-defined since 'result' is a
> relation, not a scalar. What you really want is to call an hbase lookup
> udf, to ensure you get a single record back, not to use the result of a
> table scan in another table scan.
>
> The way to get this done 'properly' is to implement partition push down in
> hbase storage. This is nontrivial and valuable. If someone has cycles to do
> it, that would be a great patch.
>
> On Feb 17, 2012, at 12:19 AM, chethan <ch...@gmail.com> wrote:
>
> > Hi,
> >
> > I want to issue command inside command in PIG Script.
> >
> > command 1 : result = load 'hbase://sample_names' using
> > org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey
> true
> > -gt 1 -lt 3') as (id:chararray);
> > command 2 : user_links = load 'hbase://sample_names' using
> > org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey
> true
> > -gt result') as (name:chararray);
> >
> > user_links = load 'hbase://sample_names' using
> > org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey
> true
> > -gt command 1') as (name:chararray);
> >
> > Thanks & Regards
> > Chethan Prakash.
>

Re: how to issue command inside command.

[Dmitriy Ryaboy <dv...@gmail.com>]

This is not currently possible. It's also ill-defined since 'result' is a relation, not a scalar. What you really want is to call an hbase lookup udf, to ensure you get a single record back, not to use the result of a table scan in another table scan. 

The way to get this done 'properly' is to implement partition push down in hbase storage. This is nontrivial and valuable. If someone has cycles to do it, that would be a great patch. 

On Feb 17, 2012, at 12:19 AM, chethan <ch...@gmail.com> wrote:

> Hi,
> 
> I want to issue command inside command in PIG Script.
> 
> command 1 : result = load 'hbase://sample_names' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey true
> -gt 1 -lt 3') as (id:chararray);
> command 2 : user_links = load 'hbase://sample_names' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey true
> -gt result') as (name:chararray);
> 
> user_links = load 'hbase://sample_names' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('cf:fname','-loadKey true
> -gt command 1') as (name:chararray);
> 
> Thanks & Regards
> Chethan Prakash.