You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by Josh Clum <jo...@gmail.com> on 2014/08/18 21:29:11 UTC
Falcon alias with filters
Is there any way to incorporate aliases into the falcon filter statements?
For instance, falcon_${input}_partition_filter_hive will produce something
like (dt='2014-08-18'). But what if in my sql query, I have a table1 and
table2 and I want to filter with (table1.dt='2014-08-18') is there no way
to do this?
Thanks,
Josh
Re: Falcon alias with filters
Posted by Josh Clum <jo...@gmail.com>.
Thanks!
On Mon, Aug 18, 2014 at 6:20 PM, Seetharam Venkatesh <
venkatesh@innerzeal.com> wrote:
> Best way to look at the materialized names and its corresponding values are
> to look at a coordinator and its job configuration.
>
>
> On Mon, Aug 18, 2014 at 3:20 PM, Seetharam Venkatesh <
> venkatesh@innerzeal.com> wrote:
>
> > Falcon already provides these filters OOTB for hive, pig, and java.
> >
> > You could use
> falcon_$input_table.falcon_$input_dated_partition_value_$key
> >
> > For each input and output for a given process, the following are
> available:
> >
> > falcon_$input_storage_type
> > falcon_$input_catalog_url
> > falcon_$input_database
> > falcon_$input_table
> >
> > falcon_$input_partitions_pig
> > falcon_$input_partitions_java
> > falcon_$input_dated_partition_value_$key
> >
> > Relevant code
> > in
> org.apache.falcon.oozie.OozieEntityBuilder#propagateCatalogTableProperties
> >
> >
> >
> > On Mon, Aug 18, 2014 at 12:29 PM, Josh Clum <jo...@gmail.com> wrote:
> >
> >> Is there any way to incorporate aliases into the falcon filter
> statements?
> >> For instance, falcon_${input}_partition_filter_hive will produce
> something
> >> like (dt='2014-08-18'). But what if in my sql query, I have a table1 and
> >> table2 and I want to filter with (table1.dt='2014-08-18') is there no
> way
> >> to do this?
> >>
> >> Thanks,
> >> Josh
> >>
> >
> >
> >
> > --
> > Regards,
> > Venkatesh
> >
> > “Perfection (in design) is achieved not when there is nothing more to
> add,
> > but rather when there is nothing more to take away.”
> > - Antoine de Saint-Exupéry
> >
>
>
>
> --
> Regards,
> Venkatesh
>
> “Perfection (in design) is achieved not when there is nothing more to add,
> but rather when there is nothing more to take away.”
> - Antoine de Saint-Exupéry
>
Re: Falcon alias with filters
Posted by Seetharam Venkatesh <ve...@innerzeal.com>.
Best way to look at the materialized names and its corresponding values are
to look at a coordinator and its job configuration.
On Mon, Aug 18, 2014 at 3:20 PM, Seetharam Venkatesh <
venkatesh@innerzeal.com> wrote:
> Falcon already provides these filters OOTB for hive, pig, and java.
>
> You could use falcon_$input_table.falcon_$input_dated_partition_value_$key
>
> For each input and output for a given process, the following are available:
>
> falcon_$input_storage_type
> falcon_$input_catalog_url
> falcon_$input_database
> falcon_$input_table
>
> falcon_$input_partitions_pig
> falcon_$input_partitions_java
> falcon_$input_dated_partition_value_$key
>
> Relevant code
> in org.apache.falcon.oozie.OozieEntityBuilder#propagateCatalogTableProperties
>
>
>
> On Mon, Aug 18, 2014 at 12:29 PM, Josh Clum <jo...@gmail.com> wrote:
>
>> Is there any way to incorporate aliases into the falcon filter statements?
>> For instance, falcon_${input}_partition_filter_hive will produce something
>> like (dt='2014-08-18'). But what if in my sql query, I have a table1 and
>> table2 and I want to filter with (table1.dt='2014-08-18') is there no way
>> to do this?
>>
>> Thanks,
>> Josh
>>
>
>
>
> --
> Regards,
> Venkatesh
>
> “Perfection (in design) is achieved not when there is nothing more to add,
> but rather when there is nothing more to take away.”
> - Antoine de Saint-Exupéry
>
--
Regards,
Venkatesh
“Perfection (in design) is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.”
- Antoine de Saint-Exupéry
Re: Falcon alias with filters
Posted by Seetharam Venkatesh <ve...@innerzeal.com>.
Falcon already provides these filters OOTB for hive, pig, and java.
You could use falcon_$input_table.falcon_$input_dated_partition_value_$key
For each input and output for a given process, the following are available:
falcon_$input_storage_type
falcon_$input_catalog_url
falcon_$input_database
falcon_$input_table
falcon_$input_partitions_pig
falcon_$input_partitions_java
falcon_$input_dated_partition_value_$key
Relevant code
in org.apache.falcon.oozie.OozieEntityBuilder#propagateCatalogTableProperties
On Mon, Aug 18, 2014 at 12:29 PM, Josh Clum <jo...@gmail.com> wrote:
> Is there any way to incorporate aliases into the falcon filter statements?
> For instance, falcon_${input}_partition_filter_hive will produce something
> like (dt='2014-08-18'). But what if in my sql query, I have a table1 and
> table2 and I want to filter with (table1.dt='2014-08-18') is there no way
> to do this?
>
> Thanks,
> Josh
>
--
Regards,
Venkatesh
“Perfection (in design) is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.”
- Antoine de Saint-Exupéry