You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by "Vikas Singh (Tech - VS)" <vi...@flipkart.com> on 2015/04/17 07:45:58 UTC
Schema Name Resolution for JDBC stores i.e Vertica
Hi
Problem Statement:
For JDBC datastores i.e. Vertica we are facing problem regarding
schema name resolution. As the actual table against which query will be
fired will be <schema_name>.<table_name>. In lens there is no concept of
schema and I guess directly it's not possible to incorporate a "." (DOT) as
table name since Hive does't support dotted names.
Proposed Workaround:
- Adding a property schemaName as part of FactTable and DimTables,
which will contain fully resolved table name.
- Doing a query rewrite for schema name at the JDBC driver level.
Please give your thoughts around this approach. Or if there is any existing
approach to support schema name resolutions.
Regards
Vikas Singh
Re: Schema Name Resolution for JDBC stores i.e Vertica
Posted by "Vikas Singh (Tech - VS)" <vi...@flipkart.com>.
Hi,
Thanks for quick response.
On Mon, Apr 20, 2015 at 9:59 AM, amareshwarisr . <am...@gmail.com>
wrote:
> Thanks for reaching out Vikas. As Rajat and Jaideep mentioned, the feature
> you are asking for already exists. You can set
> lens.metastore.native.db.name
> and lens.metastore.native.table.name as the table properties and
> JDBCDriver
> replaces them in the query rewrite.
>
> See the example at
>
> https://github.com/apache/incubator-lens/blob/master/lens-examples/src/main/resources/city_subset.xml
> which is specifying the db name.
>
> Thanks
> Amareshwari
>
> On Fri, Apr 17, 2015 at 9:06 PM, Jaideep Dhok <ja...@inmobi.com>
> wrote:
>
> > Hi Vikas,
> >
> > As Rajat mentioned JDBC driver can add a dbname.table name.
> >
> > You can check these two propertie which have to be set in the Storage
> table
> > -
> > lens.metastore.native.db.name
> > lens.metastore.native.table.name
> >
> > Relevant code - ColumnarSQLRewriter. replaceWithUnderlyingStorage
> >
> > Thanks,
> > Jaideep
> >
> > On Fri, Apr 17, 2015 at 1:35 PM, Rajat Khandelwal <
> rajatgupta59@gmail.com>
> > wrote:
> >
> > > We can add a driver level config option for jdbc driver. And have jdbc
> > > driver rewrite using that.
> > >
> > > On Fri, Apr 17, 2015 at 12:53 PM Vikas Singh (Tech - VS) <
> > > vikas.singh@flipkart.com> wrote:
> > >
> > > > Hi
> > > >
> > > > Problem Statement:
> > > > For JDBC datastores i.e. Vertica we are facing problem
> regarding
> > > > schema name resolution. As the actual table against which query will
> be
> > > > fired will be <schema_name>.<table_name>. In lens there is no concept
> > of
> > > > schema and I guess directly it's not possible to incorporate a "."
> > (DOT)
> > > as
> > > > table name since Hive does't support dotted names.
> > > >
> > > > Proposed Workaround:
> > > > - Adding a property schemaName as part of FactTable and
> > DimTables,
> > > > which will contain fully resolved table name.
> > > > - Doing a query rewrite for schema name at the JDBC driver
> level.
> > > >
> > > > Please give your thoughts around this approach. Or if there is any
> > > existing
> > > > approach to support schema name resolutions.
> > > >
> > > > Regards
> > > > Vikas Singh
> > > >
> > >
> >
> >
> >
> > --
> > -
> > Jaideep Dhok
> >
> > --
> > _____________________________________________________________
> > The information contained in this communication is intended solely for
> the
> > use of the individual or entity to whom it is addressed and others
> > authorized to receive it. It may contain confidential or legally
> privileged
> > information. If you are not the intended recipient you are hereby
> notified
> > that any disclosure, copying, distribution or taking any action in
> reliance
> > on the contents of this information is strictly prohibited and may be
> > unlawful. If you have received this communication in error, please notify
> > us immediately by responding to this email and then delete it from your
> > system. The firm is neither liable for the proper and complete
> transmission
> > of the information contained in this communication nor for any delay in
> its
> > receipt.
> >
>
Re: Schema Name Resolution for JDBC stores i.e Vertica
Posted by "amareshwarisr ." <am...@gmail.com>.
Thanks for reaching out Vikas. As Rajat and Jaideep mentioned, the feature
you are asking for already exists. You can set lens.metastore.native.db.name
and lens.metastore.native.table.name as the table properties and JDBCDriver
replaces them in the query rewrite.
See the example at
https://github.com/apache/incubator-lens/blob/master/lens-examples/src/main/resources/city_subset.xml
which is specifying the db name.
Thanks
Amareshwari
On Fri, Apr 17, 2015 at 9:06 PM, Jaideep Dhok <ja...@inmobi.com>
wrote:
> Hi Vikas,
>
> As Rajat mentioned JDBC driver can add a dbname.table name.
>
> You can check these two propertie which have to be set in the Storage table
> -
> lens.metastore.native.db.name
> lens.metastore.native.table.name
>
> Relevant code - ColumnarSQLRewriter. replaceWithUnderlyingStorage
>
> Thanks,
> Jaideep
>
> On Fri, Apr 17, 2015 at 1:35 PM, Rajat Khandelwal <ra...@gmail.com>
> wrote:
>
> > We can add a driver level config option for jdbc driver. And have jdbc
> > driver rewrite using that.
> >
> > On Fri, Apr 17, 2015 at 12:53 PM Vikas Singh (Tech - VS) <
> > vikas.singh@flipkart.com> wrote:
> >
> > > Hi
> > >
> > > Problem Statement:
> > > For JDBC datastores i.e. Vertica we are facing problem regarding
> > > schema name resolution. As the actual table against which query will be
> > > fired will be <schema_name>.<table_name>. In lens there is no concept
> of
> > > schema and I guess directly it's not possible to incorporate a "."
> (DOT)
> > as
> > > table name since Hive does't support dotted names.
> > >
> > > Proposed Workaround:
> > > - Adding a property schemaName as part of FactTable and
> DimTables,
> > > which will contain fully resolved table name.
> > > - Doing a query rewrite for schema name at the JDBC driver level.
> > >
> > > Please give your thoughts around this approach. Or if there is any
> > existing
> > > approach to support schema name resolutions.
> > >
> > > Regards
> > > Vikas Singh
> > >
> >
>
>
>
> --
> -
> Jaideep Dhok
>
> --
> _____________________________________________________________
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify
> us immediately by responding to this email and then delete it from your
> system. The firm is neither liable for the proper and complete transmission
> of the information contained in this communication nor for any delay in its
> receipt.
>
Re: Schema Name Resolution for JDBC stores i.e Vertica
Posted by Jaideep Dhok <ja...@inmobi.com>.
Hi Vikas,
As Rajat mentioned JDBC driver can add a dbname.table name.
You can check these two propertie which have to be set in the Storage table
-
lens.metastore.native.db.name
lens.metastore.native.table.name
Relevant code - ColumnarSQLRewriter. replaceWithUnderlyingStorage
Thanks,
Jaideep
On Fri, Apr 17, 2015 at 1:35 PM, Rajat Khandelwal <ra...@gmail.com>
wrote:
> We can add a driver level config option for jdbc driver. And have jdbc
> driver rewrite using that.
>
> On Fri, Apr 17, 2015 at 12:53 PM Vikas Singh (Tech - VS) <
> vikas.singh@flipkart.com> wrote:
>
> > Hi
> >
> > Problem Statement:
> > For JDBC datastores i.e. Vertica we are facing problem regarding
> > schema name resolution. As the actual table against which query will be
> > fired will be <schema_name>.<table_name>. In lens there is no concept of
> > schema and I guess directly it's not possible to incorporate a "." (DOT)
> as
> > table name since Hive does't support dotted names.
> >
> > Proposed Workaround:
> > - Adding a property schemaName as part of FactTable and DimTables,
> > which will contain fully resolved table name.
> > - Doing a query rewrite for schema name at the JDBC driver level.
> >
> > Please give your thoughts around this approach. Or if there is any
> existing
> > approach to support schema name resolutions.
> >
> > Regards
> > Vikas Singh
> >
>
--
-
Jaideep Dhok
--
_____________________________________________________________
The information contained in this communication is intended solely for the
use of the individual or entity to whom it is addressed and others
authorized to receive it. It may contain confidential or legally privileged
information. If you are not the intended recipient you are hereby notified
that any disclosure, copying, distribution or taking any action in reliance
on the contents of this information is strictly prohibited and may be
unlawful. If you have received this communication in error, please notify
us immediately by responding to this email and then delete it from your
system. The firm is neither liable for the proper and complete transmission
of the information contained in this communication nor for any delay in its
receipt.
Re: Schema Name Resolution for JDBC stores i.e Vertica
Posted by Rajat Khandelwal <ra...@gmail.com>.
We can add a driver level config option for jdbc driver. And have jdbc
driver rewrite using that.
On Fri, Apr 17, 2015 at 12:53 PM Vikas Singh (Tech - VS) <
vikas.singh@flipkart.com> wrote:
> Hi
>
> Problem Statement:
> For JDBC datastores i.e. Vertica we are facing problem regarding
> schema name resolution. As the actual table against which query will be
> fired will be <schema_name>.<table_name>. In lens there is no concept of
> schema and I guess directly it's not possible to incorporate a "." (DOT) as
> table name since Hive does't support dotted names.
>
> Proposed Workaround:
> - Adding a property schemaName as part of FactTable and DimTables,
> which will contain fully resolved table name.
> - Doing a query rewrite for schema name at the JDBC driver level.
>
> Please give your thoughts around this approach. Or if there is any existing
> approach to support schema name resolutions.
>
> Regards
> Vikas Singh
>