[jira] [Updated] (LENS-511) identify facts associated with underlying hive/jdbc tables

["Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LENS-511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amareshwari Sriramadasu updated LENS-511:
-----------------------------------------
    Labels: Hackathon-July  (was: )

> identify facts associated with underlying hive/jdbc tables
> ----------------------------------------------------------
>
>                 Key: LENS-511
>                 URL: https://issues.apache.org/jira/browse/LENS-511
>             Project: Apache Lens
>          Issue Type: Improvement
>          Components: api, cube
>    Affects Versions: 2.0
>            Reporter: Pranav Kumar Agarwal
>              Labels: Hackathon-July
>
> Consider following storage table defn inside a fact:
> <x_fact_table cube_name="sample_cube" name="fact2" weight="200.0"
> xmlns="uri:lens:cube:0.1"
>    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd ">
> ...
>   <storage_tables>
>      <storage_table>
>        <update_periods>
>          <update_period>HOURLY</update_period>
>          <update_period>DAILY</update_period>
>        </update_periods>
>        <storage_name>local</storage_name>
>        <table_desc external="true" field_delimiter=","
> table_location="/tmp/examples/fact2_local">
>          <part_cols>
>            <column comment="Time column" name="dt" type="STRING"/>
>          </part_cols>
>          <time_part_cols>dt</time_part_cols>
>        </table_desc>
>      </storage_table>
>    </storage_tables>
> </x_fact_table>
> In an event a new partition is added to the external table location
> "/tmp/examples/fact2_local" then I wish to add a new partition on the
> fact2, however I have no way to find what all facts are built on
> external table location "/tmp/examples/fact2_local". We can possibly do
> it by matching the location, however that doesn't seem to be quite
> nice.. Consider for a JDBC source the table_location is kind of dummy as
> its not really used to query the content from that location. JDBCDriver
> expects a table with the name as storageName_factName in the target
> datastore, thus no indication on which all facts to be updated.
> Problem Statement: Current storage_table definition doesn't give me
> enough detail to find where all partitions needs to be added given a new
> partition is added to an external table.
> I propose that we add a table_name property as part of table_desc and
> provide following API's:
> GET /storages/{storageName}/tableNames/
> GET /storages/{storageName}/tableNames/{tableName}/facts



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)