[jira] [Updated] (LENS-656) make Storage class more important

["Rajat Khandelwal (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LENS-656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rajat Khandelwal updated LENS-656:
----------------------------------
    Issue Type: Sub-task  (was: Improvement)
        Parent: LENS-657

> make Storage class more important
> ---------------------------------
>
>                 Key: LENS-656
>                 URL: https://issues.apache.org/jira/browse/LENS-656
>             Project: Apache Lens
>          Issue Type: Sub-task
>            Reporter: Rajat Khandelwal
>
> Current usage of Stogage in lens:
> * Storages are crated with xml, where only class name is required. It has to be a subclass of org.apache.lens.cube.metadata.Storage. Two subclasses are already provided: org.apache.lens.cube.metadata.HDFSStorage, and org.apache.lens.storage.db.DBStorage. 
> * Current requirement of storage subclass is to implement methods of org.apache.lens.cube.metadata.PartitionMetahook (will be shown below). The two provided subclasses provide a no-op implementations of this. 
> Issues:
> * DBStorage has a dbUrl variable supposed to capture DB url. But that is not used anywhere and is redundant. In the current design, the db url is required in configuration of JDBCDriver. This means that drivers and storages are deeply coupled. 
> * For querying partitions, Storage is not queried. Instead a map of storage name to partitions is maintained elsewhere. Another example of deep coupling
> Proposed changes:
> * Storage as an independent entity. Basically removing the deep coupling
> * Facts will have Storage instances instead of storage names. With loose coupling, one storage will have only one instance. 
> * Driver conf has a property "supported storages". That is currently used in the resolvers. Instead the property will become an attribute of the driver. 
> {noformat}
> public interface PartitionMetahook {
>   void preAddPartitions(List<StoragePartitionDesc> storagePartitionDesc) throws HiveException;
>   void commitAddPartitions(List<StoragePartitionDesc> storagePartitionDesc) throws HiveException;
>   void rollbackAddPartitions(List<StoragePartitionDesc> storagePartitionDesc) throws HiveException;
>   void preDropPartition(String storageTableName, List<String> partVals) throws HiveException;
>   void commitDropPartition(String storageTableName, List<String> partVals) throws HiveException;
>   void rollbackDropPartition(String storageTableName, List<String> partVals) throws HiveException;
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)