You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by "Rajat Khandelwal (JIRA)" <ji...@apache.org> on 2015/07/10 12:54:04 UTC
[jira] [Created] (LENS-656) make Storage class more important
Rajat Khandelwal created LENS-656:
-------------------------------------
Summary: make Storage class more important
Key: LENS-656
URL: https://issues.apache.org/jira/browse/LENS-656
Project: Apache Lens
Issue Type: Improvement
Reporter: Rajat Khandelwal
Current usage of Stogage in lens:
* Storages are crated with xml, where only class name is required. It has to be a subclass of org.apache.lens.cube.metadata.Storage. Two subclasses are already provided: org.apache.lens.cube.metadata.HDFSStorage, and org.apache.lens.storage.db.DBStorage.
* Current requirement of storage subclass is to implement methods of org.apache.lens.cube.metadata.PartitionMetahook (will be shown below). The two provided subclasses provide a no-op implementations of this.
Issues:
* DBStorage has a dbUrl variable supposed to capture DB url. But that is not used anywhere and is redundant. In the current design, the db url is required in configuration of JDBCDriver. This means that drivers and storages are deeply coupled.
* For querying partitions, Storage is not queried. Instead a map of storage name to partitions is maintained elsewhere. Another example of deep coupling
Proposed changes:
* Storage as an independent entity. Basically removing the deep coupling
* Facts will have Storage instances instead of storage names. With loose coupling, one storage will have only one instance.
* Driver conf has a property "supported storages". That is currently used in the resolvers. Instead the property will become an attribute of the driver.
{noformat}
public interface PartitionMetahook {
void preAddPartitions(List<StoragePartitionDesc> storagePartitionDesc) throws HiveException;
void commitAddPartitions(List<StoragePartitionDesc> storagePartitionDesc) throws HiveException;
void rollbackAddPartitions(List<StoragePartitionDesc> storagePartitionDesc) throws HiveException;
void preDropPartition(String storageTableName, List<String> partVals) throws HiveException;
void commitDropPartition(String storageTableName, List<String> partVals) throws HiveException;
void rollbackDropPartition(String storageTableName, List<String> partVals) throws HiveException;
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)