You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by Venkatesan Ramachandran <me...@gmail.com> on 2015/10/04 04:59:32 UTC
Re: Review Request 38465: FALCON-1459 : Ability to import from
database
> On Sept. 19, 2015, 3:56 p.m., Venkat Ranganathan wrote:
> > General comments: Can you please add a section in the docs for this explaining the various types of import options and the implications.
> > How does one create a Datasource entity using the REST api?
> > Please create a tracking bug for UI enhancments?
Updated
* EntitySpecification.twiki
* FalconCLI.twiki
- Venkatesan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38465/#review99409
-----------------------------------------------------------
On Sept. 23, 2015, 11:58 p.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38465/
> -----------------------------------------------------------
>
> (Updated Sept. 23, 2015, 11:58 p.m.)
>
>
> Review request for Falcon, Ajay Yadava, Balu Vellanki, Peeyush Bishnoi, Sowmya Ramesh, and Venkat Ranganathan.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> FALCON-1459 : Ability to import from database
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/LifeCycle.java 58a2a6c
> client/src/main/java/org/apache/falcon/Tag.java beeb812
> client/src/main/java/org/apache/falcon/entity/v0/EntityType.java 0657124
> client/src/main/java/org/apache/falcon/metadata/RelationshipType.java f034772
> client/src/main/resources/datasource-0.1.xsd PRE-CREATION
> client/src/main/resources/feed-0.1.xsd 2af28d2
> client/src/main/resources/jaxb-binding.xjb 6f1d6c7
> client/src/main/resources/mysql_database.xml PRE-CREATION
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java ad41674
> common/src/main/java/org/apache/falcon/entity/FeedHelper.java 572923b
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java PRE-CREATION
> common/src/main/java/org/apache/falcon/entity/parser/EntityParserFactory.java 5a33201
> common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java 4f5599e
> common/src/main/java/org/apache/falcon/entity/store/ConfigurationStore.java e27187b
> common/src/main/java/org/apache/falcon/entity/v0/EntityGraph.java bd4c6cf
> common/src/main/java/org/apache/falcon/entity/v0/EntityIntegrityChecker.java bd32852
> common/src/main/java/org/apache/falcon/metadata/EntityRelationshipGraphBuilder.java 8c3876c
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java PRE-CREATION
> common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionContext.java 4454239
> common/src/test/java/org/apache/falcon/entity/AbstractTestBase.java e9946c4
> common/src/test/java/org/apache/falcon/entity/EntityTypeTest.java 640e87d
> common/src/test/java/org/apache/falcon/entity/FeedHelperTest.java c70cfcc
> common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java PRE-CREATION
> common/src/test/java/org/apache/falcon/entity/parser/FeedEntityParserTest.java b6fdb13
> common/src/test/java/org/apache/falcon/entity/v0/EntityGraphTest.java 3863b11
> common/src/test/resources/config/datasource/datasource-0.1.xml PRE-CREATION
> common/src/test/resources/config/datasource/datasource-invalid-0.1.xml PRE-CREATION
> common/src/test/resources/config/feed/feed-import-0.1.xml PRE-CREATION
> common/src/test/resources/config/feed/feed-import-invalid-0.1.xml PRE-CREATION
> docs/src/site/twiki/EntitySpecification.twiki d4f4140
> falcon-regression/merlin-core/src/main/java/org/apache/falcon/regression/core/util/HiveAssert.java 2a934b5
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/FeedImportCoordinatorBuilder.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java a04ae95
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 3213a70
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java b819dee
> oozie/src/main/resources/action/feed/import-sqoop-database-action.xml PRE-CREATION
> pom.xml 646de69
> webapp/pom.xml 828f7f5
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java PRE-CREATION
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java d067dee
> webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java PRE-CREATION
> webapp/src/test/resources/datasource-template.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/38465/diff/
>
>
> Testing
> -------
>
> * Unit tests
> * Integration tests
> * Manual tests
> * Setup MySQL, create table and populate
> * Create datasource and feed entity with import policy in Falcon
> * Made sure the data lands up in the HDFS.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>