You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Szabolcs Vasas <va...@gmail.com> on 2016/07/15 08:07:26 UTC
Re: Review Request 37251: SQOOP-2457: Add option to automatically
compute statistics after loading date into a hive table
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37251/#review142346
-----------------------------------------------------------
Ship it!
Ship It!
- Szabolcs Vasas
On Aug. 8, 2015, 4:09 a.m., Venkat Ranganathan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37251/
> -----------------------------------------------------------
>
> (Updated Aug. 8, 2015, 4:09 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2457
> https://issues.apache.org/jira/browse/SQOOP-2457
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> With CBO and different execution engines like Tez depedning on statistics like row count heavily, it is important that we provide the option to update stats on data loaded into Hive as part of the --hive-import option. Ideally these should be Hive managed, but there are use cases where this is not automatic and hence this option will help in those cases
>
> Added a new option --hive-compute-stats which will add compute statistics statement for the loaded table/partition as the case may be for --hive-imports
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/SqoopOptions.java 9405605
> src/java/org/apache/sqoop/hive/HiveImport.java e03d33c
> src/java/org/apache/sqoop/hive/TableDefWriter.java c9962e9
> src/java/org/apache/sqoop/tool/BaseSqoopTool.java 4e2e66d
> src/test/com/cloudera/sqoop/hive/TestHiveImport.java b626964
> src/test/com/cloudera/sqoop/hive/TestTableDefWriter.java 55e572e
> testdata/hive/scripts/normalWithStatsImport.q PRE-CREATION
> testdata/hive/scripts/partitionWithStatsImport.q PRE-CREATION
>
> Diff: https://reviews.apache.org/r/37251/diff/
>
>
> Testing
> -------
>
> Added new tests and all tests pass
>
>
> Thanks,
>
> Venkat Ranganathan
>
>