[GitHub] [incubator-dolphinscheduler] snycc15 opened a new issue #4146: sqoop write to hive hcatalog orc

[GitBox <gi...@apache.org>]

snycc15 opened a new issue #4146:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4146


   建议同步写方式方式多一种选择，可考虑写到hive orc 表，性能方面会有所提升，现在是直接写到文本，例如：
   sqoop import --connect jdbc:oracle --username xxx --password xxx --table xxx.xxx --hcatalog-databse xxx  --hcatalog-table xx  --drop-and-create-hcatalog-table --hcatalog-storage-stanza 'stored as orc tblproperties ("orc.compress"="SNAPPY","transactional"="false")' --m 1


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] Eights-Li commented on issue #4146: sqoop write to hive hcatalog orc

[GitBox <gi...@apache.org>]

Eights-Li commented on issue #4146:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4146#issuecomment-737630077


   using hcatalog to import RMDBS have some limits ：
   http://sqoop.apache.org/docs/1.4.7/SqoopUserGuide.html#_unsupported_sqoop_options
   if you need custom sqoop task, you can use sqoop custom job in ds-dev-branch


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] snycc15 commented on issue #4146: sqoop write to hive hcatalog orc

[GitBox <gi...@apache.org>]

snycc15 commented on issue #4146:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4146#issuecomment-737855755


   The problem of "no support" does not exist. Import to Orc is a way to promote high performance at present.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org