You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Eric Huang (JIRA)" <ji...@apache.org> on 2014/11/04 09:02:33 UTC
[jira] [Updated] (SQOOP-1670) Propose to add banwidth control
support when using Sqoop import/export tools
[ https://issues.apache.org/jira/browse/SQOOP-1670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Huang updated SQOOP-1670:
------------------------------
Description:
As far as I know, Sqoop can't Specify the maximum amount of DB bandwidth that one Sqoop process can utilize for the import/export purpose in term of the number of bytes per second. For some cases such as online applications, DB bandwidth is very important. If one Sqoop process uses most of DB bandwidth, other apps based on the same DB will suffered a lot. So I want to provide a small patch, which is very useful for my case. When starting sqoop import/export tasks, you just need to give an argument (-Dsqoop.db.bandwidth=5242880, means specifying bandwidth to 5M) after import or export tool name, for example:
$bin/sqoop import -Dsqoop.db.bandwidth=5242880 --connect jdbc:mysql://192.168.1.28:9001/db_name --table t_order --hbase-table t_order --column-family f --hbase-create-table --username 'root' --password 'root' --m 3
was:
As far as I know, Sqoop can't Specify the maximum amount of DB bandwidth that one Sqoop process can utilize for the import/export purpose in term of the number of bytes per second. For some cases such as online applications, DB bandwidth
is very important. If one Sqoop process uses most of DB bandwidth, other apps based on the same DB will suffered a lot. So I want to provide a small patch, which is very useful for my case. When starting sqoop import/export tasks, you just need to give an argument (-Dsqoop.db.bandwidth=5242880, means specifying bandwidth to 5M) after import or export tool name, for example:
$bin/sqoop import -Dsqoop.db.bandwidth=5242880 --connect jdbc:mysql://192.168.1.28:9001/db_name --table t_order --hbase-table t_order --column-family f --hbase-create-table --username 'root' --password 'root' --m 3
> Propose to add banwidth control support when using Sqoop import/export tools
> ----------------------------------------------------------------------------
>
> Key: SQOOP-1670
> URL: https://issues.apache.org/jira/browse/SQOOP-1670
> Project: Sqoop
> Issue Type: Improvement
> Components: connectors
> Reporter: Eric Huang
> Fix For: 1.4.4
>
> Attachments: db-bandwidth-ctrl.patch
>
>
> As far as I know, Sqoop can't Specify the maximum amount of DB bandwidth that one Sqoop process can utilize for the import/export purpose in term of the number of bytes per second. For some cases such as online applications, DB bandwidth is very important. If one Sqoop process uses most of DB bandwidth, other apps based on the same DB will suffered a lot. So I want to provide a small patch, which is very useful for my case. When starting sqoop import/export tasks, you just need to give an argument (-Dsqoop.db.bandwidth=5242880, means specifying bandwidth to 5M) after import or export tool name, for example:
> $bin/sqoop import -Dsqoop.db.bandwidth=5242880 --connect jdbc:mysql://192.168.1.28:9001/db_name --table t_order --hbase-table t_order --column-family f --hbase-create-table --username 'root' --password 'root' --m 3
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)