You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Daniel Voros (JIRA)" <ji...@apache.org> on 2018/06/21 21:47:00 UTC
[jira] [Created] (SQOOP-3336) Splitting on integer column can
create more splits than necessary
Daniel Voros created SQOOP-3336:
-----------------------------------
Summary: Splitting on integer column can create more splits than necessary
Key: SQOOP-3336
URL: https://issues.apache.org/jira/browse/SQOOP-3336
Project: Sqoop
Issue Type: Bug
Affects Versions: 1.4.7
Reporter: Daniel Voros
Assignee: Daniel Voros
Fix For: 1.5.0, 3.0.0
Running an import with {{-m 2}} will result in three splits if there are only three consecutive integers in the table ({{\{1, 2, 3\}}}).
Work is (probably) spread more evenly between mappers this way, but ending up with more files than expected could be an issue.
Split-limit can also result in more values than asked for in the last chunk (due to the closed interval in the end).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)