You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Davide Grohmann (Created) (JIRA)" <ji...@apache.org> on 2012/04/06 14:17:23 UTC
[jira] [Created] (SQOOP-477) Tools (e.g., ImportTool) using
OptionBuilder are not thread safe
Tools (e.g., ImportTool) using OptionBuilder are not thread safe
----------------------------------------------------------------
Key: SQOOP-477
URL: https://issues.apache.org/jira/browse/SQOOP-477
Project: Sqoop
Issue Type: Bug
Components: tools
Affects Versions: 1.4.1-incubating, 1.4.0-incubating, 1.4.2-incubating
Environment: Software
Reporter: Davide Grohmann
Creating instance of *Tools (e.g., ImportTool, BaseSqoopTool, etc) concurrently (multiple threads) is not thread safe. The problem is in the command line parsing via OptionBuilder which is not thread safe.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-477) Tools (e.g., ImportTool) using
OptionBuilder are not thread safe
Posted by "Davide Grohmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475585#comment-13475585 ]
Davide Grohmann commented on SQOOP-477:
---------------------------------------
Hi Jarek,
well my use case is about importing several tables into Hadoop in different threads, in order to distribute the work as much as possible.
Doing this causes a race condition in the OptionBuilder because it is not thread safe.
Davide
> Tools (e.g., ImportTool) using OptionBuilder are not thread safe
> ----------------------------------------------------------------
>
> Key: SQOOP-477
> URL: https://issues.apache.org/jira/browse/SQOOP-477
> Project: Sqoop
> Issue Type: Bug
> Components: tools
> Affects Versions: 1.4.0-incubating, 1.4.1-incubating, 1.4.2
> Environment: Software
> Reporter: Davide Grohmann
>
> Creating instance of *Tools (e.g., ImportTool, BaseSqoopTool, etc) concurrently (multiple threads) is not thread safe. The problem is in the command line parsing via OptionBuilder which is not thread safe.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-477) Tools (e.g., ImportTool) using
OptionBuilder are not thread safe
Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475907#comment-13475907 ]
Jarek Jarcec Cecho commented on SQOOP-477:
------------------------------------------
Hi David,
thank you very much for description of your use case. I believe that one simple solution would be to move all Option building code to static context, which should not require significant amount of work to accomplish.
Jarcec
> Tools (e.g., ImportTool) using OptionBuilder are not thread safe
> ----------------------------------------------------------------
>
> Key: SQOOP-477
> URL: https://issues.apache.org/jira/browse/SQOOP-477
> Project: Sqoop
> Issue Type: Bug
> Components: tools
> Affects Versions: 1.4.0-incubating, 1.4.1-incubating, 1.4.2
> Environment: Software
> Reporter: Davide Grohmann
>
> Creating instance of *Tools (e.g., ImportTool, BaseSqoopTool, etc) concurrently (multiple threads) is not thread safe. The problem is in the command line parsing via OptionBuilder which is not thread safe.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-477) Tools (e.g., ImportTool) using
OptionBuilder are not thread safe
Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475360#comment-13475360 ]
Jarek Jarcec Cecho commented on SQOOP-477:
------------------------------------------
Hi David,
do you have an use case where this is causing issues to you?
Jarcec
> Tools (e.g., ImportTool) using OptionBuilder are not thread safe
> ----------------------------------------------------------------
>
> Key: SQOOP-477
> URL: https://issues.apache.org/jira/browse/SQOOP-477
> Project: Sqoop
> Issue Type: Bug
> Components: tools
> Affects Versions: 1.4.0-incubating, 1.4.1-incubating, 1.4.2
> Environment: Software
> Reporter: Davide Grohmann
>
> Creating instance of *Tools (e.g., ImportTool, BaseSqoopTool, etc) concurrently (multiple threads) is not thread safe. The problem is in the command line parsing via OptionBuilder which is not thread safe.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-477) Tools (e.g., ImportTool) using
OptionBuilder are not thread safe
Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475629#comment-13475629 ]
Jarek Jarcec Cecho commented on SQOOP-477:
------------------------------------------
Hi David,
thank you very much for your feedback. I've realized that you need to import multiple tables in paralel, however I would be interested more how you're actually executing Sqoop. The reason I'm asking is that normally users are simply executing multiple Sqoop commands in paralel from different shells - as each process is independent, there is no need to be thread safe. Second most common use case is execute Sqoop using Ooize, but even in this case each Sqoop action will be executed in different process and thus no need to be thread safe. I'm expecting that you're executing Sqoop directly from within your own application?
Jarcec
> Tools (e.g., ImportTool) using OptionBuilder are not thread safe
> ----------------------------------------------------------------
>
> Key: SQOOP-477
> URL: https://issues.apache.org/jira/browse/SQOOP-477
> Project: Sqoop
> Issue Type: Bug
> Components: tools
> Affects Versions: 1.4.0-incubating, 1.4.1-incubating, 1.4.2
> Environment: Software
> Reporter: Davide Grohmann
>
> Creating instance of *Tools (e.g., ImportTool, BaseSqoopTool, etc) concurrently (multiple threads) is not thread safe. The problem is in the command line parsing via OptionBuilder which is not thread safe.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (SQOOP-477) Tools (e.g., ImportTool) using
OptionBuilder are not thread safe
Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jarek Jarcec Cecho updated SQOOP-477:
-------------------------------------
Labels: newbie (was: )
> Tools (e.g., ImportTool) using OptionBuilder are not thread safe
> ----------------------------------------------------------------
>
> Key: SQOOP-477
> URL: https://issues.apache.org/jira/browse/SQOOP-477
> Project: Sqoop
> Issue Type: Bug
> Components: tools
> Affects Versions: 1.4.0-incubating, 1.4.1-incubating, 1.4.2
> Environment: Software
> Reporter: Davide Grohmann
> Labels: newbie
>
> Creating instance of *Tools (e.g., ImportTool, BaseSqoopTool, etc) concurrently (multiple threads) is not thread safe. The problem is in the command line parsing via OptionBuilder which is not thread safe.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-477) Tools (e.g., ImportTool) using
OptionBuilder are not thread safe
Posted by "Davide Grohmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475808#comment-13475808 ]
Davide Grohmann commented on SQOOP-477:
---------------------------------------
Hi Jarek,
yes I am executing Sqoop directly from my application. The reason is that I should import tables into Hadoop after having received some kind of requests. I can understand that the tool is designed as a batch tool and it is usually executed from the command line. However, I think it should be rather simple to customize your builder in order to have thread safeness.
Davide
> Tools (e.g., ImportTool) using OptionBuilder are not thread safe
> ----------------------------------------------------------------
>
> Key: SQOOP-477
> URL: https://issues.apache.org/jira/browse/SQOOP-477
> Project: Sqoop
> Issue Type: Bug
> Components: tools
> Affects Versions: 1.4.0-incubating, 1.4.1-incubating, 1.4.2
> Environment: Software
> Reporter: Davide Grohmann
>
> Creating instance of *Tools (e.g., ImportTool, BaseSqoopTool, etc) concurrently (multiple threads) is not thread safe. The problem is in the command line parsing via OptionBuilder which is not thread safe.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira