[jira] [Updated] (SQOOP-1301) 'run' method in Sqoop.java is not thread-safe

["Qian Xu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SQOOP-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Qian Xu updated SQOOP-1301:
---------------------------
    Fix Version/s:     (was: 1.4.6)
                   1.4.7

> 'run' method in Sqoop.java is not thread-safe
> ---------------------------------------------
>
>                 Key: SQOOP-1301
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1301
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.4.4
>            Reporter: Ajay Chitre
>             Fix For: 1.4.7
>
>
> It seems the ‘run’ method in Sqoop is not thread-safe.  When multiple Sqoop jobs are triggered exactly at the same time, they end up stepping on each other.  Here’s what seems to be the problem:  The ‘run’ method calls tool.parseArguments which in turn calls ConfigurationHelper.parseGenericOptions which in turn creates a new instance of GenericOptionsParser.  The constructor for GenericOptionsParser is not thread-safe because it ends up calling ‘buildGeneralOptions’ which uses OptionBuilder.withArgName.  This method uses instance variables thereby making it thread unsafe.
> The way we’ve got around it is by creating a ‘Lock’ object.  This seems to be working.  If there’s a better way, please let us know.  If not, please consider adding this feature.  We can create a patch if there’s an interest.  Thanks.
> Ajay Chitre (achitre@cisco.com)
> Virendra Singh (virsingh@cisco.com)
> Anyway, here’s what we’ve done:
> 1)  Created a class that extends Sqoop.java
> public class DlSqoop extends Sqoop {
> 2)  Created a Lock object:
>     private static Lock monitor = new ReentrantLock();
> 3)  Overridden ‘run’ method as follows:
>   public int run(String [] args) {
>     if (options.getConf() == null) {
>       // Configuration wasn't initialized until after the ToolRunner
>       // got us to this point. ToolRunner gave Sqoop itself a Conf
>       // though.
>       options.setConf(getConf());
>     }
>     try {
>       monitor.lock();
>       options = tool.parseArguments(args, null, options, false);
>       tool.appendArgs(this.childPrgmArgs);
>       tool.validateOptions(options);
>     } catch (Exception e) {
>       // Couldn't parse arguments.
>       // Log the stack trace for this exception
>       LOG.debug(e.getMessage(), e);
>       // Print exception message.
>       System.err.println(e.getMessage());
>       // Print the tool usage message and exit.
>       ToolOptions toolOpts = new ToolOptions();
>       tool.configureOptions(toolOpts);
>       tool.printHelp(toolOpts);
>       return 1; // Exit on exception here.
>        } finally {
>             monitor.unlock();
>         }
>     return tool.run(options);
>   }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)