You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by bowen zhang <bo...@yahoo.com> on 2013/11/19 01:01:29 UTC
issue after oozie-1584
Hi guys,
I recently rebased to the latest trunk and tried to run a sample workflow. In the new SharelibService.java, and assuming using all default config, when we try to get the sharelib from DFS in loadShareLibfromDFS(Map<String, List<Path>> shareLibMap), we will get a null back since there is really no path in the packaged oozie sharelib that starts with prefix "lib_". Consequently, in addSystemShareLibForAction() in javaActionExecutor, listOfPaths will be an empty list and throws an outOfBound exception in the following line with "listOfPaths.get(0)". And as a result, sample workflow will all fail on this exception. Puru, or Ryota, do you guys have any thoughts on this or I need to add more configuration for this?
Bowen
Re: issue after oozie-1584
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
Error is thrown while reading system launcher jar(oozie).
No system launcher jar means system failure and we should throw exception.
So it's a correct behavior.
We should throw proper error message.
We also noticed inappropriate error message(like path not found) while
testing sharelib metafile configuration.
I will create a JIRA to throw proper error code and error message for
sharelib exceptions.
Puru.
On 11/19/13 10:13 AM, "Rohini Palaniswamy" <ro...@gmail.com> wrote:
>Puru,
> You still need to fix this in code. Even if
>oozie.action.ship.launcher.jar=true
>you will still get exception if lib_ is not present. If there is no lib_
>directory, then it means user has not setup sharelib and that needs to be
>handled instead of having a outOfBound exception in
>addSystemShareLibForAction().
>
>Regards,
>Rohini
>
>
>On Mon, Nov 18, 2013 at 4:19 PM, Purshotam Shah
><pu...@yahoo-inc.com>wrote:
>
>> Two away to fix it.
>>
>> 1. You need to upload new sharelib. Please use sharelibCLI to create
>>new
>> sharelib, which copies sharelib to lib-<timestamp> directory. At start,
>> server will pick latest sharelib(based on lib_timestamp).
>>
>> 2. Set oozie.action.ship.launcher.jar=true, which will copy system
>> sharelib at server start.
>>
>> To support backward compatibility we should set
>> oozie.action.ship.launcher.jar=true, which will make Oozie server(at
>> start) to copy and load system lib.
>>
>> Puru.
>>
>>
>> On 11/18/13 4:01 PM, "bowen zhang" <bo...@yahoo.com> wrote:
>>
>> >Hi guys,
>> >I recently rebased to the latest trunk and tried to run a sample
>> >workflow. In the new SharelibService.java, and assuming using all
>>default
>> >config, when we try to get the sharelib from DFS in
>> >loadShareLibfromDFS(Map<String, List<Path>> shareLibMap), we will get a
>> >null back since there is really no path in the packaged oozie sharelib
>> >that starts with prefix "lib_". Consequently, in
>> >addSystemShareLibForAction() in javaActionExecutor, listOfPaths will be
>> >an empty list and throws an outOfBound exception in the following line
>> >with "listOfPaths.get(0)". And as a result, sample workflow will all
>>fail
>> >on this exception. Puru, or Ryota, do you guys have any thoughts on
>>this
>> >or I need to add more configuration for this?
>> >Bowen
>>
>>
Re: issue after oozie-1584
Posted by Rohini Palaniswamy <ro...@gmail.com>.
Puru,
You still need to fix this in code. Even if
oozie.action.ship.launcher.jar=true
you will still get exception if lib_ is not present. If there is no lib_
directory, then it means user has not setup sharelib and that needs to be
handled instead of having a outOfBound exception in
addSystemShareLibForAction().
Regards,
Rohini
On Mon, Nov 18, 2013 at 4:19 PM, Purshotam Shah <pu...@yahoo-inc.com>wrote:
> Two away to fix it.
>
> 1. You need to upload new sharelib. Please use sharelibCLI to create new
> sharelib, which copies sharelib to lib-<timestamp> directory. At start,
> server will pick latest sharelib(based on lib_timestamp).
>
> 2. Set oozie.action.ship.launcher.jar=true, which will copy system
> sharelib at server start.
>
> To support backward compatibility we should set
> oozie.action.ship.launcher.jar=true, which will make Oozie server(at
> start) to copy and load system lib.
>
> Puru.
>
>
> On 11/18/13 4:01 PM, "bowen zhang" <bo...@yahoo.com> wrote:
>
> >Hi guys,
> >I recently rebased to the latest trunk and tried to run a sample
> >workflow. In the new SharelibService.java, and assuming using all default
> >config, when we try to get the sharelib from DFS in
> >loadShareLibfromDFS(Map<String, List<Path>> shareLibMap), we will get a
> >null back since there is really no path in the packaged oozie sharelib
> >that starts with prefix "lib_". Consequently, in
> >addSystemShareLibForAction() in javaActionExecutor, listOfPaths will be
> >an empty list and throws an outOfBound exception in the following line
> >with "listOfPaths.get(0)". And as a result, sample workflow will all fail
> >on this exception. Puru, or Ryota, do you guys have any thoughts on this
> >or I need to add more configuration for this?
> >Bowen
>
>
Re: issue after oozie-1584
Posted by Purshotam Shah <pu...@yahoo-inc.com>.
Two away to fix it.
1. You need to upload new sharelib. Please use sharelibCLI to create new
sharelib, which copies sharelib to lib-<timestamp> directory. At start,
server will pick latest sharelib(based on lib_timestamp).
2. Set oozie.action.ship.launcher.jar=true, which will copy system
sharelib at server start.
To support backward compatibility we should set
oozie.action.ship.launcher.jar=true, which will make Oozie server(at
start) to copy and load system lib.
Puru.
On 11/18/13 4:01 PM, "bowen zhang" <bo...@yahoo.com> wrote:
>Hi guys,
>I recently rebased to the latest trunk and tried to run a sample
>workflow. In the new SharelibService.java, and assuming using all default
>config, when we try to get the sharelib from DFS in
>loadShareLibfromDFS(Map<String, List<Path>> shareLibMap), we will get a
>null back since there is really no path in the packaged oozie sharelib
>that starts with prefix "lib_". Consequently, in
>addSystemShareLibForAction() in javaActionExecutor, listOfPaths will be
>an empty list and throws an outOfBound exception in the following line
>with "listOfPaths.get(0)". And as a result, sample workflow will all fail
>on this exception. Puru, or Ryota, do you guys have any thoughts on this
>or I need to add more configuration for this?
>Bowen