You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by Dhruv Mahajan <dh...@gmail.com> on 2016/05/09 21:19:52 UTC
HADOOP_CONF_DIR is not configured or does not exist Error
Hi
While running on cluster I got the following error. I am wondering the
nature of error: is it an issue with cluster configuration or some
possible recent issue in REEF?
Thanks
Dhruv
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0 :
2016-05-09T14:15:24.2561144-07:00 0001
ERROR: ExceptionCaught TargetInvocationException encountered error
[System.Reflection.TargetInvocationException: Exception has been thrown by
the target of an invocation. ---> System.ArgumentException: HADOOP_CONF_DIR
is not configured or does not exist.
Parameter name: hadoopConfigDir
at Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider..ctor()
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[]
arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
at
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
plan)] with mesage [Exception has been thrown by the target of an
invocation.] and stack trace [ at
System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments,
Signature sig, Boolean constructor)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
at
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
plan)]
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0 :
2016-05-09T14:15:24.2631206-07:00 0001
ERROR: ExceptionThrowing InjectionException encountered error
[Org.Apache.REEF.Tang.Exceptions.InjectionException: Could not invoke
constructor: new
Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider() --->
System.Reflection.TargetInvocationException: Exception has been thrown by
the target of an invocation. ---> System.ArgumentException: HADOOP_CONF_DIR
is not configured or does not exist.
Parameter name: hadoopConfigDir
at Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider..ctor()
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[]
arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
at
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
plan)
--- End of inner exception stack trace ---] with mesage [Could not
invoke constructor: new
Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider()] and stack
trace []
Re: HADOOP_CONF_DIR is not configured or does not exist Error
Posted by Markus Weimer <ma...@weimo.de>.
Thanks for confirming!
Markus
On Mon, May 9, 2016 at 8:57 PM, Dhruv Mahajan <dh...@gmail.com> wrote:
> Turns out, It happened due to cluster update but is not a REEF bug/issue
> but issue somewhere else.
>
> Dhruv
>
> On Mon, May 9, 2016 at 8:07 PM, Markus Weimer <ma...@weimo.de> wrote:
>
>> Hi,
>>
>> this exception is thrown by the constructor of
>> `MultipleRMUrlProvider`, which indeed requires the HADOOP_CONF
>> environment variable to be set. Is this on a cluster you haven't used
>> before? Or has it recently been upgraded? Hadoop in general is moving
>> away from these environment variables, so an update might have caused
>> the error to appear.
>>
>> Markus
>>
>> On Mon, May 9, 2016 at 2:19 PM, Dhruv Mahajan <dh...@gmail.com>
>> wrote:
>> > Hi
>> >
>> > While running on cluster I got the following error. I am wondering the
>> > nature of error: is it an issue with cluster configuration or some
>> > possible recent issue in REEF?
>> >
>> > Thanks
>> > Dhruv
>> >
>> >
>> > Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0
>> :
>> > 2016-05-09T14:15:24.2561144-07:00 0001
>> > ERROR: ExceptionCaught TargetInvocationException encountered error
>> > [System.Reflection.TargetInvocationException: Exception has been thrown
>> by
>> > the target of an invocation. ---> System.ArgumentException:
>> HADOOP_CONF_DIR
>> > is not configured or does not exist.
>> > Parameter name: hadoopConfigDir
>> > at
>> Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider..ctor()
>> > --- End of inner exception stack trace ---
>> > at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[]
>> > arguments, Signature sig, Boolean constructor)
>> > at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
>> > invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
>> > at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
>> > at
>> >
>> Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
>> > plan)] with mesage [Exception has been thrown by the target of an
>> > invocation.] and stack trace [ at
>> > System.RuntimeMethodHandle.InvokeMethod(Object target, Object[]
>> arguments,
>> > Signature sig, Boolean constructor)
>> > at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
>> > invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
>> > at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
>> > at
>> >
>> Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
>> > plan)]
>> > Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0
>> :
>> > 2016-05-09T14:15:24.2631206-07:00 0001
>> > ERROR: ExceptionThrowing InjectionException encountered error
>> > [Org.Apache.REEF.Tang.Exceptions.InjectionException: Could not invoke
>> > constructor: new
>> > Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider() --->
>> > System.Reflection.TargetInvocationException: Exception has been thrown by
>> > the target of an invocation. ---> System.ArgumentException:
>> HADOOP_CONF_DIR
>> > is not configured or does not exist.
>> > Parameter name: hadoopConfigDir
>> > at
>> Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider..ctor()
>> > --- End of inner exception stack trace ---
>> > at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[]
>> > arguments, Signature sig, Boolean constructor)
>> > at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
>> > invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
>> > at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
>> > at
>> >
>> Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
>> > plan)
>> > --- End of inner exception stack trace ---] with mesage [Could not
>> > invoke constructor: new
>> > Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider()] and stack
>> > trace []
>>
Re: HADOOP_CONF_DIR is not configured or does not exist Error
Posted by Dhruv Mahajan <dh...@gmail.com>.
Turns out, It happened due to cluster update but is not a REEF bug/issue
but issue somewhere else.
Dhruv
On Mon, May 9, 2016 at 8:07 PM, Markus Weimer <ma...@weimo.de> wrote:
> Hi,
>
> this exception is thrown by the constructor of
> `MultipleRMUrlProvider`, which indeed requires the HADOOP_CONF
> environment variable to be set. Is this on a cluster you haven't used
> before? Or has it recently been upgraded? Hadoop in general is moving
> away from these environment variables, so an update might have caused
> the error to appear.
>
> Markus
>
> On Mon, May 9, 2016 at 2:19 PM, Dhruv Mahajan <dh...@gmail.com>
> wrote:
> > Hi
> >
> > While running on cluster I got the following error. I am wondering the
> > nature of error: is it an issue with cluster configuration or some
> > possible recent issue in REEF?
> >
> > Thanks
> > Dhruv
> >
> >
> > Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0
> :
> > 2016-05-09T14:15:24.2561144-07:00 0001
> > ERROR: ExceptionCaught TargetInvocationException encountered error
> > [System.Reflection.TargetInvocationException: Exception has been thrown
> by
> > the target of an invocation. ---> System.ArgumentException:
> HADOOP_CONF_DIR
> > is not configured or does not exist.
> > Parameter name: hadoopConfigDir
> > at
> Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider..ctor()
> > --- End of inner exception stack trace ---
> > at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[]
> > arguments, Signature sig, Boolean constructor)
> > at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
> > invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
> > at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
> > at
> >
> Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
> > plan)] with mesage [Exception has been thrown by the target of an
> > invocation.] and stack trace [ at
> > System.RuntimeMethodHandle.InvokeMethod(Object target, Object[]
> arguments,
> > Signature sig, Boolean constructor)
> > at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
> > invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
> > at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
> > at
> >
> Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
> > plan)]
> > Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0
> :
> > 2016-05-09T14:15:24.2631206-07:00 0001
> > ERROR: ExceptionThrowing InjectionException encountered error
> > [Org.Apache.REEF.Tang.Exceptions.InjectionException: Could not invoke
> > constructor: new
> > Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider() --->
> > System.Reflection.TargetInvocationException: Exception has been thrown by
> > the target of an invocation. ---> System.ArgumentException:
> HADOOP_CONF_DIR
> > is not configured or does not exist.
> > Parameter name: hadoopConfigDir
> > at
> Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider..ctor()
> > --- End of inner exception stack trace ---
> > at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[]
> > arguments, Signature sig, Boolean constructor)
> > at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
> > invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
> > at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
> > at
> >
> Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
> > plan)
> > --- End of inner exception stack trace ---] with mesage [Could not
> > invoke constructor: new
> > Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider()] and stack
> > trace []
>
Re: HADOOP_CONF_DIR is not configured or does not exist Error
Posted by Markus Weimer <ma...@weimo.de>.
Hi,
this exception is thrown by the constructor of
`MultipleRMUrlProvider`, which indeed requires the HADOOP_CONF
environment variable to be set. Is this on a cluster you haven't used
before? Or has it recently been upgraded? Hadoop in general is moving
away from these environment variables, so an update might have caused
the error to appear.
Markus
On Mon, May 9, 2016 at 2:19 PM, Dhruv Mahajan <dh...@gmail.com> wrote:
> Hi
>
> While running on cluster I got the following error. I am wondering the
> nature of error: is it an issue with cluster configuration or some
> possible recent issue in REEF?
>
> Thanks
> Dhruv
>
>
> Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0 :
> 2016-05-09T14:15:24.2561144-07:00 0001
> ERROR: ExceptionCaught TargetInvocationException encountered error
> [System.Reflection.TargetInvocationException: Exception has been thrown by
> the target of an invocation. ---> System.ArgumentException: HADOOP_CONF_DIR
> is not configured or does not exist.
> Parameter name: hadoopConfigDir
> at Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider..ctor()
> --- End of inner exception stack trace ---
> at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[]
> arguments, Signature sig, Boolean constructor)
> at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
> invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
> at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
> at
> Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
> plan)] with mesage [Exception has been thrown by the target of an
> invocation.] and stack trace [ at
> System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments,
> Signature sig, Boolean constructor)
> at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
> invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
> at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
> at
> Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
> plan)]
> Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0 :
> 2016-05-09T14:15:24.2631206-07:00 0001
> ERROR: ExceptionThrowing InjectionException encountered error
> [Org.Apache.REEF.Tang.Exceptions.InjectionException: Could not invoke
> constructor: new
> Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider() --->
> System.Reflection.TargetInvocationException: Exception has been thrown by
> the target of an invocation. ---> System.ArgumentException: HADOOP_CONF_DIR
> is not configured or does not exist.
> Parameter name: hadoopConfigDir
> at Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider..ctor()
> --- End of inner exception stack trace ---
> at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[]
> arguments, Signature sig, Boolean constructor)
> at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
> invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
> at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
> at
> Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
> plan)
> --- End of inner exception stack trace ---] with mesage [Could not
> invoke constructor: new
> Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider()] and stack
> trace []
RE: HADOOP_CONF_DIR is not configured or does not exist Error
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
This is existing REEF code. It reads environment variable HADOOP_CONF_DIR from Yarn environment.
-----Original Message-----
From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
Sent: Monday, May 9, 2016 2:20 PM
To: dev@reef.apache.org
Subject: HADOOP_CONF_DIR is not configured or does not exist Error
Hi
While running on cluster I got the following error. I am wondering the nature of error: is it an issue with cluster configuration or some possible recent issue in REEF?
Thanks
Dhruv
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0 :
2016-05-09T14:15:24.2561144-07:00 0001
ERROR: ExceptionCaught TargetInvocationException encountered error
[System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ArgumentException: HADOOP_CONF_DIR is not configured or does not exist.
Parameter name: hadoopConfigDir
at Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider..ctor()
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
at
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
plan)] with mesage [Exception has been thrown by the target of an
invocation.] and stack trace [ at
System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
at
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
plan)]
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl Error: 0 :
2016-05-09T14:15:24.2631206-07:00 0001
ERROR: ExceptionThrowing InjectionException encountered error
[Org.Apache.REEF.Tang.Exceptions.InjectionException: Could not invoke
constructor: new
Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider() --->
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ArgumentException: HADOOP_CONF_DIR is not configured or does not exist.
Parameter name: hadoopConfigDir
at Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider..ctor()
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags
invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
at
Org.Apache.REEF.Tang.Implementations.InjectionPlan.InjectorImpl.InjectFromPlan(InjectionPlan
plan)
--- End of inner exception stack trace ---] with mesage [Could not invoke constructor: new Org.Apache.REEF.Client.YARN.RestClient.MultipleRMUrlProvider()] and stack trace []