You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Jadhav Shweta <ja...@tcs.com> on 2014/11/13 06:52:41 UTC
Storage plugin configuration issue
Hi,
I have installed Apache drill(apache-drill-0.5.0-incubating) in distributed mode with hadoop cluster as per following link
https://cwiki.apache.org/confluence/display/DRILL/Installing+Drill+in+Distributed+Mode
Even Drill web UI is also accessible.
But i am not able to create new storage plugin or update already present one.
Whenever I hit create button, it again gets initiated to NULL.
P.S. My hadoop cluster is running with different user and I have installed apache drill in /opt/drill path with sudo root user.
Thanks and regards
Shweta Jadhav
Systems Engineer
Tata Consultancy Services Limited
Mailto: jadhav.shweta@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
Re: Storage plugin configuration issue
Posted by Aditya <ad...@gmail.com>.
Shweta,
I believe Steven requested for the text of storage plugin configuration
that you are
typing in the WebUI in attempt to create the new plugin instance.
This is to ensure that the structure of the configuration follows the
correct syntax.
I assume that you have followed[1] to create this configuration.
I also see that your ZooKeeper quorum consists of 4 nodes. Can you confirm
if these
four nodes are part of the same Zookeeper ensemble? Usually it is a bad
idea to run
an even number of ZooKeeper instance[2].
aditya...
[1]
https://cwiki.apache.org/confluence/display/DRILL/Registering+a+File+System
[2] http://hbase.apache.org/book/zookeeper.html#how_many_zks
On Thu, Nov 13, 2014 at 1:01 AM, Jadhav Shweta <ja...@tcs.com>
wrote:
> Hi
>
> I have modified following configuration files
>
> drill-override.conf
>
> drill.exec: {
> cluster-id: "mydrillcluster",
> zk.connect:
> "namenode1:2181,datanode1:2181,datanode2:2181,datanode3:2181",
> debug.error_on_leak: false,
> buffer.size: 6
> }
>
Re: Storage plugin configuration issue
Posted by Jadhav Shweta <ja...@tcs.com>.
Hi
I have modified following configuration files
drill-override.conf
drill.exec: {
cluster-id: "mydrillcluster",
zk.connect: "namenode1:2181,datanode1:2181,datanode2:2181,datanode3:2181",
debug.error_on_leak: false,
buffer.size: 6
}
drill-env.sh
#drill cluster mode
export HADOOP_HOME=/u01/DM_CoE/hadoop/hadoop-2.2.0
I tried putting IP address instead of hostname in drill-override.conf file, but still getting same error in logs.
Is there any configuration file or command to add new storage plugin.
Thanks and regards
Shweta Jadhav
Systems Engineer
Tata Consultancy Services Limited
Mailto: jadhav.shweta@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________
-----Steven Phillips <sp...@maprtech.com> wrote: -----
To: Apache Drill User <dr...@incubator.apache.org>
From: Steven Phillips <sp...@maprtech.com>
Date: 11/13/2014 02:01PM
Subject: Re: Storage plugin configuration issue
I believe the zookeeper error message you are seeing is misleading. This is
a problem that may have been fixed in recent releases (maybe not, I'm not
sure), but it used to be that when the creation of a new storage plugin
fails, the exception is swallowed in the webserver.
Under normal circumstances, the creation is successful, and then the
webserver retrieves the newly created configuration and posts it to the web
ui. In the case where creation fails, the web server still tries to get it,
but it's not there, so that explains the exception you are seeing.
Unfortunately, I don't think there is a good way to find out what the
original exception was without attaching a debugger. Can you post the
configuration you tried to add?
On Wed, Nov 12, 2014 at 10:17 PM, Jadhav Shweta <ja...@tcs.com>
wrote:
> Hi,
>
> Yes I am able to see default storage engine from web UI.
>
> I checked logs
> Its showing
>
> 2014-11-13 11:19:12,527 [qtp1832673977-70] INFO
> o.a.d.e.server.rest.StorageResources - Failure while trying to access
> storage config: shweta
> java.lang.RuntimeException: Failure while accessing Zookeeper
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
> KeeperErrorCode = NoNode for /drill/sys.storage_plugins/shweta
>
> Here I am trying to create storage plugin named shweta.
>
> I have started zookeeper on each node with user 'hadoop'
> And installed drill with used 'dm_user' who is having sudo root access.
>
> Thanks and regards
> Shweta Jadhav
> Systems Engineer
> Tata Consultancy Services Limited
> Mailto: jadhav.shweta@tcs.com
> Website: http://www.tcs.com
> ____________________________________________
> Experience certainty. IT Services
> Business Solutions
> Consulting
> ____________________________________________
>
>
> -----Neeraja Rentachintala <nr...@maprtech.com> wrote: -----
> To: "drill-user@incubator.apache.org" <dr...@incubator.apache.org>
> From: Neeraja Rentachintala <nr...@maprtech.com>
> Date: 11/13/2014 11:34AM
> Subject: Re: Storage plugin configuration issue
>
> Hi Shweta
> Do you see existing (default) storage engine configuration from the web UI.
> Let us know what error are you hitting when try to update. You might find
> more info the logs.
>
> -thanks
>
> On Wed, Nov 12, 2014 at 9:52 PM, Jadhav Shweta <ja...@tcs.com>
> wrote:
>
> >
> > Hi,
> >
> > I have installed Apache drill(apache-drill-0.5.0-incubating) in
> > distributed mode with hadoop cluster as per following link
> >
> >
> https://cwiki.apache.org/confluence/display/DRILL/Installing+Drill+in+Distributed+Mode
> >
> > Even Drill web UI is also accessible.
> >
> > But i am not able to create new storage plugin or update already present
> > one.
> > Whenever I hit create button, it again gets initiated to NULL.
> >
> > P.S. My hadoop cluster is running with different user and I have
> installed
> > apache drill in /opt/drill path with sudo root user.
> >
> >
> > Thanks and regards
> > Shweta Jadhav
> > Systems Engineer
> > Tata Consultancy Services Limited
> > Mailto: jadhav.shweta@tcs.com
> > Website: http://www.tcs.com
> > ____________________________________________
> > Experience certainty. IT Services
> > Business Solutions
> > Consulting
> > ____________________________________________
> > =====-----=====-----=====
> > Notice: The information contained in this e-mail
> > message and/or attachments to it may contain
> > confidential or privileged information. If you are
> > not the intended recipient, any dissemination, use,
> > review, distribution, printing or copying of the
> > information contained in this e-mail message
> > and/or attachments to it are strictly prohibited. If
> > you have received this communication in error,
> > please notify us by reply e-mail or telephone and
> > immediately and permanently delete the message
> > and any attachments. Thank you
> >
> >
> >
>
--
Steven Phillips
Software Engineer
mapr.com
Re: Storage plugin configuration issue
Posted by Steven Phillips <sp...@maprtech.com>.
I believe the zookeeper error message you are seeing is misleading. This is
a problem that may have been fixed in recent releases (maybe not, I'm not
sure), but it used to be that when the creation of a new storage plugin
fails, the exception is swallowed in the webserver.
Under normal circumstances, the creation is successful, and then the
webserver retrieves the newly created configuration and posts it to the web
ui. In the case where creation fails, the web server still tries to get it,
but it's not there, so that explains the exception you are seeing.
Unfortunately, I don't think there is a good way to find out what the
original exception was without attaching a debugger. Can you post the
configuration you tried to add?
On Wed, Nov 12, 2014 at 10:17 PM, Jadhav Shweta <ja...@tcs.com>
wrote:
> Hi,
>
> Yes I am able to see default storage engine from web UI.
>
> I checked logs
> Its showing
>
> 2014-11-13 11:19:12,527 [qtp1832673977-70] INFO
> o.a.d.e.server.rest.StorageResources - Failure while trying to access
> storage config: shweta
> java.lang.RuntimeException: Failure while accessing Zookeeper
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
> KeeperErrorCode = NoNode for /drill/sys.storage_plugins/shweta
>
> Here I am trying to create storage plugin named shweta.
>
> I have started zookeeper on each node with user 'hadoop'
> And installed drill with used 'dm_user' who is having sudo root access.
>
> Thanks and regards
> Shweta Jadhav
> Systems Engineer
> Tata Consultancy Services Limited
> Mailto: jadhav.shweta@tcs.com
> Website: http://www.tcs.com
> ____________________________________________
> Experience certainty. IT Services
> Business Solutions
> Consulting
> ____________________________________________
>
>
> -----Neeraja Rentachintala <nr...@maprtech.com> wrote: -----
> To: "drill-user@incubator.apache.org" <dr...@incubator.apache.org>
> From: Neeraja Rentachintala <nr...@maprtech.com>
> Date: 11/13/2014 11:34AM
> Subject: Re: Storage plugin configuration issue
>
> Hi Shweta
> Do you see existing (default) storage engine configuration from the web UI.
> Let us know what error are you hitting when try to update. You might find
> more info the logs.
>
> -thanks
>
> On Wed, Nov 12, 2014 at 9:52 PM, Jadhav Shweta <ja...@tcs.com>
> wrote:
>
> >
> > Hi,
> >
> > I have installed Apache drill(apache-drill-0.5.0-incubating) in
> > distributed mode with hadoop cluster as per following link
> >
> >
> https://cwiki.apache.org/confluence/display/DRILL/Installing+Drill+in+Distributed+Mode
> >
> > Even Drill web UI is also accessible.
> >
> > But i am not able to create new storage plugin or update already present
> > one.
> > Whenever I hit create button, it again gets initiated to NULL.
> >
> > P.S. My hadoop cluster is running with different user and I have
> installed
> > apache drill in /opt/drill path with sudo root user.
> >
> >
> > Thanks and regards
> > Shweta Jadhav
> > Systems Engineer
> > Tata Consultancy Services Limited
> > Mailto: jadhav.shweta@tcs.com
> > Website: http://www.tcs.com
> > ____________________________________________
> > Experience certainty. IT Services
> > Business Solutions
> > Consulting
> > ____________________________________________
> > =====-----=====-----=====
> > Notice: The information contained in this e-mail
> > message and/or attachments to it may contain
> > confidential or privileged information. If you are
> > not the intended recipient, any dissemination, use,
> > review, distribution, printing or copying of the
> > information contained in this e-mail message
> > and/or attachments to it are strictly prohibited. If
> > you have received this communication in error,
> > please notify us by reply e-mail or telephone and
> > immediately and permanently delete the message
> > and any attachments. Thank you
> >
> >
> >
>
--
Steven Phillips
Software Engineer
mapr.com
Re: Storage plugin configuration issue
Posted by Jadhav Shweta <ja...@tcs.com>.
Hi,
Yes I am able to see default storage engine from web UI.
I checked logs
Its showing
2014-11-13 11:19:12,527 [qtp1832673977-70] INFO o.a.d.e.server.rest.StorageResources - Failure while trying to access storage config: shweta
java.lang.RuntimeException: Failure while accessing Zookeeper
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /drill/sys.storage_plugins/shweta
Here I am trying to create storage plugin named shweta.
I have started zookeeper on each node with user 'hadoop'
And installed drill with used 'dm_user' who is having sudo root access.
Thanks and regards
Shweta Jadhav
Systems Engineer
Tata Consultancy Services Limited
Mailto: jadhav.shweta@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________
-----Neeraja Rentachintala <nr...@maprtech.com> wrote: -----
To: "drill-user@incubator.apache.org" <dr...@incubator.apache.org>
From: Neeraja Rentachintala <nr...@maprtech.com>
Date: 11/13/2014 11:34AM
Subject: Re: Storage plugin configuration issue
Hi Shweta
Do you see existing (default) storage engine configuration from the web UI.
Let us know what error are you hitting when try to update. You might find
more info the logs.
-thanks
On Wed, Nov 12, 2014 at 9:52 PM, Jadhav Shweta <ja...@tcs.com>
wrote:
>
> Hi,
>
> I have installed Apache drill(apache-drill-0.5.0-incubating) in
> distributed mode with hadoop cluster as per following link
>
> https://cwiki.apache.org/confluence/display/DRILL/Installing+Drill+in+Distributed+Mode
>
> Even Drill web UI is also accessible.
>
> But i am not able to create new storage plugin or update already present
> one.
> Whenever I hit create button, it again gets initiated to NULL.
>
> P.S. My hadoop cluster is running with different user and I have installed
> apache drill in /opt/drill path with sudo root user.
>
>
> Thanks and regards
> Shweta Jadhav
> Systems Engineer
> Tata Consultancy Services Limited
> Mailto: jadhav.shweta@tcs.com
> Website: http://www.tcs.com
> ____________________________________________
> Experience certainty. IT Services
> Business Solutions
> Consulting
> ____________________________________________
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>
Re: Storage plugin configuration issue
Posted by Neeraja Rentachintala <nr...@maprtech.com>.
Hi Shweta
Do you see existing (default) storage engine configuration from the web UI.
Let us know what error are you hitting when try to update. You might find
more info the logs.
-thanks
On Wed, Nov 12, 2014 at 9:52 PM, Jadhav Shweta <ja...@tcs.com>
wrote:
>
> Hi,
>
> I have installed Apache drill(apache-drill-0.5.0-incubating) in
> distributed mode with hadoop cluster as per following link
>
> https://cwiki.apache.org/confluence/display/DRILL/Installing+Drill+in+Distributed+Mode
>
> Even Drill web UI is also accessible.
>
> But i am not able to create new storage plugin or update already present
> one.
> Whenever I hit create button, it again gets initiated to NULL.
>
> P.S. My hadoop cluster is running with different user and I have installed
> apache drill in /opt/drill path with sudo root user.
>
>
> Thanks and regards
> Shweta Jadhav
> Systems Engineer
> Tata Consultancy Services Limited
> Mailto: jadhav.shweta@tcs.com
> Website: http://www.tcs.com
> ____________________________________________
> Experience certainty. IT Services
> Business Solutions
> Consulting
> ____________________________________________
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>