You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Boris Shulman <sh...@gmail.com> on 2012/07/11 20:42:43 UTC
Re: Failing Hive Action
Hi,
The problem I think I have is that on every TT node there is a
hive-site/hive-default files, that are probably picked up and used instead
of the setting I want. As those files point to a local derby my hive
actions fail..
Any suggestions how can I change this behavior?
Boris.
On Tue, Jul 10, 2012 at 8:47 PM, Boris Shulman <sh...@gmail.com> wrote:
> Hey Harsh,
>
> Is it possible that local hive-default.xml file is picked up from the TT
> machine? If yes how can I fix this behavior as it is not possible for me to
> change all the hive-default.xml files on all the nodes. The hive
> configuration options show the correct values and not the localhost ones.
>
> Thanks,
>
> Boris.
>
> On Tue, Jul 10, 2012 at 8:07 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Hey Boris,
>>
>> What do you specify as the hostname for your DB in the Hive
>> configuration? Do you specify 'localhost'? If so, it will not work in
>> a distributed cluster, as Oozie's actions are run as Jobs with tasks
>> executing on TT machines (remotely), and they can't all have
>> connections to localhost for a DB being valid logically.
>>
>> On Wed, Jul 11, 2012 at 8:22 AM, Boris Shulman <sh...@gmail.com>
>> wrote:
>> > Hi my hive action fails with:
>> >
>> > Failing Oozie Launcher, Main class
>> > [org.apache.oozie.action.hadoop.HiveMain], exit code [10]
>> >
>> >
>> > javax.jdo.JDOFatalDataStoreException: java.net.ConnectException :
>> > Error connecting to server localhost on port 1527 with message
>> > Connection refused
>> >
>> >
>> > it looks like it can't pick-up the right hive-defaults.xml file. I have
>> > changed the hive metastore to a SQL db in every possible place
>> (hive-site,
>> > hive-defaults), I also provided hive-default via job-config element,
>> but it
>> > still using the default value that I can't find where it is coming from.
>> >
>> > Any suggestions where it can pick the value from (I am using oozie 3.2.0
>> > and hive 0.7.1)?
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: Failing Hive Action
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Boris,
Can you send the oozie hive launcher log? that has info about the classpath
and local files that could help.
thx
On Wed, Jul 11, 2012 at 12:36 PM, Boris Shulman <sh...@gmail.com> wrote:
> The problem is that they are already there (and I have a lot of nodes).
>
> On Wed, Jul 11, 2012 at 11:56 AM, Alejandro Abdelnur <tucu@cloudera.com
> >wrote:
>
> > Why do you have hive*xml files on your nodes? Those are not required by
> > Hive, nor Oozie. If you get rid of them you should be all set.
> >
> > On Wed, Jul 11, 2012 at 11:42 AM, Boris Shulman <sh...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > The problem I think I have is that on every TT node there is a
> > > hive-site/hive-default files, that are probably picked up and used
> > instead
> > > of the setting I want. As those files point to a local derby my hive
> > > actions fail..
> > >
> > > Any suggestions how can I change this behavior?
> > >
> > > Boris.
> > >
> > > On Tue, Jul 10, 2012 at 8:47 PM, Boris Shulman <sh...@gmail.com>
> > wrote:
> > >
> > > > Hey Harsh,
> > > >
> > > > Is it possible that local hive-default.xml file is picked up from the
> > TT
> > > > machine? If yes how can I fix this behavior as it is not possible for
> > me
> > > to
> > > > change all the hive-default.xml files on all the nodes. The hive
> > > > configuration options show the correct values and not the localhost
> > ones.
> > > >
> > > > Thanks,
> > > >
> > > > Boris.
> > > >
> > > > On Tue, Jul 10, 2012 at 8:07 PM, Harsh J <ha...@cloudera.com> wrote:
> > > >
> > > >> Hey Boris,
> > > >>
> > > >> What do you specify as the hostname for your DB in the Hive
> > > >> configuration? Do you specify 'localhost'? If so, it will not work
> in
> > > >> a distributed cluster, as Oozie's actions are run as Jobs with tasks
> > > >> executing on TT machines (remotely), and they can't all have
> > > >> connections to localhost for a DB being valid logically.
> > > >>
> > > >> On Wed, Jul 11, 2012 at 8:22 AM, Boris Shulman <sh...@gmail.com>
> > > >> wrote:
> > > >> > Hi my hive action fails with:
> > > >> >
> > > >> > Failing Oozie Launcher, Main class
> > > >> > [org.apache.oozie.action.hadoop.HiveMain], exit code [10]
> > > >> >
> > > >> >
> > > >> > javax.jdo.JDOFatalDataStoreException: java.net.ConnectException :
> > > >> > Error connecting to server localhost on port 1527 with message
> > > >> > Connection refused
> > > >> >
> > > >> >
> > > >> > it looks like it can't pick-up the right hive-defaults.xml file. I
> > > have
> > > >> > changed the hive metastore to a SQL db in every possible place
> > > >> (hive-site,
> > > >> > hive-defaults), I also provided hive-default via job-config
> element,
> > > >> but it
> > > >> > still using the default value that I can't find where it is coming
> > > from.
> > > >> >
> > > >> > Any suggestions where it can pick the value from (I am using oozie
> > > 3.2.0
> > > >> > and hive 0.7.1)?
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Harsh J
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Alejandro
> >
>
--
Alejandro
Re: Failing Hive Action
Posted by Boris Shulman <sh...@gmail.com>.
The problem is that they are already there (and I have a lot of nodes).
On Wed, Jul 11, 2012 at 11:56 AM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
> Why do you have hive*xml files on your nodes? Those are not required by
> Hive, nor Oozie. If you get rid of them you should be all set.
>
> On Wed, Jul 11, 2012 at 11:42 AM, Boris Shulman <sh...@gmail.com>
> wrote:
>
> > Hi,
> >
> > The problem I think I have is that on every TT node there is a
> > hive-site/hive-default files, that are probably picked up and used
> instead
> > of the setting I want. As those files point to a local derby my hive
> > actions fail..
> >
> > Any suggestions how can I change this behavior?
> >
> > Boris.
> >
> > On Tue, Jul 10, 2012 at 8:47 PM, Boris Shulman <sh...@gmail.com>
> wrote:
> >
> > > Hey Harsh,
> > >
> > > Is it possible that local hive-default.xml file is picked up from the
> TT
> > > machine? If yes how can I fix this behavior as it is not possible for
> me
> > to
> > > change all the hive-default.xml files on all the nodes. The hive
> > > configuration options show the correct values and not the localhost
> ones.
> > >
> > > Thanks,
> > >
> > > Boris.
> > >
> > > On Tue, Jul 10, 2012 at 8:07 PM, Harsh J <ha...@cloudera.com> wrote:
> > >
> > >> Hey Boris,
> > >>
> > >> What do you specify as the hostname for your DB in the Hive
> > >> configuration? Do you specify 'localhost'? If so, it will not work in
> > >> a distributed cluster, as Oozie's actions are run as Jobs with tasks
> > >> executing on TT machines (remotely), and they can't all have
> > >> connections to localhost for a DB being valid logically.
> > >>
> > >> On Wed, Jul 11, 2012 at 8:22 AM, Boris Shulman <sh...@gmail.com>
> > >> wrote:
> > >> > Hi my hive action fails with:
> > >> >
> > >> > Failing Oozie Launcher, Main class
> > >> > [org.apache.oozie.action.hadoop.HiveMain], exit code [10]
> > >> >
> > >> >
> > >> > javax.jdo.JDOFatalDataStoreException: java.net.ConnectException :
> > >> > Error connecting to server localhost on port 1527 with message
> > >> > Connection refused
> > >> >
> > >> >
> > >> > it looks like it can't pick-up the right hive-defaults.xml file. I
> > have
> > >> > changed the hive metastore to a SQL db in every possible place
> > >> (hive-site,
> > >> > hive-defaults), I also provided hive-default via job-config element,
> > >> but it
> > >> > still using the default value that I can't find where it is coming
> > from.
> > >> >
> > >> > Any suggestions where it can pick the value from (I am using oozie
> > 3.2.0
> > >> > and hive 0.7.1)?
> > >>
> > >>
> > >>
> > >> --
> > >> Harsh J
> > >>
> > >
> > >
> >
>
>
>
> --
> Alejandro
>
Re: Failing Hive Action
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Why do you have hive*xml files on your nodes? Those are not required by
Hive, nor Oozie. If you get rid of them you should be all set.
On Wed, Jul 11, 2012 at 11:42 AM, Boris Shulman <sh...@gmail.com> wrote:
> Hi,
>
> The problem I think I have is that on every TT node there is a
> hive-site/hive-default files, that are probably picked up and used instead
> of the setting I want. As those files point to a local derby my hive
> actions fail..
>
> Any suggestions how can I change this behavior?
>
> Boris.
>
> On Tue, Jul 10, 2012 at 8:47 PM, Boris Shulman <sh...@gmail.com> wrote:
>
> > Hey Harsh,
> >
> > Is it possible that local hive-default.xml file is picked up from the TT
> > machine? If yes how can I fix this behavior as it is not possible for me
> to
> > change all the hive-default.xml files on all the nodes. The hive
> > configuration options show the correct values and not the localhost ones.
> >
> > Thanks,
> >
> > Boris.
> >
> > On Tue, Jul 10, 2012 at 8:07 PM, Harsh J <ha...@cloudera.com> wrote:
> >
> >> Hey Boris,
> >>
> >> What do you specify as the hostname for your DB in the Hive
> >> configuration? Do you specify 'localhost'? If so, it will not work in
> >> a distributed cluster, as Oozie's actions are run as Jobs with tasks
> >> executing on TT machines (remotely), and they can't all have
> >> connections to localhost for a DB being valid logically.
> >>
> >> On Wed, Jul 11, 2012 at 8:22 AM, Boris Shulman <sh...@gmail.com>
> >> wrote:
> >> > Hi my hive action fails with:
> >> >
> >> > Failing Oozie Launcher, Main class
> >> > [org.apache.oozie.action.hadoop.HiveMain], exit code [10]
> >> >
> >> >
> >> > javax.jdo.JDOFatalDataStoreException: java.net.ConnectException :
> >> > Error connecting to server localhost on port 1527 with message
> >> > Connection refused
> >> >
> >> >
> >> > it looks like it can't pick-up the right hive-defaults.xml file. I
> have
> >> > changed the hive metastore to a SQL db in every possible place
> >> (hive-site,
> >> > hive-defaults), I also provided hive-default via job-config element,
> >> but it
> >> > still using the default value that I can't find where it is coming
> from.
> >> >
> >> > Any suggestions where it can pick the value from (I am using oozie
> 3.2.0
> >> > and hive 0.7.1)?
> >>
> >>
> >>
> >> --
> >> Harsh J
> >>
> >
> >
>
--
Alejandro