You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by Aayush Bhan <aa...@gmail.com> on 2020/07/17 14:05:44 UTC
Impala - 5314
Hi Impala Dev Team,
I was assigned the ticket - 5314 on jira. As a first time contributor, I
was going through the cwiki links on how to get started. So I was setting
up my development environment in a docker container. I went ahead with all
the steps that were given. So every time I try to run the
bootstrap_development script, it leads to some build errors which i'm not
able to ascertain how to fix.
The log i'm being directed to after running the script tells me the
following -
Error in /home/impdev/Impala/testdata/bin/create-load-data.sh at line 85: fi
I got the following output while running the bootstrap_development script -
https://gist.github.com/aayushbhan/dee15ba212225e780196e2964d279681
I would really appreciate it if anyone could tell me any resources or
anything of the sort to help me fix this issue and get my development
environment set up.
Thanks,
Aayush
Re: Impala - 5314
Posted by Shant Hovsepian <sh...@superdupershant.com>.
Hi Aayush, you might want to check logs/cluster/impalad.INFO for an error
related to the impala service.
Keep in mind though from the output you had in your gist, everything
compiles and the local impala came up, what's failing is loading the
initial test data which can be quite large at times.
For example see the info here
https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala
[image: build_impala.png]
At this point you can run that last command to build and spin up the system
for testing.
-Shant
On Tue, Jul 21, 2020 at 12:31 PM Aayush Bhan <aa...@gmail.com> wrote:
> Hi Quanlong,
> I went through the 3 log files and it gave me 2 different errors.
>
> The 1st one, i.e.
>
> /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log
>
> INNER EXCEPTION: <class 'socket.error'>
> MESSAGE: [Errno 104] Connection reset by peer
>
> This error came up while - INSERT INTO TABLE tpch_kudu.lineitem SELECT *
> FROM tpch.lineitem
>
> In the second and third log files, i.e.
>
>
> /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log
> and
>
> /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log
>
> INNER EXCEPTION: <class 'thrift.transport.TTransport.TTransportException'>
> MESSAGE: Could not connect to localhost:21000
>
> Thanks,
> Aayush
>
> On Mon, Jul 20, 2020 at 7:30 PM Quanlong Huang <hu...@gmail.com>
> wrote:
>
> > Hi Aayush,
> >
> > You need to look into some log files for more details. In the console
> > output you shared, there are three log files mentioned about the
> failures.
> >
> > One is the failure on loading tpch dataset:
> >
> > > Loading TPC-H data (logging to
> > > /home/impdev/Impala/logs/data_loading/load-tpch.log)...
> > > Loading TPC-DS data (logging to
> > > /home/impdev/Impala/logs/data_loading/load-tpcds.log)...
> > > FAILED (Took: 8 min 8 sec)
> > > 'load-data tpch core' failed. Tail of log:
> > > 'load-data tpch core' failed. Tail of log:
> > > ......
> > > 11:51:28 Error executing impala SQL:
> > >
> >
> /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql
> > > See:
> > >
> >
> /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log
> >
> > You should look
> > into
> >
> /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log
> > for more details.
> >
> > The other one is the failure on loading tpcds dataset:
> >
> > > FAILED (Took: 15 min 28 sec)
> > > 'load-data tpcds core' failed. Tail of log:
> > > ......
> > > 11:58:47 Error executing impala SQL:
> > >
> >
> /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql
> > > See:
> > >
> >
> /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log
> >
> > You should look
> > into
> >
> /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log
> > for more details.
> >
> > The last one is the failure on loading functional-query dataset:
> >
> > > FAILED (Took: 33 min 22 sec)
> > > 'load-data functional-query exhaustive' failed. Tail of log:
> > > ......
> > > 12:16:42 Error executing impala SQL:
> > >
> >
> /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql
> > > See:
> > >
> >
> /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log
> >
> > You should look
> > into
> >
> /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log
> > for more details.
> >
> > Some common reasons are not enough disk spaces or memory. If these log
> > files containing failures on executing hive queries. You may need to look
> > into hive logs for more details. They are in
> > $IMPALA_HOME/logs/cluster/hive*
> >
> > Quanlong
> >
> > On Mon, Jul 20, 2020 at 6:22 PM Aayush Bhan <aa...@gmail.com>
> wrote:
> >
> > > Hey Jim,
> > > I looked into what you said and there was just enough disk space on my
> > > system.
> > > So I figured I should clean it up a bit. I tried building again after
> > > having sufficient
> > > disk space. However, the log file seems to be giving me the same
> problem
> > as
> > > before.
> > > Is it possible there could be some other reason for encountering this?
> > >
> > > Thanks,
> > > Aayush
> > >
> > > On Fri, Jul 17, 2020 at 8:10 PM Jim Apple <ap...@jbapple.com> wrote:
> > >
> > > > In my experience, a failure in the master branch during data load
> > > indicated
> > > > being out of disk space. This sometimes has manifested even when df
> -h
> > > > shows just enough space, so this was successful for me most often
> when
> > I
> > > > had more than enough space.
> > > >
> > > > On Fri, Jul 17, 2020 at 7:06 AM Aayush Bhan <aa...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Impala Dev Team,
> > > > >
> > > > > I was assigned the ticket - 5314 on jira. As a first time
> > contributor,
> > > I
> > > > > was going through the cwiki links on how to get started. So I was
> > > setting
> > > > > up my development environment in a docker container. I went ahead
> > with
> > > > all
> > > > > the steps that were given. So every time I try to run the
> > > > > bootstrap_development script, it leads to some build errors which
> i'm
> > > not
> > > > > able to ascertain how to fix.
> > > > >
> > > > > The log i'm being directed to after running the script tells me the
> > > > > following -
> > > > > Error in /home/impdev/Impala/testdata/bin/create-load-data.sh at
> line
> > > 85:
> > > > > fi
> > > > >
> > > > > I got the following output while running the bootstrap_development
> > > > script -
> > > > >
> https://gist.github.com/aayushbhan/dee15ba212225e780196e2964d279681
> > > > >
> > > > > I would really appreciate it if anyone could tell me any resources
> or
> > > > > anything of the sort to help me fix this issue and get my
> development
> > > > > environment set up.
> > > > >
> > > > > Thanks,
> > > > > Aayush
> > > > >
> > > >
> > >
> > >
> > > --
> > > aayushbhan
> > >
> >
>
>
> --
> aayushbhan
>
Re: Impala - 5314
Posted by Aayush Bhan <aa...@gmail.com>.
Hi Quanlong,
I went through the 3 log files and it gave me 2 different errors.
The 1st one, i.e.
/home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log
INNER EXCEPTION: <class 'socket.error'>
MESSAGE: [Errno 104] Connection reset by peer
This error came up while - INSERT INTO TABLE tpch_kudu.lineitem SELECT *
FROM tpch.lineitem
In the second and third log files, i.e.
/home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log
and
/home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log
INNER EXCEPTION: <class 'thrift.transport.TTransport.TTransportException'>
MESSAGE: Could not connect to localhost:21000
Thanks,
Aayush
On Mon, Jul 20, 2020 at 7:30 PM Quanlong Huang <hu...@gmail.com>
wrote:
> Hi Aayush,
>
> You need to look into some log files for more details. In the console
> output you shared, there are three log files mentioned about the failures.
>
> One is the failure on loading tpch dataset:
>
> > Loading TPC-H data (logging to
> > /home/impdev/Impala/logs/data_loading/load-tpch.log)...
> > Loading TPC-DS data (logging to
> > /home/impdev/Impala/logs/data_loading/load-tpcds.log)...
> > FAILED (Took: 8 min 8 sec)
> > 'load-data tpch core' failed. Tail of log:
> > 'load-data tpch core' failed. Tail of log:
> > ......
> > 11:51:28 Error executing impala SQL:
> >
> /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql
> > See:
> >
> /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log
>
> You should look
> into
> /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log
> for more details.
>
> The other one is the failure on loading tpcds dataset:
>
> > FAILED (Took: 15 min 28 sec)
> > 'load-data tpcds core' failed. Tail of log:
> > ......
> > 11:58:47 Error executing impala SQL:
> >
> /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql
> > See:
> >
> /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log
>
> You should look
> into
> /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log
> for more details.
>
> The last one is the failure on loading functional-query dataset:
>
> > FAILED (Took: 33 min 22 sec)
> > 'load-data functional-query exhaustive' failed. Tail of log:
> > ......
> > 12:16:42 Error executing impala SQL:
> >
> /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql
> > See:
> >
> /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log
>
> You should look
> into
> /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log
> for more details.
>
> Some common reasons are not enough disk spaces or memory. If these log
> files containing failures on executing hive queries. You may need to look
> into hive logs for more details. They are in
> $IMPALA_HOME/logs/cluster/hive*
>
> Quanlong
>
> On Mon, Jul 20, 2020 at 6:22 PM Aayush Bhan <aa...@gmail.com> wrote:
>
> > Hey Jim,
> > I looked into what you said and there was just enough disk space on my
> > system.
> > So I figured I should clean it up a bit. I tried building again after
> > having sufficient
> > disk space. However, the log file seems to be giving me the same problem
> as
> > before.
> > Is it possible there could be some other reason for encountering this?
> >
> > Thanks,
> > Aayush
> >
> > On Fri, Jul 17, 2020 at 8:10 PM Jim Apple <ap...@jbapple.com> wrote:
> >
> > > In my experience, a failure in the master branch during data load
> > indicated
> > > being out of disk space. This sometimes has manifested even when df -h
> > > shows just enough space, so this was successful for me most often when
> I
> > > had more than enough space.
> > >
> > > On Fri, Jul 17, 2020 at 7:06 AM Aayush Bhan <aa...@gmail.com>
> > wrote:
> > >
> > > > Hi Impala Dev Team,
> > > >
> > > > I was assigned the ticket - 5314 on jira. As a first time
> contributor,
> > I
> > > > was going through the cwiki links on how to get started. So I was
> > setting
> > > > up my development environment in a docker container. I went ahead
> with
> > > all
> > > > the steps that were given. So every time I try to run the
> > > > bootstrap_development script, it leads to some build errors which i'm
> > not
> > > > able to ascertain how to fix.
> > > >
> > > > The log i'm being directed to after running the script tells me the
> > > > following -
> > > > Error in /home/impdev/Impala/testdata/bin/create-load-data.sh at line
> > 85:
> > > > fi
> > > >
> > > > I got the following output while running the bootstrap_development
> > > script -
> > > > https://gist.github.com/aayushbhan/dee15ba212225e780196e2964d279681
> > > >
> > > > I would really appreciate it if anyone could tell me any resources or
> > > > anything of the sort to help me fix this issue and get my development
> > > > environment set up.
> > > >
> > > > Thanks,
> > > > Aayush
> > > >
> > >
> >
> >
> > --
> > aayushbhan
> >
>
--
aayushbhan
Re: Impala - 5314
Posted by Quanlong Huang <hu...@gmail.com>.
Hi Aayush,
You need to look into some log files for more details. In the console
output you shared, there are three log files mentioned about the failures.
One is the failure on loading tpch dataset:
> Loading TPC-H data (logging to
> /home/impdev/Impala/logs/data_loading/load-tpch.log)...
> Loading TPC-DS data (logging to
> /home/impdev/Impala/logs/data_loading/load-tpcds.log)...
> FAILED (Took: 8 min 8 sec)
> 'load-data tpch core' failed. Tail of log:
> 'load-data tpch core' failed. Tail of log:
> ......
> 11:51:28 Error executing impala SQL:
> /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql
> See:
> /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log
You should look
into /home/impdev/Impala/logs/data_loading/sql/tpch/load-tpch-core-impala-generated-kudu-none-none.sql.log
for more details.
The other one is the failure on loading tpcds dataset:
> FAILED (Took: 15 min 28 sec)
> 'load-data tpcds core' failed. Tail of log:
> ......
> 11:58:47 Error executing impala SQL:
> /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql
> See:
> /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log
You should look
into /home/impdev/Impala/logs/data_loading/sql/tpcds/invalidate-tpcds-core-impala-generated.sql.log
for more details.
The last one is the failure on loading functional-query dataset:
> FAILED (Took: 33 min 22 sec)
> 'load-data functional-query exhaustive' failed. Tail of log:
> ......
> 12:16:42 Error executing impala SQL:
> /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql
> See:
> /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log
You should look
into /home/impdev/Impala/logs/data_loading/sql/functional/invalidate-functional-query-exhaustive-impala-generated.sql.log
for more details.
Some common reasons are not enough disk spaces or memory. If these log
files containing failures on executing hive queries. You may need to look
into hive logs for more details. They are in $IMPALA_HOME/logs/cluster/hive*
Quanlong
On Mon, Jul 20, 2020 at 6:22 PM Aayush Bhan <aa...@gmail.com> wrote:
> Hey Jim,
> I looked into what you said and there was just enough disk space on my
> system.
> So I figured I should clean it up a bit. I tried building again after
> having sufficient
> disk space. However, the log file seems to be giving me the same problem as
> before.
> Is it possible there could be some other reason for encountering this?
>
> Thanks,
> Aayush
>
> On Fri, Jul 17, 2020 at 8:10 PM Jim Apple <ap...@jbapple.com> wrote:
>
> > In my experience, a failure in the master branch during data load
> indicated
> > being out of disk space. This sometimes has manifested even when df -h
> > shows just enough space, so this was successful for me most often when I
> > had more than enough space.
> >
> > On Fri, Jul 17, 2020 at 7:06 AM Aayush Bhan <aa...@gmail.com>
> wrote:
> >
> > > Hi Impala Dev Team,
> > >
> > > I was assigned the ticket - 5314 on jira. As a first time contributor,
> I
> > > was going through the cwiki links on how to get started. So I was
> setting
> > > up my development environment in a docker container. I went ahead with
> > all
> > > the steps that were given. So every time I try to run the
> > > bootstrap_development script, it leads to some build errors which i'm
> not
> > > able to ascertain how to fix.
> > >
> > > The log i'm being directed to after running the script tells me the
> > > following -
> > > Error in /home/impdev/Impala/testdata/bin/create-load-data.sh at line
> 85:
> > > fi
> > >
> > > I got the following output while running the bootstrap_development
> > script -
> > > https://gist.github.com/aayushbhan/dee15ba212225e780196e2964d279681
> > >
> > > I would really appreciate it if anyone could tell me any resources or
> > > anything of the sort to help me fix this issue and get my development
> > > environment set up.
> > >
> > > Thanks,
> > > Aayush
> > >
> >
>
>
> --
> aayushbhan
>
Re: Impala - 5314
Posted by Aayush Bhan <aa...@gmail.com>.
Hey Jim,
I looked into what you said and there was just enough disk space on my
system.
So I figured I should clean it up a bit. I tried building again after
having sufficient
disk space. However, the log file seems to be giving me the same problem as
before.
Is it possible there could be some other reason for encountering this?
Thanks,
Aayush
On Fri, Jul 17, 2020 at 8:10 PM Jim Apple <ap...@jbapple.com> wrote:
> In my experience, a failure in the master branch during data load indicated
> being out of disk space. This sometimes has manifested even when df -h
> shows just enough space, so this was successful for me most often when I
> had more than enough space.
>
> On Fri, Jul 17, 2020 at 7:06 AM Aayush Bhan <aa...@gmail.com> wrote:
>
> > Hi Impala Dev Team,
> >
> > I was assigned the ticket - 5314 on jira. As a first time contributor, I
> > was going through the cwiki links on how to get started. So I was setting
> > up my development environment in a docker container. I went ahead with
> all
> > the steps that were given. So every time I try to run the
> > bootstrap_development script, it leads to some build errors which i'm not
> > able to ascertain how to fix.
> >
> > The log i'm being directed to after running the script tells me the
> > following -
> > Error in /home/impdev/Impala/testdata/bin/create-load-data.sh at line 85:
> > fi
> >
> > I got the following output while running the bootstrap_development
> script -
> > https://gist.github.com/aayushbhan/dee15ba212225e780196e2964d279681
> >
> > I would really appreciate it if anyone could tell me any resources or
> > anything of the sort to help me fix this issue and get my development
> > environment set up.
> >
> > Thanks,
> > Aayush
> >
>
--
aayushbhan
Re: Impala - 5314
Posted by Jim Apple <ap...@jbapple.com>.
In my experience, a failure in the master branch during data load indicated
being out of disk space. This sometimes has manifested even when df -h
shows just enough space, so this was successful for me most often when I
had more than enough space.
On Fri, Jul 17, 2020 at 7:06 AM Aayush Bhan <aa...@gmail.com> wrote:
> Hi Impala Dev Team,
>
> I was assigned the ticket - 5314 on jira. As a first time contributor, I
> was going through the cwiki links on how to get started. So I was setting
> up my development environment in a docker container. I went ahead with all
> the steps that were given. So every time I try to run the
> bootstrap_development script, it leads to some build errors which i'm not
> able to ascertain how to fix.
>
> The log i'm being directed to after running the script tells me the
> following -
> Error in /home/impdev/Impala/testdata/bin/create-load-data.sh at line 85:
> fi
>
> I got the following output while running the bootstrap_development script -
> https://gist.github.com/aayushbhan/dee15ba212225e780196e2964d279681
>
> I would really appreciate it if anyone could tell me any resources or
> anything of the sort to help me fix this issue and get my development
> environment set up.
>
> Thanks,
> Aayush
>