You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@impala.apache.org by Hendry Suwanda <su...@gmail.com> on 2019/05/14 04:20:33 UTC
can't submit query after enable admission control
Hi All,
I got below error after enable Impala admission control
ERROR: Rejected query from pool default-pool: request memory needed 20.02
> MB per node is greater than memory available for admission 0 of
> centos-kudu-impala:22000.
>
> Use the MEM_LIMIT query option to indicate how much memory is required per
> node.
>
below is my config:
IMPALA_KUDU_MASTERS=<kudu master>
IMPALA_CATALOG_SERVICE_HOST=<catalog ip>
IMPALA_STATE_STORE_HOST=<state store ip>
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/log/impala
IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} "
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR}
-state_store_port=${IMPALA_STATE_STORE_PORT}"
IMPALA_SERVER_ARGS=" \
-is_coordinator=false \
-is_executor=true \
-scratch_dirs=/tmp/impala/scratch \
-mem_limit=70% \
--queue_wait_timeout_ms=180000 \
--default_pool_max_requests=2 \
--default_pool_max_queued=2 \
--default_pool_mem_limit=3g \
-log_dir=${IMPALA_LOG_DIR} \
-catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
-state_store_port=${IMPALA_STATE_STORE_PORT} \
-state_store_host=${IMPALA_STATE_STORE_HOST} \
-be_port=${IMPALA_BACKEND_PORT} \
-kudu_master_hosts=${IMPALA_KUDU_MASTERS}"
could you suggest me, what i miss?
--
Regards,
Hendry Suwanda
Github: https://github.com/hendrysuwanda
Blog: http://hendrysuwanda.github.io/
Re: can't submit query after enable admission control
Posted by Hendry Suwanda <su...@gmail.com>.
Hi Jeszy,
you right, I'm facing this issue IMPALA-8469, because i have
coordinator-only impala.
already test your suggestion & it's work.
Thank you very much
On Tue, May 14, 2019 at 1:50 PM Jeszy <je...@gmail.com> wrote:
> Hey,
>
> The most likely causes are:
> - your pool was misconfigured and has 0 mb RAM available
> - other running queries use up all the resources allocated to the pool
> - hitting IMPALA-8469, if you have a coordinator-only impalad on a
> recent release. You can work around that one by marking the impalad as
> both executor and coordinator.
>
> FWIW, it's usually more scalable to set the daemon's mem_limit as a
> specific value instead of a percentage. Also, admission control
> configuration is a bit more nuanced than the startup flags you posted
> - see
> https://impala.apache.org/docs/build3x/html/topics/impala_admission.html
> for examples.
>
> HTH
>
> On Tue, May 14, 2019 at 6:20 AM Hendry Suwanda <su...@gmail.com>
> wrote:
> >
> > Hi All,
> > I got below error after enable Impala admission control
> >
> >> ERROR: Rejected query from pool default-pool: request memory needed
> 20.02 MB per node is greater than memory available for admission 0 of
> centos-kudu-impala:22000.
> >>
> >> Use the MEM_LIMIT query option to indicate how much memory is required
> per node.
> >
> >
> > below is my config:
> > IMPALA_KUDU_MASTERS=<kudu master>
> > IMPALA_CATALOG_SERVICE_HOST=<catalog ip>
> > IMPALA_STATE_STORE_HOST=<state store ip>
> > IMPALA_STATE_STORE_PORT=24000
> > IMPALA_BACKEND_PORT=22000
> > IMPALA_LOG_DIR=/var/log/impala
> >
> > IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} "
> > IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR}
> -state_store_port=${IMPALA_STATE_STORE_PORT}"
> > IMPALA_SERVER_ARGS=" \
> > -is_coordinator=false \
> > -is_executor=true \
> > -scratch_dirs=/tmp/impala/scratch \
> > -mem_limit=70% \
> > --queue_wait_timeout_ms=180000 \
> > --default_pool_max_requests=2 \
> > --default_pool_max_queued=2 \
> > --default_pool_mem_limit=3g \
> > -log_dir=${IMPALA_LOG_DIR} \
> > -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
> > -state_store_port=${IMPALA_STATE_STORE_PORT} \
> > -state_store_host=${IMPALA_STATE_STORE_HOST} \
> > -be_port=${IMPALA_BACKEND_PORT} \
> > -kudu_master_hosts=${IMPALA_KUDU_MASTERS}"
> >
> > could you suggest me, what i miss?
> >
> > --
> > Regards,
> >
> >
> > Hendry Suwanda
> >
> > Github: https://github.com/hendrysuwanda
> > Blog: http://hendrysuwanda.github.io/
>
--
Regards,
Hendry Suwanda
Github: https://github.com/hendrysuwanda
Blog: http://hendrysuwanda.github.io/
Re: can't submit query after enable admission control
Posted by Jeszy <je...@gmail.com>.
Hey,
The most likely causes are:
- your pool was misconfigured and has 0 mb RAM available
- other running queries use up all the resources allocated to the pool
- hitting IMPALA-8469, if you have a coordinator-only impalad on a
recent release. You can work around that one by marking the impalad as
both executor and coordinator.
FWIW, it's usually more scalable to set the daemon's mem_limit as a
specific value instead of a percentage. Also, admission control
configuration is a bit more nuanced than the startup flags you posted
- see https://impala.apache.org/docs/build3x/html/topics/impala_admission.html
for examples.
HTH
On Tue, May 14, 2019 at 6:20 AM Hendry Suwanda <su...@gmail.com> wrote:
>
> Hi All,
> I got below error after enable Impala admission control
>
>> ERROR: Rejected query from pool default-pool: request memory needed 20.02 MB per node is greater than memory available for admission 0 of centos-kudu-impala:22000.
>>
>> Use the MEM_LIMIT query option to indicate how much memory is required per node.
>
>
> below is my config:
> IMPALA_KUDU_MASTERS=<kudu master>
> IMPALA_CATALOG_SERVICE_HOST=<catalog ip>
> IMPALA_STATE_STORE_HOST=<state store ip>
> IMPALA_STATE_STORE_PORT=24000
> IMPALA_BACKEND_PORT=22000
> IMPALA_LOG_DIR=/var/log/impala
>
> IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} "
> IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}"
> IMPALA_SERVER_ARGS=" \
> -is_coordinator=false \
> -is_executor=true \
> -scratch_dirs=/tmp/impala/scratch \
> -mem_limit=70% \
> --queue_wait_timeout_ms=180000 \
> --default_pool_max_requests=2 \
> --default_pool_max_queued=2 \
> --default_pool_mem_limit=3g \
> -log_dir=${IMPALA_LOG_DIR} \
> -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
> -state_store_port=${IMPALA_STATE_STORE_PORT} \
> -state_store_host=${IMPALA_STATE_STORE_HOST} \
> -be_port=${IMPALA_BACKEND_PORT} \
> -kudu_master_hosts=${IMPALA_KUDU_MASTERS}"
>
> could you suggest me, what i miss?
>
> --
> Regards,
>
>
> Hendry Suwanda
>
> Github: https://github.com/hendrysuwanda
> Blog: http://hendrysuwanda.github.io/