You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2023/08/04 19:57:00 UTC
[jira] [Updated] (HIVE-27566) Fix some yarn cluster options for tests
[ https://issues.apache.org/jira/browse/HIVE-27566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
László Bodor updated HIVE-27566:
--------------------------------
Description:
Currently, on apache master, running a minihs2 and trying to insert a record fails, because the memory defined for node managers are less then what a query asks for.
How to reproduce
{code}
mvn test -Dtest=StartMiniHS2Cluster -DminiHS2.clusterType=Tez -DminiHS2.run=true -DminiHS2.usePortsFromConf=true -Dpackaging.minimizeJar=false -T 1C -DskipShade -Dremoteresources.skip=true -Dmaven.javadoc.skip=true -Denforcer.skip=true -pl itests/hive-unit -Pitests
mvn clean install -DskipTests -Pitests
beeline -u "jdbc:hive2://localhost:10000/default" -n $USER
CREATE TABLE test_part(id int)
PARTITIONED BY(dt string)
STORED AS ORC;
insert into test_part values (1, '1');
{code}
leads to:
{code}
Error: Error while compiling statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask. org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Invalid resource request! Cannot allocate containers as requested resource is greater than maximum allowed allocation. Requested resource type=[memory-mb], Requested resource=<memory:1536, vCores:1>, maximum allowed allocation=<memory:512, vCores:4>, please note that maximum allowed allocation is calculated by scheduler based on maximum resource of registered NodeManagers, which might be less than configured maximum allocation=<memory:512, vCores:4>
{code}
> Fix some yarn cluster options for tests
> ---------------------------------------
>
> Key: HIVE-27566
> URL: https://issues.apache.org/jira/browse/HIVE-27566
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
>
> Currently, on apache master, running a minihs2 and trying to insert a record fails, because the memory defined for node managers are less then what a query asks for.
> How to reproduce
> {code}
> mvn test -Dtest=StartMiniHS2Cluster -DminiHS2.clusterType=Tez -DminiHS2.run=true -DminiHS2.usePortsFromConf=true -Dpackaging.minimizeJar=false -T 1C -DskipShade -Dremoteresources.skip=true -Dmaven.javadoc.skip=true -Denforcer.skip=true -pl itests/hive-unit -Pitests
> mvn clean install -DskipTests -Pitests
> beeline -u "jdbc:hive2://localhost:10000/default" -n $USER
> CREATE TABLE test_part(id int)
> PARTITIONED BY(dt string)
> STORED AS ORC;
> insert into test_part values (1, '1');
> {code}
> leads to:
> {code}
> Error: Error while compiling statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask. org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Invalid resource request! Cannot allocate containers as requested resource is greater than maximum allowed allocation. Requested resource type=[memory-mb], Requested resource=<memory:1536, vCores:1>, maximum allowed allocation=<memory:512, vCores:4>, please note that maximum allowed allocation is calculated by scheduler based on maximum resource of registered NodeManagers, which might be less than configured maximum allocation=<memory:512, vCores:4>
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)