You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by onmstester onmstester via user <us...@hive.apache.org> on 2023/12/24 08:09:31 UTC

Hive Tez llap creates/kills many containers after kerberized the cluster

Hi,





I'm trying to setup LLAP on Hive 3.1.3, its been enable on non-secure cluster successfully and it prints out that llap daemon instead of container while running insert into table, although it allocates a separate Tez container for every simple insert (about +10 seconds for a simple insert). I setup LLAP on HDP 3 and it did not create a seperate container for every inserts and simple inserts took less than a second in HDP/LLAP.



After kerberised the cluster, the HS2 service is running successfully, but the state of LLAP daemon is UNKNOWN, although llap0 app is running in Yarn and its container num-1 is up and running with no errors but another container is being created and finishes with exeuteStatus=0 again and again. Also hive client failed to connect with no LLAP daemons errors.
I tried to apply all configs from HDP, although failed to enable kerberizing it.
Here is the log from container-01 which is responsible to create the other container:



INFO impl.NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Container container_1703049378182_0001_01_000002

INFO instance.ComponentInstance: [COMPINSTANCE llap-0 : container_1703049378182_0001_01_000002] retrieve status after 0

INFO instance.ComponentInstance: [COMPINSTANCE llap-0 : container_1703049378182_0001_01_000002] retrieve localization statuses

INFO instance.ComponentInstance: [COMPINSTANCE llap-0 : container_1703049378182_0001_01_000002] Transitioned from INIT to STARTED on START event

INFO instance.ComponentInstance: [COMPINSTANCE llap-0 : container_1703049378182_0001_01_000002] cancelling localization retriever

WARN instance.ComponentInstance: Unable to process container ports mapping: {}

content to map due to end-of-input

[Source: (String)""; line: 1, column: 0]

INFO registry.YarnRegistryViewForProviders: [COMPINSTANCE llap-0 : container_1703049378182_0001_01_000002]: D

eleting registry path /users/hive/services/yarn-service/llap0/components/ctr-1703049378182-0001-01-000002

INFO instance.ComponentInstance: [COMPINSTANCE llap-0 : container_1703049378182_0001_01_000002] new IP = [192.168.56.113], host = ambari-server, updating registry

INFO instance.ComponentInstance: [COMPINSTANCE llap-0 : container_1703049378182_0001_01_000002] IP = [192.168.56.113], host = ambari-server, cancel container status retriever

INFO component.Component: [COMPONENT llap] Requesting for 1 container(s)

INFO component.Component: [COMPONENT llap] Submitting container request : Capability[<memory:2048, vCores:1>]Priority[0]AllocationRequestId[0]ExecutionTypeRequest[{Execution Type: GUARANTEED, Enforce Execution Type: false}]Resource Profile[null]

INFO instance.ComponentInstance: [COMPINSTANCE llap-0 : container_1703049378182_0001_01_000002]: container_1703049378182_0001_01_000002 completed. Reinsert back to pending list and requested a new container.

diagnostics=

INFO instance.ComponentInstance: [COMPINSTANCE llap-0 : container_1703049378182_0001_01_000002] Transitioned from STARTED to INIT on STOP event

INFO registry.YarnRegistryViewForProviders: [COMPINSTANCE llap-0 : container_1703049378182_0001_01_000002]: Deleting registry path /users/hive/services/yarn-service/llap0/components/ctr-1703049378182-0001-01-000002

INFO service.ServiceScheduler: 1 containers allocated.

INFO service.ServiceScheduler: [COMPONENT llap]: remove 1 outstanding container requests for allocateId 0

Sent using https://www.zoho.com/mail/