[jira] [Created] (KAFKA-13325) kafka system tests stall

["Abhijit Mane (Jira)" <ji...@apache.org>]

Abhijit Mane created KAFKA-13325:
------------------------------------

             Summary: kafka system tests stall
                 Key: KAFKA-13325
                 URL: https://issues.apache.org/jira/browse/KAFKA-13325
             Project: Kafka
          Issue Type: Bug
          Components: system tests
    Affects Versions: 2.8.0, 2.7.1
         Environment: Issue tested in environments below but is independent of h/w arch. or Linux flavor: -
1.) RHEL-8.3 on x86_64 
2.) RHEL-8.3 on IBM Power (ppc64le)
3.) apache/kafka branch tested: trunk (master)
            Reporter: Abhijit Mane
            Assignee: Abhijit Mane


Hello,

I tried apache/kafka system tests as per documentation: -

([https://github.com/apache/kafka/tree/trunk/tests#readme|https://github.com/apache/kafka/tree/trunk/tests#readme_])

=========================================================
 PROBLEM
 ~~~~~~

1.) As root user, clone kafka github repo and start "kafka system tests"
 # git clone [https://github.com/apache/kafka.git]
 # cd kafka
 # ./gradlew clean systemTestLibs
 # bash tests/docker/run_tests.sh

2.) Dockerfile issue - [https://github.com/apache/kafka/blob/trunk/tests/docker/Dockerfile]

This file has an *UID* entry as shown below: -
 -----------
 ARG *UID*="1000"
 RUN useradd -u $*UID* ducker

// {color:#de350b}*Error during docker build*{color} => useradd: UID 0 is not unique, root user id is 0
 -----------
 I ran everything as root which means the built-in bash environment variable 'UID' always

resolves to 0 and can't be changed. Hence, the docker build fails. The issue should be seen even if run as non-root.

3.) Next, as root, as per README, I ran: -

server:/kafka> *bash tests/docker/run_tests.sh*

The ducker tool builds the container images & switches to user '*ducker*' inside the container

& maps kafka root dir ('kafka') from host to '/opt/kafka-dev' in the container.

Ref: [https://github.com/apache/kafka/blob/trunk/tests/docker/ducker-ak|https://github.com/apache/kafka/blob/trunk/tests/docker/ducker-ak]

Ex:  docker run -d *-v "${kafka_dir}:/opt/kafka-dev"* <img_name>

This fails as the 'ducker' user has *no write permissions* to create files under 'kafka' root dir. Hence, it needs to be made writeable.

// *chmod -R a+w kafka* 
 – needed as container is run as 'ducker' and needs write access since kafka root volume from host is mapped to container as "/opt/kafka-dev" where the 'ducker' user writes logs
 =========================================================

=========================================================
 *FIXES needed*
 ~~~~~~~~~
 1.) Dockerfile - [https://github.com/apache/kafka/blob/trunk/tests/docker/Dockerfile]
 Change 'UID' to '*UID_DUCKER*'.

This won't conflict with built in bash env. var UID and the docker image build should succeed.
 -----------
 ARG *UID_DUCKER*="1000"
 RUN useradd -u $*UID_DUCKER* ducker

// *{color:#57d9a3}No Error{color}* => No conflict with built-in UID
 -----------

2.) README needs an update where we must ensure the kafka root dir from where the tests 
 are launched is writeable to allow the 'ducker' user to create results/logs.
 # chmod -R a+w kafka

With this, I was able to get the docker images built and system tests started successfully.
 =========================================================

Also, I wonder whether or not upstream Dockerfile & System tests are part of CI/CD and get tested for every PR. If so, this issue should have been caught.

 

*Question to kafka SME*
 -------------------------
 Do you believe this is a valid problem with the Dockerfile and the fix is acceptable? 
 Please let me know and I am happy to submit a PR with this fix.

Thanks,
 Abhijit



--
This message was sent by Atlassian Jira
(v8.3.4#803005)