[jira] [Created] (KAFKA-12847) Dockerfile needed for kafka system tests needs changes

["Abhijit Mane (Jira)" <ji...@apache.org>]

Abhijit Mane created KAFKA-12847:
------------------------------------

             Summary: Dockerfile needed for kafka system tests needs changes
                 Key: KAFKA-12847
                 URL: https://issues.apache.org/jira/browse/KAFKA-12847
             Project: Kafka
          Issue Type: Bug
          Components: system tests
    Affects Versions: 2.7.1, 2.8.0
         Environment: Issue tested in environments below but is independent of h/w arch. or Linux flavor: -
1.) RHEL-8.3 on x86_64 
2.) RHEL-8.3 on IBM Power (ppc64le)
3.) apache/kafka branch tested: trunk (master)
            Reporter: Abhijit Mane
         Attachments: Dockerfile.upstream

Hello,

I tried apache/kafka system tests as per documentation: -

(_https://github.com/apache/kafka/tree/trunk/tests#readme_)

=========================================================
PROBLEM
~~~~~~

1.) As root user, clone kafka github repo and start "kafka system tests"
 # git clone https://github.com/apache/kafka.git
 # cd kafka
 # ./gradlew clean systemTestLibs
 # bash tests/docker/run_tests.sh 
 
2.) Dockerfile issue - _https://github.com/apache/kafka/blob/trunk/tests/docker/*Dockerfile*_
 This file has an *UID* entry as shown below: -
-----------
ARG *UID*="1000"
RUN useradd -u $*UID* ducker

// {color:#de350b}*Error during docker build*{color} => useradd: UID 0 is not unique, root user id is 0
-----------
 I ran everything as root which means the built-in bash environment variable 'UID' always

resolves to 0 and can't be changed. Hence, the docker build fails.
 
3.) Next, as root, as per README, I ran: -

server:/kafka> *bash tests/docker/run_tests.sh*

The ducker tool builds the container images & switches to user '*ducker*' inside the container

& maps kafka root dir ('kafka') from host to '/opt/kafka-dev' in the container.

Ref: *https://github.com/apache/kafka/blob/trunk/tests/docker/ducker-ak*

Ex:  docker run -d *-v "${kafka_dir}:/opt/kafka-dev"* <img_name>

This fails as the 'ducker' user has *no write permissions* to create files under 'kafka' root dir. Hence, it needs to be made writeable.

// *chmod -R a+w kafka* 
 -- needed as container is run as 'ducker' and needs write access since kafka root volume from host is mapped to container as "/opt/kafka-dev" where the 'ducker' user writes logs
=========================================================


 
=========================================================
*FIXES needed*
~~~~~~~~~
 1.) Dockerfile - https://github.com/apache/kafka/blob/trunk/tests/docker/Dockerfile
 Change 'UID' to '*UID_DUCKER*'.

This won't conflict with built in bash env. var UID and the docker image build should succeed.
-----------
ARG *UID_DUCKER*="1000"
RUN useradd -u $*UID_DUCKER* ducker

// *{color:#57d9a3}No Error{color}* => No conflict with built-in UID
-----------

2.) README needs an update where we must ensure the kafka root dir from where the tests 
 are launched is writeable to allow the 'ducker' user to create results/logs.
 # chmod -R a+w kafka

With this, I was able to get the docker images built and system tests started successfully.
=========================================================
 
Also, I wonder whether or not upstream Dockerfile & System tests are part of CI/CD and get tested for every PR. If so, this issue should have been caught.

 

*Question to kafka SME*
-------------------------
Do you believe this is a valid problem with the Dockerfile and the fix is acceptable? 
Please let me know and I am happy to submit a PR with this fix.

Thanks,
Abhijit



--
This message was sent by Atlassian Jira
(v8.3.4#803005)