[jira] [Commented] (KAFKA-12847) Dockerfile needed for kafka system tests needs changes

["Abhijit Mane (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17355791#comment-17355791 ] 

Abhijit Mane commented on KAFKA-12847:
--------------------------------------

[~chia7712]
Thanks for looking into this.

Dockerfile has an UID entry as shown below: -
-----------
ARG UID="1000"       ==> Has no effect as UID value is read-only in bash
RUN useradd -u $UID ducker
// Error during docker build => useradd: UID 0 is not unique, root user id is 0
-----------

The issue is seen even if run as non-root.  'UID' is a built-in, read-only bash environment variable and resolves to id of the logged
in user. PR submitted - https://github.com/apache/kafka/pull/10782

Please let me know your thoughts.

> Dockerfile needed for kafka system tests needs changes
> ------------------------------------------------------
>
>                 Key: KAFKA-12847
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12847
>             Project: Kafka
>          Issue Type: Bug
>          Components: system tests
>    Affects Versions: 2.8.0, 2.7.1
>         Environment: Issue tested in environments below but is independent of h/w arch. or Linux flavor: -
> 1.) RHEL-8.3 on x86_64 
> 2.) RHEL-8.3 on IBM Power (ppc64le)
> 3.) apache/kafka branch tested: trunk (master)
>            Reporter: Abhijit Mane
>            Assignee: Abhijit Mane
>            Priority: Major
>              Labels: easyfix
>         Attachments: Dockerfile.upstream
>
>
> Hello,
> I tried apache/kafka system tests as per documentation: -
> ([https://github.com/apache/kafka/tree/trunk/tests#readme|https://github.com/apache/kafka/tree/trunk/tests#readme_])
> =========================================================
>  PROBLEM
>  ~~~~~~
> 1.) As root user, clone kafka github repo and start "kafka system tests"
>  # git clone [https://github.com/apache/kafka.git]
>  # cd kafka
>  # ./gradlew clean systemTestLibs
>  # bash tests/docker/run_tests.sh
> 2.) Dockerfile issue - [https://github.com/apache/kafka/blob/trunk/tests/docker/Dockerfile]
> This file has an *UID* entry as shown below: -
>  -----------
>  ARG *UID*="1000"
>  RUN useradd -u $*UID* ducker
> // {color:#de350b}*Error during docker build*{color} => useradd: UID 0 is not unique, root user id is 0
>  -----------
>  I ran everything as root which means the built-in bash environment variable 'UID' always
> resolves to 0 and can't be changed. Hence, the docker build fails. The issue should be seen even if run as non-root.
> 3.) Next, as root, as per README, I ran: -
> server:/kafka> *bash tests/docker/run_tests.sh*
> The ducker tool builds the container images & switches to user '*ducker*' inside the container
> & maps kafka root dir ('kafka') from host to '/opt/kafka-dev' in the container.
> Ref: [https://github.com/apache/kafka/blob/trunk/tests/docker/ducker-ak|https://github.com/apache/kafka/blob/trunk/tests/docker/ducker-ak]
> Ex:  docker run -d *-v "${kafka_dir}:/opt/kafka-dev"* <img_name>
> This fails as the 'ducker' user has *no write permissions* to create files under 'kafka' root dir. Hence, it needs to be made writeable.
> // *chmod -R a+w kafka* 
>  – needed as container is run as 'ducker' and needs write access since kafka root volume from host is mapped to container as "/opt/kafka-dev" where the 'ducker' user writes logs
>  =========================================================
> =========================================================
>  *FIXES needed*
>  ~~~~~~~~~
>  1.) Dockerfile - [https://github.com/apache/kafka/blob/trunk/tests/docker/Dockerfile]
>  Change 'UID' to '*UID_DUCKER*'.
> This won't conflict with built in bash env. var UID and the docker image build should succeed.
>  -----------
>  ARG *UID_DUCKER*="1000"
>  RUN useradd -u $*UID_DUCKER* ducker
> // *{color:#57d9a3}No Error{color}* => No conflict with built-in UID
>  -----------
> 2.) README needs an update where we must ensure the kafka root dir from where the tests 
>  are launched is writeable to allow the 'ducker' user to create results/logs.
>  # chmod -R a+w kafka
> With this, I was able to get the docker images built and system tests started successfully.
>  =========================================================
> Also, I wonder whether or not upstream Dockerfile & System tests are part of CI/CD and get tested for every PR. If so, this issue should have been caught.
>  
> *Question to kafka SME*
>  -------------------------
>  Do you believe this is a valid problem with the Dockerfile and the fix is acceptable? 
>  Please let me know and I am happy to submit a PR with this fix.
> Thanks,
>  Abhijit



--
This message was sent by Atlassian Jira
(v8.3.4#803005)