You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by "sarutak (via GitHub)" <gi...@apache.org> on 2026/04/16 08:23:07 UTC

[PR] [SPARK-XXXXX][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` to recover CI. [spark]

sarutak opened a new pull request, #55369:
URL: https://github.com/apache/spark/pull/55369

### What changes were proposed in this pull request?
This PR pin `meson<1.11.0` and `lxml==4.9.4` to recover CI.

### Why are the changes needed?
Recently, CI fails due to docker image building failure.
https://github.com/apache/spark/actions/runs/24489111484/job/71571576157

The root causes are:
* meson `1.11.0` was recently released and pandas 2.3.3 can not be built with it (https://github.com/pandas-dev/pandas/issues/65252)
* Newer lxml was also released (https://pypi.org/project/lxml/#history). This PR pin to `4.9.4` to consistent with the version specified in `.github/workflows/build_and_test.yml`.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Confirmed that `docker build` works for both `dev/infra/Dockerfile` and `dev/spark-test-image/pypy-310/Dockerfile`

### Was this patch authored or co-authored using generative AI tooling?
No.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI [spark]

Posted by "gaogaotiantian (via GitHub)" <gi...@apache.org>.

gaogaotiantian commented on PR #55369:
URL: https://github.com/apache/spark/pull/55369#issuecomment-4263172894

   pypy support is already dropped. Let me check how we should proceed with it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI [spark]

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.

zhengruifeng commented on PR #55369:
URL: https://github.com/apache/spark/pull/55369#issuecomment-4264509914

   thanks, merged to master


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI [spark]

Posted by "sarutak (via GitHub)" <gi...@apache.org>.

sarutak commented on code in PR #55369:
URL: https://github.com/apache/spark/pull/55369#discussion_r3137165454


##########
dev/infra/Dockerfile:
##########
@@ -92,7 +92,9 @@ RUN mkdir -p /usr/local/pypy/pypy3.10 && \
     ln -sf /usr/local/pypy/pypy3.10/bin/pypy /usr/local/bin/pypy3.10 && \
     ln -sf /usr/local/pypy/pypy3.10/bin/pypy /usr/local/bin/pypy3
 RUN curl -sS https://bootstrap.pypa.io/get-pip.py | pypy3
-RUN pypy3 -m pip install numpy 'six==1.16.0' 'pandas==2.3.3' scipy coverage matplotlib lxml
+RUN echo 'meson<1.11.0' > /tmp/constraints.txt && \

Review Comment:
   Thank you for pointing it out. That has been removed.
   https://github.com/apache/spark/pull/55505



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI [spark]

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.

zhengruifeng commented on PR #55369:
URL: https://github.com/apache/spark/pull/55369#issuecomment-4264525854

   > I believe we can just remove all the pypy related part from infra image - they are not used anymore.
   
   +1, the pypy should at least not affect the master branch


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI [spark]

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.

zhengruifeng closed pull request #55369: [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI
URL: https://github.com/apache/spark/pull/55369


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI [spark]

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.

LuciferYang commented on PR #55369:
URL: https://github.com/apache/spark/pull/55369#issuecomment-4286251177

   we can backport SPARK-56540 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI [spark]

Posted by "sarutak (via GitHub)" <gi...@apache.org>.

sarutak commented on PR #55369:
URL: https://github.com/apache/spark/pull/55369#issuecomment-4262644080

   `Base image build` seems passed.
   https://github.com/sarutak/spark/actions/runs/24499564157/job/71659528745


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-XXXXX][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI. [spark]

Posted by "sarutak (via GitHub)" <gi...@apache.org>.

sarutak commented on PR #55369:
URL: https://github.com/apache/spark/pull/55369#issuecomment-4261040803

   `docker build` on GA seems very slow today...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI [spark]

Posted by "gaogaotiantian (via GitHub)" <gi...@apache.org>.

gaogaotiantian commented on PR #55369:
URL: https://github.com/apache/spark/pull/55369#issuecomment-4263258524

   I believe we can just remove all the pypy related part from infra image - they are not used anymore.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI [spark]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.

pan3793 commented on PR #55369:
URL: https://github.com/apache/spark/pull/55369#issuecomment-4286154029

   should this(and other recent similar changes) be backported? I see that at least branch-4.1 and branch-4.0 infra image jobs are broken now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI [spark]

Posted by "sarutak (via GitHub)" <gi...@apache.org>.

sarutak commented on PR #55369:
URL: https://github.com/apache/spark/pull/55369#issuecomment-4286238499

   @pan3793 
   
   > should this(and other recent similar changes) be backported? I see that at least branch-4.1 and branch-4.0 infra image jobs are broken now.
   
   It seems that the issue resolved by #55414 is also occurring with `branch-4.0` and `branch-4.1`.
   Could you take a look at that, @LuciferYang ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-56513][INFRA] Pin `meson<1.11.0` and `lxml==4.9.4` in Dockerfile for PyPy to recover CI [spark]

Posted by "eli-schwartz (via GitHub)" <gi...@apache.org>.

eli-schwartz commented on code in PR #55369:
URL: https://github.com/apache/spark/pull/55369#discussion_r3121310019


##########
dev/infra/Dockerfile:
##########
@@ -92,7 +92,9 @@ RUN mkdir -p /usr/local/pypy/pypy3.10 && \
     ln -sf /usr/local/pypy/pypy3.10/bin/pypy /usr/local/bin/pypy3.10 && \
     ln -sf /usr/local/pypy/pypy3.10/bin/pypy /usr/local/bin/pypy3
 RUN curl -sS https://bootstrap.pypa.io/get-pip.py | pypy3
-RUN pypy3 -m pip install numpy 'six==1.16.0' 'pandas==2.3.3' scipy coverage matplotlib lxml
+RUN echo 'meson<1.11.0' > /tmp/constraints.txt && \

Review Comment:
   This is unfortunate to use `<` rather than blocking a single "bad" release. It was fixed in meson==1.11.1



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org