You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "martin-g (via GitHub)" <gi...@apache.org> on 2024/10/16 07:33:12 UTC
[PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
martin-g opened a new pull request, #3213:
URL: https://github.com/apache/avro/pull/3213
Drop more pypy3.9 usages
<!--
*Thank you very much for contributing to Apache Avro - we are happy that you want to help us improve Avro. To help the community review your contribution in the best possible way, please go through the checklist below, which will get the contribution into a shape in which it can be best reviewed.*
*Please understand that we do not do this to make contributions to Avro a hassle. In order to uphold a high standard of quality for code contributions, while at the same time managing a large number of contributions, we need contributors to prepare the contributions well, and give reviewers enough contextual information for the review. Please also understand that contributions that do not follow this guide will take longer to review and thus typically be picked up with lower priority by the community.*
## Contribution Checklist
- Make sure that the pull request corresponds to a [JIRA issue](https://issues.apache.org/jira/projects/AVRO/issues). Exceptions are made for typos in JavaDoc or documentation files, which need no JIRA issue.
- Name the pull request in the form "AVRO-XXXX: [component] Title of the pull request", where *AVRO-XXXX* should be replaced by the actual issue number.
The *component* is optional, but can help identify the correct reviewers faster: either the language ("java", "python") or subsystem such as "build" or "doc" are good candidates.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Make sure that the change passes the automated tests. You can [build the entire project](https://github.com/apache/avro/blob/main/BUILD.md) or just the [language-specific SDK](https://avro.apache.org/project/how-to-contribute/#unit-tests).
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message (including the JIRA id)
- Every commit message references Jira issues in their subject lines. In addition, commits follow the guidelines from [How to write a good git commit message](https://chris.beams.io/posts/git-commit/)
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
-->
## What is the purpose of the change
*(For example: This pull request improves file read performance by buffering data, fixing AVRO-XXXX.)*
## Verifying this change
*(Please pick one of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Extended interop tests to verify consistent valid schema names between SDKs*
- *Added test that validates that Java throws an AvroRuntimeException on invalid binary data*
- *Manually verified the change by building the website and checking the new redirect*
## Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "kojiromike (via GitHub)" <gi...@apache.org>.
kojiromike commented on PR #3213:
URL: https://github.com/apache/avro/pull/3213#issuecomment-2416694492
> For Python I used python3 -m pip install --break-system-packages ... because the CI uses short-lived VMs anyway and one should not bother with polluting the system namespace. Again feel free to improve if you see a better way!
This is fine for actions, IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "KalleOlaviNiemitalo (via GitHub)" <gi...@apache.org>.
KalleOlaviNiemitalo commented on PR #3213:
URL: https://github.com/apache/avro/pull/3213#issuecomment-2416087499
The [Dependencies](https://learn.microsoft.com/en-us/dotnet/core/install/linux-ubuntu#dependencies) list shows "libicu74 (for 24.04 or later)", and [libicu-dev on Ubuntu 24.04](https://packages.ubuntu.com/noble/libicu-dev) also depends on that.
The .NET Core 3.1.32 code that loads ICU libraries seems to be at <https://github.com/dotnet/coreclr/blob/v3.1.32/src/corefx/System.Globalization.Native/pal_icushim.c>. It checks the `CLR_ICU_VERSION_OVERRIDE` environment variable; perhaps that could be used for forcing .NET Core 3.1 to load `libicu74`. But I don't know whether these versions of ICU have compatible ABI for the functions and structures that .NET uses.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "zcsizmadia (via GitHub)" <gi...@apache.org>.
zcsizmadia commented on PR #3213:
URL: https://github.com/apache/avro/pull/3213#issuecomment-2416593922
I will have a PR soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "kojiromike (via GitHub)" <gi...@apache.org>.
kojiromike commented on PR #3213:
URL: https://github.com/apache/avro/pull/3213#issuecomment-2416537354
For doc/ref:
There are a slew issues referenced at https://github.com/actions/runner-images/issues/10636
GitHub's runner image for ubuntu-24.04 is just plain broken for Python as of now. (As opposed to just backwards-incompatible to how we were doing things).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "zcsizmadia (via GitHub)" <gi...@apache.org>.
zcsizmadia commented on PR #3213:
URL: https://github.com/apache/avro/pull/3213#issuecomment-2416547754
Just checked netcoreapp5.0, and even the plain compiler it is not working on a plain ubuntu 24.04. SO 3.1 and 5.0 should be removed (which is ok), to be able to run on ubuntu 24.04 runner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "zcsizmadia (via GitHub)" <gi...@apache.org>.
zcsizmadia commented on PR #3213:
URL: https://github.com/apache/avro/pull/3213#issuecomment-2416659935
#3214
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "martin-g (via GitHub)" <gi...@apache.org>.
martin-g commented on PR #3213:
URL: https://github.com/apache/avro/pull/3213#issuecomment-2416583564
As you may noticed I set `ubuntu-22.04` for the C# workflows to make it work again.
Feel free to open a new PR for dropping the old .NET versions.
For Python I used `python3 -m pip install --break-system-packages ...` because the CI uses short-lived VMs anyway and one should not bother with polluting the system namespace. Again feel free to improve if you see a better way!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "KalleOlaviNiemitalo (via GitHub)" <gi...@apache.org>.
KalleOlaviNiemitalo commented on PR #3213:
URL: https://github.com/apache/avro/pull/3213#issuecomment-2416051608
[Microsoft ended .NET Core 3.1 support on December 13, 2022.](https://dotnet.microsoft.com/platform/support/policy/dotnet-core), Can Apache Avro just stop building for it and testing on it? The [Supported Distributions](https://learn.microsoft.com/dotnet/core/install/linux-ubuntu#supported-distributions) table shows .NET Core 3.1 was supported on Ubuntu 20.04 and 22.04, but not 23.10 nor 24.04.
<https://github.com/apache/avro/blob/290ce8e5a626d84643d89239105f94212013551c/.github/workflows/test-lang-csharp.yml#L45-L53>
<https://github.com/apache/avro/blob/290ce8e5a626d84643d89239105f94212013551c/lang/csharp/common.props#L38-L45>
.NET 5.0 and .NET 7.0 are likewise out of support; in contrast, .NET 6.0 is a Long Term Support release and is still supported. But if there is currently no problem testing on .NET 5.0 and .NET 7.0, then I suppose they can be kept on a best-effort basis.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "KalleOlaviNiemitalo (via GitHub)" <gi...@apache.org>.
KalleOlaviNiemitalo commented on PR #3213:
URL: https://github.com/apache/avro/pull/3213#issuecomment-2416123320
It looks like .NET uses only C APIs of ICU, not C++ APIs. The functions are listed in <https://github.com/dotnet/coreclr/blob/v3.1.32/src/corefx/System.Globalization.Native/pal_icushim.h>.
Based on the [ICU Binary Compatibility](https://unicode-org.github.io/icu/userguide/icu/design.html#icu-binary-compatibility) documentation, I think the ABI will be compatible if `CLR_ICU_VERSION_OVERRIDE=74` is used to force loading `libicuuc.so.74` and `libicui18n.so.74`, which [exist in the libicu74 package](https://packages.ubuntu.com/noble/amd64/libicu74/filelist). And the code in pal_icushim.h attempts to load a range of 20 higher major versions anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "martin-g (via GitHub)" <gi...@apache.org>.
martin-g commented on PR #3213:
URL: https://github.com/apache/avro/pull/3213#issuecomment-2416010934
@KalleOlaviNiemitalo @zcsizmadia Any ideas how to fix this for ubuntu-24.04:
```
Testhost process for source(s) '/home/runner/work/avro/avro/lang/csharp/src/apache/codec/Avro.File.XZ.Test/bin/Release/netcoreapp3.1/Avro.File.XZ.Test.dll' exited with error: Process terminated. Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.
2024-10-16T07:40:09.5324651Z at System.Environment.FailFast(System.String)
2024-10-16T07:40:09.5325787Z at System.Globalization.GlobalizationMode.GetGlobalizationInvariantMode()
2024-10-16T07:40:09.5327122Z at System.Globalization.GlobalizationMode..cctor()
2024-10-16T07:40:09.5328235Z at System.Globalization.CultureData.CreateCultureWithInvariantData()
2024-10-16T07:40:09.5329310Z at System.Globalization.CultureData.get_Invariant()
2024-10-16T07:40:09.5330141Z at System.Globalization.CultureInfo..cctor()
2024-10-16T07:40:09.5330853Z at System.String.ToUpperInvariant()
2024-10-16T07:40:09.5331693Z at System.Diagnostics.Tracing.EventSource.GetGuid(System.Type)
2024-10-16T07:40:09.5333184Z at System.Diagnostics.Tracing.EventSource..ctor(System.Diagnostics.Tracing.EventSourceSettings, System.String[])
2024-10-16T07:40:09.5334978Z at System.Diagnostics.Tracing.EventSource..ctor()
2024-10-16T07:40:09.5336461Z at Microsoft.VisualStudio.TestPlatform.CoreUtilities.Tracing.TestPlatformEventSource..ctor()
2024-10-16T07:40:09.5338503Z at Microsoft.VisualStudio.TestPlatform.CoreUtilities.Tracing.TestPlatformEventSource..cctor()
2024-10-16T07:40:09.5340327Z at Microsoft.VisualStudio.TestPlatform.CoreUtilities.Tracing.TestPlatformEventSource.get_Instance()
2024-10-16T07:40:09.5341999Z at Microsoft.VisualStudio.TestPlatform.TestHost.Program.Main(System.String[])
```
?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "martin-g (via GitHub)" <gi...@apache.org>.
martin-g merged PR #3213:
URL: https://github.com/apache/avro/pull/3213
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04 [avro]
Posted by "zcsizmadia (via GitHub)" <gi...@apache.org>.
zcsizmadia commented on PR #3213:
URL: https://github.com/apache/avro/pull/3213#issuecomment-2416528034
Sorry for being late to the convo. As @KalleOlaviNiemitalo said the isse is the libucu. I wasn't aware that 3.1 cannot run at all on ubuntu 24.04 since it is so old. The only reason all the EOL net targets I still kept in the supported frameworks was to be paranoid when it comes to testing.were. It is totally safe to remove the netcoreapp3.1 support (or any of the others which are not getting supported on newer OSs).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org