You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "tirthooo7 (via GitHub)" <gi...@apache.org> on 2026/02/09 11:48:49 UTC
[PR] MINOR: Fix SSL certificate verification failure in system test worker provisioning [kafka]
tirthooo7 opened a new pull request, #21431:
URL: https://github.com/apache/kafka/pull/21431
## Description
### Summary
Update CA certificates on Ubuntu 14.04 Vagrant workers to fix SSL certificate
verification failures when cloning the kibosh repository during system test setup.
### Problem
Starting around February 7-9, 2026, system tests using Vagrant workers began
failing during worker provisioning with the following error:
```
fatal: unable to access 'https://github.com/confluentinc/kibosh.git/':
server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
```
**Root Cause:** The Vagrant workers use Ubuntu 14.04 (Trusty), which reached
end-of-life in April 2019. The CA certificate bundle on these workers is outdated
and does not include the certificate authorities needed to verify GitHub's current
SSL certificate chain. GitHub (or their CA provider) rotated certificates, causing
the verification to fail on systems with older CA bundles.
**Key observations:**
- The same worker AMI (`ami-5189a661`) worked on [February 6, 2026](https://confluent-open-source-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/fix-cloud-tagging/2026-02-06--001.7f5b8f9a-4837-486a-864c-b5ad3b17391f--1770366328--confluentinc--fix-cloud-tagging--4b6335ab0f/report.html) (see PR #21415)
- Failed on February 9, 2026 with SSL verification errors
- Only GitHub HTTPS connections failed
### Solution
Refresh the CA certificate store during worker provisioning by:
1. Installing/updating the `ca-certificates` package
2. Running `update-ca-certificates --fresh` to rebuild the certificate store
This ensures workers have an updated certificate store that can verify GitHub's
SSL certificate chain.
### Changes
**File:** `vagrant/base.sh`
**Before:**
```bash
apt-get update -y && apt-get install -y git cmake pkg-config libfuse-dev
```
**After:**
```bash
apt-get update -y && apt-get install -y git cmake pkg-config libfuse-dev ca-certificates
update-ca-certificates --fresh
```
### Testing
✅ **System-test-Kafka-Branch-Builder** - 21/21 tests passed
- **Test:** `tests/kafkatest/tests/client/client_compatibility_produce_consume_test.py`
- **Report:** [Test Results](https://confluent-open-source-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/fix-cloud-tagging/2026-02-09--001.5712fd40-c513-43bf-a4c6-c384826e6518--1770635388--confluentinc--fix-cloud-tagging--0d73e131e2/report.html)
### Notes
- This fix adds minimal overhead (~5 seconds per worker) during provisioning
- Long-term, upgrading the worker AMI from Ubuntu 14.04 to a supported LTS version
(e.g., Ubuntu 22.04) would eliminate this class of issues permanently
- The `ca-certificates` package in Ubuntu 14.04 repos is dated 2017, but running
`update-ca-certificates --fresh` rebuilds the store with all available certificates
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix SSL certificate verification failure in system test worker provisioning [kafka]
Posted by "omkreddy (via GitHub)" <gi...@apache.org>.
omkreddy merged PR #21431:
URL: https://github.com/apache/kafka/pull/21431
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Fix SSL certificate verification failure in system test worker provisioning [kafka]
Posted by "tirthooo7 (via GitHub)" <gi...@apache.org>.
tirthooo7 commented on PR #21431:
URL: https://github.com/apache/kafka/pull/21431#issuecomment-3875231434
Reviewers: @chia7712
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org