You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "emecii (via GitHub)" <gi...@apache.org> on 2026/09/07 07:42:59 UTC
[I] [Tech Debt][runtime][python][skills] Preserve skill-repository close failures [flink-agents]
emecii opened a new issue, #1108:
URL: https://github.com/apache/flink-agents/issues/1108
## Problem
`python/flink_agents/runtime/skill/skill_manager.py` currently suppresses every exception from `SkillRepository.close()` in `SkillManager.close()`. It still attempts later repositories, but a failure is neither re-raised nor logged. As a result, an extension-provided repository cleanup failure is invisible to callers.
This differs from the Java `SkillManager.closeRepos()` contract introduced in #987: it attempts every repository close, then rethrows the first failure after the loop. Python `ResourceCache.close()` follows the same close-everything-then-surface-the-first-failure pattern.
The current in-tree repository implementations do not raise on close, so this is a contract/parity defect for the public `SkillRepository` extension point rather than a known incident in a bundled repository.
## Proposed direction
Make Python `SkillManager.close()` attempt every distinct repository and rethrow the first close failure after the loop, logging later failures consistently with `ResourceCache.close()`. Add focused tests using test repositories whose `close()` raises, including a case that proves later repositories are still closed.
This is an observable behavior change, so I am seeking committer support before preparing a PR, as required by the contribution guide.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Tech Debt][runtime][python][skills] Preserve skill-repository close failures [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 closed issue #1108: [Tech Debt][runtime][python][skills] Preserve skill-repository close failures
URL: https://github.com/apache/flink-agents/issues/1108
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Tech Debt][runtime][python][skills] Preserve skill-repository close failures [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on issue #1108:
URL: https://github.com/apache/flink-agents/issues/1108#issuecomment-5790781292
Thanks for offering to help, @CodeTrainerMan! Since there hasn’t been any further activity from @emecii for a while, please go ahead and pick this up. I’ve confirmed that the issue still exists on the current main branch, and your proposed approach looks good.
@emecii, if you already have work in progress, please let us know so we can coordinate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Tech Debt][runtime][python][skills] Preserve skill-repository close failures [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on issue #1108:
URL: https://github.com/apache/flink-agents/issues/1108#issuecomment-5659833631
The proposed direction makes sense to me. Please update the issue to follow the community’s Technical Debt template, including the “Search before asking” and “Are you willing to submit a PR?” sections. The existing problem description and proposed direction can go under “Description”.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Tech Debt][runtime][python][skills] Preserve skill-repository close failures [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on issue #1108:
URL: https://github.com/apache/flink-agents/issues/1108#issuecomment-6117132534
Thanks for picking this up! I’ve updated the issue description to follow the Technical Debt template, leaving the PR willingness checkbox unchecked since the implementation is being taken on by someone other than the original reporter.
Extracting the existing `_failure_of` and `_first_or_logged` helpers into `runtime/close_utils.py` sounds good to me. Sharing them across `ResourceCache`, `FlinkRunnerContext`, and `SkillManager` avoids duplication and the circular import. Please preserve their existing behavior when moving them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Tech Debt][runtime][python][skills] Preserve skill-repository close failures [flink-agents]
Posted by "CodeTrainerMan (via GitHub)" <gi...@apache.org>.
CodeTrainerMan commented on issue #1108:
URL: https://github.com/apache/flink-agents/issues/1108#issuecomment-6102424261
Thanks @wenjin272 for the go-ahead, and @emecii for raising this.
I'm starting on it now (branch `fix/skill-manager-close-failures-1108`), following the direction @wenjin272 endorsed:
- `SkillManager.close()` attempts every distinct repository — iterating `_opened_repos` and deduping by identity, since a duplicate skill name displaces a repo in `_repos` while that repo is still owned and must be closed;
- the first `close()` failure is rethrown after the loop, matching the Java `SkillManager.closeRepos()` contract from #987 and the Python `ResourceCache.close()` pattern;
- later failures are logged the same way `ResourceCache.close()` logs them;
- the cleanup path in `_load_skills()` logs any close failure instead of letting it replace the original load failure (`ExceptionGroup` needs Python 3.11 and this package supports 3.10);
- tests with repositories whose `close()` raises, including one proving later repositories are still closed.
Two things I'd like to flag before opening the PR.
**1. Template and labels.** I can't edit the issue body or set labels myself — both return `403 Must have admin rights to Repository`, and I'm only the assignee. @emecii, could you apply the Technical Debt template below and add the `tech debt` / `priority/major` labels? @wenjin272, if it's easier for a committer to do it, that works too.
<details>
<summary>Technical Debt template version of the description</summary>
## Search before asking
- [x] I searched in the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar.
## Description
`python/flink_agents/runtime/skill/skill_manager.py` currently suppresses every exception from `SkillRepository.close()` in `SkillManager.close()`. It still attempts later repositories, but a failure is neither re-raised nor logged. As a result, an extension-provided repository cleanup failure is invisible to callers.
This differs from the Java `SkillManager.closeRepos()` contract introduced in #987: it attempts every repository close, then rethrows the first failure after the loop. Python `ResourceCache.close()` follows the same close-everything-then-surface-the-first-failure pattern.
The current in-tree repository implementations do not raise on close, so this is a contract/parity defect for the public `SkillRepository` extension point rather than a known incident in a bundled repository.
Proposed direction, as endorsed by @wenjin272 in https://github.com/apache/flink-agents/issues/1108#issuecomment-5790781292:
- `SkillManager.close()` attempts every distinct repository, iterating `_opened_repos` and deduping by identity;
- the first `close()` failure is rethrown after the loop, matching the Java `SkillManager.closeRepos()` contract from #987 and the existing Python `ResourceCache.close()` pattern;
- later failures are logged consistently with `ResourceCache.close()`;
- the cleanup path in `_load_skills()` must not let a close failure replace the original load failure. Java attaches it with `addSuppressed`; `ExceptionGroup` needs Python 3.11 and this package supports 3.10, so the cleanup failure is logged instead;
- focused tests using test repositories whose `close()` raises, including one that proves later repositories are still closed.
Still present on the current `main` branch as of 2026-09-23 (confirmed by @wenjin272).
## Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
</details>
**2. Where the shared close helpers should live.** `_failure_of` / `_first_or_logged` currently live in `resource_cache.py`, and `skill_manager` cannot import them at module level — `skill_manager -> resource_cache -> resource_context -> skill_manager` is an import cycle. I'd like to extract them into a small dependency-free `runtime/close_utils.py` and have `resource_cache`, `flink_runner_context` and `skill_manager` share it, so "consistent with `ResourceCache.close()`" is guaranteed by the code rather than by duplication. If you'd rather keep the diff to `skill_manager.py` alone, I'll keep a local copy instead — happy either way.
@emecii — if you already have something in flight, just say so and I'll step aside.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Tech Debt][runtime][python][skills] Preserve skill-repository close failures [flink-agents]
Posted by "CodeTrainerMan (via GitHub)" <gi...@apache.org>.
CodeTrainerMan commented on issue #1108:
URL: https://github.com/apache/flink-agents/issues/1108#issuecomment-5752873735
Hi @emecii and @wenjin272,
This one has been quiet since Sep 14 and I don't see a PR linked to it, so I'd like to check whether @emecii is still planning to pick it up.
If it is free, I would like to take it. My plan follows the direction @wenjin272 already endorsed:
- `SkillManager.close()` attempts every distinct repository;
- the first `close()` failure is rethrown after the loop, matching the Java `SkillManager.closeRepos()` contract from #987 and the existing Python `ResourceCache.close()` pattern;
- later failures are logged consistently with `ResourceCache.close()`;
- focused tests using test repositories whose `close()` raises, including one that proves the remaining repositories are still closed.
I will wait for a reply before branching, so we do not duplicate work. If @emecii already has something in flight, no problem, I will step aside.
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org