You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by "linghengqian (via GitHub)" <gi...@apache.org> on 2026/09/30 07:51:20 UTC

[I] Split out the unrelated changes bundled in PR #2548 [shardingsphere-elasticjob]

linghengqian opened a new issue, #2549:
URL: https://github.com/apache/shardingsphere-elasticjob/issues/2549

   ## Feature Request
   
   **For English only**, other languages will not accept.
   
   Please pay attention on issues you submitted, because we maybe need more details. 
   If no response anymore and we cannot make decision by current information, we will **close it**.
   
   Please answer these questions before submitting your issue. Thanks!
   
   ### Is your feature request related to a problem?
   
   - See https://github.com/apache/shardingsphere-elasticjob/pull/2548 .
   
   ### Describe the feature you would like.
   
   #2548 is titled *Spring Boot Starter and Spring Namespace support etcd, Nacos and in-memory registry centers*, but it also carries a number of changes that are not about Spring at all: five kernel files fixing job shutdown and configuration-removal races, two registry-center files handling a closed ZooKeeper client, the whole reachability-metadata generation pipeline, a CI action bump, and some documentation fixes. It is the same situation that #2540 ended up in, which was split in #2543, and this issue tracks the equivalent split for #2548. The items below are intended to land as separate PRs.
   
   The kernel and registry-center items all trace back to one class of problem. The `build` and GraalVM CI jobs run with `-T1C`, so several test JVMs execute concurrently and each embeds its own ZooKeeper, Nacos or etcd server; under CPU contention the registry client sessions churn and in-flight reads race the client shutdown, surfacing as `IllegalStateException: Client is not started or has been closed`, `CuratorClosedException: Expected state [STARTED] was [STOPPED]`, and `JobConfigurationException: JobConfiguration was not found`. A second, independent source is the shutdown path itself: both `ElasticJob.shutdown()` and `JobConfigurationAPI.removeJobConfiguration()` stop a job while Quartz workers and registry watch listeners are still live, so those workers keep failing against a configuration that is already gone. The `ERROR` count in a metadata-generation build has been 12, 2, 4, 4, 0, 0, 2 and 0 across consecutive runs of identical code, so the noise is timing-dependent rathe
 r than deterministic; every distinct surfacing point has been identified and is covered below, though individual runs may print nothing either way.
   
   The build and metadata items are about the `-PgenerateMetadata` pipeline rather than any product behaviour, and they exist because of two limitations in the native-maven-plugin. `metadata-copy` cannot be used to merge the agent output, because with `metadataCopy.merge=false` it only feeds `target/native/agent-output/test` into `native-image-configure` while the tracing agent writes its reports under `agent-output/test/session-{pid}-{datetime}/partial-config-with-origins.json`, so without a merge the goal copies a two-byte `{}`. `merge-agent-files` cannot be a CLI goal either, since its `context` parameter is `required=true` with no default and is not editable, and only the plugin's `NativeExtension` injects `context` into executions; it therefore has to be a bound execution. Separately, `metadata-copy` has a silent-exit path when the agent configuration is absent or the agent is neither enabled in configuration nor through the `agent` system property, which is what makes it possib
 le to scope the agent to a single module.
   
   - [ ] **Kernel: tolerate shutdown and configuration-removal races** — `JobScheduler`, `LiteJob`, `LeaderService`, `FailoverListenerManager`, `ShardingListenerManager`. Move `jobScheduleController.shutdown()` ahead of the registry teardown so that no execution outlives the registry center. Keep `waitForJobsToComplete` set to `false`: Quartz only calls `interruptJob()` in that case, and `true` blocks indefinitely while `JobScheduleController` still holds its monitor, which hangs `ScheduleJobBootstrapTest` outright. Swallow `JobConfigurationException` in `LiteJob.execute()` so that a trigger firing after removal does not reach Quartz's `JobRunShell`. Make the leader-election callback null-safe on `getJobInstance()`. Tolerate a removed configuration in `FailoverListenerManager.isFailoverEnabled()` and `ShardingListenerManager.isStaticSharding()`. Treat a deleted configuration node as an empty value in `ShardingTotalCountChangedJobListener`, because etcd delivers the removed node's v
 alue and `YamlEngine.unmarshal()` returns null for it.
   - [ ] **Registry center: closed-client reads and nested interrupts** — `ZookeeperRegistryCenter`, `RegExceptionHandler`. The `isClientClosed()` guard at the top of `getDirectly()`, `getChildrenKeys()` and `isExisted()` is defeated by a TOCTOU window: the Curator client can be closed between the check and the call, in which case `RegExceptionHandler.handleException()` throws and the fallback `return` becomes dead code, so the state needs to be re-checked after the call. Treat an `InterruptedException` nested in a wrapper as an interruption, by restoring the interrupt flag instead of wrapping it in `RegException`, since a job aborted by a shutdown interrupt currently propagates out of `LiteJob.execute()` into Quartz and is logged as an unhandled exception.
   - [ ] **Build: rewire the reachability-metadata generation pipeline** — root `pom.xml`, `test/native/pom.xml`, `test/native/native-image-filter/extra-filter.json`. Bind `merge-agent-files` as an execution at the `test` phase and drop the `build-native` and `test-native` executions. Replace the hardcoded `<agent><enabled>` with a `native.agent.enabled` property that defaults to `false` and is flipped to `true` only by the profile, so the 37 non-native modules in the reactor stop printing `Cannot find source directory` when the CLI `metadata-copy` goal is run across all 38. Add the four `excludeClasses` filters.
   - [ ] **Regenerate the GraalVM reachability metadata** — `reachability-metadata/.../generated-reachability-metadata/reachability-metadata.json`, +230 / −128 lines. It is a byproduct of the pipeline change above and its output is non-deterministic, so it should land together with that item. It also still lacks the four shaded Nacos gRPC SPI entries `com.alibaba.nacos.shaded.io.grpc.LoadBalancerProvider`, `ManagedChannelProvider`, `NameResolverProvider` and `ServerProvider`, because `extra-filter.json` excludes `com.**`; a conditional-mode agent cannot record accesses made by filtered classes, so those four would need an unconditional, resources-only `reachability-metadata.json` under `META-INF/native-image/com.alibaba.nacos/nacos-client/3.2.4/`. That is a separate item in its own right if native Nacos is wanted.
   - [ ] **CI: bump `graalvm/setup-graalvm` from v1.5.6 to v1.6.6** — `.github/workflows/graalvm.yml`, one line. While that file is being touched it is worth revisiting the `paths` filter, which only covers `graalvm.yml`, `reachability-metadata/src/**`, `test/native/native-image-filter/**` and `test/native/src/**`, and therefore does not trigger on `kernel/**`, `registry-center/**`, `spring/**` or `pom.xml`, and the `github.repository == 'apache/shardingsphere-elasticjob'` gate, which silently disables the job on forks.
   - [ ] **Docs: correctness fixes outside the Spring pages** — `docs/content/user-manual/configuration/graalvm-native-image.{en,cn}.md`, `java-api.{en,cn}.md`, `registry-center/_index.{en,cn}.md`, `registry-center/etcd.{en,cn}.md`, `registry-center/zookeeper.{en,cn}.md`. The GraalVM page fixes the repository name in the clone commands, corrects the module name in the classpath instruction, and notes that the trailing `native:metadata-copy` goal is optional. `java-api` still says the registry center "currently only supports `ZooKeeper`" and is missing an `ensembleTracker` table row. The `registry-center` index pages shed most of their content as it moved into the new per-center pages.
   - [ ] **Tests: native image coverage for the memory and Nacos registry centers** — `test/native/.../memory/MemoryJavaTest`, `MemoryOperationJavaTest`, `.../staticd/SpringBootMemoryRegistryCenterTest` and `.../nacos/NacosJavaTest`. These assert native-image compatibility of the new registry centers, which is a different concern from the Spring Boot Starter and Spring Namespace wiring, so they could move over to the native-image work. If maintainers consider native coverage part of this feature, this item can stay in `#2548`.
   - [ ] **Tests: drop the obsolete `JavaTest` TODO and widen the sharding readiness wait** — `test/native/.../operation/JavaTest.java`. The TODO documenting that `removeJobConfiguration()` always makes a listener throw is obsolete once the kernel and registry-center items land. Separately, the `Awaitility.until` predicate only waited for the sharding count to reach three, but a returned `ShardingInfo` can still have a null `serverIp`, so the wait now also requires every entry to be populated and reuses the fetched collection instead of calling `getShardingInfo()` a second time.
   
   Each of the kernel and registry-center items is a user-visible behaviour change and already has a `RELEASE-NOTES.md` entry under the 3.0.6-SNAPSHOT `Bug Fixes` section, so those entries should move with their code. The items have been verified together on this branch: `./mvnw -PgenerateMetadata -e -T1C clean test native:metadata-copy` finishes with `BUILD SUCCESS` and no `ERROR` lines, the full unit test suite passes with 845 tests and no failures, and checkstyle reports no violations. One caveat worth knowing when reviewing the kernel items: the `generateMetadata` profile restricts the surefire `includes` to `org.apache.shardingsphere.elasticjob.test.natived.**`, so that command only runs the 41 native image integration tests and no unit tests at all. A full unit test run needs to be executed separately for any kernel change.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Split out the unrelated changes bundled in PR #2548 [shardingsphere-elasticjob]

Posted by "terrymanu (via GitHub)" <gi...@apache.org>.
terrymanu closed issue #2549: Split out the unrelated changes bundled in PR #2548
URL: https://github.com/apache/shardingsphere-elasticjob/issues/2549


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Split out the unrelated changes bundled in PR #2548 [shardingsphere-elasticjob]

Posted by "linghengqian (via GitHub)" <gi...@apache.org>.
linghengqian commented on issue #2549:
URL: https://github.com/apache/shardingsphere-elasticjob/issues/2549#issuecomment-5906704986

   - I am working on the current issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org