issues@celeborn.apache.org, 2026-09

You are viewing a plain text version of this content. The canonical link for it is here.

- Re: [PR] [CELEBORN-2441] Add drainIncompleteFrame mechanism to recover from backpressure deadlock [celeborn] - posted by "buska88 (via GitHub)" <gi...@apache.org> on 2026/09/01 02:29:15 UTC, 2 replies.
- Re: [PR] [CELEBORN-2400] Recreate the netty worker EventLoopGroup when a worker event loop thread dies [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/01 08:39:10 UTC, 2 replies.
- Re: [PR] [CELEBORN-2445] Support overriding decommission forceExitTimeout via the worker exit API [celeborn] - posted by "Copilot (via GitHub)" <gi...@apache.org> on 2026/09/01 13:29:58 UTC, 2 replies.
- [I] IndexOutOfBoundsException(-1) in StoragePolicy.createFileWriter: revived PartitionLocation assigned storage type HDD though never configured [celeborn] - posted by "laxmanmamididd (via GitHub)" <gi...@apache.org> on 2026/09/01 17:00:35 UTC, 2 replies.
- Re: [PR] [CELEBORN-2440][INFRA] Post a merge summary comment on the PR in merge_pr.py [celeborn] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2026/09/02 17:32:29 UTC, 2 replies.
- Re: [PR] [CELEBORN-2447] Add a /health endpoint to master and worker [celeborn] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2026/09/03 03:02:48 UTC, 5 replies.
- Re: [PR] [CELEBORN-2390] Add celeborn-web service to docker-compose [celeborn] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2026/09/03 03:06:33 UTC, 1 replies.
- [PR] [CELEBORN-2452] Prevent device error broadcast from deleting committed shuffle files [celeborn] - posted by "wang-haihua (via GitHub)" <gi...@apache.org> on 2026/09/03 07:23:05 UTC, 3 replies.
- Re: [PR] [CELEBORN-2384] Add HDFS remote spill support for MR reduce merge to … [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/03 08:39:20 UTC, 2 replies.
- Re: [PR] [CELEBORN-2349] Support worker supplied tags and TagsManager hardening [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2026/09/04 08:06:52 UTC, 2 replies.
- [PR] [CELEBORN-2453] Master should fail fast when the Ratis server is closed unexpectedly [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/06 06:21:16 UTC, 5 replies.
- [PR] [CELEBORN-2454] Add config celeborn.master.ha.ratis.raft.server.close.threshold and docs for Ratis JvmPauseMonitor [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/06 06:21:59 UTC, 1 replies.
- Re: [PR] [CELEBORN-XXXX] Expose manually excluded worker count metric [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/06 08:33:34 UTC, 1 replies.
- Re: [PR] [CELEBORN-2370] Scope reducer metadata by partition range [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/06 08:33:38 UTC, 1 replies.
- Re: [PR] [CELEBORN-2340] Explicitly trim Flusher thread pool cache when flusher queue is empty [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/06 08:33:44 UTC, 1 replies.
- [PR] [CELEBORN-2455] Remove redundant synchronized in Dispatcher.postMessage [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/06 10:32:32 UTC, 0 replies.
- Re: [PR] [CELEBORN-2451] Make DRA safe under AUTO fallback by tying supportsReliableStorage to fallback policy [celeborn] - posted by "venkata91 (via GitHub)" <gi...@apache.org> on 2026/09/06 21:38:32 UTC, 9 replies.
- Re: [PR] [CELEBORN-194] Introduce client side metrics for celeborn [celeborn] - posted by "AmandeepSingh285 (via GitHub)" <gi...@apache.org> on 2026/09/07 05:49:28 UTC, 10 replies.
- Re: [PR] [CELEBORN-2376] Filter empty ResourceConsumption entries in WorkerInfo toString output [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 08:41:40 UTC, 1 replies.
- Re: [PR] [CELEBORN-2375] Support DecommissionThenIdle and Recommission via wor… [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 08:41:46 UTC, 1 replies.
- Re: [PR] [CELEBORN-2359] Deduplicate # HELP/# TYPE lines in PrometheusServlet output [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 08:41:52 UTC, 4 replies.
- Re: [PR] [CELEBORN-2358] Add Available Slots Count Metric [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 08:41:57 UTC, 1 replies.
- Re: [PR] [CELEBORN-2323] Optimize RegisterShuffle for large partition counts [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/07 08:42:05 UTC, 2 replies.
- [PR] [CELEBORN-2456] Fix wrong isAssignableFrom direction in Basic/Bearer authenticationSupported [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/07 16:08:00 UTC, 3 replies.
- [PR] [CELEBORN-XXXX] Invalidate transport client after push timeout [celeborn] - posted by "venkata91 (via GitHub)" <gi...@apache.org> on 2026/09/07 19:15:55 UTC, 0 replies.
- Re: [PR] [CELEBORN-2458] Invalidate transport client after push timeout [celeborn] - posted by "Copilot (via GitHub)" <gi...@apache.org> on 2026/09/08 04:06:47 UTC, 4 replies.
- [PR] [CELEBORN-2457][CIP-24] Adaptive partition write parallelism [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/08 08:06:53 UTC, 0 replies.
- Re: [PR] [CELEBORN-2378] Fix CongestionController never resuming due to Netty pool overhead in getTotalPendingBytes() [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/08 08:38:07 UTC, 1 replies.
- Re: [PR] [WIP] [CIP-20] Dynamically adjust partition write parallelism [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/08 12:57:51 UTC, 0 replies.
- Re: [PR] [CELEBORN-2415] Add Celeborn Spark UI extension [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/08 13:53:29 UTC, 6 replies.
- Re: [PR] [CELEBORN-2398] Allow ChangePartitionManager to refresh candidate workers without allocating slots [celeborn] - posted by "Kalvin2077 (via GitHub)" <gi...@apache.org> on 2026/09/09 02:35:42 UTC, 5 replies.
- Re: [PR] [CELEBORN-2434] Clamp endMapIndex in sliceSortedBufferByMapRange [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/09 04:04:03 UTC, 7 replies.
- [PR] [CELEBORN-2459] Fix race causing DataPusher to accept data during termination and drop it [celeborn] - posted by "rjvkr2021 (via GitHub)" <gi...@apache.org> on 2026/09/09 08:54:07 UTC, 12 replies.
- Re: [PR] [CELEBORN-2428] Fix TierWriter buffer accounting when CompositeByteBuf insertion fails [celeborn] - posted by "Kalvin2077 (via GitHub)" <gi...@apache.org> on 2026/09/09 10:39:20 UTC, 5 replies.
- Re: [PR] [CELEBORN-2436] Speed up slow tests and fix LifecycleManager leaks in spark-it [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/10 08:38:42 UTC, 1 replies.
- [PR] [CELEBORN-2460] Reply SHUFFLE_EXPIRED for a released shuffle instead of an empty file group [celeborn] - posted by "shlomitubul (via GitHub)" <gi...@apache.org> on 2026/09/10 20:19:43 UTC, 10 replies.
- Re: [PR] [CELEBORN-2430] Expose unexpected errors in client background threads [celeborn] - posted by "Kalvin2077 (via GitHub)" <gi...@apache.org> on 2026/09/11 03:45:06 UTC, 3 replies.
- Re: [PR] [CELEBORN-2403] Add worker metrics for disk status and unhealthy disk count [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/11 08:37:40 UTC, 0 replies.
- Re: [PR] [CELEBORN-2364] Add support to cleanup stale shuffle dependencies across all Spark apps when cluster under load [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/11 08:37:43 UTC, 1 replies.
- Re: [PR] [CELEBORN-2356] Fix NPE before null-check in handleChunkFetchRequest [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/11 08:37:48 UTC, 3 replies.
- Re: [PR] [CELEBORN-2446] Support topologySpreadConstraints for master and worker pods in Helm chart [celeborn] - posted by "strelok89 (via GitHub)" <gi...@apache.org> on 2026/09/12 17:04:26 UTC, 0 replies.
- Re: [PR] [CELEBORN-2391] Eliminate intermediate byte[] allocation in Encoders.Strings via Netty ByteBufUtil [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/13 08:35:12 UTC, 2 replies.
- Re: [PR] [CELEBORN-2443] Serialize the peer's own mapIdBitMap [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/14 08:42:14 UTC, 2 replies.
- Re: [PR] [CELEBORN-2439][TEZ] Report accurate partition statistics for shuffle output [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/14 08:42:17 UTC, 2 replies.
- Re: [PR] [CELEBORN-2447] Add a /healthz endpoint to master and worker [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2026/09/14 11:10:30 UTC, 1 replies.
- Re: [PR] [CELEBORN-2361] Unblock stage-end when CommitFilesResponse carries an empty map-id bitmap [celeborn] - posted by "Kalvin2077 (via GitHub)" <gi...@apache.org> on 2026/09/14 11:20:26 UTC, 1 replies.
- [PR] [CELEBORN-2461] Fix wrong enum casing in decommissioning.md REST examples [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/14 14:55:58 UTC, 0 replies.
- [PR] [CELEBORN-2462][WORKER] Reduce misleading WARN/ERROR logs during decommission shutdown [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/14 16:20:46 UTC, 3 replies.
- Re: [PR] [CELEBORN-2461][docs] Fix wrong enum casing in decommissioning.md REST examples [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/14 17:07:24 UTC, 5 replies.
- Re: [PR] [CELEBORN-2442] Optimize meta memory for very large partitions [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2026/09/15 06:19:01 UTC, 1 replies.
- [PR] [CELEBORN-2463] Add ManuallyExcludedWorkerCount metric to distinguish manually excluded workers [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/15 08:19:42 UTC, 0 replies.
- Re: [PR] [CELEBORN-2065] Avoid blocking fetch threads while partition files sort [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/16 08:39:31 UTC, 2 replies.
- Re: [PR] [CELEBORN-2389] Track S3 resource consumption in per-app/per-user metrics [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/16 08:39:33 UTC, 0 replies.
- [I] [FEATURE] Support size quota based s3 tiering [celeborn] - posted by "ftong2010 (via GitHub)" <gi...@apache.org> on 2026/09/17 02:52:08 UTC, 4 replies.
- Re: [PR] [CIP-14][CELEBORN-2220] Support Slow Start Push in C++ Client [celeborn] - posted by "afterincomparableyum (via GitHub)" <gi...@apache.org> on 2026/09/17 05:01:48 UTC, 0 replies.
- [PR] [CELEBORN-2357] Add Metrics Prefix [celeborn] - posted by "afterincomparableyum (via GitHub)" <gi...@apache.org> on 2026/09/17 05:13:52 UTC, 1 replies.
- Re: [PR] [CELEBORN-2388] Decouple inter-master Ratis TLS into its own SSL module [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/17 08:38:33 UTC, 0 replies.
- Re: [PR] [CELEBORN-2387] Add ReserveSlotsFailCount worker metric [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/17 08:38:38 UTC, 0 replies.
- Re: [PR] [CELEBORN-2386] Pass peer hostname to SSLEngine for SNI support [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/17 08:38:42 UTC, 0 replies.
- [PR] [CELEBORN-2465][SPARK] Proactively unregister shuffles on SQL query completion [celeborn] - posted by "yew1eb (via GitHub)" <gi...@apache.org> on 2026/09/17 11:31:19 UTC, 9 replies.
- [PR] [CELEBORN-2464] Wire Kubernetes health probes into the Helm chart [celeborn] - posted by "strelok89 (via GitHub)" <gi...@apache.org> on 2026/09/17 11:45:14 UTC, 3 replies.
- [PR] [CELEBORN-2466] Support per-zone worker StatefulSets and autoscaling in the Helm chart [celeborn] - posted by "strelok89 (via GitHub)" <gi...@apache.org> on 2026/09/17 15:40:47 UTC, 7 replies.
- Re: [PR] [CELEBORN-2065] Worker should support wait partition sort asynchronously [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/18 08:38:05 UTC, 0 replies.
- [PR] [CELEBORN-XXXX] Add backoff with jitter to push retries and log hard split once per file [celeborn] - posted by "venkata91 (via GitHub)" <gi...@apache.org> on 2026/09/20 20:56:04 UTC, 1 replies.
- Re: [PR] [CELEBORN-XXXX][WIP] Mark CelebornShuffleHandle as reliably stored for per-shuffle DRA safety [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/21 08:42:09 UTC, 0 replies.
- [PR] [CELEBORN-2468] Add transport security extension points for custom authn and authz [celeborn] - posted by "Kalvin2077 (via GitHub)" <gi...@apache.org> on 2026/09/23 07:59:52 UTC, 4 replies.
- [PR] [CELEBORN-2469] Add worker stats collection and reporting for autoscaling [celeborn] - posted by "Kalvin2077 (via GitHub)" <gi...@apache.org> on 2026/09/24 08:30:16 UTC, 3 replies.
- Re: [PR] [CELEBORN-2429][CIP-23] Batch worker/application heartbeats into aggregated raft log entries [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/09/28 08:49:08 UTC, 0 replies.
- [PR] [CELEBORN-XXXX] Skip memory file storage for replica partitions [celeborn] - posted by "venkata91 (via GitHub)" <gi...@apache.org> on 2026/09/29 00:56:10 UTC, 0 replies.
- Re: [PR] [CELEBORN-2468] Support pluggable transport authentication and authorization [celeborn] - posted by "gauravkm (via GitHub)" <gi...@apache.org> on 2026/09/30 05:55:00 UTC, 4 replies.