You are viewing a plain text version of this content. The canonical link for it is here.
- Re: [PR] [CELEBORN-1925] Support Flink 2.0 [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/04/01 03:26:43 UTC, 5 replies.
- Re: [PR] [CELEBORN-1932][CIP-14] Adapt java's serialization to support cpp serialization for GetReducerFileGroup/Response [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2025/04/01 06:51:46 UTC, 16 replies.
- Re: [PR] [CELEBORN-1903] Enable connection heartbeat and close idle connection [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/04/01 08:36:11 UTC, 7 replies.
- Re: [PR] [CELEBORN-1793] Add netty pinned memory metrics [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/04/01 08:36:13 UTC, 2 replies.
- Re: [PR] [CELEBORN-894] Add checksum for shuffle data [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/04/01 08:36:15 UTC, 1 replies.
- [PR] [CELEBORN-1894] Allow skipping already read chunks during unreplicated shuffle read retries (porting from 0.6) [celeborn] - posted by "saurabhd336 (via GitHub)" <gi...@apache.org> on 2025/04/01 08:57:59 UTC, 0 replies.
- Re: [PR] [CELEBORN-1949] Add a labeler github action to triage PRs [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/04/01 11:13:57 UTC, 0 replies.
- Re: [PR] Test add a labeler github action to triage PRs [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/04/01 11:23:52 UTC, 0 replies.
- Re: [PR] [CELEBORN-1577][FOLLOWUP] Add UpdateResourceConsumptionTime timer and prevent NPE if metrics not found [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/04/01 11:24:31 UTC, 1 replies.
- Re: [PR] [CELEBORN-1921] Broadcast large GetReducerFileGroupResponse to prevent Spark driver network exhausted [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/01 15:28:55 UTC, 2 replies.
- [PR] test [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/01 15:34:32 UTC, 3 replies.
- Re: [PR] [CELEBORN-1532][HELM][FOLLOWUP] Define template helpers for master/worker respectively [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/02 02:28:12 UTC, 0 replies.
- [PR] [MINOR] Add spec link to JavaSerializer [celeborn] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/04/02 02:29:33 UTC, 2 replies.
- Re: [PR] [CELEBORN-1952][HELM] Define template helpers for master/worker respectively [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/02 02:58:27 UTC, 2 replies.
- Re: [PR] [CELEBORN-1954][HELM] Add a new value image.registry [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/02 03:02:18 UTC, 2 replies.
- [PR] [CELEBORN-1956] Forward GitHub discussion to ASF mailing list [celeborn] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/04/02 03:09:50 UTC, 2 replies.
- [PR] [CELEBORN-1953][HELM] Split podAnnotations into master.annotations and worker.annotations [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/02 03:29:49 UTC, 12 replies.
- Re: [PR] [CELEBORN-1528][HELM] Use volumeClaimTemplates to support various storage backend [celeborn] - posted by "nqvuong1998 (via GitHub)" <gi...@apache.org> on 2025/04/02 03:33:55 UTC, 1 replies.
- [PR] [CELEBORN-1949][FOLLOWUP] Fix typo for `kind:deploy` label [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/02 03:36:22 UTC, 2 replies.
- [PR] Fix labeler typo test [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/02 03:38:57 UTC, 1 replies.
- [PR] [CELEBORN-1436][FOLLOWUP] Add swagger editor link for RESTful spec [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/02 04:10:24 UTC, 0 replies.
- Re: [PR] [CELEBORN-1436][FOLLOWUP] Add swagger editor links for RESTful spec [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/02 04:55:55 UTC, 5 replies.
- Re: [PR] [CELEBORN-1844][CIP-8] introduce tier writer proxy and simplify partition data writer [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/04/02 05:29:47 UTC, 1 replies.
- Re: [PR] [RFC][CELEBORN-894] Add support for end to end Integrity Checks [celeborn] - posted by "gauravkm (via GitHub)" <gi...@apache.org> on 2025/04/02 05:52:15 UTC, 8 replies.
- Re: [PR] [CELEBORN-1926] updated helmChart to allow different nodeSelector and tolerations for master and worker nodes [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/02 10:13:25 UTC, 1 replies.
- [PR] Convert Resource.proto from Protocol Buffers version 2 to version 3. [celeborn] - posted by "zhaohehuhu (via GitHub)" <gi...@apache.org> on 2025/04/03 07:40:11 UTC, 3 replies.
- Re: [PR] [CELEBORN-1894][0.5] Allow skipping already read chunks during unreplicated shuffle read retries [celeborn] - posted by "saurabhd336 (via GitHub)" <gi...@apache.org> on 2025/04/04 05:14:34 UTC, 8 replies.
- Re: [PR] [CELEBORN-1892] Adding register with master fail count metric for worker [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/04/04 08:35:30 UTC, 1 replies.
- Re: [PR] [CELEBORN-1896] delete data from failed to fetch shuffles [celeborn] - posted by "CodingCat (via GitHub)" <gi...@apache.org> on 2025/04/05 04:26:06 UTC, 56 replies.
- Re: [PR] [CELEBORN-1948] Worker handle replica data callback should not ignore HARD_SPLIT [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/07 02:30:06 UTC, 5 replies.
- [PR] [CELEBORN-1958][CIP-14] Add testsuite to test writing with javaClient and reading with cppClient [celeborn] - posted by "HolyLow (via GitHub)" <gi...@apache.org> on 2025/04/07 08:13:03 UTC, 4 replies.
- [PR] Set max memory of SortBasedPusher based off Spark configs [celeborn] - posted by "helenweng-stripe (via GitHub)" <gi...@apache.org> on 2025/04/07 21:45:25 UTC, 4 replies.
- [PR] Add end to end integrity checks [celeborn] - posted by "gauravkm (via GitHub)" <gi...@apache.org> on 2025/04/07 22:28:01 UTC, 2 replies.
- Re: [PR] [CELEBORN-1916] Support Aliyun OSS Based on MPU Extension Interface [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2025/04/08 02:56:44 UTC, 2 replies.
- Re: [PR] [CELEBORN-1931] use gather API for local flusher to optimize write io pattern [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2025/04/08 08:38:24 UTC, 4 replies.
- [PR] [Celeborn-81][FOLLOWUP]Correct scala test plugin args. [celeborn] - posted by "gaoyajun02 (via GitHub)" <gi...@apache.org> on 2025/04/08 12:55:37 UTC, 0 replies.
- Re: [PR] [CELEBORN-81][FOLLOWUP]Correct scala test plugin args. [celeborn] - posted by "gaoyajun02 (via GitHub)" <gi...@apache.org> on 2025/04/08 13:26:57 UTC, 2 replies.
- Re: [PR] [CELEBORN-1855] LifecycleManager return appshuffleId for non barrier stage when fetch fail has been reported [celeborn] - posted by "buska88 (via GitHub)" <gi...@apache.org> on 2025/04/08 14:27:11 UTC, 2 replies.
- Re: [PR] [CELEBORN-1912] ReadClientHandler should support reconnection for ChannelInboundHandler#channelInactive [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/04/09 08:37:27 UTC, 2 replies.
- [PR] [CELEBORN-1646][FOLLOWUP] DeviceMonitor should notifyObserversOnError with CRITICAL_ERROR disk status for input/ouput error [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/04/09 09:11:24 UTC, 6 replies.
- [PR] [CELEBORN-1960] Fix PauseSpentTime only append the interval check time [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2025/04/10 02:49:16 UTC, 2 replies.
- [PR] Avoid use `_$eq` in Scala file [celeborn] - posted by "cxzl25 (via GitHub)" <gi...@apache.org> on 2025/04/10 08:18:47 UTC, 0 replies.
- Re: [PR] [CELEBORN-1904] Cancel the running tasks in Indeterminate stage on stage rerun to save resource [celeborn] - posted by "Z1Wu (via GitHub)" <gi...@apache.org> on 2025/04/10 15:01:22 UTC, 7 replies.
- Re: [PR] [MINOR] Avoid use `_$eq` in Scala file [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2025/04/11 05:47:55 UTC, 0 replies.
- Re: [PR] [CELEBORN-1856]Support stage-rerun when read partition by chunkOffsets when enable optimize skew partition read [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/11 07:53:56 UTC, 1 replies.
- [PR] Spark celeborn patch [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/11 08:22:24 UTC, 0 replies.
- Re: [PR] [CELEBORN-1768][WRITER] Refactoring Shuffle Writer to extract common methods [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/04/11 08:35:52 UTC, 0 replies.
- Re: [PR] [CELEBORN-1856][FOLLOWUP] Check `isCelebornSkewedShuffle` before `registerCelebornSkewedShuffle` for stage rollback [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/11 08:37:55 UTC, 0 replies.
- Re: [PR] [CELEBORN-1856][FOLLOWUP] Check `isCelebornSkewedShuffle ` before `registerCelebornSkewedShuffle` for stage rollback [celeborn] - posted by "Z1Wu (via GitHub)" <gi...@apache.org> on 2025/04/11 14:35:58 UTC, 2 replies.
- [PR] Split nodeSelector into master.nodeSelector and worker.nodeSelector [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/12 04:32:22 UTC, 0 replies.
- Re: [PR] [CELEBORN-1919] Hardsplit batch tracking should be disabled when pushing only a single replica [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/04/14 06:42:50 UTC, 1 replies.
- [PR] [CELEBORN-1874][CIP-14] Add CICD procedure in github action for cppClient [celeborn] - posted by "HolyLow (via GitHub)" <gi...@apache.org> on 2025/04/14 06:46:17 UTC, 2 replies.
- Re: [PR] [CELEBORN-1922] Remove MetricSupplier usage for better compatibility [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/04/14 08:38:24 UTC, 2 replies.
- Re: [PR] [CELEBORN-1492] Introduce Celeborn Chaos Testing Framework [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/04/14 08:38:26 UTC, 1 replies.
- Re: [PR] [CELEBORN-1955][HELM] Split nodeSelector into master.nodeSelector and worker.nodeSelector [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/04/15 02:08:42 UTC, 1 replies.
- [PR] [HELM] Split tolerations into master.tolerations and worker.tolerations [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/15 03:47:53 UTC, 0 replies.
- Re: [PR] [CELEBORN-1962][HELM] Split tolerations into master.tolerations and worker.tolerations [celeborn] - posted by "codecov[bot] (via GitHub)" <gi...@apache.org> on 2025/04/15 05:40:39 UTC, 2 replies.
- Re: [D] Deploying Apache Celebron for Large-Scale Spark SQL Workloads [celeborn] - posted by "sid-habu (via GitHub)" <gi...@apache.org> on 2025/04/15 05:50:20 UTC, 0 replies.
- [PR] Unreleased partition location [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2025/04/15 15:02:30 UTC, 0 replies.
- Re: [PR] [CELEBORN-1902] Read client throws PartitionConnectionException [celeborn] - posted by "venkata91 (via GitHub)" <gi...@apache.org> on 2025/04/15 18:10:47 UTC, 5 replies.
- [PR] [CELEBORN-1966] Added fix to get userinfo in celeborn [celeborn] - posted by "AmandeepSingh285 (via GitHub)" <gi...@apache.org> on 2025/04/16 12:24:05 UTC, 15 replies.
- Re: [PR] [CELEBORN-1968] Publish metric for unreleased partition location count when worker was gracefully shutdown [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/04/17 08:06:47 UTC, 4 replies.
- [PR] [CELEBORN-1970] Use StatusCode.fromValue instead of Utils.toStatusCode [celeborn] - posted by "jiang13021 (via GitHub)" <gi...@apache.org> on 2025/04/17 12:57:00 UTC, 3 replies.
- Re: [PR] CELEBORN-1334 [CIP-12] Support HARD_SPLIT in PushMergedData should not support new version client with old version worker [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/04/18 08:35:40 UTC, 0 replies.
- [PR] [CELEBORN-1951][HELM] Rename resources.{master,worker} to {master,worker}.resoruces [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/18 10:35:51 UTC, 2 replies.
- Re: [PR] [CELEBORN-1800] Introduce ApplicationTotalCount and ApplicationFallbackCount metric to record the total and fallback count of application [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/04/20 08:34:06 UTC, 2 replies.
- [PR] [CELEBORN-1969] Remove celeborn.client.shuffle.mapPartition.split.enabled to enable shuffle partition split at default for MapPartition [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/04/21 02:56:16 UTC, 1 replies.
- Re: [PR] [CELEBORN-1961] Convert Resource.proto from Protocol Buffers version 2 to version 3. [celeborn] - posted by "zhaohehuhu (via GitHub)" <gi...@apache.org> on 2025/04/21 10:42:57 UTC, 1 replies.
- [PR] Update celeborn conf to add S3 in default and doc for policy [celeborn] - posted by "ashangit (via GitHub)" <gi...@apache.org> on 2025/04/21 21:06:08 UTC, 2 replies.
- [PR] [RFT] Enable run java UT (not a formal PR, just for discussion purposes) [celeborn] - posted by "vastian180 (via GitHub)" <gi...@apache.org> on 2025/04/22 05:28:49 UTC, 2 replies.
- [PR] [CELEBORN-1841][FOLLOWUP] Support custom implementation of EventExecutorChooser to avoid deadlock when calling await in EventLoop thread for EpollEventLoopGroup [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/04/22 07:26:41 UTC, 2 replies.
- [PR] [CELEBORN-1974] ApplicationId as metrics label should be behind a config flag [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2025/04/22 08:58:15 UTC, 25 replies.
- [PR] [CELEBORN-1976] CommitHandler should use celeborn.client.rpc.commitFiles.askTimeout for timeout of doParallelCommitFiles [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/04/23 05:27:37 UTC, 2 replies.
- [PR] [CELEBORN-1977] Add help/type on prometheus exposed metrics [celeborn] - posted by "ashangit (via GitHub)" <gi...@apache.org> on 2025/04/23 06:46:40 UTC, 3 replies.
- [PR] [CELEBORN-1760][FOLLOWUP] remove redundant release on data added in flushBuffer [celeborn] - posted by "littlexyw (via GitHub)" <gi...@apache.org> on 2025/04/23 07:49:43 UTC, 6 replies.
- [PR] [CELEBORN-1972 ][HELM] Rename affinity.{master,worker} to {master,worker}.affinity [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/24 03:06:02 UTC, 2 replies.
- [PR] [CELEBORN-1980][HELM] Split environments into master.env and worker.env [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/24 03:23:03 UTC, 2 replies.
- [PR] [CELEBORN-1981][HELM] Rename masterReplicas and workerReplicas to master.replicas and worker.replicas [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/24 03:36:01 UTC, 4 replies.
- [PR] [CELEBORN-1982] Slot Selection Perf Improvements [celeborn] - posted by "akpatnam25 (via GitHub)" <gi...@apache.org> on 2025/04/24 07:16:38 UTC, 8 replies.
- [PR] [CELEBORN-1979] Change partition manager should respect the `celeborn.storage.availableTypes` [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2025/04/24 08:58:41 UTC, 2 replies.
- Re: [PR] [CELEBORN-1841] Support custom implementation of EventExecutorChooser to avoid deadlock when calling await in EventLoop thread [celeborn] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/04/24 12:01:13 UTC, 8 replies.
- [PR] [CELEBORN-1983] Fix fetch fail not throw due to reach spark maxTaskFa… [celeborn] - posted by "buska88 (via GitHub)" <gi...@apache.org> on 2025/04/25 05:19:00 UTC, 1 replies.
- [PR] Merge Resource.proto into TransportMessages.proto [celeborn] - posted by "zhaohehuhu (via GitHub)" <gi...@apache.org> on 2025/04/25 08:04:23 UTC, 1 replies.
- Re: [PR] [CELEBORN-1811] Update default value for `celeborn.master.slot.assign.extraSlots` [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2025/04/25 12:13:50 UTC, 0 replies.
- Re: [PR] [CELEBORN-1983] Fix fetch fail not throw due to reach spark maxTaskFailures [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/27 01:37:45 UTC, 1 replies.
- [PR] [CELEBORN-1501][FOLLOWUP] Add bytes written threshold for top app consumption [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/27 21:01:43 UTC, 0 replies.
- [PR] [CELEBORN-1985][HELM] Add new values master.envFrom and worker.envFrom [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/28 06:52:39 UTC, 2 replies.
- [PR] Rename priorityClass.{master,worker} to {master,worker}.priorityClass [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/28 07:08:41 UTC, 0 replies.
- Re: [PR] [CELEBORN-1986][HELM] Rename priorityClass.{master,worker} to {master,worker}.priorityClass [celeborn] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/04/28 08:05:33 UTC, 1 replies.
- [PR] [MINOR][INFRA] Do not cancel GHA jobs on committing to main/branch-* branches [celeborn] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/04/28 08:14:25 UTC, 2 replies.
- [PR] [CELEBORN-1987][HELM] Split dnsPolicy into master.dnsPolicy and worker.dnsPolicy [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/04/28 11:25:39 UTC, 0 replies.
- [PR] åCELEBORN-1990] Add support for kubernetes PVC in helm chart [celeborn] - posted by "ashangit (via GitHub)" <gi...@apache.org> on 2025/04/28 12:22:44 UTC, 0 replies.
- [PR] CELEBORN-1991: add retries for registerShuffle on the lifecycleManager [celeborn] - posted by "shahryart (via GitHub)" <gi...@apache.org> on 2025/04/28 13:11:21 UTC, 1 replies.
- Re: [PR] [CELEBORN-1501][FOLLOWUP] Add bytes written threshold for top app consumption metrics [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/04/28 16:58:52 UTC, 1 replies.
- [PR] [CELEBORN-1912] Client should send heartbeat to worker for processing heartbeat to avoid reading idleness of worker which enables heartbeat [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/04/30 06:57:04 UTC, 4 replies.