issues@celeborn.apache.org, 2025-05

You are viewing a plain text version of this content. The canonical link for it is here.

- Re: [PR] [CELEBORN-1896] delete data from failed to fetch shuffles [celeborn] - posted by "mridulm (via GitHub)" <gi...@apache.org> on 2025/05/01 01:17:08 UTC, 26 replies.
- Re: [PR] [CELEBORN-1902] Read client throws PartitionConnectionException [celeborn] - posted by "venkata91 (via GitHub)" <gi...@apache.org> on 2025/05/02 05:12:57 UTC, 12 replies.
- Re: [PR] [CELEBORN-1912] Client should send heartbeat to worker for processing heartbeat to avoid reading idleness of worker which enables heartbeat [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/02 06:39:12 UTC, 4 replies.
- Re: [PR] [CELEBORN-1768][WRITER] Refactoring Shuffle Writer to extract common methods [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/05/02 08:36:02 UTC, 3 replies.
- Re: [PR] [CELEBORN-894] Add checksum for shuffle data [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/05/03 08:34:33 UTC, 1 replies.
- Re: [PR] [CELEBORN-1990] Add support for kubernetes PVC in helm chart [celeborn] - posted by "ashangit (via GitHub)" <gi...@apache.org> on 2025/05/03 12:20:55 UTC, 6 replies.
- [PR] [CELEBORN-1487][FOLLOWUP] Fix updateProduceBytes [celeborn] - posted by "vastian180 (via GitHub)" <gi...@apache.org> on 2025/05/05 06:04:39 UTC, 3 replies.
- [PR] [CELEBORN-1992] Ensure hadoop FS are not closed by hadoop ShutdownHookManager [celeborn] - posted by "ashangit (via GitHub)" <gi...@apache.org> on 2025/05/05 13:18:57 UTC, 1 replies.
- [D] Are these supported versions? [celeborn] - posted by "developer-guy (via GitHub)" <gi...@apache.org> on 2025/05/05 14:05:51 UTC, 3 replies.
- Re: [PR] [CELEBORN-1984] Merge ResourceRequest to transportMessageProtobuf [celeborn] - posted by "zhaohehuhu (via GitHub)" <gi...@apache.org> on 2025/05/06 06:51:47 UTC, 0 replies.
- [PR] [CELEBORN-1965] Rely on all default hadoop providers for S3 auth [celeborn] - posted by "ashangit (via GitHub)" <gi...@apache.org> on 2025/05/06 07:53:55 UTC, 2 replies.
- Re: [PR] Set max memory of SortBasedPusher based off Spark configs [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/05/06 08:37:01 UTC, 4 replies.
- Re: [PR] [CELEBORN-1855] LifecycleManager return appshuffleId for non barrier stage when fetch fail has been reported [celeborn] - posted by "buska88 (via GitHub)" <gi...@apache.org> on 2025/05/06 10:46:09 UTC, 6 replies.
- Re: [PR] [CELEBORN-1760][FOLLOWUP] Remove redundant release on data added in flushBuffer [celeborn] - posted by "littlexyw (via GitHub)" <gi...@apache.org> on 2025/05/07 03:27:29 UTC, 4 replies.
- [PR] [CELEBORN-1601][FOLLOWUP] Remove redundant logs of expired shuffle in master-worker heartbeat [celeborn] - posted by "Z1Wu (via GitHub)" <gi...@apache.org> on 2025/05/07 07:34:50 UTC, 4 replies.
- [PR] [CELEBORN-1993] CelebornConf introduces celeborn..io.threads to specify number of threads used in the server and client thread pool [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/07 12:01:13 UTC, 2 replies.
- [PR] [CELEBORN-1994] Introduce disruptor dependency to support asynchronous logging of log4j2 [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/07 12:37:47 UTC, 7 replies.
- Re: [PR] Update celeborn conf to add S3 in default and doc for policy [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2025/05/08 08:52:57 UTC, 0 replies.
- Re: [PR] [CELEBORN-1974] ApplicationId as metrics label should be behind a config flag [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/05/08 09:04:27 UTC, 4 replies.
- Re: [PR] [CELEBORN-1968] Publish metric for unreleased partition location count when worker was gracefully shutdown [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/05/08 09:58:58 UTC, 4 replies.
- [PR] [CELEBORN-1319][CELEBORN-474][FOLLOWUP] PushState uses JavaUtils#newConcurrentHashMap to speed up ConcurrentHashMap#computeIfAbsent [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/08 11:30:58 UTC, 1 replies.
- [PR] [CELEBORN-1917] supports celeborn.client.push.maxBytesSizeInFlight [celeborn] - posted by "DDDominik (via GitHub)" <gi...@apache.org> on 2025/05/08 11:56:48 UTC, 3 replies.
- [PR] [CELEBORN-1760][FOLLOWUP] Remove redundant release on data added in f… [celeborn] - posted by "littlexyw (via GitHub)" <gi...@apache.org> on 2025/05/08 12:44:01 UTC, 0 replies.
- Re: [PR] [CELEBORN-1948] Worker handle replica data callback should not ignore HARD_SPLIT [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/05/09 02:18:21 UTC, 0 replies.
- Re: [PR] [CELEBORN-1987][HELM] Split dnsPolicy into master.dnsPolicy and worker.dnsPolicy [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/05/09 02:26:50 UTC, 1 replies.
- Re: [PR] [CELEBORN-1948] Fix the issue where replica may lose data when HARD_SPLIT occurs during handlePushMergeData. [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/05/09 02:40:01 UTC, 1 replies.
- [PR] [CELEBORN-1988][HELM] Split hostNetwork into master.hostNetwork and worker.hostNetwork [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/05/09 02:40:08 UTC, 2 replies.
- [PR] [CELEBORN-1989][HELM] Split securityContext into master.podSecurityContext and worker.podSecurityContext [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/05/09 04:03:59 UTC, 5 replies.
- [PR] [CELEBORN-1978][CIP-14] Add code style checking for cppClient [celeborn] - posted by "HolyLow (via GitHub)" <gi...@apache.org> on 2025/05/09 08:16:35 UTC, 2 replies.
- Re: [PR] CELEBORN-1334 [CIP-12] Support HARD_SPLIT in PushMergedData should not support new version client with old version worker [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/05/09 08:36:31 UTC, 1 replies.
- Re: [PR] [CELEBORN-1492] Introduce Celeborn Chaos Testing Framework [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/05/10 08:34:35 UTC, 2 replies.
- Re: [PR] [CELEBORN-1926] updated helmChart to allow different nodeSelector and tolerations for master and worker nodes [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/05/10 13:52:12 UTC, 1 replies.
- [PR] [CELEBORN-1995] Optimize memory usage for push failed batches [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2025/05/12 07:02:05 UTC, 6 replies.
- Re: [PR] [CELEBORN-1800] Introduce ApplicationTotalCount and ApplicationFallbackCount metric to record the total and fallback count of application [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/12 08:36:19 UTC, 5 replies.
- [PR] [CELEBORN-1996][HELM] Rename volumes.{master,worker} to {master,worker}.volumes and {master.worker}.volumeMounts [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/05/13 10:08:38 UTC, 4 replies.
- Re: [PR] [CELEBORN-1993] CelebornConf introduces celeborn..io.threads to specify number of threads used in the client thread pool [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/13 12:05:36 UTC, 4 replies.
- Re: [PR] [CELEBORN-1903] Enable connection heartbeat and close idle connection [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/13 12:07:31 UTC, 3 replies.
- [PR] [CELEBORN-1691][FOLLOWUP] Fix the issue that upstream tasks don't rerun and the current task still retry when failed to deserialize in flink [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/13 13:12:17 UTC, 1 replies.
- [PR] Is stage end [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/16 15:04:11 UTC, 0 replies.
- Re: [PR] [CELEBORN-2000] Ignore the getReducerFileGroup timeout before shuffle stage end [celeborn] - posted by "mridulm (via GitHub)" <gi...@apache.org> on 2025/05/16 17:03:38 UTC, 9 replies.
- Re: [PR] [CELEBORN-894] End to End Integrity Checks [celeborn] - posted by "gauravkm (via GitHub)" <gi...@apache.org> on 2025/05/16 20:03:06 UTC, 21 replies.
- [PR] Is stage end v2 [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/17 06:34:00 UTC, 4 replies.
- Re: [PR] [CELEBORN-2000] Wait for the shuffle stage end before getting reducer file group [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/17 13:21:09 UTC, 5 replies.
- Re: [PR] [RFT] Enable run java UT (not a formal PR, just for discussion purposes) [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/05/18 08:34:40 UTC, 0 replies.
- [PR] Audit shuffle in separate log file [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/18 16:03:26 UTC, 0 replies.
- Re: [PR] CELEBORN-1991: add retries for registerShuffle on the lifecycleManager [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/05/19 08:38:08 UTC, 0 replies.
- Re: [PR] [CELEBORN-2002][MASTER] Audit shuffle lifecycle in separate log file [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/19 14:36:33 UTC, 4 replies.
- [PR] [CELEBORN-2004] Filter empty partition before createIntputStream [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2025/05/20 04:06:10 UTC, 4 replies.
- [PR] UT debug [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/20 14:22:10 UTC, 4 replies.
- [PR] [CELEBORN-1916][FOLLOWUP] Improve Aliyun OSS support [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/21 03:15:19 UTC, 2 replies.
- [PR] [MINOR] Change some config version [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/21 11:43:49 UTC, 2 replies.
- [PR] [CELEBORN-1775][FOLLOWUP] Improve logging around commit files [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2025/05/21 18:52:20 UTC, 3 replies.
- [PR] Release guide [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/22 01:01:02 UTC, 2 replies.
- [PR] [CELEBORN-2005] Introduce numBytesIn, numBytesOut, numBytesInPerSecond, numBytesOutPerSecond metrics for RemoteShuffleServiceFactory [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/22 04:02:57 UTC, 10 replies.
- [PR] [CELEBORN-2006] LifecycleManager should avoid parsing shufflePartitionType every time [celeborn] - posted by "cxzl25 (via GitHub)" <gi...@apache.org> on 2025/05/22 09:24:49 UTC, 2 replies.
- [PR] [CELEBORN-2007] Reduce PartitionLocation memory usage [celeborn] - posted by "cxzl25 (via GitHub)" <gi...@apache.org> on 2025/05/22 09:31:09 UTC, 3 replies.
- [PR] [CELEBORN-2008] SlotsAllocator should select disks randomly in RoundRobin mode [celeborn] - posted by "leixm (via GitHub)" <gi...@apache.org> on 2025/05/22 09:38:33 UTC, 3 replies.
- [PR] [CELEBORN-2009] Commit files request failure should exclude worker in LifecycleManager [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2025/05/22 09:51:58 UTC, 3 replies.
- [PR] [CELEBORN-1528][HELM] Use volume claim template to support various storage backend [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/05/22 11:42:59 UTC, 2 replies.
- Re: [PR] [CELEBORN-1788] Add role and roleBinding helm charts [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/05/22 11:53:46 UTC, 0 replies.
- Re: [PR] [CELEBORN-2010][INFRA] Add release guide [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/24 15:27:18 UTC, 3 replies.
- [PR] [CELEBORN-2010][FOLLOWUP] Fix svn staging dir [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/25 02:55:07 UTC, 2 replies.
- [PR] [DRAFT] [FEATURE] Inverted index based reduce partition data writer / reader [celeborn] - posted by "saurabhd336 (via GitHub)" <gi...@apache.org> on 2025/05/25 05:23:44 UTC, 6 replies.
- [PR] Add release utils [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/25 07:08:25 UTC, 0 replies.
- [PR] add license [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/26 01:29:46 UTC, 0 replies.
- Re: [PR] [CELEBORN-2011][INFRA] Add a script to simplify the process of creating release notes [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/26 01:49:40 UTC, 1 replies.
- Re: [PR] [CELEBORN-2012] Add license for http5 [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/26 02:23:06 UTC, 1 replies.
- [PR] Spark 4.0 [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/26 04:13:41 UTC, 0 replies.
- [PR] Spark 4.0 ga [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/26 05:05:46 UTC, 1 replies.
- [PR] [CELEBORN-2013] Upgrade scala binary version of spark-3.3, spark-3.4, spark-3.5 profile to 2.13.8 [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/26 06:29:44 UTC, 3 replies.
- [PR] [CELEBORN-2014] updateInterruptionNotice REST API [celeborn] - posted by "akpatnam25 (via GitHub)" <gi...@apache.org> on 2025/05/26 07:37:23 UTC, 7 replies.
- Re: [PR] [CELEBORN-1413][FOLLOWUP] Bump spark 4.0 version to 4.0.0 [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/26 10:01:03 UTC, 19 replies.
- [PR] Setup lifecycle exception [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2025/05/26 10:37:11 UTC, 0 replies.
- [PR] [MINOR][DOC] Correct configuration values in slotsallocation [celeborn] - posted by "cxzl25 (via GitHub)" <gi...@apache.org> on 2025/05/26 11:52:58 UTC, 3 replies.
- [PR] [CELEBORN-XXXX] Fix Deserialization Vulnerability [celeborn] - posted by "th555555 (via GitHub)" <gi...@apache.org> on 2025/05/26 13:45:50 UTC, 0 replies.
- Re: [PR] [CELEBORN-3288] Fix Deserialization Vulnerability [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2025/05/27 02:40:19 UTC, 0 replies.
- [PR] Spark 4.0 tests [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/27 03:45:49 UTC, 1 replies.
- Re: [I] Who is using Apache Celeborn? [celeborn] - posted by "DDDominik (via GitHub)" <gi...@apache.org> on 2025/05/27 05:02:38 UTC, 0 replies.
- [PR] Spark 4.0 tests bk [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/27 06:03:14 UTC, 1 replies.
- [PR] Co exists [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/27 06:30:43 UTC, 1 replies.
- [PR] Fix spark it [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/27 06:38:44 UTC, 1 replies.
- Re: [PR] [CELEBORN-2015] Retry IOException failures for RPC requests [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/27 09:28:11 UTC, 2 replies.
- [PR] [CELEBORN-2003] Add retry mechanism when completing S3 multipart upload [celeborn] - posted by "ashangit (via GitHub)" <gi...@apache.org> on 2025/05/27 12:20:08 UTC, 2 replies.
- [PR] [CELEBORN-2016] Add cooldown time in worker shutdown [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2025/05/27 17:05:45 UTC, 1 replies.
- [PR] [CELEBORN-2017][HELM] Add namespace to the metadata [celeborn] - posted by "ChenYi015 (via GitHub)" <gi...@apache.org> on 2025/05/28 03:49:11 UTC, 0 replies.
- [PR] [CELEBORN-2005][FOLLOWUP] Introduce ShuffleMetricGroup for numBytesIn, numBytesOut, numRecordsOut, numBytesInPerSecond, numBytesOutPerSecond, numRecordsOutPerSecond metrics [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/28 06:18:01 UTC, 5 replies.
- [PR] [CELEBORN-2018] Support min number of workers selected for shuffle [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2025/05/28 11:16:42 UTC, 8 replies.
- [PR] [CELEBORN-1413][FOLLOWUP] Support dependencies of spark-4.0 profile [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2025/05/29 07:04:54 UTC, 2 replies.
- [PR] [CELEBORN-1627][FOLLOWUP] Fix the issue where the case of name affects the metrics dashboard [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/05/30 02:03:40 UTC, 7 replies.
- [PR] auth header [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/30 02:38:32 UTC, 0 replies.
- [PR] appId for debug [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/30 05:34:13 UTC, 0 replies.
- Re: [PR] [CELEBORN-2020] Support http authentication for Celeborn CLI [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2025/05/30 07:28:47 UTC, 0 replies.
- Re: [PR] [CELEBORN-1997] RemoteShuffleInputGateDelegation should regard CelebornIOException as data consumption error for RestartPipelinedRegionFailoverStrategy [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/05/30 08:52:31 UTC, 1 replies.
- [PR] [CELEBORN-2021] Fix issues on regression HDFS and OSS before release 0.6 [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2025/05/30 09:25:58 UTC, 1 replies.