issues@celeborn.apache.org, 2024-11

You are viewing a plain text version of this content. The canonical link for it is here.

- Re: [PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2024/11/01 01:51:40 UTC, 1 replies.
- Re: [PR] [CELEBORN-1671] CelebornShuffleReader will try replica if create client failed [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2024/11/01 02:05:41 UTC, 1 replies.
- Re: [PR] [CELEBORN-1675] User with zero quota should not be allowed to submit any shuffle to cluster. [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2024/11/01 02:21:02 UTC, 4 replies.
- Re: [PR] [CELEBORN-1490][CIP-6] Add Flink Hybrid Shuffle IT test cases [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2024/11/01 02:51:50 UTC, 2 replies.
- Re: [PR] [CELEBORN-1672] Bump Spark from 3.4.3 to 3.4.4 [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2024/11/01 03:06:04 UTC, 0 replies.
- Re: [PR] [CELEBORN-1660] Using map for workers to find worker fast and WokerInfo pool to reuse objects [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2024/11/01 03:40:00 UTC, 4 replies.
- [PR] [CELEBORN-1636][FOLLOWUP] Add Factor and get available workers from requestSlots RPC [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2024/11/01 03:45:09 UTC, 3 replies.
- [PR] Revert worker info pool [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/01 05:13:08 UTC, 1 replies.
- [PR] [CELEBORN-1660] Using map for workers to find worker fast [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/01 05:16:23 UTC, 22 replies.
- Re: [PR] [CELEBORN-1490][CIP-6] Add Flink hybrid shuffle doc [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/01 05:37:25 UTC, 0 replies.
- Re: [PR] [CELEBORN-1680] Introduce ShuffleFallbackCount metrics [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/01 07:01:03 UTC, 22 replies.
- Re: [PR] [CELEBORN-1477][FOLLOWUP] Remove scala binary version from openapi-client artifactId [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/01 07:08:11 UTC, 0 replies.
- [PR] [CELEBORN-1531][FOLLOWUP] Assign the scheduleCheckTask [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/01 07:28:38 UTC, 3 replies.
- [PR] [CELEBORN-1564][FOLLOWUP] Remove unused variables [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/01 07:38:15 UTC, 1 replies.
- Re: [PR] [CELEBORN-1636][FOLLOWUP] Dynamic resources will only be utilized in case of candidates shortages [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2024/11/01 07:55:56 UTC, 10 replies.
- Re: [PR] [CELEBORN-1660] Only count the available workers device free capacity [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/01 08:35:16 UTC, 2 replies.
- [PR] [CELEBORN-1490][CIP-6] Support process large buffer in flink hybrid shuffle [celeborn] - posted by "reswqa (via GitHub)" <gi...@apache.org> on 2024/11/01 10:12:41 UTC, 6 replies.
- Re: [PR] [CELEBORN-1679] Estimated ApplicationDiskUsage in cluster should be multiplied by worker count. [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/01 11:56:07 UTC, 5 replies.
- Re: [PR] [CELEBORN-1657] Throw exception to client when exception occurred in reading sort shuffle files [celeborn] - posted by "mridulm (via GitHub)" <gi...@apache.org> on 2024/11/03 16:25:09 UTC, 1 replies.
- Re: [PR] [CELEBORN-1648] Refine AppUniqueId with UUID suffix [celeborn] - posted by "mridulm (via GitHub)" <gi...@apache.org> on 2024/11/03 16:32:50 UTC, 1 replies.
- [PR] add jdk tools into classpath [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/03 23:58:23 UTC, 0 replies.
- Re: [PR] [CELEBORN-1682] Add jave tools.jar into classpath for JVM quake [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/04 00:20:24 UTC, 0 replies.
- Re: [PR] [CELEBORN-1636][FOLLOWUP] Cache available workers in Master [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/04 00:58:14 UTC, 2 replies.
- Re: [PR] [CELEBORN-1660] Cache available workers and only count the available workers device free capacity [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/04 00:59:22 UTC, 12 replies.
- Re: [PR] [WIP][CELEBORN-1577][Phase2] QuotaManager should support interrupt shuffle. [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2024/11/04 02:29:06 UTC, 1 replies.
- Re: [PR] [CELEBORN-1682] Add java tools.jar into classpath for JVM quake [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/04 03:05:20 UTC, 0 replies.
- [PR] [CELEBORN-1678][FOLLOWUP] Improve Celeborn CLI user guide [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/04 06:00:38 UTC, 5 replies.
- Re: [PR] [WIP][CELEBORN-1634] implement queue time/processing time metrics for rpc framework [celeborn] - posted by "ErikFang (via GitHub)" <gi...@apache.org> on 2024/11/04 16:55:56 UTC, 1 replies.
- [PR] Worker version [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/04 23:42:31 UTC, 0 replies.
- [PR] [CELEBORN-1684]Modify ambiguous documents [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2024/11/05 02:25:50 UTC, 8 replies.
- Re: [PR] [CELEBORN-1683] Support to get worker version [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/05 03:10:20 UTC, 2 replies.
- Re: [PR] [CELEBORN-1531][FOLLOWUP] Assign the scheduled check task [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2024/11/05 03:18:50 UTC, 1 replies.
- Re: [PR] [CELEBORN-1684] Fix ambiguous client jar expression of document [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/05 03:42:34 UTC, 2 replies.
- Re: [PR] [CELEBORN-1619][CIP-11] Integrate tags manager with config service [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2024/11/05 03:56:07 UTC, 7 replies.
- Re: [PR] [CELEBORN-1683] Support to get worker versions [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/05 04:58:29 UTC, 0 replies.
- Re: [PR] [CELEBORN-1683] Support to check worker versions [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/05 08:14:44 UTC, 6 replies.
- Re: [PR] [CELEBORN-1413] Support Spark 4.0 [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/11/05 08:33:55 UTC, 3 replies.
- [PR] [CELEBORN-1686] Avoid return the same pushTaskQueue [celeborn] - posted by "cxzl25 (via GitHub)" <gi...@apache.org> on 2024/11/05 11:11:49 UTC, 6 replies.
- [PR] [CELEBORN-1687] Highlight flink session cluster issue in doc [celeborn] - posted by "reswqa (via GitHub)" <gi...@apache.org> on 2024/11/05 12:09:08 UTC, 1 replies.
- [PR] [CELEBORN-1190][FOLLOWUP] Fix WARNING of error prone [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/05 12:57:26 UTC, 6 replies.
- [PR] add spark version [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2024/11/05 17:49:46 UTC, 0 replies.
- Re: [PR] [CELEBORN-1528][HELM] Use volumeClaimTemplates to support various storage backend [celeborn] - posted by "zemin-piao (via GitHub)" <gi...@apache.org> on 2024/11/05 21:35:34 UTC, 0 replies.
- [PR] CELEBORN-1689: CLI Support for Ratis APIs [celeborn] - posted by "akpatnam25 (via GitHub)" <gi...@apache.org> on 2024/11/05 23:59:07 UTC, 12 replies.
- [PR] [MINOR] Remove unused TODO comments in CelebornTierProducerAgent#processBuffer [celeborn] - posted by "reswqa (via GitHub)" <gi...@apache.org> on 2024/11/06 01:53:27 UTC, 2 replies.
- [PR] [CELEBORN-1691] Fix the issue that upstream tasks don't rerun and the current task still retry when failed to decompress in flink [celeborn] - posted by "reswqa (via GitHub)" <gi...@apache.org> on 2024/11/06 02:28:54 UTC, 1 replies.
- [PR] [CELEBORN-1692] Set mount pint in fromPbFileInfoMap [celeborn] - posted by "reswqa (via GitHub)" <gi...@apache.org> on 2024/11/06 02:42:12 UTC, 0 replies.
- [PR] [CELEBORN-1693] Fix storageFetcherPool concurrent problem [celeborn] - posted by "reswqa (via GitHub)" <gi...@apache.org> on 2024/11/06 03:12:19 UTC, 9 replies.
- [PR] [CELEBORN-1696] StorageManager#cleanFile should remove file info [celeborn] - posted by "reswqa (via GitHub)" <gi...@apache.org> on 2024/11/06 05:44:53 UTC, 1 replies.
- [PR] [CELEBORN-1697] Improve ThreadStackTrace for thread dump [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/06 11:54:06 UTC, 1 replies.
- [PR] [CELEBORN-1689] optimize Master AppHeartbeat/RequestSlots RPC performance [celeborn] - posted by "xy2953396112 (via GitHub)" <gi...@apache.org> on 2024/11/06 12:36:57 UTC, 1 replies.
- Re: [PR] [CELEBORN-1692] Set mount point in fromPbFileInfoMap [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/07 06:12:50 UTC, 0 replies.
- [PR] nit [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/07 07:32:15 UTC, 0 replies.
- [PR] [CELEBORN-1685] ShuffleFallbackPolicy supports ShuffleFallbackCount metric [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/07 09:52:45 UTC, 20 replies.
- Re: [PR] [MINOR] Fix docs typo [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/07 12:34:22 UTC, 0 replies.
- [PR] format [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/07 19:16:22 UTC, 0 replies.
- Re: [PR] [CELEBORN-1601][CELEBORN-1697][FOLLOWUP] Refine the RESTful apis for revise lost shuffles and fix ThreadStack REST model [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/07 20:40:24 UTC, 9 replies.
- [PR] [CELEBORN-1699] Add GA to check the openapi codegen [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/08 00:23:00 UTC, 7 replies.
- Re: [PR] [CELEBORN-1641]Pack the output spark client jar with the major.minor version [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2024/11/08 02:56:35 UTC, 3 replies.
- Re: [PR] [CELEBORN-1601][FOLLOWUP] Refine the RESTful apis for revise lost shuffles [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/08 03:00:47 UTC, 4 replies.
- Re: [PR] [CELEBORN-1641] Pack the output spark client jar with the major.minor version [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2024/11/08 03:14:29 UTC, 10 replies.
- [PR] [CELEBORN-1071] Support stage rerun for shuffle data lost [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/08 07:04:04 UTC, 2 replies.
- Re: [PR] [CELEBORN-1451] HPA support [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/11/08 08:33:54 UTC, 1 replies.
- [PR] Separate the rest audit log file [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/08 19:08:00 UTC, 0 replies.
- [PR] use unit decbytes [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/10 06:46:08 UTC, 0 replies.
- Re: [PR] [WIP][CELEBORN-1319] Optimize skew partition logic for Reduce Mode to avoid sorting shuffle files [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/11/10 08:32:49 UTC, 4 replies.
- Re: [PR] [CELEBORN-1626] group mapTask [celeborn] - posted by "codecov[bot] (via GitHub)" <gi...@apache.org> on 2024/11/11 02:53:18 UTC, 0 replies.
- Re: [PR] [CELEBORN-1530] support MPU for S3 [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/11 03:46:23 UTC, 5 replies.
- [PR] [CELEBORN-1702] Bump Ratis version from 3.1.1 to 3.1.2 [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/11 04:20:10 UTC, 4 replies.
- [PR] [CELEBORN-1708] Bump protobuf from 3.21.7 to 3.25.5 [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/11 04:34:32 UTC, 0 replies.
- [PR] [CELEBORN-1709] Bump jetty version from 9.4.52.v20230823 to 9.4.56.v2… [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/11 04:38:49 UTC, 0 replies.
- [PR] [CELEBORN-1710] Bump commons-io from 2.13.0 to 2.17.0 [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/11 04:44:43 UTC, 0 replies.
- Re: [PR] [CELEBORN-1708] Bump protobuf version from 3.21.7 to 3.25.5 [celeborn] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2024/11/11 05:22:37 UTC, 5 replies.
- Re: [PR] [CELEBORN-1706] Use bytes(IEC) unit instead of bytes(SI) for size related metrics in prometheus dashboard [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/11 05:57:17 UTC, 2 replies.
- [PR] [CELEBORN-1711][TEST] Fix '65535' in celeborn.master/worker.http.port is invalid [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/11 06:31:47 UTC, 3 replies.
- [PR] [CELEBORN-1504][FOLLOWUP] Start test nodes in random ports to allow multiple builds run in the same ci server [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/11 07:59:36 UTC, 2 replies.
- Re: [PR] [CELEBORN-1710] Bump commons-io version from 2.13.0 to 2.17.0 [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/11 08:57:37 UTC, 0 replies.
- Re: [PR] [CELEBORN-1709] Bump jetty version from 9.4.52.v20230823 to 9.4.56.v20240826 [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/11 08:58:53 UTC, 0 replies.
- [PR] [CELEBORN-1712] Bump Netty version from 4.1.109.Final to 4.1.114.Final [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/11 09:08:01 UTC, 2 replies.
- [PR] [CELEBORN-1504] Document adds Flink 1.16 support [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/11 09:34:51 UTC, 1 replies.
- [PR] [CELEBORN-XXX] Optimize performance of handling `MapperEnd` RPC in `LifecycleManager` [celeborn] - posted by "cfmcgrady (via GitHub)" <gi...@apache.org> on 2024/11/11 11:24:01 UTC, 0 replies.
- Re: [PR] [CELEBORN-1701] Support stage rerun for shuffle data lost [celeborn] - posted by "RexXiong (via GitHub)" <gi...@apache.org> on 2024/11/12 02:09:43 UTC, 0 replies.
- [PR] [CELEBORN-1711][TEST] Setup celeborn master with retry on port BindException [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/12 05:09:33 UTC, 0 replies.
- [PR] [CELEBORN-1713] RpcTimeoutException should include RPC address in message [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/12 06:53:02 UTC, 2 replies.
- Re: [PR] [CELEBORN-1711][TEST] Fix '65535' port is invalid [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/12 07:38:37 UTC, 1 replies.
- Re: [PR] [CELEBORN-1711][TEST] Setup celeborn master/worker http server with retry on port BindException [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/12 08:36:45 UTC, 3 replies.
- [PR] [CELEBORN-1631][FOLLOWUP] Add latest snapshot index in message of HandleResponse for /snapshot/create [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/12 09:04:40 UTC, 2 replies.
- [PR] [CELEBORN-1685][FOLLOWUP] CelebornShuffleFallbackPolicyRunner should compute shuffleFallbackCounts via class name of ShuffleFallbackPolicy implementation [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/12 12:15:44 UTC, 2 replies.
- [PR] [CELEBORN-1714] Optimize handleApplicationLost [celeborn] - posted by "xy2953396112 (via GitHub)" <gi...@apache.org> on 2024/11/12 13:08:54 UTC, 1 replies.
- [PR] [CELEBORN-1715] RemoteShuffleMaster should check celeborn.client.push.replicate.enabled in constructor [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/12 14:13:09 UTC, 8 replies.
- Re: [PR] [CELEBORN-1634] implement queue time/processing time metrics for rpc framework [celeborn] - posted by "ErikFang (via GitHub)" <gi...@apache.org> on 2024/11/12 14:27:37 UTC, 2 replies.
- Re: [PR] [CELEBORN-914] Support memory file storage [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/12 18:16:51 UTC, 1 replies.
- [PR] do not return buffer dup [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/12 21:15:30 UTC, 0 replies.
- Re: [PR] [WIP][CELEBORN-1705] Fix disk buffer size negative issue [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/12 21:19:37 UTC, 0 replies.
- [PR] [CELEBORN-1705] Fix disk buffer size negative issue [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/12 21:23:01 UTC, 0 replies.
- Re: [PR] [WIP][CELEBORN-1705] Fix abnormal disk buffer size/memory file storage size [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/12 22:53:42 UTC, 0 replies.
- [PR] [CELEBORN-1717] Fix ReusedExchangedSuit UT bug [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2024/11/13 09:13:21 UTC, 2 replies.
- [PR] [CELEBORN-1716] Sleeping in CelebornShuffleReader while loop should have an upper bound time [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2024/11/13 09:26:21 UTC, 3 replies.
- Re: [PR] [CELEBORN-1504][FOLLOWUP] Document adds Flink 1.16 support [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/13 13:47:42 UTC, 0 replies.
- [PR] update the disk buffer or memory file storage after evict [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/13 18:14:25 UTC, 0 replies.
- Re: [PR] [CELEBORN-1705] Fix disk buffer size is negative issue [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/13 18:26:16 UTC, 5 replies.
- Re: [PR] [CELEBORN-1711][TEST] Fix flaky test caused by master/worker setup issue [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/13 23:41:50 UTC, 9 replies.
- [PR] [MINOR] Fix missing blank in docs [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/14 00:52:05 UTC, 0 replies.
- Re: [PR] [MINOR] Fix missing blanks in docs [celeborn] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2024/11/14 02:42:45 UTC, 1 replies.
- [PR] [CELEBORN-1718] Fix memory storage file won't hard split when memory file is full and worker has no disks [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/14 07:25:50 UTC, 1 replies.
- [PR] [CELEBORN-1705][0.5][FOLLWUP] Remove userCongestionControlContext [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/14 07:39:25 UTC, 0 replies.
- Re: [PR] [CELEBORN-1705][0.5][FOLLWUP] Fix cannot find symbol [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/14 07:46:25 UTC, 4 replies.
- [PR] [CELEBORN-1719] Introduce celeborn.client.spark.stageRerun.enabled with alternative celeborn.client.spark.fetch.throwsFetchFailure to enable spark stage rerun [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/14 08:10:06 UTC, 3 replies.
- [PR] Task [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/15 00:12:04 UTC, 0 replies.
- Re: [PR] [WIP][CELEBORN-1720] Prevent stage re-run if another task attempt is running [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/15 03:45:34 UTC, 0 replies.
- [PR] [CELEBORN-1619][CELEBORN-474][FOLLOWUP] TagsManager uses JavaUtils#newConcurrentHashMap to speed up ConcurrentHashMap#computeIfAbsent [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/15 07:52:43 UTC, 3 replies.
- [PR] [CELEBORN-1490][FOLLOWUP] Polish CelebornTierProducerAgent [celeborn] - posted by "reswqa (via GitHub)" <gi...@apache.org> on 2024/11/15 08:40:18 UTC, 2 replies.
- [PR] [CELEBORN-1721][CIP-12] Support HARD_SPLIT in PushMergedData [celeborn] - posted by "jiang13021 (via GitHub)" <gi...@apache.org> on 2024/11/15 10:32:13 UTC, 22 replies.
- Re: [PR] [CELEBORN-1707] Audit all RESTful api calls and use separate restAuditFile [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/15 11:37:28 UTC, 0 replies.
- Re: [PR] [CELEBORN-1634] Support queue time/processing time metrics for rpc framework [celeborn] - posted by "ErikFang (via GitHub)" <gi...@apache.org> on 2024/11/16 04:09:38 UTC, 2 replies.
- Re: [PR] [CELEBORN-1564] Fix actualUsableSpace of offerSlotsLoadAware condition on diskInfo [celeborn] - posted by "onebox-li (via GitHub)" <gi...@apache.org> on 2024/11/18 09:10:09 UTC, 2 replies.
- [PR] [CELEBORN-1618][CIP-11] Supporting tags via DB Config Service [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2024/11/18 14:19:53 UTC, 16 replies.
- [PR] [CELEBORN-1686][FOLLOWUP] Avoid NPE when clean up data pusher [celeborn] - posted by "Z1Wu (via GitHub)" <gi...@apache.org> on 2024/11/18 15:07:11 UTC, 4 replies.
- Re: [I] Who is using Apache Celeborn? [celeborn] - posted by "conorzhao (via GitHub)" <gi...@apache.org> on 2024/11/19 07:25:37 UTC, 2 replies.
- Re: [PR] [CELEBORN-1595] Support quota low watermark for checking quota available [celeborn] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/11/19 08:35:21 UTC, 2 replies.
- [PR] [CELEBORN-1724][CIP-14] Add environment setup tools for CppClient development [celeborn] - posted by "HolyLow (via GitHub)" <gi...@apache.org> on 2024/11/19 11:12:12 UTC, 5 replies.
- [PR] [CELEBORN-1485] Refactor addCounter, addGauge and addTimer of AbstractSource to reduce CPU utilization [celeborn] - posted by "cfmcgrady (via GitHub)" <gi...@apache.org> on 2024/11/19 12:41:12 UTC, 0 replies.
- [PR] [CELEBORN-1556][0.4] Update Github actions to v4 [celeborn] - posted by "cfmcgrady (via GitHub)" <gi...@apache.org> on 2024/11/20 02:33:30 UTC, 3 replies.
- Re: [PR] [CELEBORN-1485][0.4] Refactor addCounter, addGauge and addTimer of AbstractSource to reduce CPU utilization [celeborn] - posted by "cfmcgrady (via GitHub)" <gi...@apache.org> on 2024/11/20 03:32:49 UTC, 2 replies.
- [PR] [CELEBORN-1726] Update WorkerInfo when transition worker state [celeborn] - posted by "reswqa (via GitHub)" <gi...@apache.org> on 2024/11/20 03:33:30 UTC, 2 replies.
- [PR] [CELEBORN-1727] Correct the calculation of worker diskInfo actualUsableSpace [celeborn] - posted by "onebox-li (via GitHub)" <gi...@apache.org> on 2024/11/20 09:41:47 UTC, 3 replies.
- [PR] [CELEBORN-1700] Flink supports fallback to vanilla Flink built-in shuffle implementation [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/20 15:04:07 UTC, 9 replies.
- Re: [PR] [CELEBORN-1720] Prevent stage re-run if another task attempt is running or successful [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/20 20:00:49 UTC, 5 replies.
- [PR] fix npe [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/20 23:45:48 UTC, 0 replies.
- Re: [PR] [CELEBORN-1728] Fix NPE when failing to connect to celeborn worker [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/21 08:23:37 UTC, 0 replies.
- [I] [BUG] 我使用celeborn-client-spark-3-shaded_2.12-0.5.1.jar 与spark 3.4.4_2.12 做demo过程中报错 [celeborn] - posted by "DADA-shuai (via GitHub)" <gi...@apache.org> on 2024/11/21 08:46:49 UTC, 10 replies.
- [PR] [CELEBORN-1634][FOLLOWUP] Simplify the logic of the `RpcSource.addTimer` and `RpcSource.updateTimer` [celeborn] - posted by "cfmcgrady (via GitHub)" <gi...@apache.org> on 2024/11/21 11:01:45 UTC, 4 replies.
- [PR] [CELEBORN-1621][CIP-11] Predefined worker tags expr via dynamic configs [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2024/11/21 11:24:02 UTC, 5 replies.
- [PR] [CELEBORN-1727][FOLLOWUP] Fix CelebornHashCheckDiskSuite flaky test [celeborn] - posted by "onebox-li (via GitHub)" <gi...@apache.org> on 2024/11/21 12:54:44 UTC, 6 replies.
- [PR] [CELEBORN-1726][FOLLOWUP] Avoid NPE when transition worker state [celeborn] - posted by "reswqa (via GitHub)" <gi...@apache.org> on 2024/11/22 05:07:01 UTC, 3 replies.
- [PR] [CELEBORN-1545] Add Tez plugin skeleton and dag app master [celeborn] - posted by "GH-Gloway (via GitHub)" <gi...@apache.org> on 2024/11/22 09:47:20 UTC, 1 replies.
- [PR] [CELEBORN-1741][CIP-14] Add processBase utils to cppClient [celeborn] - posted by "HolyLow (via GitHub)" <gi...@apache.org> on 2024/11/22 17:23:01 UTC, 9 replies.
- Re: [PR] [WIP][CELEBORN-1720] Prevent stage re-run if another task attempt is running or successful [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/22 17:56:48 UTC, 1 replies.
- [PR] Ut spark on yarn [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/22 18:03:35 UTC, 1 replies.
- [PR] Ut for spark utils [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/23 00:01:58 UTC, 1 replies.
- [PR] [CELEBORN-1742] fix bounded driver inbox deadlock [celeborn] - posted by "CodingCat (via GitHub)" <gi...@apache.org> on 2024/11/25 04:28:35 UTC, 0 replies.
- Re: [PR] [CELEBORN-1742] Fix bounded driver Inbox deadlock [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/25 05:55:03 UTC, 9 replies.
- [PR] reduce the size of aws dependencies [celeborn] - posted by "zhaohehuhu (via GitHub)" <gi...@apache.org> on 2024/11/25 08:19:22 UTC, 2 replies.
- [PR] [CELEBORN-1543][FOLLOWUP] License check adds flink-1.20 profile [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/25 08:21:13 UTC, 1 replies.
- [PR] Cleaner metrics [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/25 22:45:30 UTC, 0 replies.
- [PR] [CELEBORN-1745] Fix application top disk usage [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/26 00:57:41 UTC, 5 replies.
- Re: [PR] [CELEBORN-1746] Reduce the size of aws dependencies [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/26 02:30:44 UTC, 4 replies.
- [PR] [MINOR] Update DingTalk group link [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/26 03:39:15 UTC, 3 replies.
- [PR] Remove app top usage [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/26 03:50:21 UTC, 0 replies.
- [PR] Fix flaky test - HybridShuffleWordCountTest [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/26 04:51:26 UTC, 0 replies.
- Re: [PR] [CELEBORN-1745] Remove application top disk usage code [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/26 06:24:22 UTC, 3 replies.
- Re: [PR] [CELEBORN-1747] Fix flaky test - HybridShuffleWordCountTest [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/26 06:32:19 UTC, 1 replies.
- Re: [PR] [CELEBORN-1744] Cleanup old ResourceConsumption metrics [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/26 07:04:58 UTC, 8 replies.
- [PR] [CELEBORN-1740][CIP-14] Add stackTrace utils to cppClient [celeborn] - posted by "HolyLow (via GitHub)" <gi...@apache.org> on 2024/11/26 11:56:23 UTC, 5 replies.
- [PR] Deprecate identity provider configs tied with quota [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2024/11/26 12:45:34 UTC, 0 replies.
- [PR] [CELEBORN-1621][FOLLOWUP] Support enabling worker tags via config [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2024/11/26 17:32:50 UTC, 3 replies.
- [PR] Fix metrics wrong [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/26 18:39:37 UTC, 0 replies.
- Re: [PR] [CELEBORN-1749] Fix incorrect application diskBytesWritten metrics [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/26 23:24:59 UTC, 1 replies.
- [PR] Return struct worker resource consumption information [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/27 00:23:26 UTC, 0 replies.
- [PR] [CELEBORN-1743] fix metrics data interruption and job fail due to the bolck of the lock [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2024/11/27 04:20:57 UTC, 0 replies.
- Re: [PR] [CELEBORN-1743] Resolve the metrics data interruption and the job failure caused by locked resources. [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/27 06:26:54 UTC, 8 replies.
- [PR] threadlocal [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/27 06:57:02 UTC, 1 replies.
- [PR] [CELEBORN-1751][CIP-14] Add celebornException utils to cppClient [celeborn] - posted by "HolyLow (via GitHub)" <gi...@apache.org> on 2024/11/27 12:30:41 UTC, 2 replies.
- Re: [PR] [CELEBORN-1725] Optimize performance of handling `MapperEnd` RPC in `LifecycleManager` [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/27 18:43:00 UTC, 2 replies.
- [PR] [CELEBORN-1725][FOLLOWUP] Optimize `isAllMapTasksEnd` performance [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/27 18:49:26 UTC, 11 replies.
- [PR] Shuffles api [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/28 03:29:17 UTC, 0 replies.
- Re: [PR] [CELEBORN-1752] Migration guide for shuffle RESTful api change since 0.5.2 [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/28 03:42:55 UTC, 0 replies.
- [PR] [CELEBORN-1530][FOLLOWUP] Exclude web modules by default [celeborn] - posted by "FMX (via GitHub)" <gi...@apache.org> on 2024/11/28 07:28:45 UTC, 1 replies.
- Re: [PR] [CELEBORN-1752] Migration guide for unexpected shuffle RESTful api change since 0.5.0 [celeborn] - posted by "SteNicholas (via GitHub)" <gi...@apache.org> on 2024/11/28 08:17:13 UTC, 1 replies.
- [PR] [CELEBORN-1753] Optimize the code for `exists` and `find` method [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/28 08:44:37 UTC, 0 replies.
- [PR] [CELEBORN-1755] Update doc to include S3 as one of storage layers [celeborn] - posted by "zhaohehuhu (via GitHub)" <gi...@apache.org> on 2024/11/28 08:47:05 UTC, 3 replies.
- [PR] [CELEBORN-1743] [FOLLOWUP] Make the application metrics queue later when the capacity is full without sorting [celeborn] - posted by "zaynt4606 (via GitHub)" <gi...@apache.org> on 2024/11/29 03:41:45 UTC, 0 replies.
- [PR] [CELEBORN-1756] Do not gauge hdfs metrics if hdfs storage not enabled [celeborn] - posted by "turboFei (via GitHub)" <gi...@apache.org> on 2024/11/29 06:17:18 UTC, 0 replies.
- Re: [PR] [CELEBORN-1748] Deprecate identity provider configs tied with quota [celeborn] - posted by "s0nskar (via GitHub)" <gi...@apache.org> on 2024/11/29 06:48:45 UTC, 0 replies.
- Re: [PR] [CELEBORN-1319] Optimize skew partition logic for Reduce Mode to avoid sorting shuffle files [celeborn] - posted by "wangshengjie123 (via GitHub)" <gi...@apache.org> on 2024/11/30 07:55:31 UTC, 0 replies.