You are viewing a plain text version of this content. The canonical link for it is here.
- Re: [PR] [MINOR] fix: Get and increment ATOMIC_LONG in that order everywhere [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/01 01:25:49 UTC, 0 replies.
- Re: [PR] [MINOR] fix: Identify duplicate blocks in TestUtils.validateResult [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/01 01:31:10 UTC, 0 replies.
- [I] [Improvement] flushing buffer if the memoryUsed of the first record of WriterBuffer larger than bufferSize [incubator-uniffle] - posted by "xianjingfeng (via GitHub)" <gi...@apache.org> on 2024/02/01 01:53:09 UTC, 1 replies.
- Re: [PR] [#1497] improvement(client): flushing buffer if the memoryUsed of the first record of `WriterBuffer` larger than bufferSize [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/01 06:16:52 UTC, 0 replies.
- [PR] [MINOR] build: Move dashboard module into profile [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/01 11:44:58 UTC, 7 replies.
- [I] [DOCS] What does a decomission actually do [incubator-uniffle] - posted by "connorlwilkes (via GitHub)" <gi...@apache.org> on 2024/02/01 11:58:50 UTC, 2 replies.
- [PR] [#1267][FOLLOWUP] improvement(client): INFO log level should be used in RetryUtils [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/01 14:58:45 UTC, 9 replies.
- Re: [I] [FEATURE] Provide dedicated unregister app rpc interface [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/02 02:22:14 UTC, 4 replies.
- [I] [Bug] Storage selection cache accidentally deleted when clearing stage level data [incubator-uniffle] - posted by "dingshun3016 (via GitHub)" <gi...@apache.org> on 2024/02/04 02:53:03 UTC, 0 replies.
- Re: [I] [FEATURE][Spark] Support partial sort and combine for reducing shuffle data size [incubator-uniffle] - posted by "KnightChess (via GitHub)" <gi...@apache.org> on 2024/02/04 04:02:23 UTC, 10 replies.
- [PR] fix(server): storage selection cache accidentally deleted when cleari… [incubator-uniffle] - posted by "dingshun3016 (via GitHub)" <gi...@apache.org> on 2024/02/04 06:03:32 UTC, 0 replies.
- [PR] [#1407] fix(rust): avoid checking storage type in runtime [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/04 07:15:42 UTC, 2 replies.
- [PR] [#1407] fix(rust): dont panic when no available local disks [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/04 07:17:26 UTC, 2 replies.
- Re: [PR] (#1501)fix(server): storage selection cache accidentally deleted when clearing stage level data. [incubator-uniffle] - posted by "codecov-commenter (via GitHub)" <gi...@apache.org> on 2024/02/04 07:27:24 UTC, 0 replies.
- Re: [PR] [#1501] fix(server): storage selection cache accidentally deleted when clearing stage level data. [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/04 07:34:12 UTC, 7 replies.
- Re: [PR] [#1373][part-2] partition write to multi server by reassign a faulty server [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/04 08:24:06 UTC, 10 replies.
- [PR] [MINOR] CI: Report test results [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/04 14:24:35 UTC, 3 replies.
- [PR] [#1407] fix(rust): return error when getting data from hdfs by client [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/05 06:21:16 UTC, 2 replies.
- Re: [I] [FEATURE] Print out the task shuffle write time statistics by table format in log [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/05 06:43:48 UTC, 0 replies.
- Re: [I] fix(server): storage selection cache accidentally deleted when clearing stage level data. [incubator-uniffle] - posted by "dingshun3016 (via GitHub)" <gi...@apache.org> on 2024/02/05 12:02:53 UTC, 0 replies.
- [PR] [MINOR] CI: Rework build and rust workflow events [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/05 17:31:51 UTC, 2 replies.
- Re: [PR] [FEATURE] Support Flink RSS. [incubator-uniffle] - posted by "slfan1989 (via GitHub)" <gi...@apache.org> on 2024/02/05 23:14:10 UTC, 1 replies.
- [PR] [#1407] fix(rust): drop events and release memory when errors happened [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/06 02:42:01 UTC, 3 replies.
- [PR] [FEATURE] Provide dedicated unregister app rpc interface [incubator-uniffle] - posted by "somelovelanguage (via GitHub)" <gi...@apache.org> on 2024/02/06 06:35:36 UTC, 2 replies.
- Re: [PR] [#1476] feat(spark): Provide dedicated unregister app rpc interface [incubator-uniffle] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/02/06 08:34:45 UTC, 10 replies.
- Re: [I] [Umbrella] Object Storage Support (Help Wanted) [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/07 02:01:03 UTC, 0 replies.
- [PR] [#1476] feat(rust): Provide dedicated unregister app rpc for rust based server [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/08 01:43:46 UTC, 5 replies.
- Re: [I] [Improvement] Merge data file and index file [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/08 03:23:17 UTC, 0 replies.
- Re: [I] [Improvement] `rss.client.assignment.shuffle.nodes.max` should not be less than `rss.data.replica` [incubator-uniffle] - posted by "somelovelanguage (via GitHub)" <gi...@apache.org> on 2024/02/08 03:56:56 UTC, 0 replies.
- Re: [PR] [#1373][part-1] feat: partition write to multi servers leveraging from reassignment mechanism [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/02/08 11:22:16 UTC, 10 replies.
- [I] [Improvement] Replace taskAttemptId in blockId with mapIndex [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/08 14:19:32 UTC, 4 replies.
- [PR] [MINOR] Fix invocation of kubernetes bash scripts [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/08 16:06:52 UTC, 1 replies.
- [PR] [#1512] feature: Replace taskAttemptId with mapIndex in blockId [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/08 18:02:49 UTC, 0 replies.
- Re: [PR] [#1512] feature(Spark): Replace taskAttemptId with mapIndex in blockId [incubator-uniffle] - posted by "codecov-commenter (via GitHub)" <gi...@apache.org> on 2024/02/08 20:33:26 UTC, 19 replies.
- Re: [PR] [MINOR] fix(deploy): Fix invocation of kubernetes bash scripts [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/09 01:18:19 UTC, 0 replies.
- [PR] [#1407] chore(rust): add cpu profile doc [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/09 01:51:13 UTC, 3 replies.
- [PR] [#1407] feat(rust): fix + add total grpc request metrics [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/09 01:53:20 UTC, 3 replies.
- [PR] [#1407] fix(rust): use grpc runtime worker threads and adjust default runtime config [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/09 01:57:16 UTC, 3 replies.
- Re: [PR] [#1222] feature(rust): Create mini rust server to integrate spark tests [incubator-uniffle] - posted by "codecov-commenter (via GitHub)" <gi...@apache.org> on 2024/02/09 04:35:12 UTC, 12 replies.
- [I] [Bug] RssMRUtils.convertTaskAttemptIdToLong produces too large taskAttemptIds [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/09 08:28:11 UTC, 0 replies.
- [PR] [#1472] fix(server): Inaccurate flow control leads to Shuffle server OOM when enabling Netty [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/10 20:37:47 UTC, 9 replies.
- Re: [I] [Bug] client memory leak with netty #1355 [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/11 15:51:01 UTC, 0 replies.
- [PR] [#1472][part-1] fix(server): Upgrade Netty and GRPC [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/12 07:26:42 UTC, 3 replies.
- [PR] [#1472][part-2] fix(server): Reuse ByteBuf when decoding shuffle blocks instead of reallocating it [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/12 07:30:44 UTC, 18 replies.
- [PR] [#1472][part-3] fix(client): Fix occasional IllegalReferenceCountException issues in extremely rare scenarios [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/12 07:37:07 UTC, 12 replies.
- [PR] [MINOR] fix(pom): Add missing shuffle-server dependencies to work with -Ptez [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/12 08:34:07 UTC, 3 replies.
- [PR] [#1472][part-4] feature(server): Introduce metrics for Netty's pinnedDirectMemory and usedDirectMemory [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/13 05:20:05 UTC, 12 replies.
- Re: [I] [Bug] Off-heap memory grows [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/13 06:07:51 UTC, 0 replies.
- [PR] [MINOR] feat(build): Allow to build distribution without some modules [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/14 09:30:56 UTC, 1 replies.
- [PR] [MINOR] fix(CI): Improve dashboard across the CI [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/14 09:44:23 UTC, 5 replies.
- [PR] [MINOR] refactor(common): Move blockId bit logic into common clas [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/14 12:45:42 UTC, 0 replies.
- Re: [PR] [MINOR] refactor(common): Move blockId bit logic into common class [incubator-uniffle] - posted by "codecov-commenter (via GitHub)" <gi...@apache.org> on 2024/02/14 12:56:19 UTC, 4 replies.
- [PR] [#731] feat(conf): Make blockid layout configurable [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/14 15:18:26 UTC, 5 replies.
- Re: [I] [FEATURE] Adaptive length of different parts of blockId [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/14 17:12:14 UTC, 1 replies.
- [PR] [#134] improvement(spark3): Use taskId and attemptNo as taskAttemptId [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/15 11:00:26 UTC, 24 replies.
- [PR] [MINOR} Update dashboard pom.xml to take arguments for node and npm download locations [incubator-uniffle] - posted by "connorlwilkes (via GitHub)" <gi...@apache.org> on 2024/02/15 12:52:19 UTC, 0 replies.
- Re: [PR] [MINOR] Update dashboard pom.xml to take arguments for node and npm download locations [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/15 14:21:52 UTC, 14 replies.
- [PR] [#1472][part-5] Inaccurate flow control leads to Shuffle server OOM when enabling Netty [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/15 21:02:34 UTC, 20 replies.
- Re: [PR] [#1316] improvement: detect OutputTracker API version via Spark version [incubator-uniffle] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/02/16 20:38:09 UTC, 2 replies.
- Re: [I] [Improvement] Support more tasks of the application [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/20 08:19:53 UTC, 0 replies.
- Re: [I] [Improvement] Detect OutputTracker API version via Spark version [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/20 08:23:36 UTC, 0 replies.
- [PR] [MINOR] example: Add example docker compose Uniffle/Spark cluster [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/20 10:02:13 UTC, 13 replies.
- Re: [I] [FEATURE] support shuffle partition data write to multiple shuffle servers [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/21 02:15:42 UTC, 0 replies.
- [I] [FEATURE] support shuffle partition data read from multiple shuffle servers [incubator-uniffle] - posted by "dingshun3016 (via GitHub)" <gi...@apache.org> on 2024/02/21 06:31:53 UTC, 0 replies.
- [PR] [#1472][part-5] Use UnpooledByteBufAllocator to obtain accurate ByteBuf sizes to fix inaccurate usedMemory issue causing OOM [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/21 06:52:35 UTC, 32 replies.
- Re: [I] [FEATURE] introduce dedicated healthy metrics for every local disk [incubator-uniffle] - posted by "SemyonSinchenko (via GitHub)" <gi...@apache.org> on 2024/02/21 21:39:10 UTC, 3 replies.
- [I] [Improvement] Improve write throughput for rust [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/22 06:03:52 UTC, 0 replies.
- Re: [PR] [#1355] fix(client): Netty client will leak when decoding responses [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/22 19:33:20 UTC, 0 replies.
- [PR] [#1472][FOLLOWUP] fix(client): Fix IllegalReferenceCountException issues when exceptions happened in clientReadHandler.readShuffleData() [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/22 19:39:59 UTC, 7 replies.
- Re: [I] [Bug] IllegalReferenceCountException in Shuffle Server [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/23 07:11:28 UTC, 1 replies.
- [PR] [#1459][FOLLOWUP] fix(server): Memory may leak in exceptional scenarios when flushing events [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/23 07:30:36 UTC, 16 replies.
- [I] [FEATURE] Optional blockIds report from remote shuffle-server to individual spark driver [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/24 08:48:42 UTC, 1 replies.
- [I] [FEATURE] Quick graceful upgrading for shuffle-server cluster [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/24 08:54:12 UTC, 2 replies.
- [PR] [#1472][part-6] fix(netty): Make UTs truly test Netty mode [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/24 13:59:21 UTC, 10 replies.
- [PR] [MINOR] improvement(tez): Only invoking LOG.debug when LOG.isDebugEnabled is true [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/24 16:16:10 UTC, 3 replies.
- [I] [Bug] Unexpected exceptions happened when flushing events [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/24 16:29:21 UTC, 0 replies.
- [I] [Improvement] [SPARK]Optimize shuffle reading when both sort and combine are used. [incubator-uniffle] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2024/02/26 01:24:26 UTC, 1 replies.
- Re: [I] [Serious Bug] Inaccurate flow control leads to Shuffle server OOM when enabling Netty [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/26 06:47:59 UTC, 0 replies.
- Re: [I] [Improvement] [SPARK] Modify task attempt id to avoid information redundancy. [incubator-uniffle] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2024/02/27 06:37:05 UTC, 1 replies.
- Re: [PR] [#1398] fix(MR)(TEZ): Limit attemptId to 4 bit and move it from 18 bit atomicInt to 21 bit taskAttemptId in 63 bit BlockId. [incubator-uniffle] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2024/02/27 06:43:46 UTC, 2 replies.
- [PR] [#134][FOLLOWUP] improvement(spark2): Use taskId and attemptNo as taskAttemptId [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/27 07:19:37 UTC, 3 replies.
- [PR] [MINOR] test: Use sensible partition ids in ShuffleReadClientImplTest [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/27 08:50:37 UTC, 3 replies.
- [I] [Bug] NPE cound happen before uncompressing after #1360 [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/27 09:47:34 UTC, 0 replies.
- [PR] [#1546] fix(spark): NPE cound happen before uncompressing after #1360 [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/27 11:13:41 UTC, 6 replies.
- Re: [PR] [#1546] fix(spark): NPE could happen before uncompressing after #1360 [incubator-uniffle] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/02/27 12:11:58 UTC, 1 replies.
- [I] [Bug] Duplicated shaded-grpc jar cause task fail [incubator-uniffle] - posted by "lifeSo (via GitHub)" <gi...@apache.org> on 2024/02/28 05:45:17 UTC, 1 replies.
- [I] [Bug] Uniformly throw RssException for external callers [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/28 09:22:55 UTC, 1 replies.
- [PR] [#1549] fix(common): Uniformly throw RssException for external callers [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/28 09:27:35 UTC, 4 replies.
- Re: [I] [Bug] NPE could happen before uncompressing after #1360 [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/02/28 11:11:43 UTC, 0 replies.
- [PR] [#1472][part-5][FOLLOWUP] Fix Netty transport time when sending shuffle data requests [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/28 15:32:19 UTC, 2 replies.
- Re: [PR] [#1472][part-6][FOLLOWUP] Fix Netty transport time when sending shuffle data requests [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/02/29 03:49:26 UTC, 0 replies.
- [PR] [#731] feat(spark3): Make blockid layout configurable for Spark3 [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/02/29 07:39:11 UTC, 1 replies.
- [I] [Improvement] Migrate from log4j1 to log4j2 [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/29 15:46:32 UTC, 0 replies.
- [PR] [#1552] improvement: Migrate from log4j1 to log4j2 [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/02/29 16:02:07 UTC, 7 replies.