You are viewing a plain text version of this content. The canonical link for it is here.
- Re: [PR] [#1552] improvement: Migrate from log4j1 to log4j2 [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/01 02:34:46 UTC, 20 replies.
- [I] [FEATURE] Support dynamic client conf for creating ShuffleWriteClient [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/03/01 12:06:35 UTC, 1 replies.
- Re: [PR] [#731] feat(Spark): Make blockid layout configurable for Spark clients [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/03/01 12:33:42 UTC, 10 replies.
- Re: [PR] [#859][Improvement] Set MALLOC_ARENA_MAX in start-shuffle-server.sh [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/01 19:16:45 UTC, 0 replies.
- [I] [Bug] The number of output rows in SortMergeJoin has increased after enabling RSS [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/04 03:57:24 UTC, 6 replies.
- Re: [PR] [#731] feat(spark): Make blockid layout configurable for Spark clients [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/03/04 09:21:49 UTC, 16 replies.
- [PR] build(deps): bump mio from 0.8.8 to 0.8.11 in /rust/experimental/server [incubator-uniffle] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2024/03/04 21:58:33 UTC, 4 replies.
- [PR] [#1554] feat(spark): Fetch dynamic client conf as early as possible [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/03/05 12:19:08 UTC, 23 replies.
- Re: [I] [Improvement] Migrate from log4j1 to log4j2 [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/06 03:24:44 UTC, 0 replies.
- Re: [I] [Bug] RssMRUtils.convertTaskAttemptIdToLong produces too large taskAttemptIds [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/06 03:26:06 UTC, 1 replies.
- Re: [I] [DOCS] What does a decomission actually do [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/06 03:26:50 UTC, 1 replies.
- Re: [I] [FEATURE] support shuffle partition data read from multiple shuffle servers [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/06 03:27:25 UTC, 0 replies.
- [PR] [#808][FOLLOWUP] improvement(spark): Verify the number of written records to enhance data accuracy [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/06 05:19:35 UTC, 10 replies.
- [PR] [MINOR] improvement(client): Override getClientInfo method in ShuffleServerGrpcNettyClient and remove unused getDesc method [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/06 06:39:47 UTC, 3 replies.
- [I] [Bug] A concurrency issue occurred when flushing events and removing resources at the same time [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/06 07:38:01 UTC, 3 replies.
- [I] [FEATURE] Support Spark 2.1 [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/06 08:10:47 UTC, 4 replies.
- [PR] [#1561] feat(spark): Support Spark 2.1 [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/06 08:13:18 UTC, 4 replies.
- [PR] [MINOR] improvement(test): A better computation logic for WriteAndReadMetricsTest without using reflection [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/06 13:23:27 UTC, 2 replies.
- Re: [I] [Bug] Unexpected exceptions happened when flushing events [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/06 14:18:13 UTC, 2 replies.
- Re: [PR] [#808] improvement(spark): Verify the number of written records to enhance data correctness [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/07 07:13:21 UTC, 1 replies.
- Re: [I] [FEATURE] Adaptive length of different parts of blockId [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/07 07:21:05 UTC, 0 replies.
- [I] [Bug] LocalStorageChecker hang when encounter disk is corrupted [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/07 07:33:14 UTC, 0 replies.
- [PR] [#133][FOLLOWUP] fix(netty): Let Spark use its own NettyUtils [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/07 09:51:19 UTC, 2 replies.
- [PR] [#731][FOLLOWUP] feat(Spark): Configure blockIdLayout for Spark based on max partitions [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/03/07 12:34:34 UTC, 4 replies.
- Re: [PR] [#133][FOLLOWUP] fix(netty): Let Spark2 use its own NettyUtils [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/07 15:56:19 UTC, 0 replies.
- [I] [Bug] NoSuchMethodError will be encountered when using Spark 2.x [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/07 16:19:32 UTC, 1 replies.
- Re: [PR] [#133][FOLLOWUP] fix(spark): Let Spark2 use its own NettyUtils [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/07 16:36:20 UTC, 4 replies.
- [PR] [#1564] fix(server): avoid hang when checking local disks [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/08 02:32:43 UTC, 0 replies.
- Re: [PR] [#1564] fix(server): disk health check invalid when hang [incubator-uniffle] - posted by "codecov-commenter (via GitHub)" <gi...@apache.org> on 2024/03/08 02:40:45 UTC, 10 replies.
- [I] [Flaky Test] [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/08 03:18:00 UTC, 0 replies.
- [PR] [#1569] fix(rust): flaky test for `test_ticket_manager` [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/08 03:20:57 UTC, 4 replies.
- Re: [I] [Flaky Test] test_ticket_manager for rust [incubator-uniffle] - posted by "smallzhongfeng (via GitHub)" <gi...@apache.org> on 2024/03/08 09:34:05 UTC, 0 replies.
- Re: [I] [Serious Bug] The number of output rows in SortMergeJoin has increased after enabling RSS [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/03/09 17:34:37 UTC, 7 replies.
- Re: [PR] [#1567] fix(spark): Let Spark use its own NettyUtils [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/03/11 01:53:28 UTC, 0 replies.
- [I] [Bug] Memory may leak when EventInvalidException occurs [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/12 12:38:45 UTC, 3 replies.
- Re: [I] [Bug] LocalStorageChecker hang when disk is corrupted [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/13 03:24:37 UTC, 0 replies.
- [I] [Bug] Exceptions might be discarded when spilling buffers [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/13 03:56:18 UTC, 1 replies.
- [PR] [#1572] fix(spark): Exceptions might be discarded when spilling buffers [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/13 04:00:10 UTC, 3 replies.
- [PR] [#1571] fix(server): Memory may leak when EventInvalidException occurs [incubator-uniffle] - posted by "zhangxiong0301 (via GitHub)" <gi...@apache.org> on 2024/03/13 09:33:14 UTC, 24 replies.
- [PR] build(deps): bump google.golang.org/protobuf from 1.28.0 to 1.33.0 in /deploy/kubernetes/operator [incubator-uniffle] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2024/03/13 21:50:05 UTC, 3 replies.
- Re: [I] [Bug] Duplicated shaded-grpc jar cause task fail [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/14 02:02:05 UTC, 1 replies.
- [I] [FEATURE] Provide the default no-hadoop-home deployment guide [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/14 07:50:27 UTC, 1 replies.
- [PR] [#1576] feat(doc): server deploy guide without no-hadoop-home env [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/14 08:11:58 UTC, 2 replies.
- [PR] fix(spark): add all faulty servers into blacklist while writing for stage recomputing [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/14 09:17:21 UTC, 6 replies.
- [I] [Bug] previous data is not deleted when re-register shuffle after re-assignment for stage recomputing [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/14 09:57:22 UTC, 0 replies.
- [PR] [#1373][followup] fix(client):Add client type when request shuffle assignment [incubator-uniffle] - posted by "dingshun3016 (via GitHub)" <gi...@apache.org> on 2024/03/14 10:20:19 UTC, 5 replies.
- [PR] [#1373][followup] fix(client):Fix task fail retry parameter not work [incubator-uniffle] - posted by "dingshun3016 (via GitHub)" <gi...@apache.org> on 2024/03/14 10:36:53 UTC, 12 replies.
- Re: [PR] [#1576] feat(doc): server deploy guide without hadoop-home env [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/03/14 10:40:38 UTC, 6 replies.
- [PR] [#1373][followup] fix(client):Fix partition id type is incorrect [incubator-uniffle] - posted by "dingshun3016 (via GitHub)" <gi...@apache.org> on 2024/03/14 13:12:12 UTC, 5 replies.
- [PR] [#1579] fix(spark): clear out previous stage attempt data synchronously [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/15 03:14:05 UTC, 4 replies.
- [I] [FEATURE] Add metrics about block size distribution, either overall or on a per-application basis. [incubator-uniffle] - posted by "leslizhang (via GitHub)" <gi...@apache.org> on 2024/03/15 12:42:02 UTC, 1 replies.
- [I] [Improvement] Allow Netty Worker thread pool size to dynamically adapt to the number of processor cores [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/18 08:56:46 UTC, 1 replies.
- [PR] [#1586] improvement(netty): Allow Netty Worker thread pool size to dynamically adapt to the number of processor cores [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/18 09:08:23 UTC, 8 replies.
- [I] [Bug] The exception might be swallowed when there is a bug in the code while handling the FlushEvent [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/18 09:30:42 UTC, 1 replies.
- [PR] [#1588] improvement(server): Add exception handling for the thread pool when flushing events [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/18 09:37:47 UTC, 2 replies.
- [PR] [MINOR] refactor: Calling lock() method outside try block to avoid unnecessary errors [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/18 10:10:35 UTC, 5 replies.
- [I] [FEATURE] Support Spark 3.5.1 [incubator-uniffle] - posted by "qijiale76 (via GitHub)" <gi...@apache.org> on 2024/03/19 06:54:31 UTC, 2 replies.
- [PR] [#1591] feat(spark): Support Spark 3.5.1 [incubator-uniffle] - posted by "qijiale76 (via GitHub)" <gi...@apache.org> on 2024/03/19 07:06:37 UTC, 3 replies.
- [PR] Issue 1585 [incubator-uniffle] - posted by "leslizhang (via GitHub)" <gi...@apache.org> on 2024/03/19 09:48:09 UTC, 3 replies.
- [I] [FEATURE] support generating larger block size during shuffle map task by spill partial partitions data [incubator-uniffle] - posted by "leslizhang (via GitHub)" <gi...@apache.org> on 2024/03/20 14:13:29 UTC, 2 replies.
- Re: [PR] [#1585] feat(server): Support app-level block size statistics to report metrics [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/03/21 03:29:16 UTC, 5 replies.
- Re: [PR] [#1398] fix(MR)(TEZ): Limit attemptId to 4 bit and move it from 18 bit atomicInt to 21 bit taskAttemptId in 63 bit BlockId. [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/03/21 08:50:45 UTC, 1 replies.
- [PR] [#1571][FOLLOWUP] improvement(server): Fail fast when the shuffleId is cleaned up [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/21 09:30:25 UTC, 12 replies.
- [I] [Bug] Reading local shuffle data in high-pressure scenarios may lead to high system load [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/21 12:32:33 UTC, 4 replies.
- [PR] [#1472][FOLLOWUP] improvement(server): Release memory more accurately when failing to cache shuffle data [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/21 14:02:38 UTC, 2 replies.
- Re: [PR] [#1579] fix(spark): Adjust reassgin time to avoid failure to clean up previous stage data [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/03/22 10:20:38 UTC, 8 replies.
- [I] [Bug] The used direct memory metric is inaccurate [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/23 12:10:41 UTC, 1 replies.
- [PR] [#1598] fix(server) Fix inaccurate used_direct_memory_size metric [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/23 12:21:37 UTC, 6 replies.
- [I] [FEATURE] Support for allocating direct memory when reading local files and indexes [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/24 09:25:18 UTC, 1 replies.
- [PR] [#1600] feat(netty): Support for allocating direct memory when reading local files and indexes [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/24 09:37:41 UTC, 2 replies.
- Re: [PR] [#1600] feat(netty): Support for allocating direct memory when reading local data files and index files [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/03/25 02:02:09 UTC, 13 replies.
- Re: [PR] [#1579] fix(spark): Adjust reassigned time to ensure that all previous data is cleared for stage retry [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/26 03:25:39 UTC, 0 replies.
- [I] [Improvement] Seprate code for the different roles in spark rssShuffleManager [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/26 03:40:28 UTC, 0 replies.
- Re: [PR] [#1579][part-1] fix(spark): Adjust reassigned time to ensure that all previous data is cleared for stage retry [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/26 03:55:15 UTC, 18 replies.
- [I] [Improvement] Disable dataPusher initialization for spark driver [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/26 05:51:29 UTC, 0 replies.
- [PR] [#1579][part-2] feat(spark): allow to register shuffle in parallel [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/26 06:43:47 UTC, 6 replies.
- [PR] [#1596] fix(netty): Use a ChannelFutureListener callback mechanism to release readMemory [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/26 17:16:32 UTC, 29 replies.
- [I] [Bug] Add retry mechanism for reading data(memory/local/index) [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/27 11:42:35 UTC, 0 replies.
- [I] [DOCS] Performance Benchmark Report using TPC-DS [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/27 11:51:03 UTC, 0 replies.
- Re: [I] [Bug] Add retry mechanism for reading data(memory/local/index) when there is no buffer [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/03/27 13:40:16 UTC, 1 replies.
- Re: [PR] [#1398] fix(mr,tez): Make attempId computable and move it to taskAttemptId in BlockId layout. [incubator-uniffle] - posted by "qijiale76 (via GitHub)" <gi...@apache.org> on 2024/03/28 11:44:46 UTC, 1 replies.
- [I] [Improvement] One partition data could be written to multiple servers [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/29 03:04:11 UTC, 1 replies.
- [PR] [#1608] fix(spark): re-assign only once for the same faulty server in one stage [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/29 05:49:29 UTC, 1 replies.
- Re: [PR] [#1608] fix(spark): Only share the replacement servers for faulty servers in one stage [incubator-uniffle] - posted by "codecov-commenter (via GitHub)" <gi...@apache.org> on 2024/03/29 06:08:18 UTC, 4 replies.
- Re: [PR] [#1373][part-1] add PartitionServerInfo for ShuffleHandleInfo to split partition server [incubator-uniffle] - posted by "xumanbu (via GitHub)" <gi...@apache.org> on 2024/03/29 07:31:19 UTC, 0 replies.
- [PR] [#1608][part-2] fix(spark): avoid releasing block previously when enable block resend [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/29 09:39:58 UTC, 3 replies.
- Re: [PR] [#1608][part-1] fix(spark): Only share the replacement servers for faulty servers in one stage [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/03/29 10:36:03 UTC, 0 replies.
- Re: [PR] [#1608][part-2] fix(spark): avoid releasing block in advance when enable block resend [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/03/29 11:51:30 UTC, 1 replies.
- Re: [I] [Improvement] Optimize partition data reassignment [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/03/30 00:33:53 UTC, 0 replies.