You are viewing a plain text version of this content. The canonical link for it is here.
- Re: [PR] [#1608][part-2] fix(spark): avoid releasing block in advance when enable block resend [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/04/01 01:56:29 UTC, 56 replies.
- Re: [PR] [#1608][part-1] fix(spark): Only share the replacement servers for faulty servers in one stage [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/01 03:58:05 UTC, 4 replies.
- [PR] [#1606] improvement(client): Add retry mechanism for reading data(memory/local/index) when there is no buffer [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/01 05:36:52 UTC, 8 replies.
- Re: [PR] [#1373][FOLLOWUP] fix(spark): shuffle manager rpc service invalid when partition data reassign is enabled [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/01 07:06:07 UTC, 4 replies.
- Re: [PR] [#1606] feat(client): Add retry mechanism for reading data(memory/local/index) for NO_BUFFER [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/01 09:31:10 UTC, 5 replies.
- [PR] [#1373][FOLLOWUP] fix(spark): register shuffle server partition range error when reassign faulty shuffle server for tasks [incubator-uniffle] - posted by "dingshun3016 (via GitHub)" <gi...@apache.org> on 2024/04/01 14:33:56 UTC, 5 replies.
- [I] [Improvement] Use FileChannel to replace FileOutputStream and DataOutputStream in LocalFileWriter [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/01 16:53:45 UTC, 1 replies.
- Re: [PR] [#1606] feat(client): Add retry mechanism for NO_BUFFER when reading data(memory/local/index) [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/04/02 01:47:36 UTC, 38 replies.
- Re: [I] [Feature] Partition data reassignment [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/02 02:07:09 UTC, 1 replies.
- [I] [Improvement] When committing to a shuffle server, print its IP and port [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/02 03:40:12 UTC, 0 replies.
- [PR] [#1608][part-3] feat(spark3): support reading from partition block data reassignment servers [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/02 06:39:26 UTC, 2 replies.
- [PR] [#1608][part-4] feat(server)(spark3): best effort to resend when server is inactive [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/03 06:37:51 UTC, 1 replies.
- Re: [PR] [#1608][part-4] feat(server)(spark3): activate block resend when server is inactive [incubator-uniffle] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/04/03 07:14:07 UTC, 5 replies.
- [I] [FEATURE] Show Thread Dump in DashBoard [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/03 14:14:50 UTC, 0 replies.
- Re: [PR] [#1401][part-1] The dashboard supports multiple Coordinator links. [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/03 14:26:25 UTC, 13 replies.
- [I] [FEATURE] Set up multiple Netty Servers to handle different requests for Netty Handlers separatel [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/03 14:34:12 UTC, 0 replies.
- [I] [Bug] Under high pressure situations, reading data may encounter the issue of "Blocks read inconsistent" [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/03 14:45:21 UTC, 3 replies.
- [I] [Improvement] Refactor BlockId semantics [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/04/04 08:19:52 UTC, 0 replies.
- [I] [Improvement] Remove useless ID column in Dashboard [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/04 16:45:06 UTC, 0 replies.
- [I] [FEATURE] Show logs for Coordinators and Shuffle Servers in Dashboard [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/04 16:47:48 UTC, 0 replies.
- [I] [FEATURE] Show IO/CPU/Disk usage in Dashboard [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/05 04:09:11 UTC, 0 replies.
- [PR] build(deps): bump h2 from 0.3.21 to 0.3.26 in /rust/experimental/server [incubator-uniffle] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2024/04/05 16:07:38 UTC, 2 replies.
- Re: [PR] [#1608][part-3] feat(spark3): support reading from multiple reassignment servers [incubator-uniffle] - posted by "xumanbu (via GitHub)" <gi...@apache.org> on 2024/04/08 03:31:00 UTC, 2 replies.
- Re: [PR] [#1373][FOLLOWUP] fix(spark): fix partition id inconsistency when reassign faulty shuffle server for tasks [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/08 05:50:30 UTC, 2 replies.
- Re: [I] [Improvement] Add retry mechanism for reading data(memory/local/index) when there is no buffer [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/08 05:54:53 UTC, 0 replies.
- [I] [Bug] The StorageManager cache might not function effectively under heavy IO pressure [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/08 08:54:10 UTC, 1 replies.
- Re: [PR] [#1373][FOLLOWUP] fix(spark): register with incorrect partitionRanges after reassign [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/08 09:24:12 UTC, 2 replies.
- Re: [PR] [#1608][part-3] feat(spark3): support reading partition data from multiple reassignment servers [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/08 09:32:19 UTC, 24 replies.
- [PR] [#1626] fix(server): Refresh cache with new StorageManager on write if cache expires [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/08 12:33:07 UTC, 5 replies.
- [I] [Bug] Occasionally encountering IllegalReferenceCountException when releasing ShuffleIndexResult [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/08 13:13:54 UTC, 0 replies.
- [I] [Bug] [Operator] ShuffleSever cannot be deleted even though there are no more application. [incubator-uniffle] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2024/04/09 07:07:09 UTC, 6 replies.
- Re: [PR] [#1626] fix(server): Remove the meaningless eventOfUnderStorageManagers cache [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/09 07:27:58 UTC, 12 replies.
- [PR] [#1629] fix(operator): ShuffleSever cannot be deleted even though the… [incubator-uniffle] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2024/04/09 07:52:57 UTC, 0 replies.
- Re: [PR] [#1629] fix(operator): ShuffleSever cannot be deleted even though there are no more application. [incubator-uniffle] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/04/09 08:21:51 UTC, 0 replies.
- [I] [Bug] ShuffleTaskInfo may leak when app is removed. [incubator-uniffle] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2024/04/09 10:11:47 UTC, 2 replies.
- Re: [PR] [#1629] fix(operator): ShuffleServer cannot be deleted even though there are no more application. [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/04/09 10:30:12 UTC, 16 replies.
- [PR] [#1631] fix: ShuffleTaskInfo may leak when app is removed. [incubator-uniffle] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2024/04/09 10:31:53 UTC, 1 replies.
- Re: [PR] [#1631] fix(server): ShuffleTaskInfo may leak when app is removed [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/10 03:14:46 UTC, 0 replies.
- [I] [FEATURE] Determine whether data can be written and read based on the actual disk IO situation [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/10 03:48:53 UTC, 0 replies.
- [I] [Bug] app localdisk folder remains when app is expired [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/10 07:55:16 UTC, 1 replies.
- [PR] [#1634] fix(server): remove app folder if app is expired [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/10 08:12:00 UTC, 4 replies.
- [PR] fix:read inconsistent when local file read handler init before flushing event to disk [incubator-uniffle] - posted by "dingshun3016 (via GitHub)" <gi...@apache.org> on 2024/04/10 08:26:36 UTC, 12 replies.
- [PR] chore(rust): disable flaky test of test_ticket_manager [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/10 09:18:31 UTC, 2 replies.
- [PR] chore(rust): upgrade hdfs-native to 0.9.1 version to improve writing throughput [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/10 09:24:21 UTC, 0 replies.
- Re: [PR] chore(rust): upgrade hdfs-native to 0.9.1 to improve writing throughput [incubator-uniffle] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/04/10 10:02:20 UTC, 1 replies.
- [I] [Bug] Use thread pool to control the concurrency of data reading threads when enabling Netty [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/10 14:28:08 UTC, 0 replies.
- [PR] [#1543] improvement(spark): Optimize shuffle reading when both sort and combine are used. [incubator-uniffle] - posted by "qijiale76 (via GitHub)" <gi...@apache.org> on 2024/04/11 06:51:34 UTC, 7 replies.
- [PR] [#1596][FOLLOWUP] fix(netty): Send failed responses only when the channel is writable [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/11 12:38:12 UTC, 5 replies.
- [I] [Improvement] Optimize CompositeByteBuf initialization for better performance when getting memory data [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/11 13:05:44 UTC, 2 replies.
- [PR] [#1459][FOLLOWUP] improvement(server): Print an error log when an event is dropped [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/11 15:18:11 UTC, 2 replies.
- Re: [PR] [#1163] improvement(netty): return the direct byte buffer when getting localfile data by using netty [incubator-uniffle] - posted by "pegasas (via GitHub)" <gi...@apache.org> on 2024/04/12 02:43:47 UTC, 0 replies.
- Re: [PR] [#1020] feat(spark): Support hybrid shuffle manager [incubator-uniffle] - posted by "pegasas (via GitHub)" <gi...@apache.org> on 2024/04/12 02:43:47 UTC, 0 replies.
- Re: [PR] [Improvement] Refactor the writer code with builder pattern [incubator-uniffle] - posted by "pegasas (via GitHub)" <gi...@apache.org> on 2024/04/12 02:43:48 UTC, 0 replies.
- [I] [FEATURE] Add rpc queued time and rpc process time. [incubator-uniffle] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2024/04/15 02:48:27 UTC, 3 replies.
- [I] [FEATURE] Add metrics for reading data [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/15 03:17:37 UTC, 0 replies.
- [PR] [#1645] feat(server): Add gauge metrics for reading data [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/15 07:48:15 UTC, 3 replies.
- [PR] [MINOR] Remove meaningless string concatenation [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/15 10:02:20 UTC, 11 replies.
- [PR] [#1459][FOLLOWUP] fix(server): Fix the issue of log variable printing [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/15 10:07:16 UTC, 3 replies.
- Re: [PR] [#1645] feat(server): Add gauge metrics for reading localfile data [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/16 03:47:08 UTC, 1 replies.
- [PR] [#1087]feat(spark): Support dynamic allocation for Gluten Uniffle [incubator-uniffle] - posted by "summaryzb (via GitHub)" <gi...@apache.org> on 2024/04/16 07:59:40 UTC, 2 replies.
- [PR] [#1607] docs: Performance Benchmark Report using TPC-DS [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/16 08:52:48 UTC, 25 replies.
- [I] [Improvement] Set Netty as the default server type [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/16 09:46:26 UTC, 0 replies.
- Re: [I] [FEATURE] Add gauge metrics for reading data [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/16 09:48:43 UTC, 0 replies.
- Re: [PR] [#1087] feat(spark): Support dynamic allocation for Gluten Uniffle [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/16 11:03:55 UTC, 0 replies.
- Re: [I] [FEATURE] Support dynamic allocation for Gluten Uniffle [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/16 11:03:56 UTC, 0 replies.
- Re: [PR] [#1028] improvement(operator): Make rss-env.sh and HadoopConfig configurable via ConfigMap [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/16 12:15:17 UTC, 0 replies.
- Re: [PR] [#1149] fix: GC logs in JDK 11 do not include date and time stamps. [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/16 12:16:25 UTC, 0 replies.
- Re: [PR] [#1264] feat(spark): support cancel async rpc when kill or interrupt task [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/16 12:18:51 UTC, 0 replies.
- Re: [PR] [#1607] docs: Performance report with partial TPC-DS(SF=40000) queries [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/16 14:13:41 UTC, 9 replies.
- [PR] [#1608][part-5] feat(spark3): always use the latest assignment and load balance for huge partition [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/17 02:28:05 UTC, 87 replies.
- [PR] [#1651] improvement(netty): Set Netty as the default server type [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/17 05:15:41 UTC, 4 replies.
- Re: [I] [DOCS] Performance Benchmark Report using TPC-DS [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/17 06:13:21 UTC, 0 replies.
- [I] [Umbrella] Release 0.9.0 [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/04/17 08:11:12 UTC, 0 replies.
- [I] Bump `master` to `0.10.0-SNAPSHOT`. [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/04/17 08:18:20 UTC, 1 replies.
- [PR] [#1655] Bump version to 0.10.0-SNAPSHOT [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/04/17 08:30:38 UTC, 8 replies.
- [I] Add licences and notices regarding new dashboard module [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/04/17 08:34:33 UTC, 0 replies.
- [I] [Bug] Optimized FileSegmentManagedBuffer.nioByteBuffer to avoid multiple read file [incubator-uniffle] - posted by "xumanbu (via GitHub)" <gi...@apache.org> on 2024/04/18 07:34:39 UTC, 0 replies.
- [PR] [#1658] fix(server) FileSegmentManagedBuffer#nioByteBuffer read result cacheable [incubator-uniffle] - posted by "xumanbu (via GitHub)" <gi...@apache.org> on 2024/04/18 07:48:30 UTC, 2 replies.
- Re: [PR] [#1658] fix(server): FileSegmentManagedBuffer#nioByteBuffer read result cacheable [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/18 08:43:30 UTC, 10 replies.
- Re: [PR] [#1398] fix(mr,tez): Make attempId computable and move it to taskAttemptId in BlockId layout. [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/04/18 09:27:08 UTC, 0 replies.
- [PR] [#1239] Remote merge on the shuffle server side. [incubator-uniffle] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2024/04/18 11:07:39 UTC, 16 replies.
- [PR] [#1607][FOLLOWUP] docs(benchmark): Performance report with partial TPC-DS(SF=40000) queries [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/19 03:07:17 UTC, 1 replies.
- [I] [Bug] Fix flaky tests [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/19 03:24:07 UTC, 1 replies.
- [PR] [#1662] fix(test): Fix flaky tests and update docs [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/19 03:39:35 UTC, 5 replies.
- Re: [PR] [#1662] fix(test): Fix flaky tests [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/19 05:27:26 UTC, 7 replies.
- [I] [DOCS] Update the description and default values of some configurations [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/19 07:14:02 UTC, 0 replies.
- [PR] [#1664] docs: Update the descriptions and default values of some configurations [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/19 07:30:03 UTC, 1 replies.
- [PR] [#1341] fix(mr): Fix MR Combiner ArrayIndexOutOfBoundsException Bug. [incubator-uniffle] - posted by "qijiale76 (via GitHub)" <gi...@apache.org> on 2024/04/19 11:33:12 UTC, 13 replies.
- Re: [PR] [#1664] docs: Update the descriptions and default values of outdated configurations [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/04/19 12:23:36 UTC, 8 replies.
- [PR] build(deps): bump golang.org/x/net from 0.7.0 to 0.23.0 in /deploy/kubernetes/operator [incubator-uniffle] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2024/04/19 13:49:21 UTC, 1 replies.
- Re: [PR] build(deps): bump golang.org/x/net from 0.7.0 to 0.17.0 in /deploy/kubernetes/operator [incubator-uniffle] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2024/04/19 13:49:23 UTC, 1 replies.
- Re: [I] [DOCS] Add licences and notices regarding new dashboard module [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/04/22 15:21:54 UTC, 3 replies.
- Re: [PR] [#1662] fix(test): Fix Netty related tests [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/23 11:10:33 UTC, 1 replies.
- Re: [PR] [#1664] improvement(docs): Update the descriptions and default values of outdated conf [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/25 03:07:53 UTC, 0 replies.
- Re: [I] [DOCS] Update the descriptions and default values of outdated configurations [incubator-uniffle] - posted by "zuston (via GitHub)" <gi...@apache.org> on 2024/04/25 03:07:54 UTC, 0 replies.
- [I] [Bug] Assertions will not take effect during production runtime [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/25 13:59:44 UTC, 0 replies.
- [PR] [#378][FOLLOWUP] fix(server): Fix huge_partition_num metric [incubator-uniffle] - posted by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/25 16:34:06 UTC, 3 replies.
- Re: [PR] [#1585] feat(server): Support app-level block size statistics to report metrics [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/04/26 09:07:04 UTC, 7 replies.
- Re: [PR] [#1608][part-5] feat(spark3): always use the latest assignment [incubator-uniffle] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2024/04/26 09:45:07 UTC, 2 replies.
- [PR] [#1594] improvement(client):support generating larger block size during shuffle map task by spill partial partitions data [incubator-uniffle] - posted by "leslizhang (via GitHub)" <gi...@apache.org> on 2024/04/28 04:09:42 UTC, 5 replies.
- Re: [I] [FEATURE] support generating larger block size during shuffle map task by spill partial partitions data [incubator-uniffle] - posted by "leslizhang (via GitHub)" <gi...@apache.org> on 2024/04/28 07:53:17 UTC, 0 replies.
- Re: [PR] [#1608][part-5] feat(spark3): always use the available assignment [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/04/29 02:15:36 UTC, 28 replies.
- Re: [I] [Bug] java.lang.ArrayIndexOutOfBoundsException [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/04/30 03:10:14 UTC, 0 replies.
- [PR] [#1657] build: Add license information after version 0.9.0 [incubator-uniffle] - posted by "jerqi (via GitHub)" <gi...@apache.org> on 2024/04/30 11:15:00 UTC, 2 replies.