reviews@spark.apache.org, 2025-10

You are viewing a plain text version of this content. The canonical link for it is here.

- Re: [PR] [SPARK-52807][SDP] Proto changes to support analysis inside Declarative Pipelines query functions [spark] - posted by "sryza (via GitHub)" <gi...@apache.org> on 2025/10/01 00:27:35 UTC, 0 replies.
- Re: [PR] [SPARK-52401][SQL] Fix DataFrame.collect() cache invalidation after saveAsTable append; add regression test [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/01 00:30:52 UTC, 2 replies.
- Re: [PR] [SPARK-53767] Upgrade `Dropwizard` metrics to 4.2.33 [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 01:50:18 UTC, 1 replies.
- Re: [PR] [SPARK-53768] Upgrade `lombok` to 1.18.42 [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 01:51:03 UTC, 1 replies.
- Re: [PR] [SPARK-53769] Enable `Annotation Processing` during Java compilation [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 01:51:43 UTC, 1 replies.
- Re: [PR] [SPARK-53770] Upgrade `mockito` to 5.20.0 [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 01:52:18 UTC, 1 replies.
- Re: [PR] [SPARK-52640][SDP] Propagate Python Source Code Location [spark] - posted by "sryza (via GitHub)" <gi...@apache.org> on 2025/10/01 02:31:19 UTC, 2 replies.
- Re: [PR] [SPARK-53762][SQL] Add date and time conversions simplifier rule to optimizer [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 02:48:39 UTC, 13 replies.
- [PR] [SPARK-53773][SQL] Recover alphabetic ordering of rules in `RuleIdCollection` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 03:29:32 UTC, 5 replies.
- Re: [PR] [SPARK-53638][SS][PYTHON] Limit the byte size of arrow batch for TWS to avoid OOM [spark] - posted by "zeruibao (via GitHub)" <gi...@apache.org> on 2025/10/01 04:30:26 UTC, 18 replies.
- [PR] [SPARK-53774] Update `Spark Connect`-generated Swift code with protocgen-grpc-swift-2 v2.1.1 [spark-connect-swift] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 05:15:29 UTC, 4 replies.
- Re: [PR] [SPARK-53406][SQL] Avoid unnecessary shuffle join in direct passthrough shuffle id [spark] - posted by "szehon-ho (via GitHub)" <gi...@apache.org> on 2025/10/01 05:47:51 UTC, 10 replies.
- Re: [PR] [SPARK-53760][Geo][SQL] Introduce GeometryType and GeographyType [spark] - posted by "szehon-ho (via GitHub)" <gi...@apache.org> on 2025/10/01 06:57:10 UTC, 58 replies.
- Re: [PR] [SPARK-53738][SQL] PlannedWrite should preserve custom sort order when query output contains literal [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/01 09:01:47 UTC, 22 replies.
- [PR] [SPARK-53525][CONNECT][FOLLOWUP] Spark Connect ArrowBatch Result Chunking - Scala Client [spark] - posted by "xi-db (via GitHub)" <gi...@apache.org> on 2025/10/01 09:50:50 UTC, 3 replies.
- Re: [PR] [SPARK-33349][K8S] Reset the executor pods watcher when we receive a version changed from k8s [spark] - posted by "milleniax (via GitHub)" <gi...@apache.org> on 2025/10/01 11:14:59 UTC, 0 replies.
- [PR] [DO-NOT-MERGE][PROTOTYPE] Arrow-backed Parquet vectorized reader [spark] - posted by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2025/10/01 11:59:10 UTC, 1 replies.
- Re: [PR] [SPARK-53554] Fix `sum_distinct` [spark-connect-rust] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/01 12:12:29 UTC, 1 replies.
- Re: [PR] [SPARK-51847][PYTHON] Extend PySpark testing framework util functions with basic data tests [spark] - posted by "stanlocht (via GitHub)" <gi...@apache.org> on 2025/10/01 12:13:48 UTC, 0 replies.
- Re: [PR] [SPARK-53673][CONNECT][TESTS][4.0] Fix a flaky test failure in `SparkSessionE2ESuite - interrupt tag` caused by the usage of `ForkJoinPool` [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/01 12:15:09 UTC, 0 replies.
- Re: [PR] [SPARK-53673][CONNECT][TESTS][3.5] Fix a flaky test failure in `SparkSessionE2ESuite - interrupt tag` caused by the usage of `ForkJoinPool` [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/01 12:15:43 UTC, 0 replies.
- Re: [PR] [SPARK-53536][Core] Adding a Golden File Test With Randomly Generated SQL Scripts [spark] - posted by "TeodorDjelic (via GitHub)" <gi...@apache.org> on 2025/10/01 13:03:21 UTC, 4 replies.
- Re: [PR] [SPARK-52408][BUILD][SQL] Upgrade Hive to 4.1 [spark] - posted by "vrozov (via GitHub)" <gi...@apache.org> on 2025/10/01 13:40:41 UTC, 0 replies.
- Re: [PR] Add option to skip Unix native rm explicitly [spark] - posted by "vrozov (via GitHub)" <gi...@apache.org> on 2025/10/01 14:16:37 UTC, 0 replies.
- [PR] [WIP][SPARK-53771][PYTHON][ARROW] Add support for large list type in Arrow conversion [spark] - posted by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2025/10/01 16:25:12 UTC, 0 replies.
- Re: [PR] [SPARK-52677][SQL] Simplify DataTypeUtils.canWrite and TableOutputResolver [spark] - posted by "ksbeyer (via GitHub)" <gi...@apache.org> on 2025/10/01 16:43:12 UTC, 10 replies.
- Re: [PR] [SPARK-52614][SQL] Support RowEncoder inside Product Encoder [spark] - posted by "eejbyfeldt (via GitHub)" <gi...@apache.org> on 2025/10/01 17:29:31 UTC, 3 replies.
- [PR] [SPARK-53776][CONNECT] Support Spark Context Manager Support in Connect Mode [spark] - posted by "shujingyang-db (via GitHub)" <gi...@apache.org> on 2025/10/01 17:36:01 UTC, 3 replies.
- Re: [PR] [SPARK-53741][BUILD] Upgrade ORC to 2.2.1 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 17:49:23 UTC, 1 replies.
- Re: [PR] [SPARK-39800][SQL][WIP] DataSourceV2: View Support [spark] - posted by "jzhuge (via GitHub)" <gi...@apache.org> on 2025/10/01 17:51:14 UTC, 0 replies.
- [PR] [SPARK-53777] Update `Spark Connect`-generated `Swift` source code with `4.1.0-preview2` [spark-connect-swift] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 19:10:14 UTC, 2 replies.
- [PR] [SPARK-53778] Use JDK25 Gradle Image [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 20:38:16 UTC, 0 replies.
- Re: [PR] [SPARK-53778] Use JDK25 Gradle Image as builder images [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 20:54:38 UTC, 1 replies.
- [PR] [SPARK-53536][CORE][FOLLOWUP] Fixing Flakiness of Golden File Test With Randomly Generated SQL Scripts [spark] - posted by "TeodorDjelic (via GitHub)" <gi...@apache.org> on 2025/10/01 21:10:14 UTC, 2 replies.
- Re: [PR] [SPARK-53696][PYTHON]Default to bytes for BinaryType in PySpark [spark] - posted by "ueshin (via GitHub)" <gi...@apache.org> on 2025/10/01 21:30:09 UTC, 3 replies.
- [PR] [SPARK-53780] `spark-submission-worker` should depend on `io.fabric8` directly [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/01 22:40:07 UTC, 3 replies.
- Re: [PR] [SPARK-53771][PYTHON][ARROW] Add support for large list type in Arrow conversion [spark] - posted by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2025/10/01 23:47:45 UTC, 6 replies.
- [PR] [SPARK-53781] Exclude Spark's transitive dependencies consistently across modules [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/02 00:18:37 UTC, 6 replies.
- Re: [PR] [SPARK-53737][SQL][SS] Add Real-time Mode trigger [spark] - posted by "viirya (via GitHub)" <gi...@apache.org> on 2025/10/02 00:20:23 UTC, 6 replies.
- Re: [PR] [SPARK-53757] Upgrade Jetty to 12.0.12 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/02 03:17:38 UTC, 1 replies.
- [PR] [SPARK-53782] Use `slf4j-api` as a direct dependency [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/02 03:29:36 UTC, 2 replies.
- [PR] [SPARK-53783] Use `log4j2` instead of `log4j` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/02 04:12:24 UTC, 3 replies.
- [PR] [SPARK-53784] Additional Source APIs needed to support RTM execution [spark] - posted by "jerrypeng (via GitHub)" <gi...@apache.org> on 2025/10/02 05:25:28 UTC, 5 replies.
- [PR] [WIP] [SPARK-53785][SS] Memory Source for RTM [spark] - posted by "jerrypeng (via GitHub)" <gi...@apache.org> on 2025/10/02 06:02:32 UTC, 0 replies.
- [PR] [SPARK-52614][SQL][4.0] Support RowEncoder inside Product Encoder [spark] - posted by "eejbyfeldt (via GitHub)" <gi...@apache.org> on 2025/10/02 06:08:15 UTC, 2 replies.
- [PR] [SPARK-53786][SQL] Default value should not conflict with special column name [spark] - posted by "szehon-ho (via GitHub)" <gi...@apache.org> on 2025/10/02 06:09:41 UTC, 4 replies.
- [PR] [SPARK-53787] Upgrade Spark to 4.1.0-preview2 [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/02 06:27:29 UTC, 1 replies.
- [PR] [SPARK-53788][CORE] Move VersionUtils to common utils [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/02 06:32:42 UTC, 0 replies.
- Re: [PR] [SPARK-53787] Upgrade Spark to `4.1.0-preview2` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/02 06:38:24 UTC, 1 replies.
- [PR] [SPARK-53789][SQL][CONNECT] Unify error condition CANNOT_MODIFY_CONFIG [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/02 07:06:06 UTC, 7 replies.
- Re: [PR] [SPARK-53469][SQL] Ability to cleanup shuffle in Thrift server [spark] - posted by "karuppayya (via GitHub)" <gi...@apache.org> on 2025/10/02 14:12:42 UTC, 20 replies.
- [PR] [SPARK-issuesNo][CORE][SQL] Make the rename operations multi-threaded. [spark] - posted by "Xtpacz (via GitHub)" <gi...@apache.org> on 2025/10/02 15:12:45 UTC, 0 replies.
- [PR] [SPARK-53621][CORE][FOLLOWUP] Adding Comments And Enriching Tests of Continue Handler [spark] - posted by "TeodorDjelic (via GitHub)" <gi...@apache.org> on 2025/10/02 16:02:17 UTC, 5 replies.
- Re: [PR] [SPARK-53730] Switch Spark 3.5 Java 11 docker registry images to 11-jammy [spark-docker] - posted by "peter-toth (via GitHub)" <gi...@apache.org> on 2025/10/02 16:43:03 UTC, 5 replies.
- [PR] [SPARK-53585][BUILD] Upgrade Scala to 2.13.17 [spark] - posted by "vrozov (via GitHub)" <gi...@apache.org> on 2025/10/02 17:45:56 UTC, 13 replies.
- Re: [PR] [SPARK-53720][SQL] Simplify extracting Table from DataSourceV2Relation [spark] - posted by "karuppayya (via GitHub)" <gi...@apache.org> on 2025/10/02 19:37:08 UTC, 12 replies.
- [PR] [SPARK-53793][SQL] Use DSv2 predicate to evaluate InternalRow [spark] - posted by "yhuang-db (via GitHub)" <gi...@apache.org> on 2025/10/02 20:28:59 UTC, 7 replies.
- Re: [PR] [SPARK-37466][SQL] Support subexpression elimination in higher order functions [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/03 00:25:51 UTC, 1 replies.
- Re: [PR] [SPARK-52024][SQL] Support cancel ShuffleQueryStage when propagate empty relations [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/03 00:25:52 UTC, 1 replies.
- [PR] [SPARK-53794] Add option to limit deletions per maintenance operation associated with rocksdb state provider [spark] - posted by "anishshri-db (via GitHub)" <gi...@apache.org> on 2025/10/03 01:18:47 UTC, 0 replies.
- Re: [PR] [SPARK-53794][SS] Add option to limit deletions per maintenance operation associated with rocksdb state provider [spark] - posted by "anishshri-db (via GitHub)" <gi...@apache.org> on 2025/10/03 01:20:59 UTC, 14 replies.
- Re: [PR] [SPARK-53363] Add uuid builtin function [spark-connect-rust] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/03 07:22:05 UTC, 1 replies.
- [PR] [SPARK-53795][CONNECT] Remove unused parameters in LiteralValueProtoConverter [spark] - posted by "heyihong (via GitHub)" <gi...@apache.org> on 2025/10/03 11:55:25 UTC, 2 replies.
- [PR] SPARK-53790 support generic beans in arrow encoder [spark] - posted by "steven-aerts (via GitHub)" <gi...@apache.org> on 2025/10/03 14:32:41 UTC, 0 replies.
- [PR] done [spark] - posted by "SCHJonathan (via GitHub)" <gi...@apache.org> on 2025/10/03 18:21:06 UTC, 0 replies.
- [PR] [SPARK-53797] maxBytesPerTrigger on file stream sources is incredibly slow [spark] - posted by "Kimahriman (via GitHub)" <gi...@apache.org> on 2025/10/03 18:36:09 UTC, 1 replies.
- Re: [PR] [SPARK-53796] Add `extension` field to pipeline protos to support forward compatibility [spark] - posted by "sryza (via GitHub)" <gi...@apache.org> on 2025/10/03 18:47:48 UTC, 1 replies.
- [PR] [SPARK-53799][SPARK-53800][SS][DSV2] Enable column pruning and predicate pushdown in DSV2 streaming [spark] - posted by "gengliangwang (via GitHub)" <gi...@apache.org> on 2025/10/03 23:22:32 UTC, 3 replies.
- Re: [PR] [SPARK-37467][SQL] Consolidate subexpression elimination code for whole stage and non-whole stage [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/04 00:24:50 UTC, 1 replies.
- [PR] [SPARK-53802][SDP] Support string values for user-specified schema in SDP tables [spark] - posted by "sryza (via GitHub)" <gi...@apache.org> on 2025/10/04 14:55:42 UTC, 5 replies.
- Re: [PR] [SPARK-50686][SQL] Hash to sort aggregation fallback - memory usage optimization [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/05 00:29:26 UTC, 1 replies.
- [PR] [SPARK-XXXXX][BUILD] Migrate from javax.servlet to jakarta.servlet and add enforcer rules [spark] - posted by "Vamsi-klu (via GitHub)" <gi...@apache.org> on 2025/10/05 07:24:45 UTC, 0 replies.
- [PR] [SPARK-53803][ML][Feature] Add ArimaRegression for time series forecasting in MLlib [spark] - posted by "anandexplore (via GitHub)" <gi...@apache.org> on 2025/10/05 08:50:23 UTC, 0 replies.
- Re: [PR] [SPARK-53803][ML][Feature] Added ArimaRegression for time series forecasting in MLlib [spark] - posted by "anandexplore (via GitHub)" <gi...@apache.org> on 2025/10/05 09:10:45 UTC, 8 replies.
- [PR] [SPARK-53804][SQL] Support TIME radix sort [spark] - posted by "bersprockets (via GitHub)" <gi...@apache.org> on 2025/10/05 21:02:53 UTC, 3 replies.
- Re: [PR] [WIP][SPARK-53501][PYTHON][CORE][SQL][SS] refactor python worker factory API [spark] - posted by "benrobby (via GitHub)" <gi...@apache.org> on 2025/10/05 21:55:47 UTC, 0 replies.
- Re: [PR] [SPARK-53209][YARN] Add ActiveProcessorCount JVM option to YARN executor and AM [spark] - posted by "jzhuge (via GitHub)" <gi...@apache.org> on 2025/10/05 23:59:38 UTC, 3 replies.
- Re: [PR] [SPARK-51885][SQL] Change AnalysisContext.outerPlan from Option[LogicalPlan] to Seq[LogicalPlan] [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/06 00:27:24 UTC, 1 replies.
- Re: [PR] [SPARK-42841][SQL]Assign a name to the error class _LEGACY_ERROR_TEMP_2003 [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/06 00:27:26 UTC, 1 replies.
- Re: [PR] [SPARK-51332][SQL] DS V2 supports push down BIT_AND, BIT_OR, BIT_XOR, BIT_COUNT and BIT_GET [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/06 00:27:27 UTC, 2 replies.
- [PR] [SPARK-51756][CORE][FOLLOWUP] Fix code style issues [spark] - posted by "ivoson (via GitHub)" <gi...@apache.org> on 2025/10/06 03:31:27 UTC, 5 replies.
- [PR] [SPARK-53805][SQL] Push Variant into DSv2 scan [spark] - posted by "huaxingao (via GitHub)" <gi...@apache.org> on 2025/10/06 03:42:59 UTC, 11 replies.
- Re: [PR] [SPARK-53656][SS] Refactor MemoryStream to use SparkSession instead of SQLContext [spark] - posted by "ganeshashree (via GitHub)" <gi...@apache.org> on 2025/10/06 04:23:32 UTC, 14 replies.
- [PR] [CATALYST][SQL] Allow empty input on AES decrypt to have error class [spark] - posted by "richardc-db (via GitHub)" <gi...@apache.org> on 2025/10/06 06:18:01 UTC, 0 replies.
- [PR] [SPARK-50771][SPARK-53807][CORE] Fix race condition issues between `unlock` and `releaseAllLocksForTask` in `BlockManager` [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/06 08:23:05 UTC, 0 replies.
- Re: [PR] [SPARK-53691][PS][INFRA][TESTS] Further reorganize tests for Pandas API [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/06 08:47:41 UTC, 1 replies.
- [PR] [SPARK-53691][PS][INFRA][TESTS][4.0] Further reorganize tests for Pandas API [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/06 08:52:46 UTC, 2 replies.
- Re: [PR] [SPARK-53690][SS] Fix exponential formatting of avgOffsetsBehindLatest in Kafka sources object in progress json [spark] - posted by "jayantdb (via GitHub)" <gi...@apache.org> on 2025/10/06 08:54:16 UTC, 2 replies.
- [PR] [BUILD] refer to setup-sbt by hash [spark-connect-go] - posted by "raboof (via GitHub)" <gi...@apache.org> on 2025/10/06 11:42:47 UTC, 0 replies.
- [PR] [SPARK-53808][CONNECT] Allow to pass optional JVM options to `spark-connect-scala-client` [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/06 11:51:27 UTC, 0 replies.
- [PR] Bump google.golang.org/protobuf from 1.36.7 to 1.36.10 [spark-connect-go] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2025/10/06 13:40:43 UTC, 0 replies.
- Re: [PR] Bump google.golang.org/protobuf from 1.36.7 to 1.36.9 [spark-connect-go] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2025/10/06 13:40:45 UTC, 1 replies.
- Re: [PR] [SPARK-53808][CONNECT] Allow to pass optional JVM args to `spark-connect-scala-client` [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/06 14:02:38 UTC, 4 replies.
- Re: [PR] [SPARK-53573][SQL] Use Pre-processor for generalized parameter marker handling [spark] - posted by "miland-db (via GitHub)" <gi...@apache.org> on 2025/10/06 16:04:15 UTC, 120 replies.
- [PR] [SPARK-53792] Fix rocksdbPinnedBlocksMemoryUsage when bounded memory … [spark] - posted by "zifeif2 (via GitHub)" <gi...@apache.org> on 2025/10/06 17:31:32 UTC, 2 replies.
- [PR] Initial revision [spark] - posted by "asautins (via GitHub)" <gi...@apache.org> on 2025/10/06 17:40:46 UTC, 2 replies.
- [PR] [SPARK-53809][SQL]Add canonicalization for DSv2 scan [spark] - posted by "yhuang-db (via GitHub)" <gi...@apache.org> on 2025/10/06 20:26:44 UTC, 1 replies.
- [PR] [SPARK-53786][SQL] Default value with special column name should not conflict with real column [spark] - posted by "szehon-ho (via GitHub)" <gi...@apache.org> on 2025/10/06 20:29:44 UTC, 2 replies.
- Re: [PR] [SPARK-53786][SQL] Default value with special column name should not conflict with real column [spark] - posted by "szehon-ho (via GitHub)" <gi...@apache.org> on 2025/10/06 21:08:33 UTC, 22 replies.
- Re: [PR] [SPARK-53696][PYTHON][CONNECT][SQL]Default to bytes for BinaryType in PySpark [spark] - posted by "ueshin (via GitHub)" <gi...@apache.org> on 2025/10/06 21:08:49 UTC, 7 replies.
- [PR] [SPARK-53810] Split large TWS python tests into multiple small tests to speedup the CI [spark] - posted by "huanliwang-db (via GitHub)" <gi...@apache.org> on 2025/10/06 21:48:05 UTC, 0 replies.
- [PR] [SPARK-53812][SDP] Refactor DefineDataset and DefineFlow protos to group related properties and future-proof [spark] - posted by "sryza (via GitHub)" <gi...@apache.org> on 2025/10/06 23:18:15 UTC, 16 replies.
- Re: [PR] [SPARK-53751][SDP] Explicit Versioned Checkpoint Location [spark] - posted by "JiaqiWang18 (via GitHub)" <gi...@apache.org> on 2025/10/06 23:59:01 UTC, 5 replies.
- [PR] Added Execution Frame Subclasses [spark] - posted by "TeodorDjelic (via GitHub)" <gi...@apache.org> on 2025/10/07 00:06:57 UTC, 0 replies.
- Re: [PR] [SPARK-46912][CORE] Using correct environment variables on workers of StandAlone cluster [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/07 00:26:29 UTC, 2 replies.
- [PR] [SPARK-53813] Recover `Publish Snapshot Chart` GitHub Action Job by using GitHash [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 01:28:27 UTC, 2 replies.
- [PR] [SPARK-53814] Use Java `List.of` instead of `Collections.(empty|singleton)List` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 01:57:12 UTC, 3 replies.
- [PR] [SPARK-53815] Remove `branch-0.4` from daily `publish_snapshot_*` GitHub Action jobs [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 02:13:49 UTC, 2 replies.
- [PR] [SPARK-53816] Avoid Nested ExecutionIds with Duplicate Query Plans in Streaming Queries [spark] - posted by "zeruibao (via GitHub)" <gi...@apache.org> on 2025/10/07 03:33:16 UTC, 0 replies.
- Re: [PR] [SPARK-53816][SS] Avoid Nested ExecutionIds with Duplicate Query Plans in Streaming Queries [spark] - posted by "zeruibao (via GitHub)" <gi...@apache.org> on 2025/10/07 04:09:16 UTC, 0 replies.
- [PR] [SPARK-53817] Use `o.a.s.util.Pair` instead of `o.a.commons.lang3.tuple.Pair` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 04:09:42 UTC, 3 replies.
- [PR] [SPARK-53818] Improve `SentinelManager.toString` by JEP-280 instead of `ToStringBuilder` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 04:43:29 UTC, 0 replies.
- Re: [PR] [SPARK-53818] Improve `SentinelResourceState.toString` by JEP-280 instead of `ToStringBuilder` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 06:15:58 UTC, 2 replies.
- Re: [PR] [SPARK-52509][K8S] Cleanup shuffles from fallback storage [spark] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2025/10/07 07:55:41 UTC, 3 replies.
- Re: [PR] [SPARK-52505][K8S] Allow to create executor kubernetes service [spark] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2025/10/07 07:59:05 UTC, 0 replies.
- [PR] [SPARK-53819][3.5] Upgrade commons-lang3 to 3.19.0 [spark] - posted by "eschcam (via GitHub)" <gi...@apache.org> on 2025/10/07 09:59:34 UTC, 4 replies.
- Re: [PR] [SPARK-53810][SS] Split large TWS python tests into multiple small tests to speedup the CI [spark] - posted by "anishshri-db (via GitHub)" <gi...@apache.org> on 2025/10/07 16:39:15 UTC, 0 replies.
- [PR] [SPARK-53820] Introduce `o.a.s.k8s.operator.utils.StringUtils` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 17:02:26 UTC, 4 replies.
- [PR] [WIP][SPARK-XXXXX][PS][SS][TESTS]Add Python TransformWithState test case for composite output schemas [spark] - posted by "jiateoh (via GitHub)" <gi...@apache.org> on 2025/10/07 17:37:03 UTC, 0 replies.
- [PR] [SPARK-53821] Ban `org.apache.commons.lang3` package [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 17:39:31 UTC, 3 replies.
- [PR] feat: implement transform() in Column API [spark] - posted by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2025/10/07 18:02:54 UTC, 0 replies.
- Re: [PR] [SPARK-53788][CORE] Move VersionUtils to `common` module [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 19:03:37 UTC, 0 replies.
- [PR] [SDP] typo hunting + formatting [spark] - posted by "jaceklaskowski (via GitHub)" <gi...@apache.org> on 2025/10/07 20:13:43 UTC, 1 replies.
- [PR] [SPARK-53824] Ban `org.apache.commons.collections4` package [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 20:32:46 UTC, 3 replies.
- [PR] [SPARK-53825] Use Java `MessageDigest` instead of `org.apache.commons.codec` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 21:09:20 UTC, 2 replies.
- Re: [PR] [SPARK-53822][PS][SS][TESTS]Add Python TransformWithState test case for composite/nested output schemas [spark] - posted by "bogao007 (via GitHub)" <gi...@apache.org> on 2025/10/07 21:09:36 UTC, 1 replies.
- Re: [PR] [SPARK-53792][SS] Fix rocksdbPinnedBlocksMemoryUsage when bounded memory … [spark] - posted by "micheal-o (via GitHub)" <gi...@apache.org> on 2025/10/07 21:25:52 UTC, 1 replies.
- [PR] [SPARK-53826] Use Java 25 in GitHub Action jobs consistently [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 21:29:00 UTC, 2 replies.
- [PR] [SPARK-53827] Exclude `commons-(codec|compress)` and `com.ibm.icu` transitive dependencies [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 23:33:21 UTC, 2 replies.
- [PR] [SPARK-53828] Exclude `org/apache/spark/ui/static/**` from Spark operator jar [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/07 23:54:11 UTC, 3 replies.
- [PR] [WIP][SPARK-XXXXX][PS][SS] Fix partial read bug for large state values in TransformWithStateInPySparkStateServer [spark] - posted by "jiateoh (via GitHub)" <gi...@apache.org> on 2025/10/08 00:02:35 UTC, 0 replies.
- Re: [PR] [SPARK-51051][SQL]Add an optional parameter for `array_position` function to specify starting index for matching. [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/08 00:26:20 UTC, 1 replies.
- [PR] [SPARK-53829][PYTHON] Support `datetime.time` in column operators [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/08 01:37:37 UTC, 2 replies.
- [PR] [SPARK-53830] Exclude `.idea` directory from `Spotless` check [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/08 02:21:46 UTC, 2 replies.
- Re: [PR] [SPARK-53822][PYTHON][SS][TESTS]Add Python TransformWithState test case for composite/nested output schemas [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/08 02:24:57 UTC, 2 replies.
- [PR] [SPARK-53831][INFRA] Update script `free_disk_space` [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/08 02:42:36 UTC, 4 replies.
- [PR] [SPARK-53832][K8S] Make `KubernetesClientUtils` Java-friendly [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/08 04:05:13 UTC, 3 replies.
- Re: [PR] [SPARK-53779][SQL] Implement transform() in Column API [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/08 04:42:48 UTC, 0 replies.
- [PR] [SPARK-53833][PYTHON] Update `dev/requirements.txt` to skip `torch/torchvision` in Python 3.14 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/08 04:58:32 UTC, 5 replies.
- [PR] [SPARK-53834][INFRA] Add a separate docker file for Python 3.14 daily build [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/08 05:39:08 UTC, 4 replies.
- [PR] [SPARK-53836][INFRA] Update script `free_disk_space_container` [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/08 11:59:20 UTC, 1 replies.
- Re: [PR] [SPARK-53779][SQL][CONNECT] Implement transform() in Column API [spark] - posted by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2025/10/08 15:15:46 UTC, 12 replies.
- Re: [PR] [SPARK-53806][SQL] Allow empty input on AES decrypt to have error class [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/08 15:20:52 UTC, 0 replies.
- Re: [PR] [SPARK-53125][TEST] RemoteSparkSession prints whole `spark-submit` command [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/08 15:29:47 UTC, 0 replies.
- [PR] [SQL] Fix PlanID preservation in ResolveSQLFunctions with UDFs [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/08 17:03:25 UTC, 0 replies.
- [PR] [SPARK-53843][BUILD] Upgrade `netty-tcnative` to 2.0.74.Final [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/08 17:20:00 UTC, 4 replies.
- Re: [PR] [SPARK-53810][SS][TESTS] Split large TWS python tests into multiple small tests to speedup the CI [spark] - posted by "anishshri-db (via GitHub)" <gi...@apache.org> on 2025/10/08 17:38:36 UTC, 0 replies.
- [PR] [SPARK-53844] Remove `SPARK_JENKINS` and related logics from `dev/run-tests.py` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/08 19:47:13 UTC, 0 replies.
- Re: [PR] [SPARK-52578][SQL] Add metrics for rows to track case and action in MergeRowsExec [spark] - posted by "aokolnychyi (via GitHub)" <gi...@apache.org> on 2025/10/08 20:26:01 UTC, 1 replies.
- Re: [PR] [SPARK-53844][TESTS] Remove `SPARK_JENKINS*` and related logics from `dev/run-tests.py` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/08 20:28:40 UTC, 2 replies.
- [PR] [SPARK-53846][PYTHON][TESTS] Skip `test_profile_pandas_*` tests if pandas or pyarrow are unavailable [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/08 20:37:31 UTC, 2 replies.
- [PR] [SPARK-53847] Add ContinuousMemorySink for Real-time Mode testing [spark] - posted by "jerrypeng (via GitHub)" <gi...@apache.org> on 2025/10/08 20:48:45 UTC, 11 replies.
- [PR] [SPARK-53848] Add ability to support Alpha family in Theta Aggregates [spark] - posted by "karuppayya (via GitHub)" <gi...@apache.org> on 2025/10/08 21:50:22 UTC, 17 replies.
- [PR] [SPARK-53849][BUILD] Upgrade Netty to 4.2.6.Final [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/08 22:21:44 UTC, 9 replies.
- [PR] [SPARK-53850][SDP] Define proto for Sinks [spark] - posted by "JiaqiWang18 (via GitHub)" <gi...@apache.org> on 2025/10/08 23:31:33 UTC, 2 replies.
- Re: [PR] [WIP][SPARK-51169] Set up a daily job for Python 3.14 [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/08 23:43:48 UTC, 4 replies.
- Re: [PR] [SPARK-50292] Add MapStatus RowCount optimize skewed job [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/09 00:26:18 UTC, 1 replies.
- [PR] [SPARK-53742][SQL] Push down the filter used in the count_if function [spark] - posted by "r7raul1984 (via GitHub)" <gi...@apache.org> on 2025/10/09 01:47:31 UTC, 2 replies.
- Re: [PR] [SPARK-53791][CORE][SQL] Make the rename operations multi-threaded. [spark] - posted by "Xtpacz (via GitHub)" <gi...@apache.org> on 2025/10/09 02:37:53 UTC, 5 replies.
- [PR] [SPARK-53851] Set `io.netty.noUnsafe` to `true` to suppress JEP-498 warning messages [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/09 03:37:32 UTC, 0 replies.
- [PR] [SPARK-53852] Add `spark.kubernetes.driver.pod.excludedFeatureSteps` usage example [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/09 04:13:48 UTC, 2 replies.
- Re: [PR] [SPARK-53622][CORE][TEST] Improve `UninterruptibleThreadSuite` [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/09 04:37:10 UTC, 4 replies.
- Re: [PR] [SPARK-53507][CORE] Don't use case class for BreakingChangeInfo [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/09 04:42:35 UTC, 3 replies.
- Re: [PR] [SPARK-53687][SQL][SS][SDP] Introduce WATERMARK clause in SQL statement [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/09 04:49:31 UTC, 43 replies.
- [PR] [SPARK-53853] Add `Example` section in `operators.md` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/09 05:10:30 UTC, 3 replies.
- [PR] [SPARK-53854][PYTHON][TESTS] Skip `test_collect_time` test if pandas or pyarrow are unavailable [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/09 05:39:21 UTC, 5 replies.
- Re: [PR] [SPARK-53851] Set `io.netty.noUnsafe` to `true` to avoid JEP-498 warnings [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/09 06:58:35 UTC, 1 replies.
- [PR] [SPARK-51169][PYTHON] Add Python 3.13 support in Spark Classic [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/09 08:07:24 UTC, 0 replies.
- [PR] [WIP][SPARK-53535][SQL] Fix missing structs always being assumed as nulls [spark] - posted by "ZiyaZa (via GitHub)" <gi...@apache.org> on 2025/10/09 08:34:43 UTC, 0 replies.
- Re: [PR] [SPARK-53564][CORE] Avoid DAGScheduler exits due to blockManager RPC timeout in DAGSchedulerEventProcessLoop [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/09 11:18:57 UTC, 1 replies.
- Re: [PR] [SPARK-53855][SQL] Fix PlanID preservation in ResolveSQLFunctions with UDFs [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/09 12:14:34 UTC, 2 replies.
- Re: [PR] [SPARK-53764][SQL] Collapse nested get_json_object [spark] - posted by "wecharyu (via GitHub)" <gi...@apache.org> on 2025/10/09 15:24:49 UTC, 0 replies.
- Re: [PR] [SPARK-51169][PYTHON] Add Python 3.14 support in Spark Classic [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/09 15:47:49 UTC, 1 replies.
- [PR] [SPARK-53856][CORE] Remove `blacklist` alternative config names [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/09 16:22:37 UTC, 7 replies.
- Re: [PR] [SPARK-53850][SDP] Define proto for Sinks and Rename DefineDataset to DefineOutput [spark] - posted by "JiaqiWang18 (via GitHub)" <gi...@apache.org> on 2025/10/09 17:37:19 UTC, 4 replies.
- Re: [PR] [SPARK-52798] [SQL] Add function approx_top_k_combine [spark] - posted by "gengliangwang (via GitHub)" <gi...@apache.org> on 2025/10/09 18:00:37 UTC, 23 replies.
- [PR] [SPARK-53857] Enable messageTemplate propagation to SparkThrowable [spark] - posted by "miland-db (via GitHub)" <gi...@apache.org> on 2025/10/09 18:26:58 UTC, 8 replies.
- Re: [PR] Catch InterruptedIOException and ClosedByInterruptException in TransformWithStateInPySparkStateServer to avoid worker crash [spark] - posted by "jiateoh (via GitHub)" <gi...@apache.org> on 2025/10/09 18:37:52 UTC, 0 replies.
- [PR] [SPARK-53858][PYTHON][TESTS] Skip `pyarrow` related doctests in `pyspark/sql/functions/builtin.py` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/09 21:08:31 UTC, 0 replies.
- Re: [PR] [SPARK-53796][SDP] Add `extension` field to a few pipeline protos to support forward compatibility [spark] - posted by "AnishMahto (via GitHub)" <gi...@apache.org> on 2025/10/09 21:25:58 UTC, 7 replies.
- Re: [PR] [SPARK-53858][PYTHON][TESTS] Skip `pyarrow`-related doctests in `pyspark/sql/functions/builtin.py` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/09 22:20:48 UTC, 6 replies.
- [PR] [SPARK-53859][BUILD][TESTS] Upgrade `JUnit` to 6.0.0 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/09 22:40:06 UTC, 4 replies.
- [PR] [SPARK-53860][BUILD][TESTS] Upgrade `sbt-jupiter-interface` to 0.17.0 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/09 23:02:49 UTC, 2 replies.
- [PR] [SPARK-53845] SDP Sinks [spark] - posted by "JiaqiWang18 (via GitHub)" <gi...@apache.org> on 2025/10/09 23:09:12 UTC, 6 replies.
- Re: [PR] [SPARK-51035][BUILD] Upgrade Janino to 3.1.12 [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/10 00:26:42 UTC, 1 replies.
- Re: [PR] [SPARK-53535][SQL] Fix missing structs always being assumed as nulls [spark] - posted by "Kimahriman (via GitHub)" <gi...@apache.org> on 2025/10/10 01:05:17 UTC, 22 replies.
- [PR] [SPARK-53861][PYTHON][INFRA] Factor out streaming tests from `spark-sql` and `spark-connect` [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/10 01:42:01 UTC, 0 replies.
- [PR] [SPARK-53862][DSTREAM][TESTS] Fix `CheckpointSuite.'get correct sparkdriver.[host|port] from checkpoint'` test flakiness [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/10 02:04:03 UTC, 4 replies.
- [PR] [MINOR][PYTHON][TESTS] Retry `test_observe_with_map_type` [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/10 02:04:29 UTC, 2 replies.
- Re: [PR] [SPARK-53544][PYTHON] Support complex types on observations [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/10 02:09:51 UTC, 0 replies.
- [PR] [SPARK-53562][PYTHON][TESTS][FOLLOW-UP] Add more tests for `maxBytesPerBatch` [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/10 02:36:08 UTC, 2 replies.
- [PR] [SPARK-53863] Remove `Java 17` requirement from `deploy.gradle` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/10 03:25:11 UTC, 2 replies.
- [PR] [MINOR][DOCS] Add `build_python_3.14.yml` to `Build Pipeline Status` section of `README.md` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/10 04:35:56 UTC, 2 replies.
- [PR] [SPARK-53858][PYTHON][TESTS] Skip `udf` doctests without pyarrow [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/10 05:01:59 UTC, 0 replies.
- Re: [PR] [SPARK-53858][PYTHON][TESTS] Skip doctests in `pyspark.sql.functions.builtin` if pyarrow is not installed [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/10 05:12:00 UTC, 1 replies.
- Re: [PR] [BUILD] Upgrade commons-lang3 to 3.19.0 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/10 05:25:14 UTC, 0 replies.
- Re: [PR] [SPARK-53861][PYTHON][INFRA] Factor out streaming tests from `pyspark-sql` and `pyspark-connect` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/10 05:27:25 UTC, 1 replies.
- Re: [PR] [SPARK-53864][BUILD] Upgrade `commons-lang3` to 3.19.0 [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/10 06:37:07 UTC, 2 replies.
- [PR] [SPARK-53605][INFRA] Restore pyspark execution timeout to 2 hours [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/10 09:51:23 UTC, 3 replies.
- Re: [PR] [SPARK-53779][SQL][CONNECT] Implement `transform()` in Column API [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/10 10:56:57 UTC, 1 replies.
- [PR] [SQL][SPARK-53865] Extract common logic from ResolveGenerate rule [spark] - posted by "mikhailnik-db (via GitHub)" <gi...@apache.org> on 2025/10/10 11:13:52 UTC, 4 replies.
- [PR] [SPARK-53866][PYTHON][TESTS] Skip doctest `pyspark.sql.pandas.functions` without pyarrow/pandas [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/10 12:27:05 UTC, 2 replies.
- Re: [PR] [SPARK-53455][CONNECT] Add `CloneSession` RPC [spark] - posted by "vicennial (via GitHub)" <gi...@apache.org> on 2025/10/10 13:50:08 UTC, 2 replies.
- [PR] Block unsupported aggregates [spark] - posted by "alekjarmov (via GitHub)" <gi...@apache.org> on 2025/10/10 16:14:50 UTC, 0 replies.
- [PR] [SPARK-53869] Support multiple files in `pyFiles` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/10 18:18:26 UTC, 0 replies.
- Re: [PR] [SPARK-53869] Support multiple files in `pyFiles` field [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/10 18:30:22 UTC, 4 replies.
- [PR] [SPARK-53871] Upgrade `JUnit` to 6.0.0 [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/10 20:00:02 UTC, 3 replies.
- [PR] [SPARK-53872] Revisit `JUnit` assert usage in test cases [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/10 20:50:55 UTC, 2 replies.
- [PR] [SPARK-53874] `SparkAppDriverConf` should respect `sparkVersion` of `SparkApplication` CRD [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/11 00:05:41 UTC, 4 replies.
- Re: [PR] [SPARK-53870][PYTHON][SS] Fix partial read bug for large proto messages in TransformWithStateInPySparkStateServer [spark] - posted by "jiateoh (via GitHub)" <gi...@apache.org> on 2025/10/11 01:16:58 UTC, 4 replies.
- [PR] [SPARK-53875][SQL] Fix bit_count incorrect results for negative byte/short/int values [spark] - posted by "yaooqinn (via GitHub)" <gi...@apache.org> on 2025/10/11 04:23:29 UTC, 1 replies.
- [PR] [SPARK-51166][SQL][CONNECT] Fix race condition issue related to ObservedMetrics [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/11 13:54:04 UTC, 0 replies.
- Re: [PR] [SPARK-53873][SQL] ExplodeBase.eval - Return IterableOnce to iterate over input [spark] - posted by "asautins (via GitHub)" <gi...@apache.org> on 2025/10/11 18:30:48 UTC, 0 replies.
- [PR] ExplodeBase.eval IterableOnce directly on input [spark] - posted by "asautins (via GitHub)" <gi...@apache.org> on 2025/10/11 18:32:59 UTC, 0 replies.
- [PR] [SPARK-53815][FOLLOWU] Remove `branch-0.4` from `matrix` too [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/11 19:13:13 UTC, 0 replies.
- Re: [PR] [SPARK-53815][FOLLOWUP] Remove `branch-0.4` from `matrix` too [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/11 19:46:06 UTC, 1 replies.
- Re: [PR] [SPARK-XXX][SQL][CONNECT] Fix race condition issue related to ObservedMetrics [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/11 22:57:47 UTC, 0 replies.
- Re: [PR] [SPARK-53878][SQL][CONNECT] Fix race condition issue related to ObservedMetrics [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/11 23:00:51 UTC, 11 replies.
- Re: [PR] [WIP][SQL][TESTS] Disable stable column aliases in tests if assumed [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/12 00:28:08 UTC, 1 replies.
- Re: [PR] [SPARK-53868][SQL] Only use signature with Expression[] of `visitAggregateFunction` in V2ExpressionSQBuilder [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/12 04:44:28 UTC, 1 replies.
- [PR] [SPARK-53879] Upgrade `Ammonite` to 3.0.3 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/12 18:00:40 UTC, 2 replies.
- [PR] [SPARK-XXXXX][SQL] Support DSv2 in PushVariantIntoScan [spark] - posted by "viirya (via GitHub)" <gi...@apache.org> on 2025/10/12 18:21:34 UTC, 0 replies.
- Re: [PR] [SPARK-53880][SQL] Support DSv2 in PushVariantIntoScan [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/12 18:27:05 UTC, 26 replies.
- [PR] [SPARK-53881][BUILD][TESTS] Upgrade `Selenium` to 4.32.0 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/12 18:45:43 UTC, 2 replies.
- Re: [PR] [SDP] typo hunting and code (re)formatting [spark] - posted by "jaceklaskowski (via GitHub)" <gi...@apache.org> on 2025/10/12 19:21:44 UTC, 8 replies.
- Re: [PR] [SPARK-53753][BUILD][SQL] shade antlr4-runtime [spark] - posted by "vrozov (via GitHub)" <gi...@apache.org> on 2025/10/12 23:51:27 UTC, 1 replies.
- [PR] [BUILD] Upgrade POM to SBT plugin [spark] - posted by "gemelen (via GitHub)" <gi...@apache.org> on 2025/10/13 00:54:44 UTC, 4 replies.
- [PR] [SPARK-53609][PYTHON] Limit Arrow batch sizes in SQL_GROUPED_AGG_PANDAS_UDF [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/13 03:34:12 UTC, 3 replies.
- Re: [PR] [SPARK-53636][CORE] Fix thread-safety issue in SortShuffleManager.unregisterShuffle [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/13 05:38:26 UTC, 1 replies.
- [PR] [SPARK-53877] [spark] - posted by "Pragya4 (via GitHub)" <gi...@apache.org> on 2025/10/13 05:43:43 UTC, 3 replies.
- [PR] Test SBT 1.11.7 [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/13 05:44:33 UTC, 1 replies.
- [PR] [SPARK-53738][SQL] Fix planned write when query output contains foldable orderings [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/13 05:48:11 UTC, 50 replies.
- [PR] Add documentation comparing behavioral differences between Spark Connect and Spark Classic [spark] - posted by "xi-db (via GitHub)" <gi...@apache.org> on 2025/10/13 08:35:03 UTC, 0 replies.
- [PR] [SPARK-53877] Introduce BITMAP_AND_AGG function [spark] - posted by "uros7251brick (via GitHub)" <gi...@apache.org> on 2025/10/13 08:58:23 UTC, 2 replies.
- [PR] testing [spark] - posted by "alekjarmov (via GitHub)" <gi...@apache.org> on 2025/10/13 08:59:27 UTC, 0 replies.
- [PR] [CONNECT][TESTS] Add dependency checks for Python-related tests in the `connect` module [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/13 10:06:52 UTC, 1 replies.
- Re: [PR] [SPARK-53868][SQL] Use Array.equals instead of direct reference check in V2ExpressionBuilder [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/13 10:21:30 UTC, 1 replies.
- Re: [PR] [SPARK-37210][CORE][SQL] Allow forced use of staging directory [spark] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2025/10/13 11:03:32 UTC, 2 replies.
- [PR] Always set a sql state in spark connect client [spark] - posted by "garlandz-db (via GitHub)" <gi...@apache.org> on 2025/10/13 12:45:40 UTC, 0 replies.
- Re: [PR] [SPARK-53319][SQL] Support the time type by try_make_timestamp_ltz() [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/13 13:17:13 UTC, 6 replies.
- [PR] Bump google.golang.org/grpc from 1.75.0 to 1.76.0 [spark-connect-go] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2025/10/13 14:00:04 UTC, 0 replies.
- Re: [PR] Bump google.golang.org/grpc from 1.75.0 to 1.75.1 [spark-connect-go] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2025/10/13 14:00:06 UTC, 1 replies.
- [PR] Bump golang.org/x/oauth2 from 0.30.0 to 0.32.0 [spark-connect-go] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2025/10/13 14:09:42 UTC, 0 replies.
- Re: [PR] Bump golang.org/x/oauth2 from 0.30.0 to 0.31.0 [spark-connect-go] - posted by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2025/10/13 14:09:44 UTC, 1 replies.
- Re: [PR] [SPARK-53868][SQL] Use array length check instead of direct reference check in V2ExpressionBuilder [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/13 15:13:03 UTC, 1 replies.
- [PR] Revert "[SPARK-53805][SQL] Push Variant into DSv2 scan" [spark] - posted by "viirya (via GitHub)" <gi...@apache.org> on 2025/10/13 15:22:38 UTC, 3 replies.
- [PR] [SPARK-53884][BUILD] Upgrade `ZSTD-JNI` to 1.5.7-5 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/13 15:32:24 UTC, 3 replies.
- [PR] [WIP] Tws read fully fix [spark] - posted by "jiateoh (via GitHub)" <gi...@apache.org> on 2025/10/13 16:23:39 UTC, 1 replies.
- [PR] [SPARK-53841][Python] Implement transform() in Column API [spark] - posted by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2025/10/13 16:58:06 UTC, 5 replies.
- [PR] Revert "[MINOR][PYTHON][TESTS] Retry `test_observe_with_map_type`" [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/13 18:59:46 UTC, 4 replies.
- [PR] [SPARK-53891][SQL] Model DSV2 Commit Operation Metrics API [spark] - posted by "szehon-ho (via GitHub)" <gi...@apache.org> on 2025/10/13 19:01:38 UTC, 50 replies.
- [PR] [WIP] Tws read fully fix 4.0 [spark] - posted by "jiateoh (via GitHub)" <gi...@apache.org> on 2025/10/13 19:16:07 UTC, 0 replies.
- Re: [PR] [SPARK-53882][CONNECT][DOCS] Add documentation comparing behavioral differences between Spark Connect and Spark Classic [spark] - posted by "hvanhovell (via GitHub)" <gi...@apache.org> on 2025/10/13 19:17:33 UTC, 3 replies.
- [PR] [SPARK-XXX] Remove deprecated API usages of DescribeTopicsResult [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/13 20:58:37 UTC, 0 replies.
- [PR] Feature/spark 47230 add posexplode support [spark] - posted by "IgorBerman (via GitHub)" <gi...@apache.org> on 2025/10/13 21:33:32 UTC, 0 replies.
- Re: [PR] [SPARK-53870][PYTHON][SS] Fix partial read bug for large proto messages in TransformWithStateInPySparkStateServer (branch-4.0 PR) [spark] - posted by "jiateoh (via GitHub)" <gi...@apache.org> on 2025/10/13 21:42:56 UTC, 0 replies.
- Re: [PR] [SPARK-53841][Python][Connect] Implement `transform()` in Column API [spark] - posted by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2025/10/14 00:33:50 UTC, 0 replies.
- [PR] [SPARK-53732][SQL] Remember TimeTravelSpec in DataSourceV2Relation [spark] - posted by "aokolnychyi (via GitHub)" <gi...@apache.org> on 2025/10/14 00:40:55 UTC, 78 replies.
- Re: [PR] [SPARK-53841][PYTHON][CONNECT] Implement `transform()` in Column API [spark] - posted by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2025/10/14 00:50:25 UTC, 3 replies.
- Re: [PR] [SPARK-53892][SS] Use `DescribeTopicsResult.allTopicNames` instead of the deprecated `all` API [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/14 00:55:01 UTC, 1 replies.
- Re: [PR] [SPARK-53870][PYTHON][SS][4.0] Fix partial read bug for large proto messages in TransformWithStateInPySparkStateServer [spark] - posted by "HeartSaVioR (via GitHub)" <gi...@apache.org> on 2025/10/14 02:02:11 UTC, 3 replies.
- [PR] [SPARK-53893][TESTS] Regenerate benchmark results after upgrading to Scala 2.13.17 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/14 02:10:57 UTC, 8 replies.
- [PR] [SPARK-53894][BUILD][TESTS] Upgrade `docker-java` to 3.6.0 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/14 02:23:41 UTC, 3 replies.
- [PR] [SPARK-53895] Add a missing test for ContinuousMemorySink [spark] - posted by "jerrypeng (via GitHub)" <gi...@apache.org> on 2025/10/14 02:27:50 UTC, 0 replies.
- [PR] [SPARK-53896][CORE] Enable `spark.io.compression.lzf.parallel.enabled` by default [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/14 02:44:02 UTC, 5 replies.
- [PR] [MINOR][DOCS] Fix 404 for Python Package Management link in rdd-programming-guide.md [spark] - posted by "yaooqinn (via GitHub)" <gi...@apache.org> on 2025/10/14 03:21:46 UTC, 1 replies.
- Re: [PR] [SPARK-51426][PYTHON][SQL] Fix 'Setting metadata to empty dict does not work' [spark] - posted by "petern48 (via GitHub)" <gi...@apache.org> on 2025/10/14 03:45:13 UTC, 2 replies.
- Re: [PR] [SPARK-53785][SS] Memory Source for RTM [spark] - posted by "jerrypeng (via GitHub)" <gi...@apache.org> on 2025/10/14 03:58:36 UTC, 21 replies.
- [PR] [SPARK-53867][PYTHON] Limit Arrow batch sizes in SQL_GROUPED_AGG_ARROW_UDF [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/14 04:26:56 UTC, 2 replies.
- [PR] [SPARK-53898][CORE] Shuffle cleanup should not clean MapOutputTrackerMaster.shuffleStatuses in local cluster [spark] - posted by "Ngone51 (via GitHub)" <gi...@apache.org> on 2025/10/14 04:57:56 UTC, 21 replies.
- Re: [PR] [SPARK-53897][CONNECT][TESTS] Add dependency checks for Python-related tests in the `connect` module [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/14 05:36:35 UTC, 2 replies.
- Re: [PR] [SPARK-53439][SQL] Support DSv2 ParquetScan Dynamic Partition Pruning [spark] - posted by "flaming-archer (via GitHub)" <gi...@apache.org> on 2025/10/14 08:45:19 UTC, 0 replies.
- [PR] Update DAGScheduler.scala [spark] - posted by "WHJian (via GitHub)" <gi...@apache.org> on 2025/10/14 08:55:32 UTC, 1 replies.
- [PR] [SPARK-53611][PYTHON] Limit Arrow batch sizes in window agg UDFs [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/14 10:41:02 UTC, 3 replies.
- [PR] bump josdk version to 5.1.4 [spark-kubernetes-operator] - posted by "csviri (via GitHub)" <gi...@apache.org> on 2025/10/14 11:05:51 UTC, 0 replies.
- [PR] [SPARK-53900][CONNECT] Fix unintentional `Thread.wait(0)` under rare conditions in `ExecuteGrpcResponseSender` [spark] - posted by "vicennial (via GitHub)" <gi...@apache.org> on 2025/10/14 12:52:08 UTC, 6 replies.
- Re: [PR] [SPARK-53882][CONNECT] Always set a sql state in spark connect client [spark] - posted by "hvanhovell (via GitHub)" <gi...@apache.org> on 2025/10/14 12:57:41 UTC, 0 replies.
- [PR] [SPARK-53901][CONNECT] Clear `stagedArtifacts` buffer in `SparkConnectAddArtifactsHandler` upon flush/error [spark] - posted by "vicennial (via GitHub)" <gi...@apache.org> on 2025/10/14 13:37:23 UTC, 1 replies.
- [PR] [SPARK-53902] [SQL] Add tree node pattern bits for supported expressions in `ParameterizedQuery` argument list [spark] - posted by "mihailoale-db (via GitHub)" <gi...@apache.org> on 2025/10/14 13:58:50 UTC, 3 replies.
- Re: [PR] [SPARK-53339][CONNECT] Fix an issue which occurs when an operation in pending state is interrupted [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/14 15:07:04 UTC, 2 replies.
- Re: [PR] [SPARK-53901][CONNECT] Handle potential memory leaks in`SparkConnectAddArtifactsHandler` upon flush/error [spark] - posted by "hvanhovell (via GitHub)" <gi...@apache.org> on 2025/10/14 16:11:33 UTC, 5 replies.
- [PR] [SPARK-53904][SS] Use `Consumer.poll(Duration)` instead of `Consumer.poll(long)` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/14 16:19:25 UTC, 3 replies.
- [PR] [WIP][CONNECT] Support large local relations [spark] - posted by "khakhlyuk (via GitHub)" <gi...@apache.org> on 2025/10/14 18:03:50 UTC, 0 replies.
- [PR] [SPARK-53906][K8S] Protect `ExecutorPodsAllocator.numOutstandingPods` as `protected val` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/14 19:05:15 UTC, 4 replies.
- Re: [PR] [PYTHON] Remove unused type ignores [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/14 19:17:42 UTC, 0 replies.
- [PR] Update configuration doc with appropriate prometheus sink class [spark-kubernetes-operator] - posted by "ashangit (via GitHub)" <gi...@apache.org> on 2025/10/14 19:29:51 UTC, 1 replies.
- Re: [PR] [SPARK-53899] Update `Operator SDK` to 5.1.4 [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/14 19:50:22 UTC, 0 replies.
- [PR] [SPARK-53907][K8S] Support `spark.kubernetes.allocation.maximum` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/14 20:56:40 UTC, 5 replies.
- [PR] [SPARK-53908][CONNECT] Fix observations on Spark Connect with plan cache [spark] - posted by "ueshin (via GitHub)" <gi...@apache.org> on 2025/10/14 22:45:32 UTC, 2 replies.
- [PR] [SPARK-53909] Fix RBAC to allow `Spark` driver to create `StatefulSet` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/14 23:37:17 UTC, 4 replies.
- [PR] [SPARK-53910] Add `StatefulSet`-based SparkApp example [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/15 00:25:57 UTC, 2 replies.
- [PR] [SPARK-53867][PYTHON][FOLLOW-UP] Fix `pa.concat_batches` for old pyarrow versions [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/15 01:57:13 UTC, 2 replies.
- [PR] [SPARK-53911] Support `SPARK_VERSION` placeholder in container image names [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/15 02:05:57 UTC, 4 replies.
- Re: [PR] [SPARK-53895][SS][TESTS] Add `ContinuousMemorySuite` [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/15 02:35:46 UTC, 1 replies.
- [PR] [SPARK-53912] Add `cluster-preview.yaml` and `spark-connect-server-preview.yaml` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/15 02:54:40 UTC, 3 replies.
- [PR] [SPARK-53913][DOCS] Document newly added K8s configurations [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/15 03:34:46 UTC, 4 replies.
- Re: [PR] [SPARK-53696][PYTHON][CONNECT][SQL] Default to bytes for BinaryType in PySpark [spark] - posted by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2025/10/15 03:54:09 UTC, 10 replies.
- Re: [PR] [SPARK-53789][SQL][CONNECT] Canonicalize error condition CANNOT_MODIFY_STATIC_CONFIG [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/15 04:33:00 UTC, 0 replies.
- [PR] [SPARK-53914][BUILD][CONNECT] Add connect-jdbc module [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/15 06:09:21 UTC, 12 replies.
- [PR] [WIP][SPARK-53915] Add RealTimeScanExec and ability to execute long running batches [spark] - posted by "jerrypeng (via GitHub)" <gi...@apache.org> on 2025/10/15 06:45:07 UTC, 0 replies.
- Re: [PR] [SPARK-53111][SQL][PYTHON][CONNECT] Implement the time_diff function in PySpark [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/15 06:50:58 UTC, 2 replies.
- Re: [PR] [SPARK-51692][SQL]Fix bug while replace cte ref which contains dynamicpruning [spark] - posted by "zizon (via GitHub)" <gi...@apache.org> on 2025/10/15 10:36:44 UTC, 0 replies.
- [PR] [SPARK-53916][PYTHON] Deduplicate the variables in PythonArrowInput [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/15 11:10:39 UTC, 4 replies.
- [PR] [SPARK-53919][BUILD] Make Maven plugins up-to-date [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/15 15:50:19 UTC, 7 replies.
- [PR] [SPARK-53920][Geo][SQL] Introduce GeometryType and GeographyType to Java API [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/15 17:21:02 UTC, 2 replies.
- [PR] [SPARK-53923][CORE] Rename `spark.executor.(log -> logs).redirectConsoleOutputs` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/15 17:48:58 UTC, 5 replies.
- [PR] [SPARK-53925][INFRA] Use `MacOS 26` in `build_maven_java21_macos15.yml` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/15 19:31:41 UTC, 5 replies.
- [PR] [SPARK-53926][DOCS] Document newly added `core` module configurations [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/15 20:46:45 UTC, 6 replies.
- [PR] Initial commit [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/15 21:50:30 UTC, 0 replies.
- [PR] [SPARK-53928][SQL] Enhance DSV2 partition filtering using catalyst expression [spark] - posted by "szehon-ho (via GitHub)" <gi...@apache.org> on 2025/10/15 22:43:17 UTC, 4 replies.
- [PR] [SPARK-53922][Geo][SQL] Introduce physical Geometry and Geography types [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/15 23:36:07 UTC, 9 replies.
- [PR] [SPARK-53927][KINESIS] Upgrade kinesis client and fix kinesis integration tests ((ENABLE_KINESIS_TESTS=1) [spark] - posted by "vrozov (via GitHub)" <gi...@apache.org> on 2025/10/15 23:50:21 UTC, 6 replies.
- [PR] [SPARK-53929][SQL] Support TIME in the make_timestamp and try_make_timestamp functions in Scala [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/16 00:11:50 UTC, 2 replies.
- [PR] [SPARK-53760][Geo][SQL][FOLLOWUP] Fix error message and comment for Spatial types [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/16 00:18:54 UTC, 2 replies.
- Re: [PR] [SPARK-52545][SQL][DOCS] Update string literal docs for quote escaping rules [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/16 00:27:38 UTC, 2 replies.
- Re: [PR] [SPARK-53921][Geo][PYTHON] Introduce GeometryType and GeographyType to PySpark API [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/16 00:59:58 UTC, 8 replies.
- Re: [PR] [SPARK-53880][SQL] Fix DSv2 in PushVariantIntoScan by adding SupportsPushDownVariants [spark] - posted by "viirya (via GitHub)" <gi...@apache.org> on 2025/10/16 01:00:35 UTC, 10 replies.
- Re: [PR] [SPARK-53927][KINESIS] Upgrade kinesis client and fix kinesis integration tests (ENABLE_KINESIS_TESTS=1) [spark] - posted by "vrozov (via GitHub)" <gi...@apache.org> on 2025/10/16 02:18:29 UTC, 7 replies.
- [PR] [SPARK-53931][INFRA][PYTHON] Fix scheduled job for numpy 2.1.3 [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/16 02:21:58 UTC, 8 replies.
- [PR] [SPARK-53932] Add `FAIR` schedule examples with Spark (Connect|Thrift) Servers [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/16 02:48:03 UTC, 0 replies.
- Re: [PR] Tiny doc wording clarity [spark] - posted by "hutch3232 (via GitHub)" <gi...@apache.org> on 2025/10/16 02:57:05 UTC, 0 replies.
- [PR] [SPARK-49644][SQL] Support drop multi-level partition V2 table with partial partition spec [spark] - posted by "Zouxxyy (via GitHub)" <gi...@apache.org> on 2025/10/16 03:13:15 UTC, 2 replies.
- [PR] [SPARK-53925][FOLLOW-UP][DOCS] Fix the link in readme [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/16 03:44:08 UTC, 0 replies.
- [PR] [SPARK-53933] Add `Apache Iceberg` example [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/16 03:57:14 UTC, 4 replies.
- Re: [PR] [SPARK-53932] Add `FAIR` schedule examples with `Spark (Connect|Thrift)` servers [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/16 04:04:29 UTC, 2 replies.
- Re: [PR] [SPARK-53925][FOLLOW-UP][DOCS] Fix the link in README [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/16 05:40:09 UTC, 0 replies.
- [PR] [SPARK-53935][BUILD] SBT assembly should handle META-INF correctly [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/16 06:15:28 UTC, 3 replies.
- Re: [PR] [SPARK-53914][BUILD][CONNECT] Add connect-client-jdbc module [spark] - posted by "peter-toth (via GitHub)" <gi...@apache.org> on 2025/10/16 08:19:31 UTC, 15 replies.
- [PR] [SPARK-53938][PYTHON][CONNECT] Fix decimal rescaling in createDataFrame [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/16 13:20:25 UTC, 4 replies.
- Re: [PR] [SPARK-53936][BUILD] Upgrade sbt-pom-reader from 2.4.0 to 2.5.0 [spark] - posted by "wangyum (via GitHub)" <gi...@apache.org> on 2025/10/16 14:17:24 UTC, 3 replies.
- Re: [PR] [SPARK-53927][KINESIS] Upgrade kinesis client and fix kinesis integration tests [spark] - posted by "vrozov (via GitHub)" <gi...@apache.org> on 2025/10/16 15:23:56 UTC, 1 replies.
- [PR] [SPARK-53573][SQL] Allow coalescing string literals everywhere [spark] - posted by "srielau (via GitHub)" <gi...@apache.org> on 2025/10/16 18:30:53 UTC, 54 replies.
- Re: [PR] [SPARK-53335][K8S] Optionally capture diagnostics for jobs on Kubernetes [spark] - posted by "sunchao (via GitHub)" <gi...@apache.org> on 2025/10/16 19:31:36 UTC, 30 replies.
- [PR] [SPARK-53939][PYTHON] Use batch.num_columns instead of len(batch.columns) [spark] - posted by "ueshin (via GitHub)" <gi...@apache.org> on 2025/10/16 21:19:06 UTC, 2 replies.
- Re: [PR] [SPARK-52544][SQL] Allow configuring Json datasource string length limit through SQLConf [spark] - posted by "ccage-simp (via GitHub)" <gi...@apache.org> on 2025/10/16 23:40:30 UTC, 2 replies.
- Re: [PR] [SPARK-52012][CORE][SQL] Restore IDE Index with type annotations [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/17 00:27:12 UTC, 1 replies.
- Re: [PR] [SPARK-48359][SQL] Built-in functions for Zstd compression and decompression [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/17 00:27:18 UTC, 1 replies.
- [PR] [SPARK-53149][CORE] Fix testing whether BeeLine process run in background [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/17 02:58:50 UTC, 3 replies.
- [PR] [SPARK-53940][SQL] Function version() should return full spark version instead of short version [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/17 03:45:24 UTC, 17 replies.
- Re: [PR] [SPARK-53116][CONNECT] Fix conf setter when client is built with jar containing SQLConf but not its engine counterparts [spark] - posted by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2025/10/17 04:13:25 UTC, 0 replies.
- Re: [PR] [SPARK-51728][SQL] Add SELECT EXCEPT Support [spark] - posted by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2025/10/17 04:14:50 UTC, 1 replies.
- Re: [PR] [WIP][SPARK-53938][PYTHON][CONNECT] Fix decimal rescaling in LocalDataToArrowConversion [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/17 04:36:11 UTC, 1 replies.
- [PR] [WIP][DO-NOT-MERGE][SPARK-XXXXX] Support AQE in stateless streaming workloads [spark] - posted by "HeartSaVioR (via GitHub)" <gi...@apache.org> on 2025/10/17 04:39:49 UTC, 0 replies.
- Re: [PR] [SPARK-53938][PYTHON][CONNECT] Fix decimal rescaling in LocalDataToArrowConversion [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/17 04:43:37 UTC, 7 replies.
- [PR] [WIP][SPARK-53755][CORE] Add log support in BlockManager [spark] - posted by "ivoson (via GitHub)" <gi...@apache.org> on 2025/10/17 05:07:27 UTC, 0 replies.
- [PR] [MINOR][PYTHON][DOCS] Fix the docstring of `assert_true` [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/17 05:46:16 UTC, 2 replies.
- Re: [PR] [SPARK-53927][BUILD][DSTREAM] Upgrade kinesis client and fix kinesis integration tests [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/17 05:46:17 UTC, 7 replies.
- [PR] [WIP][DO-NOT-MERGE][SPARK-XXXXX] Support changing stateless shuffle partitions upon restart of streaming query [spark] - posted by "HeartSaVioR (via GitHub)" <gi...@apache.org> on 2025/10/17 06:01:04 UTC, 0 replies.
- Re: [PR] [WIP][DO-NOT-MERGE][SPARK-53941] Support AQE in stateless streaming workloads [spark] - posted by "HeartSaVioR (via GitHub)" <gi...@apache.org> on 2025/10/17 07:31:28 UTC, 0 replies.
- Re: [PR] [WIP][DO-NOT-MERGE][SPARK-53942] Support changing stateless shuffle partitions upon restart of streaming query [spark] - posted by "HeartSaVioR (via GitHub)" <gi...@apache.org> on 2025/10/17 07:31:35 UTC, 0 replies.
- [PR] [SPARK-53446][CORE] Optimize BlockManager remove operations with cached block mappings [spark] - posted by "zml1206 (via GitHub)" <gi...@apache.org> on 2025/10/17 08:22:11 UTC, 2 replies.
- [PR] [SPARK-53943][PYTHON][DOCS] Add examples for function unwrap_udt [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/17 08:35:56 UTC, 2 replies.
- Re: [PR] [SPARK-53755][CORE] Add log support in BlockManager [spark] - posted by "ivoson (via GitHub)" <gi...@apache.org> on 2025/10/17 09:41:14 UTC, 2 replies.
- Re: [PR] [WIP][SPARK-53917][CONNECT] Support large local relations [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/17 09:41:59 UTC, 2 replies.
- Re: [PR] [SPARK-53941][SS] Support AQE in stateless streaming workloads [spark] - posted by "HeartSaVioR (via GitHub)" <gi...@apache.org> on 2025/10/17 10:55:38 UTC, 32 replies.
- Re: [PR] [SPARK-53942] Support changing stateless shuffle partitions upon restart of streaming query [spark] - posted by "HeartSaVioR (via GitHub)" <gi...@apache.org> on 2025/10/17 11:44:42 UTC, 0 replies.
- [PR] [SPARK-53930][PYTHON] Support TIME in the make_timestamp and try_make_timestamp functions in PySpark [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/17 12:38:26 UTC, 0 replies.
- [PR] [SPARK-38117][METRICS] Expose stage level metrics in prometheus endpoint [spark] - posted by "azsefi (via GitHub)" <gi...@apache.org> on 2025/10/17 15:27:34 UTC, 0 replies.
- Re: [PR] [SPARK-53659][SQL] Infer Variant shredding schema when writing to Parquet [spark] - posted by "cashmand (via GitHub)" <gi...@apache.org> on 2025/10/17 17:36:59 UTC, 7 replies.
- [PR] [SPARK-53944][K8S] Support `spark.kubernetes.executor.useDriverPodIP` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/17 18:53:06 UTC, 4 replies.
- [PR] [WIP] [SPARK-XXXXX][SQL] Optimize memory usage in session cloning with ref-counted cached local relations [spark] - posted by "pranavdev022 (via GitHub)" <gi...@apache.org> on 2025/10/17 20:57:52 UTC, 0 replies.
- Re: [PR] [SPARK-53942][SS] Support changing stateless shuffle partitions upon restart of streaming query [spark] - posted by "HeartSaVioR (via GitHub)" <gi...@apache.org> on 2025/10/17 21:23:37 UTC, 8 replies.
- [PR] [SPARK-53945][BUILD] Upgrade `semanticdb-shared` to `4.13.10` [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/17 23:25:41 UTC, 2 replies.
- [PR] [SPARK-53946][BUILD] Upgrade SBT to 1.11.7 [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/17 23:46:44 UTC, 8 replies.
- Re: [PR] [WIP][SPARK-51224][BUILD] Test Maven 4 [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/18 00:25:35 UTC, 1 replies.
- Re: [PR] [SPARK-51883][DOCS][PYTHON] Python Data Source user guide for filter pushdown [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/18 00:25:39 UTC, 1 replies.
- [PR] [SPARK-53947][SQL]Count null in approx_top_k [spark] - posted by "yhuang-db (via GitHub)" <gi...@apache.org> on 2025/10/18 22:05:44 UTC, 0 replies.
- Re: [PR] [SPARK-52495][SQL] Allow including partition columns in the single variant column [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/19 00:30:57 UTC, 1 replies.
- [PR] Adds comprehensive top-level documentation to the repository [spark] - posted by "GizzZmo (via GitHub)" <gi...@apache.org> on 2025/10/19 17:39:10 UTC, 3 replies.
- [PR] [SPARK-XXXXX][SQL][3.5] Fix deadlock in Observation [spark] - posted by "viirya (via GitHub)" <gi...@apache.org> on 2025/10/19 22:14:58 UTC, 0 replies.
- Re: [PR] [SPARK-51489][SQL]Represent SQL scripts in Spark UI [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/20 00:58:45 UTC, 3 replies.
- [PR] [SPARK-53949][CONNECT] Use `Utils. getRootCause` instead of `Throwables.getRootCause` [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/20 06:03:40 UTC, 0 replies.
- [PR] [SPARK-53950][BUILD] Upgrade scala-xml to 2.4.0 [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/20 06:07:57 UTC, 3 replies.
- [PR] [SPARK-53951][BUILD] Upgrade `protobuf-java` to 4.33.0 [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/20 06:11:31 UTC, 2 replies.
- [PR] [SPARK-53952][INFRA] Balance GHA CI jobs [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/20 07:27:31 UTC, 1 replies.
- Re: [PR] [SPARK-53949][CONNECT] Use `Utils.getRootCause` instead of `Throwables.getRootCause` [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/20 07:37:07 UTC, 3 replies.
- [PR] [SPARK-42857][PYTHON][TESTS] Enable parity test `test_supported_types` [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/20 08:51:59 UTC, 2 replies.
- Re: [PR] [SPARK-53289][SQL] Invoke cancelJobsWithTag before start passing the SparkListenerSQLExecutionEnd event to prevent execution of canceled tasks. [spark] - posted by "zhouyuan (via GitHub)" <gi...@apache.org> on 2025/10/20 08:58:29 UTC, 10 replies.
- [PR] [SPARK-53953][BUILD][3.5] Bump Avro 1.11.5 [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/20 09:13:42 UTC, 1 replies.
- [PR] [SPARK-53954][BUILD] Bump Avro 1.12.1 [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/20 09:17:20 UTC, 5 replies.
- [PR] [SPARK-53955][CORE] Prefer to detect Java Home from env JAVA_HOME on finding jmap [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/20 10:11:06 UTC, 7 replies.
- Re: [PR] [SPARK-53952][INFRA] Balance GHA CI jobs by shifting connect tests [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/20 10:35:18 UTC, 5 replies.
- [PR] [WIP][SPARK-53956][SQL][PYTHON Support TIME in the try_make_timestamp function in PySpark [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/20 12:11:11 UTC, 0 replies.
- Re: [PR] [DRAFT][PYTHON] Remove unused type ignores [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/20 12:29:31 UTC, 0 replies.
- [PR] [SPARK-51658][Geo][SQL] Support GEOGRAPHY and GEOMETRY in the SpatialReferenceSystemMapper [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/20 12:52:02 UTC, 0 replies.
- Re: [PR] [SPARK-53956][PYTHON] Support TIME in the try_make_timestamp function in PySpark [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/20 13:14:18 UTC, 10 replies.
- [PR] [SPARK-53958][BUILD] Simplify Jackson deps management by using BOM [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/20 14:28:22 UTC, 4 replies.
- [PR] [SPARK-52578][SQL] Add numSourceRows metric for MergeIntoExec [spark] - posted by "asl3 (via GitHub)" <gi...@apache.org> on 2025/10/20 16:40:51 UTC, 0 replies.
- Re: [PR] [SPARK-53948][SQL][3.5] Fix deadlock in Observation [spark] - posted by "gengliangwang (via GitHub)" <gi...@apache.org> on 2025/10/20 16:44:37 UTC, 2 replies.
- Re: [PR] [SPARK-53947][SQL] Count null in approx_top_k [spark] - posted by "gengliangwang (via GitHub)" <gi...@apache.org> on 2025/10/20 19:42:47 UTC, 5 replies.
- [PR] [SPARK-53959][CONNECT] Throw a client-side error when creating a dataframe from a pandas dataframe with an index but no data [spark] - posted by "khakhlyuk (via GitHub)" <gi...@apache.org> on 2025/10/20 19:57:13 UTC, 4 replies.
- Re: [PR] [SPARK-53666][DOCS] Add script to generate llms.txt file for Spark main website [spark] - posted by "allisonwang-db (via GitHub)" <gi...@apache.org> on 2025/10/20 21:04:58 UTC, 0 replies.
- [PR] [SPARK-53961][SQL][TESTS] Fix `FileStreamSinkSuite` flakiness by using `walkFileTree` instead of `walk` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/20 21:06:27 UTC, 4 replies.
- [PR] [SPARK-53962][BUILD] Upgrade ASM to 9.9 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/20 21:32:29 UTC, 2 replies.
- [PR] [SPARK-53960][SQL] Let approx_top_k_accumulate/combine/estimate handle NULLs [spark] - posted by "yhuang-db (via GitHub)" <gi...@apache.org> on 2025/10/20 23:06:59 UTC, 7 replies.
- Re: [PR] [SPARK-53930][PYTHON] Support TIME in the make_timestamp function in PySpark [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/21 01:15:02 UTC, 5 replies.
- Re: [PR] [SPARK-53955][CORE][3.5] Prefer to detect Java Home from env JAVA_HOME on finding jmap [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/21 03:13:09 UTC, 3 replies.
- [PR] [SPARK-53963][PYTHON][TESTS] Drop temporary functions in regular UDF tests [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/21 03:19:17 UTC, 2 replies.
- [PR] [MINOR][PYTHON][TESTS] Update `test_arrow_udf_output_timestamps_ltz` to check the timezone conversion [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/21 03:26:36 UTC, 2 replies.
- [PR] [SPARK-53964][BUILD] Correct Java Home finding for SBT unidoc [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/21 04:28:22 UTC, 1 replies.
- [PR] [SPARK-53965][CONNECT][SS][PYTHON] Upgrade buf plugins to `v29.5` [spark] - posted by "LuciferYang (via GitHub)" <gi...@apache.org> on 2025/10/21 06:45:42 UTC, 2 replies.
- Re: [PR] [WIP][SPARK-53757] Upgrade Jetty to 12.0.12 [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/21 07:00:16 UTC, 2 replies.
- Re: [PR] [SPARK-28098][SQL]Support read hive table while LeafDir had multi-level paths [spark] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2025/10/21 07:04:30 UTC, 2 replies.
- [PR] [SPARK-53966][CORE] Reduce default page size by LONG_ARRAY_OFFSET if ZGC and ON_HEAP are used [spark] - posted by "wankunde (via GitHub)" <gi...@apache.org> on 2025/10/21 08:07:30 UTC, 5 replies.
- [PR] [DO-NOT-MERGE] Prototype: runtime profiling of Python workers [spark] - posted by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2025/10/21 08:24:10 UTC, 1 replies.
- [PR] [WIP][PYTHON] Avoid intermediate pandas dataframe creation in `df.toPandas` [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/21 10:31:33 UTC, 0 replies.
- [PR] [SPARK-53968][SQL] Store decimal precision loss conf in arithmetic expressions [spark] - posted by "stefankandic (via GitHub)" <gi...@apache.org> on 2025/10/21 12:10:19 UTC, 2 replies.
- [PR] [SPARK-53969][PYTHON][TESTS] Drop temporary functions in Arrow UDF tests [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/21 12:12:26 UTC, 1 replies.
- Re: [PR] [SPARK-33737][K8S] Support getting pod state using Informers + Listers [spark] - posted by "littlexyw (via GitHub)" <gi...@apache.org> on 2025/10/21 12:24:00 UTC, 0 replies.
- [PR] [SPARK-53694][SQL][TESTS][4.0] Improve `V1WriteHiveCommandSuite` test coverage [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/21 13:52:01 UTC, 8 replies.
- [PR] [SPARK-53971][BUILD] Bump zstd-jni 1.5.7-6 [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/21 14:48:18 UTC, 2 replies.
- Re: [PR] [SPARK-53964][BUILD] Simplify Java Home finding for SBT unidoc [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/21 15:30:37 UTC, 0 replies.
- [PR] add analysis context proto [spark] - posted by "cookiedough77 (via GitHub)" <gi...@apache.org> on 2025/10/21 17:37:09 UTC, 0 replies.
- [PR] [SPARK-53973][Avro] Classify errors for AvroOptions boolean casting failure [spark] - posted by "siying (via GitHub)" <gi...@apache.org> on 2025/10/21 17:46:22 UTC, 2 replies.
- [PR] [SPARK-53974][BUILD] Bump Jackson 2.20.0 [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/21 18:09:38 UTC, 5 replies.
- [PR] [SPARK-53972][SS] Fix streaming query recentProgress regression in classic pyspark [spark] - posted by "zifeif2 (via GitHub)" <gi...@apache.org> on 2025/10/21 19:19:27 UTC, 4 replies.
- Re: [PR] [SPARK-53927][BUILD][DSTREAM][4.0] Upgrade kinesis client and fix kinesis integration tests [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/21 20:19:58 UTC, 8 replies.
- Re: [PR] [SPARK-53955][CORE][3.5] Prefer to detect Java Home from env JAVA_HOME on finding jmap for JDK 8 [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/21 20:30:01 UTC, 3 replies.
- Re: [PR] [SPARK-46485][SQL] V1Write should not add Sort when not needed [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/21 21:32:55 UTC, 5 replies.
- [PR] [SPARK-53975][PYTHON] Adds basic logging support [spark] - posted by "ueshin (via GitHub)" <gi...@apache.org> on 2025/10/21 23:58:56 UTC, 0 replies.
- Re: [PR] [SPARK-52457][SQL]ParseToDate/ParseToTimestamp can return incorrect value for TimestampNTZ [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/22 00:28:42 UTC, 1 replies.
- Re: [PR] [SPARK-53975][PYTHON] Adds basic Python worker logging support [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/22 02:03:44 UTC, 13 replies.
- [PR] [SPARK-53979][PYTHON][TESTS] Drop temporary functions in Pandas UDF tests [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/22 03:39:06 UTC, 2 replies.
- [PR] [DRAFT] Huanliwang db/split test 2 [spark] - posted by "huanliwang-db (via GitHub)" <gi...@apache.org> on 2025/10/22 04:35:36 UTC, 0 replies.
- [PR] [SPARK-53980][CORE] Add more flexible getAllWithPrefix for SparkConf [spark] - posted by "beliefer (via GitHub)" <gi...@apache.org> on 2025/10/22 08:06:36 UTC, 6 replies.
- Re: [PR] [SPARK-46485][SQL][3.5] V1Write should not add Sort when not needed [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/22 08:18:56 UTC, 4 replies.
- [PR] [SPARK-28098][SQL]Support read hive table when location have multi-level non partitioned paths" [spark] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2025/10/22 09:35:19 UTC, 0 replies.
- Re: [PR] [SPARK-28098][SQL]Support read hive table when location have multi-level non partitioned paths [spark] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2025/10/22 09:38:53 UTC, 1 replies.
- [PR] [SPARK-53981][BUILD] Upgrade Netty to 4.2.7.Final [spark] - posted by "yaooqinn (via GitHub)" <gi...@apache.org> on 2025/10/22 09:55:45 UTC, 3 replies.
- [PR] Add optional sqlstate to sparkthrowables [spark] - posted by "garlandz-db (via GitHub)" <gi...@apache.org> on 2025/10/22 12:02:29 UTC, 0 replies.
- Re: [PR] [SPARK-53917][CONNECT] Support large local relations [spark] - posted by "asf-gitbox-commits (via GitHub)" <gi...@apache.org> on 2025/10/22 13:49:34 UTC, 0 replies.
- Re: [PR] [SC-208390][SDP] Add PipelineAnalysisContext message to support pipeline analysis during Spark Connect query execution [spark] - posted by "sryza (via GitHub)" <gi...@apache.org> on 2025/10/22 15:13:55 UTC, 3 replies.
- [PR] Update deps, fix some warnings [spark-connect-rust] - posted by "hntd187 (via GitHub)" <gi...@apache.org> on 2025/10/22 15:25:57 UTC, 0 replies.
- Re: [PR] [SPARK-53738][SQL][3.5] Fix planned write when query output contains foldable orderings [spark] - posted by "peter-toth (via GitHub)" <gi...@apache.org> on 2025/10/22 17:35:18 UTC, 1 replies.
- [PR] [SPARK-53983] Increase `s.k.o.reconciler.foregroundRequestTimeoutSeconds` to `60s` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 17:42:21 UTC, 3 replies.
- [PR] [SPARK-53984] Use CRD `v1` instead of `v1beta1` in `benchmark` script [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 17:45:23 UTC, 4 replies.
- [PR] [SPARK-53985] Suppress `StatusRecorder` warning messages [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 17:55:14 UTC, 3 replies.
- [PR] [SPARK-53987] Upgrade `JaCoCo` to 0.8.14 for official Java 25 support [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 18:39:18 UTC, 3 replies.
- [PR] [SPARK-53988] [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 18:55:59 UTC, 0 replies.
- [PR] [SPARK-53989] Optimize `sparkapps.sh` to use `kubectl delete --all` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 19:33:21 UTC, 2 replies.
- Re: [PR] [SPARK-53988] Upgrade `Spotless` to 8.0.0 [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 19:59:33 UTC, 1 replies.
- Re: [PR] Add PipelineAnalysisContext message to support pipeline analysis during Spark Connect query execution [spark] - posted by "SCHJonathan (via GitHub)" <gi...@apache.org> on 2025/10/22 20:26:10 UTC, 6 replies.
- [PR] [SPARK-53990] Use Java `(Map|Set).of` instead of `Collections.(empty|singleton)(Set|Map)` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 20:37:04 UTC, 0 replies.
- Re: [PR] [SPARK-53990] Use Java `(Map|Set).of` instead of `Collections.(empty|singleton)(Map|Set)` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 21:08:21 UTC, 1 replies.
- [PR] [SPARK-53992] Add `SparkOperatorConfManager.getAll` method [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 21:16:14 UTC, 3 replies.
- Re: [PR] [SPARK-52780] Add StreamRows and Arrow Record Streaming [spark-connect-go] - posted by "caldempsey (via GitHub)" <gi...@apache.org> on 2025/10/22 21:57:38 UTC, 2 replies.
- [PR] [SPARK-53993] Support `spark.logConf` configuration [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 22:01:04 UTC, 2 replies.
- [PR] [SPARK-53994] Exclude `KerberosConfDriverFeatureStep` during benchmarking [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/22 22:23:02 UTC, 3 replies.
- Re: [PR] [SPARK-52578][SQL] Add `numSourceRows` metric for `MergeIntoExec` [spark] - posted by "szehon-ho (via GitHub)" <gi...@apache.org> on 2025/10/22 23:21:03 UTC, 5 replies.
- Re: [PR] [SPARK-53986][PYTHON][SS][TESTS] Reorganize Python streaming TWS test [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/23 00:41:49 UTC, 2 replies.
- [PR] [SPARK-53996][SQL] Improve InferFiltersFromConstraints to infers from complex join expressions [spark] - posted by "vanekjar (via GitHub)" <gi...@apache.org> on 2025/10/23 01:14:50 UTC, 0 replies.
- Re: [PR] [SPARK-53967][PYTHON] Avoid intermediate pandas dataframe creation in `df.toPandas` [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/23 02:15:36 UTC, 0 replies.
- Re: [PR] [SPARK-52937][SDP] Sinks [spark] - posted by "sryza (via GitHub)" <gi...@apache.org> on 2025/10/23 03:11:50 UTC, 0 replies.
- [PR] [SPARK-53610][PYTHON] Limit Arrow batch sizes in CoGrouped applyInPandas and applyInArrow [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/23 03:30:13 UTC, 2 replies.
- [PR] [SPARK-47836][SQL] Use doubles sketch replace the GK algorithm for ap… [spark] - posted by "lifulong (via GitHub)" <gi...@apache.org> on 2025/10/23 03:39:17 UTC, 4 replies.
- [PR] [WIP][SPARK-53998] Add addition E2E tests for RTM [spark] - posted by "jerrypeng (via GitHub)" <gi...@apache.org> on 2025/10/23 04:58:01 UTC, 0 replies.
- [PR] [SPARK-53999][CORE] Native KQueue Transport support on BSD/MacOS [spark] - posted by "yaooqinn (via GitHub)" <gi...@apache.org> on 2025/10/23 07:32:52 UTC, 10 replies.
- [PR] [SPARK-50205][SQL][TEST] Re-enable `SparkSessionJobTaggingAndCancellationSuite.Cancellation APIs in SparkSession are isolated` [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/23 11:07:20 UTC, 1 replies.
- [PR] [SPARK-53934][CONNECT] Initial implement Connect JDBC driver [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/23 12:20:47 UTC, 7 replies.
- [PR] [SPARK-54002][DEPLOY] Support integrating BeeLine with Connect JDBC driver [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/23 12:36:50 UTC, 9 replies.
- Re: [PR] Add client env proto to spark connect client requests [spark] - posted by "garlandz-db (via GitHub)" <gi...@apache.org> on 2025/10/23 12:49:42 UTC, 0 replies.
- [PR] [SPARK-53914][BUILD][FOLLOWUP] Fix branch-4.0 daily maven test [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/23 13:17:00 UTC, 4 replies.
- Re: [PR] [SPARK-53966][CORE] Add utility functions to detect JVM GCs [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/23 16:12:47 UTC, 1 replies.
- [PR] [SPARK-53966][CORE][FOLLOWUP] Support `is(Serial|Parallel)GC` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/23 16:51:31 UTC, 5 replies.
- [PR] [SPARK-54005][CORE][TESTS] Remove no-op `spark.shuffle.blockTransferService` configuration [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/23 18:21:49 UTC, 2 replies.
- [PR] [MINOR][CORE] Remove invalid `NioBlockTransferService` comment [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/23 18:27:26 UTC, 2 replies.
- Re: [PR] [SPARK-52509][K8S] Cleanup individual shuffles from fallback storage on `RemoveShuffle` event [spark] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2025/10/23 18:39:22 UTC, 3 replies.
- [PR] [SPARK-54007][CORE][SQL] Use Java `Set.of` instead of `Collections.emptySet` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/23 19:09:41 UTC, 4 replies.
- Re: [PR] [SPARK-54001][SQL] Optimize memory usage in session cloning with ref-counted cached local relations [spark] - posted by "hvanhovell (via GitHub)" <gi...@apache.org> on 2025/10/23 19:29:23 UTC, 1 replies.
- [PR] [SPARK-54004] Fix uncaching table by name without cascading [spark] - posted by "aokolnychyi (via GitHub)" <gi...@apache.org> on 2025/10/23 19:51:10 UTC, 2 replies.
- Re: [PR] [SPARK-42360][SQL] Rule to convert Left Outer Join with suitable filter to Left Anti Join [spark] - posted by "ekoifman (via GitHub)" <gi...@apache.org> on 2025/10/23 19:52:31 UTC, 0 replies.
- Re: [PR] [SPARK-54004][SQL] Fix uncaching table by name without cascading [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/23 20:00:10 UTC, 11 replies.
- [PR] [SPARK-54008] Skip QO for DESCRIBE QUERY [spark] - posted by "gene-db (via GitHub)" <gi...@apache.org> on 2025/10/23 20:12:16 UTC, 3 replies.
- Re: [PR] [SPARK-53324][K8S] Introduce pending pod limit per ResourceProfile [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/23 20:21:38 UTC, 0 replies.
- [PR] [SPARK-53324][K8S][FOLLOWUP] Use invokePrivate to access `numOutstandingPods` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/23 20:57:01 UTC, 0 replies.
- Re: [PR] [SPARK-53324][K8S][FOLLOWUP] Use `invokePrivate` to access `numOutstandingPods` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/23 21:06:58 UTC, 4 replies.
- Re: [PR] [MINOR] Update configuration doc with appropriate prometheus sink class [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/23 21:27:16 UTC, 0 replies.
- [PR] [SPARK-54006][TESTS][DSTREAM][3.5] Fix for NoClassDefFoundError in WithAggregationKinesisBackedBlockRDDSuite [spark] - posted by "vrozov (via GitHub)" <gi...@apache.org> on 2025/10/23 21:28:48 UTC, 2 replies.
- [PR] [WIP][SPARK-53614] Add `applyInPandas` [spark] - posted by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2025/10/23 22:05:26 UTC, 0 replies.
- [PR] [SPARK-54009][CORE] Support `spark.io.mode.default` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/23 22:25:01 UTC, 5 replies.
- Re: [PR] [SPARK-53891][SQL] Model DSV2 Commit Write Summary API [spark] - posted by "szehon-ho (via GitHub)" <gi...@apache.org> on 2025/10/23 23:39:05 UTC, 3 replies.
- [PR] [SPARK-54010] Support restart counter in Spark app [spark-kubernetes-operator] - posted by "jiangzho (via GitHub)" <gi...@apache.org> on 2025/10/24 00:23:04 UTC, 1 replies.
- Re: [PR] [SPARK-52598][DOCS] Reorganize Spark Connect programming guide [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/24 00:24:50 UTC, 1 replies.
- [PR] test [spark] - posted by "vitaliili-db (via GitHub)" <gi...@apache.org> on 2025/10/24 01:12:59 UTC, 0 replies.
- [PR] [SPARK-28098][SQL] Supports reading hive tables when there are subdirectories under the table or partition location. [spark] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2025/10/24 01:44:46 UTC, 1 replies.
- [PR] [SPARK-54011][CORE] Switch direct ctors to IoHandlerFactories for EventLoopGroups [spark] - posted by "yaooqinn (via GitHub)" <gi...@apache.org> on 2025/10/24 03:06:50 UTC, 4 replies.
- Re: [PR] [SPARK-53915] Add RealTimeScanExec and ability to execute long running batches [spark] - posted by "viirya (via GitHub)" <gi...@apache.org> on 2025/10/24 06:50:38 UTC, 12 replies.
- [PR] [SPARK-54003][SQL] Use the staging directory as the output path then move to final path. [spark] - posted by "zhengchenyu (via GitHub)" <gi...@apache.org> on 2025/10/24 07:18:13 UTC, 0 replies.
- Re: [PR] [SPARK-52509][CORE] Cleanup individual shuffles from fallback storage on `RemoveShuffle` event [spark] - posted by "EnricoMi (via GitHub)" <gi...@apache.org> on 2025/10/24 09:20:45 UTC, 3 replies.
- [PR] [SPARK-54015][PYTHON] Relax Py4J requirement to py4j>=0.10.9.7,<0.10.9.10 [spark] - posted by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2025/10/24 12:56:43 UTC, 4 replies.
- Re: [PR] [SPARK-53980][CORE] Add `SparkConf.getAllWithPrefix(String, String => K)` API [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/24 15:00:06 UTC, 2 replies.
- Re: [PR] [SPARK-54006][TESTS][DSTREAM][3.5] Fix for NoClassDefFoundError in WithAggregationKinesisBackedBlockRDDSuite and KinesisTestUitls [spark] - posted by "vrozov (via GitHub)" <gi...@apache.org> on 2025/10/24 15:45:25 UTC, 3 replies.
- [PR] [SPARK-54018][K8S][INFRA][DOCS] Upgrade Volcano to 1.13.0 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/24 16:07:46 UTC, 2 replies.
- [PR] [SPARK-54021][Geo][SQL] Implement Geography and Geometry accessors across Catalyst [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/24 17:10:12 UTC, 2 replies.
- Re: [PR] [SPARK-53957][Geo][SQL] Support GEOGRAPHY and GEOMETRY in the SpatialReferenceSystemMapper [spark] - posted by "mkaravel (via GitHub)" <gi...@apache.org> on 2025/10/24 17:33:35 UTC, 13 replies.
- [PR] [SPARK-54023][CORE] Support `AUTO` Netty IO Mode [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/24 18:45:35 UTC, 1 replies.
- [PR] [SPARK-54024][BUILD] Add sbt-dependency-tree to SBT plugins [spark] - posted by "vrozov (via GitHub)" <gi...@apache.org> on 2025/10/24 20:03:05 UTC, 2 replies.
- [PR] [SPARK-52798][SQL][TESTS][FOLLOWUP] Fix a unit test failure due to the ANSI assumption [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/24 22:06:10 UTC, 5 replies.
- [PR] [MINOR][TESTS]skip two duplicate tests related to mapping BinaryType to bytes in PySpark [spark] - posted by "xianzhe-databricks (via GitHub)" <gi...@apache.org> on 2025/10/24 22:22:28 UTC, 2 replies.
- Re: [PR] [SPARK-53335][K8S] Support `spark.kubernetes.driver.annotateExitException` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/24 23:26:45 UTC, 2 replies.
- Re: [PR] [SPARK-54021][SQL] Implement Geography and Geometry accessors across Catalyst [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/24 23:29:43 UTC, 0 replies.
- [PR] [SPARK-54026][BUILD] Use BOM for AWS Java SDK V2 dependency management [spark] - posted by "vrozov (via GitHub)" <gi...@apache.org> on 2025/10/25 00:06:52 UTC, 8 replies.
- [PR] [WIP] [SPARK-54027] Kafka Source RTM support [spark] - posted by "jerrypeng (via GitHub)" <gi...@apache.org> on 2025/10/25 00:15:59 UTC, 0 replies.
- [PR] SPARK-54028: Use empty schema when altering a view which is not Hive compatible [spark] - posted by "cravani (via GitHub)" <gi...@apache.org> on 2025/10/25 12:57:24 UTC, 0 replies.
- [PR] [SPARK-54029][SQL] Add detailed error message for table metadata corruption to ease debugging [spark] - posted by "ganeshashree (via GitHub)" <gi...@apache.org> on 2025/10/25 15:04:23 UTC, 10 replies.
- [PR] [SPARK-54030][SQL] Add user-friendly assertion message for view metadata corruption [spark] - posted by "ganeshashree (via GitHub)" <gi...@apache.org> on 2025/10/25 16:16:28 UTC, 4 replies.
- [PR] [SQL][SPARK-54031] Add new golden file tests for analysis edge-cases [spark] - posted by "vladimirg-db (via GitHub)" <gi...@apache.org> on 2025/10/25 21:23:09 UTC, 6 replies.
- Re: [PR] [SPARK-54023][CORE] Support `AUTO` IO Mode [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/25 22:16:40 UTC, 6 replies.
- Re: [PR] [SPARK-52790][CORE] Introduce new grid testing method in SparkFunSuite [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/26 00:29:56 UTC, 1 replies.
- Re: [PR] [SPARK-53002][SS] Show accurate memory usage statistics when bounded memory is enabled for RocksDB [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/26 01:02:41 UTC, 0 replies.
- Re: [PR] Enable -Xsource:3 compiler flag [spark] - posted by "Noobzik (via GitHub)" <gi...@apache.org> on 2025/10/26 12:52:26 UTC, 0 replies.
- [PR] [SDP] Allow full-refresh-all with dp.create_sink (and no flows) [spark] - posted by "jaceklaskowski (via GitHub)" <gi...@apache.org> on 2025/10/26 16:08:03 UTC, 1 replies.
- [PR] [SPARK-54032][CORE] Prefer to use native Netty transports by default [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/26 20:08:40 UTC, 4 replies.
- [PR] [SPARK-54033][Geo][SQL] Introduce Catalyst server-side geospatial execution classes [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/26 21:09:40 UTC, 3 replies.
- [PR] [SPARK-54034][CORE] Fix `Utils.isBindCollision` to detect port conflict NativeIoException correctly [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/26 22:31:34 UTC, 0 replies.
- Re: [PR] [SPARK-54034][CORE] Fix `Utils.isBindCollision` to detect port conflict `NativeIoException` correctly [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/26 22:40:17 UTC, 5 replies.
- Re: [PR] [SPARK-52865][SQL]Remove usage of deprecated FileCommitProtocol.newTaskTempFile method [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/27 00:30:27 UTC, 1 replies.
- Re: [PR] Keep coverage data when running pip tests [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/27 00:30:28 UTC, 1 replies.
- Re: [PR] [SPARK-52864] [CORE] [Tests] Let LocalSparkContext clear active context in beforeAll [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/27 00:30:29 UTC, 1 replies.
- Re: [PR] [SPARK-52439][SQL] Support creating check constraint with NULL [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/27 00:30:30 UTC, 1 replies.
- [PR] [MINOR][DOCS] Fix MacOS 26 CI link in README.md [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/27 01:08:15 UTC, 2 replies.
- [PR] [SPARK-54036][INFRA] Add `build_python_3.11_macos26.yml` GitHub Action job [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/27 03:00:58 UTC, 3 replies.
- [PR] [SPARK-54013][CONNECT] Implement SparkConnectDatabaseMetaData simple methods [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/27 03:36:12 UTC, 24 replies.
- [PR] [SPARK-54014][CONNECT] Support max rows for SparkConnectStatement [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/27 10:07:59 UTC, 5 replies.
- [PR] [SPARK-54040][BUILD] Remove unused commons-collections 3.x [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/27 14:55:29 UTC, 1 replies.
- [PR] [SPARK-54042] Use 4.1.0-preview3 in `integration-test-(token|mac-spark41)` [spark-connect-swift] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/27 16:12:35 UTC, 3 replies.
- [PR] [SPARK-54041] [SQL] Refactor ParameterizedQuery arguments validation [spark] - posted by "mihailoale-db (via GitHub)" <gi...@apache.org> on 2025/10/27 16:27:06 UTC, 7 replies.
- [PR] [SPARK-54039] [SS] Add TaskContext logging to KafkaDataConsumer [spark] - posted by "Nishanth28 (via GitHub)" <gi...@apache.org> on 2025/10/27 16:29:17 UTC, 2 replies.
- [PR] [SPARK-52510][Geo][CONNECT][FOLLOWUP] Reserve expression proto numbers for Geometry and Geography [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/27 19:31:22 UTC, 3 replies.
- [PR] [SPARK-54043] Update `Spark Connect`-generated `Swift` source code with `4.1.0-preview3` RC1 [spark-connect-swift] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/27 20:07:22 UTC, 5 replies.
- Re: [PR] [SPARK-54024][BUILD] Add sbt-dependency-tree to SBT plugin [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/27 21:33:48 UTC, 2 replies.
- [PR] [SPARK-54044] Upgrade `gRPC Swift NIO Transport` to 2.2.0 [spark-connect-swift] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/27 21:58:42 UTC, 2 replies.
- [PR] Release buffers [spark] - posted by "viirya (via GitHub)" <gi...@apache.org> on 2025/10/27 22:08:59 UTC, 2 replies.
- [PR] [SPARK-54045] Upgrade `FlatBuffers` to v25.9.23 [spark-connect-swift] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/27 22:25:37 UTC, 4 replies.
- [PR] [SPARK-54046][INFRA] Upgrade PyArrow to 22.0.0 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/27 22:46:00 UTC, 8 replies.
- [PR] [SPARK-54047][PYTHON] Use a difference error message when kill-on-idle-timeout [spark] - posted by "ueshin (via GitHub)" <gi...@apache.org> on 2025/10/27 23:37:20 UTC, 2 replies.
- [PR] [SPARK-54048][PYTHON] Update `dev/requirements.txt` to install `torch(vision)` in Python 3.14 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/27 23:56:17 UTC, 2 replies.
- Re: [PR] added the file in readme [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/28 00:26:55 UTC, 1 replies.
- [PR] [SPARK-53835][INFRA] Install `pyarrow/torch/torchvision` packages to `Python 3.14` Dockefile [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/28 01:56:51 UTC, 4 replies.
- [PR] [SPARK-53941][FOLLOWUP][SS] Introduce a SQL config to control AQE for stateless streaming workload specifically [spark] - posted by "HeartSaVioR (via GitHub)" <gi...@apache.org> on 2025/10/28 02:30:53 UTC, 4 replies.
- [PR] [SPARK-54050][PYTHON][DOCS] Update the documents of arrow-batching related configures [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/28 03:03:30 UTC, 2 replies.
- [PR] [SPARK-53966][CORE] Reduce default page size by LONG_ARRAY_OFFSET if ZGC or ShenandoahGC and ON_HEAP are used [spark] - posted by "wankunde (via GitHub)" <gi...@apache.org> on 2025/10/28 03:16:42 UTC, 3 replies.
- Re: [PR] [SPARK-53614] Add `Iterator[pandas.DataFrame]` support to `applyInPandas` [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/28 03:26:14 UTC, 5 replies.
- [PR] [SPARK-54052][PYTHON] Add SparkThrowable wrapper to workaround Py4J limitation [spark] - posted by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2025/10/28 03:44:33 UTC, 2 replies.
- [PR] [SPARK-54051] Keep coverage data when running pip tests [spark] - posted by "gaogaotiantian (via GitHub)" <gi...@apache.org> on 2025/10/28 03:52:01 UTC, 0 replies.
- [PR] [SPARK-54053] Use Swift 6.2 for all Linux CIs [spark-connect-swift] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/28 05:02:12 UTC, 4 replies.
- [PR] [SPARK-54054][CONNECT] Support row position for SparkConnectResultSet [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/28 05:36:48 UTC, 7 replies.
- [PR] [SPARK-54038][CONNECT] Support getSQLKeywords for SparkConnectDatabaseMetaData [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/28 05:55:11 UTC, 10 replies.
- Re: [PR] [SPARK-54039] [SS] Add TaskContext information to KafkaDataConsumer release() logs for better debugging [spark] - posted by "Nishanth28 (via GitHub)" <gi...@apache.org> on 2025/10/28 08:55:01 UTC, 6 replies.
- [PR] [SPARK-53535] Fix findCheapestGroupField returning invalid Map [spark] - posted by "ZiyaZa (via GitHub)" <gi...@apache.org> on 2025/10/28 09:03:49 UTC, 0 replies.
- [PR] resolve substitution for SQLConf settings in Catalogs [spark] - posted by "EugeneYushin (via GitHub)" <gi...@apache.org> on 2025/10/28 10:03:06 UTC, 0 replies.
- Re: [PR] [SPARK-53535][SQL][FOLLOWUP] Fix findCheapestGroupField returning invalid Map [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/28 11:05:48 UTC, 1 replies.
- [PR] [SPARK-54057][BUILD] Retrieve dependency version from `pom.xml` for `DependencyOverrides` in `SparkBuild.scala` [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/28 12:14:37 UTC, 4 replies.
- Re: [PR] [SPARK-54059][CORE] Reduce default page size by LONG_ARRAY_OFFSET if ZGC or ShenandoahGC and ON_HEAP are used [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/28 15:39:35 UTC, 3 replies.
- [PR] [SPARK-51658][Geo][SQL] Introduce Geometry and Geography in-memory wrapper formats [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/28 16:51:58 UTC, 3 replies.
- [PR] [SPARK-54061] Wrap IllegalArgumentException with proper error code for invalid datetime patterns [spark] - posted by "miland-db (via GitHub)" <gi...@apache.org> on 2025/10/28 17:29:06 UTC, 14 replies.
- [PR] [SPARK-54062][SQL] MergeScalarSubqueries code cleanup [spark] - posted by "peter-toth (via GitHub)" <gi...@apache.org> on 2025/10/28 17:52:29 UTC, 3 replies.
- Re: [PR] [SPARK-54010] Support `applicationTolerations.restartConfig.restartCounterResetMillis` [spark-kubernetes-operator] - posted by "peter-toth (via GitHub)" <gi...@apache.org> on 2025/10/28 18:20:33 UTC, 3 replies.
- [PR] [SPARK-54022][SQL] Make DSv2 table resolution aware of cached tables [spark] - posted by "aokolnychyi (via GitHub)" <gi...@apache.org> on 2025/10/28 18:23:26 UTC, 5 replies.
- [PR] [DRAFT] [SPARK-53573][SQL] IDENTIFIER everywhere [spark] - posted by "srielau (via GitHub)" <gi...@apache.org> on 2025/10/28 18:24:17 UTC, 0 replies.
- Re: [PR] [SPARK-54052][PYTHON] Add a bridge object to workaround Py4J limitation [spark] - posted by "ueshin (via GitHub)" <gi...@apache.org> on 2025/10/28 18:32:56 UTC, 4 replies.
- [PR] [SPARK-54058][SS][TESTS] Increase `kafka.metadata.max.age.ms` from `1` to `500` to make a test in `KafkaMicroBatchV1SourceWithConsumerSuite` stable [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/28 18:39:06 UTC, 1 replies.
- [PR] [SPARK-54049][BUILD] Spark-network-common no longer shades all of Guava [spark] - posted by "vinodkc (via GitHub)" <gi...@apache.org> on 2025/10/28 18:40:04 UTC, 10 replies.
- [PR] [SPARK-54064][PYTHON] Simplify recursion detection of `cloudpickle` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/28 18:59:07 UTC, 7 replies.
- Re: [PR] [SPARK-54051][INFRA][PYTHON] Keep coverage data when running pip tests [spark] - posted by "gaogaotiantian (via GitHub)" <gi...@apache.org> on 2025/10/28 19:12:56 UTC, 3 replies.
- [PR] [SPARK-54066][PYTHON][TESTS] Skip `test_in_memory_data_source` in Python 3.14 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/28 19:55:37 UTC, 4 replies.
- [PR] [SPARK-54067][SDP] Avoid long "User application exited with 1" errors & traces on spark-pipelines errors [spark] - posted by "sryza (via GitHub)" <gi...@apache.org> on 2025/10/28 20:36:45 UTC, 7 replies.
- [PR] [SPARK-54069][PYTHON][TESTS] Skip `test_to_feather` in Python 3.14 [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/28 20:57:37 UTC, 3 replies.
- [PR] [SPARK-53921][Geo][PYTHON][FOLLOWUP] Fix invalid algorithm error for Geography JSON parsing [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/28 20:59:18 UTC, 2 replies.
- Re: [PR] [SPARK-53482][SQL] MERGE INTO support for when source has less nested field than target [spark] - posted by "szehon-ho (via GitHub)" <gi...@apache.org> on 2025/10/28 21:35:24 UTC, 10 replies.
- [PR] [SPARK-53726]Add `spark.logConf` configuration to operator docs [spark-kubernetes-operator] - posted by "jiangzho (via GitHub)" <gi...@apache.org> on 2025/10/28 22:03:22 UTC, 1 replies.
- Re: [PR] [SPARK-54047][PYTHON] Use a difference error when kill-on-idle-timeout [spark] - posted by "ueshin (via GitHub)" <gi...@apache.org> on 2025/10/28 23:48:06 UTC, 1 replies.
- [PR] [SPARK-54063] Trigger snapshot for next batch when upload lag [spark] - posted by "zifeif2 (via GitHub)" <gi...@apache.org> on 2025/10/29 02:34:30 UTC, 2 replies.
- [PR] [SPARK-54072][SS] Ensure that we don't upload empty files in RocksDB snapshot [spark] - posted by "micheal-o (via GitHub)" <gi...@apache.org> on 2025/10/29 02:47:50 UTC, 1 replies.
- [PR] [SPARK-53610][PYTHON][TESTS][FOLLOW-UP] Add tests for negative maxRecordsPerBatch [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/29 03:04:32 UTC, 2 replies.
- Re: [PR] [SPARK-54070] Add `spark.logConf` configuration to operator docs [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/29 03:33:09 UTC, 3 replies.
- [PR] [SPARK-54073] Improve `ConfOptionDocGenerator` to generate a sorted config doc by key [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/29 04:41:42 UTC, 1 replies.
- Re: [PR] [SPARK-54073] Improve `ConfOptionDocGenerator` to generate a sorted doc by config key [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/29 06:00:54 UTC, 2 replies.
- [PR] [SPARK-51756][CORE][FOLLOWUP] Avoid the risk of overflow of long [spark] - posted by "beliefer (via GitHub)" <gi...@apache.org> on 2025/10/29 10:23:40 UTC, 10 replies.
- [PR] [SPARK-54074][SS][TESTS] Increase `kafka.metadata.max.age.ms` for an unavailable offset relevant test in `KafkaMicroBatchV1SourceWithConsumerSuite` [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/29 10:46:59 UTC, 6 replies.
- Re: [PR] [SPARK-45956][BUILD] Upgrade `Apache ZooKeeper` to 3.9.1 [spark] - posted by "Dzeri96 (via GitHub)" <gi...@apache.org> on 2025/10/29 13:13:02 UTC, 2 replies.
- Re: [PR] [SPARK-51069][SQL] Add big-endian support to UnsafeRowUtils.validateStructuralIntegrityWithReasonImpl [spark] - posted by "jonathan-albrecht-ibm (via GitHub)" <gi...@apache.org> on 2025/10/29 13:17:14 UTC, 0 replies.
- [PR] Add OpenLineage to Spark 3.5.4 [spark] - posted by "ysung (via GitHub)" <gi...@apache.org> on 2025/10/29 14:34:36 UTC, 1 replies.
- [PR] [SPARK-54075] [SQL] Make `ResolvedCollation` evaluable [spark] - posted by "mihailoale-db (via GitHub)" <gi...@apache.org> on 2025/10/29 15:16:57 UTC, 2 replies.
- [PR] [SPARK-54076][DOCKER][TESTS] Fix Flaky Test:`OracleJoinPushdownIntegrationSuite` [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/29 15:22:35 UTC, 1 replies.
- [PR] [SPARK-53905][SQL] Refactor RelationResolution to enable code reuse [spark] - posted by "aokolnychyi (via GitHub)" <gi...@apache.org> on 2025/10/29 15:28:34 UTC, 6 replies.
- Re: [PR] [SPARK-54074][SS][TESTS] Remove `kafka.metadata.max.age.ms` from an unavailable offset relevant test in `KafkaMicroBatchV1SourceWithConsumerSuite` [spark] - posted by "sarutak (via GitHub)" <gi...@apache.org> on 2025/10/29 16:24:05 UTC, 2 replies.
- Re: [PR] [SPARK-54060][Geo][SQL] Introduce Geometry and Geography in-memory wrapper formats [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/29 16:30:47 UTC, 2 replies.
- [PR] [SPARK-54077] Enable all disabled tests on Linux [spark-connect-swift] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/29 16:37:54 UTC, 4 replies.
- [PR] Avoid traversing nested folders if recursiveFileLookup = false [spark] - posted by "mzhang (via GitHub)" <gi...@apache.org> on 2025/10/29 16:41:44 UTC, 1 replies.
- [PR] [SPARK-54078] Deflake StateStoreSuite `SPARK-40492: maintenance before unload` [spark] - posted by "liviazhu (via GitHub)" <gi...@apache.org> on 2025/10/29 16:51:27 UTC, 0 replies.
- Re: [PR] [SPARK-54076][SQL][TESTS] Use `sarutak/oracle-free` image to use `PASSWORD_INIT_TIMEOUT` for `OracleDatabaseOnDocker` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/29 17:17:14 UTC, 4 replies.
- [PR] [SPARK-54079][Geo][SQL] Introduce the framework for adding ST expressions in Catalyst [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/29 17:53:41 UTC, 4 replies.
- [PR] [SPARK-54080] Use Swift 6.2 as the minimum supported version [spark-connect-swift] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/29 18:05:03 UTC, 2 replies.
- [PR] [SPARK-53976][PYTHON] Support logging in Pandas/Arrow UDFs [spark] - posted by "ueshin (via GitHub)" <gi...@apache.org> on 2025/10/29 21:11:28 UTC, 6 replies.
- [PR] [SPARK-52134][SQL][FOLLOW UP] SQL Script execution code - follow-up refactor [spark] - posted by "davidm-db (via GitHub)" <gi...@apache.org> on 2025/10/29 21:21:33 UTC, 0 replies.
- Re: [PR] [SPARK-54078][SS] Deflake StateStoreSuite `SPARK-40492: maintenance before unload` [spark] - posted by "anishshri-db (via GitHub)" <gi...@apache.org> on 2025/10/29 21:46:09 UTC, 1 replies.
- Re: [PR] [SPARK-52134][SQL][FOLLOW UP] SQL Script execution code - refactor follow up [spark] - posted by "davidm-db (via GitHub)" <gi...@apache.org> on 2025/10/29 23:27:43 UTC, 2 replies.
- [PR] [SPARK-54081] Add `word-count-preview.yaml` Example [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/29 23:46:33 UTC, 4 replies.
- Re: [PR] [SPARK-53614][PYTHON] Add `Iterator[pandas.DataFrame]` support to `applyInPandas` [spark] - posted by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2025/10/30 00:19:27 UTC, 12 replies.
- Re: [PR] [SPARK-52794][PYTHON] Performance improvement of arrow-optimized python UDTF with LocalDataToArrowConversion check [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/30 00:28:50 UTC, 1 replies.
- Re: [PR] [SPARK-52486][SQL] Fix Spark Driver Planning OOM issue due to unworthwhile dpp expression before Execution when enabling AQE [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/30 00:28:52 UTC, 1 replies.
- Re: [PR] [SPARK-51400] Replace ArrayContains nodes to InSet [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/30 00:28:53 UTC, 1 replies.
- Re: [PR] [SPARK-xxxxx][SQL] Release Arrow buffers early [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/30 02:15:59 UTC, 7 replies.
- Re: [PR] [DRAFT] JDBC Driver for Spark Connect Server [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/30 02:44:33 UTC, 1 replies.
- [PR] [SPARK-54082] Upgrade Spark to `4.1.0-preview3` [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/30 05:00:06 UTC, 3 replies.
- [PR] [SPARK-51972][SS] Introduce State Store file integrity verification using checksum [spark] - posted by "micheal-o (via GitHub)" <gi...@apache.org> on 2025/10/30 05:00:30 UTC, 4 replies.
- [PR] [SPARK-54083] Use `4.1.0-preview3` instead of `RC1` [spark-connect-swift] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/30 05:07:19 UTC, 5 replies.
- [PR] [SPARK-54084] Publish Apache Spark 4.1.0-preview3 to docker registry [spark-docker] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/30 05:23:04 UTC, 6 replies.
- [PR] [SPARK-XXXXX][SQL] Extract common methods to KafkaOffsetReaderBase [spark] - posted by "viirya (via GitHub)" <gi...@apache.org> on 2025/10/30 07:49:46 UTC, 1 replies.
- Re: [PR] [SPARK-54027] Kafka Source RTM support [spark] - posted by "viirya (via GitHub)" <gi...@apache.org> on 2025/10/30 08:02:34 UTC, 17 replies.
- [PR] [SPARK-54085] Fix job submission failure for standalone cluster [spark] - posted by "EricGao888 (via GitHub)" <gi...@apache.org> on 2025/10/30 08:18:46 UTC, 2 replies.
- [PR] [SPARK-53921][Geo][PYTHON][FOLLOWUP] Add GeographyType and GeometryType to `__all__` in types [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/30 08:29:36 UTC, 3 replies.
- [PR] [SPARK-54086][CORE][SHUFFLE] Support `IO_URING `Netty IO Mode [spark] - posted by "yaooqinn (via GitHub)" <gi...@apache.org> on 2025/10/30 08:49:06 UTC, 3 replies.
- [PR] [SPARK-54087][CORE] Spark Executor launch task failed should return task killed message [spark] - posted by "AngersZhuuuu (via GitHub)" <gi...@apache.org> on 2025/10/30 09:02:55 UTC, 0 replies.
- [PR] [SPARK-54088][CORE] When TaskSchedulerImpl searches for an executor to schedule an unschedulable task, it should exclude executors that are pending to remove [spark] - posted by "AngersZhuuuu (via GitHub)" <gi...@apache.org> on 2025/10/30 09:45:36 UTC, 2 replies.
- Re: [PR] [SPARK-38498][STREAM] Support customized StreamingListener by configuration [spark] - posted by "AngersZhuuuu (via GitHub)" <gi...@apache.org> on 2025/10/30 09:59:48 UTC, 5 replies.
- Re: [PR] [SPARK-54028][SQL] Use empty schema when altering a view which is not Hive compatible [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/30 11:12:07 UTC, 10 replies.
- [PR] [SPARK-53535][SQL][FOLLOWUP] Remove duplicated tests [spark] - posted by "ZiyaZa (via GitHub)" <gi...@apache.org> on 2025/10/30 11:52:44 UTC, 0 replies.
- [PR] [SPARK-54091][Geo][SQL] Implement the ST_Srid expression in SQL [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/30 13:20:56 UTC, 2 replies.
- Re: [PR] [SPARK-53535][SQL][TESTS][FOLLOWUP] Remove duplicated tests [spark] - posted by "cloud-fan (via GitHub)" <gi...@apache.org> on 2025/10/30 13:57:21 UTC, 1 replies.
- [PR] [SPARK-54092] Use Java-friendly `KubernetesClientUtils` APIs [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/30 15:25:35 UTC, 4 replies.
- Re: [PR] [SPARK-54085][CORE] Fix `initialize` to add `CREATE` option additionally in `DriverRunner` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/30 15:35:15 UTC, 3 replies.
- [PR] Grammar improvement in docstring [spark] - posted by "nebloc (via GitHub)" <gi...@apache.org> on 2025/10/30 15:57:47 UTC, 0 replies.
- Re: [PR] [MINOR][SQL] Extract common methods to KafkaOffsetReaderBase [spark] - posted by "viirya (via GitHub)" <gi...@apache.org> on 2025/10/30 16:31:13 UTC, 3 replies.
- Re: [PR] [SPARK-54067][CORE] Avoid long "User application exited with 1" errors & traces on spark-pipelines errors [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/30 17:37:04 UTC, 0 replies.
- Re: [PR] [SPARK-54067][CORE] Improve `SparkSubmit` to invoke `exitFn` with the root cause instead of `SparkUserAppException` [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/30 17:45:36 UTC, 1 replies.
- [PR] [SPARK-53891][SQL][FOLLOW-UP] Clarify javadocs for case when merge summary metric is not found [spark] - posted by "szehon-ho (via GitHub)" <gi...@apache.org> on 2025/10/30 18:19:58 UTC, 4 replies.
- [PR] [SPARK-53977][PYTHON] Support logging in UDTFs [spark] - posted by "ueshin (via GitHub)" <gi...@apache.org> on 2025/10/30 18:25:36 UTC, 3 replies.
- [PR] [SPARK-54096][Geo][PYTHON] Support Spatial Reference System mapping in PySpark [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/30 19:37:58 UTC, 4 replies.
- [PR] [SPARK-53991][SQL] Add SQL support for KLL quantiles functions based on DataSketches [spark] - posted by "dtenedor (via GitHub)" <gi...@apache.org> on 2025/10/30 19:47:07 UTC, 24 replies.
- Re: [PR] [SPARK-53996][SQL] Improve InferFiltersFromConstraints to infer filters from complex join expressions [spark] - posted by "andylam-db (via GitHub)" <gi...@apache.org> on 2025/10/30 20:21:07 UTC, 2 replies.
- Re: [PR] [SPARK-53809][SQL]Add canonicalization for DataSourceV2ScanRelation [spark] - posted by "yhuang-db (via GitHub)" <gi...@apache.org> on 2025/10/30 20:33:04 UTC, 4 replies.
- Re: [PR] [SPARK-54094][SQL] Extract common methods to KafkaOffsetReaderBase [spark] - posted by "viirya (via GitHub)" <gi...@apache.org> on 2025/10/30 21:04:21 UTC, 1 replies.
- [PR] [SPARK-54097] Upgrade `Gradle` to 9.2.0 [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/30 21:11:29 UTC, 3 replies.
- [PR] [SPARK-54098] Set `--sun-misc-unsafe-memory-access=allow` for `JavaExec` Gradle tasks [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/30 21:26:28 UTC, 1 replies.
- [PR] [SPARK-54099][SQL] XML variant parser should fall back to string on decimal parsing errors [spark] - posted by "xiaonanyang-db (via GitHub)" <gi...@apache.org> on 2025/10/30 21:45:48 UTC, 4 replies.
- [PR] [SPARK-54100][BUILD] Remove `ignore.symbol.file` Javac option [spark] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/30 22:10:28 UTC, 6 replies.
- [PR] [SPARK-54101][Geo][SQL] Introduce the framework for adding ST functions in Scala [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/30 22:53:40 UTC, 4 replies.
- [PR] [SPARK-54103][Geo][SQL] Introduce client-side Geography and Geometry classes [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/31 00:02:20 UTC, 6 replies.
- [PR] [SPARK-54033][Geo][SQL][FOLLOWUP] Fix Java test suite names for geospatial execution classes [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/31 00:08:03 UTC, 2 replies.
- Re: [PR] [SPARK-52767][SQL] Optimize the performance of maxRows for join and union [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/31 00:27:48 UTC, 0 replies.
- Re: [PR] [SPARK-51554][SQL] Add the time_trunc() function for TIME datatype [spark] - posted by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2025/10/31 00:27:52 UTC, 0 replies.
- [PR] [SPARK-54104][Geo][SQL] Disallow casting geospatial types to/from other data types [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/31 02:28:17 UTC, 8 replies.
- [PR] [SPARK-54079][Geo][SQL][FOLLOWUP] Fix catalyst expressions ST package and imports [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/31 02:35:09 UTC, 0 replies.
- [PR] [SPARK-53978][PYTHON] Support logging in driver-side workers [spark] - posted by "ueshin (via GitHub)" <gi...@apache.org> on 2025/10/31 04:08:38 UTC, 3 replies.
- Re: [PR] [SPARK-54056][SQL] resolve substitution for SQLConf settings in Catalogs [spark] - posted by "EugeneYushin (via GitHub)" <gi...@apache.org> on 2025/10/31 04:51:09 UTC, 3 replies.
- [PR] [SPARK-54105] Update `integration-test-ubuntu-spark41` to use Spark `4.1.0-preview3-java21` [spark-connect-swift] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/31 05:18:51 UTC, 2 replies.
- [PR] [SPARK-54106][SS] State store row checksum implementation [spark] - posted by "micheal-o (via GitHub)" <gi...@apache.org> on 2025/10/31 05:27:29 UTC, 0 replies.
- [PR] [SPARK-54107] Use `4.1.0-preview3-java21-scala` image for preview examples [spark-kubernetes-operator] - posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2025/10/31 05:42:09 UTC, 2 replies.
- [PR] [SPARK-54108][CONNECT] Revise execute* methods of SparkConnectStatement [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/31 07:07:09 UTC, 4 replies.
- [PR] [SPARK-54109][PYTHON][TESTS] Avoid function conflicts in test_pandas_grouped_map [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/31 07:49:21 UTC, 2 replies.
- [PR] [SPARK-53731][PYTHON] Update the type hints of iterator APIs [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/31 08:29:15 UTC, 0 replies.
- [PR] [SPARK-54110][Geo][SQL] Introduce type encoders for Geography and Geometry types [spark] - posted by "uros-db (via GitHub)" <gi...@apache.org> on 2025/10/31 08:38:01 UTC, 3 replies.
- [PR] [MINOR][PYTHON][DOCS] Fix the doctest of `pivot` [spark] - posted by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2025/10/31 09:22:10 UTC, 0 replies.
- [PR] [SPARK-54111][CONNECT] Support getCatalogs for SparkConnectDatabaseMetaData [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/31 09:34:26 UTC, 0 replies.
- [PR] [SPARK-54115][CORE][UI][CONNECT] Display connect server execution threads first in thread dump page [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/31 10:24:31 UTC, 2 replies.
- Re: [PR] [SPARK-54086][CORE][SHUFFLE] Support `IO_URING` Netty IO Mode [spark] - posted by "yaooqinn (via GitHub)" <gi...@apache.org> on 2025/10/31 10:28:09 UTC, 2 replies.
- [PR] [SPARK-54116][SQL] Add off-heap mode support for LongHashedRelation [spark] - posted by "zhztheplayer (via GitHub)" <gi...@apache.org> on 2025/10/31 12:24:49 UTC, 0 replies.
- [PR] [CONNECT][SQL] Convert IllegalArgumentException to SparkException with proper error classes in Spark Connect server [spark] - posted by "grundprinzip (via GitHub)" <gi...@apache.org> on 2025/10/31 15:39:47 UTC, 0 replies.
- [PR] [SPARK-54112][CONNECT] Support getSchemas for SparkConnectDatabaseMetaData [spark] - posted by "pan3793 (via GitHub)" <gi...@apache.org> on 2025/10/31 17:43:25 UTC, 1 replies.
- [PR] [SPARK-54118] Improve the put/merge operation in ListState when there are multiple values [spark] - posted by "huanliwang-db (via GitHub)" <gi...@apache.org> on 2025/10/31 20:29:28 UTC, 0 replies.
- [PR] [SPARK-54120] Update assertGeneratedCRDMatchesHelmChart to include diff [spark-kubernetes-operator] - posted by "jiangzho (via GitHub)" <gi...@apache.org> on 2025/10/31 21:36:22 UTC, 1 replies.
- Re: [PR] [SPARK-54118][SS] Improve the put/merge operation in ListState when there are multiple values [spark] - posted by "anishshri-db (via GitHub)" <gi...@apache.org> on 2025/10/31 22:08:26 UTC, 1 replies.
- [PR] [SPARK-54121][SS] Automatic Snapshot Repair for State store [spark] - posted by "micheal-o (via GitHub)" <gi...@apache.org> on 2025/10/31 23:01:55 UTC, 0 replies.
- [PR] [SPARK-54117] Throw better error to indicate that TWS is only supported with RocksDB state store provider [spark] - posted by "fedimser (via GitHub)" <gi...@apache.org> on 2025/10/31 23:11:35 UTC, 0 replies.