You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by da...@apache.org on 2025/10/06 14:19:22 UTC

(beam) branch users/damccorm/combinePerKey updated (846d9c068d4 -> 521b6d1c92c)

This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch users/damccorm/combinePerKey
in repository https://gitbox.apache.org/repos/asf/beam.git


    from 846d9c068d4 Don't lift combinebykey
     add 9f8e00e4393 Switch streaming engine worker harness based on job settings (#35901)
     add b9c277244f2 Add Jpms tests for Java 25 (#36380)
     add 800e2d7da5f Implement histogram user metric for python SDK (#36335)
     add 82952c8c29a Update python metrics example in programming guide (#36334)
     add 5af82f5f78c feat(subprocess_server): add fallback to Google Maven mirror  (#36365)
     add 1e973630e67 fix(sdk:python): Avoid AttributeError for transforms without hints (#36251)
     add 5485467f230 Handle pending adjustment for processing time bundle correctly. (#36384)
     add 312509fd019 Add back missing licenses (#36386)
     add c1fc3698a38 Java GroupByEncryptedKey (#36217)
     add bd77a1eff91 ci: update Python version from 3.12 to 3.13 in Flink pre-commit workflow (#36389)
     add 7174991d487 fix(python-sdk): restrict transformers version for python 3.9 compat (#36385)
     add 60436605e19 Add schema provider support for Kafka redistribute options (#36332)
     add 517bef3290f fix: handle empty requests in milvus search to avoid connection attempts (#36388)
     add 0b9fe991555 [Python] cloudsql_it_test  fixture runs before setupclass (#36406)
     add 25f65521f5b input is reshuffled so we can't guarantee that same input row will get specific job id based on which we have assert. (#36402)
     add 10a049ecf81 Fix python 3.13 version for different workflows (#36399)
     add 90e80b66522 Merge branch 'master' of https://github.com/apache/beam into users/damccorm/combinePerKey
     add 521b6d1c92c Lint

No new revisions were added by this update.

Summary of changes:
 .github/workflows/README.md                        |   9 +-
 ....yml => beam_PostCommit_Java_Jpms_Dataflow.yml} |  16 +-
 ...eam_PostCommit_Java_Jpms_Dataflow_Versions.yml} |  23 +-
 ...11.yml => beam_PostCommit_Java_Jpms_Direct.yml} |  17 +-
 .../beam_PostCommit_Java_Jpms_Direct_Java21.yml    | 101 ----
 ... beam_PostCommit_Java_Jpms_Direct_Versions.yml} |  23 +-
 .../beam_PostCommit_Python_Examples_Dataflow.yml   |   4 +-
 .../beam_PostCommit_Python_MongoDBIO_IT.yml        |   6 +-
 ..._PostCommit_XVR_PythonUsingJavaSQL_Dataflow.yml |   4 +-
 .../workflows/beam_PreCommit_Python_PVR_Flink.yml  |   4 +-
 .../org/apache/beam/gradle/BeamModulePlugin.groovy |  16 +-
 runners/google-cloud-dataflow-java/build.gradle    |   4 +-
 .../options/DataflowStreamingPipelineOptions.java  |   3 +
 .../dataflow/worker/StreamingDataflowWorker.java   | 512 +++++++++++++++------
 .../harness/SingleSourceWorkerHarness.java         |  13 +-
 .../dataflow/worker/FakeWindmillServer.java        |  47 ++
 .../worker/StreamingDataflowWorkerTest.java        | 148 ++++++
 .../worker/windmill/src/main/proto/windmill.proto  |   9 +
 runners/prism/java/build.gradle                    |   3 +-
 .../prism/internal/engine/elementmanager.go        |  44 +-
 .../resources/beam/checkstyle/suppressions.xml     |   2 +
 sdks/java/core/build.gradle                        |   5 +
 .../beam/sdk/transforms/GroupByEncryptedKey.java   | 254 ++++++++++
 .../java/org/apache/beam/sdk/util/GcpSecret.java   |  58 +++
 .../{SerializableSupplier.java => Secret.java}     |  18 +-
 .../sdk/transforms/GroupByEncryptedKeyTest.java    | 189 ++++++++
 .../KafkaReadSchemaTransformConfiguration.java     |  30 ++
 .../io/kafka/KafkaReadSchemaTransformProvider.java |  29 ++
 .../KafkaReadSchemaTransformProviderTest.java      |   7 +-
 sdks/python/apache_beam/internal/metrics/cells.py  |  21 +-
 .../apache_beam/internal/metrics/metric_test.py    |  39 ++
 .../apache_beam/io/gcp/bigquery_file_loads_test.py |   6 +-
 sdks/python/apache_beam/metrics/execution.py       |  12 +-
 sdks/python/apache_beam/metrics/metric.py          |   1 +
 .../python/apache_beam/metrics/monitoring_infos.py |  39 +-
 .../apache_beam/metrics/monitoring_infos_test.py   |  35 ++
 .../apache_beam/ml/rag/enrichment/milvus_search.py |   3 +
 sdks/python/apache_beam/pipeline.py                |  33 +-
 .../iqr_test.py => programming_guide_test.py}      |  44 +-
 .../apache_beam/runners/direct/direct_metrics.py   |  33 ++
 sdks/python/apache_beam/transforms/core.py         |   2 +-
 .../enrichment_handlers/cloudsql_it_test.py        |   3 +-
 sdks/python/apache_beam/utils/histogram.py         |  72 ++-
 sdks/python/apache_beam/utils/subprocess_server.py |  69 ++-
 .../apache_beam/utils/subprocess_server_test.py    |  50 ++
 sdks/python/setup.py                               |  12 +-
 .../content/en/documentation/programming-guide.md  |  16 +-
 47 files changed, 1673 insertions(+), 415 deletions(-)
 rename .github/workflows/{beam_PostCommit_Java_Jpms_Dataflow_Java11.yml => beam_PostCommit_Java_Jpms_Dataflow.yml} (88%)
 rename .github/workflows/{beam_PostCommit_Java_Jpms_Dataflow_Java17.yml => beam_PostCommit_Java_Jpms_Dataflow_Versions.yml} (83%)
 rename .github/workflows/{beam_PostCommit_Java_Jpms_Direct_Java11.yml => beam_PostCommit_Java_Jpms_Direct.yml} (87%)
 delete mode 100644 .github/workflows/beam_PostCommit_Java_Jpms_Direct_Java21.yml
 rename .github/workflows/{beam_PostCommit_Java_Jpms_Direct_Java17.yml => beam_PostCommit_Java_Jpms_Direct_Versions.yml} (83%)
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupByEncryptedKey.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/util/GcpSecret.java
 copy sdks/java/core/src/main/java/org/apache/beam/sdk/util/{SerializableSupplier.java => Secret.java} (62%)
 create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/GroupByEncryptedKeyTest.java
 copy sdks/python/apache_beam/{ml/anomaly/detectors/iqr_test.py => programming_guide_test.py} (50%)