You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2022/09/10 04:19:33 UTC

[beam] branch nightly-refs/heads/master updated (7a3ac2da4e0 -> 1526ca8c4cc)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git


    from 7a3ac2da4e0 Support models returning a dictionary of outputs (#23087)
     add 134648f0764 [TPC-DS] Store metrics into BigQuery and InfluxDB (#22545)
     add 139ef904f6f [Website] add table-container-wrapper #22896
     add 7ab04666da0 Merge pull request #22915: [Website] Update from-spark page table content overflow
     add bc929ee694c [Website] add paddings to pillars-item, change styles of footer logos
     add 5734d3e3af6 Merge pull request #2281: [Website] update homepage mobile styles
     add 98f1f75459e Use a ClassLoadingStrategy that is compatible with Java 17+
     add ef0bb6d1cbf Merge pull request #23055 from cushon/b243697303
     add ec4aaa434bc [Website] update case-studies logo images #22799 (#22793)
     add 88036d9b3b9 [Website] change media-query max-width variable to ak-breakpoint-xl #22521 (#22498)
     add 80284ef58f7 [Website] add overflow to code tags #22888 (#22427)
     add 8c6dcaeceb0 Clean up Kafka Cluster and pubsub topic in rc validation script (#23021)
     add a75f115d20d Fix assertions in the Spanner IO IT tests (#23098)
     add ce80ee33c5f [Website] update shortcode languages from duplicate go to typescript #23004
     add 733b0568fe7 Merge pull request #22275: [Website] update shortcode languages
     add 973475fde19 Use existing pickle_library flag in expansion service. (#23111)
     add 07439a0ab8a Assert pipeline results in performance tests (#23027)
     add d17914862da Consolidate Samza TranslationContext and PortableTranslationContext (#23072)
     add 1526ca8c4cc Improvements to SchemaTransform implementations for BQ and Kafka (#23045)

No new revisions were added by this update.

Summary of changes:
 ...rties.groovy => TpcdsDatabaseProperties.groovy} |  16 +-
 .../jenkins/job_PerformanceTests_KafkaIO_IT.groovy |  29 +--
 .../job_PostCommit_Java_Tpcds_Dataflow.groovy      |  18 +-
 .../jenkins/job_PostCommit_Java_Tpcds_Flink.groovy |  15 +-
 .../jenkins/job_PostCommit_Java_Tpcds_Spark.groovy | 109 ++------
 ...assic_New_SQL.json => TPC-DS_Dataflow_SQL.json} | 279 +++++++++++----------
 ..._Classic_New_SQL.json => TPC-DS_Flink_SQL.json} | 279 +++++++++++----------
 ...dard.json => TPC-DS_Spark_Classic_New_SQL.json} | 224 +++++++++--------
 release/src/main/scripts/run_rc_validation.sh      |  16 +-
 .../samza/translation/GroupByKeyTranslator.java    |   2 +-
 .../translation/ParDoBoundMultiTranslator.java     |   2 +-
 .../translation/PortableTranslationContext.java    |  51 +---
 .../samza/translation/ReshuffleTranslator.java     |   2 +-
 .../SamzaPortablePipelineTranslator.java           |   8 +-
 .../samza/translation/TranslationContext.java      |  15 +-
 .../reflect/ByteBuddyDoFnInvokerFactory.java       |  29 ++-
 .../extensions/python/PythonExternalTransform.java |   3 +-
 .../provider/bigquery/BigQueryIOPushDownIT.java    |  13 +-
 .../beam/sdk/bigqueryioperftests/BigQueryIOIT.java |  10 +-
 .../java/org/apache/beam/sdk/io/cdap/CdapIOIT.java |   5 +-
 .../java/org/apache/beam/sdk/io/avro/AvroIOIT.java |   5 +-
 .../apache/beam/sdk/io/parquet/ParquetIOIT.java    |   5 +-
 .../java/org/apache/beam/sdk/io/text/TextIOIT.java |   5 +-
 .../apache/beam/sdk/io/tfrecord/TFRecordIOIT.java  |   8 +-
 .../java/org/apache/beam/sdk/io/xml/XmlIOIT.java   |   5 +-
 .../io/gcp/bigquery/BigQuerySchemaIOProvider.java  |  20 +-
 .../sdk/io/hadoop/format/HadoopFormatIOIT.java     |   8 +-
 .../java/org/apache/beam/sdk/io/jdbc/JdbcIOIT.java |   8 +-
 .../KafkaSchemaTransformReadConfiguration.java     | 170 ++-----------
 .../io/kafka/KafkaSchemaTransformReadProvider.java | 167 ++++++++++++
 .../org/apache/beam/sdk/io/kafka/KafkaIOIT.java    |  14 +-
 .../KafkaSchemaTransformReadProviderTest.java      | 124 +++++++++
 .../apache/beam/sdk/io/mongodb/MongoDBIOIT.java    |   9 +-
 sdks/java/testing/tpcds/build.gradle               |   3 +
 .../apache/beam/sdk/tpcds/SqlTransformRunner.java  | 102 +++++++-
 .../org/apache/beam/sdk/tpcds/TpcdsOptions.java    |  64 +++++
 .../java/org/apache/beam/sdk/tpcds/TpcdsRun.java   |   5 +
 .../org/apache/beam/sdk/tpcds/TpcdsRunResult.java  |   4 +-
 .../java/org/apache/beam/sdk/tpcds/TpcdsUtils.java |  12 +
 .../io/gcp/experimental/spannerio_read_it_test.py  |  15 +-
 .../runners/portability/expansion_service_main.py  |   9 +-
 website/www/site/assets/scss/_case_study.scss      |   7 +-
 website/www/site/assets/scss/_ctas.sass            |   2 +-
 website/www/site/assets/scss/_footer.sass          |   4 +
 website/www/site/assets/scss/_global.sass          |   7 +
 website/www/site/assets/scss/_pillars.scss         |   4 +
 website/www/site/content/en/case-studies/hop.md    |   2 +-
 website/www/site/content/en/case-studies/lyft.md   |   2 +-
 .../content/en/documentation/programming-guide.md  |   6 +-
 .../www/site/content/en/get-started/from-spark.md  |   4 +
 50 files changed, 1170 insertions(+), 755 deletions(-)
 copy .test-infra/jenkins/{NexmarkDatabaseProperties.groovy => TpcdsDatabaseProperties.groovy} (78%)
 copy .test-infra/metrics/grafana/dashboards/perftests_metrics/{Nexmark_Spark_Classic_New_SQL.json => TPC-DS_Dataflow_SQL.json} (65%)
 copy .test-infra/metrics/grafana/dashboards/perftests_metrics/{Nexmark_Spark_Classic_New_SQL.json => TPC-DS_Flink_SQL.json} (65%)
 copy .test-infra/metrics/grafana/dashboards/perftests_metrics/{Nexmark_Spark_Classic_New_Standard.json => TPC-DS_Spark_Classic_New_SQL.json} (71%)
 create mode 100644 sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaSchemaTransformReadProvider.java
 create mode 100644 sdks/java/io/kafka/src/test/java/org/apache/beam/sdk/io/kafka/KafkaSchemaTransformReadProviderTest.java