You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "Abacn (via GitHub)" <gi...@apache.org> on 2023/09/12 20:41:26 UTC
[GitHub] [beam] Abacn opened a new issue, #28426: [Task]: Merge PythonFormatter and PythonLint
Abacn opened a new issue, #28426:
URL: https://github.com/apache/beam/issues/28426
### What needs to happen?
Currently, there are two precommit for python code format, running three tests
- PythonFormatter PreCommit
- run yapf
- PythonLint PreCommit
- run lint (format check)
- run mypy (type check)
For convenience we should merge them into one. Either tests does not take long to run.
### Issue Priority
Priority: 2 (default / most normal work should be filed as P2)
### Issue Components
- [ ] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #28426: [Task]: Shutdown Jenkins PreCommits
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #28426:
URL: https://github.com/apache/beam/issues/28426#issuecomment-1728293446
This comments tracks the test suites only available on GitHub Action
| Name | GitHub status | Phrase |
|-----|-----|-----|
| PostCommit_Go_Dataflow_ARM | [](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Go_Dataflow_ARM.yml?query=event%3Aschedule) | `Run Go PostCommit Dataflow ARM` |
| PostCommit_Java_Examples_Dataflow_ARM | [](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_Examples_Dataflow_ARM.yml?query=event%3Aschedule) | `Run Java_Examples_Dataflow_ARM PostCommit 8` `Run Java_Examples_Dataflow_ARM PostCommit 11` `Run Java_Examples_Dataflow_ARM PostCommit 17` |
| Python_ValidatesContainer_Dataflow_ARM | [](https://github.com/apache/beam/actions/workflows/beam_Python_ValidatesContainer_Dataflow_ARM.yml?query=event%3Aschedule) | `Run Python ValidatesContainer Dataflow ARM 3.8` `Run Python ValidatesContainer Dataflow ARM 3.9` `Run Python ValidatesContainer Dataflow ARM 3.10` `Run Python ValidatesContainer Dataflow ARM 3.11` |
| PreCommit_Java_Flink_Versions | [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Flink_Versions.yml?query=event%3Aschedule) | `Run Java_Flink_Versions PreCommit` |
| PreCommit_ItFramework | [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_ItFramework.yml?query=event%3Aschedule) | `Run It_Framework PreCommit` |
| PostCommit_Java_Examples_Dataflow | [](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_Examples_Dataflow.yml?query=event%3Aschedule) | `Run Java examples on Dataflow` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Shutdown Jenkins PreCommits [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #28426:
URL: https://github.com/apache/beam/issues/28426#issuecomment-1747593427
Current Performance tests:
| Name | Phrase | Jenkins status | GitHub Action status |
|-----|-----|-----|-----|
| PerformanceTests_AvroIOIT | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_AvroIOIT.yml?query=event%3Aschedule)
| PerformanceTests_BigQueryIO_Batch_Java_Avro | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Batch_Java_Avro/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_BigQueryIO_Batch_Java_Avro.yml?query=event%3Aschedule)
| PerformanceTests_BigQueryIO_Batch_Java_Json | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Batch_Java_Json/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_BigQueryIO_Batch_Java_Json.yml?query=event%3Aschedule)
| PerformanceTests_BigQueryIO_Streaming_Java | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Streaming_Java/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_BigQueryIO_Streaming_Java.yml?query=event%3Aschedule)
| PerformanceTests_BiqQueryIO_Read_Python | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Read_Python/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_BiqQueryIO_Read_Python.yml?query=event%3Aschedule)
| PerformanceTests_BiqQueryIO_Write_Python_Batch | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_BiqQueryIO_Write_Python_Batch/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_BiqQueryIO_Write_Python_Batch.yml?query=event%3Aschedule)
| PerformanceTests_Cdap | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_Cdap/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_Cdap.yml?query=event%3Aschedule)
| PerformanceTests_Compressed_TextIOIT | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_Compressed_TextIOIT.yml?query=event%3Aschedule)
| PerformanceTests_HadoopFormat | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_HadoopFormat.yml?query=event%3Aschedule)
| PerformanceTests_JDBC | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_JDBC/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_JDBC.yml?query=event%3Aschedule)
| PerformanceTests_Kafka_IO | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_Kafka_IO.yml?query=event%3Aschedule)
| PerformanceTests_ManyFiles_TextIOIT | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_ManyFiles_TextIOIT.yml?query=event%3Aschedule)
| PerformanceTests_ManyFiles_TextIOIT_HDFS | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS.yml?query=event%3Aschedule)
| PerformanceTests_MongoDBIO_IT | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_MongoDBIO_IT.yml?query=event%3Aschedule)
| PerformanceTests_ParquetIOIT | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_ParquetIOIT.yml?query=event%3Aschedule)
| PerformanceTests_ParquetIOIT_HDFS | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_ParquetIOIT_HDFS.yml?query=event%3Aschedule)
| PerformanceTests_PubsubIOIT_Python_Streaming | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_PubsubIOIT_Python_Streaming/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_PubsubIOIT_Python_Streaming.yml?query=event%3Aschedule)
| PerformanceTests_SQLBigQueryIO_Batch_Java | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_SQLBigQueryIO_Batch_Java/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_SQLBigQueryIO_Batch_Java.yml?query=event%3Aschedule)
| PerformanceTests_SpannerIO_Read_2GB_Python | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_SpannerIO_Read_2GB_Python/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_SpannerIO_Read_2GB_Python.yml?query=event%3Aschedule)
| PerformanceTests_SpannerIO_Write_2GB_Python_Batch | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_SpannerIO_Write_2GB_Python_Batch/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_SpannerIO_Write_2GB_Python_Batch.yml?query=event%3Aschedule)
| PerformanceTests_SparkReceiver_IO | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_SparkReceiver_IO/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_SparkReceiver_IO.yml?query=event%3Aschedule)
| PerformanceTests_TFRecordIOIT | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_TFRecordIOIT.yml?query=event%3Aschedule)
| PerformanceTests_TFRecordIOIT_HDFS | - | N/A | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_TFRecordIOIT_HDFS.yml?query=event%3Aschedule)
| PerformanceTests_TextIOIT | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_TextIOIT/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_TextIOIT.yml?query=event%3Aschedule)
| PerformanceTests_TextIOIT_HDFS | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_TextIOIT_HDFS.yml?query=event%3Aschedule)
| PerformanceTests_TextIOIT_Python | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_TextIOIT_Python/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_TextIOIT_Python.yml?query=event%3Aschedule)
| PerformanceTests_WordCountIT_PythonVersions | - | N/A | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_WordCountIT_PythonVersions.yml?query=event%3Aschedule)
| PerformanceTests_XmlIOIT | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_XmlIOIT.yml?query=event%3Aschedule)
| PerformanceTests_XmlIOIT_HDFS | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_XmlIOIT_HDFS.yml?query=event%3Aschedule)
| PerformanceTests_xlang_KafkaIO_Python | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_xlang_KafkaIO_Python/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_xlang_KafkaIO_Python.yml?query=event%3Aschedule)
Flaky:
| Name | Phrase | Jenkins status | GitHub Action status |
|-----|-----|-----|-----|
| PerformanceTests_AvroIOIT_HDFS | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_AvroIOIT_HDFS.yml?query=event%3Aschedule)
| PerformanceTests_Compressed_TextIOIT_HDFS | - | [](https://ci-beam.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/) | [](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_Compressed_TextIOIT_HDFS.yml?query=event%3Aschedule)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Shutdown Jenkins PreCommits [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #28426: [Task]: Shutdown Jenkins PreCommits
URL: https://github.com/apache/beam/issues/28426
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #28426: [Task]: Remove Jenkins PreCommits
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #28426:
URL: https://github.com/apache/beam/issues/28426#issuecomment-1719985552
This comment tracks the test suites that are shutdown in Jenkins
| Name | Jenkins status | GitHub Action status |
|-----|-----|-----|
| PreCommit_PythonDocs | [](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/) | [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_PythonDocs.yml)
| PreCommit_PythonLint | [](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/) | [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_PythonLint.yml)
| PreCommit_RAT | [](https://ci-beam.apache.org/job/beam_PreCommit_RAT_Cron/) | [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_RAT.yml)
| PreCommit_Spotless | [](https://ci-beam.apache.org/job/beam_PreCommit_Spotless_Cron/) | [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Spotless.yml)
| PreCommit_Website | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/) | [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Website.yml)
| PreCommit_Website_Stage_GCS | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/) | [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Website_Stage_GCS.yml)
| PreCommit_Whitespace | [](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/) | [](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Whitespace.yml)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org