[GitHub] [beam] AnandInguva commented on a diff in pull request #25242: Add runtime metric to TFT tests

["AnandInguva (via GitHub)" <gi...@apache.org>]

AnandInguva commented on code in PR #25242:
URL: https://github.com/apache/beam/pull/25242#discussion_r1097898583


##########
sdks/python/apache_beam/testing/benchmarks/cloudml/cloudml_benchmark_test.py:
##########
@@ -32,6 +35,27 @@
 _OUTPUT_GCS_BUCKET_ROOT = 'gs://temp-storage-for-end-to-end-tests/tft/'
 
 
+def _publish_metrics(pipeline, metric_value, metrics_table):
+  influx_options = InfluxDBMetricsPublisherOptions(
+      metrics_table,
+      pipeline.get_option('influx_db_name'),
+      pipeline.get_option('influx_hostname'),
+      os.getenv('INFLUXDB_USER'),
+      os.getenv('INFLUXDB_USER_PASSWORD'),
+  )
+  metric_reader = MetricsReader(
+      project_name=pipeline.get_option('project'),
+      bq_table=metrics_table,
+      bq_dataset=pipeline.get_option('metrics_dataset'),
+      publish_to_bq=True,
+      influxdb_options=influx_options,
+  )
+  metric_reader.publish_values([(
+      'runtime',
+      metric_value,

Review Comment:
   1. Changed it.
   2. Yes, that is right, once the changes are in, I will create a grafana dashboard.
   3. I am not sure if that would be useful. MetricsReader is all we need and to publish it to influx DB we would need InfluxDB related information, which I passed it through pipeline options. Other benchmark tests, they have an abstract class `LoadTest`, which takes care of these things. Other benchmark test like fastavro_it_test do it like the way I did it.
   4. Added a param to the function to accept any metric name.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org