You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by bb...@apache.org on 2022/03/08 16:27:22 UTC
[airflow] branch mapped-task-drawer updated (a32d887 -> 1ef4d27)
This is an automated email from the ASF dual-hosted git repository.
bbovenzi pushed a change to branch mapped-task-drawer
in repository https://gitbox.apache.org/repos/asf/airflow.git.
discard a32d887 make side panel collapsible, useTasks,
discard 2f8e444 use API
discard 2192c4a add tooltip info to details
discard 5cb4def switch from drawer to details section
discard ce953b3 fix hover and extra prop
discard eb96150 improve rendering and add selected dag run
discard ee86937 reformat grid background colors
discard 780d6a2 basic slide drawer
discard adfc609 make UI and tree work with mapped tasks
add 3052268 Chart: move updating note about config removal (#21919)
add 84ed747 Unpin ``pandas-gbq`` and remove unused code (#21915)
add a0e2eba Unpin `google-cloud-memcache` (#21912)
add f42559a Add `test_connection` method to `AzureDataFactoryHook` (#21924)
add b2fa135 py files doesn't have to be checked is_zipfiles in refresh_dag (#21926)
add 2c57ad4 Fix incorrect data provided to tries & landing times charts (#21928)
add 8b276c6 Ensure deps is set, convert BaseSensorOperator to classvar (#21815)
add bc95db9 Only allow mapping against return value XCom (#21930)
add c16228d Fix single backticks in rst changelogs (#21922)
add ac77c89 Add map_index to pods launched by KubernetesExecutor (#21871)
add 6c37e47 Add map_index label to mapped KubernetesPodOperator (#21916)
add 58a7945 update freshworks company users (#21932)
add 3f9295a Update ECS sample DAG and Docs to new standards (#21828)
add 3c4dd3f Chart: 1.5.0 changelog (#21906)
add 42e2b80 Rename `xcom.dagrun_id` to `xcom.dag_run_id` (#21806)
add d34ebcb Exit with appropriate message for shell in PROD image in breeze (#21848)
add ba79adb Make container creation configurable when uploading files via WasbHook (#20510)
add af79df6 Add RedshiftDataHook (#19137)
add 5b45a78 Add docs for `db upgrade` / `db downgrade` (#21879)
add 0b9257b Add CI jobs and tooling to aid with tracking backtracking pip issues (#21825)
add 1956f38 Enhance magic methods on XComArg for UX (#21882)
add ffacf3e enter the shell breeze2 environment (#21145)
add cc4b056 Proposed policy for Provider's minimum supported version. (#21696)
add c929780 Add detailed email docs for Sendgrid (#21958)
add 3b4f5df Dev: Update script used in generating issues for status of testing of RC (#21950)
add 7e806cf Fix typo in docs (#21964)
add 0d856b1 Remove unnecessary loggings from offline sql generation command (#21962)
add 8d8d072 Change KubePodOperator labels from exeuction_date to run_id (#21960)
add b65e522 More explicit mapped argument validation (#21933)
add d6a079c Ensure that `airflow dags tests` works for mapped DAGs (#21969)
add aa8bffd Add compat shim for SQLAlchemy to avoid warnings (#21959)
add 17d3e78 replace extra links value (#21971)
add 3008fc3 Run inclusive language check on CHANGELOG (#21980)
add 2b4d146 Fix changelog/updating typos (#21979)
add 9ce45ff Rename 'S3' hook name to 'Amazon S3' (#21988)
add 1949f5d Change the storage of frame to use threadLocal rather than Dict (#21993)
add b153a2e Add test to run DB downgrade in the CI (#21273)
add e81527c Simplify tests for CLI connections commands (#21983)
add 28e3d65 Add per-DAG delete permissions (#21938)
add e9b42d6 Don't validate that Params are JSON when NOTSET (#22000)
add f5f2216 Minor cleanup on Celery config (#21880)
add 8c63a8d Add brief examples of integration test dags you might want (#22009)
add de29eb6 Upgrade `moto` library to version 3.0 (#22005)
add 652b859 Local kubernetes executor (#19729)
add a9b7dd6 Resolve mypy issue in athena example dag (#22020)
add 26e8d6d Updates FTPHook provider to have test_connection (#21997)
add 9020b3a Add autodetect arg to external table creation in GCSToBigQueryOperator (#21944)
add d8b730a Fix typo in docs (#22024)
add 2157787 Remove back the limitation of Yandex as a bug in 0.142 is fixed (#22023)
add 76150f1 Fix failing main after merging SFTP hook test connection (#22026)
add f968eba Added AWS RDS operators (#20907)
add 80c52a1 Added AWS RDS sensors (#21231)
add 9a9d54b Fixing bug when roles list is empty (#18590)
add 7acc190 added docker network_mode options (#21986)
add 9be3c50 Removes limitations from Dask dependencies (#22017)
add 7724a5a Move S3ToRedshiftOperator documentation to transfer dir (#21975)
add 7be2041 Default args type check (#21809)
add c22fb31 Cleanup RedshiftSQLOperator documentation (#21976)
add 73c6bf0 Switch oss hook tests in alibaba-provider to use Mocks (17617) (#21992)
add 188ac51 Change default python executable to python3 for docker decorator (#21973)
add 01a1a26 retry on very specific eni provision failures (#22002)
add 0209a5c Add docs and sample dags for AWS Batch (#22010)
add 4e05bbc Dev: Allow easy repacking of Providers (#22018)
add 2ab8f24 Fix mypy errors resulting from typing updates in marshmallow (#22044)
add 082aafd Update limits of dependencies after `dask` test disabling (#22046)
add 42d3ab1 missing quotes (#21990)
add db76f0b Add information on DAG pausing/deactivation/deletion (#22025)
add 6db9b00 Add Looker PDT operators (#20882)
add 3f18ffc Update best-practices.rst (#22053)
add 6126c4e Fix spelling (#22054)
add f5b9631 Add documentation for Feb Providers release (#22056)
add e7fed6b DB upgrade is required when updating Airflow (#22061)
add c1faaf3 Add sample dag and doc for RedshiftToS3Operator (#22060)
add a11d831 Allow for uploading metadata with GCS Hook Upload (#22058)
add 40944fc Clean up the `pre-commit` config (#22052)
add 374354d Dynamo to S3 Sample DAG and Docs (#21920)
add 064b39f Add example config of sql_alchemy_connect_args (#22045)
add c8d49f6 fixes query status polling logic (#21423)
add a150ee0 Add template fields to DynamoDBToS3Operator (#22080)
add 4014194 Add new options to DatabricksCopyIntoOperator (#22076)
add 5ace37a Store callbacks in database if standalone_dag_processor config is True. (#21731)
add ee9d245 Add Mapped task instance endpoint (#21965)
add 184a46f refactors polling logic for athena queries (#21488)
add c7286e5 Bug-fix GCSToS3Operator (#22071)
add 7f4935b Use yaml safe load (#22085)
add 564c7ca Improve guidelines for contributors for provider testing (#22087)
add eba9703 Revert "Use yaml safe load (#22085)" (#22089)
add d3c168c Fixed dask executor and tests (#22027)
add 99cf75d Fix/21994 liveness probe (#22041)
new db7a659 make UI and tree work with mapped tasks
new f2836fa basic slide drawer
new 5390dbc reformat grid background colors
new e22bcc6 improve rendering and add selected dag run
new 28cf8f0 fix hover and extra prop
new bcb3242 switch from drawer to details section
new e5764b3 add tooltip info to details
new a301881 use API
new 1ef4d27 make side panel collapsible, useTasks,
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a32d887)
\
N -- N -- N refs/heads/mapped-task-drawer (1ef4d27)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 9 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/build-images.yml | 48 +-
.github/workflows/ci.yml | 8 +
.pre-commit-config.yaml | 30 +-
BREEZE.rst | 1 +
CHANGELOG.txt | 130 +-
INTHEWILD.md | 2 +-
README.md | 13 +
UPDATING.md | 44 +-
airflow/api/common/mark_tasks.py | 24 +-
.../endpoints/task_instance_endpoint.py | 63 +-
airflow/api_connexion/openapi/v1.yaml | 46 +-
.../api_connexion/schemas/task_instance_schema.py | 1 +
airflow/callbacks/callback_requests.py | 30 +-
.../database_callback_sink.py} | 30 +-
airflow/cli/cli_parser.py | 41 +-
airflow/cli/commands/connection_command.py | 1 +
airflow/cli/commands/task_command.py | 4 +-
airflow/cli/commands/user_command.py | 33 +-
.../provider.yaml => compat/sqlalchemy.py} | 28 +-
airflow/config_templates/config.yml | 41 +-
airflow/config_templates/default_airflow.cfg | 25 +-
airflow/config_templates/default_celery.py | 6 +-
airflow/contrib/hooks/bigquery_hook.py | 1 -
airflow/dag_processing/manager.py | 34 +-
airflow/decorators/__init__.pyi | 8 +
airflow/decorators/base.py | 36 +-
.../example_local_kubernetes_executor.py | 67 +
airflow/exceptions.py | 4 +
airflow/executors/celery_kubernetes_executor.py | 2 +-
airflow/executors/dask_executor.py | 2 +-
airflow/executors/executor_constants.py | 1 +
airflow/executors/executor_loader.py | 14 +
airflow/executors/kubernetes_executor.py | 39 +-
...es_executor.py => local_kubernetes_executor.py} | 91 +-
airflow/jobs/scheduler_job.py | 88 +-
airflow/kubernetes/kubernetes_helper_functions.py | 9 +-
airflow/kubernetes/pod_generator.py | 4 +
airflow/migrations/utils.py | 43 +
...6b6f902_increase_length_of_fab_ab_view_menu_.py | 6 +-
...5d12_add_max_active_runs_column_to_dagmodel_.py | 3 +-
.../versions/1507a7289a2f_create_is_encrypted.py | 5 +-
.../2c6edca13270_resource_based_permissions.py | 2 +-
...7a1ff4_add_kubernetes_resource_checkpointing.py | 7 +-
.../versions/3c20cacc0044_add_dagrun_run_type.py | 4 +-
...27fdd3_increase_length_of_email_and_username.py | 40 +-
...e3737b18f_added_timetable_description_column.py | 7 +-
.../7b2661a43ba3_taskinstance_keyed_to_dagrun.py | 40 +-
...7b86_increase_pool_name_size_in_taskinstance.py | 11 +-
.../versions/92c57b58940d_add_fab_tables.py | 7 +-
...93827b8_add_queued_at_column_to_dagrun_table.py | 3 +-
.../b25a55525161_increase_length_of_pool_name.py | 1 -
.../bbf4a7ad0465_remove_id_column_from_xcom.py | 12 +-
...a23_add_has_import_errors_column_to_dagmodel.py | 3 +-
.../bef4f3d11e8b_drop_kuberesourceversion_and_.py | 7 +-
...c306b5b5ae4a_switch_xcom_table_to_use_run_id.py | 20 +-
... => c97c2ab6aa23_add_callback_request_table.py} | 33 +-
...623dc7_add_max_tries_column_to_task_instance.py | 8 +-
.../versions/cf5dc11e79ad_drop_user_and_chart.py | 5 +-
.../e38be357a868_update_schema_for_smart_sensor.py | 6 +-
.../versions/e3a246e0dc1_current_schema.py | 4 +-
...53f75_add_taskmap_and_map_id_on_taskinstance.py | 4 +-
...4a3141f0_make_xcom_pkey_columns_non_nullable.py | 11 +-
...ac86c_change_field_in_dagcode_to_mediumtext_.py | 5 +-
...62e7089_add_task_log_filename_template_model.py | 1 +
airflow/models/__init__.py | 2 +
airflow/models/baseoperator.py | 54 +-
airflow/models/dag.py | 16 +-
airflow/models/db_callback_request.py | 52 +
airflow/models/mappedoperator.py | 93 +-
airflow/models/param.py | 23 +-
airflow/models/taskinstance.py | 148 +-
airflow/models/xcom.py | 20 +-
airflow/models/xcom_arg.py | 18 +-
airflow/operators/python.py | 2 +
airflow/providers/airbyte/CHANGELOG.rst | 23 +
airflow/providers/airbyte/provider.yaml | 1 +
airflow/providers/alibaba/CHANGELOG.rst | 31 +
airflow/providers/alibaba/provider.yaml | 1 +
airflow/providers/amazon/CHANGELOG.rst | 48 +
.../amazon/aws/example_dags/example_athena.py | 2 +-
.../amazon/aws/example_dags/example_batch.py | 66 +
.../aws/example_dags/example_dynamodb_to_s3.py} | 53 +-
.../example_dynamodb_to_s3_segmented.py | 60 +
.../amazon/aws/example_dags/example_ecs_fargate.py | 74 +-
.../amazon/aws/example_dags/example_rds.py | 151 ++
.../example_redshift_data_execute_sql.py | 80 ++
...example_redshift.py => example_redshift_sql.py} | 51 +-
.../aws/example_dags/example_redshift_to_s3.py} | 53 +-
.../aws/example_dags/example_s3_to_redshift.py | 37 +-
airflow/providers/amazon/aws/exceptions.py | 11 +
airflow/providers/amazon/aws/hooks/athena.py | 13 +-
airflow/providers/amazon/aws/hooks/emr.py | 15 +-
.../providers/amazon/aws/hooks/redshift_data.py | 52 +
airflow/providers/amazon/aws/hooks/s3.py | 2 +-
airflow/providers/amazon/aws/operators/batch.py | 5 +-
airflow/providers/amazon/aws/operators/ecs.py | 40 +-
airflow/providers/amazon/aws/operators/rds.py | 560 ++++++++
.../amazon/aws/operators/redshift_data.py | 159 +++
airflow/providers/amazon/aws/sensors/batch.py | 4 +
airflow/providers/amazon/aws/sensors/rds.py | 150 ++
.../amazon/aws/transfers/dynamodb_to_s3.py | 39 +-
.../providers/amazon/aws/transfers/gcs_to_s3.py | 10 +-
.../amazon/aws/transfers/redshift_to_s3.py | 4 +
.../provider.yaml => amazon/aws/utils/rds.py} | 18 +-
airflow/providers/amazon/provider.yaml | 25 +-
airflow/providers/apache/beam/CHANGELOG.rst | 30 +
airflow/providers/apache/beam/provider.yaml | 1 +
airflow/providers/apache/cassandra/CHANGELOG.rst | 23 +
airflow/providers/apache/cassandra/provider.yaml | 1 +
airflow/providers/apache/drill/CHANGELOG.rst | 23 +
airflow/providers/apache/drill/provider.yaml | 1 +
airflow/providers/apache/druid/CHANGELOG.rst | 11 +
airflow/providers/apache/druid/provider.yaml | 1 +
airflow/providers/apache/hdfs/CHANGELOG.rst | 18 +
airflow/providers/apache/hdfs/provider.yaml | 1 +
airflow/providers/apache/hive/CHANGELOG.rst | 25 +
airflow/providers/apache/hive/provider.yaml | 1 +
airflow/providers/apache/kylin/CHANGELOG.rst | 26 +
airflow/providers/apache/kylin/provider.yaml | 1 +
airflow/providers/apache/livy/CHANGELOG.rst | 28 +
airflow/providers/apache/livy/provider.yaml | 1 +
airflow/providers/apache/pig/CHANGELOG.rst | 23 +
airflow/providers/apache/pig/provider.yaml | 1 +
airflow/providers/apache/pinot/CHANGELOG.rst | 15 +
airflow/providers/apache/pinot/provider.yaml | 1 +
airflow/providers/apache/spark/CHANGELOG.rst | 16 +
airflow/providers/apache/spark/provider.yaml | 1 +
airflow/providers/apache/sqoop/CHANGELOG.rst | 13 +
airflow/providers/apache/sqoop/provider.yaml | 1 +
airflow/providers/asana/CHANGELOG.rst | 27 +
airflow/providers/asana/provider.yaml | 1 +
airflow/providers/celery/CHANGELOG.rst | 17 +
airflow/providers/celery/provider.yaml | 1 +
airflow/providers/cloudant/CHANGELOG.rst | 15 +
airflow/providers/cloudant/provider.yaml | 1 +
airflow/providers/cncf/kubernetes/CHANGELOG.rst | 20 +
.../cncf/kubernetes/operators/kubernetes_pod.py | 17 +-
airflow/providers/cncf/kubernetes/provider.yaml | 1 +
airflow/providers/databricks/CHANGELOG.rst | 27 +
.../databricks/operators/databricks_sql.py | 71 +-
airflow/providers/databricks/provider.yaml | 1 +
airflow/providers/datadog/CHANGELOG.rst | 17 +
airflow/providers/datadog/provider.yaml | 1 +
airflow/providers/dingding/CHANGELOG.rst | 23 +
airflow/providers/dingding/provider.yaml | 1 +
airflow/providers/discord/CHANGELOG.rst | 18 +
airflow/providers/discord/provider.yaml | 1 +
airflow/providers/docker/CHANGELOG.rst | 20 +
airflow/providers/docker/decorators/docker.py | 20 +-
.../docker/example_dags/example_docker.py | 1 -
airflow/providers/docker/operators/docker.py | 7 +
airflow/providers/docker/provider.yaml | 1 +
airflow/providers/elasticsearch/CHANGELOG.rst | 18 +
airflow/providers/elasticsearch/provider.yaml | 1 +
airflow/providers/exasol/CHANGELOG.rst | 11 +
airflow/providers/exasol/provider.yaml | 1 +
airflow/providers/facebook/CHANGELOG.rst | 14 +
airflow/providers/facebook/provider.yaml | 1 +
airflow/providers/ftp/CHANGELOG.rst | 23 +
airflow/providers/ftp/hooks/ftp.py | 11 +-
airflow/providers/ftp/provider.yaml | 1 +
airflow/providers/github/CHANGELOG.rst | 15 +-
airflow/providers/github/provider.yaml | 1 +
airflow/providers/google/CHANGELOG.rst | 38 +
.../example_dags/example_cloud_memorystore.py | 4 +-
.../google/cloud/example_dags/example_looker.py | 63 +
airflow/providers/google/cloud/hooks/bigquery.py | 28 +-
airflow/providers/google/cloud/hooks/gcs.py | 6 +
airflow/providers/google/cloud/hooks/looker.py | 258 ++++
.../google/cloud/operators/cloud_memorystore.py | 9 +-
airflow/providers/google/cloud/operators/looker.py | 105 ++
airflow/providers/google/cloud/sensors/looker.py | 84 ++
.../google/cloud/transfers/gcs_to_bigquery.py | 1 +
airflow/providers/google/provider.yaml | 16 +
airflow/providers/grpc/CHANGELOG.rst | 22 +
airflow/providers/grpc/provider.yaml | 1 +
airflow/providers/hashicorp/CHANGELOG.rst | 22 +
airflow/providers/hashicorp/provider.yaml | 1 +
airflow/providers/http/CHANGELOG.rst | 17 +
airflow/providers/http/provider.yaml | 1 +
airflow/providers/imap/CHANGELOG.rst | 11 +
airflow/providers/imap/provider.yaml | 1 +
airflow/providers/influxdb/CHANGELOG.rst | 20 +
airflow/providers/influxdb/provider.yaml | 1 +
airflow/providers/jdbc/CHANGELOG.rst | 12 +
airflow/providers/jdbc/provider.yaml | 1 +
airflow/providers/jenkins/CHANGELOG.rst | 15 +
airflow/providers/jenkins/provider.yaml | 1 +
airflow/providers/jira/CHANGELOG.rst | 19 +
airflow/providers/jira/provider.yaml | 1 +
airflow/providers/microsoft/azure/CHANGELOG.rst | 20 +
.../microsoft/azure/hooks/data_factory.py | 22 +-
airflow/providers/microsoft/azure/hooks/wasb.py | 104 +-
airflow/providers/microsoft/azure/provider.yaml | 1 +
.../microsoft/azure/transfers/local_to_wasb.py | 12 +-
.../microsoft/azure/transfers/sftp_to_wasb.py | 12 +-
airflow/providers/microsoft/mssql/CHANGELOG.rst | 11 +
airflow/providers/microsoft/mssql/provider.yaml | 1 +
airflow/providers/microsoft/psrp/CHANGELOG.rst | 12 +
airflow/providers/microsoft/psrp/provider.yaml | 1 +
airflow/providers/microsoft/winrm/CHANGELOG.rst | 12 +
airflow/providers/microsoft/winrm/provider.yaml | 1 +
airflow/providers/mongo/CHANGELOG.rst | 14 +
airflow/providers/mongo/provider.yaml | 1 +
airflow/providers/mysql/CHANGELOG.rst | 13 +
airflow/providers/mysql/provider.yaml | 1 +
airflow/providers/neo4j/CHANGELOG.rst | 14 +
airflow/providers/neo4j/provider.yaml | 1 +
airflow/providers/odbc/CHANGELOG.rst | 11 +
airflow/providers/odbc/provider.yaml | 1 +
airflow/providers/openfaas/CHANGELOG.rst | 20 +
airflow/providers/openfaas/provider.yaml | 1 +
airflow/providers/opsgenie/CHANGELOG.rst | 14 +
airflow/providers/opsgenie/provider.yaml | 1 +
airflow/providers/oracle/CHANGELOG.rst | 16 +
airflow/providers/oracle/provider.yaml | 1 +
airflow/providers/pagerduty/CHANGELOG.rst | 15 +
airflow/providers/pagerduty/provider.yaml | 1 +
airflow/providers/papermill/CHANGELOG.rst | 14 +
airflow/providers/papermill/provider.yaml | 1 +
airflow/providers/plexus/CHANGELOG.rst | 21 +
airflow/providers/plexus/provider.yaml | 1 +
airflow/providers/postgres/CHANGELOG.rst | 26 +-
airflow/providers/postgres/provider.yaml | 1 +
airflow/providers/presto/CHANGELOG.rst | 29 +
airflow/providers/presto/provider.yaml | 1 +
airflow/providers/qubole/CHANGELOG.rst | 12 +
airflow/providers/qubole/provider.yaml | 1 +
airflow/providers/redis/CHANGELOG.rst | 18 +
airflow/providers/redis/provider.yaml | 1 +
airflow/providers/salesforce/CHANGELOG.rst | 20 +
airflow/providers/salesforce/provider.yaml | 1 +
airflow/providers/samba/CHANGELOG.rst | 17 +
airflow/providers/samba/provider.yaml | 1 +
airflow/providers/segment/CHANGELOG.rst | 19 +-
airflow/providers/segment/provider.yaml | 1 +
airflow/providers/sendgrid/CHANGELOG.rst | 10 +
airflow/providers/sendgrid/provider.yaml | 1 +
airflow/providers/sftp/CHANGELOG.rst | 20 +
airflow/providers/sftp/hooks/sftp.py | 2 +-
airflow/providers/sftp/provider.yaml | 1 +
airflow/providers/singularity/CHANGELOG.rst | 23 +
airflow/providers/singularity/provider.yaml | 1 +
airflow/providers/slack/CHANGELOG.rst | 11 +
airflow/providers/slack/provider.yaml | 1 +
airflow/providers/snowflake/CHANGELOG.rst | 11 +
airflow/providers/snowflake/provider.yaml | 1 +
airflow/providers/sqlite/CHANGELOG.rst | 11 +
airflow/providers/sqlite/provider.yaml | 1 +
airflow/providers/ssh/CHANGELOG.rst | 11 +
airflow/providers/ssh/provider.yaml | 1 +
airflow/providers/tableau/CHANGELOG.rst | 11 +
airflow/providers/tableau/provider.yaml | 1 +
airflow/providers/telegram/CHANGELOG.rst | 24 +
airflow/providers/telegram/provider.yaml | 1 +
airflow/providers/trino/CHANGELOG.rst | 24 +
airflow/providers/trino/provider.yaml | 1 +
airflow/providers/vertica/CHANGELOG.rst | 11 +
airflow/providers/vertica/provider.yaml | 1 +
airflow/providers/yandex/CHANGELOG.rst | 16 +
.../yandex/operators/yandexcloud_dataproc.py | 8 +-
airflow/providers/yandex/provider.yaml | 1 +
airflow/providers/zendesk/CHANGELOG.rst | 12 +
airflow/providers/zendesk/provider.yaml | 1 +
airflow/security/permissions.py | 2 +-
airflow/sensors/base.py | 14 +-
airflow/settings.py | 1 +
airflow/ti_deps/dependencies_deps.py | 4 +-
.../{dagrun_id_dep.py => dagrun_backfill_dep.py} | 10 +-
airflow/ti_deps/deps/ready_to_reschedule.py | 4 +
airflow/utils/db.py | 40 +-
airflow/utils/db_cleanup.py | 2 +
airflow/utils/task_group.py | 1 -
airflow/www/fab_security/sqla/manager.py | 4 +-
airflow/www/security.py | 12 +-
airflow/www/views.py | 26 +-
breeze | 8 +-
breeze-complete | 2 +-
chart/CHANGELOG.txt | 40 +-
chart/Chart.yaml | 76 +
chart/UPDATING.rst | 12 +-
.../templates/scheduler/scheduler-deployment.yaml | 23 +-
.../templates/triggerer/triggerer-deployment.yaml | 23 +-
chart/tests/test_scheduler.py | 4 +
chart/tests/test_triggerer.py | 5 +
chart/values.schema.json | 14 +
chart/values.yaml | 37 +
dev/ISSUE_TEMPLATE.md.jinja2 | 2 +-
dev/README_RELEASE_HELM_CHART.md | 22 +
dev/README_RELEASE_PROVIDER_PACKAGES.md | 18 +-
dev/TRACKING_BACKTRACKING_ISSUES.md | 220 +++
dev/breeze/setup.cfg | 4 +
dev/breeze/src/airflow_breeze/breeze.py | 140 +-
dev/breeze/src/airflow_breeze/cache.py | 30 +-
dev/breeze/src/airflow_breeze/ci/build_image.py | 17 +-
dev/breeze/src/airflow_breeze/ci/build_params.py | 28 +-
.../docs_generator/build_documentation.py | 6 +-
dev/breeze/src/airflow_breeze/global_constants.py | 136 +-
.../breeze/src/airflow_breeze/shell}/__init__.py | 0
dev/breeze/src/airflow_breeze/shell/enter_shell.py | 240 ++++
.../src/airflow_breeze/shell/shell_builder.py | 231 +++
.../airflow_breeze/utils/docker_command_utils.py | 117 +-
.../src/airflow_breeze/utils/host_info_utils.py | 62 +
dev/breeze/src/airflow_breeze/utils/path_utils.py | 23 +-
dev/breeze/src/airflow_breeze/utils/run_utils.py | 154 +-
dev/breeze/src/airflow_breeze/visuals/__init__.py | 4 +-
.../src/airflow_ci/find_newer_dependencies.py | 142 ++
dev/breeze/tests/test_cache.py | 32 +-
dev/breeze/tests/test_commands.py | 9 +-
dev/breeze/tests/test_docker_command_utils.py | 180 +++
.../breeze/tests/test_host_info_utils.py | 18 +-
dev/breeze/tests/test_run_utils.py | 53 +
dev/prepare_release_issue.py | 39 +-
.../PROVIDER_ISSUE_TEMPLATE.md.jinja2 | 4 +
dev/provider_packages/prepare_provider_packages.py | 44 +-
docs/apache-airflow-providers-airbyte/commits.rst | 93 +-
docs/apache-airflow-providers-airbyte/index.rst | 6 +-
docs/apache-airflow-providers-alibaba/commits.rst | 58 +-
docs/apache-airflow-providers-alibaba/index.rst | 2 +-
docs/apache-airflow-providers-amazon/commits.rst | 1070 +++++++-------
docs/apache-airflow-providers-amazon/index.rst | 31 +-
.../operators/batch.rst | 65 +
.../operators/ecs.rst | 35 +-
.../operators/rds.rst | 200 +++
.../operators/redshift_data.rst | 52 +
.../operators/redshift_sql.rst | 88 +-
.../operators/transfer/dynamodb_to_s3.rst | 61 +
.../operators/transfer/redshift_to_s3.rst | 56 +
.../operators/{ => transfer}/s3_to_redshift.rst | 57 +-
docs/apache-airflow-providers-amazon/redirects.txt | 18 +
.../commits.rst | 127 +-
.../apache-airflow-providers-apache-beam/index.rst | 8 +-
.../commits.rst | 199 +--
.../index.rst | 4 +-
.../commits.rst | 49 +-
.../index.rst | 2 +-
.../commits.rst | 261 ++--
.../index.rst | 6 +-
.../commits.rst | 26 +-
.../apache-airflow-providers-apache-hdfs/index.rst | 2 +-
.../commits.rst | 363 ++---
.../apache-airflow-providers-apache-hive/index.rst | 15 +-
.../commits.rst | 160 ++-
.../index.rst | 2 +-
.../commits.rst | 194 +--
.../apache-airflow-providers-apache-livy/index.rst | 6 +-
.../commits.rst | 185 +--
docs/apache-airflow-providers-apache-pig/index.rst | 2 +-
.../commits.rst | 166 +--
.../index.rst | 4 +-
.../commits.rst | 294 ++--
.../index.rst | 2 +-
.../commits.rst | 210 +--
.../index.rst | 2 +-
docs/apache-airflow-providers-asana/commits.rst | 67 +-
docs/apache-airflow-providers-asana/index.rst | 2 +-
docs/apache-airflow-providers-celery/commits.rst | 151 +-
docs/apache-airflow-providers-celery/index.rst | 5 +-
docs/apache-airflow-providers-cloudant/commits.rst | 153 +-
docs/apache-airflow-providers-cloudant/index.rst | 2 +-
.../commits.rst | 514 +++----
.../index.rst | 2 +-
.../commits.rst | 281 ++--
docs/apache-airflow-providers-databricks/index.rst | 22 +-
.../operators/copy_into.rst | 6 +
docs/apache-airflow-providers-datadog/commits.rst | 165 ++-
docs/apache-airflow-providers-datadog/index.rst | 2 +-
.../apache-airflow-providers-dbt-cloud/commits.rst | 21 +-
docs/apache-airflow-providers-dbt-cloud/index.rst | 57 +-
docs/apache-airflow-providers-dingding/commits.rst | 211 +--
docs/apache-airflow-providers-dingding/index.rst | 6 +-
docs/apache-airflow-providers-discord/commits.rst | 166 ++-
docs/apache-airflow-providers-discord/index.rst | 6 +-
docs/apache-airflow-providers-docker/commits.rst | 360 ++---
docs/apache-airflow-providers-docker/index.rst | 2 +-
.../commits.rst | 287 ++--
.../index.rst | 2 +-
docs/apache-airflow-providers-exasol/commits.rst | 195 +--
docs/apache-airflow-providers-exasol/index.rst | 4 +-
docs/apache-airflow-providers-facebook/commits.rst | 192 +--
docs/apache-airflow-providers-facebook/index.rst | 2 +-
docs/apache-airflow-providers-ftp/commits.rst | 193 +--
docs/apache-airflow-providers-ftp/index.rst | 2 +-
docs/apache-airflow-providers-github/commits.rst | 21 +-
docs/apache-airflow-providers-github/index.rst | 2 +-
docs/apache-airflow-providers-google/commits.rst | 1480 ++++++++++----------
.../connections/gcp_looker.rst | 75 +
docs/apache-airflow-providers-google/index.rst | 39 +-
.../operators/cloud/dataflow.rst | 2 +-
.../operators/cloud/looker.rst | 70 +
docs/apache-airflow-providers-grpc/commits.rst | 190 +--
docs/apache-airflow-providers-grpc/index.rst | 2 +-
.../apache-airflow-providers-hashicorp/commits.rst | 222 +--
docs/apache-airflow-providers-hashicorp/index.rst | 8 +-
docs/apache-airflow-providers-http/commits.rst | 275 ++--
docs/apache-airflow-providers-http/index.rst | 2 +-
docs/apache-airflow-providers-imap/commits.rst | 199 +--
docs/apache-airflow-providers-imap/index.rst | 2 +-
docs/apache-airflow-providers-influxdb/commits.rst | 27 +-
docs/apache-airflow-providers-influxdb/index.rst | 4 +-
docs/apache-airflow-providers-jdbc/commits.rst | 207 +--
docs/apache-airflow-providers-jdbc/index.rst | 2 +-
docs/apache-airflow-providers-jenkins/commits.rst | 250 ++--
docs/apache-airflow-providers-jenkins/index.rst | 2 +-
docs/apache-airflow-providers-jira/commits.rst | 183 +--
docs/apache-airflow-providers-jira/index.rst | 2 +-
.../commits.rst | 480 ++++---
.../index.rst | 8 +-
.../commits.rst | 203 +--
.../index.rst | 4 +-
.../commits.rst | 62 +-
.../index.rst | 4 +-
.../commits.rst | 228 +--
.../index.rst | 4 +-
docs/apache-airflow-providers-mongo/commits.rst | 198 +--
docs/apache-airflow-providers-mongo/index.rst | 10 +-
docs/apache-airflow-providers-mysql/commits.rst | 293 ++--
docs/apache-airflow-providers-mysql/index.rst | 10 +-
docs/apache-airflow-providers-neo4j/commits.rst | 116 +-
docs/apache-airflow-providers-neo4j/index.rst | 2 +-
docs/apache-airflow-providers-odbc/commits.rst | 153 +-
docs/apache-airflow-providers-odbc/index.rst | 2 +-
docs/apache-airflow-providers-openfaas/commits.rst | 151 +-
docs/apache-airflow-providers-openfaas/index.rst | 2 +-
docs/apache-airflow-providers-opsgenie/commits.rst | 200 +--
docs/apache-airflow-providers-opsgenie/index.rst | 2 +-
docs/apache-airflow-providers-oracle/commits.rst | 250 ++--
docs/apache-airflow-providers-oracle/index.rst | 2 +-
.../apache-airflow-providers-pagerduty/commits.rst | 169 ++-
docs/apache-airflow-providers-pagerduty/index.rst | 4 +-
.../apache-airflow-providers-papermill/commits.rst | 186 +--
docs/apache-airflow-providers-papermill/index.rst | 2 +-
docs/apache-airflow-providers-plexus/commits.rst | 133 +-
docs/apache-airflow-providers-plexus/index.rst | 2 +-
docs/apache-airflow-providers-postgres/commits.rst | 294 ++--
docs/apache-airflow-providers-postgres/index.rst | 6 +-
docs/apache-airflow-providers-presto/commits.rst | 186 +--
docs/apache-airflow-providers-presto/index.rst | 31 +-
docs/apache-airflow-providers-qubole/commits.rst | 278 ++--
docs/apache-airflow-providers-qubole/index.rst | 2 +-
docs/apache-airflow-providers-redis/commits.rst | 178 +--
docs/apache-airflow-providers-redis/index.rst | 2 +-
.../commits.rst | 28 +-
docs/apache-airflow-providers-salesforce/index.rst | 8 +-
docs/apache-airflow-providers-samba/commits.rst | 175 +--
docs/apache-airflow-providers-samba/index.rst | 2 +-
docs/apache-airflow-providers-segment/commits.rst | 176 +--
docs/apache-airflow-providers-segment/index.rst | 2 +-
docs/apache-airflow-providers-sendgrid/commits.rst | 99 +-
docs/apache-airflow-providers-sendgrid/index.rst | 4 +-
docs/apache-airflow-providers-sftp/commits.rst | 288 ++--
docs/apache-airflow-providers-sftp/index.rst | 8 +-
.../commits.rst | 181 +--
.../apache-airflow-providers-singularity/index.rst | 2 +-
docs/apache-airflow-providers-slack/commits.rst | 263 ++--
docs/apache-airflow-providers-slack/index.rst | 8 +-
.../apache-airflow-providers-snowflake/commits.rst | 357 ++---
docs/apache-airflow-providers-snowflake/index.rst | 6 +-
docs/apache-airflow-providers-sqlite/commits.rst | 209 +--
docs/apache-airflow-providers-sqlite/index.rst | 2 +-
docs/apache-airflow-providers-ssh/commits.rst | 289 ++--
docs/apache-airflow-providers-ssh/index.rst | 4 +-
docs/apache-airflow-providers-tableau/commits.rst | 135 +-
docs/apache-airflow-providers-tableau/index.rst | 2 +-
docs/apache-airflow-providers-telegram/commits.rst | 107 +-
docs/apache-airflow-providers-telegram/index.rst | 4 +-
docs/apache-airflow-providers-trino/commits.rst | 78 +-
docs/apache-airflow-providers-trino/index.rst | 32 +-
docs/apache-airflow-providers-vertica/commits.rst | 181 +--
docs/apache-airflow-providers-vertica/index.rst | 2 +-
docs/apache-airflow-providers-yandex/commits.rst | 222 +--
docs/apache-airflow-providers-yandex/index.rst | 4 +-
docs/apache-airflow-providers-zendesk/commits.rst | 168 +--
docs/apache-airflow-providers-zendesk/index.rst | 2 +-
docs/apache-airflow/best-practices.rst | 42 +-
docs/apache-airflow/concepts/dags.rst | 37 +
docs/apache-airflow/executor/index.rst | 1 +
.../apache-airflow/executor/local_kubernetes.rst | 30 +-
docs/apache-airflow/howto/email-config.rst | 58 +-
docs/apache-airflow/howto/set-up-database.rst | 11 +-
docs/apache-airflow/img/email_connection.png | Bin 0 -> 59515 bytes
docs/apache-airflow/installation/upgrading.rst | 6 +-
docs/apache-airflow/migrations-ref.rst | 4 +-
docs/apache-airflow/security/webserver.rst | 2 +-
docs/apache-airflow/usage-cli.rst | 60 +
docs/helm-chart/quick-start.rst | 3 +-
docs/integration-logos/gcp/Cloud-Looker.png | Bin 0 -> 8094 bytes
docs/spelling_wordlist.txt | 23 +
images/candidates_for_backtrack_triggers.png | Bin 0 -> 529103 bytes
kubernetes_tests/test_kubernetes_pod_operator.py | 20 +-
.../test_kubernetes_pod_operator_backcompat.py | 14 +-
.../ci_push_empty_ci_images.sh} | 26 +-
.../ci/images/ci_push_empty_production_images.sh | 38 +
scripts/ci/libraries/_testing.sh | 38 +
scripts/ci/libraries/_verify_image.sh | 16 +-
.../ci_run_single_airflow_test_in_docker.sh | 37 +-
scripts/ci/testing/run_downgrade_test.sh | 73 +
setup.cfg | 2 -
setup.py | 28 +-
tests/{models => always}/test_connection.py | 6 +-
tests/api/common/test_mark_tasks.py | 28 +-
tests/api_connexion/endpoints/test_log_endpoint.py | 2 +-
.../test_mapped_task_instance_endpoint.py | 256 ++++
.../endpoints/test_task_instance_endpoint.py | 25 +-
.../api_connexion/endpoints/test_xcom_endpoint.py | 2 +-
.../schemas/test_task_instance_schema.py | 4 +-
tests/api_connexion/schemas/test_xcom_schema.py | 2 +-
tests/{timetables => callbacks}/__init__.py | 0
tests/callbacks/test_callback_requests.py | 70 +
tests/cli/commands/test_connection_command.py | 376 ++---
tests/cli/commands/test_user_command.py | 69 +-
tests/core/test_config_templates.py | 1 +
tests/dag_processing/test_manager.py | 139 +-
tests/dag_processing/test_processor.py | 10 +-
tests/executors/test_celery_executor.py | 4 +-
tests/executors/test_dask_executor.py | 24 +-
tests/executors/test_local_kubernetes_executor.py | 62 +
tests/jobs/test_backfill_job.py | 20 +-
tests/jobs/test_scheduler_job.py | 28 +-
tests/kubernetes/test_pod_generator.py | 37 +
tests/models/test_baseoperator.py | 9 +-
tests/models/test_dag.py | 2 +-
tests/models/test_param.py | 22 +-
tests/models/test_taskinstance.py | 4 +-
tests/models/test_xcom.py | 8 +-
tests/models/test_xcom_arg.py | 14 +
tests/providers/alibaba/cloud/hooks/test_oss.py | 123 +-
.../providers/alibaba/cloud/utils/oss_mock.py | 31 +-
.../amazon/aws/hooks/test_emr_containers.py | 42 +
tests/providers/amazon/aws/hooks/test_kinesis.py | 2 +-
.../amazon/aws/hooks/test_redshift_data.py | 19 +-
.../providers/amazon/aws/operators/test_athena.py | 6 +-
tests/providers/amazon/aws/operators/test_ecs.py | 34 +-
tests/providers/amazon/aws/operators/test_rds.py | 452 ++++++
.../amazon/aws/operators/test_redshift_data.py | 108 ++
tests/providers/amazon/aws/sensors/test_rds.py | 227 +++
.../amazon/aws/transfers/test_gcs_to_s3.py | 29 +
tests/providers/apache/hive/operators/test_hive.py | 4 +-
.../kubernetes/operators/test_kubernetes_pod.py | 37 +-
.../databricks/operators/test_databricks_sql.py | 86 ++
tests/providers/ftp/hooks/test_ftp.py | 14 +
tests/providers/google/cloud/hooks/test_gcs.py | 10 +-
tests/providers/google/cloud/hooks/test_looker.py | 111 ++
.../google/cloud/operators/test_looker.py | 148 ++
.../providers/google/cloud/sensors/test_looker.py | 107 ++
.../google/cloud/transfers/test_gcs_to_bigquery.py | 2 +
.../azure/hooks/test_azure_data_factory.py | 68 +-
tests/providers/microsoft/azure/hooks/test_wasb.py | 89 +-
.../azure/transfers/test_local_to_wasb.py | 20 +-
.../microsoft/azure/transfers/test_sftp_to_wasb.py | 29 +-
tests/sensors/test_base.py | 10 +-
tests/serialization/test_dag_serialization.py | 37 +-
tests/test_utils/api_connexion_utils.py | 5 +
tests/test_utils/db.py | 6 +
tests/ti_deps/deps/test_dagrun_id_dep.py | 16 +-
tests/www/views/test_views_acl.py | 3 +-
tests/www/views/test_views_tasks.py | 52 +-
557 files changed, 19645 insertions(+), 9621 deletions(-)
copy airflow/{contrib/hooks/bigquery_hook.py => callbacks/database_callback_sink.py} (55%)
copy airflow/{providers/sendgrid/provider.yaml => compat/sqlalchemy.py} (64%)
create mode 100644 airflow/example_dags/example_local_kubernetes_executor.py
copy airflow/executors/{celery_kubernetes_executor.py => local_kubernetes_executor.py} (67%)
create mode 100644 airflow/migrations/utils.py
copy airflow/migrations/versions/{97cdd93827b8_add_queued_at_column_to_dagrun_table.py => c97c2ab6aa23_add_callback_request_table.py} (53%)
create mode 100644 airflow/models/db_callback_request.py
create mode 100644 airflow/providers/amazon/aws/example_dags/example_batch.py
copy airflow/providers/{docker/example_dags/example_docker.py => amazon/aws/example_dags/example_dynamodb_to_s3.py} (52%)
create mode 100644 airflow/providers/amazon/aws/example_dags/example_dynamodb_to_s3_segmented.py
create mode 100644 airflow/providers/amazon/aws/example_dags/example_rds.py
create mode 100644 airflow/providers/amazon/aws/example_dags/example_redshift_data_execute_sql.py
rename airflow/providers/amazon/aws/example_dags/{example_redshift.py => example_redshift_sql.py} (60%)
copy airflow/providers/{docker/example_dags/example_docker.py => amazon/aws/example_dags/example_redshift_to_s3.py} (52%)
create mode 100644 airflow/providers/amazon/aws/hooks/redshift_data.py
create mode 100644 airflow/providers/amazon/aws/operators/rds.py
create mode 100644 airflow/providers/amazon/aws/operators/redshift_data.py
create mode 100644 airflow/providers/amazon/aws/sensors/rds.py
copy airflow/providers/{sendgrid/provider.yaml => amazon/aws/utils/rds.py} (81%)
create mode 100644 airflow/providers/google/cloud/example_dags/example_looker.py
create mode 100644 airflow/providers/google/cloud/hooks/looker.py
create mode 100644 airflow/providers/google/cloud/operators/looker.py
create mode 100644 airflow/providers/google/cloud/sensors/looker.py
rename airflow/ti_deps/deps/{dagrun_id_dep.py => dagrun_backfill_dep.py} (84%)
create mode 100644 dev/TRACKING_BACKTRACKING_ISSUES.md
copy {tests/timetables => dev/breeze/src/airflow_breeze/shell}/__init__.py (100%)
create mode 100644 dev/breeze/src/airflow_breeze/shell/enter_shell.py
create mode 100644 dev/breeze/src/airflow_breeze/shell/shell_builder.py
create mode 100644 dev/breeze/src/airflow_breeze/utils/host_info_utils.py
create mode 100644 dev/breeze/src/airflow_ci/find_newer_dependencies.py
create mode 100644 dev/breeze/tests/test_docker_command_utils.py
copy airflow/providers/sendgrid/provider.yaml => dev/breeze/tests/test_host_info_utils.py (79%)
create mode 100644 dev/breeze/tests/test_run_utils.py
create mode 100644 docs/apache-airflow-providers-amazon/operators/batch.rst
create mode 100644 docs/apache-airflow-providers-amazon/operators/rds.rst
create mode 100644 docs/apache-airflow-providers-amazon/operators/redshift_data.rst
create mode 100644 docs/apache-airflow-providers-amazon/operators/transfer/dynamodb_to_s3.rst
create mode 100644 docs/apache-airflow-providers-amazon/operators/transfer/redshift_to_s3.rst
rename docs/apache-airflow-providers-amazon/operators/{ => transfer}/s3_to_redshift.rst (50%)
create mode 100644 docs/apache-airflow-providers-google/connections/gcp_looker.rst
create mode 100644 docs/apache-airflow-providers-google/operators/cloud/looker.rst
copy airflow/providers/influxdb/CHANGELOG.rst => docs/apache-airflow/executor/local_kubernetes.rst (59%)
create mode 100644 docs/apache-airflow/img/email_connection.png
create mode 100644 docs/integration-logos/gcp/Cloud-Looker.png
create mode 100644 images/candidates_for_backtrack_triggers.png
copy scripts/ci/{libraries/_verify_image.sh => images/ci_push_empty_ci_images.sh} (51%)
mode change 100644 => 100755
create mode 100755 scripts/ci/images/ci_push_empty_production_images.sh
create mode 100755 scripts/ci/testing/run_downgrade_test.sh
rename tests/{models => always}/test_connection.py (99%)
create mode 100644 tests/api_connexion/endpoints/test_mapped_task_instance_endpoint.py
copy tests/{timetables => callbacks}/__init__.py (100%)
create mode 100644 tests/callbacks/test_callback_requests.py
create mode 100644 tests/executors/test_local_kubernetes_executor.py
copy airflow/contrib/hooks/bigquery_hook.py => tests/providers/alibaba/cloud/utils/oss_mock.py (60%)
copy airflow/providers/sendgrid/provider.yaml => tests/providers/amazon/aws/hooks/test_redshift_data.py (69%)
create mode 100644 tests/providers/amazon/aws/operators/test_rds.py
create mode 100644 tests/providers/amazon/aws/operators/test_redshift_data.py
create mode 100644 tests/providers/amazon/aws/sensors/test_rds.py
create mode 100644 tests/providers/google/cloud/hooks/test_looker.py
create mode 100644 tests/providers/google/cloud/operators/test_looker.py
create mode 100644 tests/providers/google/cloud/sensors/test_looker.py
[airflow] 01/09: make UI and tree work with mapped tasks
Posted by bb...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bbovenzi pushed a commit to branch mapped-task-drawer
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit db7a659fbec0c74e5f5ea398d59e58f542778452
Author: Brent Bovenzi <br...@gmail.com>
AuthorDate: Wed Feb 16 11:27:29 2022 -0500
make UI and tree work with mapped tasks
---
airflow/www/static/js/tree/InstanceTooltip.jsx | 15 +++
airflow/www/utils.py | 136 ++++++++++++-------------
2 files changed, 78 insertions(+), 73 deletions(-)
diff --git a/airflow/www/static/js/tree/InstanceTooltip.jsx b/airflow/www/static/js/tree/InstanceTooltip.jsx
index e22189d..bc0d58c 100644
--- a/airflow/www/static/js/tree/InstanceTooltip.jsx
+++ b/airflow/www/static/js/tree/InstanceTooltip.jsx
@@ -25,6 +25,21 @@ import { Box, Text } from '@chakra-ui/react';
import { formatDateTime, getDuration, formatDuration } from '../datetime_utils';
import { finalStatesMap } from '../utils';
+const STATES = [
+ ['success', 0],
+ ['failed', 0],
+ ['upstream_failed', 0],
+ ['up_for_retry', 0],
+ ['up_for_reschedule', 0],
+ ['running', 0],
+ ['deferred', 0],
+ ['sensing', 0],
+ ['queued', 0],
+ ['scheduled', 0],
+ ['skipped', 0],
+ ['no_status', 0],
+];
+
const InstanceTooltip = ({
group,
instance: {
diff --git a/airflow/www/utils.py b/airflow/www/utils.py
index b1f5e0d..ed765d4 100644
--- a/airflow/www/utils.py
+++ b/airflow/www/utils.py
@@ -43,9 +43,8 @@ from airflow.models import errors
from airflow.models.taskinstance import TaskInstance
from airflow.utils import timezone
from airflow.utils.code_utils import get_python_source
-from airflow.utils.helpers import alchemy_to_dict
from airflow.utils.json import AirflowJsonEncoder
-from airflow.utils.state import State, TaskInstanceState
+from airflow.utils.state import State
from airflow.www.forms import DateTimeWithTimezoneField
from airflow.www.widgets import AirflowDateTimePickerWidget
@@ -56,82 +55,13 @@ def datetime_to_string(value: Optional[DateTime]) -> Optional[str]:
return value.isoformat()
-def get_mapped_instances(task_instance, session):
- return (
- session.query(TaskInstance)
- .filter(
- TaskInstance.dag_id == task_instance.dag_id,
- TaskInstance.run_id == task_instance.run_id,
- TaskInstance.task_id == task_instance.task_id,
- TaskInstance.map_index >= 0,
- )
- .all()
- )
-
-
-def get_instance_with_map(task_instance, session):
- if task_instance.map_index == -1:
- return alchemy_to_dict(task_instance)
- mapped_instances = get_mapped_instances(task_instance, session)
- return get_mapped_summary(task_instance, mapped_instances)
-
-
-def get_mapped_summary(parent_instance, task_instances):
- priority = [
- TaskInstanceState.FAILED,
- TaskInstanceState.UPSTREAM_FAILED,
- TaskInstanceState.UP_FOR_RETRY,
- TaskInstanceState.UP_FOR_RESCHEDULE,
- TaskInstanceState.QUEUED,
- TaskInstanceState.SCHEDULED,
- TaskInstanceState.DEFERRED,
- TaskInstanceState.SENSING,
- TaskInstanceState.RUNNING,
- TaskInstanceState.SHUTDOWN,
- TaskInstanceState.RESTARTING,
- TaskInstanceState.REMOVED,
- TaskInstanceState.SUCCESS,
- TaskInstanceState.SKIPPED,
- ]
-
- mapped_states = [ti.state for ti in task_instances]
-
- group_state = None
- for state in priority:
- if state in mapped_states:
- group_state = state
- break
-
- group_start_date = datetime_to_string(
- min((ti.start_date for ti in task_instances if ti.start_date), default=None)
- )
- group_end_date = datetime_to_string(
- max((ti.end_date for ti in task_instances if ti.end_date), default=None)
- )
-
- return {
- 'task_id': parent_instance.task_id,
- 'run_id': parent_instance.run_id,
- 'state': group_state,
- 'start_date': group_start_date,
- 'end_date': group_end_date,
- 'mapped_states': mapped_states,
- 'operator': parent_instance.operator,
- 'execution_date': datetime_to_string(parent_instance.execution_date),
- 'try_number': parent_instance.try_number,
- }
-
-
def encode_ti(
- task_instance: Optional[TaskInstance], is_mapped: Optional[bool], session: Optional[Session]
+ task_instance: Optional[TaskInstance], is_mapped: Optional[bool], session: Session
) -> Optional[Dict[str, Any]]:
if not task_instance:
return None
- if is_mapped:
- return get_mapped_summary(task_instance, task_instances=get_mapped_instances(task_instance, session))
-
- return {
+ summary = {
'task_id': task_instance.task_id,
'dag_id': task_instance.dag_id,
'run_id': task_instance.run_id,
@@ -144,6 +74,66 @@ def encode_ti(
'try_number': task_instance.try_number,
}
+ def get_mapped_summary(task_instances):
+ priority = [
+ 'failed',
+ 'upstream_failed',
+ 'up_for_retry',
+ 'up_for_reschedule',
+ 'queued',
+ 'scheduled',
+ 'deferred',
+ 'sensing',
+ 'running',
+ 'shutdown',
+ 'restarting',
+ 'removed',
+ 'no_status',
+ 'success',
+ 'skipped',
+ ]
+
+ mapped_states = [ti.state for ti in task_instances]
+
+ group_state = None
+ for state in priority:
+ if state in mapped_states:
+ group_state = state
+ break
+
+ group_start_date = datetime_to_string(
+ min((ti.start_date for ti in task_instances if ti.start_date), default=None)
+ )
+ group_end_date = datetime_to_string(
+ max((ti.end_date for ti in task_instances if ti.end_date), default=None)
+ )
+
+ return {
+ 'task_id': task_instance.task_id,
+ 'run_id': task_instance.run_id,
+ 'state': group_state,
+ 'start_date': group_start_date,
+ 'end_date': group_end_date,
+ 'mapped_states': mapped_states,
+ 'operator': task_instance.operator,
+ 'execution_date': datetime_to_string(task_instance.execution_date),
+ 'try_number': task_instance.try_number,
+ }
+
+ if is_mapped:
+ return get_mapped_summary(
+ session.query(TaskInstance)
+ .filter(
+ TaskInstance.dag_id == task_instance.dag_id,
+ TaskInstance.run_id == task_instance.run_id,
+ TaskInstance.task_id == task_instance.task_id,
+ TaskInstance.map_index >= 0,
+ )
+ .all()
+ )
+
+ return summary
+
def encode_dag_run(dag_run: Optional[models.DagRun]) -> Optional[Dict[str, Any]]:
if not dag_run:
[airflow] 07/09: add tooltip info to details
Posted by bb...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bbovenzi pushed a commit to branch mapped-task-drawer
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit e5764b3ad7dc34f1c8121625e180a7ea252dc8e4
Author: Brent Bovenzi <br...@gmail.com>
AuthorDate: Tue Mar 1 18:19:27 2022 -0500
add tooltip info to details
---
.../www/static/js/tree/details/content/DagRun.jsx | 79 ++++++++++-
.../js/tree/details/content/TaskInstance.jsx | 146 ++++++++++++++++++++-
airflow/www/static/js/tree/details/index.jsx | 2 +-
3 files changed, 217 insertions(+), 10 deletions(-)
diff --git a/airflow/www/static/js/tree/details/content/DagRun.jsx b/airflow/www/static/js/tree/details/content/DagRun.jsx
index 58e4b92..abcf908 100644
--- a/airflow/www/static/js/tree/details/content/DagRun.jsx
+++ b/airflow/www/static/js/tree/details/content/DagRun.jsx
@@ -17,15 +17,86 @@
* under the License.
*/
+/* global moment */
+
import React from 'react';
import {
Text,
+ Box,
} from '@chakra-ui/react';
+import { MdPlayArrow } from 'react-icons/md';
+
+import { formatDateTime, formatDuration } from '../../../datetime_utils';
-const DagRun = () => (
- <>
- <Text>dag run details</Text>
- </>
+const DagRun = ({
+ dagRun: {
+ state, runId, duration, dataIntervalStart, dataIntervalEnd, startDate, endDate, runType,
+ },
+}) => (
+ <Box fontSize="12px" py="4px">
+ <Text>
+ <Text as="strong">Status:</Text>
+ {' '}
+ {state || 'no status'}
+ </Text>
+ <br />
+ <Text whiteSpace="nowrap">
+ Run Id:
+ {' '}
+ {runId}
+ </Text>
+ <Text>
+ Run Type:
+ {' '}
+ {runType === 'manual' && <MdPlayArrow style={{ display: 'inline' }} />}
+ {runType}
+ </Text>
+ <Text>
+ Duration:
+ {' '}
+ {formatDuration(duration)}
+ </Text>
+ <br />
+ <Text as="strong">Data Interval:</Text>
+ <Text>
+ Start:
+ {' '}
+ {formatDateTime(dataIntervalStart)}
+ </Text>
+ <Text>
+ End:
+ {' '}
+ {formatDateTime(dataIntervalEnd)}
+ </Text>
+ <br />
+ <Text as="strong">UTC</Text>
+ <Text>
+ Started:
+ {' '}
+ {formatDateTime(moment.utc(startDate))}
+ </Text>
+ <Text>
+ Ended:
+ {' '}
+ {endDate && formatDateTime(moment.utc(endDate))}
+ </Text>
+ <br />
+ <Text as="strong">
+ Local:
+ {' '}
+ {moment().format('Z')}
+ </Text>
+ <Text>
+ Started:
+ {' '}
+ {formatDateTime(startDate)}
+ </Text>
+ <Text>
+ Ended:
+ {' '}
+ {endDate && formatDateTime(endDate)}
+ </Text>
+ </Box>
);
export default DagRun;
diff --git a/airflow/www/static/js/tree/details/content/TaskInstance.jsx b/airflow/www/static/js/tree/details/content/TaskInstance.jsx
index 4ab46e5..bc60737 100644
--- a/airflow/www/static/js/tree/details/content/TaskInstance.jsx
+++ b/airflow/www/static/js/tree/details/content/TaskInstance.jsx
@@ -17,15 +17,151 @@
* under the License.
*/
+/* global moment */
+
import React from 'react';
import {
Text,
+ Box,
} from '@chakra-ui/react';
-const TaskInstance = () => (
- <>
- <Text>task instance details</Text>
- </>
-);
+import { formatDateTime, getDuration, formatDuration } from '../../../datetime_utils';
+
+const TaskInstance = ({
+ instance: {
+ duration, operator, startDate, endDate, state, taskId, runId, // mappedStates,
+ },
+}) => {
+ const isGroup = false; // !!group.children;
+ const groupSummary = [];
+ // const mapSummary = [];
+
+ // if (isGroup) {
+ // const numMap = finalStatesMap();
+ // group.children.forEach((child) => {
+ // const taskInstance = child.instances.find((ti) => ti.runId === runId);
+ // if (taskInstance) {
+ // const stateKey = taskInstance.state == null ? 'no_status' : taskInstance.state;
+ // if (numMap.has(stateKey)) numMap.set(stateKey, numMap.get(stateKey) + 1);
+ // }
+ // });
+ // numMap.forEach((key, val) => {
+ // if (key > 0) {
+ // groupSummary.push(
+ // // eslint-disable-next-line react/no-array-index-key
+ // <Text key={val} ml="10px">
+ // {val}
+ // {': '}
+ // {key}
+ // </Text>,
+ // );
+ // }
+ // });
+ // }
+
+ // if (group.isMapped && mappedStates) {
+ // const numMap = finalStatesMap();
+ // mappedStates.forEach((s) => {
+ // const stateKey = s || 'no_status';
+ // if (numMap.has(stateKey)) numMap.set(stateKey, numMap.get(stateKey) + 1);
+ // });
+ // numMap.forEach((key, val) => {
+ // if (key > 0) {
+ // mapSummary.push(
+ // // eslint-disable-next-line react/no-array-index-key
+ // <Text key={val} ml="10px">
+ // {val}
+ // {': '}
+ // {key}
+ // </Text>,
+ // );
+ // }
+ // });
+ // }
+
+ const taskIdTitle = isGroup ? 'Task Group Id: ' : 'Task Id: ';
+
+ return (
+ <Box fontSize="12px" py="4px">
+ {/* {group.tooltip && (
+ <Text>{group.tooltip}</Text>
+ )} */}
+ <Text>
+ <Text as="strong">Status:</Text>
+ {' '}
+ {state || 'no status'}
+ </Text>
+ {isGroup && (
+ <>
+ <br />
+ <Text as="strong">Group Summary</Text>
+ {groupSummary}
+ </>
+ )}
+ {/* {group.isMapped && (
+ <>
+ <br />
+ <Text as="strong">
+ {mappedStates.length}
+ {' '}
+ {mappedStates.length === 1 ? 'Task ' : 'Tasks '}
+ Mapped
+ </Text>
+ {mapSummary}
+ </>
+ )} */}
+ <br />
+ <Text>
+ {taskIdTitle}
+ {taskId}
+ </Text>
+ <Text whiteSpace="nowrap">
+ Run Id:
+ {' '}
+ {runId}
+ </Text>
+ {operator && (
+ <Text>
+ Operator:
+ {' '}
+ {operator}
+ </Text>
+ )}
+ <Text>
+ Duration:
+ {' '}
+ {formatDuration(duration || getDuration(startDate, endDate))}
+ </Text>
+ <br />
+ <Text as="strong">UTC</Text>
+ <Text>
+ Started:
+ {' '}
+ {startDate && formatDateTime(moment.utc(startDate))}
+ </Text>
+ <Text>
+ Ended:
+ {' '}
+ {endDate && formatDateTime(moment.utc(endDate))}
+ </Text>
+ <br />
+ <Text as="strong">
+ Local:
+ {' '}
+ {moment().format('Z')}
+ </Text>
+ <Text>
+ Started:
+ {' '}
+ {startDate && formatDateTime(startDate)}
+ </Text>
+ <Text>
+ Ended:
+ {' '}
+ {endDate && formatDateTime(endDate)}
+ </Text>
+ </Box>
+ );
+};
export default TaskInstance;
diff --git a/airflow/www/static/js/tree/details/index.jsx b/airflow/www/static/js/tree/details/index.jsx
index b8f960e..31610a7 100644
--- a/airflow/www/static/js/tree/details/index.jsx
+++ b/airflow/www/static/js/tree/details/index.jsx
@@ -39,7 +39,7 @@ const Details = ({
<Box>
{/* TODO: get full instance data from the API */}
{!selected.runId && !selected.taskId && <DagContent />}
- {selected.runId && !selected.taskId && <DagRunContent />}
+ {selected.runId && !selected.taskId && <DagRunContent dagRun={selected} />}
{selected.taskId && <TaskInstanceContent instance={selected} />}
</Box>
</Flex>
[airflow] 03/09: reformat grid background colors
Posted by bb...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bbovenzi pushed a commit to branch mapped-task-drawer
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 5390dbc75db3128c11a937ade63e19406e3caa6c
Author: Brent Bovenzi <br...@gmail.com>
AuthorDate: Fri Feb 25 11:58:23 2022 -0500
reformat grid background colors
---
airflow/www/static/js/tree/InstanceTooltip.jsx | 16 ++---
airflow/www/static/js/tree/SidePanel.jsx | 45 +++++++++++--
airflow/www/static/js/tree/StatusBox.jsx | 19 +++---
airflow/www/static/js/tree/Tree.jsx | 28 ++++----
airflow/www/static/js/tree/dagRuns/Bar.jsx | 9 ++-
airflow/www/static/js/tree/dagRuns/index.jsx | 3 +-
airflow/www/static/js/tree/renderTaskRows.jsx | 88 ++++++++++++++------------
7 files changed, 128 insertions(+), 80 deletions(-)
diff --git a/airflow/www/static/js/tree/InstanceTooltip.jsx b/airflow/www/static/js/tree/InstanceTooltip.jsx
index bc0d58c..b0d0584 100644
--- a/airflow/www/static/js/tree/InstanceTooltip.jsx
+++ b/airflow/www/static/js/tree/InstanceTooltip.jsx
@@ -125,29 +125,29 @@ const InstanceTooltip = ({
</>
)}
<br />
- <Text>
+ {/* <Text>
{taskIdTitle}
{taskId}
- </Text>
- <Text whiteSpace="nowrap">
+ </Text> */}
+ {/* <Text whiteSpace="nowrap">
Run Id:
{' '}
{runId}
- </Text>
- {operator && (
+ </Text> */}
+ {/* {operator && (
<Text>
Operator:
{' '}
{operator}
</Text>
- )}
+ )} */}
<Text>
Duration:
{' '}
{formatDuration(duration || getDuration(startDate, endDate))}
</Text>
<br />
- <Text as="strong">UTC</Text>
+ {/* <Text as="strong">UTC</Text>
<Text>
Started:
{' '}
@@ -173,7 +173,7 @@ const InstanceTooltip = ({
Ended:
{' '}
{endDate && formatDateTime(endDate)}
- </Text>
+ </Text> */}
</Box>
);
};
diff --git a/airflow/www/static/js/tree/SidePanel.jsx b/airflow/www/static/js/tree/SidePanel.jsx
index 3515f1c..7a21eba 100644
--- a/airflow/www/static/js/tree/SidePanel.jsx
+++ b/airflow/www/static/js/tree/SidePanel.jsx
@@ -21,29 +21,60 @@
import React from 'react';
import {
- Box,
chakra,
Flex,
Text,
+ useDisclosure,
+ CloseButton,
+ Button,
+ IconButton,
} from '@chakra-ui/react';
+import { MdKeyboardArrowLeft, MdKeyboardArrowRight } from 'react-icons/md';
import { formatDateTime } from '../datetime_utils';
-const SidePanel = ({ instance: { runId, taskId, executionDate }, isOpen }) => (
- <Box bg="gray.200" maxWidth={isOpen ? 300 : 0} minWidth={isOpen ? 300 : 0} transition="all 0.5s" position="relative" overflow="hidden">
- <Flex right={isOpen ? 0 : -300} top={0} transition="right 0.5s, max-width 0.5s" width={300} flexDirection="column" m={2}>
- <Text as="h4">
+const SidePanel = ({ instance: { runId, taskId, executionDate } }) => {
+ const { isOpen, onOpen, onClose } = useDisclosure();
+ if (!isOpen) {
+ return (
+ <IconButton
+ ml={2}
+ icon={<MdKeyboardArrowLeft size={18} />}
+ aria-label="Open Details Panel"
+ onClick={onOpen}
+ />
+ );
+ }
+ const title = runId && taskId
+ ? (
+ <>
<chakra.span>Task Instance: </chakra.span>
<chakra.b>{taskId}</chakra.b>
<chakra.span> at </chakra.span>
<chakra.b>{formatDateTime(moment.utc(executionDate))}</chakra.b>
+ </>
+ )
+ : (
+ <chakra.span>Dag Details: </chakra.span>
+ );
+
+ return (
+ <Flex bg="gray.200" maxWidth={300} minWidth={300} flexDirection="column" p={3}>
+ <IconButton
+ ml={2}
+ icon={<MdKeyboardArrowRight size={18} />}
+ aria-label="Close Details Panel"
+ onClick={onClose}
+ />
+ <Text as="h4">
+ {title}
</Text>
<Text>
{/* eslint-disable-next-line max-len */}
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Nunc vel risus commodo viverra maecenas accumsan. Ut porttitor leo a diam sollicitudin tempor id eu. Molestie at elementum eu facilisis sed odio morbi quis commodo. Facilisis leo vel fringilla est ullamcorper eget nulla facilisi etiam. Est sit amet facilisis magna etiam tempor orci eu. Id semper risus in hendrerit gravida rutrum. Ac odio tempor orci dapibus [...]
</Text>
</Flex>
- </Box>
-);
+ );
+};
export default SidePanel;
diff --git a/airflow/www/static/js/tree/StatusBox.jsx b/airflow/www/static/js/tree/StatusBox.jsx
index dce0fda..b4abc93 100644
--- a/airflow/www/static/js/tree/StatusBox.jsx
+++ b/airflow/www/static/js/tree/StatusBox.jsx
@@ -24,9 +24,10 @@ import {
Flex,
Box,
Tooltip,
+ useTheme,
} from '@chakra-ui/react';
-import { callModal } from '../dag';
+// import { callModal } from '../dag';
import InstanceTooltip from './InstanceTooltip';
const StatusBox = ({
@@ -35,21 +36,19 @@ const StatusBox = ({
const {
executionDate, taskId, tryNumber = 0, operator, runId,
} = instance;
+ const { colors } = useTheme();
+ const hoverBlue = `${colors.blue[100]}50`;
- const onOpenModal = () => executionDate && callModal(taskId, executionDate, extraLinks, tryNumber, operator === 'SubDagOperator' || undefined, runId);
- const onClick = () => {
- if (group.isMapped) {
- onSelectInstance(instance);
- } else {
- onSelectInstance({});
- onOpenModal();
- }
+ // const onOpenModal = () => executionDate && callModal(taskId, executionDate, extraLinks, tryNumber, operator === 'SubDagOperator' || undefined, runId);
+ const onClick = (e) => {
+ e.stopPropagation();
+ onSelectInstance(instance);
};
// Fetch the corresponding column element and set its background color when hovering
const onMouseOver = () => {
[...containerRef.current.getElementsByClassName(`js-${runId}`)]
- .forEach((e) => { e.style.backgroundColor = 'rgba(113, 128, 150, 0.1)'; });
+ .forEach((e) => { e.style.backgroundColor = hoverBlue; });
};
const onMouseLeave = () => {
[...containerRef.current.getElementsByClassName(`js-${runId}`)]
diff --git a/airflow/www/static/js/tree/Tree.jsx b/airflow/www/static/js/tree/Tree.jsx
index 095adbd..ff1402f 100644
--- a/airflow/www/static/js/tree/Tree.jsx
+++ b/airflow/www/static/js/tree/Tree.jsx
@@ -59,7 +59,7 @@ const Tree = () => {
return (
<Box position="relative" ref={containerRef}>
- <FormControl display="flex" alignItems="center" justifyContent="flex-end" width="100%">
+ <FormControl display="flex" alignItems="center" justifyContent="flex-end" width="100%" mb={2}>
{isRefreshOn && <Spinner color="blue.500" speed="1s" mr="4px" />}
<FormLabel htmlFor="auto-refresh" mb={0} fontSize="12px" fontWeight="normal">
Auto-refresh
@@ -68,19 +68,21 @@ const Tree = () => {
</FormControl>
<Text transform="rotate(-90deg)" position="absolute" left="-6px" top="130px">Runs</Text>
<Text transform="rotate(-90deg)" position="absolute" left="-6px" top="190px">Tasks</Text>
- <Box pl="24px">
+ <Box pl="24px" height="100%" onClick={() => setSelectedInstance({})}>
<Flex position="relative" flexDirection="row" justifyContent="space-between" overflow="hidden">
- <Table mr="24px" overflowX="auto" ref={scrollRef} height={0}>
- <Thead>
- <DagRuns containerRef={containerRef} />
- </Thead>
- <Tbody>
- {renderTaskRows({
- task: groups, containerRef, onSelectInstance,
- })}
- </Tbody>
- </Table>
- <SidePanel isOpen={!!runId} instance={selectedInstance} />
+ <Box mr="12px" pb="12px" overflowX="auto" ref={scrollRef} maxWidth="60vw">
+ <Table height={0}>
+ <Thead>
+ <DagRuns containerRef={containerRef} selectedInstance={selectedInstance} />
+ </Thead>
+ <Tbody>
+ {renderTaskRows({
+ task: groups, containerRef, onSelectInstance, selectedInstance,
+ })}
+ </Tbody>
+ </Table>
+ </Box>
+ <SidePanel instance={selectedInstance} />
</Flex>
</Box>
</Box>
diff --git a/airflow/www/static/js/tree/dagRuns/Bar.jsx b/airflow/www/static/js/tree/dagRuns/Bar.jsx
index a0fa5b1..4dbe20d 100644
--- a/airflow/www/static/js/tree/dagRuns/Bar.jsx
+++ b/airflow/www/static/js/tree/dagRuns/Bar.jsx
@@ -26,6 +26,7 @@ import {
Tooltip,
Text,
VStack,
+ useTheme,
} from '@chakra-ui/react';
import { MdPlayArrow } from 'react-icons/md';
@@ -35,13 +36,16 @@ import { callModalDag } from '../../dag';
const BAR_HEIGHT = 100;
const DagRunBar = ({
- run, max, index, totalRuns, containerRef,
+ run, max, index, totalRuns, containerRef, selectedInstance,
}) => {
+ const { colors } = useTheme();
+ const hoverBlue = `${colors.blue[100]}50`;
let highlightHeight = '100%';
if (containerRef && containerRef.current) {
const table = containerRef.current.getElementsByTagName('tbody')[0];
highlightHeight = table.offsetHeight + BAR_HEIGHT;
}
+ const isSelected = run.runId === selectedInstance.runId;
return (
<Box position="relative">
<Flex
@@ -93,7 +97,8 @@ const DagRunBar = ({
top="1px"
height={highlightHeight}
className={`js-${run.runId}`}
- _peerHover={{ backgroundColor: 'rgba(113, 128, 150, 0.1)' }}
+ bg={isSelected ? 'blue.100' : undefined}
+ _peerHover={!isSelected && { backgroundColor: hoverBlue }}
zIndex={0}
transition="background-color 0.2s"
/>
diff --git a/airflow/www/static/js/tree/dagRuns/index.jsx b/airflow/www/static/js/tree/dagRuns/index.jsx
index a0a4931..7de949f 100644
--- a/airflow/www/static/js/tree/dagRuns/index.jsx
+++ b/airflow/www/static/js/tree/dagRuns/index.jsx
@@ -36,7 +36,7 @@ const DurationTick = ({ children, ...rest }) => (
</Text>
);
-const DagRuns = ({ containerRef }) => {
+const DagRuns = ({ containerRef, selectedInstance }) => {
const { data: { dagRuns = [] } } = useTreeData();
const durations = [];
const runs = dagRuns.map((dagRun) => {
@@ -97,6 +97,7 @@ const DagRuns = ({ containerRef }) => {
index={i}
totalRuns={runs.length}
containerRef={containerRef}
+ selectedInstance={selectedInstance}
/>
))}
</Flex>
diff --git a/airflow/www/static/js/tree/renderTaskRows.jsx b/airflow/www/static/js/tree/renderTaskRows.jsx
index f5a7ba0..7eff252 100644
--- a/airflow/www/static/js/tree/renderTaskRows.jsx
+++ b/airflow/www/static/js/tree/renderTaskRows.jsx
@@ -28,6 +28,7 @@ import {
Flex,
useDisclosure,
Collapse,
+ useTheme,
} from '@chakra-ui/react';
import { FiChevronUp, FiChevronDown } from 'react-icons/fi';
@@ -40,7 +41,7 @@ import { getMetaValue } from '../utils';
const dagId = getMetaValue('dag_id');
const renderTaskRows = ({
- task, containerRef, level = 0, isParentOpen, onSelectInstance,
+ task, containerRef, level = 0, isParentOpen, onSelectInstance, selectedInstance,
}) => task.children.map((t) => (
<Row
key={t.id}
@@ -50,40 +51,37 @@ const renderTaskRows = ({
prevTaskId={task.id}
isParentOpen={isParentOpen}
onSelectInstance={onSelectInstance}
+ selectedInstance={selectedInstance}
/>
));
const TaskName = ({
isGroup = false, isMapped = false, onToggle, isOpen, level, taskName,
}) => (
- <Box _groupHover={{ backgroundColor: 'rgba(113, 128, 150, 0.1)' }} transition="background-color 0.2s">
- <Flex
- as={isGroup ? 'button' : 'div'}
- onClick={() => isGroup && onToggle()}
- color={level > 4 && 'white'}
- aria-label={taskName}
- title={taskName}
- mr={4}
- width="100%"
- backgroundColor={`rgba(203, 213, 224, ${0.25 * level})`}
- alignItems="center"
+ <Flex
+ as={isGroup ? 'button' : 'div'}
+ onClick={() => isGroup && onToggle()}
+ aria-label={taskName}
+ title={taskName}
+ mr={4}
+ width="100%"
+ alignItems="center"
+ >
+ <Text
+ display="inline"
+ fontSize="12px"
+ ml={level * 4 + 4}
+ isTruncated
>
- <Text
- display="inline"
- fontSize="12px"
- ml={level * 4 + 4}
- isTruncated
- >
- {taskName}
- {isMapped && (
- ' [ ]'
- )}
- </Text>
- {isGroup && (
- isOpen ? <FiChevronDown data-testid="open-group" /> : <FiChevronUp data-testid="closed-group" />
+ {taskName}
+ {isMapped && (
+ ' [ ]'
)}
- </Flex>
- </Box>
+ </Text>
+ {isGroup && (
+ isOpen ? <FiChevronDown data-testid="open-group" /> : <FiChevronUp data-testid="closed-group" />
+ )}
+ </Flex>
);
const TaskInstances = ({
@@ -104,16 +102,20 @@ const TaskInstances = ({
onSelectInstance={onSelectInstance}
/>
)
- : <Box key={`${run.runId}-${task.id}`} width="18px" data-testid="blank-task" />;
+ : <Box key={`${run.runId}-${task.id}`} width="16px" data-testid="blank-task" />;
})}
</Flex>
);
-const Row = ({
- task, containerRef, level, prevTaskId, isParentOpen = true, onSelectInstance,
-}) => {
+const Row = (props) => {
+ const {
+ task, containerRef, level, prevTaskId, isParentOpen = true, onSelectInstance, selectedInstance,
+ } = props;
const { data: { dagRuns = [] } } = useTreeData();
+ const { colors } = useTheme();
+ const hoverBlue = `${colors.blue[100]}50`;
const isGroup = !!task.children;
+ const isSelected = selectedInstance.taskId === task.id;
const taskName = prevTaskId ? task.id.replace(`${prevTaskId}.`, '') : task.id;
@@ -136,21 +138,23 @@ const Row = ({
return (
<>
<Tr
- backgroundColor={`rgba(203, 213, 224, ${0.25 * level})`}
+ bg={isSelected ? 'blue.100' : undefined}
borderBottomWidth={isFullyOpen ? 1 : 0}
- borderBottomColor={level > 1 ? 'white' : 'gray.200'}
+ borderBottomColor="gray.200"
role="group"
+ _hover={!isSelected && { bg: hoverBlue }}
+ transition="background-color 0.2s"
>
<Td
- _groupHover={level > 3 && {
- color: 'white',
- }}
+ bg={isSelected ? 'blue.100' : 'white'}
+ _groupHover={!isSelected && ({ bg: 'blue.50' })}
p={0}
+ transition="background-color 0.2s"
lineHeight="18px"
position="sticky"
left={0}
- backgroundColor="white"
borderBottom={0}
+ zIndex={2}
>
<Collapse in={isFullyOpen}>
<TaskName
@@ -164,7 +168,11 @@ const Row = ({
</Collapse>
</Td>
<Td width={0} p={0} borderBottom={0} />
- <Td p={0} align="right" _groupHover={{ backgroundColor: 'rgba(113, 128, 150, 0.1)' }} transition="background-color 0.2s" borderBottom={0}>
+ <Td
+ p={0}
+ align="right"
+ borderBottom={0}
+ >
<Collapse in={isFullyOpen}>
<TaskInstances
dagRuns={dagRuns}
@@ -177,7 +185,9 @@ const Row = ({
</Tr>
{isGroup && (
renderTaskRows({
- task, containerRef, level: level + 1, isParentOpen: isOpen, onSelectInstance,
+ ...props,
+ level: level + 1,
+ isParentOpen: isOpen,
})
)}
</>
[airflow] 06/09: switch from drawer to details section
Posted by bb...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bbovenzi pushed a commit to branch mapped-task-drawer
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit bcb3242cc75ce485c76804d74ecbd1772e081721
Author: Brent Bovenzi <br...@gmail.com>
AuthorDate: Tue Mar 1 14:00:11 2022 -0500
switch from drawer to details section
---
airflow/www/forms.py | 4 +-
airflow/www/static/js/tree/InstanceTooltip.jsx | 83 ++------------------
airflow/www/static/js/tree/SidePanel.jsx | 90 ----------------------
airflow/www/static/js/tree/StatusBox.jsx | 6 +-
airflow/www/static/js/tree/Tree.jsx | 66 ++++++++--------
airflow/www/static/js/tree/dagRuns/Bar.jsx | 6 +-
airflow/www/static/js/tree/dagRuns/Tooltip.jsx | 51 +-----------
airflow/www/static/js/tree/dagRuns/index.jsx | 6 +-
airflow/www/static/js/tree/details/Header.jsx | 85 ++++++++++++++++++++
airflow/www/static/js/tree/details/content/Dag.jsx | 31 ++++++++
.../www/static/js/tree/details/content/DagRun.jsx | 31 ++++++++
.../js/tree/details/content/TaskInstance.jsx | 31 ++++++++
airflow/www/static/js/tree/details/index.jsx | 48 ++++++++++++
airflow/www/static/js/tree/renderTaskRows.jsx | 22 +++---
14 files changed, 291 insertions(+), 269 deletions(-)
diff --git a/airflow/www/forms.py b/airflow/www/forms.py
index 3a07cbe..3c3d0ac 100644
--- a/airflow/www/forms.py
+++ b/airflow/www/forms.py
@@ -104,13 +104,13 @@ class DateTimeWithNumRunsForm(FlaskForm):
)
num_runs = SelectField(
"Number of runs",
- default=25,
+ default=10,
choices=(
(5, "5"),
+ (10, "10"),
(25, "25"),
(50, "50"),
(100, "100"),
- (365, "365"),
),
)
diff --git a/airflow/www/static/js/tree/InstanceTooltip.jsx b/airflow/www/static/js/tree/InstanceTooltip.jsx
index b0d0584..e5c1117 100644
--- a/airflow/www/static/js/tree/InstanceTooltip.jsx
+++ b/airflow/www/static/js/tree/InstanceTooltip.jsx
@@ -17,33 +17,16 @@
* under the License.
*/
-/* global moment */
-
import React from 'react';
import { Box, Text } from '@chakra-ui/react';
-import { formatDateTime, getDuration, formatDuration } from '../datetime_utils';
import { finalStatesMap } from '../utils';
-
-const STATES = [
- ['success', 0],
- ['failed', 0],
- ['upstream_failed', 0],
- ['up_for_retry', 0],
- ['up_for_reschedule', 0],
- ['running', 0],
- ['deferred', 0],
- ['sensing', 0],
- ['queued', 0],
- ['scheduled', 0],
- ['skipped', 0],
- ['no_status', 0],
-];
+import { formatDateTime } from '../datetime_utils';
const InstanceTooltip = ({
group,
instance: {
- duration, operator, startDate, endDate, state, taskId, runId, mappedStates,
+ startDate, state, runId, mappedStates,
},
}) => {
const isGroup = !!group.children;
@@ -93,25 +76,20 @@ const InstanceTooltip = ({
});
}
- const taskIdTitle = isGroup ? 'Task Group Id: ' : 'Task Id: ';
-
return (
- <Box fontSize="12px" py="4px">
+ <Box fontSize="12px" py="2px">
{group.tooltip && (
<Text>{group.tooltip}</Text>
)}
<Text>
- <Text as="strong">Status:</Text>
+ <Text as="strong">
+ {isGroup ? 'Overall ' : ''}
+ Status:
+ </Text>
{' '}
{state || 'no status'}
</Text>
- {isGroup && (
- <>
- <br />
- <Text as="strong">Group Summary</Text>
- {groupSummary}
- </>
- )}
+ {isGroup && groupSummary}
{group.isMapped && (
<>
<br />
@@ -124,56 +102,11 @@ const InstanceTooltip = ({
{mapSummary}
</>
)}
- <br />
- {/* <Text>
- {taskIdTitle}
- {taskId}
- </Text> */}
- {/* <Text whiteSpace="nowrap">
- Run Id:
- {' '}
- {runId}
- </Text> */}
- {/* {operator && (
- <Text>
- Operator:
- {' '}
- {operator}
- </Text>
- )} */}
- <Text>
- Duration:
- {' '}
- {formatDuration(duration || getDuration(startDate, endDate))}
- </Text>
- <br />
- {/* <Text as="strong">UTC</Text>
- <Text>
- Started:
- {' '}
- {startDate && formatDateTime(moment.utc(startDate))}
- </Text>
- <Text>
- Ended:
- {' '}
- {endDate && formatDateTime(moment.utc(endDate))}
- </Text>
- <br />
- <Text as="strong">
- Local:
- {' '}
- {moment().format('Z')}
- </Text>
<Text>
Started:
{' '}
{startDate && formatDateTime(startDate)}
</Text>
- <Text>
- Ended:
- {' '}
- {endDate && formatDateTime(endDate)}
- </Text> */}
</Box>
);
};
diff --git a/airflow/www/static/js/tree/SidePanel.jsx b/airflow/www/static/js/tree/SidePanel.jsx
deleted file mode 100644
index 88439a0..0000000
--- a/airflow/www/static/js/tree/SidePanel.jsx
+++ /dev/null
@@ -1,90 +0,0 @@
-/*!
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-/* global moment */
-
-import React from 'react';
-import {
- chakra,
- Flex,
- Text,
- useDisclosure,
- IconButton,
-} from '@chakra-ui/react';
-import { MdKeyboardArrowLeft, MdKeyboardArrowRight } from 'react-icons/md';
-
-import { formatDateTime } from '../datetime_utils';
-
-const SidePanel = ({ instance: { runId, taskId, executionDate } }) => {
- const { isOpen, onOpen, onClose } = useDisclosure();
- if (!isOpen) {
- return (
- <IconButton
- m={2}
- icon={<MdKeyboardArrowLeft size={18} />}
- aria-label="Open Details Panel"
- onClick={onOpen}
- />
- );
- }
- let title = '';
- if (runId && taskId) {
- title = (
- <>
- <chakra.span>Task Instance: </chakra.span>
- <chakra.b>{taskId}</chakra.b>
- <chakra.span> at </chakra.span>
- <chakra.b>{formatDateTime(moment.utc(executionDate))}</chakra.b>
- </>
- );
- } else if (runId) {
- title = (
- <>
- <chakra.span>Dag Run: </chakra.span>
- <chakra.b>{runId}</chakra.b>
- <chakra.span> at </chakra.span>
- <chakra.b>{formatDateTime(moment.utc(executionDate))}</chakra.b>
- </>
- );
- } else {
- title = (
- <chakra.span>Dag Details: </chakra.span>
- );
- }
-
- return (
- <Flex bg="gray.200" maxWidth={isOpen ? '50%' : 0} minWidth={isOpen ? 300 : 0} flexDirection="column" p={3}>
- <IconButton
- m={2}
- icon={<MdKeyboardArrowRight size={18} />}
- aria-label="Close Details Panel"
- onClick={onClose}
- />
- <Text as="h4">
- {title}
- </Text>
- <Text>
- {/* eslint-disable-next-line max-len */}
- Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Nunc vel risus commodo viverra maecenas accumsan. Ut porttitor leo a diam sollicitudin tempor id eu. Molestie at elementum eu facilisis sed odio morbi quis commodo. Facilisis leo vel fringilla est ullamcorper eget nulla facilisi etiam. Est sit amet facilisis magna etiam tempor orci eu. Id semper risus in hendrerit gravida rutrum. Ac odio tempor orci dapibus [...]
- </Text>
- </Flex>
- );
-};
-
-export default SidePanel;
diff --git a/airflow/www/static/js/tree/StatusBox.jsx b/airflow/www/static/js/tree/StatusBox.jsx
index 0953a79..9609d9a 100644
--- a/airflow/www/static/js/tree/StatusBox.jsx
+++ b/airflow/www/static/js/tree/StatusBox.jsx
@@ -30,7 +30,7 @@ import {
import InstanceTooltip from './InstanceTooltip';
const StatusBox = ({
- group, instance, containerRef, onSelectInstance, selectedInstance, ...rest
+ group, instance, containerRef, onSelect, selected, ...rest
}) => {
const { runId } = instance;
const { colors } = useTheme();
@@ -38,7 +38,7 @@ const StatusBox = ({
// Fetch the corresponding column element and set its background color when hovering
const onMouseEnter = () => {
- if (selectedInstance.runId !== runId) {
+ if (selected.runId !== runId) {
[...containerRef.current.getElementsByClassName(`js-${runId}`)]
.forEach((e) => { e.style.backgroundColor = hoverBlue; });
}
@@ -51,7 +51,7 @@ const StatusBox = ({
const onClick = (e) => {
e.stopPropagation();
onMouseLeave();
- onSelectInstance(instance);
+ onSelect(instance);
};
return (
diff --git a/airflow/www/static/js/tree/Tree.jsx b/airflow/www/static/js/tree/Tree.jsx
index 199e373..1f60d03 100644
--- a/airflow/www/static/js/tree/Tree.jsx
+++ b/airflow/www/static/js/tree/Tree.jsx
@@ -34,13 +34,13 @@ import {
import useTreeData from './useTreeData';
import renderTaskRows from './renderTaskRows';
import DagRuns from './dagRuns';
-import SidePanel from './SidePanel';
+import Details from './details';
const Tree = () => {
const containerRef = useRef();
const scrollRef = useRef();
const { data: { groups = {}, dagRuns = [] }, isRefreshOn, onToggleRefresh } = useTreeData();
- const [selectedInstance, setSelectedInstance] = useState({});
+ const [selected, setSelected] = useState({});
const dagRunIds = dagRuns.map((dr) => dr.runId);
@@ -52,46 +52,42 @@ const Tree = () => {
}
}, []);
- const { runId, taskId } = selectedInstance;
- const onSelectInstance = (newInstance) => (
+ const { runId, taskId } = selected;
+ const onSelect = (newInstance) => (
(newInstance.runId === runId && newInstance.taskId === taskId)
- ? setSelectedInstance({})
- : setSelectedInstance(newInstance)
+ ? setSelected({})
+ : setSelected(newInstance)
);
return (
- <Box position="relative" ref={containerRef}>
- <FormControl display="flex" alignItems="center" justifyContent="flex-end" width="100%" mb={2}>
- {isRefreshOn && <Spinner color="blue.500" speed="1s" mr="4px" />}
- <FormLabel htmlFor="auto-refresh" mb={0} fontSize="12px" fontWeight="normal">
- Auto-refresh
- </FormLabel>
- <Switch id="auto-refresh" onChange={onToggleRefresh} isChecked={isRefreshOn} size="lg" />
- </FormControl>
+ <Flex pl="24px" position="relative" flexDirection="row" justifyContent="space-between" ref={containerRef}>
<Text transform="rotate(-90deg)" position="absolute" left="-6px" top="130px">Runs</Text>
<Text transform="rotate(-90deg)" position="absolute" left="-6px" top="190px">Tasks</Text>
- <Box pl="24px" height="100%" onClick={() => setSelectedInstance({})}>
- <Flex position="relative" flexDirection="row" justifyContent="space-between" overflow="hidden">
- <Box mr="12px" pb="12px" overflowX="auto" ref={scrollRef} maxWidth="60vw">
- <Table height={0}>
- <Thead>
- <DagRuns
- containerRef={containerRef}
- selectedInstance={selectedInstance}
- onSelectInstance={onSelectInstance}
- />
- </Thead>
- <Tbody>
- {renderTaskRows({
- task: groups, containerRef, onSelectInstance, selectedInstance, dagRunIds,
- })}
- </Tbody>
- </Table>
- </Box>
- <SidePanel instance={selectedInstance} />
- </Flex>
+ <Box mr="12px" pb="12px" overflowX="auto" ref={scrollRef} maxWidth="300px" minWidth="300px" position="relative">
+ <FormControl display="flex" alignItems="center" justifyContent="flex-end" width="100%" mb={2}>
+ {isRefreshOn && <Spinner color="blue.500" speed="1s" mr="4px" />}
+ <FormLabel htmlFor="auto-refresh" mb={0} fontSize="12px" fontWeight="normal">
+ Auto-refresh
+ </FormLabel>
+ <Switch id="auto-refresh" onChange={onToggleRefresh} isChecked={isRefreshOn} size="lg" />
+ </FormControl>
+ <Table height={0}>
+ <Thead>
+ <DagRuns
+ containerRef={containerRef}
+ selected={selected}
+ onSelect={onSelect}
+ />
+ </Thead>
+ <Tbody>
+ {renderTaskRows({
+ task: groups, containerRef, onSelect, selected, dagRunIds,
+ })}
+ </Tbody>
+ </Table>
</Box>
- </Box>
+ <Details selected={selected} onSelect={onSelect} />
+ </Flex>
);
};
diff --git a/airflow/www/static/js/tree/dagRuns/Bar.jsx b/airflow/www/static/js/tree/dagRuns/Bar.jsx
index c64abdd..1c8dd6b 100644
--- a/airflow/www/static/js/tree/dagRuns/Bar.jsx
+++ b/airflow/www/static/js/tree/dagRuns/Bar.jsx
@@ -35,7 +35,7 @@ import DagRunTooltip from './Tooltip';
const BAR_HEIGHT = 100;
const DagRunBar = ({
- run, max, index, totalRuns, containerRef, selectedInstance, onSelectInstance,
+ run, max, index, totalRuns, containerRef, selected, onSelect,
}) => {
const { colors } = useTheme();
const hoverBlue = `${colors.blue[100]}50`;
@@ -44,7 +44,7 @@ const DagRunBar = ({
const table = containerRef.current.getElementsByTagName('tbody')[0];
highlightHeight = table.offsetHeight + BAR_HEIGHT;
}
- const isSelected = run.runId === selectedInstance.runId;
+ const isSelected = run.runId === selected.runId;
return (
<Box position="relative">
<Flex
@@ -59,7 +59,7 @@ const DagRunBar = ({
zIndex={1}
onClick={(e) => {
e.stopPropagation();
- onSelectInstance(run);
+ onSelect(run);
}}
position="relative"
data-peer
diff --git a/airflow/www/static/js/tree/dagRuns/Tooltip.jsx b/airflow/www/static/js/tree/dagRuns/Tooltip.jsx
index 4e42cde..70b7f6a 100644
--- a/airflow/www/static/js/tree/dagRuns/Tooltip.jsx
+++ b/airflow/www/static/js/tree/dagRuns/Tooltip.jsx
@@ -17,8 +17,6 @@
* under the License.
*/
-/* global moment */
-
import React from 'react';
import { Box, Text } from '@chakra-ui/react';
import { MdPlayArrow } from 'react-icons/md';
@@ -27,20 +25,19 @@ import { formatDateTime, formatDuration } from '../../datetime_utils';
const DagRunTooltip = ({
dagRun: {
- state, runId, duration, dataIntervalStart, dataIntervalEnd, startDate, endDate, runType,
+ state, duration, dataIntervalEnd, runType,
},
}) => (
- <Box fontSize="12px" py="4px">
+ <Box fontSize="12px" py="2px">
<Text>
<Text as="strong">Status:</Text>
{' '}
{state || 'no status'}
</Text>
- <br />
<Text whiteSpace="nowrap">
- Run Id:
+ Run:
{' '}
- {runId}
+ {formatDateTime(dataIntervalEnd)}
</Text>
<Text>
Run Type:
@@ -53,46 +50,6 @@ const DagRunTooltip = ({
{' '}
{formatDuration(duration)}
</Text>
- <br />
- <Text as="strong">Data Interval:</Text>
- <Text>
- Start:
- {' '}
- {formatDateTime(dataIntervalStart)}
- </Text>
- <Text>
- End:
- {' '}
- {formatDateTime(dataIntervalEnd)}
- </Text>
- <br />
- <Text as="strong">UTC</Text>
- <Text>
- Started:
- {' '}
- {formatDateTime(moment.utc(startDate))}
- </Text>
- <Text>
- Ended:
- {' '}
- {endDate && formatDateTime(moment.utc(endDate))}
- </Text>
- <br />
- <Text as="strong">
- Local:
- {' '}
- {moment().format('Z')}
- </Text>
- <Text>
- Started:
- {' '}
- {formatDateTime(startDate)}
- </Text>
- <Text>
- Ended:
- {' '}
- {endDate && formatDateTime(endDate)}
- </Text>
</Box>
);
diff --git a/airflow/www/static/js/tree/dagRuns/index.jsx b/airflow/www/static/js/tree/dagRuns/index.jsx
index 7459634..7ff4e39 100644
--- a/airflow/www/static/js/tree/dagRuns/index.jsx
+++ b/airflow/www/static/js/tree/dagRuns/index.jsx
@@ -36,7 +36,7 @@ const DurationTick = ({ children, ...rest }) => (
</Text>
);
-const DagRuns = ({ containerRef, selectedInstance, onSelectInstance }) => {
+const DagRuns = ({ containerRef, selected, onSelect }) => {
const { data: { dagRuns = [] } } = useTreeData();
const durations = [];
const runs = dagRuns.map((dagRun) => {
@@ -97,8 +97,8 @@ const DagRuns = ({ containerRef, selectedInstance, onSelectInstance }) => {
index={i}
totalRuns={runs.length}
containerRef={containerRef}
- selectedInstance={selectedInstance}
- onSelectInstance={onSelectInstance}
+ selected={selected}
+ onSelect={onSelect}
/>
))}
</Flex>
diff --git a/airflow/www/static/js/tree/details/Header.jsx b/airflow/www/static/js/tree/details/Header.jsx
new file mode 100644
index 0000000..b9d3086
--- /dev/null
+++ b/airflow/www/static/js/tree/details/Header.jsx
@@ -0,0 +1,85 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import React from 'react';
+import {
+ Breadcrumb,
+ BreadcrumbItem,
+ BreadcrumbLink,
+ Box,
+ Heading,
+} from '@chakra-ui/react';
+import { MdPlayArrow } from 'react-icons/md';
+
+import useTreeData from '../useTreeData';
+import { formatDateTime } from '../../datetime_utils';
+import getMetaValue from '../../meta_value';
+
+const dagId = getMetaValue('dag_id');
+
+const LabelValue = ({ label, value }) => (
+ <Box position="relative">
+ <Heading as="h5" size="sm" color="gray.300" position="absolute" top="-12px">{label}</Heading>
+ <Heading as="h3" size="md">{value}</Heading>
+ </Box>
+);
+
+const Header = ({
+ selected: { taskId, runId },
+ onSelect,
+}) => {
+ const { data: { dagRuns = [] } } = useTreeData();
+ const dagRun = dagRuns.find((r) => r.runId === runId);
+ // console.log(dagRun);
+ let runLabel = dagRun ? formatDateTime(dagRun.dataIntervalEnd) : '';
+ if (dagRun && dagRun.runType === 'manual') {
+ runLabel = (
+ <>
+ <MdPlayArrow style={{ display: 'inline' }} />
+ {runLabel}
+ </>
+ );
+ }
+
+ return (
+ <Breadcrumb>
+ <BreadcrumbItem isCurrentPage={!runId && !taskId}>
+ <BreadcrumbLink onClick={() => onSelect({})}>
+ <LabelValue label="DAG" value={dagId} />
+ </BreadcrumbLink>
+ </BreadcrumbItem>
+ {runId && (
+ <BreadcrumbItem isCurrentPage={runId && !taskId}>
+ <BreadcrumbLink onClick={() => onSelect({ runId })}>
+ <LabelValue label="Run" value={runLabel} />
+ </BreadcrumbLink>
+ </BreadcrumbItem>
+ )}
+ {taskId && (
+ <BreadcrumbItem isCurrentPage>
+ <BreadcrumbLink>
+ <LabelValue label="Task" value={taskId} />
+ </BreadcrumbLink>
+ </BreadcrumbItem>
+ )}
+ </Breadcrumb>
+ );
+};
+
+export default Header;
diff --git a/airflow/www/static/js/tree/details/content/Dag.jsx b/airflow/www/static/js/tree/details/content/Dag.jsx
new file mode 100644
index 0000000..be460ba
--- /dev/null
+++ b/airflow/www/static/js/tree/details/content/Dag.jsx
@@ -0,0 +1,31 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import React from 'react';
+import {
+ Text,
+} from '@chakra-ui/react';
+
+const Dag = () => (
+ <>
+ <Text>dag details</Text>
+ </>
+);
+
+export default Dag;
diff --git a/airflow/www/static/js/tree/details/content/DagRun.jsx b/airflow/www/static/js/tree/details/content/DagRun.jsx
new file mode 100644
index 0000000..58e4b92
--- /dev/null
+++ b/airflow/www/static/js/tree/details/content/DagRun.jsx
@@ -0,0 +1,31 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import React from 'react';
+import {
+ Text,
+} from '@chakra-ui/react';
+
+const DagRun = () => (
+ <>
+ <Text>dag run details</Text>
+ </>
+);
+
+export default DagRun;
diff --git a/airflow/www/static/js/tree/details/content/TaskInstance.jsx b/airflow/www/static/js/tree/details/content/TaskInstance.jsx
new file mode 100644
index 0000000..4ab46e5
--- /dev/null
+++ b/airflow/www/static/js/tree/details/content/TaskInstance.jsx
@@ -0,0 +1,31 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import React from 'react';
+import {
+ Text,
+} from '@chakra-ui/react';
+
+const TaskInstance = () => (
+ <>
+ <Text>task instance details</Text>
+ </>
+);
+
+export default TaskInstance;
diff --git a/airflow/www/static/js/tree/details/index.jsx b/airflow/www/static/js/tree/details/index.jsx
new file mode 100644
index 0000000..b8f960e
--- /dev/null
+++ b/airflow/www/static/js/tree/details/index.jsx
@@ -0,0 +1,48 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import React from 'react';
+import {
+ Flex,
+ Box,
+ Divider,
+} from '@chakra-ui/react';
+
+import Header from './Header';
+import TaskInstanceContent from './content/TaskInstance';
+import DagRunContent from './content/DagRun';
+import DagContent from './content/Dag';
+
+const Details = ({
+ selected,
+ onSelect,
+}) => (
+ <Flex borderLeftWidth="1px" flexDirection="column" p={3} flexGrow={1}>
+ <Header selected={selected} onSelect={onSelect} />
+ <Divider my={2} />
+ <Box>
+ {/* TODO: get full instance data from the API */}
+ {!selected.runId && !selected.taskId && <DagContent />}
+ {selected.runId && !selected.taskId && <DagRunContent />}
+ {selected.taskId && <TaskInstanceContent instance={selected} />}
+ </Box>
+ </Flex>
+);
+
+export default Details;
diff --git a/airflow/www/static/js/tree/renderTaskRows.jsx b/airflow/www/static/js/tree/renderTaskRows.jsx
index 7440774..5e8234e 100644
--- a/airflow/www/static/js/tree/renderTaskRows.jsx
+++ b/airflow/www/static/js/tree/renderTaskRows.jsx
@@ -40,7 +40,7 @@ import { getMetaValue } from '../utils';
const dagId = getMetaValue('dag_id');
const renderTaskRows = ({
- task, containerRef, level = 0, isParentOpen, onSelectInstance, selectedInstance, dagRunIds
+ task, containerRef, level = 0, isParentOpen, onSelect, selected, dagRunIds,
}) => task.children.map((t) => (
<Row
key={t.id}
@@ -49,8 +49,8 @@ const renderTaskRows = ({
containerRef={containerRef}
prevTaskId={task.id}
isParentOpen={isParentOpen}
- onSelectInstance={onSelectInstance}
- selectedInstance={selectedInstance}
+ onSelect={onSelect}
+ selected={selected}
dagRunIds={dagRunIds}
/>
));
@@ -85,7 +85,7 @@ const TaskName = ({
);
const TaskInstances = ({
- task, containerRef, dagRunIds, onSelectInstance, selectedInstance,
+ task, containerRef, dagRunIds, onSelect, selected,
}) => (
<Flex justifyContent="flex-end">
{dagRunIds.map((runId) => {
@@ -99,8 +99,8 @@ const TaskInstances = ({
instance={instance}
containerRef={containerRef}
group={task}
- onSelectInstance={onSelectInstance}
- selectedInstance={selectedInstance}
+ onSelect={onSelect}
+ selected={selected}
/>
)
: <Box key={key} width="16px" data-testid="blank-task" />;
@@ -115,15 +115,15 @@ const Row = (props) => {
level,
prevTaskId,
isParentOpen = true,
- onSelectInstance,
- selectedInstance,
+ onSelect,
+ selected,
dagRunIds,
} = props;
// const { data: { dagRuns = [] } } = useTreeData();
const { colors } = useTheme();
const hoverBlue = `${colors.blue[100]}50`;
const isGroup = !!task.children;
- const isSelected = selectedInstance.taskId === task.id;
+ const isSelected = selected.taskId === task.id;
const taskName = prevTaskId ? task.id.replace(`${prevTaskId}.`, '') : task.id;
@@ -186,8 +186,8 @@ const Row = (props) => {
dagRunIds={dagRunIds}
task={task}
containerRef={containerRef}
- onSelectInstance={onSelectInstance}
- selectedInstance={selectedInstance}
+ onSelect={onSelect}
+ selected={selected}
/>
</Collapse>
</Td>
[airflow] 08/09: use API
Posted by bb...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bbovenzi pushed a commit to branch mapped-task-drawer
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit a301881f86ecc64efe36e44741459471c2d7926d
Author: Brent Bovenzi <br...@gmail.com>
AuthorDate: Tue Mar 1 19:44:16 2022 -0500
use API
---
airflow/www/package.json | 2 +
airflow/www/static/js/tree/StatusBox.jsx | 6 +-
airflow/www/static/js/tree/Tree.jsx | 16 ++--
.../tree/{details/content/Dag.jsx => api/index.js} | 21 +++--
.../{details/content/Dag.jsx => api/useDag.js} | 19 ++--
.../{details/content/Dag.jsx => api/useTasks.js} | 19 ++--
airflow/www/static/js/tree/dagRuns/Bar.jsx | 2 +-
airflow/www/static/js/tree/details/Header.jsx | 5 +-
airflow/www/static/js/tree/details/content/Dag.jsx | 36 ++++++-
.../js/tree/details/content/TaskInstance.jsx | 100 ++++++++++----------
airflow/www/static/js/tree/details/index.jsx | 34 ++++---
airflow/www/static/js/tree/index.jsx | 13 ++-
airflow/www/yarn.lock | 104 +++++++++++++++++++--
13 files changed, 257 insertions(+), 120 deletions(-)
diff --git a/airflow/www/package.json b/airflow/www/package.json
index 6d52987..717d5b1 100644
--- a/airflow/www/package.json
+++ b/airflow/www/package.json
@@ -74,6 +74,7 @@
"@emotion/cache": "^11.4.0",
"@emotion/react": "^11.4.1",
"@emotion/styled": "^11",
+ "axios": "^0.26.0",
"bootstrap-3-typeahead": "^4.0.2",
"camelcase-keys": "^7.0.0",
"codemirror": "^5.59.1",
@@ -92,6 +93,7 @@
"react": "^17.0.2",
"react-dom": "^17.0.2",
"react-icons": "^4.2.0",
+ "react-query": "^3.34.16",
"redoc": "^2.0.0-rc.63",
"url-search-params-polyfill": "^8.1.0"
},
diff --git a/airflow/www/static/js/tree/StatusBox.jsx b/airflow/www/static/js/tree/StatusBox.jsx
index 9609d9a..62762c5 100644
--- a/airflow/www/static/js/tree/StatusBox.jsx
+++ b/airflow/www/static/js/tree/StatusBox.jsx
@@ -32,7 +32,7 @@ import InstanceTooltip from './InstanceTooltip';
const StatusBox = ({
group, instance, containerRef, onSelect, selected, ...rest
}) => {
- const { runId } = instance;
+ const { runId, taskId } = instance;
const { colors } = useTheme();
const hoverBlue = `${colors.blue[100]}50`;
@@ -51,7 +51,9 @@ const StatusBox = ({
const onClick = (e) => {
e.stopPropagation();
onMouseLeave();
- onSelect(instance);
+ onSelect({
+ taskId, runId, instance, task: group,
+ });
};
return (
diff --git a/airflow/www/static/js/tree/Tree.jsx b/airflow/www/static/js/tree/Tree.jsx
index 1f60d03..aeeaeb5 100644
--- a/airflow/www/static/js/tree/Tree.jsx
+++ b/airflow/www/static/js/tree/Tree.jsx
@@ -63,14 +63,14 @@ const Tree = () => {
<Flex pl="24px" position="relative" flexDirection="row" justifyContent="space-between" ref={containerRef}>
<Text transform="rotate(-90deg)" position="absolute" left="-6px" top="130px">Runs</Text>
<Text transform="rotate(-90deg)" position="absolute" left="-6px" top="190px">Tasks</Text>
- <Box mr="12px" pb="12px" overflowX="auto" ref={scrollRef} maxWidth="300px" minWidth="300px" position="relative">
- <FormControl display="flex" alignItems="center" justifyContent="flex-end" width="100%" mb={2}>
- {isRefreshOn && <Spinner color="blue.500" speed="1s" mr="4px" />}
- <FormLabel htmlFor="auto-refresh" mb={0} fontSize="12px" fontWeight="normal">
- Auto-refresh
- </FormLabel>
- <Switch id="auto-refresh" onChange={onToggleRefresh} isChecked={isRefreshOn} size="lg" />
- </FormControl>
+ <FormControl display="flex" position="absolute" left="220px">
+ {isRefreshOn && <Spinner color="blue.500" speed="1s" mr="4px" />}
+ <FormLabel htmlFor="auto-refresh" mb={0} fontSize="12px" fontWeight="normal">
+ Auto-refresh
+ </FormLabel>
+ <Switch id="auto-refresh" onChange={onToggleRefresh} isChecked={isRefreshOn} size="lg" />
+ </FormControl>
+ <Box mr="12px" pb="12px" overflowX="auto" ref={scrollRef} maxWidth="300px" minWidth="300px" position="relative" mt="24px">
<Table height={0}>
<Thead>
<DagRuns
diff --git a/airflow/www/static/js/tree/details/content/Dag.jsx b/airflow/www/static/js/tree/api/index.js
similarity index 72%
copy from airflow/www/static/js/tree/details/content/Dag.jsx
copy to airflow/www/static/js/tree/api/index.js
index be460ba..58edd88 100644
--- a/airflow/www/static/js/tree/details/content/Dag.jsx
+++ b/airflow/www/static/js/tree/api/index.js
@@ -17,15 +17,18 @@
* under the License.
*/
-import React from 'react';
-import {
- Text,
-} from '@chakra-ui/react';
+import axios from 'axios';
+import camelcaseKeys from 'camelcase-keys';
-const Dag = () => (
- <>
- <Text>dag details</Text>
- </>
+import useDag from './useDag';
+import useTasks from './useTasks';
+
+axios.defaults.baseURL = '/api/v1';
+axios.interceptors.response.use(
+ (res) => (res.data ? camelcaseKeys(res.data, { deep: true }) : res),
);
-export default Dag;
+export {
+ useDag,
+ useTasks,
+};
diff --git a/airflow/www/static/js/tree/details/content/Dag.jsx b/airflow/www/static/js/tree/api/useDag.js
similarity index 80%
copy from airflow/www/static/js/tree/details/content/Dag.jsx
copy to airflow/www/static/js/tree/api/useDag.js
index be460ba..6c19ee4 100644
--- a/airflow/www/static/js/tree/details/content/Dag.jsx
+++ b/airflow/www/static/js/tree/api/useDag.js
@@ -17,15 +17,12 @@
* under the License.
*/
-import React from 'react';
-import {
- Text,
-} from '@chakra-ui/react';
+import axios from 'axios';
+import { useQuery } from 'react-query';
-const Dag = () => (
- <>
- <Text>dag details</Text>
- </>
-);
-
-export default Dag;
+export default function useDag(dagId) {
+ return useQuery(
+ ['dag', dagId],
+ () => axios.get(`/dags/${dagId}`),
+ );
+}
diff --git a/airflow/www/static/js/tree/details/content/Dag.jsx b/airflow/www/static/js/tree/api/useTasks.js
similarity index 80%
copy from airflow/www/static/js/tree/details/content/Dag.jsx
copy to airflow/www/static/js/tree/api/useTasks.js
index be460ba..3dd11e30 100644
--- a/airflow/www/static/js/tree/details/content/Dag.jsx
+++ b/airflow/www/static/js/tree/api/useTasks.js
@@ -17,15 +17,12 @@
* under the License.
*/
-import React from 'react';
-import {
- Text,
-} from '@chakra-ui/react';
+import axios from 'axios';
+import { useQuery } from 'react-query';
-const Dag = () => (
- <>
- <Text>dag details</Text>
- </>
-);
-
-export default Dag;
+export default function useTasks(dagId) {
+ return useQuery(
+ ['tasks', dagId],
+ () => axios.get(`/dags/${dagId}/tasks`),
+ );
+}
diff --git a/airflow/www/static/js/tree/dagRuns/Bar.jsx b/airflow/www/static/js/tree/dagRuns/Bar.jsx
index 1c8dd6b..94c84e7 100644
--- a/airflow/www/static/js/tree/dagRuns/Bar.jsx
+++ b/airflow/www/static/js/tree/dagRuns/Bar.jsx
@@ -59,7 +59,7 @@ const DagRunBar = ({
zIndex={1}
onClick={(e) => {
e.stopPropagation();
- onSelect(run);
+ onSelect({ runId: run.runId, dagRun: run });
}}
position="relative"
data-peer
diff --git a/airflow/www/static/js/tree/details/Header.jsx b/airflow/www/static/js/tree/details/Header.jsx
index b9d3086..c362606 100644
--- a/airflow/www/static/js/tree/details/Header.jsx
+++ b/airflow/www/static/js/tree/details/Header.jsx
@@ -27,9 +27,8 @@ import {
} from '@chakra-ui/react';
import { MdPlayArrow } from 'react-icons/md';
-import useTreeData from '../useTreeData';
import { formatDateTime } from '../../datetime_utils';
-import getMetaValue from '../../meta_value';
+import { getMetaValue } from '../../utils';
const dagId = getMetaValue('dag_id');
@@ -43,8 +42,8 @@ const LabelValue = ({ label, value }) => (
const Header = ({
selected: { taskId, runId },
onSelect,
+ dagRuns,
}) => {
- const { data: { dagRuns = [] } } = useTreeData();
const dagRun = dagRuns.find((r) => r.runId === runId);
// console.log(dagRun);
let runLabel = dagRun ? formatDateTime(dagRun.dataIntervalEnd) : '';
diff --git a/airflow/www/static/js/tree/details/content/Dag.jsx b/airflow/www/static/js/tree/details/content/Dag.jsx
index be460ba..e71d9c2 100644
--- a/airflow/www/static/js/tree/details/content/Dag.jsx
+++ b/airflow/www/static/js/tree/details/content/Dag.jsx
@@ -20,12 +20,38 @@
import React from 'react';
import {
Text,
+ Tag,
+ Code,
+ Flex,
+ HStack,
} from '@chakra-ui/react';
-const Dag = () => (
- <>
- <Text>dag details</Text>
- </>
-);
+import { getMetaValue } from '../../../utils';
+import { useDag } from '../../api';
+
+const dagId = getMetaValue('dag_id');
+
+const Dag = () => {
+ const { data: dag } = useDag(dagId);
+ if (!dag) return null;
+ const {
+ description, tags, fileloc, owners,
+ } = dag;
+ return (
+ <>
+ {description && <Text>{description}</Text>}
+ <HStack>{tags.map((tag) => <Tag key={tag.name} size="lg">{tag.name}</Tag>)}</HStack>
+ <Text>
+ Relative File Location:
+ {' '}
+ <Code colorScheme="blackAlpha">{fileloc}</Code>
+ </Text>
+ <Flex>
+ <Text mr={2}>Owner:</Text>
+ {owners.map((o) => <Text key={o}>{o}</Text>)}
+ </Flex>
+ </>
+ );
+};
export default Dag;
diff --git a/airflow/www/static/js/tree/details/content/TaskInstance.jsx b/airflow/www/static/js/tree/details/content/TaskInstance.jsx
index bc60737..0c0a21e 100644
--- a/airflow/www/static/js/tree/details/content/TaskInstance.jsx
+++ b/airflow/www/static/js/tree/details/content/TaskInstance.jsx
@@ -24,68 +24,70 @@ import {
Text,
Box,
} from '@chakra-ui/react';
+import { finalStatesMap } from '../../../utils';
import { formatDateTime, getDuration, formatDuration } from '../../../datetime_utils';
const TaskInstance = ({
instance: {
- duration, operator, startDate, endDate, state, taskId, runId, // mappedStates,
+ duration, operator, startDate, endDate, state, taskId, runId, mappedStates,
},
+ task,
}) => {
- const isGroup = false; // !!group.children;
+ const isGroup = !!task.children;
const groupSummary = [];
- // const mapSummary = [];
+ const mapSummary = [];
- // if (isGroup) {
- // const numMap = finalStatesMap();
- // group.children.forEach((child) => {
- // const taskInstance = child.instances.find((ti) => ti.runId === runId);
- // if (taskInstance) {
- // const stateKey = taskInstance.state == null ? 'no_status' : taskInstance.state;
- // if (numMap.has(stateKey)) numMap.set(stateKey, numMap.get(stateKey) + 1);
- // }
- // });
- // numMap.forEach((key, val) => {
- // if (key > 0) {
- // groupSummary.push(
- // // eslint-disable-next-line react/no-array-index-key
- // <Text key={val} ml="10px">
- // {val}
- // {': '}
- // {key}
- // </Text>,
- // );
- // }
- // });
- // }
+ if (isGroup) {
+ const numMap = finalStatesMap();
+ task.children.forEach((child) => {
+ const taskInstance = child.instances.find((ti) => ti.runId === runId);
+ if (taskInstance) {
+ const stateKey = taskInstance.state == null ? 'no_status' : taskInstance.state;
+ if (numMap.has(stateKey)) numMap.set(stateKey, numMap.get(stateKey) + 1);
+ }
+ });
+ numMap.forEach((key, val) => {
+ if (key > 0) {
+ groupSummary.push(
+ // eslint-disable-next-line react/no-array-index-key
+ <Text key={val} ml="10px">
+ {val}
+ {': '}
+ {key}
+ </Text>,
+ );
+ }
+ });
+ }
- // if (group.isMapped && mappedStates) {
- // const numMap = finalStatesMap();
- // mappedStates.forEach((s) => {
- // const stateKey = s || 'no_status';
- // if (numMap.has(stateKey)) numMap.set(stateKey, numMap.get(stateKey) + 1);
- // });
- // numMap.forEach((key, val) => {
- // if (key > 0) {
- // mapSummary.push(
- // // eslint-disable-next-line react/no-array-index-key
- // <Text key={val} ml="10px">
- // {val}
- // {': '}
- // {key}
- // </Text>,
- // );
- // }
- // });
- // }
+ if (task.isMapped && mappedStates) {
+ const numMap = finalStatesMap();
+ mappedStates.forEach((s) => {
+ const stateKey = s || 'no_status';
+ if (numMap.has(stateKey)) numMap.set(stateKey, numMap.get(stateKey) + 1);
+ });
+ numMap.forEach((key, val) => {
+ if (key > 0) {
+ mapSummary.push(
+ // eslint-disable-next-line react/no-array-index-key
+ <Text key={val} ml="10px">
+ {val}
+ {': '}
+ {key}
+ </Text>,
+ );
+ }
+ });
+ }
const taskIdTitle = isGroup ? 'Task Group Id: ' : 'Task Id: ';
return (
<Box fontSize="12px" py="4px">
- {/* {group.tooltip && (
- <Text>{group.tooltip}</Text>
- )} */}
+ {task.tooltip && (
+ <Text>{task.tooltip}</Text>
+ )}
<Text>
<Text as="strong">Status:</Text>
{' '}
@@ -98,7 +100,7 @@ const TaskInstance = ({
{groupSummary}
</>
)}
- {/* {group.isMapped && (
+ {task.isMapped && (
<>
<br />
<Text as="strong">
@@ -109,7 +111,7 @@ const TaskInstance = ({
</Text>
{mapSummary}
</>
- )} */}
+ )}
<br />
<Text>
{taskIdTitle}
diff --git a/airflow/www/static/js/tree/details/index.jsx b/airflow/www/static/js/tree/details/index.jsx
index 31610a7..ee3b044 100644
--- a/airflow/www/static/js/tree/details/index.jsx
+++ b/airflow/www/static/js/tree/details/index.jsx
@@ -28,21 +28,31 @@ import Header from './Header';
import TaskInstanceContent from './content/TaskInstance';
import DagRunContent from './content/DagRun';
import DagContent from './content/Dag';
+import useTreeData from '../useTreeData';
const Details = ({
selected,
onSelect,
-}) => (
- <Flex borderLeftWidth="1px" flexDirection="column" p={3} flexGrow={1}>
- <Header selected={selected} onSelect={onSelect} />
- <Divider my={2} />
- <Box>
- {/* TODO: get full instance data from the API */}
- {!selected.runId && !selected.taskId && <DagContent />}
- {selected.runId && !selected.taskId && <DagRunContent dagRun={selected} />}
- {selected.taskId && <TaskInstanceContent instance={selected} />}
- </Box>
- </Flex>
-);
+}) => {
+ const { data: { dagRuns = [] } } = useTreeData();
+ console.log(selected);
+ return (
+ <Flex borderLeftWidth="1px" flexDirection="column" p={3} flexGrow={1}>
+ <Header selected={selected} onSelect={onSelect} dagRuns={dagRuns} />
+ <Divider my={2} />
+ <Box>
+ {/* TODO: get full instance data from the API */}
+ {!selected.runId && !selected.taskId && <DagContent />}
+ {selected.runId && !selected.taskId && <DagRunContent dagRun={selected.dagRun} />}
+ {selected.taskId && (
+ <TaskInstanceContent
+ instance={selected.instance}
+ task={selected.task}
+ />
+ )}
+ </Box>
+ </Flex>
+ );
+};
export default Details;
diff --git a/airflow/www/static/js/tree/index.jsx b/airflow/www/static/js/tree/index.jsx
index 454f4f0..4eea821 100644
--- a/airflow/www/static/js/tree/index.jsx
+++ b/airflow/www/static/js/tree/index.jsx
@@ -24,6 +24,7 @@ import ReactDOM from 'react-dom';
import { ChakraProvider } from '@chakra-ui/react';
import { CacheProvider } from '@emotion/react';
import createCache from '@emotion/cache';
+import { QueryClient, QueryClientProvider } from 'react-query';
import Tree from './Tree';
// create shadowRoot
@@ -36,12 +37,22 @@ const myCache = createCache({
const mainElement = document.getElementById('react-container');
shadowRoot.appendChild(mainElement);
+const queryClient = new QueryClient({
+ defaultOptions: {
+ queries: {
+ refetchOnWindowFocus: false,
+ },
+ },
+});
+
function App() {
return (
<React.StrictMode>
<CacheProvider value={myCache}>
<ChakraProvider>
- <Tree />
+ <QueryClientProvider client={queryClient}>
+ <Tree />
+ </QueryClientProvider>
</ChakraProvider>
</CacheProvider>
</React.StrictMode>
diff --git a/airflow/www/yarn.lock b/airflow/www/yarn.lock
index 79c0165..0e9a1ee 100644
--- a/airflow/www/yarn.lock
+++ b/airflow/www/yarn.lock
@@ -1263,7 +1263,7 @@
core-js-pure "^3.16.0"
regenerator-runtime "^0.13.4"
-"@babel/runtime@^7.0.0", "@babel/runtime@^7.12.13":
+"@babel/runtime@^7.0.0", "@babel/runtime@^7.12.13", "@babel/runtime@^7.5.5", "@babel/runtime@^7.6.2":
version "7.17.2"
resolved "https://registry.yarnpkg.com/@babel/runtime/-/runtime-7.17.2.tgz#66f68591605e59da47523c631416b18508779941"
integrity sha512-hzeyJyMA1YGdJTuWU0e/j4wKXrU4OMFvY2MSlaI9B7VQb0r5cxTE3EAIS2Q7Tn2RIcDkRvTA/v2JsAEhxe99uw==
@@ -3254,6 +3254,13 @@ axe-core@^4.0.2:
resolved "https://registry.yarnpkg.com/axe-core/-/axe-core-4.3.3.tgz#b55cd8e8ddf659fe89b064680e1c6a4dceab0325"
integrity sha512-/lqqLAmuIPi79WYfRpy2i8z+x+vxU3zX2uAm0gs1q52qTuKwolOj1P8XbufpXcsydrpKx2yGn2wzAnxCMV86QA==
+axios@^0.26.0:
+ version "0.26.0"
+ resolved "https://registry.yarnpkg.com/axios/-/axios-0.26.0.tgz#9a318f1c69ec108f8cd5f3c3d390366635e13928"
+ integrity sha512-lKoGLMYtHvFrPVt3r+RBMp9nh34N0M8zEfCWqdWZx6phynIEhQqAdydpyBAAG211zlhX9Rgu08cOamy6XjE5Og==
+ dependencies:
+ follow-redirects "^1.14.8"
+
axobject-query@^2.2.0:
version "2.2.0"
resolved "https://registry.yarnpkg.com/axobject-query/-/axobject-query-2.2.0.tgz#943d47e10c0b704aa42275e20edf3722648989be"
@@ -3594,6 +3601,11 @@ base@^0.11.1:
mixin-deep "^1.2.0"
pascalcase "^0.1.1"
+big-integer@^1.6.16:
+ version "1.6.51"
+ resolved "https://registry.yarnpkg.com/big-integer/-/big-integer-1.6.51.tgz#0df92a5d9880560d3ff2d5fd20245c889d130686"
+ integrity sha512-GPEid2Y9QU1Exl1rpO9B2IPJGHPSupF5GnVIP0blYvNOMer2bTvSWs1jGOUg04hTmu67nmLsQ9TBo1puaotBHg==
+
big.js@^3.1.3:
version "3.2.0"
resolved "https://registry.yarnpkg.com/big.js/-/big.js-3.2.0.tgz#a5fc298b81b9e0dca2e458824784b65c52ba588e"
@@ -3682,6 +3694,20 @@ braces@^3.0.1, braces@~3.0.2:
dependencies:
fill-range "^7.0.1"
+broadcast-channel@^3.4.1:
+ version "3.7.0"
+ resolved "https://registry.yarnpkg.com/broadcast-channel/-/broadcast-channel-3.7.0.tgz#2dfa5c7b4289547ac3f6705f9c00af8723889937"
+ integrity sha512-cIAKJXAxGJceNZGTZSBzMxzyOn72cVgPnKx4dc6LRjQgbaJUQqhy5rzL3zbMxkMWsGKkv2hSFkPRMEXfoMZ2Mg==
+ dependencies:
+ "@babel/runtime" "^7.7.2"
+ detect-node "^2.1.0"
+ js-sha3 "0.8.0"
+ microseconds "0.2.0"
+ nano-time "1.0.0"
+ oblivious-set "1.0.0"
+ rimraf "3.0.2"
+ unload "2.2.0"
+
brorand@^1.0.1, brorand@^1.1.0:
version "1.1.0"
resolved "https://registry.yarnpkg.com/brorand/-/brorand-1.1.0.tgz#12c25efe40a45e3c323eb8675a0a0ce57b22371f"
@@ -5082,6 +5108,11 @@ detect-node-es@^1.1.0:
resolved "https://registry.yarnpkg.com/detect-node-es/-/detect-node-es-1.1.0.tgz#163acdf643330caa0b4cd7c21e7ee7755d6fa493"
integrity sha512-ypdmJU/TbBby2Dxibuv7ZLW3Bs1QEmM7nHjEANfohJLvE0XVujisn1qPJcZxg+qDucsr+bP6fLD1rPS3AhJ7EQ==
+detect-node@^2.0.4, detect-node@^2.1.0:
+ version "2.1.0"
+ resolved "https://registry.yarnpkg.com/detect-node/-/detect-node-2.1.0.tgz#c9c70775a49c3d03bc2c06d9a73be550f978f8b1"
+ integrity sha512-T0NIuQpnTvFDATNuHN5roPwSBG83rFsuO+MXXH9/3N1eFbn4wcPjttvjMLEPWJ0RGUYgQE7cGgS3tNxbqCGM7g==
+
diff-sequences@^27.0.6:
version "27.0.6"
resolved "https://registry.yarnpkg.com/diff-sequences/-/diff-sequences-27.0.6.tgz#3305cb2e55a033924054695cc66019fd7f8e5723"
@@ -5994,6 +6025,11 @@ focus-lock@^0.9.1:
dependencies:
tslib "^2.0.3"
+follow-redirects@^1.14.8:
+ version "1.14.9"
+ resolved "https://registry.yarnpkg.com/follow-redirects/-/follow-redirects-1.14.9.tgz#dd4ea157de7bfaf9ea9b3fbd85aa16951f78d8d7"
+ integrity sha512-MQDfihBQYMcyy5dhRDJUHcw7lb2Pv/TuE6xP1vyraLukNDHKbDxDNaOE3NbCAdKQApno+GPRyo1YAp89yCjK4w==
+
for-in@^1.0.2:
version "1.0.2"
resolved "https://registry.yarnpkg.com/for-in/-/for-in-1.0.2.tgz#81068d295a8142ec0ac726c6e2200c30fb6d5e80"
@@ -7621,6 +7657,11 @@ js-levenshtein@^1.1.6:
resolved "https://registry.yarnpkg.com/js-levenshtein/-/js-levenshtein-1.1.6.tgz#c6cee58eb3550372df8deb85fad5ce66ce01d59d"
integrity sha512-X2BB11YZtrRqY4EnQcLX5Rh373zbK4alC1FW7D7MBhL2gtcC17cTnr6DmfHZeS0s2rTHjUTMMHfG7gO8SSdw+g==
+js-sha3@0.8.0:
+ version "0.8.0"
+ resolved "https://registry.yarnpkg.com/js-sha3/-/js-sha3-0.8.0.tgz#b9b7a5da73afad7dedd0f8c463954cbde6818840"
+ integrity sha512-gF1cRrHhIzNfToc802P800N8PpXS+evLLXfsVpowqmAFR9uwbi89WvXg2QspOmXL8QL86J4T1EpFu+yUkwJY3Q==
+
"js-tokens@^3.0.0 || ^4.0.0", js-tokens@^4.0.0:
version "4.0.0"
resolved "https://registry.yarnpkg.com/js-tokens/-/js-tokens-4.0.0.tgz#19203fb59991df98e3a287050d4647cdeaf32499"
@@ -8072,6 +8113,14 @@ marked@^4.0.10:
resolved "https://registry.yarnpkg.com/marked/-/marked-4.0.12.tgz#2262a4e6fd1afd2f13557726238b69a48b982f7d"
integrity sha512-hgibXWrEDNBWgGiK18j/4lkS6ihTe9sxtV4Q1OQppb/0zzyPSzoFANBa5MfsG/zgsWklmNnhm0XACZOH/0HBiQ==
+match-sorter@^6.0.2:
+ version "6.3.1"
+ resolved "https://registry.yarnpkg.com/match-sorter/-/match-sorter-6.3.1.tgz#98cc37fda756093424ddf3cbc62bfe9c75b92bda"
+ integrity sha512-mxybbo3pPNuA+ZuCUhm5bwNkXrJTbsk5VWbR5wiwz/GC6LIiegBGn2w3O08UG/jdbYLinw51fSQ5xNU1U3MgBw==
+ dependencies:
+ "@babel/runtime" "^7.12.5"
+ remove-accents "0.4.2"
+
mathml-tag-names@^2.1.3:
version "2.1.3"
resolved "https://registry.yarnpkg.com/mathml-tag-names/-/mathml-tag-names-2.1.3.tgz#4ddadd67308e780cf16a47685878ee27b736a0a3"
@@ -8198,6 +8247,11 @@ micromatch@^4.0.2, micromatch@^4.0.4:
braces "^3.0.1"
picomatch "^2.2.3"
+microseconds@0.2.0:
+ version "0.2.0"
+ resolved "https://registry.yarnpkg.com/microseconds/-/microseconds-0.2.0.tgz#233b25f50c62a65d861f978a4a4f8ec18797dc39"
+ integrity sha512-n7DHHMjR1avBbSpsTBj6fmMGh2AGrifVV4e+WYc3Q9lO+xnSZ3NyhcBND3vzzatt05LFhoKFRxrIyklmLlUtyA==
+
miller-rabin@^4.0.0:
version "4.0.1"
resolved "https://registry.yarnpkg.com/miller-rabin/-/miller-rabin-4.0.1.tgz#f080351c865b0dc562a8462966daa53543c78a4d"
@@ -8422,6 +8476,13 @@ nan@^2.12.1:
resolved "https://registry.yarnpkg.com/nan/-/nan-2.14.2.tgz#f5376400695168f4cc694ac9393d0c9585eeea19"
integrity sha512-M2ufzIiINKCuDfBSAUr1vWQ+vuVcA9kqx8JJUsbQi6yf1uGRyb7HfpdfUr5qLXf3B/t8dPvcjhKMmlfnP47EzQ==
+nano-time@1.0.0:
+ version "1.0.0"
+ resolved "https://registry.yarnpkg.com/nano-time/-/nano-time-1.0.0.tgz#b0554f69ad89e22d0907f7a12b0993a5d96137ef"
+ integrity sha1-sFVPaa2J4i0JB/ehKwmTpdlhN+8=
+ dependencies:
+ big-integer "^1.6.16"
+
nanoid@^3.1.23:
version "3.1.23"
resolved "https://registry.yarnpkg.com/nanoid/-/nanoid-3.1.23.tgz#f744086ce7c2bc47ee0a8472574d5c78e4183a81"
@@ -8756,6 +8817,11 @@ object.values@^1.1.4:
define-properties "^1.1.3"
es-abstract "^1.18.2"
+oblivious-set@1.0.0:
+ version "1.0.0"
+ resolved "https://registry.yarnpkg.com/oblivious-set/-/oblivious-set-1.0.0.tgz#c8316f2c2fb6ff7b11b6158db3234c49f733c566"
+ integrity sha512-z+pI07qxo4c2CulUHCDf9lcqDlMSo72N/4rLUpRXf6fu+q8vjt8y0xS+Tlf8NTJDdTXHbdeO1n3MlbctwEoXZw==
+
once@^1.3.0, once@^1.3.1, once@^1.4.0:
version "1.4.0"
resolved "https://registry.yarnpkg.com/once/-/once-1.4.0.tgz#583b1aa775961d4b113ac17d9c50baef9dd76bd1"
@@ -9735,6 +9801,15 @@ react-is@^17.0.1:
resolved "https://registry.yarnpkg.com/react-is/-/react-is-17.0.2.tgz#e691d4a8e9c789365655539ab372762b0efb54f0"
integrity sha512-w2GsyukL62IJnlaff/nRegPQR94C/XXamvMWmSHRJ4y7Ts/4ocGRmTHvOs8PSE6pB3dWOrD/nueuU5sduBsQ4w==
+react-query@^3.34.16:
+ version "3.34.16"
+ resolved "https://registry.yarnpkg.com/react-query/-/react-query-3.34.16.tgz#279ea180bcaeaec49c7864b29d1711ee9f152594"
+ integrity sha512-7FvBvjgEM4YQ8nPfmAr+lJfbW95uyW/TVjFoi2GwCkF33/S8ajx45tuPHPFGWs4qYwPy1mzwxD4IQfpUDrefNQ==
+ dependencies:
+ "@babel/runtime" "^7.5.5"
+ broadcast-channel "^3.4.1"
+ match-sorter "^6.0.2"
+
react-remove-scroll-bar@^2.1.0:
version "2.2.0"
resolved "https://registry.yarnpkg.com/react-remove-scroll-bar/-/react-remove-scroll-bar-2.2.0.tgz#d4d545a7df024f75d67e151499a6ab5ac97c8cdd"
@@ -10007,6 +10082,11 @@ remark@^13.0.0:
remark-stringify "^9.0.0"
unified "^9.1.0"
+remove-accents@0.4.2:
+ version "0.4.2"
+ resolved "https://registry.yarnpkg.com/remove-accents/-/remove-accents-0.4.2.tgz#0a43d3aaae1e80db919e07ae254b285d9e1c7bb5"
+ integrity sha1-CkPTqq4egNuRngeuJUsoXZ4ce7U=
+
remove-trailing-separator@^1.0.1:
version "1.1.0"
resolved "https://registry.yarnpkg.com/remove-trailing-separator/-/remove-trailing-separator-1.1.0.tgz#c24bce2a283adad5bc3f58e0d48249b92379d8ef"
@@ -10127,6 +10207,13 @@ rgba-regex@^1.0.0:
resolved "https://registry.yarnpkg.com/rgba-regex/-/rgba-regex-1.0.0.tgz#43374e2e2ca0968b0ef1523460b7d730ff22eeb3"
integrity sha1-QzdOLiyglosO8VI0YLfXMP8i7rM=
+rimraf@3.0.2, rimraf@^3.0.0, rimraf@^3.0.2:
+ version "3.0.2"
+ resolved "https://registry.yarnpkg.com/rimraf/-/rimraf-3.0.2.tgz#f1a5402ba6220ad52cc1282bac1ae3aa49fd061a"
+ integrity sha512-JZkJMZkAGFFPP2YqXZXPbMlMBgsxzE8ILs4lMIX/2o0L9UBw9O/Y3o6wFw/i9YLapcUJWwqbi3kdxIPdC62TIA==
+ dependencies:
+ glob "^7.1.3"
+
rimraf@^2.5.4, rimraf@^2.6.3:
version "2.7.1"
resolved "https://registry.yarnpkg.com/rimraf/-/rimraf-2.7.1.tgz#35797f13a7fdadc566142c29d4f07ccad483e3ec"
@@ -10134,13 +10221,6 @@ rimraf@^2.5.4, rimraf@^2.6.3:
dependencies:
glob "^7.1.3"
-rimraf@^3.0.0, rimraf@^3.0.2:
- version "3.0.2"
- resolved "https://registry.yarnpkg.com/rimraf/-/rimraf-3.0.2.tgz#f1a5402ba6220ad52cc1282bac1ae3aa49fd061a"
- integrity sha512-JZkJMZkAGFFPP2YqXZXPbMlMBgsxzE8ILs4lMIX/2o0L9UBw9O/Y3o6wFw/i9YLapcUJWwqbi3kdxIPdC62TIA==
- dependencies:
- glob "^7.1.3"
-
ripemd160@^2.0.0, ripemd160@^2.0.1:
version "2.0.2"
resolved "https://registry.yarnpkg.com/ripemd160/-/ripemd160-2.0.2.tgz#a1c1a6f624751577ba5d07914cbc92850585890c"
@@ -11365,6 +11445,14 @@ universalify@^0.1.0, universalify@^0.1.2:
resolved "https://registry.yarnpkg.com/universalify/-/universalify-0.1.2.tgz#b646f69be3942dabcecc9d6639c80dc105efaa66"
integrity sha512-rBJeI5CXAlmy1pV+617WB9J63U6XcazHHF2f2dbJix4XzpUF0RS3Zbj0FGIOCAva5P/d/GBOYaACQ1w+0azUkg==
+unload@2.2.0:
+ version "2.2.0"
+ resolved "https://registry.yarnpkg.com/unload/-/unload-2.2.0.tgz#ccc88fdcad345faa06a92039ec0f80b488880ef7"
+ integrity sha512-B60uB5TNBLtN6/LsgAf3udH9saB5p7gqJwcFfbOEZ8BcBHnGwCf6G/TGiEqkRAxX7zAFIUtzdrXQSdL3Q/wqNA==
+ dependencies:
+ "@babel/runtime" "^7.6.2"
+ detect-node "^2.0.4"
+
unset-value@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/unset-value/-/unset-value-1.0.0.tgz#8376873f7d2335179ffb1e6fc3a8ed0dfc8ab559"
[airflow] 02/09: basic slide drawer
Posted by bb...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bbovenzi pushed a commit to branch mapped-task-drawer
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit f2836fa20b4aa09d83247f40caac18e76b7482c7
Author: Brent Bovenzi <br...@gmail.com>
AuthorDate: Wed Feb 16 13:57:33 2022 -0500
basic slide drawer
---
airflow/www/static/js/tree/SidePanel.jsx | 49 +++++++++++++++++
airflow/www/static/js/tree/StatusBox.jsx | 75 +++++++++++++++------------
airflow/www/static/js/tree/Tree.jsx | 25 ++++++---
airflow/www/static/js/tree/renderTaskRows.jsx | 19 +++++--
4 files changed, 125 insertions(+), 43 deletions(-)
diff --git a/airflow/www/static/js/tree/SidePanel.jsx b/airflow/www/static/js/tree/SidePanel.jsx
new file mode 100644
index 0000000..3515f1c
--- /dev/null
+++ b/airflow/www/static/js/tree/SidePanel.jsx
@@ -0,0 +1,49 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/* global moment */
+
+import React from 'react';
+import {
+ Box,
+ chakra,
+ Flex,
+ Text,
+} from '@chakra-ui/react';
+
+import { formatDateTime } from '../datetime_utils';
+
+const SidePanel = ({ instance: { runId, taskId, executionDate }, isOpen }) => (
+ <Box bg="gray.200" maxWidth={isOpen ? 300 : 0} minWidth={isOpen ? 300 : 0} transition="all 0.5s" position="relative" overflow="hidden">
+ <Flex right={isOpen ? 0 : -300} top={0} transition="right 0.5s, max-width 0.5s" width={300} flexDirection="column" m={2}>
+ <Text as="h4">
+ <chakra.span>Task Instance: </chakra.span>
+ <chakra.b>{taskId}</chakra.b>
+ <chakra.span> at </chakra.span>
+ <chakra.b>{formatDateTime(moment.utc(executionDate))}</chakra.b>
+ </Text>
+ <Text>
+ {/* eslint-disable-next-line max-len */}
+ Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Nunc vel risus commodo viverra maecenas accumsan. Ut porttitor leo a diam sollicitudin tempor id eu. Molestie at elementum eu facilisis sed odio morbi quis commodo. Facilisis leo vel fringilla est ullamcorper eget nulla facilisi etiam. Est sit amet facilisis magna etiam tempor orci eu. Id semper risus in hendrerit gravida rutrum. Ac odio tempor orci dapibus [...]
+ </Text>
+ </Flex>
+ </Box>
+);
+
+export default SidePanel;
diff --git a/airflow/www/static/js/tree/StatusBox.jsx b/airflow/www/static/js/tree/StatusBox.jsx
index c22f0b2..dce0fda 100644
--- a/airflow/www/static/js/tree/StatusBox.jsx
+++ b/airflow/www/static/js/tree/StatusBox.jsx
@@ -30,12 +30,21 @@ import { callModal } from '../dag';
import InstanceTooltip from './InstanceTooltip';
const StatusBox = ({
- group, instance, containerRef, extraLinks = [], ...rest
+ group, instance, containerRef, extraLinks = [], onSelectInstance, ...rest
}) => {
const {
executionDate, taskId, tryNumber = 0, operator, runId,
} = instance;
- const onClick = () => executionDate && callModal(taskId, executionDate, extraLinks, tryNumber, operator === 'SubDagOperator' || undefined, runId);
+
+ const onOpenModal = () => executionDate && callModal(taskId, executionDate, extraLinks, tryNumber, operator === 'SubDagOperator' || undefined, runId);
+ const onClick = () => {
+ if (group.isMapped) {
+ onSelectInstance(instance);
+ } else {
+ onSelectInstance({});
+ onOpenModal();
+ }
+ };
// Fetch the corresponding column element and set its background color when hovering
const onMouseOver = () => {
@@ -48,37 +57,39 @@ const StatusBox = ({
};
return (
- <Tooltip
- label={<InstanceTooltip instance={instance} group={group} />}
- fontSize="md"
- portalProps={{ containerRef }}
- hasArrow
- placement="top"
- openDelay={400}
- >
- <Flex
- p="1px"
- my="1px"
- mx="2px"
- justifyContent="center"
- alignItems="center"
- onClick={onClick}
- cursor={!group.children && 'pointer'}
- data-testid="task-instance"
- zIndex={1}
- onMouseEnter={onMouseOver}
- onMouseLeave={onMouseLeave}
- {...rest}
+ <>
+ <Tooltip
+ label={<InstanceTooltip instance={instance} group={group} />}
+ fontSize="md"
+ portalProps={{ containerRef }}
+ hasArrow
+ placement="top"
+ openDelay={400}
>
- <Box
- width="10px"
- height="10px"
- backgroundColor={stateColors[instance.state] || 'white'}
- borderRadius="2px"
- borderWidth={instance.state ? 0 : 1}
- />
- </Flex>
- </Tooltip>
+ <Flex
+ p="1px"
+ my="1px"
+ mx="2px"
+ justifyContent="center"
+ alignItems="center"
+ onClick={onClick}
+ cursor={!group.children && 'pointer'}
+ data-testid="task-instance"
+ zIndex={1}
+ onMouseEnter={onMouseOver}
+ onMouseLeave={onMouseLeave}
+ {...rest}
+ >
+ <Box
+ width="10px"
+ height="10px"
+ backgroundColor={stateColors[instance.state] || 'white'}
+ borderRadius="2px"
+ borderWidth={instance.state ? 0 : 1}
+ />
+ </Flex>
+ </Tooltip>
+ </>
);
};
diff --git a/airflow/www/static/js/tree/Tree.jsx b/airflow/www/static/js/tree/Tree.jsx
index 450e002..095adbd 100644
--- a/airflow/www/static/js/tree/Tree.jsx
+++ b/airflow/www/static/js/tree/Tree.jsx
@@ -17,7 +17,7 @@
* under the License.
*/
-import React, { useRef, useEffect } from 'react';
+import React, { useRef, useEffect, useState } from 'react';
import {
Table,
Tbody,
@@ -28,16 +28,19 @@ import {
Spinner,
Text,
Thead,
+ Flex,
} from '@chakra-ui/react';
import useTreeData from './useTreeData';
import renderTaskRows from './renderTaskRows';
import DagRuns from './dagRuns';
+import SidePanel from './SidePanel';
const Tree = () => {
const containerRef = useRef();
const scrollRef = useRef();
const { data: { groups = {} }, isRefreshOn, onToggleRefresh } = useTreeData();
+ const [selectedInstance, setSelectedInstance] = useState({});
useEffect(() => {
// Set initial scroll to far right if it is scrollable
@@ -47,6 +50,13 @@ const Tree = () => {
}
}, []);
+ const { runId, taskId } = selectedInstance;
+ const onSelectInstance = (newInstance) => (
+ (newInstance.runId === runId && newInstance.taskId === taskId)
+ ? setSelectedInstance({})
+ : setSelectedInstance(newInstance)
+ );
+
return (
<Box position="relative" ref={containerRef}>
<FormControl display="flex" alignItems="center" justifyContent="flex-end" width="100%">
@@ -58,17 +68,20 @@ const Tree = () => {
</FormControl>
<Text transform="rotate(-90deg)" position="absolute" left="-6px" top="130px">Runs</Text>
<Text transform="rotate(-90deg)" position="absolute" left="-6px" top="190px">Tasks</Text>
- <Box px="24px">
- <Box position="relative" width="100%" overflowX="auto" ref={scrollRef}>
- <Table>
+ <Box pl="24px">
+ <Flex position="relative" flexDirection="row" justifyContent="space-between" overflow="hidden">
+ <Table mr="24px" overflowX="auto" ref={scrollRef} height={0}>
<Thead>
<DagRuns containerRef={containerRef} />
</Thead>
<Tbody>
- {renderTaskRows({ task: groups, containerRef })}
+ {renderTaskRows({
+ task: groups, containerRef, onSelectInstance,
+ })}
</Tbody>
</Table>
- </Box>
+ <SidePanel isOpen={!!runId} instance={selectedInstance} />
+ </Flex>
</Box>
</Box>
);
diff --git a/airflow/www/static/js/tree/renderTaskRows.jsx b/airflow/www/static/js/tree/renderTaskRows.jsx
index afd97e6..f5a7ba0 100644
--- a/airflow/www/static/js/tree/renderTaskRows.jsx
+++ b/airflow/www/static/js/tree/renderTaskRows.jsx
@@ -40,7 +40,7 @@ import { getMetaValue } from '../utils';
const dagId = getMetaValue('dag_id');
const renderTaskRows = ({
- task, containerRef, level = 0, isParentOpen,
+ task, containerRef, level = 0, isParentOpen, onSelectInstance,
}) => task.children.map((t) => (
<Row
key={t.id}
@@ -49,6 +49,7 @@ const renderTaskRows = ({
containerRef={containerRef}
prevTaskId={task.id}
isParentOpen={isParentOpen}
+ onSelectInstance={onSelectInstance}
/>
));
@@ -85,7 +86,9 @@ const TaskName = ({
</Box>
);
-const TaskInstances = ({ task, containerRef, dagRuns }) => (
+const TaskInstances = ({
+ task, containerRef, dagRuns, onSelectInstance,
+}) => (
<Flex justifyContent="flex-end">
{dagRuns.map((run) => {
// Check if an instance exists for the run, or return an empty box
@@ -98,6 +101,7 @@ const TaskInstances = ({ task, containerRef, dagRuns }) => (
containerRef={containerRef}
extraLinks={task.extraLinks}
group={task}
+ onSelectInstance={onSelectInstance}
/>
)
: <Box key={`${run.runId}-${task.id}`} width="18px" data-testid="blank-task" />;
@@ -106,7 +110,7 @@ const TaskInstances = ({ task, containerRef, dagRuns }) => (
);
const Row = ({
- task, containerRef, level, prevTaskId, isParentOpen = true,
+ task, containerRef, level, prevTaskId, isParentOpen = true, onSelectInstance,
}) => {
const { data: { dagRuns = [] } } = useTreeData();
const isGroup = !!task.children;
@@ -162,13 +166,18 @@ const Row = ({
<Td width={0} p={0} borderBottom={0} />
<Td p={0} align="right" _groupHover={{ backgroundColor: 'rgba(113, 128, 150, 0.1)' }} transition="background-color 0.2s" borderBottom={0}>
<Collapse in={isFullyOpen}>
- <TaskInstances dagRuns={dagRuns} task={task} containerRef={containerRef} />
+ <TaskInstances
+ dagRuns={dagRuns}
+ task={task}
+ containerRef={containerRef}
+ onSelectInstance={onSelectInstance}
+ />
</Collapse>
</Td>
</Tr>
{isGroup && (
renderTaskRows({
- task, containerRef, level: level + 1, isParentOpen: isOpen,
+ task, containerRef, level: level + 1, isParentOpen: isOpen, onSelectInstance,
})
)}
</>
[airflow] 04/09: improve rendering and add selected dag run
Posted by bb...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bbovenzi pushed a commit to branch mapped-task-drawer
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit e22bcc622c84b67f0cf96ee8ea29ef3ec5a31ad2
Author: Brent Bovenzi <br...@gmail.com>
AuthorDate: Mon Feb 28 12:24:27 2022 -0500
improve rendering and add selected dag run
---
airflow/www/static/js/tree/SidePanel.jsx | 28 +++++++++++++++++--------
airflow/www/static/js/tree/StatusBox.jsx | 24 ++++++++++-----------
airflow/www/static/js/tree/Tree.jsx | 12 ++++++++---
airflow/www/static/js/tree/dagRuns/Bar.jsx | 9 ++++----
airflow/www/static/js/tree/dagRuns/index.jsx | 3 ++-
airflow/www/static/js/tree/renderTaskRows.jsx | 30 ++++++++++++++++++---------
6 files changed, 65 insertions(+), 41 deletions(-)
diff --git a/airflow/www/static/js/tree/SidePanel.jsx b/airflow/www/static/js/tree/SidePanel.jsx
index 7a21eba..88439a0 100644
--- a/airflow/www/static/js/tree/SidePanel.jsx
+++ b/airflow/www/static/js/tree/SidePanel.jsx
@@ -25,8 +25,6 @@ import {
Flex,
Text,
useDisclosure,
- CloseButton,
- Button,
IconButton,
} from '@chakra-ui/react';
import { MdKeyboardArrowLeft, MdKeyboardArrowRight } from 'react-icons/md';
@@ -38,30 +36,42 @@ const SidePanel = ({ instance: { runId, taskId, executionDate } }) => {
if (!isOpen) {
return (
<IconButton
- ml={2}
+ m={2}
icon={<MdKeyboardArrowLeft size={18} />}
aria-label="Open Details Panel"
onClick={onOpen}
/>
);
}
- const title = runId && taskId
- ? (
+ let title = '';
+ if (runId && taskId) {
+ title = (
<>
<chakra.span>Task Instance: </chakra.span>
<chakra.b>{taskId}</chakra.b>
<chakra.span> at </chakra.span>
<chakra.b>{formatDateTime(moment.utc(executionDate))}</chakra.b>
</>
- )
- : (
+ );
+ } else if (runId) {
+ title = (
+ <>
+ <chakra.span>Dag Run: </chakra.span>
+ <chakra.b>{runId}</chakra.b>
+ <chakra.span> at </chakra.span>
+ <chakra.b>{formatDateTime(moment.utc(executionDate))}</chakra.b>
+ </>
+ );
+ } else {
+ title = (
<chakra.span>Dag Details: </chakra.span>
);
+ }
return (
- <Flex bg="gray.200" maxWidth={300} minWidth={300} flexDirection="column" p={3}>
+ <Flex bg="gray.200" maxWidth={isOpen ? '50%' : 0} minWidth={isOpen ? 300 : 0} flexDirection="column" p={3}>
<IconButton
- ml={2}
+ m={2}
icon={<MdKeyboardArrowRight size={18} />}
aria-label="Close Details Panel"
onClick={onClose}
diff --git a/airflow/www/static/js/tree/StatusBox.jsx b/airflow/www/static/js/tree/StatusBox.jsx
index b4abc93..815e6a1 100644
--- a/airflow/www/static/js/tree/StatusBox.jsx
+++ b/airflow/www/static/js/tree/StatusBox.jsx
@@ -27,34 +27,32 @@ import {
useTheme,
} from '@chakra-ui/react';
-// import { callModal } from '../dag';
import InstanceTooltip from './InstanceTooltip';
const StatusBox = ({
- group, instance, containerRef, extraLinks = [], onSelectInstance, ...rest
+ group, instance, containerRef, onSelectInstance, selectedInstance, ...rest
}) => {
- const {
- executionDate, taskId, tryNumber = 0, operator, runId,
- } = instance;
+ const { runId } = instance;
const { colors } = useTheme();
const hoverBlue = `${colors.blue[100]}50`;
- // const onOpenModal = () => executionDate && callModal(taskId, executionDate, extraLinks, tryNumber, operator === 'SubDagOperator' || undefined, runId);
- const onClick = (e) => {
- e.stopPropagation();
- onSelectInstance(instance);
- };
-
// Fetch the corresponding column element and set its background color when hovering
const onMouseOver = () => {
- [...containerRef.current.getElementsByClassName(`js-${runId}`)]
- .forEach((e) => { e.style.backgroundColor = hoverBlue; });
+ if (selectedInstance.runId !== runId) {
+ [...containerRef.current.getElementsByClassName(`js-${runId}`)]
+ .forEach((e) => { e.style.backgroundColor = hoverBlue; });
+ }
};
const onMouseLeave = () => {
[...containerRef.current.getElementsByClassName(`js-${runId}`)]
.forEach((e) => { e.style.backgroundColor = null; });
};
+ const onClick = (e) => {
+ e.stopPropagation();
+ onSelectInstance(instance);
+ };
+
return (
<>
<Tooltip
diff --git a/airflow/www/static/js/tree/Tree.jsx b/airflow/www/static/js/tree/Tree.jsx
index ff1402f..199e373 100644
--- a/airflow/www/static/js/tree/Tree.jsx
+++ b/airflow/www/static/js/tree/Tree.jsx
@@ -39,9 +39,11 @@ import SidePanel from './SidePanel';
const Tree = () => {
const containerRef = useRef();
const scrollRef = useRef();
- const { data: { groups = {} }, isRefreshOn, onToggleRefresh } = useTreeData();
+ const { data: { groups = {}, dagRuns = [] }, isRefreshOn, onToggleRefresh } = useTreeData();
const [selectedInstance, setSelectedInstance] = useState({});
+ const dagRunIds = dagRuns.map((dr) => dr.runId);
+
useEffect(() => {
// Set initial scroll to far right if it is scrollable
const runsContainer = scrollRef.current;
@@ -73,11 +75,15 @@ const Tree = () => {
<Box mr="12px" pb="12px" overflowX="auto" ref={scrollRef} maxWidth="60vw">
<Table height={0}>
<Thead>
- <DagRuns containerRef={containerRef} selectedInstance={selectedInstance} />
+ <DagRuns
+ containerRef={containerRef}
+ selectedInstance={selectedInstance}
+ onSelectInstance={onSelectInstance}
+ />
</Thead>
<Tbody>
{renderTaskRows({
- task: groups, containerRef, onSelectInstance, selectedInstance,
+ task: groups, containerRef, onSelectInstance, selectedInstance, dagRunIds,
})}
</Tbody>
</Table>
diff --git a/airflow/www/static/js/tree/dagRuns/Bar.jsx b/airflow/www/static/js/tree/dagRuns/Bar.jsx
index 4dbe20d..c64abdd 100644
--- a/airflow/www/static/js/tree/dagRuns/Bar.jsx
+++ b/airflow/www/static/js/tree/dagRuns/Bar.jsx
@@ -31,12 +31,11 @@ import {
import { MdPlayArrow } from 'react-icons/md';
import DagRunTooltip from './Tooltip';
-import { callModalDag } from '../../dag';
const BAR_HEIGHT = 100;
const DagRunBar = ({
- run, max, index, totalRuns, containerRef, selectedInstance,
+ run, max, index, totalRuns, containerRef, selectedInstance, onSelectInstance,
}) => {
const { colors } = useTheme();
const hoverBlue = `${colors.blue[100]}50`;
@@ -53,14 +52,14 @@ const DagRunBar = ({
alignItems="flex-end"
justifyContent="center"
mb="2px"
- // py="2px"
px="2px"
mx="1px"
cursor="pointer"
width="14px"
zIndex={1}
- onClick={() => {
- callModalDag({ execution_date: run.executionDate, dag_id: run.dagId, run_id: run.runId });
+ onClick={(e) => {
+ e.stopPropagation();
+ onSelectInstance(run);
}}
position="relative"
data-peer
diff --git a/airflow/www/static/js/tree/dagRuns/index.jsx b/airflow/www/static/js/tree/dagRuns/index.jsx
index 7de949f..7459634 100644
--- a/airflow/www/static/js/tree/dagRuns/index.jsx
+++ b/airflow/www/static/js/tree/dagRuns/index.jsx
@@ -36,7 +36,7 @@ const DurationTick = ({ children, ...rest }) => (
</Text>
);
-const DagRuns = ({ containerRef, selectedInstance }) => {
+const DagRuns = ({ containerRef, selectedInstance, onSelectInstance }) => {
const { data: { dagRuns = [] } } = useTreeData();
const durations = [];
const runs = dagRuns.map((dagRun) => {
@@ -98,6 +98,7 @@ const DagRuns = ({ containerRef, selectedInstance }) => {
totalRuns={runs.length}
containerRef={containerRef}
selectedInstance={selectedInstance}
+ onSelectInstance={onSelectInstance}
/>
))}
</Flex>
diff --git a/airflow/www/static/js/tree/renderTaskRows.jsx b/airflow/www/static/js/tree/renderTaskRows.jsx
index 7eff252..9b454f0 100644
--- a/airflow/www/static/js/tree/renderTaskRows.jsx
+++ b/airflow/www/static/js/tree/renderTaskRows.jsx
@@ -32,7 +32,6 @@ import {
} from '@chakra-ui/react';
import { FiChevronUp, FiChevronDown } from 'react-icons/fi';
-import useTreeData from './useTreeData';
import StatusBox from './StatusBox';
import { getMetaValue } from '../utils';
@@ -41,7 +40,7 @@ import { getMetaValue } from '../utils';
const dagId = getMetaValue('dag_id');
const renderTaskRows = ({
- task, containerRef, level = 0, isParentOpen, onSelectInstance, selectedInstance,
+ task, containerRef, level = 0, isParentOpen, onSelectInstance, selectedInstance, dagRunIds
}) => task.children.map((t) => (
<Row
key={t.id}
@@ -52,6 +51,7 @@ const renderTaskRows = ({
isParentOpen={isParentOpen}
onSelectInstance={onSelectInstance}
selectedInstance={selectedInstance}
+ dagRunIds={dagRunIds}
/>
));
@@ -85,33 +85,42 @@ const TaskName = ({
);
const TaskInstances = ({
- task, containerRef, dagRuns, onSelectInstance,
+ task, containerRef, dagRunIds, onSelectInstance, selectedInstance,
}) => (
<Flex justifyContent="flex-end">
- {dagRuns.map((run) => {
+ {dagRunIds.map((runId) => {
// Check if an instance exists for the run, or return an empty box
- const instance = task.instances.find((gi) => gi.runId === run.runId);
+ const instance = task.instances.find((gi) => gi.runId === runId);
+ const key = `${runId}-${task.id}`;
return instance
? (
<StatusBox
- key={`${run.runId}-${task.id}`}
+ key={key}
instance={instance}
containerRef={containerRef}
extraLinks={task.extraLinks}
group={task}
onSelectInstance={onSelectInstance}
+ selectedInstance={selectedInstance}
/>
)
- : <Box key={`${run.runId}-${task.id}`} width="16px" data-testid="blank-task" />;
+ : <Box key={key} width="16px" data-testid="blank-task" />;
})}
</Flex>
);
const Row = (props) => {
const {
- task, containerRef, level, prevTaskId, isParentOpen = true, onSelectInstance, selectedInstance,
+ task,
+ containerRef,
+ level,
+ prevTaskId,
+ isParentOpen = true,
+ onSelectInstance,
+ selectedInstance,
+ dagRunIds,
} = props;
- const { data: { dagRuns = [] } } = useTreeData();
+ // const { data: { dagRuns = [] } } = useTreeData();
const { colors } = useTheme();
const hoverBlue = `${colors.blue[100]}50`;
const isGroup = !!task.children;
@@ -175,10 +184,11 @@ const Row = (props) => {
>
<Collapse in={isFullyOpen}>
<TaskInstances
- dagRuns={dagRuns}
+ dagRunIds={dagRunIds}
task={task}
containerRef={containerRef}
onSelectInstance={onSelectInstance}
+ selectedInstance={selectedInstance}
/>
</Collapse>
</Td>
[airflow] 09/09: make side panel collapsible, useTasks,
Posted by bb...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bbovenzi pushed a commit to branch mapped-task-drawer
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 1ef4d274ba4b82c4054fb2cebdf5bc72b12991e2
Author: Brent Bovenzi <br...@gmail.com>
AuthorDate: Wed Mar 2 14:23:10 2022 -0500
make side panel collapsible, useTasks,
---
airflow/www/static/js/tree/StatusBox.jsx | 3 +-
airflow/www/static/js/tree/Tree.jsx | 126 +++++++++++++------
airflow/www/static/js/tree/api/useDag.js | 2 +-
airflow/www/static/js/tree/dagRuns/Bar.jsx | 5 +-
airflow/www/static/js/tree/details/Header.jsx | 3 +-
airflow/www/static/js/tree/details/content/Dag.jsx | 27 +++-
airflow/www/static/js/tree/details/index.jsx | 1 -
airflow/www/static/js/tree/renderTaskRows.jsx | 8 +-
airflow/www/utils.py | 136 +++++++++++----------
9 files changed, 196 insertions(+), 115 deletions(-)
diff --git a/airflow/www/static/js/tree/StatusBox.jsx b/airflow/www/static/js/tree/StatusBox.jsx
index 62762c5..0f5294f 100644
--- a/airflow/www/static/js/tree/StatusBox.jsx
+++ b/airflow/www/static/js/tree/StatusBox.jsx
@@ -48,8 +48,7 @@ const StatusBox = ({
.forEach((e) => { e.style.backgroundColor = null; });
};
- const onClick = (e) => {
- e.stopPropagation();
+ const onClick = () => {
onMouseLeave();
onSelect({
taskId, runId, instance, task: group,
diff --git a/airflow/www/static/js/tree/Tree.jsx b/airflow/www/static/js/tree/Tree.jsx
index aeeaeb5..0d84639 100644
--- a/airflow/www/static/js/tree/Tree.jsx
+++ b/airflow/www/static/js/tree/Tree.jsx
@@ -17,6 +17,8 @@
* under the License.
*/
+/* global localStorage */
+
import React, { useRef, useEffect, useState } from 'react';
import {
Table,
@@ -29,18 +31,32 @@ import {
Text,
Thead,
Flex,
+ useDisclosure,
+ IconButton,
} from '@chakra-ui/react';
+import { MdArrowForward, MdArrowBack } from 'react-icons/md';
import useTreeData from './useTreeData';
import renderTaskRows from './renderTaskRows';
import DagRuns from './dagRuns';
import Details from './details';
+import { callModal, callModalDag } from '../dag';
+
+const sidePanelKey = 'showSidePanel';
const Tree = () => {
const containerRef = useRef();
const scrollRef = useRef();
const { data: { groups = {}, dagRuns = [] }, isRefreshOn, onToggleRefresh } = useTreeData();
- const [selected, setSelected] = useState({});
+ const [selected, setSelected] = useState({}); // selected task instance or dag run
+ const isPanelOpen = JSON.parse(localStorage.getItem(sidePanelKey));
+ const { isOpen, onToggle } = useDisclosure({ defaultIsOpen: isPanelOpen });
+
+ const toggleSidePanel = () => {
+ if (!isOpen) localStorage.setItem(sidePanelKey, true);
+ else localStorage.setItem(sidePanelKey, false);
+ onToggle();
+ };
const dagRunIds = dagRuns.map((dr) => dr.runId);
@@ -48,46 +64,88 @@ const Tree = () => {
// Set initial scroll to far right if it is scrollable
const runsContainer = scrollRef.current;
if (runsContainer && runsContainer.scrollWidth > runsContainer.clientWidth) {
- runsContainer.scrollBy(runsContainer.clientWidth, 0);
+ runsContainer.scrollBy(runsContainer.clientWidth, 250);
}
- }, []);
+ }, [isOpen]); // isOpen is to redo the scroll when the side panel opens/closes
const { runId, taskId } = selected;
- const onSelect = (newInstance) => (
- (newInstance.runId === runId && newInstance.taskId === taskId)
- ? setSelected({})
- : setSelected(newInstance)
- );
+
+ // show task/run info in the side panel, or just call the regular action modal
+ const onSelect = (newSelected) => {
+ if (isOpen) {
+ const isSame = newSelected.runId === runId && newSelected.taskId === taskId;
+ setSelected(isSame ? {} : newSelected);
+ } else if (!isOpen) {
+ if (newSelected.dagRun) {
+ const { dagRun } = newSelected;
+ callModalDag({
+ execution_date: dagRun.executionDate,
+ dag_id: dagRun.dagId,
+ run_id: dagRun.runId,
+ });
+ } else if (newSelected.instance) {
+ const extraLinks = newSelected.task.extraLinks || [];
+ const { instance } = newSelected;
+ callModal(
+ taskId,
+ instance.executionDate,
+ extraLinks,
+ instance.tryNumber,
+ instance.operator === 'SubDagOperator' || undefined,
+ instance.runId,
+ );
+ }
+ }
+ };
return (
- <Flex pl="24px" position="relative" flexDirection="row" justifyContent="space-between" ref={containerRef}>
+ <Box pl="24px" position="relative" ref={containerRef}>
<Text transform="rotate(-90deg)" position="absolute" left="-6px" top="130px">Runs</Text>
<Text transform="rotate(-90deg)" position="absolute" left="-6px" top="190px">Tasks</Text>
- <FormControl display="flex" position="absolute" left="220px">
- {isRefreshOn && <Spinner color="blue.500" speed="1s" mr="4px" />}
- <FormLabel htmlFor="auto-refresh" mb={0} fontSize="12px" fontWeight="normal">
- Auto-refresh
- </FormLabel>
- <Switch id="auto-refresh" onChange={onToggleRefresh} isChecked={isRefreshOn} size="lg" />
- </FormControl>
- <Box mr="12px" pb="12px" overflowX="auto" ref={scrollRef} maxWidth="300px" minWidth="300px" position="relative" mt="24px">
- <Table height={0}>
- <Thead>
- <DagRuns
- containerRef={containerRef}
- selected={selected}
- onSelect={onSelect}
- />
- </Thead>
- <Tbody>
- {renderTaskRows({
- task: groups, containerRef, onSelect, selected, dagRunIds,
- })}
- </Tbody>
- </Table>
- </Box>
- <Details selected={selected} onSelect={onSelect} />
- </Flex>
+ <Flex flexGrow={1} justifyContent="flex-end" alignItems="center">
+ <FormControl display="flex" width="auto" mr={2}>
+ {isRefreshOn && <Spinner color="blue.500" speed="1s" mr="4px" />}
+ <FormLabel htmlFor="auto-refresh" mb={0} fontSize="12px" fontWeight="normal">
+ Auto-refresh
+ </FormLabel>
+ <Switch id="auto-refresh" onChange={onToggleRefresh} isChecked={isRefreshOn} size="lg" />
+ </FormControl>
+ <IconButton onClick={toggleSidePanel}>
+ {isOpen
+ ? <MdArrowForward size="18px" aria-label="Collapse Details" title="Collapse Details" />
+ : <MdArrowBack size="18px" title="Expand Details" aria-label="Expand Details" />}
+ </IconButton>
+ </Flex>
+ <Flex flexDirection="row" justifyContent="space-between">
+ <Box
+ mr="12px"
+ mt="24px"
+ pb="12px"
+ overflowX="auto"
+ ref={scrollRef}
+ maxWidth={isOpen && '300px'}
+ minWidth={isOpen && '300px'}
+ >
+ <Table height={0}>
+ <Thead>
+ <DagRuns
+ containerRef={containerRef}
+ selected={selected}
+ onSelect={onSelect}
+ />
+ </Thead>
+ <Tbody>
+ {renderTaskRows({
+ task: groups, containerRef, onSelect, selected, dagRunIds,
+ })}
+ </Tbody>
+ </Table>
+ </Box>
+ {isOpen && (
+ <Details selected={selected} onSelect={onSelect} />
+ )}
+ </Flex>
+ </Box>
);
};
diff --git a/airflow/www/static/js/tree/api/useDag.js b/airflow/www/static/js/tree/api/useDag.js
index 6c19ee4..1343302 100644
--- a/airflow/www/static/js/tree/api/useDag.js
+++ b/airflow/www/static/js/tree/api/useDag.js
@@ -23,6 +23,6 @@ import { useQuery } from 'react-query';
export default function useDag(dagId) {
return useQuery(
['dag', dagId],
- () => axios.get(`/dags/${dagId}`),
+ () => axios.get(`/dags/${dagId}/details`),
);
}
diff --git a/airflow/www/static/js/tree/dagRuns/Bar.jsx b/airflow/www/static/js/tree/dagRuns/Bar.jsx
index 94c84e7..14df756 100644
--- a/airflow/www/static/js/tree/dagRuns/Bar.jsx
+++ b/airflow/www/static/js/tree/dagRuns/Bar.jsx
@@ -57,10 +57,7 @@ const DagRunBar = ({
cursor="pointer"
width="14px"
zIndex={1}
- onClick={(e) => {
- e.stopPropagation();
- onSelect({ runId: run.runId, dagRun: run });
- }}
+ onClick={() => onSelect({ runId: run.runId, dagRun: run })}
position="relative"
data-peer
>
diff --git a/airflow/www/static/js/tree/details/Header.jsx b/airflow/www/static/js/tree/details/Header.jsx
index c362606..2982a4e 100644
--- a/airflow/www/static/js/tree/details/Header.jsx
+++ b/airflow/www/static/js/tree/details/Header.jsx
@@ -45,7 +45,6 @@ const Header = ({
dagRuns,
}) => {
const dagRun = dagRuns.find((r) => r.runId === runId);
- // console.log(dagRun);
let runLabel = dagRun ? formatDateTime(dagRun.dataIntervalEnd) : '';
if (dagRun && dagRun.runType === 'manual') {
runLabel = (
@@ -65,7 +64,7 @@ const Header = ({
</BreadcrumbItem>
{runId && (
<BreadcrumbItem isCurrentPage={runId && !taskId}>
- <BreadcrumbLink onClick={() => onSelect({ runId })}>
+ <BreadcrumbLink onClick={() => onSelect({ runId, dagRun })}>
<LabelValue label="Run" value={runLabel} />
</BreadcrumbLink>
</BreadcrumbItem>
diff --git a/airflow/www/static/js/tree/details/content/Dag.jsx b/airflow/www/static/js/tree/details/content/Dag.jsx
index e71d9c2..7ee19ec 100644
--- a/airflow/www/static/js/tree/details/content/Dag.jsx
+++ b/airflow/www/static/js/tree/details/content/Dag.jsx
@@ -27,13 +27,23 @@ import {
} from '@chakra-ui/react';
import { getMetaValue } from '../../../utils';
-import { useDag } from '../../api';
+import { useDag, useTasks } from '../../api';
const dagId = getMetaValue('dag_id');
const Dag = () => {
const { data: dag } = useDag(dagId);
- if (!dag) return null;
+ const { data: taskData } = useTasks(dagId);
+ if (!dag || !taskData) return null;
+ const { tasks = [], totalEntries = '' } = taskData;
+ const operators = {};
+ tasks.forEach((t) => {
+ if (!operators[t.classRef.className]) {
+ operators[t.classRef.className] = 1;
+ } else {
+ operators[t.classRef.className] += 1;
+ }
+ });
const {
description, tags, fileloc, owners,
} = dag;
@@ -50,6 +60,19 @@ const Dag = () => {
<Text mr={2}>Owner:</Text>
{owners.map((o) => <Text key={o}>{o}</Text>)}
</Flex>
+ <Text>
+ {totalEntries}
+ {' '}
+ Tasks
+ </Text>
+ {Object.entries(operators).map(([key, value]) => (
+ <Text key={key}>
+ {value}
+ {' '}
+ {key}
+ {value > 1 && 's'}
+ </Text>
+ ))}
</>
);
};
diff --git a/airflow/www/static/js/tree/details/index.jsx b/airflow/www/static/js/tree/details/index.jsx
index ee3b044..c932b05 100644
--- a/airflow/www/static/js/tree/details/index.jsx
+++ b/airflow/www/static/js/tree/details/index.jsx
@@ -35,7 +35,6 @@ const Details = ({
onSelect,
}) => {
const { data: { dagRuns = [] } } = useTreeData();
- console.log(selected);
return (
<Flex borderLeftWidth="1px" flexDirection="column" p={3} flexGrow={1}>
<Header selected={selected} onSelect={onSelect} dagRuns={dagRuns} />
diff --git a/airflow/www/static/js/tree/renderTaskRows.jsx b/airflow/www/static/js/tree/renderTaskRows.jsx
index 5e8234e..00dad85 100644
--- a/airflow/www/static/js/tree/renderTaskRows.jsx
+++ b/airflow/www/static/js/tree/renderTaskRows.jsx
@@ -40,18 +40,14 @@ import { getMetaValue } from '../utils';
const dagId = getMetaValue('dag_id');
const renderTaskRows = ({
- task, containerRef, level = 0, isParentOpen, onSelect, selected, dagRunIds,
+ task, level = 0, ...rest
}) => task.children.map((t) => (
<Row
+ {...rest}
key={t.id}
task={t}
level={level}
- containerRef={containerRef}
prevTaskId={task.id}
- isParentOpen={isParentOpen}
- onSelect={onSelect}
- selected={selected}
- dagRunIds={dagRunIds}
/>
));
diff --git a/airflow/www/utils.py b/airflow/www/utils.py
index ed765d4..b1f5e0d 100644
--- a/airflow/www/utils.py
+++ b/airflow/www/utils.py
@@ -43,8 +43,9 @@ from airflow.models import errors
from airflow.models.taskinstance import TaskInstance
from airflow.utils import timezone
from airflow.utils.code_utils import get_python_source
+from airflow.utils.helpers import alchemy_to_dict
from airflow.utils.json import AirflowJsonEncoder
-from airflow.utils.state import State
+from airflow.utils.state import State, TaskInstanceState
from airflow.www.forms import DateTimeWithTimezoneField
from airflow.www.widgets import AirflowDateTimePickerWidget
@@ -55,13 +56,82 @@ def datetime_to_string(value: Optional[DateTime]) -> Optional[str]:
return value.isoformat()
+def get_mapped_instances(task_instance, session):
+ return (
+ session.query(TaskInstance)
+ .filter(
+ TaskInstance.dag_id == task_instance.dag_id,
+ TaskInstance.run_id == task_instance.run_id,
+ TaskInstance.task_id == task_instance.task_id,
+ TaskInstance.map_index >= 0,
+ )
+ .all()
+ )
+
+
+def get_instance_with_map(task_instance, session):
+ if task_instance.map_index == -1:
+ return alchemy_to_dict(task_instance)
+ mapped_instances = get_mapped_instances(task_instance, session)
+ return get_mapped_summary(task_instance, mapped_instances)
+
+
+def get_mapped_summary(parent_instance, task_instances):
+ priority = [
+ TaskInstanceState.FAILED,
+ TaskInstanceState.UPSTREAM_FAILED,
+ TaskInstanceState.UP_FOR_RETRY,
+ TaskInstanceState.UP_FOR_RESCHEDULE,
+ TaskInstanceState.QUEUED,
+ TaskInstanceState.SCHEDULED,
+ TaskInstanceState.DEFERRED,
+ TaskInstanceState.SENSING,
+ TaskInstanceState.RUNNING,
+ TaskInstanceState.SHUTDOWN,
+ TaskInstanceState.RESTARTING,
+ TaskInstanceState.REMOVED,
+ TaskInstanceState.SUCCESS,
+ TaskInstanceState.SKIPPED,
+ ]
+
+ mapped_states = [ti.state for ti in task_instances]
+
+ group_state = None
+ for state in priority:
+ if state in mapped_states:
+ group_state = state
+ break
+
+ group_start_date = datetime_to_string(
+ min((ti.start_date for ti in task_instances if ti.start_date), default=None)
+ )
+ group_end_date = datetime_to_string(
+ max((ti.end_date for ti in task_instances if ti.end_date), default=None)
+ )
+
+ return {
+ 'task_id': parent_instance.task_id,
+ 'run_id': parent_instance.run_id,
+ 'state': group_state,
+ 'start_date': group_start_date,
+ 'end_date': group_end_date,
+ 'mapped_states': mapped_states,
+ 'operator': parent_instance.operator,
+ 'execution_date': datetime_to_string(parent_instance.execution_date),
+ 'try_number': parent_instance.try_number,
+ }
+
+
def encode_ti(
- task_instance: Optional[TaskInstance], is_mapped: Optional[bool], session: Session
+ task_instance: Optional[TaskInstance], is_mapped: Optional[bool], session: Optional[Session]
) -> Optional[Dict[str, Any]]:
if not task_instance:
return None
- summary = {
+ if is_mapped:
+ return get_mapped_summary(task_instance, task_instances=get_mapped_instances(task_instance, session))
+
+ return {
'task_id': task_instance.task_id,
'dag_id': task_instance.dag_id,
'run_id': task_instance.run_id,
@@ -74,66 +144,6 @@ def encode_ti(
'try_number': task_instance.try_number,
}
- def get_mapped_summary(task_instances):
- priority = [
- 'failed',
- 'upstream_failed',
- 'up_for_retry',
- 'up_for_reschedule',
- 'queued',
- 'scheduled',
- 'deferred',
- 'sensing',
- 'running',
- 'shutdown',
- 'restarting',
- 'removed',
- 'no_status',
- 'success',
- 'skipped',
- ]
-
- mapped_states = [ti.state for ti in task_instances]
-
- group_state = None
- for state in priority:
- if state in mapped_states:
- group_state = state
- break
-
- group_start_date = datetime_to_string(
- min((ti.start_date for ti in task_instances if ti.start_date), default=None)
- )
- group_end_date = datetime_to_string(
- max((ti.end_date for ti in task_instances if ti.end_date), default=None)
- )
-
- return {
- 'task_id': task_instance.task_id,
- 'run_id': task_instance.run_id,
- 'state': group_state,
- 'start_date': group_start_date,
- 'end_date': group_end_date,
- 'mapped_states': mapped_states,
- 'operator': task_instance.operator,
- 'execution_date': datetime_to_string(task_instance.execution_date),
- 'try_number': task_instance.try_number,
- }
-
- if is_mapped:
- return get_mapped_summary(
- session.query(TaskInstance)
- .filter(
- TaskInstance.dag_id == task_instance.dag_id,
- TaskInstance.run_id == task_instance.run_id,
- TaskInstance.task_id == task_instance.task_id,
- TaskInstance.map_index >= 0,
- )
- .all()
- )
-
- return summary
-
def encode_dag_run(dag_run: Optional[models.DagRun]) -> Optional[Dict[str, Any]]:
if not dag_run:
[airflow] 05/09: fix hover and extra prop
Posted by bb...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bbovenzi pushed a commit to branch mapped-task-drawer
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 28cf8f03cceb0dd599d85adacdd738c7993f838f
Author: Brent Bovenzi <br...@gmail.com>
AuthorDate: Mon Feb 28 21:01:53 2022 -0500
fix hover and extra prop
---
airflow/www/static/js/tree/StatusBox.jsx | 5 +++--
airflow/www/static/js/tree/renderTaskRows.jsx | 1 -
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/airflow/www/static/js/tree/StatusBox.jsx b/airflow/www/static/js/tree/StatusBox.jsx
index 815e6a1..0953a79 100644
--- a/airflow/www/static/js/tree/StatusBox.jsx
+++ b/airflow/www/static/js/tree/StatusBox.jsx
@@ -37,7 +37,7 @@ const StatusBox = ({
const hoverBlue = `${colors.blue[100]}50`;
// Fetch the corresponding column element and set its background color when hovering
- const onMouseOver = () => {
+ const onMouseEnter = () => {
if (selectedInstance.runId !== runId) {
[...containerRef.current.getElementsByClassName(`js-${runId}`)]
.forEach((e) => { e.style.backgroundColor = hoverBlue; });
@@ -50,6 +50,7 @@ const StatusBox = ({
const onClick = (e) => {
e.stopPropagation();
+ onMouseLeave();
onSelectInstance(instance);
};
@@ -73,7 +74,7 @@ const StatusBox = ({
cursor={!group.children && 'pointer'}
data-testid="task-instance"
zIndex={1}
- onMouseEnter={onMouseOver}
+ onMouseEnter={onMouseEnter}
onMouseLeave={onMouseLeave}
{...rest}
>
diff --git a/airflow/www/static/js/tree/renderTaskRows.jsx b/airflow/www/static/js/tree/renderTaskRows.jsx
index 9b454f0..7440774 100644
--- a/airflow/www/static/js/tree/renderTaskRows.jsx
+++ b/airflow/www/static/js/tree/renderTaskRows.jsx
@@ -98,7 +98,6 @@ const TaskInstances = ({
key={key}
instance={instance}
containerRef={containerRef}
- extraLinks={task.extraLinks}
group={task}
onSelectInstance={onSelectInstance}
selectedInstance={selectedInstance}