(superset) 01/01: feat(ci): add Chromatic visual regression testing for Storybook

[ru...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

rusackas pushed a commit to branch feat/chromatic-visual-testing
in repository https://gitbox.apache.org/repos/asf/superset.git

commit c56d46adc0699656c38525d356f440c30057fe07
Author: Claude Code <no...@anthropic.com>
AuthorDate: Wed Sep 9 11:15:06 2026 -0700

    feat(ci): add Chromatic visual regression testing for Storybook
    
    Publishes superset-frontend's Storybook to Chromatic on pushes to master
    and on pull requests touching superset-frontend, using the existing
    CHROMATIC_PROJECT_TOKEN repo secret.
    
    Chromatic was implemented here once before (#21095, 2021) and removed
    in #27232 (Feb 2024) for being unmaintained and overlapping with
    Applitools, which has since also been discontinued (no live references
    left, only historical CHANGELOG mentions) -- so that overlap no longer
    applies.
    
    Design choices, to keep this from repeating the prior setup's history
    of "fix:"/"remove, it has errors" churn:
    
    - Fork PRs don't receive CHROMATIC_PROJECT_TOKEN (GitHub withholds repo
      secrets from pull_request runs triggered by a fork); the publish
      steps guard on CHROMATIC_PROJECT_TOKEN != '' and simply no-op for
      them rather than failing, following the same env-var-gate pattern
      frontend-bundle-size-nightly.yml already uses for an optional secret.
    - Non-blocking for now (exitZeroOnChanges: true): visual changes show
      up as a PR check + comment for review, without gating merges. Can
      flip to a required check once the team trusts the signal.
    - TurboSnap enabled (onlyChanged: true) to keep runs fast against this
      monorepo's large chart-plugin story set; fetch-depth: 0 on checkout
      so it has the git history it needs to diff against the baseline.
    - Reuses the existing superset-node-ci Docker target and the
      build-then-mount-output pattern frontend-bundle-size-nightly.yml
      already established, rather than installing Node directly on the
      runner.
    
    chromaui/action is already wildcard-allowlisted in the ASF allowlist
    (chromaui/action@*), so this doesn't touch that check. Verified locally:
    pre-commit (zizmor) passes, actionlint passes (aside from a stale-label
    false positive on ubuntu-26.04 and $TAG-quoting notices that are already
    pervasive throughout the rest of this repo's workflows), and the real
    check_asf_allowlist.py script accepts the new chromaui/action ref.
    
    Co-Authored-By: Claude Sonnet 5 <no...@anthropic.com>
---
 .github/workflows/chromatic.yml | 108 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 108 insertions(+)

diff --git a/.github/workflows/chromatic.yml b/.github/workflows/chromatic.yml
new file mode 100644
index 00000000000..ad49c4330c8
--- /dev/null
+++ b/.github/workflows/chromatic.yml
@@ -0,0 +1,108 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Publishes superset-frontend's Storybook to Chromatic for visual
+# regression testing. See https://www.chromatic.com/docs/github-actions
+#
+# Runs on pushes to master (keeps the Chromatic baseline in sync with
+# mainline) and on pull requests that touch superset-frontend. Fork PRs
+# don't receive CHROMATIC_PROJECT_TOKEN -- GitHub withholds repository
+# secrets from pull_request runs triggered by a fork -- so the publish
+# steps below no-op for them (via the CHROMATIC_PROJECT_TOKEN != '' guard)
+# rather than failing.
+#
+# Non-blocking for now (exitZeroOnChanges: true): visual changes are
+# surfaced as a PR check/comment for review, not enforced as a merge gate.
+# A prior Chromatic setup here (#21095) was removed in #27232 for being
+# unmaintained and overlapping with Applitools (since also discontinued).
+# Keep this one simple and watch it before considering a required check.
+name: Chromatic
+
+on:
+  push:
+    branches:
+      - master
+    paths:
+      - "superset-frontend/**"
+  pull_request:
+    types: [synchronize, opened, reopened, ready_for_review]
+    paths:
+      - "superset-frontend/**"
+
+# cancel previous workflow jobs for PRs
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: true
+
+env:
+  TAG: apache/superset:chromatic-${{ github.run_id }}
+  CHROMATIC_PROJECT_TOKEN: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
+
+permissions:
+  contents: read
+
+jobs:
+  chromatic:
+    runs-on: ubuntu-26.04
+    timeout-minutes: 30
+    # pull-requests: write lets chromaui/action post its check and PR
+    # comment. Withheld automatically by GitHub for fork-triggered
+    # pull_request runs, same as CHROMATIC_PROJECT_TOKEN above.
+    permissions:
+      contents: read
+      pull-requests: write
+    steps:
+      - name: Checkout Code
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+          # TurboSnap (onlyChanged below) needs full history to diff
+          # against the baseline commit.
+          fetch-depth: 0
+          ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+
+      - name: Build Docker Image
+        if: ${{ env.CHROMATIC_PROJECT_TOKEN != '' }}
+        run: |
+          docker buildx build \
+            -t $TAG \
+            --cache-from=type=registry,ref=apache/superset-cache:3.11-slim-trixie \
+            --target superset-node-ci \
+            .
+
+      - name: Build Storybook
+        if: ${{ env.CHROMATIC_PROJECT_TOKEN != '' }}
+        run: |
+          mkdir -p ${{ github.workspace }}/superset-frontend/storybook-static
+          docker run \
+          -v ${{ github.workspace }}/superset-frontend/storybook-static:/app/superset-frontend/storybook-static \
+          --rm $TAG \
+          bash -c "npm i && npm run build-storybook"
+
+      - name: Publish to Chromatic
+        if: ${{ env.CHROMATIC_PROJECT_TOKEN != '' }}
+        uses: chromaui/action@6b31c4307e3f5a150ab5345b051bb40a62923a5f # v18.7.3
+        with:
+          projectToken: ${{ env.CHROMATIC_PROJECT_TOKEN }}
+          token: ${{ secrets.GITHUB_TOKEN }}
+          workingDir: superset-frontend
+          storybookBuildDir: storybook-static
+          # TurboSnap: only re-snapshot stories affected by files changed
+          # since the baseline build.
+          onlyChanged: true
+          exitZeroOnChanges: true
+          zip: true