integrations/.github/workflows/sweep-dashboard-data-scope.yml at master · efd6/integrations · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
name: "Sweeper: Dashboard Data Scope and Filter Integrity"
on:
  schedule:
    - cron: "0 9 * * 4"
  workflow_dispatch:

permissions:
  actions: read
  contents: read
  copilot-requests: write
  issues: write
  pull-requests: read

jobs:
  run:
    # Disabled 2026-07-02 — no current owner. Re-enable by removing this
    # `if: false` line via a reviewed PR so the change is visible.
    if: false
    uses: elastic/ai-github-actions/.github/workflows/gh-aw-code-quality-audit.lock.yml@v0
    with:
      title-prefix: "[dashboard-data-scope]"
      severity-threshold: "high"
      additional-instructions: |
        You are auditing dashboards in the elastic/integrations repository for data scope
        bugs. Find cases that would matter to a real user: dashboards where opening the page
        shows data from unrelated integrations, or where a panel is completely broken because
        its data source reference does not exist. One confirmed bug that any user with multiple
        integrations installed would immediately notice is worth more than a list of dashboards
        with broad-looking patterns that turn out to be harmless.

        ## What dashboards guarantee

        Dashboard JSON files live at `packages/<pkg>/kibana/dashboard/*.json`. Each dashboard
        belongs to exactly one integration and should only query that integration's own data
        streams. A dashboard that queries `logs-*` instead of `logs-aws.cloudtrail-*` shows
        data from every other integration installed in the cluster — the user opens the AWS
        CloudTrail dashboard and sees Kubernetes, Windows, and Cisco logs mixed in. Control
        panels are the most damaging: a filter dropdown querying `logs-*` shows field values
        from every integration's data, making it impossible to filter meaningfully.

        Stale `indexPatternId` references — a static Kibana saved object ID rather than a
        dynamically resolved data view — produce "no data source" errors on panels in Kibana
        installations where that saved object does not exist.

        ## How to investigate

        Scan for candidates:
        ```bash
        python3 -c "
        import json, glob, re
        for f in glob.glob('packages/*/kibana/dashboard/*.json'):
            with open(f) as fh:
                try: content = fh.read()
                except: continue
            if re.search(r'\"(logs|metrics)-\*\"', content):
                print(f)
        " 2>/dev/null
        ```

        For each candidate, open the file and understand what role the broad pattern plays.
        This is the critical judgment: a `logs-*` string in the `indexRefName` of a
        visualization or the data source of a control panel is always wrong. A `logs-*`
        string in a panel title, description, or filter label that does not affect what data
        is actually queried is harmless. Do not file the harmless ones.

        For confirmed scope bugs, determine the correct scoped pattern for the package. It
        is always `<type>-<package>.<datastream>-*`, e.g. `logs-aws.cloudtrail-*`. Read
        the package's `manifest.yml` and data stream definitions to confirm.

        Check `validation.yml` for suppressed `SVR00002` checks:
        ```bash
        grep -rl 'SVR00002\|exclude_checks' packages/*/validation.yml 2>/dev/null
        ```
        A suppressed check without a comment explaining why is itself a finding — the
        problem was identified by the validator and deliberately left unresolved.

        For any confirmed finding in a high-traffic package, check the git history:
        `git log --oneline -- packages/<pkg>/kibana/dashboard/`
        Dashboards are sometimes regenerated in bulk migrations that inadvertently broaden
        data view references. A broad pattern introduced in a recent bulk commit is a
        regression affecting everyone who updated.

        ## Verify before filing

        For each scope finding, confirm all three:
        1. The broad pattern is the actual data source for a query or control panel, not an
           incidental string in metadata
        2. The package has a correct scoped pattern that should be used instead
        3. The bug is visible to any user who has more than one integration installed

        For stale `indexPatternId` references, confirm the referenced ID is not a data view
        the integration installs as part of its own assets.

        ## Output

        File a single issue containing:
        - Control panels and visualizations with over-broad data sources, by dashboard, with
          the correct scoped pattern for each
        - Stale `indexPatternId` references that produce "no data source" errors, by package
        - Packages with `SVR00002` suppressed without explanation
        - Total dashboards scanned and number with confirmed findings