feat: new manifest format

[atilafassina]

This branch evolves the plugin manifest system from v1.0 to v2.0, adding three major capabilities to support smarter scaffolding by the databricks apps init CLI.

Schema Extensions (commit 4c42edd)

• discoveryDescriptor — new schema definition describing how the CLI auto-discovers resource field values via Databricks CLI commands (e.g.
  listing warehouses). Supports cliCommand (with placeholder), selectField/displayField for interactive selection, dependsOn for
  inter-field ordering, and a shortcut for single-value fast paths.
• postScaffoldStep — ordered instructions shown to the user after project init (e.g. "run migrations", "configure secrets").
• origin field on resource entries — computed value indicating where a field's value comes from ("platform", "static", "cli", or "user").

Origin Computation & Scaffolding Descriptor (commit 7fe3d74)

• computeOrigin() derives origin from field properties:
  - local only → platform
  - value present → static
  - resolve present → cli
  - user input → user.
• enrichFieldsWithOrigin() injects computed origin into every resource field at sync time.
• Adds a scaffolding descriptor to the template manifest with the databricks apps init command signature, flags, and guardrail rules
  (never/must).
• Bumps manifest version from "1.0" → "2.0".

Semantic Validation (commit b905167)

Four new cross-field validators added to appkit plugin validate:

1. dependsOn cycle detection — DFS-based check for circular dependencies between discovery fields.
2. dependsOn dangling reference — errors if a field references a non-existent sibling.
3. Discovery <PROFILE> placeholder — ensures every cliCommand includes the placeholder.
4. Discovery/origin coherence — warns if a field has discovery but its computed origin isn't "user" (meaning discovery would never be used).
5. postScaffold structural validation — verifies array structure and non-empty instructions.

Core Plugin Annotations (commit d8cbbf4)

Annotates all four built-in plugin manifests (analytics, files, genie, lakebase) with:

• discovery descriptors on resource fields (e.g. warehouse listing, catalog/schema browsing)
• postScaffold steps (e.g. "Create a SQL warehouse", "Enable Lakebase")

Schema Fix (commit eb65776)

Inlines resourceFieldEntry and resourceRequirement definitions in the template schema so origin and discovery are properly represented without $ref resolution issues.

Test Results

Run 1

Metric	Delta	Value
Generated	--	100% (5/5)
Eval Score	↑ 0.03	1.00
Gen Turns	↓ 4.40	41.2
Build Pass	--	5 / 5
Tests Pass	↑ 1	10 (U5 S5)
Edit Gen %	--	100% (4/4)
Edit Score	--	100% (4/4)
Edit Turns	↑ 2	22.5
Edit Tests	--	4 / 4

[Copilot]

Pull request overview

This PR upgrades the AppKit plugin manifest ecosystem to a v2.0 template manifest format to support smarter databricks apps init scaffolding, including field discovery metadata, post-scaffold user instructions, computed field origins, and semantic (cross-field) validation during plugin validate.

Changes:

• Bump template plugin manifest version to 2.0 and add a top-level scaffolding descriptor.
• Extend plugin/template schemas with discovery (CLI command-based value discovery) and postScaffold steps; generate/propagate computed origin into template manifests during sync.
• Add semantic validation (dependsOn cycles/dangling refs, <PROFILE> placeholder, discovery/origin coherence, postScaffold structure) and associated tests/docs updates.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
template/appkit.plugins.json	Updates template manifest to v2.0 and annotates built-in plugins with discovery/origin/postScaffold plus scaffolding descriptor.
packages/shared/src/schemas/template-plugins.schema.json	Expands template manifest schema to support v2.0 and scaffolding descriptor; inlines field/requirement defs for origin.
packages/shared/src/schemas/plugin-manifest.schema.json	Adds discovery to resource fields and postScaffold to plugin manifests.
packages/shared/src/schemas/plugin-manifest.generated.ts	Updates generated TS types to include discovery/postScaffold types.
packages/shared/src/plugin.ts	Re-exports new generated types (DiscoveryDescriptor, PostScaffoldStep).
packages/shared/src/cli/commands/plugin/validate/validate.ts	Runs semantic validation and formats semantic errors/warnings in CLI output.
packages/shared/src/cli/commands/plugin/validate/validate-manifest.ts	Implements semantic validation rules and issue formatting.
packages/shared/src/cli/commands/plugin/validate/validate-manifest.test.ts	Adds test coverage for new semantic validation behavior.
packages/shared/src/cli/commands/plugin/sync/sync.ts	Computes/injects origin, bumps template manifest to v2.0, and adds scaffolding descriptor on write.
packages/shared/src/cli/commands/plugin/sync/sync.test.ts	Adds unit tests for origin computation.
packages/shared/src/cli/commands/plugin/manifest-types.ts	Adds scaffolding descriptor types and exports new manifest-related types.
packages/appkit/src/plugins/lakebase/manifest.json	Adds discovery descriptors and postScaffold steps to the lakebase built-in plugin manifest.
packages/appkit/src/plugins/genie/manifest.json	Adds schema reference, discovery descriptor, and postScaffold steps to genie plugin manifest.
packages/appkit/src/plugins/files/manifest.json	Adds discovery descriptor and postScaffold steps to files plugin manifest.
packages/appkit/src/plugins/analytics/manifest.json	Adds discovery descriptor and postScaffold steps to analytics plugin manifest.
docs/static/schemas/template-plugins.schema.json	Publishes updated template plugins schema for docs site.
docs/static/schemas/plugin-manifest.schema.json	Publishes updated plugin manifest schema for docs site.
docs/static/appkit-ui/styles.gen.css	Updates generated UI styles (tailwind output) used by docs UI.
docs/docs/api/appkit/Interface.ResourceFieldEntry.md	Documents the new discovery field on ResourceFieldEntry.
docs/docs/api/appkit/Interface.PluginManifest.md	Documents the new postScaffold field on PluginManifest.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.