Skip to content

feat(kit): add skill loader foundation#366

Open
gene9831 wants to merge 2 commits into
opentiny:developfrom
gene9831:codex/skill-loader-foundation
Open

feat(kit): add skill loader foundation#366
gene9831 wants to merge 2 commits into
opentiny:developfrom
gene9831:codex/skill-loader-foundation

Conversation

@gene9831

@gene9831 gene9831 commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

PR 描述

新增 skill 基础定义与 loader 能力,为后续 storage 和 skillPlugin 提供统一的 SkillDefinition 输入。

主要改动:

  • 新增 SkillDefinitionSkillResourceDescriptorSkillCandidate 等基础类型。
  • 新增 browser loader,支持 fileList / directoryHandle 加载 skill。
  • 新增 node loader,支持本地 fs skill 目录加载。
  • 新增 GitHub loader,支持从 repo/path/ref 加载 skill,并带最多 5 次指数退避重试。
  • 新增 loadSkill / loadSkillWithDetails,默认返回 SkillDefinition,details API 返回 warnings。
  • 新增 cancellable SkillLoadJob
  • 新增 @opentiny/tiny-robot-kit/node 子入口。
  • 新增 loader fixture 下载脚本和 browser/definition/node loader 测试。

验证:

pnpm -F @opentiny/tiny-robot-kit test -- --run src/skills/test/loaderBrowser.test.ts src/skills/test/loaderDefinition.test.ts src/skills/test/loaderNode.test.ts
pnpm build

Summary by CodeRabbit

Release Notes

  • New Features

    • Added multi-source skill loading capabilities supporting filesystem, GitHub, and browser-based sources.
    • Introduced new ./node package export for Node environment skill loading.
    • Added YAML parsing support for skill metadata.
  • Chores

    • Expanded build configuration to include additional entry points.

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@gene9831, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 31 minutes and 1 second. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: dd5b97b2-1aa8-437e-935d-09bf63ca651f

📥 Commits

Reviewing files that changed from the base of the PR and between 4acd728 and fa9522c.

📒 Files selected for processing (6)
  • packages/kit/scripts/download-skill-fixtures.mjs
  • packages/kit/src/skills/loader/browser.ts
  • packages/kit/src/skills/loader/definition.ts
  • packages/kit/src/skills/loader/github.ts
  • packages/kit/src/skills/test/loaderDefinition.test.ts
  • packages/kit/src/skills/test/loaderNode.test.ts

Walkthrough

Adds a complete skill loading subsystem to packages/kit. New modules define core SkillDefinition types, loader contracts, shared utilities (cancellation, YAML frontmatter parsing, path normalization), source-specific file loaders for browser (File API), filesystem (Node fs/promises), and GitHub (Contents API with retry), a definition builder, and separate browser/Node entrypoints. A fixture download script and comprehensive Vitest tests are also included. The yaml dependency and ./node package subpath export are added.

Changes

Skill Loader Subsystem

Layer / File(s) Summary
Core skill data model and path utilities
packages/kit/src/skills/types/index.ts, packages/kit/src/skills/utils.ts
SkillDefinition, SkillCandidate, and SkillResourceDescriptor (discriminated union for text/binary resources with loaded-vs-reader variants) are defined. normalizeSkillPath, isTextSkillFilePath, and getExtension path helpers are added.
Loader type contracts
packages/kit/src/skills/loader/type.ts
Defines SkillLoadWarning, SkillLoadResult, SkillLoadJob (Promise augmented with cancel()), SkillLoadContext with AbortSignal, SkillLoadProgressPhase/Event, SkillLoadBaseOptions, LoadableSkillFile, and the three discriminated option types (BrowserSkillLoadOptions, FsSkillLoadOptions, GithubSkillLoadOptions).
Loader utilities and job infrastructure
packages/kit/src/skills/loader/utils.ts
Adds SkillLoadCancelledError, throwIfSkillLoadCancelled, createSkillLoadJob (AbortController-based cancellable job factory), parseMarkdownFrontmatter (via yaml), stripRootDirectory, getFallbackSkillName, pushWarning (strict-mode throw or accumulate), and getString/getRecord type guards.
Skill definition builder and its tests
packages/kit/src/skills/loader/definition.ts, packages/kit/src/skills/test/loaderDefinition.test.ts
createSkillDefinition selects and validates the entry file, parses frontmatter+body, converts other files into SkillResourceDescriptor objects via toSkillResource, and returns SkillLoadResult with warnings. normalizeFiles deduplicates and sorts paths. Tests cover success paths, binary/JSON resources, missing/invalid entry, duplicate paths, and strict mode.
Browser loader and its tests
packages/kit/src/skills/loader/browser.ts, packages/kit/src/skills/test/loaderBrowser.test.ts
loadBrowserSkillFiles supports fileList and directoryHandle modes using the File System Access API with cancellation. loadBrowserFile classifies and reads text/binary content. Tests verify root-directory stripping, binary resource payloads, fallback naming, empty warnings, and cancel() rejection.
Filesystem and GitHub source loaders
packages/kit/src/skills/loader/fs.ts, packages/kit/src/skills/loader/github.ts
loadFsSkillFiles recursively walks options.root skipping ignored/hidden directories. loadGithubSkillFiles resolves the ref, walks the GitHub Contents API recursively, and wraps all network calls in fetchGithubWithRetry with exponential backoff (429/5xx), abort-aware delay, and toSkillRelativePath path mapping.
Public entrypoints and package wiring
packages/kit/src/skills/loader/index.ts, packages/kit/src/skills/loader/node.ts, packages/kit/src/node.ts, packages/kit/package.json
loader/index.ts dispatches to browser/github loaders; loader/node.ts dispatches to fs/github loaders. src/node.ts is the barrel re-export. package.json adds the ./node subpath export, extends build/dev scripts, and adds yaml to dependencies.
Fixture download script and node loader tests
packages/kit/scripts/download-skill-fixtures.mjs, packages/kit/src/skills/test/loaderNode.test.ts, packages/kit/src/skills/test/.gitignore
download-skill-fixtures.mjs downloads and caches fixture directories from pinned GitHub SHAs using .fixture-source.json marker-based cache invalidation. Node loader tests cover fs loading, multi-file resources, GitHub loading, retry behavior via fake timers and stubbed fetch, and loadSkillWithDetails warnings.

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant loadSkill
  participant createSkillLoadJob
  participant loadGithubSkillFiles
  participant fetchGithubWithRetry
  participant GitHubContentsAPI
  participant createSkillDefinition

  Caller->>loadSkill: loadSkill({ source: 'github', repo, path })
  loadSkill->>createSkillLoadJob: createSkillLoadJob(load)
  createSkillLoadJob-->>Caller: SkillLoadJob (with cancel())

  Note over createSkillLoadJob,loadGithubSkillFiles: job awaited
  createSkillLoadJob->>loadGithubSkillFiles: loadGithubSkillFiles(options, context)
  loadGithubSkillFiles->>fetchGithubWithRetry: fetchGithubJson(contentsURL)
  fetchGithubWithRetry->>GitHubContentsAPI: fetch(url, { signal })
  GitHubContentsAPI-->>fetchGithubWithRetry: 429 → retry with backoff
  GitHubContentsAPI-->>fetchGithubWithRetry: 200 OK → directory listing
  fetchGithubWithRetry-->>loadGithubSkillFiles: JSON entries
  loadGithubSkillFiles->>GitHubContentsAPI: fetchGithubBytes(download_url) per file
  GitHubContentsAPI-->>loadGithubSkillFiles: Uint8Array
  loadGithubSkillFiles-->>createSkillLoadJob: LoadableSkillFile[]
  createSkillLoadJob->>createSkillDefinition: createSkillDefinition(files, options)
  createSkillDefinition-->>createSkillLoadJob: SkillLoadResult
  createSkillLoadJob-->>Caller: SkillDefinition
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

🐇 Hoppity-hop through the skill-loading maze,
From browser to filesystem, GitHub ablaze!
With frontmatter parsed and retry in place,
Each LoadableSkillFile finds its right space.
cancel() the job if the warren gets slow —
The rabbit delivers clean SkillLoadResults, oh!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat(kit): add skill loader foundation' accurately summarizes the main change—introducing the foundational skill loader system with core types, three loader implementations (browser, Node, GitHub), and public APIs.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

Review ran into problems

🔥 Problems

Stopped waiting for pipeline failures after 30000ms. One of your pipelines takes longer than our 30000ms fetch window to run, so review may not consider pipeline-failure results for inline comments if any failures occurred after the fetch window. Increase the timeout if you want to wait longer or run a @coderabbit review after the pipeline has finished.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

✅ Preview build completed successfully!

Click the image above to preview.
Preview will be automatically removed when this PR is closed.

@github-actions

github-actions Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

@gene9831

Copy link
Copy Markdown
Collaborator Author

Warning

Review limit reached

@gene9831, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 24 minutes and 41 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?
CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: b517fe08-b010-4ea4-bce7-c6f375f78ac3

📥 Commits
Reviewing files that changed from the base of the PR and between 045d26b and 4acd728.

📒 Files selected for processing (17)

  • packages/kit/package.json
  • packages/kit/scripts/download-skill-fixtures.mjs
  • packages/kit/src/node.ts
  • packages/kit/src/skills/loader/browser.ts
  • packages/kit/src/skills/loader/definition.ts
  • packages/kit/src/skills/loader/fs.ts
  • packages/kit/src/skills/loader/github.ts
  • packages/kit/src/skills/loader/index.ts
  • packages/kit/src/skills/loader/node.ts
  • packages/kit/src/skills/loader/type.ts
  • packages/kit/src/skills/loader/utils.ts
  • packages/kit/src/skills/test/.gitignore
  • packages/kit/src/skills/test/loaderBrowser.test.ts
  • packages/kit/src/skills/test/loaderDefinition.test.ts
  • packages/kit/src/skills/test/loaderNode.test.ts
  • packages/kit/src/skills/types/index.ts
  • packages/kit/src/skills/utils.ts

✨ Finishing Touches
🧪 Generate unit tests (beta)

  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai review now

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown

@gene9831 I'll kick off a full review of this PR right away!

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (1)
packages/kit/src/skills/test/loaderDefinition.test.ts (1)

124-223: ⚡ Quick win

Add a regression test for non-canonical entryFile option values.

Line 124 onward covers entry-file error cases well, but there’s no assertion for entryFile inputs like ./SKILL.md (or backslash style) resolving correctly after path normalization. A focused case here would prevent regressions.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/kit/src/skills/test/loaderDefinition.test.ts` around lines 124 -
223, Add a new test case after the existing entry file error tests (after the
"throws when the entry file has no instructions" test) that verifies the
createSkillDefinition function correctly handles non-canonical entryFile option
values such as ./SKILL.md or paths with backslashes. The test should pass these
non-canonical paths in the options object and assert that they resolve correctly
to the same SKILL.md file, confirming that path normalization works as expected
and prevents regressions when users provide paths with different separators or
relative path prefixes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/kit/scripts/download-skill-fixtures.mjs`:
- Around line 32-59: The fetchJson and fetchBytes functions lack timeout and
retry mechanisms, causing test suite hangs on transient GitHub failures or
stalled connections. Implement a retry wrapper that wraps both functions to
retry on transient errors (429 and 5xx status codes) with exponential backoff,
and add an AbortController-based timeout to the fetch calls in both fetchJson
and fetchBytes (with a reasonable timeout like 30 seconds) to prevent indefinite
hangs. Apply consistent retry and timeout logic to both functions to ensure
robust fixture downloading in the pretest phase.

In `@packages/kit/src/skills/loader/browser.ts`:
- Around line 13-17: Replace the unbounded parallel file loading in the
Promise.all() call that processes all files from options.fileList
simultaneously. Instead of mapping all files to loadBrowserFile calls at once,
implement bounded concurrency control to limit the number of concurrent file
reads (typically using a concurrency library or a queue-based approach). This
will prevent memory spikes from reading all files in parallel and ensure proper
resource management for large file lists, while the actual implementation loads
files in controlled batches rather than all at once through the current
Promise.all() with array map pattern.

In `@packages/kit/src/skills/loader/definition.ts`:
- Around line 15-18: The entryFile variable is not being normalized before being
compared against the normalized file paths in the find operation. Apply the same
normalization logic to options.entryFile that is used in the normalizeFiles
function call, then assign the result to entryFile so that the path comparison
on line 17 correctly matches normalized file paths regardless of input format
(with or without ./ prefixes or backslash variants).

In `@packages/kit/src/skills/loader/github.ts`:
- Around line 126-147: The for loop condition in the GitHub fetch retry logic
uses a less-than-or-equal-to comparison operator (`<=`) which causes one extra
iteration compared to the configured retry limit. Change the loop condition from
`retryCount <= maxGithubFetchRetries` to `retryCount < maxGithubFetchRetries` so
that with a maxGithubFetchRetries value of 5, exactly 5 total attempts are made
instead of 6.

In `@packages/kit/src/skills/test/loaderNode.test.ts`:
- Around line 64-80: The test "loads weather skill from GitHub over the network"
is making actual network calls to GitHub, which makes it unreliable for CI/CD.
Mark this test as skipped by default using either it.skip or a conditional check
for an environment variable/flag that allows it to run only when explicitly
enabled, similar to how integration tests are typically gated. This ensures the
unit test suite runs deterministically without external dependencies while
allowing developers to opt-in to the network-dependent test when needed.

---

Nitpick comments:
In `@packages/kit/src/skills/test/loaderDefinition.test.ts`:
- Around line 124-223: Add a new test case after the existing entry file error
tests (after the "throws when the entry file has no instructions" test) that
verifies the createSkillDefinition function correctly handles non-canonical
entryFile option values such as ./SKILL.md or paths with backslashes. The test
should pass these non-canonical paths in the options object and assert that they
resolve correctly to the same SKILL.md file, confirming that path normalization
works as expected and prevents regressions when users provide paths with
different separators or relative path prefixes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 192db42e-6270-45c1-977c-9bcc6b1d38ce

📥 Commits

Reviewing files that changed from the base of the PR and between 045d26b and 4acd728.

📒 Files selected for processing (17)
  • packages/kit/package.json
  • packages/kit/scripts/download-skill-fixtures.mjs
  • packages/kit/src/node.ts
  • packages/kit/src/skills/loader/browser.ts
  • packages/kit/src/skills/loader/definition.ts
  • packages/kit/src/skills/loader/fs.ts
  • packages/kit/src/skills/loader/github.ts
  • packages/kit/src/skills/loader/index.ts
  • packages/kit/src/skills/loader/node.ts
  • packages/kit/src/skills/loader/type.ts
  • packages/kit/src/skills/loader/utils.ts
  • packages/kit/src/skills/test/.gitignore
  • packages/kit/src/skills/test/loaderBrowser.test.ts
  • packages/kit/src/skills/test/loaderDefinition.test.ts
  • packages/kit/src/skills/test/loaderNode.test.ts
  • packages/kit/src/skills/types/index.ts
  • packages/kit/src/skills/utils.ts

Comment thread packages/kit/scripts/download-skill-fixtures.mjs Outdated
Comment thread packages/kit/src/skills/loader/browser.ts Outdated
Comment thread packages/kit/src/skills/loader/definition.ts Outdated
Comment thread packages/kit/src/skills/loader/github.ts Outdated
Comment thread packages/kit/src/skills/test/loaderNode.test.ts Outdated
@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown

@gene9831 I'll kick off a full review of this PR right away!

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant