Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,6 @@ lib

output
*.mp4

.cache/
output/
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,15 @@ run it:
bun run hello.tsx
```

## agent usage (confirmation before costs)

when using the agent in this repo:

- ask for confirmation before any command that triggers paid api calls
- create tsx when needed, then run: `bun run src/cli/index.ts render <file.tsx> -o output/<name>.mp4`
- use `bun run src/cli/index.ts preview <file.tsx>` for zero-cost checks
- use single-step actions with: `bun run src/cli/index.ts run <action> ...`

## installation

```bash
Expand Down Expand Up @@ -534,4 +543,3 @@ see [CONTRIBUTING.md](CONTRIBUTING.md) for development setup.

Apache-2.0 — see [LICENSE.md](LICENSE.md)


172 changes: 19 additions & 153 deletions SKILLS.md
Original file line number Diff line number Diff line change
@@ -1,173 +1,39 @@
# agent skills

this sdk includes claude code agent skills for each service. each skill is co-located with its service code.
this repo exposes a small set of claude code skills plus a cli. use skills for guidance and run the cli to produce outputs.

## available skills

### service skills
1. **varg-video-generation** (`skills/varg-video-generation/`)
- generate videos with the react engine
- agent should create tsx if needed, then run a render after confirmation
- cli: `bun run src/cli/index.ts render <file.tsx> [-o output.mp4]`
- preview: `bun run src/cli/index.ts preview <file.tsx>`

located in `service/<name>/SKILL.md`:
2. **vargai-pipeline-cookbooks** (`skills/vargai-pipeline-cookbooks/`)
- end-to-end workflows (talking character, text-to-tiktok, round selfie, trendwatching, remotion)
- source cookbooks in `pipeline/cookbooks/`

1. **image-generation** (`service/image/`)
- generate ai images using fal (flux models) or higgsfield soul characters
- cli: `bun run service/image fal|soul <prompt> [options]`
## agent execution policy (important)

2. **video-generation** (`service/video/`)
- generate videos from images (local or url) or text prompts using fal.ai
- supports local image files - automatically uploads to fal storage
- cli: `bun run service/video from_image|from_text <args>`
- ask for confirmation before running any command that triggers paid api calls
- if the user explicitly says "run/render/generate now," proceed without extra confirmation
- prefer cli execution over leaving tsx files unrun
- write outputs to `output/` and do not commit new outputs

3. **voice-synthesis** (`service/voice/`)
- generate realistic text-to-speech audio using elevenlabs
- cli: `bun run service/voice generate|elevenlabs <text> [options]`
## cli actions (non-skill)

3b. **music-generation** (`lib/elevenlabs.ts`)
- generate music from text prompts using elevenlabs
- generate sound effects from descriptions
- cli: `bun run lib/elevenlabs.ts music|sfx <prompt> [options]`

4. **video-lipsync** (`service/sync/`)
- sync video with audio using wav2lip or simple overlay
- cli: `bun run service/sync sync|wav2lip|overlay <args>`

5. **video-captions** (`service/captions/`)
- add auto-generated or custom subtitles to videos
- cli: `bun run service/captions <videoPath> [options]`

6. **video-editing** (`service/edit/`)
- edit videos with ffmpeg (resize, trim, concat, social media prep)
- cli: `bun run service/edit social|montage|trim|resize|merge_audio <args>`

7. **audio-transcription** (`service/transcribe/`)
- transcribe audio to text or subtitles using groq/fireworks
- cli: `bun run service/transcribe <audioUrl> <provider> [outputPath]`

### library skills

8. **apify-scraping** (`lib/apify.ts`)
- run apify actors for web scraping (tiktok, google maps, social media, etc.)
- cli: `bun run lib/apify.ts run <actor_id> [input_json]`
- example: `bun run lib/apify.ts run clockworks/tiktok-scraper '{"hashtags":["viral"],"resultsPerPage":10}'`

### utility skills

9. **telegram-send** (external: `/Users/aleks/Github/Badaboom1995/rumble-b2c`)
- send videos to telegram users/channels as round videos
- automatically converts to 512x512 square format for telegram
- cli: `cd /Users/aleks/Github/Badaboom1995/rumble-b2c && bun run scripts/telegram-send-video.ts <videoPath> <@username>`
- example: `cd /Users/aleks/Github/Badaboom1995/rumble-b2c && bun run scripts/telegram-send-video.ts /path/to/video.mp4 @caffeinum`

### pipeline skills

located in `pipeline/cookbooks/SKILL.md`:

10. **talking-character-pipeline** (`pipeline/cookbooks/`)
- complete workflow to create talking character videos
- combines: character generation → voiceover → animation → lipsync → captions → social prep

11. **round-video-character** (`pipeline/cookbooks/round-video-character.md`)
- create realistic round selfie videos for telegram using nano banana pro + wan 2.5
- workflow: generate selfie first frame (person in setting) → voiceover → wan 2.5 video
- uses: `bun run lib/fal.ts`, `bun run lib/replicate.ts`, `bun run lib/elevenlabs.ts`
- input: text script + profile photo
- output: extreme close-up selfie video with authentic camera shake, lighting, and audio

12. **trendwatching** (`pipeline/cookbooks/trendwatching.md`)
- discover viral tiktok content for any topic/hashtag
- uses apify tiktok scraper to find trending videos
- get engagement metrics: plays, likes, shares, comments
- cli: `bun run lib/apify.ts run clockworks/tiktok-scraper '{"hashtags":["topic"],"resultsPerPage":10}'`

## structure

each skill follows this pattern:

```
service/<name>/
├── index.ts # service implementation
└── SKILL.md # claude code agent skill
```

## how skills work

skills are **model-invoked** - claude autonomously decides when to use them based on your request and the skill's description.

**example:**
- you say: "create a talking character video"
- claude reads `talking-character-pipeline` skill
- claude executes the workflow using the pipeline steps

## using skills

### in claude code

skills are automatically discovered when you're in the sdk directory:

```
user: create an image of a sunset
claude: [uses image-generation skill]
bun run service/image fal "beautiful sunset over mountains"
```

### manually

you can also run services directly:
use `varg run` for single-step actions:

```bash
# generate image
bun run service/image fal "sunset over mountains" true

# generate video from that image
bun run service/video from_image "camera pan" https://image-url.jpg 5 true

# add voice
bun run service/voice elevenlabs "this is a beautiful sunset" rachel true

# sync with video
bun run service/sync wav2lip https://video-url.mp4 https://audio-url.mp3
bun run src/cli/index.ts run image|video|voice|music|transcribe|captions|sync|upload|edit --help
```

## skill features

each skill includes:

- **name**: unique skill identifier
- **description**: when claude should use this skill
- **allowed-tools**: restricted to Read, Bash for safety
- **usage examples**: cli and programmatic examples
- **when to use**: specific use cases
- **tips**: best practices
- **environment variables**: required api keys

## benefits

- **discoverability**: claude knows all available services
- **context**: skills provide usage examples and best practices
- **safety**: `allowed-tools` limits to read-only and bash execution
- **documentation**: skills serve as living documentation

## skill reference

| skill | service | primary use case |
|-------|---------|------------------|
| image-generation | image | create ai images, character headshots |
| video-generation | video | animate images, generate video clips |
| voice-synthesis | voice | text-to-speech, voiceovers |
| music-generation | elevenlabs | generate music, create sound effects |
| video-lipsync | sync | sync audio with video, talking characters |
| video-captions | captions | add subtitles, accessibility |
| video-editing | edit | resize, trim, social media optimization |
| audio-transcription | transcribe | speech-to-text, subtitle generation |
| apify-scraping | lib/apify | web scraping via apify actors (tiktok, etc.) |
| telegram-send | external | send videos to telegram as round videos |
| talking-character-pipeline | pipeline | end-to-end talking character videos |
| round-video-character | pipeline | telegram round selfie videos with wan 2.5 |
| trendwatching | pipeline | discover viral tiktok content by hashtag |

## see also

- [README.md](README.md) - sdk overview and installation
- [STRUCTURE.md](STRUCTURE.md) - detailed module organization
- [docs/react.md](docs/react.md) - jsx rendering
- [docs/sdk.md](docs/sdk.md) - ai sdk usage
- [pipeline/cookbooks/talking-character.md](pipeline/cookbooks/talking-character.md) - talking character workflow
- [pipeline/cookbooks/round-video-character.md](pipeline/cookbooks/round-video-character.md) - telegram round selfie video cookbook
- [pipeline/cookbooks/trendwatching.md](pipeline/cookbooks/trendwatching.md) - discover viral tiktok content
8 changes: 8 additions & 0 deletions docs/react.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,14 @@ await render(
);
```

## agent execution (confirmation before costs)

when using the agent in this repo:

- ask for confirmation before any command that triggers paid api calls
- create tsx when needed, then run `bun run src/cli/index.ts render <file.tsx> -o output/<name>.mp4`
- use `bun run src/cli/index.ts preview <file.tsx>` for zero-cost checks

## core concepts

### everything is cached
Expand Down
10 changes: 10 additions & 0 deletions docs/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,16 @@ const { video } = await generateVideo({
});
```

## agent execution (confirmation before costs)

when using the agent in this repo:

- ask for confirmation before any command that triggers paid api calls
- prefer cli execution for outputs:
- `bun run src/cli/index.ts run <action> ...`
- `bun run src/cli/index.ts render <file.tsx> -o output/<name>.mp4`
- `bun run src/cli/index.ts preview <file.tsx>` for zero-cost checks

---

## core concepts
Expand Down
23 changes: 19 additions & 4 deletions skills/varg-video-generation/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: varg-video-generation
description: Generate AI videos using varg SDK React engine. Use when creating videos, animations, talking characters, slideshows, or social media content. Always run onboarding first to check API keys.
description: This skill should be used when the user asks to "create a video", "generate a video", "render a tsx", "make a slideshow", "create a talking character", or "animate an image" using the varg React engine.
license: MIT
metadata:
author: vargHQ
Expand All @@ -13,15 +13,23 @@ allowed-tools: Bash(bun:*) Bash(cat:*) Read Write Edit

Generate AI videos using declarative JSX syntax with automatic caching and parallel generation.

## Execution policy (must follow)

- Ask for confirmation before running any command that triggers paid API calls.
- Proceed without extra confirmation only when the user explicitly asks to run/render/generate now.
- Prefer CLI execution after creating TSX; do not leave TSX files unrun.
- Prefer `preview` for zero-cost checks before paid renders.
- Write outputs to `output/` and do not commit new outputs.

## Quick Setup

Run the setup script to initialize a project:

```bash
bun scripts/setup.ts
bun run src/init.ts
```

Or manually check API keys:
Or check API keys directly:

```bash
cat .env 2>/dev/null | grep -E "^(FAL_API_KEY|ELEVENLABS_API_KEY)=" || echo "No API keys found"
Expand Down Expand Up @@ -142,6 +150,13 @@ See [references/templates.md](references/templates.md) for more templates.
## Running Videos

```bash
# preferred: render via CLI after creating TSX
bun run src/cli/index.ts render your-video.tsx -o output/your-video.mp4

# optional: fast, zero-cost preview
bun run src/cli/index.ts preview your-video.tsx

# direct execution (acceptable but less consistent)
bun run your-video.tsx
```

Expand Down Expand Up @@ -207,7 +222,7 @@ const character = Image({ prompt: "blue robot" });

## Next Steps

1. Run `bun scripts/setup.ts` to initialize project
1. Run `bun run src/init.ts` to initialize project
2. Add your FAL_API_KEY to `.env`
3. Run `bun run examples/my-first-video.tsx`
4. Or ask the agent: "create a 10 second tiktok video about cats"
29 changes: 29 additions & 0 deletions skills/vargai-pipeline-cookbooks/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
name: vargai-pipeline-cookbooks
description: This skill should be used when the user asks to "create a talking character pipeline", "make a text-to-tiktok video", "build a round selfie video for Telegram", "do trendwatching with Apify", or "create a Remotion video pipeline".
version: 0.1.0
allowed-tools: Read, Bash
---

# pipeline cookbooks

Provide complete, multi-step workflows for common varg pipelines. Select the closest cookbook and follow its steps end-to-end.

## select a cookbook

- **Talking character (full workflow)**: `references/talking-character-pipeline.md`
- **Talking character (short form)**: `references/talking-character.md`
- **Text-to-TikTok**: `references/text-to-tiktok.md`
- **Round video character (Telegram selfie)**: `references/round-video-character.md`
- **Trendwatching (Apify + TikTok)**: `references/trendwatching.md`
- **Remotion editing pipeline**: `references/remotion-video.md`

## scripts

Use helper scripts from `scripts/` when the cookbook calls for them. Avoid editing scripts unless requested.

## notes

- Assume Bun and the varg SDK are available.
- Many steps require API keys (`FAL_API_KEY`, `ELEVENLABS_API_KEY`, `REPLICATE_API_TOKEN`, `GROQ_API_KEY`).
- Prefer the full talking-character pipeline when quality matters; use the short form for quick tests.
Loading