Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions .agents/skills/hackerai-user-research/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
---
name: hackerai-user-research
description: Run privacy-safe HackerAI customer research from a Linear question and a product-data cohort. Use for requests to understand user types, recurring jobs, workflows, friction, value drivers, reasons to pay, or customer avatars from actual HackerAI messages, including HAC-65-style top-spender research. Also use when a PM asks how to run, repeat, or interpret the `pm-user-research` Trigger task. Do not use for support investigations, decisions about one person's eligibility or risk, or exporting raw customer content.
---

# HackerAI User Research

Turn a research question and 3-20 internal user IDs into restricted per-user
profiles and an aggregated cohort report. The deployed task samples messages,
redacts sensitive data, and uses Grok 4.6 with reasoning disabled.

Read [references/privacy-policy.md](references/privacy-policy.md) and
[references/pm-runbook.md](references/pm-runbook.md) before running the task.

## Workflow

1. Read the owning Linear issue. Extract the research question, cohort rule,
exclusions, requested output, and privacy constraints. Confirm the responsible
owner explicitly approved customer-message research. If no approved issue
exists, create or update one and stop until approval is recorded; creating the
issue does not itself grant approval.
2. Select the cohort in PostHog. Use Stripe-synced revenue in PostHog when its
freshness and account mapping are sufficient. Check Stripe directly only for
unmatched customers, refunds/disputes, payer-versus-user ambiguity, or other
reconciliation gaps. Never use Google Drive.
3. Resolve each cohort member to the internal user ID used by Convex. Exclude
internal/test/fraud accounts and deduplicate payer or organization
relationships before triggering analysis. Stop unless 3-20 unique internal
user IDs remain after filtering.
4. Discover the Trigger task `pm-user-research` and inspect its current schema.
Trigger it in the intended environment with the Linear issue ID, exact
question, descriptive cohort label, 3-20 unique user IDs, PM name/handle, and
optional chat limit. Never call the worker task directly.
5. Wait for the run to complete. Keep the returned `analysisId`; it is the audit
and lookup key for the restricted Convex records.
6. Present only the aggregate answer, evidence coverage, supported user types,
avatars, primary/secondary target, confidence, unknowns, and experiments.
Detailed pseudonym-level profiles remain in restricted Convex records and are
not returned through Trigger.
7. Update Linear only when asked. Copy aggregate findings, coverage, confidence,
unknowns, and experiments. Never copy cohort IDs, pseudonym-level profiles,
raw evidence, direct identifiers, or per-user findings or targeting decisions.

## Trigger payload

Use the current task schema as the authority. A typical HAC-65 run is:

```json
{
"linearIssueId": "HAC-65",
"question": "What kinds of users are our highest-spending customers, what recurring work do they use HackerAI for, and why do they pay?",
"cohortLabel": "Top 10 users by reconciled lifetime net paid spend",
"userIds": ["internal-user-id-1", "internal-user-id-2", "internal-user-id-3"],
"requestedBy": "PM name or handle",
"maxChatsPerUser": 12
}
```

Do not place email addresses, Stripe customer IDs, or message content in the
payload. `userIds` must be the internal Convex/WorkOS user IDs.

## Quality checks

- Treat a profile as directional when fewer than three chats were available or
confidence is low.
- Verify `usersAnalyzed`, `chatsReviewed`, and `messagesReviewed` before using a
conclusion.
- Do not turn one-off requests into an avatar. Prefer patterns supported across
multiple chats and users.
- Keep observed product behavior separate from acquisition or messaging
hypotheses.
- Say `unknown` when the evidence does not establish context. Never infer a
company or occupation from an email address.
- A failed or partial run is not permission to inspect messages manually. Fix
cohort mapping or deployment/configuration and rerun the bounded task.

## Result boundary

The Trigger result contains only aggregate internal research. Detailed profiles
remain restricted and deletion-aware in Convex. The aggregate report is the only
part that may be copied to Linear, under the owning issue's privacy rules.
4 changes: 4 additions & 0 deletions .agents/skills/hackerai-user-research/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
interface:
display_name: "HackerAI User Research"
short_description: "Analyze user workflows from private product evidence"
default_prompt: "Use $hackerai-user-research to answer this Linear research question from privacy-safe product evidence."
38 changes: 38 additions & 0 deletions .agents/skills/hackerai-user-research/references/pm-runbook.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# PM runbook

## 1. Prepare the cohort

Start only after the responsible owner explicitly approves customer-message
research in the Linear issue. Creating or editing the issue is not approval. For
spend-ranked research, use
PostHog's Stripe-synced lifetime net paid amount when available and current.
Exclude refunds/disputes, internal and test users, fraud, duplicates, and
unmatched customers. Open Stripe only to resolve discrepancies or payer/account
ownership. Produce internal Convex/WorkOS user IDs, not emails or Stripe IDs.

## 2. Run through Codex

Ask Codex:

> Use $hackerai-user-research for HAC-65. Select the reconciled top-spender
> cohort, run the analysis, wait for it, and give me the aggregate findings with
> coverage, confidence, unknowns, and recommended experiments.

Codex should use Trigger's task discovery/schema tools, trigger
`pm-user-research` in production, then wait for completion. The task runs one
parallel worker per user and a final cohort synthesis. Both calls use
`x-ai/grok-4.6` with OpenRouter reasoning explicitly disabled and zero-data-
retention routing required.

## 3. Interpret the result

Use the aggregate report to understand supported user types, customer avatars,
and decisions. Always include coverage and confidence. Detailed profiles remain
restricted in Convex. Treat acquisition channels and marketing messages as
hypotheses until a separate experiment validates them.

## 4. Share safely

Keep the complete Trigger result and Convex records restricted. A Linear update
may include only the aggregate answer, avatars, coverage, confidence, unknowns,
and experiments. Do not include the cohort IDs or pseudonym-level profiles.
36 changes: 36 additions & 0 deletions .agents/skills/hackerai-user-research/references/privacy-policy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Customer research privacy policy

Use customer messages only for an approved, specific internal research purpose.
The owning Linear issue must define the cohort and intended output.

## Allowed

- Product behavior: recurring jobs, Ask/Agent usage, workflow stages, broad tool
or environment categories, friction, value, and reasons to pay.
- Analyst-visible per-user profile content uses generated pseudonyms. Restricted
Convex records retain the internal user ID needed for deletion and lifecycle
handling; never expose the pseudonym-to-user linkage.
- Cohort-level avatars, confidence, unknowns, and testable hypotheses.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

## Prohibited

- Sensitive-trait, demographic, health, political, religious, sexual, or other
personal profiling.
- Inferring identity, employer, company, occupation, geography, or legitimacy
from an email address or isolated clue.
- User contact, sales outreach, public marketing claims, eligibility decisions,
fraud decisions, or adverse actions based on this research.
- Copying cohort IDs, pseudonym-level profiles, raw prompts, transcripts, direct
quotes, identifiers, evidence, files, code, commands, payloads, secrets, or
per-user targets, findings, or targeting decisions into Linear or task output.
- Sanitized aggregate report fields may be copied to the approved Linear issue.
- Manual browsing as a fallback when the bounded task fails.

## Storage and deletion

Raw message excerpts exist only in the analysis worker's memory and model
request. Model calls require an OpenRouter zero-data-retention route and fail
closed if no such Grok 4.6 endpoint is available. Convex stores the run audit,
pseudonymized structured profiles, and the aggregate report. Account deletion
removes that user's stored profile and run-membership linkage; runs and reports
are retained only as cohort-level outputs from cohorts of at least three.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
57 changes: 57 additions & 0 deletions convex/__tests__/userDeletion.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -535,6 +535,49 @@ function seedTables(userId = "user_123", otherUserId = "user_other"): Tables {
],
processed_webhooks: [{ _id: "webhook", event_id: "evt_1" }],
processed_checkout_sessions: [{ _id: "checkout", session_key: "cs_1" }],
research_runs: [
{
_id: "research-run",
analysis_id: "analysis-1",
cohort_size: 3,
status: "completed",
},
],
research_reports: [
{
_id: "research-report",
analysis_id: "analysis-1",
report: { answerToQuestion: "Cohort-level answer" },
},
],
research_user_profiles: [
{
_id: "research-profile-user",
analysis_id: "analysis-1",
user_id: userId,
pseudonym: "U01",
},
{
_id: "research-profile-other",
analysis_id: "analysis-1",
user_id: otherUserId,
pseudonym: "U02",
},
],
research_run_members: [
{
_id: "research-member-user",
analysis_id: "analysis-1",
user_id: userId,
pseudonym: "U01",
},
{
_id: "research-member-other",
analysis_id: "analysis-1",
user_id: otherUserId,
pseudonym: "U02",
},
],
};
}

Expand Down Expand Up @@ -573,6 +616,20 @@ describe("userDeletion", () => {
expect(row(tables, "chats", "chat-other")).toBeTruthy();
expect(row(tables, "feedback", "feedback-other")).toBeTruthy();
expect(row(tables, "files", "file-other")).toBeTruthy();
expect(
row(tables, "research_user_profiles", "research-profile-user"),
).toBeUndefined();
expect(
row(tables, "research_user_profiles", "research-profile-other"),
).toBeTruthy();
expect(
row(tables, "research_run_members", "research-member-user"),
).toBeUndefined();
expect(
row(tables, "research_run_members", "research-member-other"),
).toBeTruthy();
expect(row(tables, "research_runs", "research-run")).toBeTruthy();
expect(row(tables, "research_reports", "research-report")).toBeTruthy();
expect(row(tables, "temp_streams", "temp-stream-user")).toBeUndefined();
expect(row(tables, "temp_streams", "temp-stream-other")).toBeTruthy();

Expand Down
Loading