feat(geo): add llms.txt AI crawler manifest

[prudentbird]

Summary

Serves an llms.txt manifest at the site root (src/app/llms.txt/route.ts, text/plain) so AI crawlers get a clear, authoritative description instead of guessing.

Contents (generated from the shared site config so it can't drift):

• Product summary (title + blockquote).
• Core pages — Home, How it works, FAQ, Contact (with descriptions).
• Company — Ajared Research Inc., links, and contact email.
• Key features — the platform's capabilities.
• Legal — Privacy & Terms.

Done when (issue #110)

• An llms.txt manifest is served at the site root with a product summary, the core pages, a company description, and key features.

Validation

• npm run typecheck ✅ / npm run build ✅ — /llms.txt builds as a static route.
• Verified via next start: served as text/plain; charset=utf-8 with correct absolute URLs.

Notes

Stacked on #109 (targets feat/seo-109-structured-data).

Closes #110

[claude]

Claude finished @prudentbird's task —— View job

---

Code Review Complete ✅

Reviewed PR #121: feat(geo): add llms.txt AI crawler manifest

Todo List

• Read and analyze the new llms.txt route implementation
• Check code quality and Next.js best practices
• Review for potential bugs or issues
• Assess performance considerations
• Examine security concerns
• Evaluate test coverage needs
• Provide comprehensive feedback

Code Review Feedback

Overall Assessment

✅ Strong implementation - This is a well-structured Next.js API route that follows best practices for serving static content.

Code Quality & Best Practices

✅ Excellent Structure:

• Clean, readable implementation following Next.js 13+ App Router conventions
• Proper separation of concerns with configuration imported from ~/lib/site
• Good use of TypeScript with proper typing
• Consistent formatting and naming conventions

✅ Configuration Management:

• Smart use of shared siteConfig from ~/lib/site.ts prevents content drift
• Dynamic URL generation with absoluteUrl() utility ensures correct domain handling
• Features array is maintainable and easily extensible

Security Analysis

✅ No Security Issues Identified:

• Static content generation with no user input
• No SQL queries, external API calls, or file system operations
• Content is templated from configuration, reducing injection risks
• Proper Content-Type header prevents MIME confusion

Performance Considerations

✅ Excellent Performance:

• Minimal computational overhead - just string template generation
• No database queries or external API dependencies
• Response is cacheable (though no explicit cache headers set)
• Small payload size (~1KB estimated)

💡 Enhancement Opportunity:
Consider adding cache headers for better performance:

return new Response(body, {
  headers: {
    'Content-Type': 'text/plain; charset=utf-8',
    'Cache-Control': 'public, max-age=3600, s-maxage=86400',
  },
});

Technical Implementation

✅ Correct Next.js Patterns:

• Proper App Router API route structure (route.ts in app directory)
• Correct HTTP method export (GET)
• Proper Response object usage with appropriate headers
• File location (/llms.txt/route.ts) correctly serves content at /llms.txt

Potential Issues

⚠️ Minor Considerations:

1. Link Validation: The anchor links (#how-it-works, #faq) assume specific page structure. Consider validating these exist.

2. Content Length: No validation that generated content fits within reasonable limits (though unlikely to be an issue).

3. Error Handling: While the simple implementation is unlikely to fail, consider wrapping in try-catch for production robustness:

export function GET() {
  try {
    const body = `# ${siteConfig.name}...`;
    return new Response(body, {
      headers: {
        'Content-Type': 'text/plain; charset=utf-8',
      },
    });
  } catch (error) {
    return new Response('Error generating manifest', { status: 500 });
  }
}

Test Coverage

📝 Test Recommendations:

• Unit test to verify content structure and formatting
• Integration test to ensure route serves correct content-type
• Test with different siteConfig values to verify dynamic generation
• Validate all generated URLs are properly formatted

Example test structure:

describe('/llms.txt route', () => {
  it('should return plain text content', async () => {
    const response = await GET();
    expect(response.headers.get('content-type')).toBe('text/plain; charset=utf-8');
  });
  
  it('should include all required sections', async () => {
    const response = await GET();
    const text = await response.text();
    expect(text).toContain('# Retailytics');
    expect(text).toContain('## Core pages');
    expect(text).toContain('## Company');
    expect(text).toContain('## Key features');
    expect(text).toContain('## Legal');
  });
});

Standards Compliance

✅ LLMs.txt Specification:

• Follows the emerging llms.txt standard for AI crawler manifests
• Well-structured markdown with clear sections
• Includes essential information: product description, key pages, company details
• Provides both human-readable and machine-parseable content

Final Verdict

🎉 Excellent Implementation - This PR demonstrates solid engineering practices with a clean, secure, and performant solution. The code is production-ready with only minor enhancement opportunities.

Recommendation: ✅ APPROVE - No blocking issues identified. The implementation is well-crafted and follows best practices.

---