diff --git a/.env.example b/.env.example index 5c460c4..91f3c08 100644 --- a/.env.example +++ b/.env.example @@ -1,3 +1,5 @@ +# Preferred: run `codeturtle` and save to ~/.codeturtle/config.toml +# This file is optional (cwd .env). Config.toml is read first. GITHUB_TOKEN= OLLAMA_MODEL=qwen2.5:7b OLLAMA_BASE_URL=http://localhost:11434 @@ -6,4 +8,3 @@ LLM_BACKEND=ollama # OPENAI_API_KEY= # LANGFUSE_PUBLIC_KEY= # LANGFUSE_SECRET_KEY= -# GRAPHIFY_GRAPH_PATH= diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..9e7971c --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Venkat Pachala + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index ea38671..03c4e94 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,35 @@ # CodeTurtle -Local-first GitHub pull request reviewer. +Local-first GitHub PR reviewer. Graphify structure, clamped **MERGE** / **COMMENT** / **REQUEST_CHANGES**. Runs on your machine. -Deterministic gates (PR facts, path grounding, hunk verification, coverage clamp) -plus a specialist agent swarm. Structure comes from Graphify. Reviews run on -your machine. Nothing is posted unless you pass `--comment`. +```bash +uv tool install "git+https://github.com/venkatpachala/CodeTurtle.git@v0.3.0" +codeturtle +``` + +Requires: **Ollama** (or `OPENAI_API_KEY`), **git**, and a **GitHub token** (`gh auth login` or paste once). + +`codeturtle` with no arguments opens a menu: save token → pick an Ollama model → type `owner/repo` and a PR number. The CLI clones into `~/.codeturtle/repos/`, builds a Graphify code-only graph if needed, and dry-runs the review. + +```text +CodeTurtle + [1] GitHub token detected / paste once → ~/.codeturtle/config.toml + [2] Model list from Ollama: qwen2.5:7b, llama3.2, ... + [3] Review a PR owner/repo 3292 + [4] Quit +``` + +Scripts still work: ```bash -uv tool install git+https://github.com/venkatpachala/CodeTurtle.git@v0.2.0 +codeturtle review confident-ai/deepeval 3292 codeturtle review owner/repo 123 --dry-run ``` +Default is **dry-run**. Nothing is posted unless you pass `--comment`. Qdrant and Neo4j are not required. + +--- + ## What is CodeTurtle? CodeTurtle is a CLI. It fetches a GitHub PR with your token, builds a @@ -18,6 +37,8 @@ code-only Graphify graph of the repository, reviews **change units** (hunks) instead of a truncated diff, and emits **MERGE**, **COMMENT**, or **REQUEST_CHANGES**. +Config lives in `%USERPROFILE%\.codeturtle\config.toml` (or `~/.codeturtle/config.toml`), not a random project `.env`. + Agents propose findings. Gates decide what ships: - a finding that only cites `.github/wordlist.txt` is dropped @@ -54,45 +75,34 @@ The model does not own file selection, line identity, or the final GitHub event ### Prerequisites - Python >= 3.11 -- GitHub token with `public_repo` (or fine-grained PR read) -- An LLM: Ollama (default) or OpenAI-compatible -- Graphify code-only graph for the repo you review +- git +- GitHub token with `public_repo` (or fine-grained PR read), or `gh auth login` +- An LLM: [Ollama](https://ollama.com/download) (`ollama pull qwen2.5:7b`) or `OPENAI_API_KEY` -### Install +You cannot ship a 7B model inside `uv tool install`. Ollama (or an API key) is the only extra install. -```bash -uv tool install git+https://github.com/venkatpachala/CodeTurtle.git@v0.2.0 - -# from a clone -git clone https://github.com/venkatpachala/CodeTurtle.git -cd CodeTurtle -pip install -e ".[ollama,graphify]" -codeturtle --help -``` - -### Setup +### Install ```bash -cp .env.example .env -# set GITHUB_TOKEN -# OLLAMA_MODEL=qwen2.5:7b -ollama pull qwen2.5:7b +uv tool install "git+https://github.com/venkatpachala/CodeTurtle.git@v0.3.0" +codeturtle ``` -Index structure once per repo: +From a clone: ```bash -git clone https://github.com/owner/repo repos/owner_repo -cd repos/owner_repo -graphify . --code-only +git clone https://github.com/venkatpachala/CodeTurtle.git +cd CodeTurtle +pip install -e . +codeturtle --help ``` -Qdrant and Neo4j are not required for `--help` or `--dry-run`. +Graphify is included in the default install. -### Review +### Commands ```bash -codeturtle new-session +codeturtle codeturtle review owner/repo 123 --dry-run codeturtle review owner/repo 123 --comment codeturtle graphify-test owner/repo --stats @@ -100,17 +110,7 @@ codeturtle graphify-test owner/repo --stats Default is dry-run. `--comment` on a repo without write access fails closed and prints the body. -### Smoke test (v0.2.0) - -| Field | Value | -|-------|--------| -| PR | [confident-ai/deepeval#3288](https://github.com/confident-ai/deepeval/pull/3288) | -| Classification | source | -| Change units | 2 | -| Investigate | 6 hops | -| Decision | REQUEST_CHANGES | -| Graphify | ~19k nodes / ~57k edges | -| Post | none (`--dry-run`) | +### Dev eval ```bash uv run python -m tests.evaluation.run_eval --offline @@ -139,4 +139,3 @@ scripts/ # maintainer helpers ## Contributing See [CONTRIBUTING.md](CONTRIBUTING.md). - diff --git a/cli/commands/graphify_cmd.py b/cli/commands/graphify_cmd.py index 82691d4..5f29606 100644 --- a/cli/commands/graphify_cmd.py +++ b/cli/commands/graphify_cmd.py @@ -27,6 +27,9 @@ def graphify_test( ): """Handshake Graphify MCP for a specific repository graph.""" try: + from core.workspace import ensure_workspace + + ensure_workspace(repo) graph_path = resolve_graph_path(repo) provider = get_knowledge_provider(repo=repo) diff --git a/cli/commands/init.py b/cli/commands/init.py index d9fea21..b0e525c 100644 --- a/cli/commands/init.py +++ b/cli/commands/init.py @@ -18,6 +18,9 @@ def init(): token = Prompt.ask("Enter your GitHub Token (or press Enter to use GITHUB_TOKEN env)") if token: os.environ["GITHUB_TOKEN"] = token + from core.user_config import save_user_config + + save_user_config({"github_token": token}) console.print("[green]✓[/green] CodeTurtle initialized successfully!") - console.print("You can now run: [bold]codeturtle review owner/repo PR_NUMBER[/bold]") \ No newline at end of file + console.print("You can now run: [bold]codeturtle[/bold] or [bold]codeturtle review owner/repo PR_NUMBER[/bold]") \ No newline at end of file diff --git a/cli/commands/review.py b/cli/commands/review.py index 3f1c878..9454bdf 100644 --- a/cli/commands/review.py +++ b/cli/commands/review.py @@ -49,8 +49,11 @@ class PipelineContext: def get_current_session() -> str: from core.ci import ensure_session + from core.user_config import session_file - return ensure_session() + path = session_file() + path.parent.mkdir(parents=True, exist_ok=True) + return ensure_session(str(path)) def _as_dict(obj: Any) -> dict: @@ -166,6 +169,17 @@ def run( ) ) + from core.ci import resolve_github_token + from core.workspace import WorkspaceError, ensure_workspace + + token = resolve_github_token(fallback=str(settings.github_token or "")) + try: + graph = ensure_workspace(repo, number, token=token) + print(f"[Workspace] graph={graph}") + except WorkspaceError as exc: + console.print(f"[red]{exc}[/red]") + raise SystemExit(1) from exc + self._load_knowledge_base() self._fetch_pr() self._build_full_diff() diff --git a/cli/commands/session.py b/cli/commands/session.py index d82fbed..0b9ff87 100644 --- a/cli/commands/session.py +++ b/cli/commands/session.py @@ -16,10 +16,12 @@ def new_session(): try: conversation_id = memory.create_new_session() - # Save as current active session try: - with open(".current_session", "w") as f: - f.write(conversation_id) + from core.user_config import session_file + + path = session_file() + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(conversation_id + "\n", encoding="utf-8") except Exception as e: raise Exception("Failed to save current session") from e diff --git a/cli/commands/wizard.py b/cli/commands/wizard.py new file mode 100644 index 0000000..114be07 --- /dev/null +++ b/cli/commands/wizard.py @@ -0,0 +1,183 @@ +"""First-run menu: token, model, review. No new-session ceremony.""" + +from __future__ import annotations + +import os +import subprocess +from typing import List, Optional + +from rich.console import Console +from rich.panel import Panel +from rich.prompt import Prompt + +from config import settings +from core.user_config import ( + config_path, + detect_github_token, + save_user_config, +) + +console = Console() + + +def list_ollama_models(ollama_bin: str = "ollama") -> List[str]: + try: + proc = subprocess.run( + [ollama_bin, "list"], + capture_output=True, + text=True, + timeout=10, + check=False, + ) + except (OSError, subprocess.TimeoutExpired): + return [] + if proc.returncode != 0: + return [] + names: List[str] = [] + for i, line in enumerate((proc.stdout or "").splitlines()): + line = line.strip() + if not line: + continue + name = line.split()[0] + if i == 0 and name.lower() in {"name", "model"}: + continue + if name: + names.append(name) + return names + + +def _token_status() -> str: + tok = detect_github_token() + if tok: + return "detected" + return "missing" + + +def _ensure_token() -> str: + existing = detect_github_token() + if existing: + os.environ["GITHUB_TOKEN"] = existing + settings.github_token = existing + save_user_config({"github_token": existing}) + console.print(f"[green]GitHub token[/green] detected → {config_path()}") + return existing + console.print( + "Paste a GitHub token (classic `public_repo`, or [bold]Enter[/bold] to try [cyan]gh auth token[/cyan])." + ) + pasted = Prompt.ask("GitHub token", password=True, default="").strip() + token = pasted or detect_github_token() + if not token: + console.print("[red]No GitHub token. Set GITHUB_TOKEN or run gh auth login.[/red]") + return "" + os.environ["GITHUB_TOKEN"] = token + settings.github_token = token + save_user_config({"github_token": token}) + console.print(f"[green]Saved[/green] {config_path()}") + return token + + +def _ensure_model() -> str: + current = (getattr(settings, "ollama_model", "") or "").strip() + models = list_ollama_models() + if not models: + console.print( + "[yellow]Install Ollama and pull a model — you cannot ship 5GB weights inside uv tool install.[/yellow]\n" + " https://ollama.com/download\n" + " ollama pull qwen2.5:7b" + ) + if current: + return current + fallback = Prompt.ask("Model name", default=current or "qwen2.5:7b").strip() + if fallback: + settings.ollama_model = fallback + settings.llm_backend = "ollama" + os.environ["OLLAMA_MODEL"] = fallback + save_user_config({"ollama_model": fallback, "llm_backend": "ollama"}) + return fallback + console.print("[bold]Ollama models[/bold]") + for i, name in enumerate(models, start=1): + mark = " [green](current)[/green]" if name == current else "" + console.print(f" {i}. {name}{mark}") + default = "1" + if current in models: + default = str(models.index(current) + 1) + raw = Prompt.ask("Pick a model", default=default).strip() + chosen = current or models[0] + if raw.isdigit() and 1 <= int(raw) <= len(models): + chosen = models[int(raw) - 1] + elif raw in models: + chosen = raw + settings.ollama_model = chosen + settings.llm_backend = "ollama" + os.environ["OLLAMA_MODEL"] = chosen + save_user_config({"ollama_model": chosen, "llm_backend": "ollama"}) + console.print(f"[green]Model[/green] {chosen}") + return chosen + + +def _run_review(repo: str, number: int) -> None: + from cli.commands.review import ReviewPipeline + + ReviewPipeline().run( + repo=repo, + number=number, + dry_run=True, + verbose=False, + execute_tests=False, + execute_install=False, + comment=False, + config_path="", + ) + + +def _prompt_pr() -> Optional[tuple[str, int]]: + repo = Prompt.ask("Repository (owner/repo)", default="").strip().strip("/") + if not repo or "/" not in repo: + console.print("[red]Expected owner/repo (e.g. confident-ai/deepeval).[/red]") + return None + raw = Prompt.ask("PR number", default="").strip() + try: + number = int(raw) + except ValueError: + console.print("[red]PR number must be an integer.[/red]") + return None + if number <= 0: + console.print("[red]PR number must be positive.[/red]") + return None + return repo, number + + +def run_wizard() -> None: + console.print( + Panel.fit( + "[bold cyan]CodeTurtle[/bold cyan]\n" + "Local PR review. Graphify + your model. Default is dry-run.", + title="Welcome", + ) + ) + while True: + token_state = _token_status() + model = (getattr(settings, "ollama_model", "") or "").strip() or "not set" + console.print() + console.print(f" [1] GitHub token {token_state} → {config_path()}") + console.print(f" [2] Model {model}") + console.print(" [3] Review a PR owner/repo N") + console.print(" [4] Quit") + choice = Prompt.ask("Select", choices=["1", "2", "3", "4"], default="3") + if choice == "1": + _ensure_token() + elif choice == "2": + _ensure_model() + elif choice == "3": + if not detect_github_token(): + if not _ensure_token(): + continue + if not (getattr(settings, "ollama_model", "") or "").strip(): + _ensure_model() + picked = _prompt_pr() + if not picked: + continue + repo, number = picked + _run_review(repo, number) + else: + return diff --git a/cli/main.py b/cli/main.py index 48a129b..8216b8d 100644 --- a/cli/main.py +++ b/cli/main.py @@ -1,3 +1,6 @@ +import os +import sys + import typer from rich.console import Console from rich.panel import Panel @@ -24,15 +27,24 @@ @app.callback(invoke_without_command=True) def main(ctx: typer.Context): - """Global entry point with error handling""" - if ctx.invoked_subcommand is None: + """Open the first-run menu, or run a subcommand.""" + if ctx.invoked_subcommand is not None: + return + if os.environ.get("CODETURTLE_NO_WIZARD") or not sys.stdin.isatty(): console = Console() - console.print(Panel.fit( - "[bold cyan]CodeTurtle[/bold cyan]\n\n" - "Use [bold]codeturtle --help[/bold] to see available commands.\n" - "Start with: [bold]codeturtle new-session[/bold]", - title="Welcome" - )) + console.print( + Panel.fit( + "[bold cyan]CodeTurtle[/bold cyan]\n\n" + "Use [bold]codeturtle --help[/bold] to see commands.\n" + "Interactive: run [bold]codeturtle[/bold] in a terminal.\n" + "Scripts: [bold]codeturtle review owner/repo 123 --dry-run[/bold]", + title="Welcome", + ) + ) + return + from cli.commands.wizard import run_wizard + + run_wizard() if __name__ == "__main__": @@ -40,4 +52,5 @@ def main(ctx: typer.Context): app() except Exception as e: from core.utils import handle_error + handle_error(e, verbose=False) diff --git a/config.py b/config.py index 8a9852c..de477f6 100644 --- a/config.py +++ b/config.py @@ -2,13 +2,18 @@ from pydantic import ConfigDict from dotenv import load_dotenv import os +import sys -# Explicitly load .env +# ~/.codeturtle/config.toml first, then cwd .env (does not override existing env). +from core.user_config import apply_user_config_to_environ, home_dir, repos_dir + +apply_user_config_to_environ() load_dotenv() + class Settings(BaseSettings): # LLM Configuration - llm_backend: str = "ollama" # ← Added this + llm_backend: str = "ollama" ollama_model: str = "qwen2.5:7b" ollama_base_url: str = "http://localhost:11434" @@ -16,22 +21,21 @@ class Settings(BaseSettings): github_token: str = "" # Paths - memory_path: str = "~/.codeturtle/memory" - traces_path: str = "~/.codeturtle/traces" + memory_path: str = str(home_dir() / "memory") + traces_path: str = str(home_dir() / "traces") model_config = ConfigDict( - env_file = ".env", - extra = "ignore" # ← This allows extra variables without error - + env_file=".env", + extra="ignore", ) # Graphify (structural knowledge) graphify_enabled: bool = False graphify_graph_path: str = "graphify-out/graph.json" graphify_http_url: str = "http://localhost:8080/mcp" - graphify_python: str = "python" + graphify_python: str = sys.executable graphify_project_path: str = "" - # config - repos_root: str = "repos" + # empty = ~/.codeturtle/repos + repos_root: str = "" graphify_graph_filename: str = "graphify-out/graph.json" graphify_transport: str = "stdio" graphify_only_retrieval: bool = True @@ -41,19 +45,18 @@ class Settings(BaseSettings): execute_tests: bool = False execute_timeout_s: int = 120 execute_max_files: int = 8 - # 4.3b/c — install in the PR worktree (off by default; implies network) execute_install: bool = False execute_install_timeout_s: int = 180 execute_allow_npm: bool = True execute_allow_npm_scripts: bool = False execute_network: bool = False - # Phase 6.2 — inline review comments (post path only) inline_max: int = 8 inline_lockfile: bool = False - # Phase 7.3 — MERGE only if packed/total >= this (empty KEEP) coverage_merge_min: float = 0.5 -# Create the settings instance -settings = Settings() \ No newline at end of file + +settings = Settings() +if not (getattr(settings, "repos_root", None) or "").strip(): + settings.repos_root = str(repos_dir()) diff --git a/core/memory/database.py b/core/memory/database.py index f0432ab..8e82789 100644 --- a/core/memory/database.py +++ b/core/memory/database.py @@ -3,13 +3,21 @@ from datetime import datetime import json -DB_PATH = Path("data/codeturtle.db") -DB_PATH.parent.mkdir(parents=True, exist_ok=True) + +def _db_path() -> Path: + try: + from core.user_config import home_dir + + p = home_dir() / "codeturtle.db" + except Exception: + p = Path("data/codeturtle.db") + p.parent.mkdir(parents=True, exist_ok=True) + return p def get_connection(): """Get SQLite connection""" - conn = sqlite3.connect(DB_PATH) + conn = sqlite3.connect(_db_path()) conn.row_factory = sqlite3.Row return conn diff --git a/core/repository_knowledge/factory.py b/core/repository_knowledge/factory.py index d028aff..c43ef82 100644 --- a/core/repository_knowledge/factory.py +++ b/core/repository_knowledge/factory.py @@ -12,9 +12,9 @@ def get_knowledge_provider(repo: Optional[str] = None) -> RepositoryKnowledgePro Build a Graphify-backed knowledge provider. If repo is given (owner/name), resolve: - repos//graphify-out/graph.json + ~/.codeturtle/repos//graphify-out/graph.json - If repo is None, fall back to settings.graphify_graph_path (manual override). + If repo is None, fall back to settings.graphify_graph_path. """ from config import settings diff --git a/core/repository_knowledge/paths.py b/core/repository_knowledge/paths.py index 2c93a3e..2be4aaf 100644 --- a/core/repository_knowledge/paths.py +++ b/core/repository_knowledge/paths.py @@ -1,30 +1,51 @@ from __future__ import annotations +import os from pathlib import Path from typing import Optional -from config import settings - def repo_to_folder(repo: str) -> str: """FalkorDB/QueryWeaver -> FalkorDB_QueryWeaver""" return repo.strip().replace("\\", "/").replace("/", "_") +def _default_repos_root() -> Path: + from core.user_config import repos_dir + + return repos_dir() + + def resolve_repo_dir(repo: str, repos_root: Optional[str] = None) -> Path: - root = Path(repos_root or getattr(settings, "repos_root", "repos")) - return root / repo_to_folder(repo) + folder = repo_to_folder(repo) + if repos_root: + return Path(str(repos_root)).expanduser() / folder + env_root = (os.environ.get("CODETURTLE_REPOS_ROOT") or "").strip() + if env_root: + return Path(env_root).expanduser() / folder + home_dir = _default_repos_root() / folder + cwd_legacy = Path("repos") / folder + if home_dir.exists(): + return home_dir + if cwd_legacy.exists(): + return cwd_legacy + return home_dir def resolve_graph_path(repo: str, repos_root: Optional[str] = None) -> Path: """ Universal graph location: - repos//graphify-out/graph.json + ~/.codeturtle/repos//graphify-out/graph.json """ repo_dir = resolve_repo_dir(repo, repos_root) - # Prefer settings override pattern if present, else standard Graphify layout - rel = getattr(settings, "graphify_graph_filename", "graphify-out/graph.json") + rel = "graphify-out/graph.json" + try: + from config import settings + + rel = getattr(settings, "graphify_graph_filename", rel) or rel + except Exception: + pass return (repo_dir / rel).resolve() @@ -35,8 +56,8 @@ def ensure_graph_exists(repo: str) -> Path: raise FileNotFoundError( f"Graphify graph not found for '{repo}'.\n" f"Expected: {path}\n\n" - f"Build it with:\n" + f"CodeTurtle builds this automatically on review. Or:\n" f" cd {repo_dir}\n" - f" graphify . --code-only\n" + f" graphify extract . --code-only\n" ) - return path \ No newline at end of file + return path diff --git a/core/user_config.py b/core/user_config.py new file mode 100644 index 0000000..a6eaa5e --- /dev/null +++ b/core/user_config.py @@ -0,0 +1,163 @@ +"""User config under ~/.codeturtle (not a cwd .env).""" + +from __future__ import annotations + +import os +import subprocess +from pathlib import Path +from typing import Any, Dict, Mapping, Optional + +CONFIG_NAME = "config.toml" +REPOS_DIRNAME = "repos" +SESSION_NAME = "current_session" + +_TOML_KEYS = ( + "github_token", + "ollama_model", + "llm_backend", + "ollama_base_url", + "openai_api_key", +) + +_ENV_FOR_KEY = { + "github_token": "GITHUB_TOKEN", + "ollama_model": "OLLAMA_MODEL", + "llm_backend": "LLM_BACKEND", + "ollama_base_url": "OLLAMA_BASE_URL", + "openai_api_key": "OPENAI_API_KEY", +} + + +def home_dir() -> Path: + raw = (os.environ.get("CODETURTLE_HOME") or "").strip() + if raw: + return Path(raw).expanduser().resolve() + return (Path.home() / ".codeturtle").resolve() + + +def config_path() -> Path: + return home_dir() / CONFIG_NAME + + +def repos_dir() -> Path: + raw = (os.environ.get("CODETURTLE_REPOS_ROOT") or "").strip() + if raw: + return Path(raw).expanduser().resolve() + return home_dir() / REPOS_DIRNAME + + +def session_file() -> Path: + if os.environ.get("GITHUB_ACTIONS"): + return Path(".current_session") + return home_dir() / SESSION_NAME + + +def ensure_home() -> Path: + p = home_dir() + p.mkdir(parents=True, exist_ok=True) + return p + + +def load_user_config(path: Optional[Path] = None) -> Dict[str, Any]: + p = path or config_path() + if not p.is_file(): + return {} + try: + import tomllib + + data = tomllib.loads(p.read_text(encoding="utf-8")) + except Exception: + return {} + if not isinstance(data, dict): + return {} + return {k: data[k] for k in _TOML_KEYS if k in data and data[k] not in (None, "")} + + +def dumps_toml(data: Mapping[str, Any]) -> str: + lines = [ + "# CodeTurtle user config. Written by `codeturtle` (first-run wizard).", + "# Environment variables override these values.", + "", + ] + for key in _TOML_KEYS: + if key not in data or data[key] in (None, ""): + continue + val = data[key] + if isinstance(val, bool): + lines.append(f"{key} = {'true' if val else 'false'}") + elif isinstance(val, (int, float)) and not isinstance(val, bool): + lines.append(f"{key} = {val}") + else: + text = str(val).replace("\\", "\\\\").replace('"', '\\"') + lines.append(f'{key} = "{text}"') + return "\n".join(lines).rstrip() + "\n" + + +def save_user_config(updates: Mapping[str, Any], path: Optional[Path] = None) -> Path: + p = path or config_path() + p.parent.mkdir(parents=True, exist_ok=True) + current = load_user_config(p) + for key in _TOML_KEYS: + if key in updates and updates[key] not in (None, ""): + current[key] = updates[key] + p.write_text(dumps_toml(current), encoding="utf-8") + return p + + +def apply_user_config_to_environ( + data: Optional[Mapping[str, Any]] = None, + environ: Optional[Mapping[str, str]] = None, +) -> None: + """Copy config.toml into os.environ only for keys that are not already set.""" + env = os.environ if environ is None else environ + blob = dict(data) if data is not None else load_user_config() + for key, env_key in _ENV_FOR_KEY.items(): + val = blob.get(key) + if not val: + continue + if not str(env.get(env_key) or "").strip(): + env[env_key] = str(val) # type: ignore[index] + + +def apply_user_config_to_settings(settings_obj: Any) -> None: + data = load_user_config() + if data.get("github_token") and not getattr(settings_obj, "github_token", ""): + settings_obj.github_token = str(data["github_token"]) + if data.get("ollama_model"): + settings_obj.ollama_model = str(data["ollama_model"]) + if data.get("llm_backend"): + settings_obj.llm_backend = str(data["llm_backend"]) + if data.get("ollama_base_url"): + settings_obj.ollama_base_url = str(data["ollama_base_url"]) + + +def detect_github_token( + environ: Optional[Mapping[str, str]] = None, + *, + gh_token: Optional[str] = None, +) -> str: + """config.toml / env, then `gh auth token`. Does not prompt.""" + env = os.environ if environ is None else environ + for key in ("CODETURTLE_GITHUB_TOKEN", "GITHUB_TOKEN"): + v = str(env.get(key) or "").strip() + if v: + return v + cfg = load_user_config() + v = str(cfg.get("github_token") or "").strip() + if v: + return v + if gh_token is not None: + return str(gh_token).strip() + try: + proc = subprocess.run( + ["gh", "auth", "token"], + capture_output=True, + text=True, + timeout=8, + check=False, + ) + if proc.returncode == 0: + return (proc.stdout or "").strip() + except (OSError, subprocess.TimeoutExpired): + return "" + return "" diff --git a/core/workspace.py b/core/workspace.py new file mode 100644 index 0000000..4e33e22 --- /dev/null +++ b/core/workspace.py @@ -0,0 +1,222 @@ +"""Clone + Graphify index into ~/.codeturtle/repos. No GRAPHIFY_GRAPH_PATH.""" + +from __future__ import annotations + +import os +import shutil +import subprocess +import sys +from pathlib import Path +from typing import Callable, List, Optional, Sequence + +from core.repository_knowledge.paths import ( + repo_to_folder, + resolve_graph_path, + resolve_repo_dir, +) + +RunFn = Callable[..., subprocess.CompletedProcess] +REV_SIDECAR = ".codeturtle-rev" + + +class WorkspaceError(RuntimeError): + pass + + +def graphify_executable() -> str: + env = (os.environ.get("GRAPHIFY_BIN") or "").strip() + if env: + return env + found = shutil.which("graphify") + if found: + return found + sibling = Path(sys.executable).resolve().parent / ( + "graphify.exe" if os.name == "nt" else "graphify" + ) + if sibling.is_file(): + return str(sibling) + return "graphify" + + +def _run_default( + argv: Sequence[str], + *, + check: bool = True, + capture_output: bool = False, + timeout: Optional[int] = None, + cwd: Optional[str] = None, +) -> subprocess.CompletedProcess: + return subprocess.run( + list(argv), + check=check, + capture_output=capture_output, + text=True, + timeout=timeout, + cwd=cwd, + ) + + +def _head_sha(repo_dir: Path, run: RunFn) -> str: + try: + proc = run( + ["git", "-C", str(repo_dir), "rev-parse", "HEAD"], + check=True, + capture_output=True, + timeout=30, + ) + return (proc.stdout or "").strip() + except Exception: + return "" + + +def _sidecar_path(graph: Path) -> Path: + return graph.parent / REV_SIDECAR + + +def graph_is_stale(repo_dir: Path, graph: Path, run: RunFn) -> bool: + if not graph.is_file(): + return True + sha = _head_sha(repo_dir, run) + if not sha: + return False + side = _sidecar_path(graph) + if not side.is_file(): + return True + return side.read_text(encoding="utf-8").strip() != sha + + +def clone_url(repo: str, token: str = "") -> str: + slug = repo.strip().strip("/") + if token: + return f"https://x-access-token:{token}@github.com/{slug}.git" + return f"https://github.com/{slug}.git" + + +def ensure_clone( + repo: str, + *, + run: Optional[RunFn] = None, + token: str = "", +) -> Path: + """Shallow-clone owner/repo into ~/.codeturtle/repos/owner_repo if missing.""" + run = run or _run_default + dest = resolve_repo_dir(repo) + if (dest / ".git").exists() or dest.joinpath("graphify-out", "graph.json").is_file(): + return dest + dest.parent.mkdir(parents=True, exist_ok=True) + url = clone_url(repo, token=token) + print(f"[Workspace] clone {repo} → {dest}") + try: + run( + ["git", "clone", "--depth", "1", url, str(dest)], + check=True, + timeout=300, + ) + except subprocess.CalledProcessError as exc: + err = (getattr(exc, "stderr", None) or getattr(exc, "stdout", None) or str(exc)) + raise WorkspaceError(f"git clone failed for {repo}: {err}") from exc + except OSError as exc: + raise WorkspaceError( + "git is required on PATH to clone repositories." + ) from exc + return dest + + +def fetch_pull_ref( + repo_dir: Path, + number: int, + *, + run: Optional[RunFn] = None, +) -> None: + """Best-effort: check out the PR head so Graphify sees new files.""" + if number <= 0: + return + run = run or _run_default + branch = f"codeturtle-pr-{int(number)}" + try: + run( + [ + "git", + "-C", + str(repo_dir), + "fetch", + "origin", + f"pull/{int(number)}/head:{branch}", + "--depth", + "1", + ], + check=True, + capture_output=True, + timeout=180, + ) + run( + ["git", "-C", str(repo_dir), "checkout", branch], + check=True, + capture_output=True, + timeout=60, + ) + except Exception: + return + + +def _graphify_argv(repo_dir: Path) -> List[str]: + return [ + graphify_executable(), + "extract", + str(repo_dir), + "--code-only", + "--no-cluster", + ] + + +def ensure_index( + repo: str, + *, + run: Optional[RunFn] = None, + force: bool = False, +) -> Path: + """Build graphify-out/graph.json with `graphify extract --code-only` if missing/stale.""" + run = run or _run_default + dest = resolve_repo_dir(repo) + graph = resolve_graph_path(repo) + if not force and not graph_is_stale(dest, graph, run): + print(f"[Workspace] graph ready {graph}") + return graph + dest.mkdir(parents=True, exist_ok=True) + argv = _graphify_argv(dest) + print(f"[Workspace] graphify {' '.join(argv[1:])}") + try: + run(argv, check=True, capture_output=False, timeout=1800, cwd=str(dest)) + except FileNotFoundError as exc: + raise WorkspaceError( + "Graphify is not on PATH. Reinstall with: " + 'uv tool install "git+https://github.com/venkatpachala/CodeTurtle.git@v0.3.0"' + ) from exc + except subprocess.CalledProcessError as exc: + raise WorkspaceError(f"graphify failed for {repo}: {exc}") from exc + if not graph.is_file(): + raise WorkspaceError( + f"graphify finished but {graph} is missing. " + "Expected graphify-out/graph.json under the clone." + ) + sha = _head_sha(dest, run) + if sha: + graph.parent.mkdir(parents=True, exist_ok=True) + _sidecar_path(graph).write_text(sha + "\n", encoding="utf-8") + return graph + + +def ensure_workspace( + repo: str, + number: int = 0, + *, + run: Optional[RunFn] = None, + token: str = "", + force_index: bool = False, +) -> Path: + """Clone (if needed) + Graphify code-only index. Returns graph.json path.""" + run = run or _run_default + dest = ensure_clone(repo, run=run, token=token) + if number: + fetch_pull_ref(dest, number, run=run) + return ensure_index(repo, run=run, force=force_index) diff --git a/pyproject.toml b/pyproject.toml index 903d3d6..dbfe03f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "codeturtle-review" -version = "0.2.0" +version = "0.3.0" description = "Local-first agentic GitHub PR reviewer" readme = "README.md" requires-python = ">=3.11" @@ -22,6 +22,9 @@ dependencies = [ "langchain-core>=0.3.0", "pyyaml>=6.0", "gitpython>=3.1.50", + "mcp>=1,<3", + "graphifyy[mcp]>=0.9.50", + "langchain-ollama>=1.1.0", ] [project.optional-dependencies] @@ -37,7 +40,7 @@ langfuse = [ graphify = [ "mcp>=1,<3", "graphifyy[mcp]>=0.9.50", -] +] # included in the default install; extra kept for older docs qdrant = [ "qdrant-client>=1.18.0", "langchain-qdrant>=1.1.0", diff --git a/tests/test_cli_entrypoint.py b/tests/test_cli_entrypoint.py index fb11757..d476adc 100644 --- a/tests/test_cli_entrypoint.py +++ b/tests/test_cli_entrypoint.py @@ -27,6 +27,22 @@ def _run(*args: str, timeout: int = 60) -> subprocess.CompletedProcess: class TestCliHelp(unittest.TestCase): + def test_no_args_noninteractive_does_not_hang(self): + env = os.environ.copy() + env["PYTHONPATH"] = str(ROOT) + os.pathsep + env.get("PYTHONPATH", "") + env["CODETURTLE_NO_WIZARD"] = "1" + proc = subprocess.run( + [sys.executable, "-m", "cli.main"], + capture_output=True, + text=True, + cwd=str(ROOT), + env=env, + timeout=60, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + blob = ((proc.stdout or "") + (proc.stderr or "")).lower() + self.assertIn("review", blob) + def test_root_help_lists_commands(self): proc = _run("--help") self.assertEqual(proc.returncode, 0, proc.stderr) diff --git a/tests/test_packaging.py b/tests/test_packaging.py index 6e5c2a9..831cf49 100644 --- a/tests/test_packaging.py +++ b/tests/test_packaging.py @@ -18,7 +18,7 @@ def setUpClass(cls): def test_project_name_and_version(self): project = self.data["project"] self.assertEqual(project["name"], "codeturtle-review") - self.assertEqual(project["version"], "0.2.0") + self.assertEqual(project["version"], "0.3.0") self.assertGreaterEqual(project["requires-python"], ">=3.11") def test_console_script(self): @@ -30,9 +30,10 @@ def test_build_system(self): self.assertIn("hatchling", " ".join(build["requires"])) self.assertEqual(build["build-backend"], "hatchling.build") - def test_required_deps_do_not_force_optional_backends(self): + def test_required_deps_include_graphify_not_qdrant(self): deps = " ".join(self.data["project"]["dependencies"]).lower() - for needle in ("qdrant", "neo4j", "langfuse", "graphifyy", "langchain-ollama"): + self.assertIn("graphifyy", deps) + for needle in ("qdrant", "neo4j", "langfuse"): self.assertNotIn(needle, deps) def test_optional_extras_declared(self): diff --git a/tests/test_phase63_action.py b/tests/test_phase63_action.py index 892bd3d..b4c337b 100644 --- a/tests/test_phase63_action.py +++ b/tests/test_phase63_action.py @@ -90,14 +90,20 @@ def test_review_get_current_session_creates(self): cwd = os.getcwd() try: os.chdir(td) - with patch("core.memory.manager.init_db"): - with patch( - "core.memory.manager.MemoryManager.create_new_session", - return_value="auto-sess", - ): - sid = get_current_session() - self.assertEqual(sid, "auto-sess") - self.assertTrue(Path(".current_session").is_file()) + with patch.dict( + os.environ, + {"CODETURTLE_HOME": td, "GITHUB_ACTIONS": ""}, + clear=False, + ): + os.environ.pop("GITHUB_ACTIONS", None) + with patch("core.memory.manager.init_db"): + with patch( + "core.memory.manager.MemoryManager.create_new_session", + return_value="auto-sess", + ): + sid = get_current_session() + self.assertEqual(sid, "auto-sess") + self.assertTrue((Path(td) / "current_session").is_file()) finally: os.chdir(cwd) diff --git a/tests/test_workspace.py b/tests/test_workspace.py new file mode 100644 index 0000000..e26ed21 --- /dev/null +++ b/tests/test_workspace.py @@ -0,0 +1,195 @@ +"""Auto clone + Graphify index. Clone/graphify are mocked — no network.""" + +from __future__ import annotations + +import os +import sys +import tempfile +import unittest +from pathlib import Path +from unittest.mock import MagicMock, patch + +ROOT = Path(__file__).resolve().parents[1] +if str(ROOT) not in sys.path: + sys.path.insert(0, str(ROOT)) + + +class _Proc: + def __init__(self, stdout: str = "", returncode: int = 0): + self.stdout = stdout + self.stderr = "" + self.returncode = returncode + + +class TestReposHome(unittest.TestCase): + def test_resolve_graph_path_uses_codeturtle_home_repos(self): + from core.repository_knowledge.paths import resolve_graph_path, resolve_repo_dir + + with tempfile.TemporaryDirectory() as td: + home = Path(td) + with patch.dict( + os.environ, + {"CODETURTLE_HOME": str(home), "CODETURTLE_REPOS_ROOT": ""}, + clear=False, + ): + os.environ.pop("CODETURTLE_REPOS_ROOT", None) + repo_dir = resolve_repo_dir("confident-ai/deepeval") + graph = resolve_graph_path("confident-ai/deepeval") + self.assertEqual(repo_dir, (home / "repos" / "confident-ai_deepeval")) + self.assertTrue( + str(graph).replace("\\", "/").endswith( + "repos/confident-ai_deepeval/graphify-out/graph.json" + ) + ) + + +class TestEnsureClone(unittest.TestCase): + def test_clone_invoked_when_missing(self): + from core.workspace import ensure_clone + + with tempfile.TemporaryDirectory() as td: + home = Path(td) + calls = [] + + def run(argv, **kwargs): + calls.append(list(argv)) + dest = Path(argv[-1]) + dest.mkdir(parents=True, exist_ok=True) + (dest / ".git").mkdir() + return _Proc() + + with patch.dict(os.environ, {"CODETURTLE_HOME": str(home)}, clear=False): + os.environ.pop("CODETURTLE_REPOS_ROOT", None) + dest = ensure_clone("confident-ai/deepeval", run=run) + self.assertTrue((dest / ".git").is_dir()) + self.assertEqual(calls[0][:3], ["git", "clone", "--depth"]) + self.assertIn("https://github.com/confident-ai/deepeval.git", calls[0]) + + def test_clone_skipped_when_git_dir_exists(self): + from core.workspace import ensure_clone + + with tempfile.TemporaryDirectory() as td: + home = Path(td) + dest = home / "repos" / "owner_repo" + dest.mkdir(parents=True) + (dest / ".git").mkdir() + run = MagicMock(side_effect=AssertionError("clone should not run")) + with patch.dict(os.environ, {"CODETURTLE_HOME": str(home)}, clear=False): + os.environ.pop("CODETURTLE_REPOS_ROOT", None) + out = ensure_clone("owner/repo", run=run) + self.assertEqual(out, dest) + run.assert_not_called() + + +class TestEnsureIndex(unittest.TestCase): + def test_graphify_extract_when_graph_missing(self): + from core.workspace import ensure_index + + with tempfile.TemporaryDirectory() as td: + home = Path(td) + dest = home / "repos" / "owner_repo" + dest.mkdir(parents=True) + (dest / ".git").mkdir() + + def run(argv, **kwargs): + if argv[:1] == ["git"]: + return _Proc(stdout="abc123\n") + graph = dest / "graphify-out" / "graph.json" + graph.parent.mkdir(parents=True, exist_ok=True) + graph.write_text("{}", encoding="utf-8") + return _Proc() + + with patch.dict(os.environ, {"CODETURTLE_HOME": str(home)}, clear=False): + os.environ.pop("CODETURTLE_REPOS_ROOT", None) + graph = ensure_index("owner/repo", run=run) + self.assertTrue(graph.is_file()) + self.assertEqual(graph.name, "graph.json") + + def test_graphify_skipped_when_fresh(self): + from core.workspace import ensure_index + + with tempfile.TemporaryDirectory() as td: + home = Path(td) + dest = home / "repos" / "owner_repo" + gdir = dest / "graphify-out" + gdir.mkdir(parents=True) + (dest / ".git").mkdir() + (gdir / "graph.json").write_text("{}", encoding="utf-8") + (gdir / ".codeturtle-rev").write_text("abc123\n", encoding="utf-8") + + def run(argv, **kwargs): + if argv[:1] == ["git"] and "rev-parse" in argv: + return _Proc(stdout="abc123\n") + raise AssertionError(f"unexpected {argv}") + + with patch.dict(os.environ, {"CODETURTLE_HOME": str(home)}, clear=False): + os.environ.pop("CODETURTLE_REPOS_ROOT", None) + graph = ensure_index("owner/repo", run=run) + self.assertTrue(graph.is_file()) + + +class TestUserConfig(unittest.TestCase): + def test_save_and_load_roundtrip(self): + from core.user_config import load_user_config, save_user_config + + with tempfile.TemporaryDirectory() as td: + path = Path(td) / "config.toml" + save_user_config( + { + "github_token": "gho_test", + "ollama_model": "qwen2.5:7b", + "llm_backend": "ollama", + }, + path=path, + ) + data = load_user_config(path) + self.assertEqual(data["github_token"], "gho_test") + self.assertEqual(data["ollama_model"], "qwen2.5:7b") + self.assertEqual(data["llm_backend"], "ollama") + + def test_toml_applied_before_missing_env(self): + from core.user_config import apply_user_config_to_environ + + env = {} + apply_user_config_to_environ( + {"github_token": "from-toml", "ollama_model": "qwen2.5:7b"}, + environ=env, + ) + self.assertEqual(env["GITHUB_TOKEN"], "from-toml") + self.assertEqual(env["OLLAMA_MODEL"], "qwen2.5:7b") + env2 = {"GITHUB_TOKEN": "already"} + apply_user_config_to_environ({"github_token": "from-toml"}, environ=env2) + self.assertEqual(env2["GITHUB_TOKEN"], "already") + + def test_detect_prefers_env_over_gh(self): + from core.user_config import detect_github_token + + tok = detect_github_token( + environ={"GITHUB_TOKEN": "env-token"}, + gh_token="gh-token", + ) + self.assertEqual(tok, "env-token") + + +class TestOllamaList(unittest.TestCase): + def test_parses_ollama_list_table(self): + from cli.commands.wizard import list_ollama_models + + fake = ( + "NAME ID SIZE MODIFIED\n" + "qwen2.5:7b abc 4.7 GB 5 weeks ago\n" + "llama3.2:latest def 2.0 GB 1 day ago\n" + ) + + def fake_run(argv, **kwargs): + self.assertEqual(argv[:2], ["ollama", "list"]) + p = _Proc(stdout=fake, returncode=0) + return p + + with patch("cli.commands.wizard.subprocess.run", side_effect=fake_run): + names = list_ollama_models() + self.assertEqual(names, ["qwen2.5:7b", "llama3.2:latest"]) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/uv.lock b/uv.lock index fb106e0..4707576 100644 --- a/uv.lock +++ b/uv.lock @@ -252,12 +252,15 @@ wheels = [ [[package]] name = "codeturtle-review" -version = "0.2.0" +version = "0.3.0" source = { editable = "." } dependencies = [ { name = "gitpython" }, + { name = "graphifyy", extra = ["mcp"] }, { name = "langchain-core" }, + { name = "langchain-ollama" }, { name = "langgraph" }, + { name = "mcp" }, { name = "pydantic" }, { name = "pydantic-settings" }, { name = "pygithub" }, @@ -293,14 +296,17 @@ qdrant = [ [package.metadata] requires-dist = [ { name = "gitpython", specifier = ">=3.1.50" }, + { name = "graphifyy", extras = ["mcp"], specifier = ">=0.9.50" }, { name = "graphifyy", extras = ["mcp"], marker = "extra == 'graphify'", specifier = ">=0.9.50" }, { name = "langchain-core", specifier = ">=0.3.0" }, + { name = "langchain-ollama", specifier = ">=1.1.0" }, { name = "langchain-ollama", marker = "extra == 'ollama'", specifier = ">=1.1.0" }, { name = "langchain-ollama", marker = "extra == 'qdrant'", specifier = ">=1.1.0" }, { name = "langchain-openai", marker = "extra == 'openai'", specifier = ">=1.3.5" }, { name = "langchain-qdrant", marker = "extra == 'qdrant'", specifier = ">=1.1.0" }, { name = "langfuse", marker = "extra == 'langfuse'", specifier = ">=4.14.0" }, { name = "langgraph", specifier = ">=0.2.0" }, + { name = "mcp", specifier = ">=1,<3" }, { name = "mcp", marker = "extra == 'graphify'", specifier = ">=1,<3" }, { name = "neo4j", marker = "extra == 'neo4j'", specifier = ">=6.2.0" }, { name = "pydantic", specifier = ">=2.0.0" },