Skip to content

feat(go): third-party type resolution from vendor/GOMODCACHE#641

Open
shivasurya wants to merge 7 commits intomainfrom
shiva/pr-01-golang-thirdparty
Open

feat(go): third-party type resolution from vendor/GOMODCACHE#641
shivasurya wants to merge 7 commits intomainfrom
shiva/pr-01-golang-thirdparty

Conversation

@shivasurya
Copy link
Copy Markdown
Owner

Summary

Adds foundational Go third-party type resolution: validated method calls on gorm, gin, grpc, and any other vendored/cached dependency — replacing the previous best-effort fallback.

What this PR delivers

  • GoThirdPartyLoader interface (core/types.go) — mirrors GoStdlibLoader, reuses GoStdlibType/GoStdlibPackage (zero new types)
  • GoThirdPartyLocalLoader (registry/go_thirdparty_local.go) — parses vendor/ or GOMODCACHE via tree-sitter; extracts exported types, methods, fields, functions; flattens embedded interface methods (same-package recursive + well-known cross-package io.Closer/io.Reader/etc.)
  • Pattern 1b Check 2.5 (builder/go_builder.go) — sits between Check 2 (stdlib) and Check 3 (promoted methods); validates third-party method calls against extracted metadata
  • Return type inference (extraction/go_variables.go) — inferTypeFromThirdPartyFunction() infers return types from third-party constructors (e.g. gorm.Open → *gorm.DB)
  • Disk cache{userCacheDir}/code-pathfinder/go-thirdparty/{sha256(projectRoot)[:12]}/ with cache-index.json version tracking; cold run ~50-200ms, warm run ~5ms; version mismatch triggers re-extraction
  • --refresh-rules integration — reuses existing flag to also flush the go-thirdparty disk cache (no new flag)

Resolution chain after this PR

Check 1   → user code (callGraph.Functions)
Check 2   → stdlib CDN (167 packages)
Check 2.5 → third-party vendor/GOMODCACHE  ← NEW
Check 3   → promoted methods (struct embedding)
Check 4   → best-effort FQN fallback

PoC validation (from branch shiva/go-thirdparty-poc)

  • vendored gorm + gin stubs: 7/7 calls resolved via Check 2.5
  • real vendored posthog-go: 4/4 calls resolved (including embedded io.Closer and EnqueueClient)

Test plan

  • TestDiskCacheWriteAndRead — cold run writes JSON, warm run reads from disk without vendor/
  • TestCacheVersionMismatch — go.mod version bump triggers re-extraction
  • TestRefreshCacheFlushrefreshCache=true wipes stale cache and re-extracts
  • TestGormCheck25 / TestGinSubpackageResolution — full pipeline integration tests
  • All existing registry and builder tests pass (go test ./... -count=1)

🤖 Generated with Claude Code

shivasurya and others added 3 commits April 5, 2026 14:48
Adds GoThirdPartyLoader interface and GoThirdPartyLocalLoader that
parses third-party Go packages from vendor/ or GOMODCACHE using
tree-sitter. Extracts exported types, methods, fields, and functions
into the same GoStdlibPackage format used by stdlib resolution.

Integration:
- Pattern 1b Check 2.5: validates method existence on third-party types
  between stdlib (Check 2) and promoted methods (Check 3)
- Return type inference: inferTypeFromThirdPartyFunction for variable
  type bindings from third-party function calls
- scan.go: InitGoThirdPartyLoader wired after stdlib loader init

Validated: gorm.io/gorm.DB.Raw, gin.Context.Query resolve at 100% rate
in integration tests with vendored source.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixes interface embedding resolution for types like posthog.Client
which embeds EnqueueClient and io.Closer. Previously, only direct
method_spec nodes were extracted — embedded interfaces were silently
skipped, causing methods like Enqueue() and Close() to fall through
to Check 4 (best-effort) instead of Check 2.5 (validated).

Changes:
- extractInterfaceMethods now captures type_elem → type_identifier
  and type_elem → qualified_type nodes as embedded interface names
- flattenEmbeddedMethods resolves same-package embeds by looking up
  in pkg.Types (handles multi-level embedding recursively)
- resolveWellKnownEmbeds handles cross-package stdlib embeds
  (io.Closer, io.Reader, io.Writer, fmt.Stringer, error) via
  hardcoded method table
- Embeds field added to GoStdlibType for tracking

Validated: posthog.Client.Enqueue() and .Close() now resolve via
Check 2.5 from vendored source with 100% resolution rate.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ader

- Add disk cache layer to GoThirdPartyLocalLoader:
  - Cache path: {userCacheDir}/code-pathfinder/go-thirdparty/{sha256(projectRoot)[:12]}/
  - cache-index.json tracks version per module for invalidation on go.mod upgrades
  - Cold run extracts via tree-sitter and writes JSON; warm run reads from disk (~5ms vs ~200ms)
  - Version mismatch (go.mod bumped) triggers re-extraction automatically

- Reuse --refresh-rules flag to also flush go-thirdparty disk cache (no new CLI flag)
  - InitGoThirdPartyLoader now accepts refreshCache bool; passed from scan.go refreshRules

- Update NewGoThirdPartyLocalLoader signature (add refreshCache bool param)
- Fix all call sites in tests and builder

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@shivasurya shivasurya added enhancement New feature or request go Pull requests that update go code labels Apr 5, 2026
@shivasurya shivasurya self-assigned this Apr 5, 2026
@shivasurya shivasurya added enhancement New feature or request go Pull requests that update go code labels Apr 5, 2026
@safedep
Copy link
Copy Markdown

safedep bot commented Apr 5, 2026

SafeDep Report Summary

Green Malicious Packages Badge Green Vulnerable Packages Badge Green Risky License Badge

No dependency changes detected. Nothing to scan.

View complete scan results →

This report is generated by SafeDep Github App

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 5, 2026

Code Pathfinder Security Scan

Pass Critical High Medium Low Info

No security issues detected.

Metric Value
Files Scanned 11
Rules 205

Powered by Code Pathfinder

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 5, 2026

Codecov Report

❌ Patch coverage is 83.98510% with 86 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.21%. Comparing base (e11335d) to head (994c4bd).

Files with missing lines Patch % Lines
...ne/graph/callgraph/registry/go_thirdparty_local.go 83.60% 48 Missing and 33 partials ⚠️
...-engine/graph/callgraph/extraction/go_variables.go 80.95% 2 Missing and 2 partials ⚠️
sast-engine/cmd/scan.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main     #641    +/-   ##
========================================
  Coverage   84.21%   84.21%            
========================================
  Files         161      162     +1     
  Lines       22719    23256   +537     
========================================
+ Hits        19133    19586   +453     
- Misses       2886     2936    +50     
- Partials      700      734    +34     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

shivasurya and others added 4 commits April 5, 2026 15:21
- godot: add trailing periods to 3 comments
- tagliatelle: rename json tag cached_at → cachedAt (camelCase)
- nilnil: introduce errPackageSourceNotFound sentinel; return it
  instead of (nil, nil) from getOrLoadPackage
- unused: remove resolveEmbeddedFromExternal (dead code from PoC)
- prealloc: preallocate fieldNames slice in test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…artyFunction

Registry (go_thirdparty_local_test.go):
- Add TestPackageCount, TestGoThirdPartyLocalGetType/GetFunction_NotFound
- Add TestGetType_PackageNotFound (errPackageSourceNotFound path)
- Add TestFindPackageSource_GOMODCACHE and _Subpackage (GOMODCACHE branch)
- Add TestLoadCacheIndex_InvalidJSON, TestWriteToDiskCache_NilDiskIndex
- Add TestIsExported_EmptyString, TestExtractMethodDecl_NoReceiverType
- Add TestInitDiskCache_NoGoMod

Extraction (go_variables_stdlib_test.go):
- Add mockThirdPartyLoader and 5 tests covering all paths of
  inferTypeFromThirdPartyFunction (nil loader, unknown pkg, func not found,
  error-only return, success)

Builder (go_version_test.go):
- Add TestInitGoThirdPartyLoader_{NilReg,NoDependencies,WithDependencies,RefreshCache}

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- TestModuleKeyFor_NoMatch: fallback return importPath unchanged
- TestLoadFromDiskCache_MissingFile: ReadFile error → nil
- TestLoadFromDiskCache_WithLogger: debug log branch on cache hit
- TestSaveCacheIndex_EmptyCacheDir: no-op when cacheDir is ""
- TestWriteToDiskCache_WriteFailure: WriteFile error → log + return
- TestInitDiskCache_MkdirAllFailure: MkdirAll failure → diskIndex stays nil
- TestInitDiskCache_RefreshWithLogger: refreshCache=true with logger
- TestExtractMethodDecl_UnexportedMethod: unexported methods skipped

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request go Pull requests that update go code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant