Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 11 additions & 9 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ all of them.
┌───────── Frontends (in-process Go, one binary) ─────────┐
Go ────► x/tools SSA ───┐
Python ──► python3 ast ──┤
JS ────► goja AST ──┤
JS ────► esbuild AST ──┤
Java ────► JVM bytecode ──┤─► lower ─► gIR (core + intrinsics, canonical FQNs)
Rust ────► rustc MIR ──┤ │
Ruby ────► ruby Ripper ──┤ │
Expand Down Expand Up @@ -177,14 +177,16 @@ Python/Java/Rust/Ruby shelling out to a toolchain on `PATH`.
lives in `http.HandleFunc` closures. Emits `go:` names.
- **Python** (`converters/python/`) — shells out to `python3` for an `ast` JSON
dump, then lowers it to a real CFG (`ssabuild`). Emits `py:` names; requires `python3`.
- **JavaScript** (`converters/javascript/`) — pure-Go parse via **goja**, then
lowers. TS/JSX/ESM are stripped/lowered in-process by esbuild (no Node), with source
maps remapping positions back; a plain `.js` is handed to goja first and only falls
back to esbuild when the parse fails, since one extension covers four dialects.
Flow, which esbuild cannot load at all, is blanked in place beforehand
(`flowstrip.go`) so byte offsets — and therefore positions — are preserved. `.vue`/`.svelte` SFCs are also
handled (`sfc.go`): the `<script>` block lowers as JS/TS and each dangerous template
directive (`v-html`, `{@html}`) compiles to a synthetic sink call. Emits `js:` names.
- **JavaScript** (`converters/javascript/`) — pure-Go parse of **esbuild's AST**
(`github.com/bytevet/esbuild-jsast`, which re-exports the parser esbuild keeps under
`internal/`), then lowers. TypeScript, JSX and ES modules are parsed as themselves —
no text transform, no Node, and no sourcemap in the position path, since a node's byte
offset already indexes the source as written. One extension covers four dialects, so the
dialect is found by trying them (`parseLadder`) rather than predicted. Flow, which is
neither JS nor TS, is blanked in place beforehand (`flowstrip.go`) so byte offsets — and
therefore positions — are preserved. `.vue`/`.svelte` SFCs are also handled (`sfc.go`):
the `<script>` block lowers as JS/TS and each dangerous template directive (`v-html`,
`{@html}`) compiles to a synthetic sink call. Emits `js:` names.
- **Java** (`converters/java/`) — analyzes JVM **bytecode**. An embedded helper
(`JavaDump.java`, run via a JDK 24+ `java`) compiles `.java` in-process and reads
`.class` with `java.lang.classfile`; `lower.go` simulates the operand stack to
Expand Down
7 changes: 4 additions & 3 deletions BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,16 +43,17 @@ toolchain-gated, net-new frontends, or deferred perf work.
| FE-3 | crit | ✅ `6fb8ad5` | Rust bin crates + workspaces via `cargo metadata` per-target MIR emit. |
| FE-4 | high | ✅ `9300e96` | Java CFG reconstruction + operand-stack/local PHI merge at control-flow joins. |
| FE-5 | high | ✅ | "Default if empty" branch-merge PHI in Python, JS, and Rust (block-by-block for MIR). |
| FE-6 | high | ✅ `803dcfd` | TypeScript / JSX / `.mjs`/`.cjs` / ESM via in-process esbuild transform + sourcemap remap. |
| FE-6 | high | ✅ `803dcfd` | TypeScript / JSX / `.mjs`/`.cjs` / ESM via in-process esbuild transform + sourcemap remap. Mechanism superseded by FE-16; the dialects it covered are still covered. |
| FE-7 | high | ✅ `0a60df8` | Python dict/set literals lowered as sequences so inner sources/sinks fire. |
| FE-8 | high | ✅ `12389e9` | Java findings anchor to each class's `.java` via the SourceFile attribute. |
| FE-9 | med | ✅ `f866600` | Java probes `java -version` and surfaces the real javac diagnostic on failure. |
| FE-10 | med | ✅ `f866600` | Rust MIR-shape smoke test warns on rustc format drift. |
| FE-11 | high | ✅ `da50352` | `.js` dialect chosen by PARSE FAILURE, not prediction. A content sniff (and before it, the extension alone) had to guess which of plain-script/ESM/Flow/JSX a `.js` was; guessing cost the whole file when wrong and 7% of `Scan_JS` when right. goja parses first, `loaderLadder` runs only on failure. |
| FE-12 | high | ✅ `9357b76`,`4d1729c` | Flow-typed `.js` recovered by blanking Flow syntax in place (`flowstrip.go`), offset-preserving because `go-sourcemap` is consumer-only and positions are mandatory. parse-server converts **all 193 source files** (196 gIR modules) (187/196 after the first cut, 177/196 before it). The residual was not only casts: `export`/`declare` modifiers stranded by the blank, multi-line unions cut at their first newline, casts nested in a call or applied to a pattern, and Flow's generic bound `<T: X>`. Three silent-corruption bugs surfaced with them, each blanking VALUE code while staying brace-balanced so the result still parsed: a `{}` in a ternary branch reset the pending-`?` counter; `type` used as an ORDINARY identifier (`(type as X)`, pervasive in React) read as a type alias; and a `{` opening an import/export specifier list was taken for a block, so `import { type Config } from …` blanked the import and everything after it. Per-bracket state is now one frame stack, and a declaration position is a property of the enclosing frame rather than of the previous byte. Verified on parse-server and React (1,874 files): zero blanked spans contain executable tokens. |
| FE-11 | high | ✅ `da50352` | `.js` dialect chosen by PARSE FAILURE, not prediction. A content sniff (and before it, the extension alone) had to guess which of plain-script/ESM/Flow/JSX a `.js` was; guessing cost the whole file when wrong and 7% of `Scan_JS` when right. Mechanism superseded by FE-16: `parseLadder` IS the parse, from its first rung. |
| FE-12 | high | ✅ `9357b76`,`4d1729c` | Flow-typed `.js` recovered by blanking Flow syntax in place (`flowstrip.go`), offset-preserving because positions are mandatory and a node's offset indexes the buffer the parser was handed. parse-server converts **all 193 source files** (196 gIR modules) (187/196 after the first cut, 177/196 before it). The residual was not only casts: `export`/`declare` modifiers stranded by the blank, multi-line unions cut at their first newline, casts nested in a call or applied to a pattern, and Flow's generic bound `<T: X>`. Three silent-corruption bugs surfaced with them, each blanking VALUE code while staying brace-balanced so the result still parsed: a `{}` in a ternary branch reset the pending-`?` counter; `type` used as an ORDINARY identifier (`(type as X)`, pervasive in React) read as a type alias; and a `{` opening an import/export specifier list was taken for a block, so `import { type Config } from …` blanked the import and everything after it. Per-bracket state is now one frame stack, and a declaration position is a property of the enclosing frame rather than of the previous byte. Verified on parse-server and React (1,874 files): zero blanked spans contain executable tokens. |
| FE-13 | high | ✅ `4f2871c` | Ruby binary operators reach gIR. `lowerBinary` never read the operator, so every binary expression lowered as `BIN_OP_ADD` — the universal propagator — and `user == "admin"` carried taint. `<<`→ADD (shovel is append), `%`→REM (so `ssrf.go` reads it as a format template), comparisons→`builtin.compare`. |
| FE-14 | med | ⏸ | Ruby `.to_s` is not a modeled propagator, so it DROPS taint; `unary` (`!x`) and `ifop` (`cond ? a : b`) are unlowered and sink taint into `ruby.unsupported`. The ternary is a real false negative. Found while writing FE-13's samples. |
| FE-15 | med | ⏸ | React still drops 237 of 1,874 files: Flow syntax `flowstrip.go`'s subset does not cover -- dropped, with no mis-lowering observed in the parse-server/React sweep (which bounds the corpus, not the construct space). Full fidelity would mean `hermes-parser` (C++→WASM) under a pure-Go WASM runtime; its emscripten ABI needs a spike before any of it is reachable. Spun out of FE-12 so a closed row does not carry open work. |
| FE-16 | high | ✅ | JS parses esbuild's AST directly (`github.com/bytevet/esbuild-jsast`), replacing goja + `api.Transform` + sourcemap remapping. The round trip cost precision as well as time: printing was ~79% of the transform, and goja's ragged grammar forced downlevels that rewrote object spread into a helper taint did not survive. Top-level await was an outright gap — expressible only in ESM output, which the CommonJS-consuming lowering could not read — and is now supported. Positions are correct by construction (a node offset indexes the source as written), verified byte-identical against the pre-change build over all 58 `test/js` findings. |

## Detection & secrets coverage (COV)

Expand Down
15 changes: 8 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,13 +67,14 @@ changing one:
parameter — see its doc comment.
- `python/`, `ruby/` — shell out to `python3` / `ruby` for an AST dump via an embedded helper
(`pyast.py`, `rbdump.rb`), then lower it.
- `javascript/` — pure-Go goja parse. Extensions that always need one (`.ts/.tsx/.jsx/.mjs/.cjs`) take
esbuild's in-process `Transform` up front, with a source-map consumer remapping positions back
(`transform.go`). Plain `.js` does NOT: it is the ambiguous extension — plain script, ESM, Flow and
JSX all ship as `.js` — so it goes straight to goja and the PARSE FAILURE drives `loaderLadder`,
which is exact where predicting the dialect was not. `flowstrip.go` is that ladder's last rung:
Flow has no esbuild loader, so it blanks Flow-only syntax IN PLACE, every removed byte becoming a
space, because positions must survive and the sourcemap library cannot compose two maps.
- `javascript/` — pure-Go parse of esbuild's own AST, via `github.com/bytevet/esbuild-jsast` (the
parser is walled off under esbuild's `internal/`; that module is the seam). There is NO text
transform and no sourcemap: TS/JSX/ESM arrive as themselves, and a node offset indexes the source
as written, so `dialect.go`'s `lineIndex` is the ONE place an offset becomes a position. Which
dialect is decided by PARSE FAILURE, not prediction (`parseLadder`), which is exact where
guessing was not; the rung ORDER is load-bearing and pinned by a test. `flowstrip.go` is that ladder's last rung:
Flow is neither JS nor TS, so it blanks Flow-only syntax IN PLACE, every removed byte becoming a
space, because a node's position is an offset into the buffer the parser was handed.
`.vue`/`.svelte` SFCs (`sfc.go`) lower the `<script>` block as the module body and append each
dangerous template directive (`v-html`, `{@html}`) as a synthetic sink CALL.
- `java/` — JVM **bytecode**, via an embedded `JavaDump.java` (JDK 24+, `java.lang.classfile`) plus an
Expand Down
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,13 @@ fmt:
gofmt -l cmd converters internal test/corpus
vet:
go vet ./...
gate: fmt vet build test
# CI's blocking check, and a strict superset of `go vet`. golangci-lint refuses to
# run when it was built with an older Go than this module targets, so an outdated
# binary fails closed with a config-load error rather than a clean report:
# GOTOOLCHAIN=go1.26.5 go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.12.2
lint:
golangci-lint run
gate: fmt vet lint build test

# --- with the C/C++/Rust LLVM frontends (cgo + libLLVM) ---
build-llvm:
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ flowchart LR
## Install

```bash
go install godzilla/cmd/godzilla@latest # or, from a clone:
go install github.com/bytevet/godzilla/cmd/godzilla@latest # or, from a clone:
go build -o godzilla ./cmd/godzilla
```

Expand Down Expand Up @@ -153,7 +153,7 @@ releases; `edge`/`edge-full` track `main`. Multi-arch (amd64 + arm64).

| | Go | Python | JavaScript | Java | Rust | Ruby |
|---|---|---|---|---|---|---|
| Parser | `golang.org/x/tools` SSA | `python3` `ast` | goja (pure Go); TS/JSX/ESM via esbuild; Flow blanked in place; `.vue`/`.svelte` SFCs | JVM bytecode (`java.lang.classfile`) | rustc MIR | `ruby` Ripper; `.erb` templates |
| Parser | `golang.org/x/tools` SSA | `python3` `ast` | esbuild AST (pure Go); TS/JSX/ESM natively; Flow blanked in place; `.vue`/`.svelte` SFCs | JVM bytecode (`java.lang.classfile`) | rustc MIR | `ruby` Ripper; `.erb` templates |
| SQL injection | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Command injection | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Path traversal | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Expand Down
24 changes: 12 additions & 12 deletions cmd/godzilla/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,18 @@ import (
"slices"
"strings"

"godzilla/internal/analysis"
"godzilla/internal/buildpolicy"
"godzilla/internal/config"
"godzilla/internal/llm"
"godzilla/internal/memlimit"
"godzilla/internal/proc"
"godzilla/internal/report"
"godzilla/internal/rules"
"godzilla/internal/rules/loader"
"godzilla/internal/scan"
"godzilla/internal/triage"
ir "godzilla/pkg/ir/v1"
"github.com/bytevet/godzilla/internal/analysis"
"github.com/bytevet/godzilla/internal/buildpolicy"
"github.com/bytevet/godzilla/internal/config"
"github.com/bytevet/godzilla/internal/llm"
"github.com/bytevet/godzilla/internal/memlimit"
"github.com/bytevet/godzilla/internal/proc"
"github.com/bytevet/godzilla/internal/report"
"github.com/bytevet/godzilla/internal/rules"
"github.com/bytevet/godzilla/internal/rules/loader"
"github.com/bytevet/godzilla/internal/scan"
"github.com/bytevet/godzilla/internal/triage"
ir "github.com/bytevet/godzilla/pkg/ir/v1"
)

// version is the tool version, overridable at build time via
Expand Down
2 changes: 1 addition & 1 deletion cmd/godzilla/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"strings"
"testing"

"godzilla/internal/testsupport"
"github.com/bytevet/godzilla/internal/testsupport"
)

// runCLI builds and runs the godzilla CLI (via `go run .`) with args, returning
Expand Down
6 changes: 3 additions & 3 deletions cmd/godzilla/rules.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ import (
"slices"
"strings"

"godzilla/internal/rules"
"godzilla/internal/rules/loader"
"godzilla/internal/ruletest"
"github.com/bytevet/godzilla/internal/rules"
"github.com/bytevet/godzilla/internal/rules/loader"
"github.com/bytevet/godzilla/internal/ruletest"
)

const rulesUsageText = `usage: godzilla rules <list|lint|test> [args]
Expand Down
8 changes: 4 additions & 4 deletions converters/cpp/converter.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ import (
"path/filepath"
"strings"

"godzilla/converters/frontend"
llvm_converter "godzilla/converters/llvm"
"godzilla/internal/proc"
ir "godzilla/pkg/ir/v1"
"github.com/bytevet/godzilla/converters/frontend"
llvm_converter "github.com/bytevet/godzilla/converters/llvm"
"github.com/bytevet/godzilla/internal/proc"
ir "github.com/bytevet/godzilla/pkg/ir/v1"
)

// Converter lowers C/C++ sources into gIR: the shared frontend.Driver surface
Expand Down
4 changes: 2 additions & 2 deletions converters/cpp/converter_stub.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ package cpp_converter
import (
"fmt"

"godzilla/internal/walkignore"
ir "godzilla/pkg/ir/v1"
"github.com/bytevet/godzilla/internal/walkignore"
ir "github.com/bytevet/godzilla/pkg/ir/v1"
)

type Converter struct{}
Expand Down
4 changes: 2 additions & 2 deletions converters/frontend/frontend.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ import (
"strings"
"sync"

"godzilla/internal/walkignore"
ir "godzilla/pkg/ir/v1"
"github.com/bytevet/godzilla/internal/walkignore"
ir "github.com/bytevet/godzilla/pkg/ir/v1"
)

// Driver is the embeddable front half of a Batch-based Converter: the
Expand Down
2 changes: 1 addition & 1 deletion converters/go/converter.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import (
"strings"
"sync"

ir "godzilla/pkg/ir/v1"
ir "github.com/bytevet/godzilla/pkg/ir/v1"
"golang.org/x/tools/go/packages"
"golang.org/x/tools/go/ssa"
"golang.org/x/tools/go/ssa/ssautil"
Expand Down
2 changes: 1 addition & 1 deletion converters/go/converter_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"strings"
"testing"

ir "godzilla/pkg/ir/v1"
ir "github.com/bytevet/godzilla/pkg/ir/v1"
)

func TestConvertFile(t *testing.T) {
Expand Down
2 changes: 1 addition & 1 deletion converters/go/format_marker_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"path/filepath"
"testing"

ir "godzilla/pkg/ir/v1"
ir "github.com/bytevet/godzilla/pkg/ir/v1"
)

// TestFormatMarkerExactCalleeMatch is the regression guard for the
Expand Down
6 changes: 3 additions & 3 deletions converters/java/branchmerge_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ package java_converter
import (
"testing"

"godzilla/internal/analysis"
"godzilla/internal/rules"
ir "godzilla/pkg/ir/v1"
"github.com/bytevet/godzilla/internal/analysis"
"github.com/bytevet/godzilla/internal/rules"
ir "github.com/bytevet/godzilla/pkg/ir/v1"
)

// TestBranchMerge_TernaryKeepsTaint is a hermetic guard (no JDK) for FE-4. It
Expand Down
8 changes: 4 additions & 4 deletions converters/java/converter.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ import (
"strings"
"sync"

"godzilla/internal/buildpolicy"
"godzilla/internal/proc"
"godzilla/internal/walkignore"
ir "godzilla/pkg/ir/v1"
"github.com/bytevet/godzilla/internal/buildpolicy"
"github.com/bytevet/godzilla/internal/proc"
"github.com/bytevet/godzilla/internal/walkignore"
ir "github.com/bytevet/godzilla/pkg/ir/v1"
)

//go:embed JavaDump.java
Expand Down
6 changes: 3 additions & 3 deletions converters/java/converter_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ import (
"strings"
"testing"

"godzilla/internal/irwalk"
"godzilla/internal/testsupport"
ir "godzilla/pkg/ir/v1"
"github.com/bytevet/godzilla/internal/irwalk"
"github.com/bytevet/godzilla/internal/testsupport"
ir "github.com/bytevet/godzilla/pkg/ir/v1"
)

// requireJava skips when no JDK `java` launcher is on PATH (the frontend runs
Expand Down
4 changes: 2 additions & 2 deletions converters/java/lower.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import (
"slices"
"strings"

"godzilla/converters/ssabuild"
ir "godzilla/pkg/ir/v1"
"github.com/bytevet/godzilla/converters/ssabuild"
ir "github.com/bytevet/godzilla/pkg/ir/v1"
)

// convertClass turns one dumped class into a gIR module (one function per method).
Expand Down
Loading
Loading