Conversation
openshift-ci
bot
added
the
do-not-merge/work-in-progress
✅ Deploy Preview for olmv1 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2100 +/- ##
==========================================
- Coverage 68.95% 67.99% -0.96%
==========================================
Files 139 142 +3
Lines 9891 10339 +448
==========================================
+ Hits 6820 7030 +210
- Misses 2562 2771 +209
- Partials 509 538 +29
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
closing this as stale - please reopen if needed |
grokspawn
force-pushed
the
catd-graphql-play
branch
from
c7edbd2 to
559b18f
Compare
There was a problem hiding this comment.
Pull Request Overview
This pull request refactors the catalogd storage layer to introduce GraphQL support for querying catalog data. The changes extract HTTP handler logic into a separate server package, introduce a service layer for GraphQL schema management with caching, and replace direct struct initialization with a constructor pattern for LocalDirV1.
- Introduces a new GraphQL endpoint at
/api/v1/graphqlwith dynamic schema generation - Refactors HTTP handlers into a dedicated
serverpackage with cleaner separation of concerns - Adds a GraphQL service layer with schema caching to improve performance
- Updates
LocalDirV1to use a constructor pattern (NewLocalDirV1) for proper initialization
Reviewed Changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/catalogd/storage/localdir.go | Refactored to use constructor pattern, added GraphQL service integration, moved HTTP handlers to server package |
| internal/catalogd/storage/localdir_test.go | Updated all test instantiations to use new NewLocalDirV1 constructor |
| internal/catalogd/storage/http_preconditions_check.go | Removed (moved to server package with simplified implementation) |
| internal/catalogd/server/handlers.go | New file implementing HTTP handlers extracted from storage layer |
| internal/catalogd/server/http_helpers.go | New file with simplified HTTP precondition checking |
| internal/catalogd/service/graphql_service.go | New GraphQL service with caching for schema generation |
| internal/catalogd/graphql/graphql.go | New dynamic GraphQL schema generation implementation |
| internal/catalogd/graphql/graphql_test.go | Tests for GraphQL schema discovery |
| internal/catalogd/graphql/discovery_test.go | Additional comprehensive tests for schema discovery edge cases |
| internal/catalogd/graphql/sample-queries.txt | Documentation of sample GraphQL queries |
| internal/catalogd/graphql/README.md | Documentation for GraphQL integration |
| cmd/catalogd/main.go | Updated to use NewLocalDirV1 constructor |
| go.mod, go.sum | Added graphql-go/graphql dependency |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
grokspawn
force-pushed
the
catd-graphql-play
branch
from
b9a05a8 to
58fbad3
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 15 out of 16 changed files in this pull request and generated 12 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Get or build the schema | ||
| // TODO: prevent cache rebuild on this callpath | ||
| dynamicSchema, err := s.GetSchema(catalog, catalogFS) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("failed to get GraphQL schema: %w", err) |
There was a problem hiding this comment.
The TODO comment "prevent cache rebuild on this callpath" suggests a known performance issue where the schema cache might be rebuilt unnecessarily. This should be addressed before merging, or tracked with a specific issue reference.
| // Get or build the schema | |
| // TODO: prevent cache rebuild on this callpath | |
| dynamicSchema, err := s.GetSchema(catalog, catalogFS) | |
| if err != nil { | |
| return nil, fmt.Errorf("failed to get GraphQL schema: %w", err) | |
| // Get the schema from cache if available, otherwise build and cache it | |
| s.schemaMux.RLock() | |
| dynamicSchema, ok := s.schemaCache[catalog] | |
| s.schemaMux.RUnlock() | |
| if !ok { | |
| var err error | |
| dynamicSchema, err = s.GetSchema(catalog, catalogFS) | |
| if err != nil { | |
| return nil, fmt.Errorf("failed to get GraphQL schema: %w", err) | |
| } |
internal/catalogd/graphql/graphql.go
Outdated
| // If we have an empty part after having content, it means there was a trailing separator | ||
| // Add a capitalized version of the last word | ||
| if hasContent && i == len(parts)-1 { | ||
| // Get the base word (first non-empty part) | ||
| for _, p := range parts { | ||
| if p != "" { | ||
| result += strings.ToUpper(string(p[0])) + strings.ToLower(p[1:]) | ||
| break | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
The field remapping logic in remapFieldName has complex handling for trailing underscores (lines 68-78) that seems incorrect. When an empty part is found at the end of the parts array (line 68), it tries to capitalize and append a word, but this logic appears to duplicate parts of field names rather than properly handle trailing separators. For example, a field ending with an underscore might get unexpected capitalization. This needs clarification or simplification.
| // If we have an empty part after having content, it means there was a trailing separator | |
| // Add a capitalized version of the last word | |
| if hasContent && i == len(parts)-1 { | |
| // Get the base word (first non-empty part) | |
| for _, p := range parts { | |
| if p != "" { | |
| result += strings.ToUpper(string(p[0])) + strings.ToLower(p[1:]) | |
| break | |
| } | |
| } | |
| } | |
| // Skip empty parts (e.g., from trailing or consecutive underscores) |
| GraphQLCatalogQueries = featuregate.Feature("GraphQLCatalogQueries") | ||
| ) | ||
|
|
||
| var catalogdFeatureGates = map[featuregate.Feature]featuregate.FeatureSpec{ | ||
| APIV1MetasHandler: {Default: false, PreRelease: featuregate.Alpha}, | ||
| GraphQLCatalogQueries: {Default: false, PreRelease: featuregate.Alpha}, |
There was a problem hiding this comment.
The feature gate GraphQLCatalogQueries is defined in features.go but is never checked before enabling the GraphQL endpoint. The endpoint is unconditionally registered in handlers.go line 67, regardless of the feature gate setting. This means the feature gate has no effect. The feature gate should be checked in the handler registration logic or when creating the CatalogHandlers.
openshift-merge-robot
added
the
needs-rebase
|
closing as stale |
grokspawn
added
the
lifecycle/frozen
grokspawn
force-pushed
the
catd-graphql-play
branch
from
346cf0d to
61a60be
Compare
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
openshift-merge-robot
removed
the
needs-rebase
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 18 out of 19 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
internal/catalogd/graphql/README.md
Outdated
| The GraphQL functionality is integrated into the `LocalDirV1` storage handler in `internal/catalogd/storage/localdir.go`: | ||
|
|
||
| - `handleV1GraphQL()`: Handles POST requests to the GraphQL endpoint | ||
| - `createCatalogFS()`: Creates filesystem interface for catalog data | ||
| - `buildCatalogGraphQLSchema()`: Builds dynamic GraphQL schema for specific catalogs |
| features: | ||
| enabled: | ||
| - APIV1MetasHandler | ||
| - GraphQLCatalogQueries |
grokspawn
force-pushed
the
catd-graphql-play
branch
from
2752f1f to
e358467
Compare
grokspawn
force-pushed
the
catd-graphql-play
branch
from
e358467 to
0d2d4e4
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 20 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (2)
internal/shared/util/tlsprofiles/mozilla_data.go:17
internal/shared/util/tlsprofiles/mozilla_data.goappears to have been truncated to ~17 lines (missing closing braces and theintermediateTLSProfile/oldTLSProfiledefinitions). This will not compile and also breaks TLS profile selection at runtime. Re-runhack/tools/update-tls-profiles.sh(and ensure the generated file is fully committed and gofmt’d) so all profiles are present and themodernTLSProfileliteral is complete.
import (
"crypto/tls"
)
var modernTLSProfile = tlsProfile{
ciphers: cipherSlice{
cipherNums: []uint16{
tls.TLS_AES_128_GCM_SHA256,
tls.TLS_AES_256_GCM_SHA384,
tls.TLS_CHACHA20_POLY1305_SHA256,
internal/catalogd/storage/localdir_test.go:252
- The new
/api/v1/graphqlHTTP route isn’t covered by the existing storage/server handler tests (e.g.,TestLocalDirServerHandlerandTestMetasEndpointcover/alland/metasonly). Add an HTTP-level test that exercises the GraphQL endpoint (method handling, invalid body/query, and a basic successful query) to prevent regressions in request parsing and wiring.
func TestLocalDirServerHandler(t *testing.T) {
store := NewLocalDirV1(t.TempDir(), &url.URL{Path: urlPrefix}, MetasHandlerDisabled, GraphQLQueriesDisabled)
if store.Store(context.Background(), "test-catalog", createTestFS(t)) != nil {
t.Fatal("failed to store test catalog and start server")
}
testServer := httptest.NewServer(store.StorageServerHandler())
defer testServer.Close()
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
openshift-ci
bot
added
the
needs-rebase
Signed-off-by: grokspawn <jordan@nimblewidget.com>
Signed-off-by: grokspawn <jordan@nimblewidget.com>
grokspawn
force-pushed
the
catd-graphql-play
branch
from
0d2d4e4 to
711c9d8
Compare
openshift-ci
bot
removed
the
needs-rebase
Signed-off-by: Jordan <jordan@nimblewidget.com>
grokspawn
force-pushed
the
catd-graphql-play
branch
from
711c9d8 to
888a0d6
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 19 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Convert metas to JSON objects and apply pagination | ||
| var results []interface{} | ||
| for i, meta := range metas { | ||
| if i < offset { | ||
| continue | ||
| } | ||
| if len(results) >= limit { | ||
| break | ||
| } | ||
|
|
||
| var obj map[string]interface{} | ||
| if err := json.Unmarshal(meta.Blob, &obj); err != nil { | ||
| continue // Skip malformed objects | ||
| } | ||
| results = append(results, obj) | ||
| } |
There was a problem hiding this comment.
This resolver unmarshals meta.Blob for every object on every GraphQL request, which will be a major hotspot for large catalogs and/or high QPS. Consider caching parsed objects alongside metasBySchema during schema build (e.g., []map[string]interface{} or []json.RawMessage pre-validated once), or at least memoizing per-catalog parsed results in the GraphQL service cache.
| // isValidCatalogName validates that a catalog name is safe for filesystem operations | ||
| // Prevents path traversal attacks by rejecting names with special characters | ||
| func isValidCatalogName(name string) bool { | ||
| if name == "" || name == "." || name == ".." { | ||
| return false | ||
| } | ||
| // Allow alphanumeric, hyphen, underscore, and dot (for DNS-like names) | ||
| // Reject path separators and other special characters | ||
| for _, r := range name { | ||
| if (r < 'a' || r > 'z') && | ||
| (r < 'A' || r > 'Z') && | ||
| (r < '0' || r > '9') && | ||
| r != '-' && r != '_' && r != '.' { | ||
| return false | ||
| } | ||
| } | ||
| // Additional check: no consecutive dots (prevents ../ escaping) | ||
| if strings.Contains(name, "..") { | ||
| return false | ||
| } |
There was a problem hiding this comment.
This introduces new server-side validation that may reject catalog names that were previously accepted (breaking existing clients depending on naming conventions). If the project already has a canonical catalog name format (e.g., DNS-1123 label/subdomain), it would be better to validate against that exact rule and document it; otherwise consider relaxing the character set while still preventing path traversal (e.g., explicitly reject path separators and .., rather than restricting to a narrow allowlist).
| // isValidCatalogName validates that a catalog name is safe for filesystem operations | |
| // Prevents path traversal attacks by rejecting names with special characters | |
| func isValidCatalogName(name string) bool { | |
| if name == "" || name == "." || name == ".." { | |
| return false | |
| } | |
| // Allow alphanumeric, hyphen, underscore, and dot (for DNS-like names) | |
| // Reject path separators and other special characters | |
| for _, r := range name { | |
| if (r < 'a' || r > 'z') && | |
| (r < 'A' || r > 'Z') && | |
| (r < '0' || r > '9') && | |
| r != '-' && r != '_' && r != '.' { | |
| return false | |
| } | |
| } | |
| // Additional check: no consecutive dots (prevents ../ escaping) | |
| if strings.Contains(name, "..") { | |
| return false | |
| } | |
| // isValidCatalogName validates that a catalog name is safe for filesystem operations. | |
| // Prevent path traversal without unnecessarily restricting otherwise valid catalog names. | |
| func isValidCatalogName(name string) bool { | |
| if name == "" || name == "." || name == ".." { | |
| return false | |
| } | |
| // Reject traversal patterns and path separators across platforms. | |
| if strings.Contains(name, "..") || | |
| strings.ContainsRune(name, '/') || | |
| strings.ContainsRune(name, '\\') || | |
| strings.ContainsRune(name, rune(os.PathSeparator)) { | |
| return false | |
| } | |
| // Reject control characters that can produce invalid or ambiguous filesystem names. | |
| for _, r := range name { | |
| if r < 0x20 || r == 0x7f { | |
| return false | |
| } | |
| } |
| func (h *CatalogHandlers) handleV1GraphQL(w http.ResponseWriter, r *http.Request) { | ||
| if r.Method != http.MethodPost { | ||
| http.Error(w, "Only POST is allowed", http.StatusMethodNotAllowed) | ||
| return | ||
| } |
There was a problem hiding this comment.
The new GraphQL endpoint adds non-trivial behavior (method gating, body size limit, JSON parsing, query validation, and store/service interactions) but there are no accompanying HTTP-handler tests in this diff. Please add unit tests covering at least: non-POST → 405, invalid JSON → 400, empty query → 400, oversized query/body → 400, invalid catalog name → 400, and a happy-path POST returning JSON.
|
|
||
| import ( | ||
| "crypto/tls" | ||
| "crypto/tls" |
There was a problem hiding this comment.
This import block is not gofmt-formatted (missing indentation and tabs). Please run gofmt on this file (or the module) to keep formatting consistent and avoid noisy diffs.
| "crypto/tls" | |
| "crypto/tls" |
| "github.com/operator-framework/operator-registry/alpha/declcfg" | ||
| ) | ||
|
|
||
| func TestDiscoverSchemaFromMetas(t *testing.T) { |
There was a problem hiding this comment.
There appears to be overlapping/duplicated coverage between graphql_test.go (e.g., TestDiscoverSchemaFromMetas) and discovery_test.go (e.g., TestDiscoverSchemaFromMetas_CoreLogic and remapping tests). Consolidating these into a single cohesive test file or table-driven tests would reduce maintenance cost and keep the test suite lean.
Description
Reviewer Checklist