feat: redact sensitive feed data in structured logs#903
Conversation
There was a problem hiding this comment.
Pull request overview
This PR tightens security around request-scoped structured logging by redacting sensitive feed tokens and replacing logged source URLs with sanitized metadata, while consolidating observability/security emission through a shared JSON logger (including rack-timeout).
Changes:
- Introduces
AppLogger,LogEvent, andLogSanitizerto centralize structured logging and sanitize sensitive fields. - Updates
ObservabilityandSecurityLoggerto emit through the shared structured logger. - Redacts
/api/v1/feeds/:tokenin request context and routes rack-timeout logs through the same JSON formatter.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| spec/html2rss/web/request_context_middleware_spec.rb | Adds coverage for redacting feed tokens in request context path. |
| spec/html2rss/web/log_sanitizer_spec.rb | New specs for path redaction, URL sanitization, and log formatting behavior. |
| app/web/telemetry/observability.rb | Switches observability emission to the shared LogEvent emitter. |
| app/web/telemetry/log_sanitizer.rb | Adds sanitizers for feed-token paths and URL fields in log details. |
| app/web/telemetry/log_event.rb | Introduces a shared emitter that merges request context + sanitized payload. |
| app/web/telemetry/app_logger.rb | Adds a shared JSON logger/formatter (JSON + logfmt parsing). |
| app/web/security/security_logger.rb | Routes security events through LogEvent and shared logger state. |
| app/web/request/request_context_middleware.rb | Redacts feed tokens when building request context. |
| app/web/boot/setup.rb | Wires rack-timeout logging to use the shared JSON logger. |
| def sanitize_path(path) | ||
| return if path.nil? | ||
|
|
||
| path_string = path.to_s | ||
| suffix = feed_suffix(path_string) | ||
| token_path = suffix ? path_string.delete_suffix(suffix) : path_string | ||
|
|
||
| token_path.gsub(FEED_TOKEN_ROUTE, "\\1[REDACTED]#{suffix}") | ||
| end |
There was a problem hiding this comment.
sanitize_path always strips a .json/.xml/.rss suffix before attempting the feed-token replacement. If the path ends with one of those suffixes but does not match the /api/v1/feeds/:token pattern, the method returns the suffix-stripped path, which will corrupt logged paths (e.g., /api/v1/health.json -> /api/v1/health). Consider matching the full feed-token route (including an optional suffix) and only redacting when that match succeeds, otherwise return the original path_string unchanged.
|
|
||
| RSpec.describe Html2rss::Web::LogSanitizer do | ||
| let(:io) { StringIO.new } | ||
| let(:logger) { Logger.new(io).tap { |log| log.formatter = Html2rss::Web::AppLogger.send(:method, :format_entry) } } |
There was a problem hiding this comment.
This spec sets the logger formatter via Html2rss::Web::AppLogger.send(:method, :format_entry), but format_entry is a private singleton method in AppLogger. method(:format_entry) typically raises NameError for private methods, so this can fail when running the spec. Prefer private_method(:format_entry) (or expose a small public helper on AppLogger intended for tests).
| let(:logger) { Logger.new(io).tap { |log| log.formatter = Html2rss::Web::AppLogger.send(:method, :format_entry) } } | |
| let(:logger) { Logger.new(io).tap { |log| log.formatter = Html2rss::Web::AppLogger.send(:private_method, :format_entry) } } |
| # @param url [String] | ||
| # @return [Hash{Symbol=>String}] | ||
| def sanitized_url(host, url) | ||
| { host:, scheme: 'https', hash: url_hash(url) } | ||
| end |
There was a problem hiding this comment.
The helper def sanitized_url(host, url) defined later in this spec overrides the earlier let(:sanitized_url) helper method. After this definition, any call to sanitized_url without arguments (e.g. eq(url: sanitized_url)) will raise an ArgumentError. Rename one of these helpers (e.g., expected_sanitized_url for the let, or build_sanitized_url for the helper) to avoid the method name collision.
Summary
Verification
Notes