Expose new transforms and formatters design

CHANGELOG.md

@@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+- New `formatter:` options for the IO target: `:json` and `:passthrough` (#25)
+- New `transform:` options for the IO target: `:cloud_watch` and `:passthrough` (#25)
+
+### Changed
+- A custom `formatter:` on the IO target now receives the transformed telemetry. Pass `transform: :passthrough` to get the same telemetry as before (#25)
+
+### Dropped
+- The `Targets::IOTarget::JSONFormatter` constant. Its behavior moved to `Formatters::JSONFormatter` and `Transforms::CloudWatchTransform` (#25)
+
 ### Fixed
 - Stop the telemetry runner when puma shuts down or when the target IO stream is closed. This prevents `IOError: closed stream` errors during shutdown (#31, #45)
 - Log target errors with `unknown_error` instead of `error`, so a failed publish does not make puma exit (#31, #45)

README.md

@@ -41,22 +41,35 @@ Puma::Plugin::Telemetry.configure do |config|
 end
 ```
 
-### Basic
+### Basic IO Target
 
-Output telemetry as JSON to `STDOUT`
+A basic I/O target will emit telemetry data to `STDOUT`, formatted in JSON.
 
 ```ruby
-  config.add_target :io
+config.add_target :io
 ```
 
+#### Options
+
+This target has configurable `formatter:` and `transform:` options.
+The `formatter:` options are
+
+* `:json` _(default)_ - Print the logs in JSON.
+* `:passthrough` - A passthrough formatter which returns the telemetry `Hash` unaltered, passing it directly to the `io:` instance.
+
+The `transform:` options are
+
+* `:cloud_watch` _(default)_ - Transforms telemetry keys, replacing dots with dashes to support AWS CloudWatch Log Metrics filters.
+* `:passthrough` -  A passthrough transform which returns the telemetry `Hash` unaltered.
+
 ### Datadog StatsD target
 
-Given gem provides built in target for Datadog StatsD client, that uses batch operation to publish metrics.
+A target for the Datadog StatsD client, that uses batch operation to publish metrics.
 
-**NOTE** Be sure to have `dogstatsd` gem installed.
+**NOTE** Be sure to have the `dogstatsd` gem installed.
 
 ```ruby
-  config.add_target :dogstatsd, client: Datadog::Statsd.new
+config.add_target :dogstatsd, client: Datadog::Statsd.new
 ```
 
 You can provide all the tags, namespaces, and other configuration options as always to `Datadog::Statsd.new` method.
@@ -73,7 +86,7 @@ Puma::Plugin::Telemetry.configure do |config|
   config.puma_telemetry = %w[workers.requests_count queue.backlog queue.capacity]
   config.socket_telemetry!
   config.socket_parser = :inspect
-  config.add_target :io, formatter: :json, io: StringIO.new
+  config.add_target :io, io: StringIO.new, formatter: :json, transform: :passthrough
   config.add_target :dogstatsd, client: Datadog::Statsd.new(tags: { env: ENV["RAILS_ENV"] })
 end
 ```
@@ -85,8 +98,8 @@ Target is a simple object that implements `call` methods that accepts `telemetry
 Just be mindful that if the API takes long to call, it will slow down frequency with which telemetry will get reported.
 
 ```ruby
-  # Example logfmt to stdout target
-  config.add_target proc { |telemetry| puts telemetry.map { |k, v| "#{k}=#{v.inspect}" }.join(" ") }
+  # Example key/value log to `STDOUT` target
+  config.add_target ->(telemetry) { puts telemetry.map { |k, v| "#{k}=#{v.inspect}" }.join(" ") }
 ```
 
 ## Extra middleware

lib/puma/plugin/telemetry/formatters/json_formatter.rb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+require 'json'
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        # JSON formatter, expects `call` method accepting telemetry hash
+        class JSONFormatter
+          def self.call(telemetry)
+            ::JSON.dump(telemetry)
+          end
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/formatters/passthrough_formatter.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        # A passthrough formatter - it returns the telemetry Hash it was given
+        class PassthroughFormatter
+          def self.call(telemetry)
+            telemetry
+          end
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/targets/base_formatting_target.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+require_relative '../formatters/json_formatter'
+require_relative '../formatters/passthrough_formatter'
+require_relative '../transforms/cloud_watch_transform'
+require_relative '../transforms/passthrough_transform'
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Targets
+        # A base class for other Targets concerned with formatting telemetry
+        class BaseFormattingTarget
+          def initialize(formatter: :json, transform: :cloud_watch)
+            @transform = case transform
+                         when :cloud_watch then Transforms::CloudWatchTransform
+                         when :passthrough then Transforms::PassthroughTransform
+                         else transform
+                         end
+            @formatter = case formatter
+                         when :json then Formatters::JSONFormatter
+                         when :passthrough then Formatters::PassthroughFormatter
+                         else formatter
+                         end
+          end
+
+          def call(_telemetry)
+            raise NotImplementedError, "#{__method__} must be implemented by #{self.class.name}"
+          end
+
+          private
+
+          attr_reader :formatter, :transform
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/targets/io_target.rb

@@ -1,40 +1,25 @@
 # frozen_string_literal: true
 
-require 'json'
+require_relative 'base_formatting_target'
 
 module Puma
   class Plugin
     module Telemetry
       module Targets
         # Simple IO Target, publishing metrics to STDOUT or logs
-        #
-        class IOTarget
-          # JSON formatter for IO, expects `call` method accepting telemetry hash
-          #
-          class JSONFormatter
-            # NOTE: Replace dots with dashes for better support of AWS CloudWatch
-            #       Log Metric filters, as they don't support dots in key names.
-            def self.call(telemetry)
-              log = telemetry.transform_keys { |k| k.tr('.', '-') }
-
-              log['name'] = 'Puma::Plugin::Telemetry'
-              log['message'] = 'Publish telemetry'
-
-              ::JSON.dump(log)
-            end
-          end
-
-          def initialize(io: $stdout, formatter: :json)
+        class IOTarget < BaseFormattingTarget
+          def initialize(io: $stdout, formatter: :json, transform: :cloud_watch)
+            super(formatter: formatter, transform: transform)
             @io = io
-            @formatter = case formatter
-                         when :json then JSONFormatter
-                         else formatter
-                         end
           end
 
           def call(telemetry)
-            @io.puts(@formatter.call(telemetry))
+            io.puts(formatter.call(transform.call(telemetry)))
           end
+
+          private
+
+          attr_reader :io
         end
       end
     end

lib/puma/plugin/telemetry/transforms/cloud_watch_transform.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Transforms
+        # Replace dots with dashes for better support of AWS CloudWatch Log
+        # Metric filters, as they don't support dots in key names.
+        # Expects `call` method accepting telemetry Hash
+        class CloudWatchTransform
+          def self.call(telemetry)
+            telemetry.transform_keys { |k| String(k).tr('.', '-') }.tap do |data|
+              data['name'] = 'Puma::Plugin::Telemetry'
+              data['message'] = 'Publish telemetry'
+            end
+          end
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/transforms/passthrough_transform.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Transforms
+        # A passthrough transform - it returns the telemetry Hash it was given
+        class PassthroughTransform
+          def self.call(telemetry)
+            telemetry
+          end
+        end
+      end
+    end
+  end
+end

spec/fixtures/sockets.rb

@@ -21,7 +21,7 @@
 
 Puma::Plugin::Telemetry.configure do |config|
   # Simple `key=value` formatter
-  config.add_target :io, formatter: ->(t) { t.map { |r| r.join('=') }.join(' ') }
+  config.add_target(:io, formatter: ->(t) { t.map { |r| r.join('=') }.join(' ') }, transform: :passthrough)
   config.frequency = 1
   config.enabled = true
 

spec/puma/plugin/telemetry/formatters/json_formatter_spec.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        RSpec.describe JSONFormatter do
+          subject(:formatter) { described_class }
+
+          it 'formats the telemetry as a JSON string' do
+            string = formatter.call('foo' => 'bar')
+
+            data = ::JSON.parse(string)
+            expect(data.fetch('foo')).to eq('bar')
+          end
+
+          it 'handles symbol keys' do
+            string = formatter.call(foo: 'bar')
+
+            data = ::JSON.parse(string)
+            expect(data.fetch('foo')).to eq('bar')
+          end
+        end
+      end
+    end
+  end
+end

spec/puma/plugin/telemetry/formatters/passthrough_formatter_spec.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        RSpec.describe PassthroughFormatter do
+          subject(:formatter) { described_class }
+
+          it 'returns the telemetry, unaltered' do
+            telemetry_data = { 'foo' => 'bar' }
+            formatted_data = formatter.call(telemetry_data)
+
+            expect(formatted_data).to eq(telemetry_data)
+          end
+        end
+      end
+    end
+  end
+end

spec/puma/plugin/telemetry/targets/io_target_spec.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Targets
+        RSpec.describe IOTarget do
+          subject(:target) { described_class.new(io: io, formatter: logfmt) }
+          let(:io) { StringIO.new }
+          let(:telemetry) { { foo: 'bar' } }
+          let(:logfmt) { ->(telemetry) { telemetry.map { |k, v| "#{k}=#{v}" }.join(' ') } }
+
+          it 'puts to the io object' do
+            target.call(telemetry)
+
+            expect(io.string).to include('foo=bar')
+          end
+
+          context 'with passthrough formatter and transform' do
+            subject(:target) { described_class.new(io: io, formatter: :passthrough, transform: :passthrough) }
+
+            it 'puts the telemetry unaltered' do
+              target.call(telemetry)
+
+              expect(io.string).to eq("#{telemetry}\n")
+            end
+          end
+        end
+      end
+    end
+  end
+end

spec/puma/plugin/telemetry/transforms/cloud_watch_transform_spec.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Transforms
+        RSpec.describe CloudWatchTransform do
+          subject(:transform) { described_class }
+
+          it 'replaces dots with dashes in keys' do
+            data = transform.call('the.foo' => 'the.bar')
+
+            expect(data.fetch('the-foo')).to eq('the.bar')
+          end
+
+          it 'handles symbol keys' do
+            data = transform.call(foo: 'bar')
+
+            expect(data.fetch('foo')).to eq('bar')
+          end
+        end
+      end
+    end
+  end
+end

spec/puma/plugin/telemetry/transforms/passthrough_transform_spec.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Transforms
+        RSpec.describe PassthroughTransform do
+          subject(:transform) { described_class }
+
+          it 'returns the telemetry, unaltered' do
+            telemetry_data = { 'foo' => 'bar' }
+            transformed_data = transform.call(telemetry_data)
+
+            expect(transformed_data).to eq(telemetry_data)
+          end
+        end
+      end
+    end
+  end
+end