Add L2Met Transform

CHANGELOG.md

@@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+- New `LogTarget` for logging metrics to any `::Logger` compatible logger
+- New `formatter:` option `:logfmt`.
+- New `transform:` option `:l2met`.
+
 ## [1.2.0]
 
 ### Added
@@ -43,7 +48,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Check for support for 'ubuntu-20.04'
 - Check for support for Ruby 2.6 and 2.7
 
-
 ## [1.1.4]
 
 ### Changed

README.md

@@ -46,7 +46,7 @@ end
 A basic I/O target will emit telemetry data to `STDOUT`, formatted in JSON.
 
 ```ruby
-config.add_target :io
+config.add_target(:io)
 ```
 
 #### Options
@@ -55,13 +55,39 @@ This target has configurable `formatter:` and `transform:` options.
 The `formatter:` options are
 
 * `:json` _(default)_ - Print the logs in JSON.
+* `:logfmt` - Print the logs in key/value pairs, as per `logfmt`.
 * `:passthrough` - A passthrough formatter which returns the telemetry `Hash` unaltered, passing it directly to the `io:` instance.
 
 The `transform:` options are
 
 * `:cloud_watch` _(default)_ - Transforms telemetry keys, replacing dots with dashes to support AWS CloudWatch Log Metrics filters.
+* `:l2met` - Transforms telemetry keys, prepending `sample#` for [L2Met][l2met] consumption.
 * `:passthrough` -  A passthrough transform which returns the telemetry `Hash` unaltered.
 
+### Log target
+
+While emitting to `STDOUT` via the basic `IOTarget` can work for getting telemetry into logs, we also provide an explicit `LogTarget`.
+This target will defaults to emitting telemetry at the `INFO` log level via a [standard library `::Logger`][logger] instance.
+That default logger will print to `STDOUT` in [the `logfmt` format][logfmt].
+
+```ruby
+config.add_target(:log)
+```
+
+You can pass an explicit `logger:` option if you wanted to, for example, use the same logger as Rails.
+
+```ruby
+config.add_target(:log, logger: Rails.logger)
+```
+
+This target also has configurable `formatter:` and `transform:` options.
+The [possible options are the same as for the `IOTarget`](#options), but the defaults are different.
+The `LogTarget` defaults to `formatter: :logfmt`, and `transform: :passthrough`.
+
+[l2met]:  https://github.com/ryandotsmith/l2met "l2met - Logs to metrics"
+[logfmt]: https://brandur.org/logfmt "logfmt - Structured log format"
+[logger]: https://rubyapi.org/o/logger "Ruby's Logger, from the stdlib"
+
 ### Datadog StatsD target
 
 A target for the Datadog StatsD client, that uses batch operation to publish metrics.
@@ -73,7 +99,7 @@ gem "dogstatsd-ruby"
 ```
 
 ```ruby
-config.add_target :dogstatsd, client: Datadog::Statsd.new
+config.add_target(:dogstatsd, client: Datadog::Statsd.new)
 ```
 
 You can provide all the tags, namespaces, and other configuration options as always to `Datadog::Statsd.new` method.

lib/puma/plugin/telemetry.rb

@@ -7,6 +7,7 @@
 require 'puma/plugin/telemetry/data'
 require 'puma/plugin/telemetry/targets/datadog_statsd_target'
 require 'puma/plugin/telemetry/targets/io_target'
+require 'puma/plugin/telemetry/targets/log_target'
 require 'puma/plugin/telemetry/targets/open_telemetry_target'
 require 'puma/plugin/telemetry/config'
 

lib/puma/plugin/telemetry/config.rb

@@ -35,6 +35,7 @@ class Config
         TARGETS = {
           dogstatsd: Telemetry::Targets::DatadogStatsdTarget,
           io: Telemetry::Targets::IOTarget,
+          log: Telemetry::Targets::LogTarget,
           open_telemetry: Telemetry::Targets::OpenTelemetryTarget
         }.freeze
 

lib/puma/plugin/telemetry/formatters/logfmt_formatter.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        # Logfmt formatter, expects `call` method accepting telemetry hash
+        class LogfmtFormatter
+          def self.call(telemetry)
+            telemetry.map { |k, v| "#{String(k)}=#{v.inspect}" }.join(' ')
+          end
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/targets/base_formatting_target.rb

@@ -1,8 +1,10 @@
 # frozen_string_literal: true
 
 require_relative '../formatters/json_formatter'
+require_relative '../formatters/logfmt_formatter'
 require_relative '../formatters/passthrough_formatter'
 require_relative '../transforms/cloud_watch_transform'
+require_relative '../transforms/l2met_transform'
 require_relative '../transforms/passthrough_transform'
 
 module Puma
@@ -12,16 +14,8 @@ module Targets
         # A base class for other Targets concerned with formatting telemetry
         class BaseFormattingTarget
           def initialize(formatter: :json, transform: :cloud_watch)
-            @transform = case transform
-                         when :cloud_watch then Transforms::CloudWatchTransform
-                         when :passthrough then Transforms::PassthroughTransform
-                         else transform
-                         end
-            @formatter = case formatter
-                         when :json then Formatters::JSONFormatter
-                         when :passthrough then Formatters::PassthroughFormatter
-                         else formatter
-                         end
+            @formatter = FORMATTERS.fetch(formatter) { formatter }
+            @transform = TRANSFORMS.fetch(transform) { transform }
           end
 
           def call(_telemetry)
@@ -31,6 +25,20 @@ def call(_telemetry)
           private
 
           attr_reader :formatter, :transform
+
+          FORMATTERS = {
+            json: Formatters::JSONFormatter,
+            logfmt: Formatters::LogfmtFormatter,
+            passthrough: Formatters::PassthroughFormatter
+          }.freeze
+          private_constant :FORMATTERS
+
+          TRANSFORMS = {
+            cloud_watch: Transforms::CloudWatchTransform,
+            l2met: Transforms::L2metTransform,
+            passthrough: Transforms::PassthroughTransform
+          }.freeze
+          private_constant :TRANSFORMS
         end
       end
     end

lib/puma/plugin/telemetry/targets/log_target.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+require 'logger'
+require_relative 'base_formatting_target'
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Targets
+        # Simple Log Target, publishing metrics to a Ruby ::Logger at stdout
+        # at the INFO log level
+        class LogTarget < BaseFormattingTarget
+          def initialize(logger: ::Logger.new($stdout), formatter: :logfmt, transform: :passthrough)
+            super(formatter: formatter, transform: transform)
+            @logger = logger
+          end
+
+          def call(telemetry)
+            logger.info(formatter.call(transform.call(telemetry)))
+          end
+
+          private
+
+          attr_reader :logger
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/transforms/l2met_transform.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+require 'English'
+require 'pathname'
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Transforms
+        # L2Met (Logs to Metrics) transform that makes all keys a `sample#` in the L2Met format.
+        class L2metTransform
+          def self.call(telemetry)
+            new.call(telemetry)
+          end
+
+          def initialize(host_env: ENV, program_name: $PROGRAM_NAME, socket: Socket)
+            @host_env = host_env
+            @program_name = program_name
+            @socket = socket
+          end
+
+          def call(telemetry)
+            telemetry.transform_keys { |k| "sample##{k}" }.tap do |data|
+              data['name'] ||= 'Puma::Plugin::Telemetry'
+              data['source'] ||= source
+            end
+          end
+
+          private
+
+          attr_reader :host_env, :program_name, :socket
+
+          def source
+            @source ||= host_env['L2MET_SOURCE'] ||
+                        host_env['DYNO'] || # For Heroku
+                        host_with_exe_name # Last-ditch effort
+          end
+
+          def host_with_exe_name
+            "#{socket.gethostname}/#{Pathname(program_name).basename}"
+          end
+        end
+      end
+    end
+  end
+end

spec/puma/plugin/telemetry/formatters/logfmt_formatter_spec.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        RSpec.describe LogfmtFormatter do
+          subject(:formatter) { described_class }
+
+          it 'formats the telemetry in key/value pairs' do
+            string = formatter.call('foo' => 'bar', 'count' => 2)
+
+            expect(string).to include('foo="bar"').and(include('count=2'))
+          end
+
+          it 'handles symbol keys' do
+            string = formatter.call(foo: 'bar')
+
+            expect(string).to include('foo="bar"')
+          end
+        end
+      end
+    end
+  end
+end

spec/puma/plugin/telemetry/targets/log_target_spec.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Targets
+        RSpec.describe LogTarget do
+          subject(:target) { described_class.new(logger: logger, formatter: kvpair) }
+          let(:logger) { ::Logger.new(io) }
+          let(:io) { StringIO.new }
+          let(:telemetry) { { foo: 'bar' } }
+          let(:kvpair) { ->(telemetry) { telemetry.map { |k, v| "#{k}=#{v}" }.join(' ') } }
+
+          it 'logs the telemetry at the INFO level' do
+            target.call(telemetry)
+
+            expect(io.string).to include('INFO').and(include('foo=bar'))
+          end
+        end
+      end
+    end
+  end
+end

spec/puma/plugin/telemetry/transforms/l2met_transform_spec.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Transforms
+        RSpec.describe L2metTransform do
+          subject(:transform) { described_class.new(host_env: fake_env, socket: fake_socket) }
+
+          let(:fake_env) { {} }
+          let(:fake_socket) { class_double('Socket', gethostname: 'GIBSON') }
+
+          it 'transforms the telemetry key to an L2Met sample' do
+            data = transform.call('widgets.size' => 2, 'speed' => 10.5)
+
+            expect(data).to include('sample#widgets.size' => 2, 'sample#speed' => 10.5)
+          end
+
+          it 'handles symbol keys' do
+            data = transform.call('queue.depth': 2)
+
+            expect(data).to include('sample#queue.depth' => 2)
+          end
+
+          it 'adds source from L2MET_SOURCE in ENV' do
+            fake_env['L2MET_SOURCE'] = 'some-machine'
+
+            data = transform.call('queue.depth' => 2)
+
+            expect(data).to include('source' => 'some-machine')
+          end
+
+          it 'adds source from DYNO in ENV' do
+            fake_env['DYNO'] = 'web.10'
+
+            data = transform.call('queue.depth' => 2)
+
+            expect(data).to include('source' => 'web.10')
+          end
+
+          it 'adds source from $PROGRAM_NAME when L2MET_SOURCE nor DYNO are in ENV' do
+            data = transform.call('queue.depth' => 2)
+
+            expect(data).to include('source' => 'GIBSON/rspec')
+          end
+        end
+      end
+    end
+  end
+end