Use Case
Ingesting Telegraf metrics into Snowflake for analytics, dashboarding, and long-term storage. The existing SQL output plugin can connect to Snowflake via gosnowflake, but it is a generic SQL driver with no awareness of Snowflake-specific optimizations: no array binding for batch inserts, no automatic table creation or schema evolution, no Snowpipe Streaming integration, and no template-based table routing.
A dedicated Snowpipe Streaming plugin provides production-grade Snowflake ingestion with efficient batch inserts (array binding), key-pair authentication, automatic DDL, template-based table routing (e.g. metrics_{{.Name}}), retry with exponential backoff, and NaN/Inf sanitization.
Expected behavior
A dedicated outputs.snowpipe_streaming plugin that:
- Authenticates via RSA key-pair (Snowflake best practice for service accounts)
- Routes metrics to tables using Go templates based on metric name
- Batches inserts efficiently using gosnowflake array binding
- Automatically creates tables and evolves schema as new fields appear
- Retries transient errors with exponential backoff
- Sanitizes NaN/Inf values to NULL
- Caches schema with configurable TTL to minimize DESCRIBE TABLE calls
Actual behavior
No dedicated Snowflake output plugin exists. Users must use the generic SQL output plugin which lacks batch optimization, schema management, table routing, and Snowpipe Streaming awareness. This results in suboptimal insert performance and requires manual table creation and schema management.
Additional info
A PR implementing this plugin is available: #18721
Use Case
Ingesting Telegraf metrics into Snowflake for analytics, dashboarding, and long-term storage. The existing SQL output plugin can connect to Snowflake via gosnowflake, but it is a generic SQL driver with no awareness of Snowflake-specific optimizations: no array binding for batch inserts, no automatic table creation or schema evolution, no Snowpipe Streaming integration, and no template-based table routing.
A dedicated Snowpipe Streaming plugin provides production-grade Snowflake ingestion with efficient batch inserts (array binding), key-pair authentication, automatic DDL, template-based table routing (e.g.
metrics_{{.Name}}), retry with exponential backoff, and NaN/Inf sanitization.Expected behavior
A dedicated
outputs.snowpipe_streamingplugin that:Actual behavior
No dedicated Snowflake output plugin exists. Users must use the generic SQL output plugin which lacks batch optimization, schema management, table routing, and Snowpipe Streaming awareness. This results in suboptimal insert performance and requires manual table creation and schema management.
Additional info
A PR implementing this plugin is available: #18721