add support for automated splunk update when new version is available - #97
add support for automated splunk update when new version is available#97asifulhaque786 wants to merge 1 commit into
Conversation
bf4b4f3 to
e4391f5
Compare
b4d3c34 to
cf3bc00
Compare
Adds an opt-in auto-upgrade path for the Splunk Universal Forwarder: leaving version/build unset tracks the latest release via a new splunk_forwarder_latest_version() function (scraped from Splunk's download page, cached 6h, with stale-cache/hardcoded fallbacks so a lookup failure never breaks catalog compilation). Pinning version requires build to match, enforced by a fail-fast check. The pin-vs- fetch decision lives in the function itself - it takes version/build as required (but nullable) params and returns them as-is when both are given, only falling through to the cache/fetch path when neither is. Upgrades/downgrades are applied via package_ensure => 'latest': both the dpkg and rpm providers read the target version directly from the already-staged source file and, if it differs from what's installed, apply it in place via `dpkg -i`/`rpm -U --oldpackage` - which handles both directions natively and leaves etc/system/local/* (including seeded admin credentials) untouched, since it isn't a packaged conffile. Also extends the accept-tos exec's timeout from Puppet's default 300s to 900s via a resource collector override (a fresh install's first start can take longer than that), and makes Service resources respect noop_value like the rest of this profile's resources.
a507961 to
9ae73a0
Compare
| return { "version" => version, "build" => build } if version && build | ||
|
|
||
| cached = read_cache | ||
| return cached if cached && !stale?(cached) |
There was a problem hiding this comment.
P1 — this returns a value that violates the return_type declared on line 33, breaking every catalog on the second compile.
write_cache persists data.merge("fetched_at" => ...), and read_cache is a plain JSON.parse, so cached is {"version"=>..., "build"=>..., "fetched_at"=>...}. Puppet's Struct is strict — unlisted keys are rejected.
So: first compile fetches, returns a clean 2-key hash, passes. Every compile in the next 6h hits this line and dies with a return-type mismatch. cached || FALLBACK on line 48 has the same defect.
Stripping on read rather than on write keeps the timestamp available for stale?:
return cached.slice("version", "build") if cached && !stale?(cached)
...
rescue StandardError => e
Puppet.warning(...)
cached ? cached.slice("version", "build") : FALLBACK
end| # compilation for every node using this class. | ||
| # | ||
| Puppet::Functions.create_function(:splunk_forwarder_latest_version) do | ||
| DOWNLOAD_PAGE = "https://www.splunk.com/en_us/download/universal-forwarder.html" |
There was a problem hiding this comment.
These constants leak onto top-level Object.
Ruby resolves constant assignment lexically, and a block is not a constant scope — so inside create_function do ... end these define ::DOWNLOAD_PAGE, ::CACHE_PATH, ::CACHE_TTL_SECONDS and ::FALLBACK on Object in the puppetserver JRuby.
Two consequences: a collision with any other function that picks the same identifier, and warning: already initialized constant on every environment reload.
Private methods returning the literals (def cache_path = "/opt/..."), or nesting them under a real module, both avoid it.
|
|
||
| def write_cache(data) | ||
| FileUtils.mkdir_p(File.dirname(CACHE_PATH)) | ||
| tmp_path = "#{CACHE_PATH}.tmp" |
There was a problem hiding this comment.
Fixed temp filename races under concurrent compiles.
A puppetserver compiles many catalogs in parallel. When the TTL expires, every in-flight thread fetches and writes this same path, then renames it into place — so the cache can land truncated or spliced.
read_cache rescues the parse error and returns nil, so it degrades to "refetch on every compile" rather than a hard failure. But that's a live HTTP call to splunk.com per catalog, which is exactly what the cache exists to prevent.
tmp_path = "#{cache_path}.#{Process.pid}.#{Thread.current.object_id}.tmp"plus an ensure block to unlink it on failure.
| Eit_types::Noop_Value $noop_value = undef, | ||
| Hash[String[1], Hash] $addons = {}, | ||
| String[1] $password_hash, | ||
| Optional[Eit_types::Version] $version = undef, |
There was a problem hiding this comment.
Dropping the 7.2.4 / 8a94541dcfac defaults is the highest-blast-radius part of this PR.
grep over data/ and hieradata/ finds no splunk version/build pin anywhere, so every node with manage => true is currently on 7.2.4 by way of these defaults. After this merge the same node resolves whatever splunk.com advertises (10.4.2 today) and package_ensure => 'latest' applies it on the next run — a 3-major-version jump on the agent that ships customer logs, unattended, keyed off a page scrape.
The machinery is right; it's the default I'd flip. Keeping the pin as the default and making tracking explicit (version => 'latest', or a track_latest boolean) gets the same capability without the fleet moving on its own.
| # | ||
| Puppet::Functions.create_function(:splunk_forwarder_latest_version) do | ||
| DOWNLOAD_PAGE = "https://www.splunk.com/en_us/download/universal-forwarder.html" | ||
| CACHE_PATH = "/opt/obmondo/cache/splunk_forwarder_latest_version.json" |
There was a problem hiding this comment.
/opt/obmondo/cache is referenced nowhere else in the repo — nothing creates or manages it.
FileUtils.mkdir_p runs as the puppetserver user, so if /opt/obmondo isn't writable by it, write_cache warns and every compile refetches. Same end state as the tmp-path race: a live HTTP call per catalog, with open_timeout: 5, read_timeout: 10 bounding one request but not the aggregate.
Puppet[:vardir] is writable by the compiling process by construction and would remove the failure mode. A short negative-result cache would also bound the worst case when Splunk's page is slow rather than down.
… https://gitea.obmondo.com/EnableIT/abbnoa6nlk/issues/2554