HMS-9952: Add NewlyAddedErrata job

[swadeley]

Summary

HMS-9952 add job to send update events for all templates with added advisories

Testing steps

snapshot a repo
make a template
make kafka-topic-consume KAFKA_TOPIC=platform.content-sources.template # in term 1
go run cmd/jobs/main.go newly-added-errata # in term 2

[xbhouse]

https://issues.redhat.com/browse/HMS-9952

[rverdile]

Bryttanie's PR (https://github.com/content-services/content-sources-backend/pull/1536/changes) adds calculating advisory changes. I wonder if this would just need to enqueue update-template-content jobs to trigger events for updates. What do you think?

[swadeley]

Bryttanie's PR (https://github.com/content-services/content-sources-backend/pull/1536/changes) adds calculating advisory changes. I wonder if this would just need to enqueue update-template-content jobs to trigger events for updates. What do you think?

Hi, yes.
I could refactor now to add InternalOnlyListSnapshotErrataIDs or rebase to use it after it is merged.

EDIT: I will pull in InternalOnlyListSnapshotErrataIDs now rather than wait for that PR to be merged.

[rverdile]

@swadeley I'm a little confused about what the ticket is asking for. Is this understanding correct?

This job meant to run before the changes in Bryttanie's PR start sending template update events, meaning it populates the initial errata for the template, and then Bryttanie's PR starts calculating the diff.

[swadeley]

@swadeley I'm a little confused about what the ticket is asking for. Is this understanding correct?

This job meant to run before the changes in Bryttanie's PR start sending template update events, meaning it populates the initial errata for the template, and then Bryttanie's PR starts calculating the diff.

Hi @rverdile , yes, Justin confirmed, my job is a standalone backfill job that will be run when Patch is ready to accept and make use of the updates Bryttanie's PR code is sending.

[xbhouse]

@swadeley I'm a little confused about what the ticket is asking for. Is this understanding correct?

This job meant to run before the changes in Bryttanie's PR start sending template update events, meaning it populates the initial errata for the template, and then Bryttanie's PR starts calculating the diff.

sorry for the confusion @rverdile! as @swadeley said, Justin confirmed this job is meant to be run after my PR is merged and after patch can store these relationships, so the ticket to run this job should be done last. i've updated the dependencies for each ticket so the order is clearer

[rverdile]

Thanks guys, I'm still a little confused where this sits in context with everything else. What is the reason for the backfill here? Are the updates added in @xbhouse's PR already being sent before this job runs? I noticed that PR has a feature flag - does that stay off until this job runs?

[swadeley]

Thanks guys, I'm still a little confused where this sits in context with everything else. What is the reason for the backfill here? Are the updates added in @xbhouse's PR already being sent before this job runs? I noticed that PR has a feature flag - does that stay off until this job runs?

Hi @rverdile , code in PR# 1536 can be merged and send updates and but nothing will be done with them until feature is turned on.

Feature will not be turned on until:

1. Patch is updated to accept and hold the data coming from content-sources
2. My backfill job is run

[xbhouse]

Thanks guys, I'm still a little confused where this sits in context with everything else. What is the reason for the backfill here? Are the updates added in @xbhouse's PR already being sent before this job runs? I noticed that PR has a feature flag - does that stay off until this job runs?

the reason for this job is to let patch know about the advisories for all existing templates. the diff updates from my PR will already be sending whenever a template is updated, but that wouldn't include the full set.

regarding the flag: patch should just drop message fields if it doesn't understand them, Michael suggested adding a feature flag though just in case. so we'd turn the flag on first, then run this job.

[swadeley]

Would it be better to do:

if err != nil {
    log.Error().Err(err).Msg("failed to fetch templates")
    return
}

[swadeley]

I think paginate through the templates here?