feat: support cr for k8s entity generator#2542
Open
StartE wants to merge 21 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
该 PR 为 LoongCollector 的 Kubernetes Meta v2 增加 按配置采集第三方自定义资源(CR) 的能力:通过 dynamic informer 采集 unstructured CR,对外导出对应 实体(entity),并可选生成 Pod↔CR、Namespace↔CR 的关联(link),同时补充 informer 的 watch 错误 give-up 与 cache sync 逻辑,以及配套文档与单测。
Changes:
- 在
service_kubernetes_meta配置中新增CustomResources,并在 meta collector 中注册 CR 实体与链路事件处理与发送。 - 在
pkg/helper/k8smeta引入 CR 采集配置类型、crUnifiedCache(dynamic informer)、以及 Pod→CR / CR→Namespace 的 link 生成。 - 新增/更新单测与中文文档,补充 CustomResources 的字段、RBAC 与故障行为说明。
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| plugins/input/kubernetesmetav2/service_meta.go | 新增 CustomResources 配置入口与解析辅助方法 |
| plugins/input/kubernetesmetav2/meta_collector.go | 启动流程中注册/启动 CR collector,并注册对应 send/link 处理 |
| plugins/input/kubernetesmetav2/meta_collector_cr.go | 新增 CR 实体与 CR 链路(Pod/Namespace)事件转 pipeline event |
| plugins/input/kubernetesmetav2/meta_collector_cr_test.go | 覆盖 CR 配置归一化、去重、以及 CR link/entity 处理逻辑 |
| pkg/helper/k8smeta/k8s_meta_manager.go | MetaManager 支持注册 CR cache、保存 rest.Config、并发读写 cacheMap 加锁 |
| pkg/helper/k8smeta/k8s_meta_link.go | LinkGenerator 增加 Pod→CR 与 CR→Namespace 链路生成 |
| pkg/helper/k8smeta/k8s_meta_link_test.go | 覆盖 Pod→CR(ownerRef/label fallback)与 CR→Namespace link 生成 |
| pkg/helper/k8smeta/k8s_meta_informer_lifecycle.go | 新增 informer 生命周期辅助:give-up stop 合并、watch error 处理、cache sync |
| pkg/helper/k8smeta/k8s_meta_informer_giveup.go | 定义“应 give-up”的错误分类与阈值 |
| pkg/helper/k8smeta/k8s_meta_custom_resource.go | CustomResources 配置结构、Normalize、GVR/LinkType 辅助函数 |
| pkg/helper/k8smeta/k8s_meta_cr_unified_cache.go | dynamic informer CR cache 实现(unstructured + trimmed copy) |
| pkg/helper/k8smeta/k8s_meta_cache.go | 内置 typed informer 引入 give-up handler 与 cache sync helper |
| pkg/helper/k8smeta/k8s_meta_const.go | 新增 PodCustomResource / NamespaceCustomResource 结构定义 |
| go.mod | 将 controller-runtime 作为直接依赖(用于 config/apiutil) |
| docs/cn/plugins/input/extended/service-kubernetesmeta-v2.md | 文档补充 CustomResources、RBAC、行为说明与示例 |
linrunqi08
requested changes
Apr 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
K8s Meta v2:支持 CR 动态采集与实体/关联导出,并增强 Informer 错误与缓存同步处理。
背景 / 要解决什么
LoongCollector 的 Kubernetes Meta v2 原先主要面向 内置资源(如 Pod、Service)。在实际场景里,很多工作负载信息在 自定义资源(CR) 上(例如 Argo Workflow),需要和 Pod、Namespace 等一起做 实体(entity) 和 关联(link) 采集,便于检索与关联分析。
本 PR 在 不破坏现有内置资源行为 的前提下,增加 按配置采集指定 GVR 的 CR,并支持可选的 Pod↔CR、Namespace↔CR 关联导出。
行为变化
unstructured),与现有 Clientset + Typed Informer 并存;Dynamic 侧使用 JSON 内容协商(与 Clientset 可能使用的 protobuf 区分)。架构
MetaManager 在 Init 时注入
rest.Config;内置资源走原有 cache;CR 走统一 dynamic informer cache,事件进入同一套 link / entity 管线,由 kubernetesmetav2 插件按配置转成 pipeline 事件。主要改动
pkg/helper/k8smeta/cr_unified_cache、与k8s_meta_link/k8s_meta_manager集成;k8s_meta_informer_*处理 watch 错误与 cache syncplugins/input/kubernetesmetav2/meta_collector增加 CR 实体与 Namespace→CR 等导出逻辑;service_meta接配置docs/cn/plugins/input/extended/service-kubernetesmeta-v2.md功能使用示例
CR资源使用示例(1) 采集argoWorkflow:
CR资源使用示例(2)采集Sandbox: