fix(terraform): drop the unencrypted-upload opt-out and reject collector + managed scan together

.github/workflows/terraform-validate.yml

@@ -34,7 +34,10 @@ jobs:
       matrix:
         # AWS rows run the declared floor (Terraform 1.8, the version the
         # terraform-test job also runs); examples that use ephemeral resources
-        # (v6 multi-cluster, v6 managed-scan) need 1.10.
+        # (v6 multi-cluster, v6 managed-scan) need 1.10. Every modules/ directory
+        # is listed so a module added without a test suite is still parsed and
+        # fmt-checked; the validation blocks and preconditions themselves are
+        # only reached by the terraform-test job below, which supplies values.
         include:
           - directory: gcp-integration-setup/terraform
             terraform: 1.9.5
@@ -54,6 +57,10 @@ jobs:
             terraform: 1.8.5
           - directory: aws-integration-setup/terraform/modules/eks-managed-scan-access
             terraform: 1.8.5
+          - directory: aws-integration-setup/terraform/modules/k8s-resources
+            terraform: 1.8.5
+          - directory: aws-integration-setup/terraform/modules/nullify-aws-integration
+            terraform: 1.8.5
           - directory: aws-integration-setup/terraform-v6
             terraform: 1.8.5
           - directory: aws-integration-setup/terraform-v6/examples/basic
@@ -64,6 +71,10 @@ jobs:
             terraform: 1.10.5
           - directory: aws-integration-setup/terraform-v6/modules/eks-managed-scan-access
             terraform: 1.8.5
+          - directory: aws-integration-setup/terraform-v6/modules/k8s-resources
+            terraform: 1.8.5
+          - directory: aws-integration-setup/terraform-v6/modules/nullify-aws-integration
+            terraform: 1.8.5
           - directory: azure-integration-setup/terraform
             terraform: 1.9.5
           - directory: azure-integration-setup/terraform/examples/management-group

aws-integration-setup/terraform-v6/README.md

@@ -114,7 +114,7 @@ The default `authorization = "rbac"` needs the list-only `nullify-readonly` Clus
 - `tags`: Resource tags
 - `cronjob_schedule`: Deprecated and unused here; set the schedule on `k8s-resources`
 
-`k8s-resources` requires `cluster_name` and `kms_key_arn` whenever `enable_collector` is true. `cluster_name` must match your actual cluster name exactly, must match a cluster registered on the Nullify configure page, and must differ per cluster: the collector uploads to `<prefix>/k8s-collector/<cluster_name>-data.json`, and Nullify joins that upload on the name. See "Kubernetes Resources" and "KMS" in `../terraform/README.md`.
+`k8s-resources` requires `cluster_name` and `kms_key_arn` whenever `enable_collector` is true, with no unencrypted opt-out, and rejects `enable_collector` together with `enable_managed_scan_rbac`. `cluster_name` must match your actual cluster name exactly, must match a cluster registered on the Nullify configure page, and must differ per cluster: the collector uploads to `<prefix>/k8s-collector/<cluster_name>-data.json`, and Nullify joins that upload on the name. See "Kubernetes Resources" and "KMS" in `../terraform/README.md`.
 
 `k8s-resources` defaults `collector_image` to `public.ecr.aws/w4o2j2x4/integrations:k8s-collector-3.46.0`, the same build as the `k8s-collector-latest` tag the Helm chart deploys. Replace any explicit `nullify/k8s-collector:latest`: Nullify does not publish that Docker Hub image.
 

aws-integration-setup/terraform-v6/examples/multi-cluster-complete/terraform.tfvars.example

@@ -34,8 +34,10 @@ aws_region = "ap-southeast-2"
 # nullify_s3_access_point_arn = "arn:aws:s3:ap-southeast-2:123456789012:accesspoint/acme-corp-k8s-collector"
 
 # KMS ARN from the Nullify configure page. A key ARN or an alias ARN both work.
-# REQUIRED when scan_mode is collector: the collector refuses to upload without KMS encryption.
-kms_key_arn = "arn:aws:kms:ap-southeast-2:123456789012:key/12345678-1234-1234-1234-123456789012"
+# REQUIRED when scan_mode is collector: the collector refuses to upload without KMS
+# encryption, and there is no opt-out. The key lives in Nullify's account, not yours,
+# so it is not the account id used for nullify_role_arn or eks_cluster_arns above.
+# kms_key_arn = "arn:aws:kms:<nullify-region>:<nullify-account-id>:key/<from-the-configure-page>"
 
 # Kubernetes namespace for Nullify resources
 kubernetes_namespace = "nullify"

aws-integration-setup/terraform-v6/modules/k8s-resources/main.tf

@@ -166,8 +166,8 @@ resource "kubernetes_cron_job_v1" "k8s_collector" {
     }
 
     precondition {
-      condition     = var.kms_key_arn != "" || var.allow_unencrypted_upload
-      error_message = "kms_key_arn is required when enable_collector is true: the collector walks the cluster, then refuses to upload without KMS encryption and exits 1, so every CronJob run fails and Nullify receives nothing. Take the value from the Nullify configure page. Set allow_unencrypted_upload = true only if Nullify issued you no key."
+      condition     = var.kms_key_arn != ""
+      error_message = "kms_key_arn is required when enable_collector is true: the collector walks the cluster, then refuses to upload without KMS encryption and exits 1, so every CronJob run fails and Nullify receives nothing. There is no opt-out. Nullify's upload bucket sets SSE-KMS as its default encryption, so an upload sent with no encryption headers is still encrypted with Nullify's key, and S3 rejects it unless the caller holds kms:GenerateDataKey on that key -- which this role only gets from kms_key_arn. Take the value from the Nullify configure page."
     }
   }
 
@@ -232,14 +232,6 @@ resource "kubernetes_cron_job_v1" "k8s_collector" {
                 }
               }
 
-              dynamic "env" {
-                for_each = var.kms_key_arn == "" && var.allow_unencrypted_upload ? [1] : []
-                content {
-                  name  = "ALLOW_UNENCRYPTED_UPLOAD"
-                  value = "true"
-                }
-              }
-
               dynamic "env" {
                 for_each = var.enable_debug ? [1] : []
                 content {

aws-integration-setup/terraform-v6/modules/k8s-resources/managed_scan.tf

@@ -32,6 +32,13 @@ locals {
 resource "kubernetes_cluster_role_v1" "nullify_readonly" {
   count = var.enable_managed_scan_rbac ? 1 : 0
 
+  lifecycle {
+    precondition {
+      condition     = !var.enable_collector
+      error_message = "enable_collector and enable_managed_scan_rbac cannot both be true for one cluster. The collector's upload is registered as an on-prem cluster keyed on cluster_name; the managed scan registers the same cluster as its EKS ARN. Nothing joins the two, so the cluster appears twice in inventory with its pods and containers duplicated. Pick one mode: apply it, then ask Nullify to remove the old cluster registration."
+    }
+  }
+
   metadata {
     name = var.managed_scan_cluster_role_name
     labels = {

aws-integration-setup/terraform-v6/modules/k8s-resources/tests/k8s_resources.tftest.hcl

@@ -17,7 +17,7 @@ run "collector_only_is_the_default" {
 
   assert {
     condition     = length([for env in kubernetes_cron_job_v1.k8s_collector[0].spec[0].job_template[0].spec[0].template[0].spec[0].container[0].env : env if env.name == "ALLOW_UNENCRYPTED_UPLOAD"]) == 0
-    error_message = "ALLOW_UNENCRYPTED_UPLOAD must never be set unless allow_unencrypted_upload is true"
+    error_message = "ALLOW_UNENCRYPTED_UPLOAD must never be set: the module has no unencrypted path, and Nullify's bucket defaults to SSE-KMS so an unencrypted upload is rejected by KMS anyway"
   }
 
   assert {
@@ -60,9 +60,9 @@ run "collector_requires_a_cluster_name" {
 }
 
 # The published collector images exit 1 on upload with "upload blocked: no KMS
-# encryption and unencrypted uploads not explicitly allowed" when neither is
-# set, after a successful collection. Failing at plan is the only signal the
-# customer gets before that.
+# encryption and unencrypted uploads not explicitly allowed" when the key is
+# missing, after a successful collection. Failing at plan is the only signal
+# the customer gets before that.
 run "collector_requires_a_kms_key" {
   command = plan
 
@@ -75,24 +75,34 @@ run "collector_requires_a_kms_key" {
   expect_failures = [kubernetes_cron_job_v1.k8s_collector]
 }
 
-run "unencrypted_upload_is_an_explicit_opt_out" {
+run "collector_and_managed_scan_rbac_cannot_both_run" {
   command = plan
 
   variables {
     iam_role_arn             = "arn:aws:iam::123456789012:role/AWSIntegration-acme-NullifyReadOnlyRole"
     cluster_name             = "acme-prod"
     s3_bucket_name           = "nullify-collector-uploads"
-    allow_unencrypted_upload = true
+    kms_key_arn              = "arn:aws:kms:eu-central-1:111122223333:alias/nullify-customer-uploads"
+    enable_collector         = true
+    enable_managed_scan_rbac = true
   }
 
-  assert {
-    condition     = length([for env in kubernetes_cron_job_v1.k8s_collector[0].spec[0].job_template[0].spec[0].template[0].spec[0].container[0].env : env if env.name == "ALLOW_UNENCRYPTED_UPLOAD" && env.value == "true"]) == 1
-    error_message = "allow_unencrypted_upload must be the only thing that sets ALLOW_UNENCRYPTED_UPLOAD"
+  expect_failures = [kubernetes_cluster_role_v1.nullify_readonly]
+}
+
+run "the_kms_key_reaches_the_collector" {
+  command = plan
+
+  variables {
+    iam_role_arn   = "arn:aws:iam::123456789012:role/AWSIntegration-acme-NullifyReadOnlyRole"
+    cluster_name   = "acme-prod"
+    s3_bucket_name = "nullify-collector-uploads"
+    kms_key_arn    = "arn:aws:kms:eu-central-1:111122223333:alias/nullify-customer-uploads"
   }
 
   assert {
-    condition     = length([for env in kubernetes_cron_job_v1.k8s_collector[0].spec[0].job_template[0].spec[0].template[0].spec[0].container[0].env : env if env.name == "NULLIFY_KMS_KEY_ARN"]) == 0
-    error_message = "NULLIFY_KMS_KEY_ARN must not be set when no key was given"
+    condition     = length([for env in kubernetes_cron_job_v1.k8s_collector[0].spec[0].job_template[0].spec[0].template[0].spec[0].container[0].env : env if env.name == "NULLIFY_KMS_KEY_ARN" && env.value == "arn:aws:kms:eu-central-1:111122223333:alias/nullify-customer-uploads"]) == 1
+    error_message = "kms_key_arn must reach the collector as NULLIFY_KMS_KEY_ARN, which S3 takes as SSEKMSKeyId"
   }
 }
 

aws-integration-setup/terraform-v6/modules/k8s-resources/variables.tf

@@ -83,7 +83,7 @@ variable "cronjob_schedule" {
 
 variable "kms_key_arn" {
   type        = string
-  description = "The KMS ARN shown on the Nullify configure page, required when enable_collector is true: a key ARN or an alias ARN. The collector sends it to S3 as SSEKMSKeyId and refuses to upload without it. The key is in Nullify's account, so a bare key ID or a bare alias/<name> would resolve against your own account and is rejected here"
+  description = "The KMS ARN shown on the Nullify configure page, required when enable_collector is true: a key ARN or an alias ARN. The collector sends it to S3 as SSEKMSKeyId and refuses to upload without it, and there is no unencrypted path -- Nullify's bucket defaults to SSE-KMS, so an unencrypted upload needs kms:GenerateDataKey on Nullify's key anyway. The key is in Nullify's account, so a bare key ID or a bare alias/<name> would resolve against your own account and is rejected here"
   default     = ""
 
   validation {
@@ -92,12 +92,6 @@ variable "kms_key_arn" {
   }
 }
 
-variable "allow_unencrypted_upload" {
-  type        = bool
-  description = "WARNING: send the cluster inventory to Nullify's S3 area with no KMS encryption. Sets ALLOW_UNENCRYPTED_UPLOAD on the collector, which otherwise exits 1 rather than upload unencrypted. This is the only thing that sets that variable. Use it only if Nullify issued you no KMS key; otherwise leave it false and set kms_key_arn"
-  default     = false
-}
-
 variable "enable_debug" {
   type        = bool
   description = "Enable debug logging for troubleshooting"

aws-integration-setup/terraform-v6/modules/nullify-aws-integration/variables.tf

@@ -97,7 +97,7 @@ variable "tags" {
 
 variable "kms_key_arn" {
   type        = string
-  description = "The KMS ARN shown on the Nullify configure page (optional): a key ARN, including multi-Region key/mrk-... keys, or an alias ARN. A key ARN is granted on its own. An alias ARN also grants key/* in the same Nullify account and region, because IAM does not resolve an alias in a policy Resource. Nullify's key policy decides which of those keys the role can use"
+  description = "The KMS ARN shown on the Nullify configure page (optional): a key ARN, including multi-Region key/mrk-... keys, or an alias ARN. The key Nullify hands out is in Nullify's account, so the policy grants the ARN you pass and key/* in that same account and region, for key ARNs and alias ARNs alike: IAM does not resolve an alias in a policy Resource, and Nullify's key policy is the real gate on a key Nullify owns. The wildcard is derived only when the ARN's account differs from yours, so an ARN pasted from your own account grants that key alone. The kms_policy_resources output lists what was granted"
   default     = ""
 
   validation {

aws-integration-setup/terraform/README.md

@@ -277,7 +277,9 @@ terraform init && terraform apply
 
 The key Nullify hands out is in Nullify's account, so the policy grants the ARN you pass and `arn:<partition>:kms:<region>:<account>:key/*` in that same account and region: IAM does not resolve an alias ARN in a policy's `Resource`, and Nullify's key policy is the real gate on a key Nullify owns. The wildcard is derived only when the ARN's account differs from yours, so an ARN pasted from your own account grants that key alone. The `kms_policy_resources` module output lists what was granted.
 
-`k8s-resources` takes the same variable and writes no IAM policy: it forwards the value to the collector as `NULLIFY_KMS_KEY_ARN`, which S3 takes as `SSEKMSKeyId`. **The collector refuses to upload without it** — it walks the cluster, then exits 1 with `upload blocked: no KMS encryption and unencrypted uploads not explicitly allowed` — so `kms_key_arn` is required whenever `enable_collector` is true, and the module fails at plan without it. `allow_unencrypted_upload = true` is the only way to run the collector without a key; use it only if Nullify issued you none.
+`k8s-resources` takes the same variable and writes no IAM policy: it forwards the value to the collector as `NULLIFY_KMS_KEY_ARN`, which S3 takes as `SSEKMSKeyId`. S3 resolves a bare key ID or a bare `alias/<name>` against the calling account, and the key is in Nullify's, so only the two ARN forms above work there too; both modules validate for them.
+
+**The collector refuses to upload without a key** — it walks the cluster, then exits 1 with `upload blocked: no KMS encryption and unencrypted uploads not explicitly allowed` — so `kms_key_arn` is required whenever `enable_collector` is true, and the module fails at plan without it. There is no opt-out, and one would not help: Nullify's upload bucket sets SSE-KMS as its default encryption, so an object written with no encryption headers is still encrypted with Nullify's key, and S3 refuses the `PutObject` unless the caller holds `kms:GenerateDataKey` on it. `kms_key_arn` is the only thing that grants the role that.
 
 ## Module Usage
 
@@ -330,6 +332,8 @@ This is the same value as `collector.clusterName` in the `nullify-k8s-collector`
 
 `k8s-resources` variables include `enable_collector` (default `true`), `enable_managed_scan_rbac` (default `false`), `collector_image`, `cronjob_schedule` (default `0 0 * * *`), `kubernetes_namespace`, `service_account_name` and `enable_debug`.
 
+`enable_collector` and `enable_managed_scan_rbac` are mutually exclusive for one cluster, and the module fails at plan if both are true. The collector's upload registers the cluster as on-prem keyed on `cluster_name`; the managed scan registers the same cluster as its EKS ARN. Nothing joins the two, so enabling both lists the cluster twice with its pods and containers duplicated. To change mode, apply the new one and ask Nullify to remove the old cluster registration.
+
 `collector_image` defaults to `public.ecr.aws/w4o2j2x4/integrations:k8s-collector-3.46.0`, a pinned tag of Nullify's ECR Public image. It is the same build as `k8s-collector-latest`, which the `nullify-k8s-collector` Helm chart deploys, so both install paths run the same collector. Earlier versions defaulted to `nullify/k8s-collector:latest` on Docker Hub, which Nullify does not publish; if you set that value explicitly, replace it.
 
 ## Outputs
@@ -371,8 +375,10 @@ module "nullify_k8s" {
 - Terraform >= 1.8 is now required. It is the floor CI validates at and the floor the module test suites run at, so the declared version is the tested one.
 - The root and examples require AWS provider `>= 5.33`. Run `terraform init -upgrade` if your lock file pins an older 5.x.
 - `k8s-resources` now requires `cluster_name` while `enable_collector` is true, and sets it as the collector's `CLUSTER_NAME`. A deployment that never set it was uploading to `k8s-collector/default-name-data.json`; after this change it uploads to `k8s-collector/<cluster_name>-data.json`. The old object is in Nullify's bucket, which your role cannot read or delete: ask Nullify to remove the stale `default-name` cluster, and register the new name on the configure page before the next run.
-- `k8s-resources` now requires `kms_key_arn` while `enable_collector` is true. The shipped collector images refuse to upload without KMS encryption, so a deployment that left it empty was collecting successfully and then failing every upload. Set the value from the configure page, or set `allow_unencrypted_upload = true` if Nullify issued you no key.
-- `scan_mode = "both"` is no longer accepted in `examples/multi-cluster-complete`. The collector registers a cluster as on-prem, keyed on `cluster_name`; the managed scan registers the same cluster as its EKS ARN. Nothing joins them, so "both" listed one cluster twice with its pods and containers duplicated.
+- `k8s-resources` now requires `kms_key_arn` while `enable_collector` is true, with no opt-out. The shipped collector images refuse to upload without KMS encryption, so a deployment that left it empty was collecting successfully and then failing every upload. Set the value from the configure page; ask Nullify for a key if you have none.
+- `k8s-resources` no longer declares `allow_unencrypted_upload`. Passing it is now a plan error. It never worked against Nullify's bucket: leaving `kms_key_arn` empty also leaves the IAM role with no KMS statement, and the bucket's SSE-KMS default encryption makes `kms:GenerateDataKey` on Nullify's key a requirement of every upload, encrypted headers or not. The variable turned a self-explaining collector error into an S3 `AccessDenied`.
+- `k8s-resources` now rejects `enable_collector` and `enable_managed_scan_rbac` together, which `examples/multi-cluster-complete` already rejected as `scan_mode = "both"`. The two paths register the same cluster under different identities and nothing joins them.
+- `scan_mode = "both"` is no longer accepted in `examples/multi-cluster-complete`, and the module enforces the same rule. The collector registers a cluster as on-prem, keyed on `cluster_name`; the managed scan registers the same cluster as its EKS ARN. Nothing joins them, so "both" listed one cluster twice with its pods and containers duplicated.
 - `k8s-resources` collector resources moved to `count` instances. `moved` blocks keep existing state, so a plan should show no changes; check it before applying.
 - The root configuration no longer declares the Kubernetes provider. It never deployed Kubernetes resources.
 - A cluster outside the AWS provider's region now fails at plan time with an explanation instead of a not-found error.

aws-integration-setup/terraform/examples/multi-cluster-complete/terraform.tfvars.example

@@ -34,8 +34,10 @@ aws_region = "ap-southeast-2"
 # nullify_s3_access_point_arn = "arn:aws:s3:ap-southeast-2:123456789012:accesspoint/acme-corp-k8s-collector"
 
 # KMS ARN from the Nullify configure page. A key ARN or an alias ARN both work.
-# REQUIRED when scan_mode is collector: the collector refuses to upload without KMS encryption.
-kms_key_arn = "arn:aws:kms:ap-southeast-2:123456789012:key/12345678-1234-1234-1234-123456789012"
+# REQUIRED when scan_mode is collector: the collector refuses to upload without KMS
+# encryption, and there is no opt-out. The key lives in Nullify's account, not yours,
+# so it is not the account id used for nullify_role_arn or eks_cluster_arns above.
+# kms_key_arn = "arn:aws:kms:<nullify-region>:<nullify-account-id>:key/<from-the-configure-page>"
 
 # Kubernetes namespace for Nullify resources
 kubernetes_namespace = "nullify"