What

This was brought to my attention and discovered by 💪@SimKev2 💪

Resources which are marked for termination, on gatekeeper startup, cause gatekeeper to fail its readiness probes indefinitely (assuming gatekeeper is trying to sync those resources). This seems to be because gatekeeper is trying to ensure it has successfully loaded the sync cache before it handles any traffic. However, it "expects" but fails to "observe" terminating resources, resulting in a deadlock for the readiness check.

Steps

1. Create the following namespace.

apiVersion: v1
kind: Namespace
metadata:
  finalizers:
  - rob.test.io/abcdefg
  name: rob-test

2. Delete it with kubectl delete ns rob-test (This should hang, since the finalizer won't resolve) The purpose of this is to put a resource permanently into the terminating state.

3. Ensure Namespace is in the sync config

apiVersion: config.gatekeeper.sh/v1alpha1
kind: Config
metadata:
  name: config
  namespace: gatekeeper-system
spec:
  sync:
    syncOnly:
    - kind: Namespace
      version: v1

4. Start Gatekeeper

After gatekeeper starts up it should fail its readiness checks indefinitely.

This seems to be because when setting the expectations for the objectTracker we take into consideration the rob-test namespace (even though it's terminating). This becomes a problem later when running the sync controller, since it doesn't observe resources marked for termination.

I think what makes sense is to run Observe on resources which have been marked for termination.

So add this r.tracker.ForData(gvk).Observe(instance) here

FYI @shomron

Environment:

• Gatekeeper version: v3.1.0-beta.9
• Kubernetes version: 1.15