EKS Cluster Scanning

EKS cluster scanning is an option on the AWS integration. It discovers the Amazon Elastic Kubernetes Service (EKS) clusters in every enabled region of the connected accounts and scans them remotely from Defakto, with nothing installed in the cluster. Ledger finds application workloads and the credentials they use, then links them to secrets found by other integrations, the same way the Kubernetes integration does.

The integration reads Secrets through the Kubernetes API, so secret values pass through the integration's memory while it checks them for credentials. Ledger stores only fingerprints, redacted hints, and non-secret identifiers, never full secret values.

What Data is Collected?

1. Applications - Deployments, StatefulSets, DaemonSets, CronJobs, and Jobs.
2. Secrets - Metadata about environment variables, Kubernetes Secrets, and ConfigMaps that contain suspected credentials (e.g. key name, resource name, and detected type).
3. Metadata - Cluster details such as the cloud provider and region.

Integration Permissions

EKS cluster scanning adds the following statements to the AWS integration permissions. IntegrationID is the integration ID, which begins with sc-, and ClusterScannerRoleName is ledger-cluster-scanner-<integration-id>.

EKS Statements in the Integration Role Policy

PolicyDocument:
  Version: '2012-10-17'
  Statement:
    # Discover clusters in every enabled region.
    # These APIs do not support resource-level permissions.
    - Effect: Allow
      Action:
        - ec2:DescribeRegions
        - eks:ListClusters
      Resource: '*'
    - Effect: Allow
      Action:
        - eks:DescribeCluster
      Resource: !Sub "arn:aws:eks:*:${AWS::AccountId}:cluster/*"
    # Create an access entry only for the cluster scanner role, as a STANDARD
    # entry tagged with this integration ID and nothing else, with no
    # Kubernetes username or groups.
    - Effect: Allow
      Action:
        - eks:CreateAccessEntry
      Resource: !Sub "arn:aws:eks:*:${AWS::AccountId}:cluster/*"
      Condition:
        StringEquals:
          "eks:principalArn": !Sub "arn:aws:iam::${AWS::AccountId}:role/${ClusterScannerRoleName}"
          "eks:accessEntryType": STANDARD
          "aws:RequestTag/defakto:integration-id": !Ref IntegrationID
        ForAllValues:StringEquals:
          "aws:TagKeys":
            - defakto:integration-id
        "Null":
          "eks:username": "true"
          "eks:kubernetesGroups": "true"
    # Tag the scanner role's access entry with this integration ID.
    - Effect: Allow
      Action:
        - eks:TagResource
      Resource: !Sub "arn:aws:eks:*:${AWS::AccountId}:access-entry/*/role/${AWS::AccountId}/${ClusterScannerRoleName}/*"
      Condition:
        StringEquals:
          "aws:RequestTag/defakto:integration-id": !Ref IntegrationID
        ForAllValues:StringEquals:
          "aws:TagKeys":
            - defakto:integration-id
    # Read the scanner role's access entry. Reads carry no tag condition so
    # that an entry this integration does not own can be inspected and refused.
    - Effect: Allow
      Action:
        - eks:DescribeAccessEntry
        - eks:ListAssociatedAccessPolicies
      Resource: !Sub "arn:aws:eks:*:${AWS::AccountId}:access-entry/*/role/${AWS::AccountId}/${ClusterScannerRoleName}/*"
    # Attach only the two read-only access policies, at cluster scope, to
    # entries this integration owns.
    - Effect: Allow
      Action:
        - eks:AssociateAccessPolicy
      Resource: !Sub "arn:aws:eks:*:${AWS::AccountId}:access-entry/*/role/${AWS::AccountId}/${ClusterScannerRoleName}/*"
      Condition:
        StringEquals:
          "eks:principalArn": !Sub "arn:aws:iam::${AWS::AccountId}:role/${ClusterScannerRoleName}"
          "eks:accessEntryType": STANDARD
          "aws:ResourceTag/defakto:integration-id": !Ref IntegrationID
          "eks:accessScope": cluster
        ArnEquals:
          "eks:policyArn":
            - arn:aws:eks::aws:cluster-access-policy/AmazonEKSViewPolicy
            - arn:aws:eks::aws:cluster-access-policy/AmazonEKSSecretReaderPolicy
    # Delete only access entries this integration owns.
    - Effect: Allow
      Action:
        - eks:DeleteAccessEntry
      Resource: !Sub "arn:aws:eks:*:${AWS::AccountId}:access-entry/*/role/${AWS::AccountId}/${ClusterScannerRoleName}/*"
      Condition:
        StringEquals:
          "eks:principalArn": !Sub "arn:aws:iam::${AWS::AccountId}:role/${ClusterScannerRoleName}"
          "eks:accessEntryType": STANDARD
          "aws:ResourceTag/defakto:integration-id": !Ref IntegrationID
    # Cluster contents are read through the scanner role, never through the
    # integration role.
    - Effect: Allow
      Action:
        - sts:AssumeRole
      Resource: !Sub "arn:aws:iam::${AWS::AccountId}:role/${ClusterScannerRoleName}"

Organization Integration: Management Account Scanner Role Statement

- Effect: Allow
  Action:
    - sts:AssumeRole
  Resource: !Sub "arn:aws:iam::*:role/${ClusterScannerRoleName}"
  Condition:
    StringEquals:
      "aws:ResourceOrgID": !Ref OrganizationId

The template also creates a scanner role in each account. The scanner role trusts only the integration role.

Cluster Scanner Role

ClusterScannerRole:
  Type: AWS::IAM::Role
  Properties:
    RoleName: !Ref ClusterScannerRoleName
    AssumeRolePolicyDocument:
      Version: '2012-10-17'
      Statement:
        - Effect: Allow
          Principal:
            AWS: !GetAtt IntegrationRole.Arn
          Action: sts:AssumeRole
    Policies:
      - PolicyName: ledger-cluster-scanner-permissions
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - eks:DescribeCluster
              Resource: !Sub "arn:aws:eks:*:${AWS::AccountId}:cluster/*"

On each cluster, Ledger creates an EKS access entry for the scanner role, tagged defakto:integration-id, and associates the AWS-managed AmazonEKSViewPolicy and AmazonEKSSecretReaderPolicy at cluster scope. AmazonEKSSecretReaderPolicy allows the scanner role to read every Secret in the cluster. See the AWS access policy permissions reference for the exact Kubernetes permissions.

Installation

Prerequisites

Each cluster you want scanned must have:

1. Status ACTIVE.
2. Authentication mode API or API_AND_CONFIG_MAP. To switch a CONFIG_MAP cluster, run the following command. The switch is one-way, and existing aws-auth grants keep working.

   aws eks update-cluster-config --name <cluster-name> \
     --access-config authenticationMode=API_AND_CONFIG_MAP

3. Public access to the Kubernetes API endpoint. If you restrict the public access CIDRs, add the integration's egress addresses. Addresses must be obtained from Defakto directly.

Clusters that do not meet these requirements are listed with the reason they were skipped.

Steps

EKS cluster scanning is set when you create the AWS integration and cannot be changed afterward. In the CLI, add --cluster-scanning to spirlctl ledger integration aws add.

1. Follow the AWS integration steps. In the "Kubernetes" section, leave "EKS scanning" checked. The box is checked by default.
2. Apply the Terraform template. The template includes the EKS permissions and the scanner role.
3. Once the integration connects, Ledger discovers clusters and scans them on the integration's schedule. Use the "Run now" button to scan immediately.

warning

When you delete the integration, Ledger removes its access entries in the background using the integration role, usually within a few minutes. Wait until the entries are gone before you destroy the integration's Terraform resources, or the remaining entries stay on your clusters. To check a cluster, run:

aws eks list-access-entries \
  --cluster-name <cluster-name> \
  --region <region> \
  --query "accessEntries[?contains(@, 'ledger-cluster-scanner-')]"

An empty list means the cluster has no Ledger access entries left. If an entry is still listed an hour after you deleted the integration, remove it with aws eks delete-access-entry before you destroy the Terraform resources.