K8s 教程

Secret 管理和审计

Secret 管理和审计

Secret 管理和审计日志是 Kubernetes 安全的最后一道防线。

Secret 管理

Secret 基础

# 创建 Secret
apiVersion: v1
kind: Secret
metadata:
  name: app-secret
  namespace: production
type: Opaque
stringData:  # 自动 base64 编码
  username: admin
  password: P@ssw0rd123!
  api-key: abc123xyz789

在 Pod 中使用 Secret

方式 1：环境变量

apiVersion: v1
kind: Pod
metadata:
  name: myapp
spec:
  containers:
  - name: app
    image: myapp:1.0
    env:
    # 单个 key
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: app-secret
          key: password
    
    # 所有 key
    envFrom:
    - secretRef:
        name: app-secret

方式 2：Volume 挂载（推荐）

apiVersion: v1
kind: Pod
metadata:
  name: myapp
spec:
  containers:
  - name: app
    image: myapp:1.0
    volumeMounts:
    - name: secret-volume
      mountPath: /etc/secrets
      readOnly: true
  
  volumes:
  - name: secret-volume
    secret:
      secretName: app-secret
      defaultMode: 0400  # 只读权限

Secret 加密存储

Kubernetes 默认将 Secret 以 base64 编码存储在 etcd，并非加密！

启用 etcd 加密

# /etc/kubernetes/encryption-config.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
  - secrets
  providers:
  - aescbc:
      keys:
      - name: key1
        secret: <base64-encoded-32-byte-key>
  - identity: {}  # 回退到未加密（用于读取旧数据）

# 生成加密密钥
head -c 32 /dev/urandom | base64

# 配置 kube-apiserver
# 在 /etc/kubernetes/manifests/kube-apiserver.yaml 中添加：
--encryption-provider-config=/etc/kubernetes/encryption-config.yaml

加密所有现有 Secret

# 重新写入所有 Secret 以使用新的加密配置
kubectl get secrets --all-namespaces -o json | \
  kubectl replace -f -

外部 Secret 管理

HashiCorp Vault

安装 Vault

# 使用 Helm 安装
helm repo add hashicorp https://helm.releases.hashicorp.com
helm install vault hashicorp/vault \
  --namespace vault \
  --create-namespace \
  --set server.dev.enabled=true

配置 Kubernetes Auth

# 进入 Vault Pod
kubectl exec -it vault-0 -n vault -- /bin/sh

# 启用 Kubernetes 认证
vault auth enable kubernetes

# 配置 Kubernetes 认证
vault write auth/kubernetes/config \
  kubernetes_host="https://kubernetes.default.svc:443"

# 创建策略
vault policy write myapp - <<EOF
path "secret/data/myapp/*" {
  capabilities = ["read"]
}
EOF

# 创建角色
vault write auth/kubernetes/role/myapp \
  bound_service_account_names=myapp-sa \
  bound_service_account_namespaces=production \
  policies=myapp \
  ttl=24h

存储 Secret

# 写入 Secret
vault kv put secret/myapp/database \
  username=dbuser \
  password=dbpass123

# 读取 Secret
vault kv get secret/myapp/database

在 Pod 中使用 Vault

apiVersion: v1
kind: Pod
metadata:
  name: myapp
  namespace: production
spec:
  serviceAccountName: myapp-sa
  containers:
  - name: app
    image: myapp:1.0
    env:
    - name: VAULT_ADDR
      value: "http://vault.vault:8200"
  
  # 使用 Vault Agent Sidecar
  initContainers:
  - name: vault-agent
    image: vault:1.15.0
    args:
    - agent
    - -config=/vault/config/agent.hcl
    volumeMounts:
    - name: config
      mountPath: /vault/config
    - name: shared-data
      mountPath: /vault/secrets
  
  volumes:
  - name: config
    configMap:
      name: vault-agent-config
  - name: shared-data
    emptyDir: {}

External Secrets Operator

# 安装 External Secrets Operator
helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets \
  external-secrets/external-secrets \
  -n external-secrets-system \
  --create-namespace

配置 SecretStore

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: vault-backend
  namespace: production
spec:
  provider:
    vault:
      server: "http://vault.vault:8200"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "myapp"
          serviceAccountRef:
            name: "myapp-sa"

创建 ExternalSecret

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: myapp-secret
  namespace: production
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  
  target:
    name: myapp-secret
    creationPolicy: Owner
  
  data:
  - secretKey: database-password
    remoteRef:
      key: myapp/database
      property: password
  
  - secretKey: api-key
    remoteRef:
      key: myapp/api
      property: key

Sealed Secrets

Sealed Secrets 允许加密 Secret 后提交到 Git。

# 安装 Sealed Secrets Controller
kubectl apply -f https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.24.0/controller.yaml

# 安装 kubeseal CLI
brew install kubeseal

使用 Sealed Secrets

# 1. 创建普通 Secret
kubectl create secret generic mysecret \
  --from-literal=password=mypassword \
  --dry-run=client -o yaml > secret.yaml

# 2. 加密 Secret
kubeseal -f secret.yaml -w sealed-secret.yaml

# 3. 应用加密的 Secret（安全，可提交到 Git）
kubectl apply -f sealed-secret.yaml

# 4. Controller 自动解密为普通 Secret
kubectl get secret mysecret

审计日志

审计日志记录所有对 API Server 的请求，是安全审计的关键。

启用审计日志

审计策略配置

# /etc/kubernetes/audit-policy.yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
# 不记录只读请求
- level: None
  verbs: ["get", "list", "watch"]

# 记录 Secret 的所有操作
- level: RequestResponse
  resources:
  - group: ""
    resources: ["secrets"]

# 记录认证令牌
- level: Metadata
  resources:
  - group: ""
    resources: ["serviceaccounts/token"]

# 记录重要的配置变更
- level: Request
  verbs: ["create", "update", "patch", "delete"]
  resources:
  - group: ""
    resources: ["configmaps", "services"]
  - group: "apps"
    resources: ["deployments", "statefulsets", "daemonsets"]
  - group: "rbac.authorization.k8s.io"
    resources: ["roles", "rolebindings", "clusterroles", "clusterrolebindings"]

# 默认记录元数据
- level: Metadata

配置 kube-apiserver

# /etc/kubernetes/manifests/kube-apiserver.yaml
spec:
  containers:
  - command:
    - kube-apiserver
    - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
    - --audit-log-path=/var/log/kubernetes/audit.log
    - --audit-log-maxage=30
    - --audit-log-maxbackup=10
    - --audit-log-maxsize=100
    volumeMounts:
    - name: audit-policy
      mountPath: /etc/kubernetes/audit-policy.yaml
      readOnly: true
    - name: audit-log
      mountPath: /var/log/kubernetes
  volumes:
  - name: audit-policy
    hostPath:
      path: /etc/kubernetes/audit-policy.yaml
      type: File
  - name: audit-log
    hostPath:
      path: /var/log/kubernetes
      type: DirectoryOrCreate

审计日志示例

{
  "kind": "Event",
  "apiVersion": "audit.k8s.io/v1",
  "level": "RequestResponse",
  "auditID": "abc123-def456-ghi789",
  "stage": "ResponseComplete",
  "requestURI": "/api/v1/namespaces/production/secrets/app-secret",
  "verb": "get",
  "user": {
    "username": "alice",
    "groups": ["developers", "system:authenticated"]
  },
  "sourceIPs": ["192.168.1.100"],
  "userAgent": "kubectl/v1.28.0",
  "objectRef": {
    "resource": "secrets",
    "namespace": "production",
    "name": "app-secret",
    "apiVersion": "v1"
  },
  "responseStatus": {
    "code": 200
  },
  "requestReceivedTimestamp": "2024-01-08T12:00:00.000Z",
  "stageTimestamp": "2024-01-08T12:00:00.100Z"
}

审计日志分析

# 查看谁访问了 Secret
cat /var/log/kubernetes/audit.log | \
  jq 'select(.objectRef.resource=="secrets") | 
      {user: .user.username, secret: .objectRef.name, verb: .verb, time: .requestReceivedTimestamp}'

# 查看失败的认证尝试
cat /var/log/kubernetes/audit.log | \
  jq 'select(.responseStatus.code >= 400 and .responseStatus.code < 500)'

# 查看删除操作
cat /var/log/kubernetes/audit.log | \
  jq 'select(.verb=="delete") | 
      {user: .user.username, resource: .objectRef.resource, name: .objectRef.name}'

发送审计日志到 Elasticsearch

# kube-apiserver 配置
--audit-webhook-config-file=/etc/kubernetes/audit-webhook.yaml
--audit-webhook-batch-max-size=100

# audit-webhook.yaml
apiVersion: v1
kind: Config
clusters:
- name: elasticsearch
  cluster:
    server: http://elasticsearch.logging:9200/_bulk
contexts:
- context:
    cluster: elasticsearch
  name: default
current-context: default

Falco 运行时安全

Falco 监控运行时异常行为。

Falco 规则示例

# 检测访问 Secret 的异常行为
- rule: Suspicious Secret Access
  desc: Detect unusual access to Kubernetes secrets
  condition: >
    k8s_audit and ka.verb="get" and 
    ka.target.resource="secrets" and
    not user_known_secret_access
  output: >
    Suspicious secret access 
    (user=%ka.user.name secret=%ka.target.name 
    namespace=%ka.target.namespace)
  priority: WARNING

# 检测容器中执行的敏感命令
- rule: Sensitive Command in Container
  desc: Detect execution of sensitive commands
  condition: >
    spawned_process and container and
    (proc.name in (nc, ncat, nmap, socat, tcpdump, tshark))
  output: >
    Sensitive command executed in container
    (user=%user.name command=%proc.cmdline 
    container=%container.id)
  priority: CRITICAL

# 检测 SSH 登录到容器
- rule: SSH Connection to Container
  desc: Detect SSH connection to a container
  condition: >
    inbound_outbound and container and
    fd.sport=22 and fd.l4proto=tcp
  output: >
    SSH connection to container
    (container=%container.id ip=%fd.rip)
  priority: WARNING

Falco 告警集成

# 配置 Falco 发送告警到 Slack
# /etc/falco/falco.yaml
json_output: true
json_include_output_property: true

http_output:
  enabled: true
  url: "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"

安全扫描工具

Kube-bench

检查集群是否符合 CIS Kubernetes Benchmark。

# 运行 kube-bench
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml

# 查看结果
kubectl logs -f job/kube-bench

# 输出示例：
# [INFO] 1 Master Node Security Configuration
# [PASS] 1.1.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive
# [FAIL] 1.1.2 Ensure that the API server pod specification file ownership is set to root:root
# [WARN] 1.2.1 Ensure that the --anonymous-auth argument is set to false

Kube-hunter

寻找集群安全漏洞。

# 运行 kube-hunter
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-hunter/main/job.yaml

# 查看结果
kubectl logs -f job/kube-hunter

Kubescape

Kubernetes 安全合规性扫描。

# 安装
brew install kubescape

# 扫描集群
kubescape scan

# 扫描特定框架
kubescape scan framework nsa,mitre

# 扫描 YAML 文件
kubescape scan deployment.yaml

最佳实践清单

Secret 管理

  • ✅ 使用外部 Secret 管理系统（Vault）
  • ✅ 启用 etcd 加密
  • ✅ 定期轮换 Secret
  • ✅ 使用 Volume 挂载而非环境变量
  • ✅ 限制 Secret 访问权限（RBAC）
  • ❌ 不要在日志中打印 Secret
  • ❌ 不要将 Secret 提交到 Git

审计日志

  • ✅ 启用审计日志
  • ✅ 记录 Secret 访问
  • ✅ 记录 RBAC 变更
  • ✅ 定期分析审计日志
  • ✅ 设置日志告警
  • ✅ 保留足够的日志历史（30-90天）

运行时安全

  • ✅ 部署 Falco 监控异常行为
  • ✅ 定期运行安全扫描（kube-bench）
  • ✅ 监控容器行为
  • ✅ 设置告警通知
  • ✅ 定期审查安全策略

持续安全

# 在 CI/CD 中集成安全检查
stages:
- build
- scan
- deploy

security-scan:
  stage: scan
  script:
    # 镜像漏洞扫描
    - trivy image --severity HIGH,CRITICAL myapp:$CI_COMMIT_SHA
    
    # YAML 安全检查
    - kubescape scan k8s/*.yaml
    
    # Secret 检测
    - gitleaks detect --source .
    
    # Dockerfile 最佳实践检查
    - hadolint Dockerfile

小结

本节介绍了 Secret 管理和审计：

✅ Secret 管理：加密存储、外部管理（Vault）、Sealed Secrets
✅ 审计日志：配置、分析、告警
✅ Falco：运行时安全监控
✅ 安全扫描：kube-bench、kube-hunter、kubescape
✅ 最佳实践：Secret、审计、运行时、持续安全

至此，安全最佳实践章节全部完成！🎉

下一章：生产环境实践。

下一章
什么是 Kubernetes