Mar 24, 2022
Container Security Checklist
Cloud Native Concepts
Cloud Native Concepts
Table by Aqua Cloud Native Security Platform, more details download here
Container Threat Model
- Insecure Host
- Misconfiguration container
- Vulnerable application
- Supply chain attacks
- Expose secrets
- Insecure networking
- Integrity and confidentiality of OS images
- Container escape vulnerabilities
Container Security Checklist
Checklist to build and secure the images across the following phases:
- Secure the Build
- Secure the Container Registry
- Secure the Container Runtime
- Secure the Infrastructure
- Secure the Data
- Secure the Workloads
Secure the Build
Secure Supply Chain
- Know where images, packages came from.
Hardening Code — Secure SDLC (Software Development Life Cycle)
- Do a static analysis of the code and libraries used by the code to surface any vulnerabilities in the code and its dependencies.
- Improve the security and quality of their code. OWASP Open Source Application Security tools: SAST, IAST.
Secure the Image — Hardening
You can build the container images using Docker, Kaniko.
- Reduce the attack surface
Package a single application per container. Small container images. Minimize the number of layers.
- Use the minimal OS image:
- Alpine images
- Scratch images
- Distroless images
- Use OS optimized for running containers:
- Flatcar images
- CodeOS by Fedora replaced the Project Atomic.
- Bottlerocket by Aws
- k3os by Rancher
- Container-Optimized OS — COS by Google, based on Chromium-os used by Google
- Do you use Alpine, distroless or vanilla images? …
- 7 Google best practices for building containers
- Multi-staged builds.
A well-designed multi-stage build contains only the minimal binary files and dependencies required for the final image, with no build tools or intermediate files. Optimize cache.
- Use official base images.
- Avoid unknown public images.
- Rootless. Run as a non-root user. Least privileged user
- Create a dedicated user and group on the image.
Do not use a UID below 10,000. For best security, always run your processes as a UID above 10,000. Remove setuid and setgid permissions from the images
- Avoid privileged containers, which lets a container run as root on the local machine.
- Use only the necessary Privileged Capabilities.
- Drop kernel modules, system time, trace processes (CAP_SYS_MODULE, CAP_SYS_TIME, CAP_SYS_PTRACE ).
- Enable the
--read-onlymode in docker, if it's possible. - Don’t leave sensitive information (secrets, tokens, keys, etc) in the image.
- Not mounting Host Path.
- Use Metadata Labels for Images, such as licensing information, sources, names of authors, and relation of containers to projects or components.
- Used fixed image tag for inmutability.
- Tagging using semantic versioning.
- Not use mutable tags(latest,staging,etc). Use Inmutable tags(SHA-256, commit).
- The challengue of uniquely identifying your images
Pulling images by digest
docker images --digests
docker pull alpine@sha256:b7233dafbed64e3738630b69382a8b231726aa1014ccaabc1947c5308a8910a7- Enanbled Security profiles: SELinux, AppArmor, Seccomp.
- Static code analysys tool for Dockerfile like a linter. Detect misconfigurations
- Hadolint
- Packers (including encrypters), and downloaders are all able to evade static scanning by, for example, encrypting binary code that is only executed in memory, making the malware active only in runtime.
- Trivy detect missconfigurations
Image Scanning
- Check image for Common Vulnerabilities and Exposures (CVE)
- Check image for secrets
- Prevent attacks using the Supply Chain Attack
- Scan container images for CVE (Common Vulnerabilities and Exposures).
- Used dynamic analysis techniques for containers.
Container Security Scanners
Comparing the container scanners results:
- Container Vulnerability Scanning Fun by Rory
- Comparison — Anchore Engine vs Clair vs Trivy by Alfredo Pardo
Image Signing
Sign and verify images to mitigate MITM attacks. Docker offers a Content Trust mechanism that allows you to cryptographically sign images using a private key. This guarantees the image, and its tags, have not been modified.
- Notary. Implementation of TUF specification.
- sigstore/Cosign
- Sigstore: A Solution to Software Supply Chain Security
- Zero-Trust supply chains with Sigstore and SPIFFE/SPIRE
More Material about build containers
Secure the Build
Secure Supply Chain
- Know where images, packages came from.
Hardening Code — Secure SDLC (Software Development Life Cycle)
- Do a static analysis of the code and libraries used by the code to surface any vulnerabilities in the code and its dependencies.
- Improve the security and quality of their code. OWASP Open Source Application Security tools: SAST, IAST.
Secure the Image — Hardening
You can build the container images using Docker, Kaniko.
- Reduce the attack surface
Package a single application per container. Small container images. Minimize the number of layers.
- Use the minimal OS image:
- Alpine images
- Scratch images
- Distroless images
- Use OS optimized for running containers:
- Flatcar images
- CodeOS by Fedora replaced the Project Atomic.
- Bottlerocket by Aws
- k3os by Rancher
- Container-Optimized OS — COS by Google, based on Chromium-os used by Google
- Do you use Alpine, distroless or vanilla images? …
- 7 Google best practices for building containers
- Multi-staged builds.
A well-designed multi-stage build contains only the minimal binary files and dependencies required for the final image, with no build tools or intermediate files. Optimize cache.
- Use official base images.
- Avoid unknown public images.
- Rootless. Run as a non-root user. Least privileged user
- Create a dedicated user and group on the image.
Do not use a UID below 10,000. For best security, always run your processes as a UID above 10,000. Remove setuid and setgid permissions from the images
- Avoid privileged containers, which lets a container run as root on the local machine.
- Use only the necessary Privileged Capabilities.
- Drop kernel modules, system time, trace processes (CAP_SYS_MODULE, CAP_SYS_TIME, CAP_SYS_PTRACE ).
- Enable the
--read-onlymode in docker, if it's possible. - Don’t leave sensitive information (secrets, tokens, keys, etc) in the image.
- Not mounting Host Path.
- Use Metadata Labels for Images, such as licensing information, sources, names of authors, and relation of containers to projects or components.
- Used fixed image tag for inmutability.
- Tagging using semantic versioning.
- Not use mutable tags(latest,staging,etc). Use Inmutable tags(SHA-256, commit).
- The challengue of uniquely identifying your images
Pulling images by digest
docker images --digests
docker pull alpine@sha256:b7233dafbed64e3738630b69382a8b231726aa1014ccaabc1947c5308a8910a7- Enanbled Security profiles: SELinux, AppArmor, Seccomp.
- Static code analysys tool for Dockerfile like a linter. Detect misconfigurations
- Hadolint
- Packers (including encrypters), and downloaders are all able to evade static scanning by, for example, encrypting binary code that is only executed in memory, making the malware active only in runtime.
- Trivy detect missconfigurations
Image Scanning
- Check image for Common Vulnerabilities and Exposures (CVE)
- Check image for secrets
- Prevent attacks using the Supply Chain Attack
- Scan container images for CVE (Common Vulnerabilities and Exposures).
- Used dynamic analysis techniques for containers.
Container Security Scanners
Comparing the container scanners results:
- Container Vulnerability Scanning Fun by Rory
- Comparison — Anchore Engine vs Clair vs Trivy by Alfredo Pardo
Image Signing
Sign and verify images to mitigate MITM attacks. Docker offers a Content Trust mechanism that allows you to cryptographically sign images using a private key. This guarantees the image, and its tags, have not been modified.
- Notary. Implementation of TUF specification.
- sigstore/Cosign
- Sigstore: A Solution to Software Supply Chain Security
- Zero-Trust supply chains with Sigstore and SPIFFE/SPIRE
More Material about build containers
Secure the Container Registry
Best configurations with ECR, ACR, Harbor, etc. Best practices.
- Lock down access to the image registry (who can push/pull) to restrict which users can upload and download images from it. Uses Role Based Access Control (RBAC)
There is no guarantee that the image you are pulling from the registry is trusted. It may unintentionally contain security vulnerabilities, or may have intentionally been replaced with an image compromised by attackers.
- Use a private registry deployed behind firewall, to reduce the risk of tampering.
Registry Resources
Secure the Container Runtime
See the following container runtimes, there are three main types of container runtimes — low-level runtimes, high-level runtimes, and virtualized runtimes or sandboxed.
- Low-Level Container Runtimes:
- High-Level Container Runtimes
- Docker Engine
- Podman
- CRI-O — OCI-based implementation of Kubernetes Container Runtime Interface
- Mirantes Container Runtime
- Sandboxed and Virtualized Container Runtimes
Why is important Runtime Security?
- Detection of IOC (Indicator Of Compromise)
- Detect Zero Days attack
- Compliance requirement
- Recommended in highly dynamic environments
Constraints
- Event context
- Safety
- Low overhead
- Wide support of kernels
Enable detection of anomalous behaviour in applications.
- Applied the secure configurations in the container runtime. By default is insecure.
- Restrict access to container runtime daemon/APIs
- Use if it’s possible in Rootless Mode.
Docker Security
- Avoid misconfigured exposed Docker API Ports, attackers used the misconfigured port to deploy and run a malicious image that contained malware that was specifically designed to evade static scanning.
- TLS encryption between the Docker client and daemon. Do not expose the Docker engine using Unix socket or remotely using http.
Never make the daemon socket available for remote connections, unless you are using Docker’s encrypted HTTPS socket, which supports authentication.
- Limit the usage of mount Docker socket in a container in an untrusted environment.
- Do not run Docker images with an option that exposes the socket in the container.
-v /var/run/docker.sock://var/run/docker.sock
The Docker daemon socket is a Unix network socket that facilitates communication with the Docker API. By default, this socket is owned by the root user. If anyone else obtains access to the socket, they will have permissions equivalent to root access to the host.
- Run Docker in Rootless Mode.
docker context use rootless - Enable the user namespaces.
- Enable Docker Content Trust. Docker.
DOCKER_CONTENT_TRUST=1. Docker Content Trust implements The Update Framework (TUF) . Powered by Notary, an open-source TUF-client and server that can operate over arbitrary trusted collections of data.
More Material about Docker Security
Secure the Infrastructure
Risk:
- If host is compromised, the container will be too.
- Kernel exploits
Best practices:
- Keep the host kernel patched to prevent a range of known vulnerabilities, many of which can result in container escape. Since the kernel is shared by the container and the host, the kernel exploits when an attacker manages to run on a container can directly affect the host.
- Use CIS-Benchmark for the operating system.
- Use secure computing (seccomp) to restrict host system call (syscall) access from containers.
- Use Security-Enhanced Linux (SELinux) to further isolate containers.
Secure the Data
- Don’t leak sensitive info in the images, avoid using environment variables for your sensitive information.
Secrets are Digital credentials:
- passwords
- API keys & Tokens
- SSH keys
- Private certificates for secure communication, transmitting and receiving data (TLS, SSL, and so on)
- Private encryption keys for systems like PGP
- Database names or connection strings.
- Sensitive configuration settings (email address, usernames, debug flags, etc.)
- Use a proper filesystem encryption technology for container storage
- Use volume mounts to pass secrets to a container at runtime
- Provide write/execute access only to the containers that need to modify the data in a specific host filesystem path
- OPA to write controls like only allowing Read-only Root Filesystem access, listing allowed host filesystem paths to mount, and listing allowed Flex volume drivers.
- Automatically scan container images for sensitive data such as credentials, tokens, SSH keys, TLS certificates, database names or connection strings and so on, before pushing them to a container registry (can be done locally and in CI).
- Limit storage related syscalls and capabilities to prevent runtime privilege escalation.
- Implement RBAC, or role-based access control. Every human or application only needs the minimum secrets required to operate, nothing more. Principle of Least Privilege.
- Run audits regularly. Centralized audit trails are the key to knowing all the key security events.
- Rotate secrets, a standard security practice.
- Automatically create and store secrets
Secrets Management Tools
Open source tools:
- detect-secrets by Yelp: detecting and preventing secrets in code.
- git-secrets by awslabs: Prevents you from committing secrets and credentials into git repositories
Cloud Provider Key Management
Enterprise secrets vault:
Secure the Workloads … Running the containers
- Avoid privileged containers
- • Root access to all devices • Ability to tamper with Linux security modules like AppArmor and SELinux • Ability to install a new instance of the Docker platform, using the host’s kernel capabilities, and run Docker within Docker.
To check if the container is running in privileged mode
docker inspect --format =’{{. HostConfig.Privileged}}’[container_id]
- Limit container resources.
When a container is compromised, attackers may try to make use of the underlying host resources to perform malicious activity. Set memory and CPU usage limits to minimize the impact of breaches for resource-intensive containers.
docker run -d --name container-1 --cpuset-cpus 0 --cpu-shares 768 cpu-stress- Preventing a fork bomb.
docker run --rm -it --pids-limit 200 debian:jessie - Segregate container networks.
- The default bridge network exists on all Docker hosts — if you do not specify a different network, new containers automatically connect to it.
- Use custom bridge networks to control which containers can communicate between them, and to enable automatic DNS resolution from container name to IP address.
- Ensure that containers can connect to each other only if absolutely necessary, and avoid connecting sensitive containers to public-facing networks.
- Docker provides network drivers that let you create your own bridge network, overlay network, or macvlan network. If you need more control, you can create a Docker network plugin.
- Improve container isolation.
Protecting a container is exactly the same as protecting any process running on Linux. Ideally, the operating system on a container host should protect the host kernel from container escapes, and prevent mutual influence between containers.
- Set filesystem and volumes to Read only.
This can prevent malicious activity such as deploying malware on the container or modifying configuration.
docker run --read-only alpine
- Complete lifecycle management restrict system calls from Within Containers
- Monitor Container Activity. Analyze collected events to detect suspicious behaviourial patterns.
- Create an incident response process to ensure rapid response in the case of an attack.
- Apply automated patching
- Confirms Inmutability. Implement drift prevention to ensure container inmutability.
- Ensure you have robust auditing and forensics for quick troubleshooting and compliance reporting.
