Slow Scan Command in Containers

Problem

When running the Golang agent scan command on your Go-based project in a containerized environment without a local cache, the Go toolchain must repeatedly download all module dependencies from the internet, which can significantly slow down builds. This also leads to increased CPU usage and memory consumption during repeated dependency downloads.

Cause

As part of its instrumentation step in the scan command, the Go agent performs a full compilation and therefore must resolve all project dependencies beforehand. The lack of a build cache means that each new container running the scan command starts from scratch, fetching all necessary modules.

This process repeats for every build without caching, causing heavy network usage and long compile times.

Solution

By combining pre-fetching with a shared cache, you drastically reduce both build times and network overhead, keeping container builds faster, more consistent, and less dependent on external network availability.

Pre-Fetch Dependencies

Starting with Go 1.16, run go mod download to download and cache all dependencies ahead of time. This streamlines subsequent steps and surfaces any network or dependency issues sooner.

Use a Shared Cache

Mount a Directory for File Caching

Mount a volume inside your container to store the module downloads and build artifacts. By preserving this directory across builds, you no longer need to re-download dependencies every time.

To persist the cache across builds, mount a host directory into the container using Docker’s -v option:

docker run --rm \
  -v $(pwd)/cache:/cache \
  -v $(pwd):/app \
  my-go-image

Or with Docker Compose:

services:
  app:
    image: my-go-image
    volumes:
      - ./cache:/cache
      - ./:/app

When working with multiple branches or different Go versions in the same environment, consider isolating cache directories per branch or version to avoid conflicts or unexpected behavior due to incompatible module or build artifacts.

Configure Go to Use the Mounted Cache

Then, configure Go’s environment variables GOMODCACHE (module cache) and GOCACHE (build cache) to point to directories inside the mounted /cache folder. For example:

FROM golang:1.20 AS builder

ENV GOMODCACHE=/cache/go-mod
ENV GOCACHE=/cache/go-build

WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download

COPY . .
# Download and execute Golang agent commands

The mounted /cache directory will then contain Pre-fetched Go modules (from go mod download) and Build artifacts (from go build). This setup enables faster, more efficient builds by leveraging persistent caching.