Vulnerability Management

An unfortunate reality of the process of developing, providing, and running software is that security vulnerabilities will exist. For this reason, we have established a vulnerability management program that strives to identify vulnerabilities in our applications and infrastructure early and often.

Detection

We aim to detect vulnerabilities through many means which include internal testing, external testing, and automated scanning. The security team oversees the configuration and enforcement of these detection capabilities. Some of the solutions implemented to ensure broad coverage are:

• Host scans - We use the Wiz Unified Vulnerability Management platform to scan all hosts in our network for vulnerabilities and compliance with our secure configuration baselines - based upon the Center for Internet Security (CIS) Level 2 benchmarks.
• Database scans - We use Wiz to continuously monitor our cloud database services for security risks, including misconfigurations, unintended exposure, and drift from our secure configuration baselines. In addition, AWS and GCP monitor and manage vulnerability and compliance controls for cloud-provider–managed components in their respective environments, consistent with the shared responsibility model.
• Web Application & External Scans - All internet-facing APIs and web applications are scanned using ZAP by Checkmarx. Scans are performed with authenticated access and include both passive and active testing to evaluate application functionality and identify potential security vulnerabilities.
• Penetration Testing - We employ independent external penetration testers to perform targeted testing of our infrastructure and applications at least annually (and typically as a part of our FedRAMP assessment).
• Bug Bounty Program - We run a bug bounty program through Bugcrowd for continuous penetration testing of our systems. Read more about the program on our bug bounty page.

Further, we have a variety of tools that aid in vulnerability detection, but are primarily implemented in a manner to help us prevent vulnerabilities from even being introduced. Read about them below.

Prevention

As described in the secure development document, we implement a number of security checks into the development process in order to prevent vulnerabilities from being introduced to our systems and software. Some of the tooling we use to implement these checks are:

• Software composition analysis - We use GitHub Advanced Security Dependency Review in our CI/CD pipelines to identify vulnerabilities in open-source or third-party software packages used in our applications. The scanner checks all dependencies against a vulnerability database and will block builds if any vulnerabilities exist. We also continually monitor our established dependencies using Dependabot.
• Static code analysis - We use CodeQL in our CI/CD pipelines to perform static code analysis which flags code vulnerabilities and security hot-spots in addition to code quality checks. Quality gates have been established which require all security hot-spots to be reviewed before allowing a build to succeed.
• Secret scanning - We use GitHub Advanced Security to identify committed or leaked secrets (e.g., API keys, tokens, and credentials) in source code and pull requests. Identified secrets are blocked or remediated prior to merge, and exposed credentials are rotated in accordance with our incident response procedures.
• Infrastructure-as-Code (IaC) scanning - We integrate the Wiz CLI into our CI/CD pipeline to scan Terraform plans for security misconfigurations (consistent with CIS Level 2 requirements) before deployment. This includes controls to prevent unintended exposure and to enforce secure configuration baselines (e.g., blocking public S3 buckets or ensuring databases are encrypted).

By integrating all of these tools into our CI/CD pipelines, we significantly reduce the likelihood of service being deployed with new vulnerabilities present.

Additonally, an SBOM is generated for each container image using Syft (the Wiz CLI can also generate an SBOM). The container image and its associated build artifacts—including the SBOM and a Cosign signature—are stored together in the container registry. During deployment, the Kyverno admission controller verifies both the image signature and the SBOM to detect tampering and block untrusted images from being admitted to the cluster.

Resolution

Resolution of identified vulnerabilities is integrated with our software development processes. Vulnerabilities introduced during the development process are expected to be resolved before code is merged. Where vulnerabilities cannot be resolved immediately (e.g., recently disclosed open source dependencies), tasks are generated for remediation assigned to the appropriate development team with due dates set in accordance with our resolution timelines. Vulnerabilities identified through those means described in detection are prioritized based on their severity level. Severity levels are based on the CVSS v3 scores. Tickets are generated for the appropriate team with due dates set in accordance with our resolution timelines.

We make our best effort to resolve vulnerabilities based on severity within the following resolution times from identification:

Severity	Resolution Time
High	30 days
Moderate	90 days
Low	180 days

The information security team provides oversight of the vulnerability remediation process by coordination with our product and engineering managers to ensure tasks are prioritized against our product road map appropriately.