Since we are building security products, following a strict development lifecycle is paramount to release quality software. Security checks are inserted into each phase of development from coding to testing to deploying our services.
Annually we require all of our developers to complete training courses on the latest secure coding practices. Training covers topics such as the Open Web Application Security Project (OWASP) Top 10 and how to prevent those risks when designing and coding solutions.
No code gets merged without a peer review and code changes to critical code bases require reviews by the internal experts of the high impact functions. This is true of our application code changes and infrastructure changes (we manage all of our infrastructure as code).
Before code is even peer reviewed, automated static code analysis is performed on each requested code change through CI/CD pipelines. These checks will identify problems with the quality of the code, security issues presented by the changes, and vulnerabilities in third-party code through software composition analysis. CI/CD checks also include unit testing to ensure code changes do not introduce bugs.
As changes are deployed to pre-production environments, the security team runs scheduled dynamic security scans on the systems to detect vulnerabilities in the systems when in use. New issues detected are reported back to the developers for remediation before code is promoted to production environments. In addition, a full suite of integration tests are run and manual QA testing is performed, as appropriate.
Once code has passed the various automated and manual checks, CI/CD pipelines handle the deployment of the changes. All of our services run on immutable infrastructure, so no changes are ever applied directly on running services by an engineer. Deployment pipelines will deploy a single or small set of the modified application and run checks to verify that service is healthy and operating as expected before replacing all instances of the running service with the updated code.