Touchpoints: Security Testing

This post is part of a series on DevSecOps and CI/CD security. Check out the overview for context and links to the rest of the series.

Testing is a core part of software development and CI/CD. This post explores software testing for DevSecOps:

• Security Unit Tests
• Security Integration Tests
• Behavior-driven Development Frameworks

Security Unit Tests

Unit tests run quickly, testing fine-grained behavior at the individual function or method level. They’re mostly self-contained, and don’t depend on specific states of external systems. Unit testing can happen in multiple places:

• Local development environments (triggered manually or via git hooks, in the editor or command line)
• Build environments (triggered manually or via pushes/merges, executed by a CI server)

Security unit tests validate fine-grained behavior. For example:

• Logging: sensitive application operations like login attempts, password resets, fund transfers, or profile updates require logging. Unit tests for these operations might check that logs are generated, and include security-relevant details such as timestamp, user identifier, and session duration. They might also validate that the logs don’t contain sensitive information like session tokens or passwords.
• Input validation: test that invalid and malicious values are rejected from user-controlled (attacker-controlled) fields. For example, an order quantity field should be an integer >= 0. The function accepting input for this field might perform a type check and numeric comparison. A unit test for this function should confirm rejection of invalid or malicious values such as -2, .725, or AaBbCc'";[]{}.
• Authorization: test that all administrative functionality can only by invoked by authenticated, properly-privileged users. For a multi-user application with role-based access control, an UpdateRoleMembership() function that alters role membership need to be restricted to users with the Administrator role and an active session. Logic in the UpdateRoleMembership() function confirms the calling user has an active session, and that the calling user is a member of the Administrator role. Three unit tests for this:
  • Invoke UpdateRoleMembership() from an unauthenticated/anonymous session: the function should return an HTTP 401/403
  • Invoke UpdateRoleMembership() from an authenticated session of a user without the Administrator role: the function should return an HTTP 401/403
  • Invoke UpdateRoleMembership() from an authenticated session of a user with the Administrator role: the function should succeed with an HTTP 200, and role membership should be updated

  Depending on how the application implements session management and authorization checks (internally vs. relying on external systems), authorization tests could be considered unit or integration tests.

OWASP provides detailed guidance on security unit testing.

Some quick-running tests or tools from BDD frameworks (discussed below) can also be adapted for unit testing.

Security Integration Tests

Some security requirements involve code that touches multiple systems, calls third-party APIs, or crosses network boundaries. Integration testing tests code that interacts with external systems.

Generally, integration tests take longer than unit tests and test multi-step, more complex behavior. They typically require some setup before execution: stuff like external databases, cloud resources, networking configuration, or service account credentials need to be in the state that the integration test expects. There’s a natural pairing between integration tests and using infrastructure-as-code to place external systems in the desired state for testing.

Integration testing can happen in multiple places:

• In an integration/staging environment, executed by a CI pipeline
• In a local development environment that’s mocking a production-like environment (using containers or virtual machines)

Integration testing extends to security. Longer-running integration tests are well-suited to run at particular events: merging a feature branch, before a significant release, or periodically during off-hours.

Some examples of security integration tests:

• Session invalidation: test that when a user logs out of a web application, the session is fully destroyed. For example: database or cache entries of active sessions are cleared, and making authenticated requests with the invalidated session token fail with an HTTP 401 or redirect to the login page. Authenticated application functionality like fund transfers, payments, or accessing personal data should be impossible without a valid session.
• Account lockout: test that your application implements account lockout (blocking credential-stuffing or brute-force attacks). For example, a test might make 10 invalid login attempts in quick succession to the same account, followed by a valid login attempt. The valid login attempt should fail until the account is unlocked.
• Encryption in transit: validate that the application is served using TLS (HTTPS) with a strong configuration. This includes protocol and cipher suite selection, using the HTTP Strict-Transport-Security header, and configuring the application to be unavailable over HTTP (or implementing a 301 HTTP -> HTTPS redirect). SSLyze is handy for this, and supports CI/CD.

Behavior-driven Development Frameworks

Unfortunately, as of this writing in 2023 the two main behavior-driven development (BDD) security frameworks are not actively maintained. Security engineering effort is better spent in other places. However, this tool category holds promise so I’ve included it for reference.

BDD security frameworks (BDD-Security and Gauntlt) fit naturally with security integration testing. These tools translate security requirements into executable tests that run against deployed web applications or APIs. They don’t require access to source code, making them programming language and framework agnostic. This also makes them less precise than custom-written tests or static analysis tools. The tests can run locally, or through a CI server in a build or integration environment.

BDD-Security and GauntLt:

1. Use Gherkin syntax to translate security requirements into executable tests, and define aspects of test execution:
   • Target selection: hosts/ports to run tests against
   • Test/attack selection: which tools to run and their parameters
   • Test criteria: expected tool output, pass/fail conditions for a given test
   • False positive filtering: what tool output indicates a false positive
2. Wrap existing tools like OWASP ZAP, Selenium, nmap, sqlmap, SSLyze, and others to execute the test definitions. Tests execute over the network.
3. Report test results: GauntLt supports Unix-style exit codes and sends results to STDOUT for further processing. It also supports HTML reports. BDD-Security supports HTML, JSON, and XML reports.

Investing time in tuning BDD security frameworks to your application and security requirements is key to realize their value. Some tools wrapped by BDD frameworks require per-application tuning, such as:

• Credentials: application username/password
• Scoping: spidering, application route inclusion/exclusion
• Authentication details: application login/logout URLs for authenticated scanning
• Inactive session detection: detection patterns in HTTP responses indicating active vs. inactive sessions
• Rate-limiting: how quickly to send HTTP requests, serially or in parallel

BDD-Security has two significant advantages over GauntLt:

1. It supports Selenium WebDriver for ad-hoc HTTP requests. Selenium automates actions in a real browser, resulting in more realistic HTTP traffic and server-side behavior compared with GauntLt’s curl adapter for ad-hoc HTTP requests.
2. It generates JSON and XML reports. This is especially useful for filtering false positives and integrating with issue trackers.

Thanks for reading. If you like what you read, check out the next post in the series on patching and the software supply chain.