The previous lessons in this chapter have focused on functional correctness. Our test suite, powered by GoogleTest and managed by CTest, answers the question: "Does my code do what it's supposed to do?"

But we generally want our build systems to automatically detect other problems too. For example:

Are there any obvious memory errors and undefined behaviors?
Does the code run efficiently?
Is the coding style clean, consistent, and maintainable?

This chapter expands our utility belt with tools that help us implement such checks. We'll survey three categories of automated analysis and learn how to integrate them into our CMake projects:

Dynamic Analysis with AddressSanitizer to find memory bugs at runtime.
Performance Analysis with Google Benchmark to measure our code's speed.
Static Analysis with Clang-Tidy to enforce good practices and find bugs before we even compile.

Finding Memory Errors with AddressSanitizer (ASan)

Dynamic analysis is the process of checking code for errors while it is running. The most powerful tools in this category are the sanitizers, which are special modes built into modern compilers like Clang and GCC.

The most famous of these is the AddressSanitizer (ASan). It instruments our code with extra checks to detect a wide range of memory errors the moment they happen, such as:

Buffer overflows (reading or writing past the end of an array).
Use-after-free (using a pointer to memory that has already been deallocated).
Use-after-return (using a pointer to a local variable after its function has returned).

These bugs are notoriously difficult to debug because they often cause crashes or unintended behaviors much later in the program's execution, far from the original error. ASan makes them trivial to find by stopping the program as soon as they happen, and providing a detailed stack trace.

Integrating ASan with CMake

Enabling ASan involves simply setting a compiler and linker flag: -fsanitize=address on GCC/Clang, or /fsanitize=address in MSVC.

To improve stack traces when an error is detected, we generally also want to provide the -fno-omit-frame-pointer and -g arguments for GCC/Clang, or /Zi on MSVC.

The best way to manage this is with a dedicated build type and preset. Let's create a module file to handle this:

cmake/Sanitize.cmake

1cmake_minimum_required(VERSION 3.23)
2
3set(SANITIZE_CONDITION "$<CONFIG:Sanitize>")
4
5if (MSVC)
6  set(
7    SANITIZE_FLAGS 
8    "$<${SANITIZE_CONDITION}:/fsanitize=address>"
9  )
10  add_compile_options("$<${SANITIZE_CONDITION}:/Zi>")
11else()
12  set(SANITIZE_FLAGS
13    "$<${SANITIZE_CONDITION}:-fsanitize=address>"
14    "$<${SANITIZE_CONDITION}:-fno-omit-frame-pointer>"
15    "$<${SANITIZE_CONDITION}:-g>"
16  )
17endif()
18
19add_compile_options(${SANITIZE_FLAGS})
20add_link_options(${SANITIZE_FLAGS})

And we'll include() it in our root CMakeLists.txt file:

CMakeLists.txt

1cmake_minimum_required(VERSION 3.23)
2project(Greeter)
3
4include(cmake/Coverage.cmake)
5include(cmake/Sanitize.cmake)
6
7add_subdirectory(app)
8add_subdirectory(greeter)
9
10enable_testing()
11add_subdirectory(tests)

Finally, let's add the corresponding presets to CMakePresets.json.

CMakePresets.json

1{
2  "version": 3,
3  "configurePresets": [
4    // ... other presets
5    {
6      "name": "sanitize",
7      "displayName": "Address Sanitizer (ASan)",
8      "inherits": "default",
9      "cacheVariables": {
10        "CMAKE_BUILD_TYPE": "Sanitize"
11      }
12    }
13  ],
14  "buildPresets": [
15    // ... other presets
16    {
17      "name": "sanitize",
18      "configurePreset": "sanitize"
19    }
20  ],
21  "testPresets": [
22    // ... other presets
23    {
24      "name": "sanitize",
25      "inherits": "default",
26      "configurePreset": "sanitize"
27    }
28  ]
29}

Seeing ASan in Action

Now, let's introduce a classic memory bug to the start of our greet() function in GreeterLib. We'll create a buffer of 5 bytes, then try to read index 5 of that buffer - ie, the byte at the 6th position:

greeter/src/Greeter.cpp

1#include <greeter/Greeter.h>
2#include <iostream>
3
4// ...
5
6std::string Greeter::greet() const {
7  char buffer[5];
8  std::cout << buffer[5];
9  
10  // ...
11}

ASan can only detect a memory issue if we actually cause that issue to occur when running our ASan-enabled build. For this reason, we get the most benefit out ASan when we combine it with a test suite that covers as much of our code as possible.

The test suite executes as many functions and branches of our code as possible, whilst ASan is keeping watch to detect if any of those actions cause memory issues.

First though, let's run our tests against our standard build, without ASan enabled. As long as our memory issue doesn't cause the program to crash, our test suite will likely pass:

1cmake --preset default

1cmake --build --preset default

1ctest --preset default

1...
2100% tests passed, 0 tests failed out of 4
3...

Let's try again, but this time with ASan enabled.

Note if you're using GCC in a MinGW environment, such as MSYS2 UCRT64, these steps may not work. We explain why this is and how to address it at the end of this section.

On other environments, we can delete our existing build directory, and then configure, build, and test our project from scratch, but this time using our new sanitize preset:

1cmake --preset sanitize

1cmake --build --preset sanitize

1ctest --preset sanitize

This time, the test should fail with a detailed report from ASan. This will be a huge dump of information containing everything we might need to debug a complex issue.

For our simple program, some of the key output is shown below, where it tells us the type of error that was detected (stack-buffer-overflow) and points us to exactly where it happened:

1ERROR: AddressSanitizer: stack-buffer-overflow on address 0x008d920fe905
2
3...
4
5Address 0x008d920fe905 is located in stack of thread T0 at offset 101 in frame
6    #0 0x7ff6a39dc73f in Greeter::greet()
7
8  This frame has 2 object(s):
9    [32, 56) 'day_str' (line 18)
10    [96, 101) 'buffer' (line 25) <== Memory access at offset 101 overflows this variable
11
12...
13
140% tests passed, 4 tests failed out of 4

ASan on Windows (MSYS2 / MinGW)

At the time of writing, ASan is not supported by GCC in MinGW-w64 environments, including MSYS2 UCRT64.

If you want to follow along, you can use the MSYS64 CLANG64 environment instead of UCRT64. Both are included with MSYS64 installations.

Each MSYS64 environment is isolated, so you will need to install the required tools on this new environment if you haven't already.

This can be done in the same way we did previously via pacman, ensuring we're installing the clang variations instead of ucrt64. The following commands install the compiler toolchain, CMake, and the Ninja build system:

1pacman -Syu

1pacman -S mingw-w64-clang-x86_64-toolchain \
2  mingw-w64-clang-x86_64-cmake \
3  mingw-w64-clang-x86_64-ninja

The CLANG64 shell uses the Clang compiler rather than GCC. But, as long as we've kept our build scripts portable, the project should be configurable, buildable, and testable in exactly the same way:

1cmake --preset sanitize

1cmake --build --preset sanitize

1ctest --preset sanitize

Configuring ASan Runtime Behavior

AddressSanitizer's default settings find many common memory errors, but some more advanced checks are disabled by default because they can have a performance impact or are specific to certain bug classes. You can enable these checks at runtime to get even more detailed analysis. This is done by setting the ASAN_OPTIONS environment variable before running an instrumented executable.

A classic and dangerous C++ bug is a use-after-return. This happens when a function returns a pointer to one of its local variables. Once the function returns, its stack frame is destroyed, and the memory that the variable occupied is now free to be reused. The returned pointer is now "dangling," and any attempt to use it results in undefined behavior.

The ASan check for this, detect_stack_use_after_return, is disabled by default in many configurations. Let's write a test to demonstrate this bug and see how we can configure ASan to detect it.

Creating a "Use-After-Return" Bug in a Test

We'll add a new test case to tests/greeter/test_greeter.cpp. This test will contain its own helper function that intentionally creates this bug.

tests/greeter/test_greeter.cpp

1#include <gtest/gtest.h>
2#include <greeter/Greeter.h>
3
4// ... existing tests
5
6const char* getDanglingPointer() {
7  char local_buffer[] = "This is temporary";
8
9  // DANGER: returning pointer to local variable
10  return local_buffer; 
11}
12
13TEST(
14  ASanRuntimeOptionTests,
15  FailsToDetectUseAfterReturnByDefault
16) {
17  const char* dangling_ptr = getDanglingPointer();
18
19  // The memory dangling_ptr points to is no longer valid.
20  // Accessing it is undefined behavior.
21  char first_char = dangling_ptr[0]; // <w>
22
23  // This assertion might pass by chance if
24  // the memory hasn't been overwritten.
25  EXPECT_NE(first_char, '\0');
26}

First, let's run this test with our sanitize preset and the default ASan options.

1cmake --preset sanitize

1cmake --build --preset sanitize

1ctest --preset sanitize

In most environments, this test will pass. The undefined behavior doesn't cause an immediate crash, and since the specific check is disabled, ASan remains silent.

1...
2100% tests passed, 0 tests failed out of 5
3...

Enabling the Check in CMake

Now, let's enable the detect_stack_use_after_return check. We'll set the ASAN_OPTIONS environment variable. We can do this in several ways. One option is to set it system-wide or for our terminal session, using techniques we covered earlier.

We can also apply it when a CMake preset is being used via the environment key:

CMakePresets.json

1{
2  "version": 3,
3  "configurePresets": [/*...*/],
4  "buildPresets": [/*...*/],
5  "testPresets": [{
6    "name": "sanitize",
7    "inherits": "default",
8    "configurePreset": "sanitize",
9    "environment": {
10      "ASAN_OPTIONS": "detect_stack_use_after_return=1"
11    }
12  }]
13}

Alternatively, we can apply it to a set of tests found by a gtest_discover_tests() invocation:

tests/greeter/CMakeLists.txt

1cmake_minimum_required(VERSION 3.23)
2find_package(GTest REQUIRED)
3
4add_executable(GreeterLibTests test_greeter.cpp)
5
6target_link_libraries(GreeterLibTests PRIVATE
7  GreeterLib
8  GTest::gtest
9  GTest::gmock
10  GTest::gmock_main
11)
12
13gtest_discover_tests(GreeterLibTests PROPERTIES
14  LABELS "library"
15  ENVIRONMENT 
16    "ASAN_OPTIONS=detect_stack_use_after_return=1" 
17)

After setting this environment variable using any of these approaches, we can rebuild and run the tests again:

1cmake --preset sanitize

1cmake --build --preset sanitize

1ctest --preset sanitize

This time, the test should fail. ASan, with the check now enabled, intercepts the invalid memory access and provides a detailed report:

1ERROR: AddressSanitizer: stack-use-after-return on address 0x020481232e20
2...
3Address 0x020481232e20 is located in ... getDanglingPointer()
4...
580% tests passed, 1 tests failed out of 5

This is just one example of the many ways you can customize ASan's behavior. For a full list of options and detailed explanations of the errors ASan can find, the official AddressSanitizer documentation is an excellent resource.

Summary

In this lesson, we've seen how to integrate dynamic analysis into our build process to catch a whole class of memory bugs that are often missed by standard testing.

Dynamic Analysis: Sanitizers like ASan find errors by instrumenting your code to perform checks as it runs.
AddressSanitizer (ASan): The most common sanitizer, it excels at finding memory errors like buffer overflows and use-after-free bugs.
CMake Integration: We enabled ASan by adding the -fsanitize=address flag and created a dedicated Sanitize build type and preset to manage it.
Clear Error Reports: When ASan detects an error, it immediately terminates the program and provides a detailed stack trace, making bugs easy to locate and fix.

Using AddressSanitizer (ASan)

Finding Memory Errors with AddressSanitizer (ASan)

Integrating ASan with CMake

cmake/Sanitize.cmake

CMakeLists.txt

CMakePresets.json

Seeing ASan in Action

greeter/src/Greeter.cpp

Configuring ASan Runtime Behavior

Creating a "Use-After-Return" Bug in a Test

tests/greeter/test_greeter.cpp

Enabling the Check in CMake

CMakePresets.json

tests/greeter/CMakeLists.txt

Summary

Using Google Benchmark

Managing C++ Projects Using CMake