Assertions and Parameterized Tests

Sometimes, a simple equality or relational check isn't enough. You might want to verify that a string has a certain format, that a collection contains a specific element, or that a value falls within a complex range.

We could implement each of these checks manually within our test function, but more commonly, test libraries like Google Test allow us to define matchers that integrate with their APIs more elegantly.

Matchers are objects that represent a predicate. In Google Test, they are used with the EXPECT_THAT(value, matcher) assertion, which reads like natural language: "I expect that this value matches this condition."

We can create any matcher (predicate function) that we might need, but The Google Mock library, which is bundled with Google Test, includes a large collection of generally useful examples.

To use them, we typically need to include the <gmock/gmock.h> header and link against the GTest::gmock library.

Let's update tests/CMakeLists.txt to include it.

1cmake_minimum_required(VERSION 3.23)
2find_package(GTest REQUIRED)
3
4add_executable(GreeterTests main.cpp)
5
6target_link_libraries(GreeterTests PRIVATE
7  GreeterLib
8  GTest::gtest
9  GTest::gmock # Add the gmock library 
10  GTest::gtest_main
11)
12
13gtest_discover_tests(GreeterTests)

Here are a few of the many built-in matchers that Google Mock provides:

• testing::Eq(value): Equivalent to ==.
• testing::Gt(value): Greater than (>).
• testing::StartsWith(str): Checks if a string starts with a given prefix.
• testing::EndsWith(str): Checks if a string ends with a given suffix.
• testing::ContainsRegex(regex): Checks if a string contains a substring that matches a regular expression.
• testing::Each(matcher): Checks if every element in a container matches an inner matcher.
• testing::AllOf(m1, m2, ...): A logical AND; succeeds if all nested matchers succeed.

The official docs contain the full list of available matchers.

Practical Example

In our parameterized test, we checked for the exact number of exclamation marks. This is a bit brittle; what if we only care that the greeting has the correct base string and ends with some number of exclamation marks? A matcher is perfect for this.

Let's add a test that uses a regular expression matcher to verify the format:

1#include <gtest/gtest.h>
2#include <gmock/gmock.h> // Include for matchers 
3#include <greeter/Greeter.h>
4#include <tuple>
5
6class GreeterTest : public testing::Test {
7protected:
8  Greeter my_greeter;
9};
10
11TEST_F(GreeterTest, GreetHasCorrectFormat) {
12  my_greeter.setEnthusiasm(0.8f);
13  EXPECT_THAT(
14    my_greeter.greet(),
15    testing::MatchesRegex("Hello from the Greeter class!*")
16  );
17}

The * character is a special regex token that matches any quantity (including 0) of the proceeding token. We have dedicated lessons on regular expressions in our C++ course.

In this example, our test is ensuring that the function returns "Hello from the Greeter class" followed by any number of exclamation marks.

1cmake --build --preset default

1ctest --preset default

1Test project D:/Projects/cmake/build
2    Start 1: GreeterTest.GreetHasCorrectFormat
31/1 Test #1: GreeterTest.GreetHasCorrectFormat ...   Passed    0.01 sec
4
5100% tests passed, 0 tests failed out of 1
6
7Total Test time (real) =   0.04 sec

This test is more robust to small, insignificant changes in the implementation of greet().

Let's go back to our previous example where we were testing the exact output of greet(), not just testing the general format.

The exact behavior that depends an externally-provided parameter - the enthusiasm level. What if we want to test the greet() method for several different enthusiasm levels? We could write a separate test for each one:

1TEST_F(GreeterTest, GreetsWithNoExclamations) {
2  my_greeter.setEnthusiasm(0.1f);
3  EXPECT_EQ(
4    my_greeter.greet(),
5    "Hello from the Greeter class"
6  );
7}
8
9TEST_F(GreeterTest, GreetsWithOneExclamation) {
10  my_greeter.setEnthusiasm(0.4f);
11  EXPECT_EQ(
12    my_greeter.greet(),
13    "Hello from the Greeter class!"
14  );
15}
16// ... and so on ...

This is repetitive and doesn't scale up to real-world scenarios where we might want to test dozens or hundreds of different parameter combinations. The solution is a parameterized test. This allows you to run the same test logic with a wide range of different input values.

Let's refactor our enthusiasm tests into a single parameterized test. The setup involves four steps.

Step 1: Use TestWithParam

First, we'll update our a test fixture class to inherit from testing::TestWithParam<T>, instead of testing::Test.

The template parameter T in TestWithParam is the type of our parameter. We often use a std::tuple for this, as our parameter typically contain multiple values of different types.

In our case, our tuple will contain a float representing the enthusiasm_ level, and a std::string representing what we expect our function to return given that enthusiasm_ level:

1#include <gtest/gtest.h>
2#include <greeter/Greeter.h>
3#include <tuple>
4
5class GreeterEnthusiasmTest :
6  public testing::TestWithParam<
7    std::tuple<float, std::string>
8  >
9{
10protected:
11  Greeter my_greeter;
12};
13
14// ...

Step 2: Writing a Parameterized Test with TEST_P

Next, we use the TEST_P macro (the 'P' is for Parameterized) instead of TEST_F.

Inside the test, we can access the current parameter value with the GetParam() method.

We'll pull our enthusiasm and expected greeting from our tuple. We'll then set our enthusiasm to that level, and assert that the greet() method then returns the expected string:

1#include <gtest/gtest.h>
2#include <greeter/Greeter.h>
3#include <tuple>
4
5class GreeterEnthusiasmTest :
6  public testing::TestWithParam<
7    std::tuple<float, std::string>
8  >
9{
10protected:
11  Greeter my_greeter;
12};
13
14TEST_P(
15  GreeterEnthusiasmTest,
16  GreetReturnsCorrectExclamations
17) {
18  auto [enthusiasm, expected_greeting]{GetParam()};
19  my_greeter.setEnthusiasm(enthusiasm);
20  EXPECT_EQ(my_greeter.greet(), expected_greeting);
21}
22
23// ...

Step 3: Instantiating Tests

Finally, we just need to instantiate our tests with different parameter values.

We use the INSTANTIATE_TEST_SUITE_P macro for this, passing a name as the first argument, our TEST_P's name as the second argument, and the set of parameters we want to test as the third.

The testing::Values() function helps us to provide these parameters. This allowing us to now instantiate each test simply by constructing a parameter - often a single line of code:

1#include <gtest/gtest.h>
2#include <greeter/Greeter.h>
3#include <tuple>
4
5class GreeterEnthusiasmTest :
6  public testing::TestWithParam<
7    std::tuple<float, std::string>
8  >
9{
10protected:
11  Greeter my_greeter;
12};
13
14TEST_P(
15  GreeterEnthusiasmTest,
16  GreetReturnsCorrectExclamations
17) {
18  auto [enthusiasm, expected_greeting]{GetParam()};
19  my_greeter.setEnthusiasm(enthusiasm);
20  EXPECT_EQ(my_greeter.greet(), expected_greeting);
21}
22
23INSTANTIATE_TEST_SUITE_P(
24  EnthusiasmLevels,
25  GreeterEnthusiasmTest,
26  testing::Values(
27    std::make_tuple(0.0f, "Hello from the Greeter class"),
28    std::make_tuple(0.3f, "Hello from the Greeter class"),
29    std::make_tuple(0.4f, "Hello from the Greeter class!"),
30    std::make_tuple(0.7f, "Hello from the Greeter class!!"),
31    std::make_tuple(1.0f, "Hello from the Greeter class!!!")
32  )
33);

Step 4: Running the Tests

When we run our tests now, CTest and Google Test will automatically generate and run five separate test cases based on this one definition, each with a different set of inputs from the Values list.

1cmake --build --preset default

1ctest --preset default

1...
2100% tests passed, 0 tests failed out of 5
3...

This lesson introduced most of the core features we need to write tests.

• Assertions: We learned the difference between fatal ASSERT_* and non-fatal EXPECT_* assertions, and saw specialized assertions for floating-point numbers (EXPECT_NEAR) and exceptions (ASSERT_THROW).
• Parameterized Tests: We saw how to use TEST_P and INSTANTIATE_TEST_SUITE_P to eliminate repetitive code by running the same test logic against a large set of input data.
• Custom Matchers: We learned to use EXPECT_THAT with Google Mock's library of matchers to create expressive assertions for complex conditions, such as matching a regular expression.