Regex Capture Groups

In this lesson, we’ll further build our regex knowledge, introducing capture groups, non-capture groups, and lazy selectors. Our previous lesson covered a broad introduction of regex, and how to use it to determine if a string matches our criteria.

Here, we will focus on a more advanced use case, where we want to extract information from the strings we receive.

Use cases for this include:

• Extracting data from a string, such as retrieving the domain name from an email address
• Reformatting a string, such as changing the order of date elements from MM/DD/YYYY to YYYY/MM/DD
• Finding and replacing specific parts of a string, such as replacing all instances of a word with a different word while preserving the rest of the string

Finally, we’ll see how we can make use of these regex concepts in C++

This is intended to be a follow-up to our introductory lesson on Regular Expressions in C++, so familiarity with the concepts covered there is recommended:

Regular Expressions

An introduction to regular expressions, and how to use them in C++ with the standard library's regex, regex_match, and regex_search

Capture Groups

In regular expressions, capture groups are defined by parenthesis: ( and ). For example, if we wanted to capture all of the times "hello" appears in our string, our regex would look like this:

1(hello)

If we wanted to capture only the "hello"s that proceed " world", it would look like this:

1(hello) world

Within the capture group, we still have all of our usual regex powers, for example:

• (hello|goodbye) - Capture hello or goodbye
• ([hc]ello) - Capture hello and cello
• ([hc]ello?) - Capture hell, hello, cell and cello
• (hel*o) - Capture heo, helo, hello, etc
• (hel\w) - Capture hell, help, etc
• hello(.*)world - Capture everything between hello and world

Just like the order of operations in maths and programming can be manipulated using brackets, so too can the order of operations within regex.

Operators like ? and * can be applied to a capture group, while | within a capture group constrains its effects to that group

1✔️ The( big)? cat -> The cat
2✔️ The( big)? cat -> The big cat
3❌ The( big)? cat -> The big big cat
4
5✔️ The( big)* cat -> The cat
6✔️ The( big)* cat -> The big cat
7✔️ The( big)* cat -> The big big cat
8
9✔️ The (big|chonky) cat -> The big cat
10✔️ The (big|chonky) cat -> The chonky cat
11❌ The (big|chonky) cat -> The big chonky cat
12
13❌ The( big| chonky)+ cat -> The cat
14✔️ The( big| chonky)+ cat -> The big cat
15✔️ The( big| chonky)+ cat -> The big chonky cat
16✔️ The( big| chonky)+ cat -> The big chonky big cat

Note, if just want to manipulate the order of operations, and have no need to capture the content of our group, we can use a non-capture group instead. These are introduced a little later in this lesson.

1Creating a capture group:
2❌ The (big) cat -> The (big) cat
3
4Searching for a pattern containing brackets
5✔️ The \(big\) cat -> The (big) cat

Greedy and Lazy Quantifiers

Let's imagine we want to get a breakdown of what email providers our users have signed up with. We have a list of emails, like example@gmail.com, and we want to generate a list of email providers, like gmail.

Our first attempt at a regular expression might be to capture everything from the @ to the literal .:

1@(.*)\.

Some basic testing would indicate this works - given a string like example@gmail.com, the substring of gmail is captured as intended.

However, given the string example@gmail.co.uk, the substring of gmail.co is captured.

This is because, by default, repetition quantifiers such as *, +, and {x,y} are greedy. They capture as much as possible.

In this example, the .* in (.*)\. is capturing everything until the last period in the string, not the next period.

We can change this by appending a ? after the quantifier, thereby making it lazy:

1@(.*?)\.

This also applies to the other quantifiers. The following lists what would be captured by various regular expressions, given the string of 54321!.

We apply different quantifiers to the \d character, to remove some of the digits from what is ultimately captured:

1* matches as many repetitions as possible
2✔️ \d*(.*) => 54321! = !
3
4*? matches as few repetitions as possible
5✔️ \d*?(.*) => 54321! = 54321!
6
7+ matches as many repetitions as possible, but at least 1
8✔️ \d+(.*) => 54321! = !
9
10+? matches as few repetitions as possible, but at least 1
11✔️ \d+?(.*) => 54321! = 4321!
12
13{2,4} matches 2, 3 or 4 repetitions, preferring more
14✔️ \d{2,4}(.*) => 54321! = 1!
15
16{2,4}? matches 2, 3 or 4 repetitions, preferring fewer
17✔️ \d{2,4}?(.*) => 54321! = 321!

Non-Capture Groups

The ability to create groups of tokens within our regular expression is generally useful, even if we don’t need to capture them. For these, we have non-capture groups. Non-capture groups start with (?: and end with ).

They have many use cases, including the following examples:

Applying the optional operator (?) to sub-patterns

Below, we make the string "brown "  optional:

1✔️ The (?:brown )?fox -> The fox
2✔️ The (?:brown )?fox -> The brown fox
3❌ The (?:brown )?fox -> The red fox

Applying the repetition quantifiers (``, +, and {}) to sub-patterns

Below, we allow the string "red " to appear 0-2 times:

1✔️ The (?:red ){0,2}fox -> The fox
2✔️ The (?:red ){0,2}fox -> The red fox
3✔️ The (?:red ){0,2}fox -> The red red fox
4❌ The (?:red ){0,2}fox -> The red red red fox

Manipulating the order of operations

The following example shows how we can use a non-capturing group to manipulate which part of the pattern the alternation operator | applies to:

1Default order of operators
2✔️ The brown|red fox -> The brown
3✔️ The brown|red fox -> red fox
4
5Controlling it using a non-capture group
6❌ The (?:brown|red) fox -> The brown
7❌ The (?:brown|red) fox -> red fox
8✔️ The (?:brown|red) fox -> The brown fox
9✔️ The (?:brown|red) fox -> The red fox

Using Capture Groups in C++

The rest of this lesson will focus on how we can use capture groups within the C++ standard library’s regex helpers, available by including <regex>

std::match_results and std::smatch

The std::match_results<> template class is typically how we want to store the output of our regex operations. This is templated so it can be used with different types of strings.

However, an instance of the template class that works with std::string has already been aliased for us. It is called std::smatch, which is what we'll be using here.

The std::regex_search() function has an overload that accepts a std::match_results object as the second argument:

1#include <regex>
2
3int main() {
4  std::string Input{"Hello There"};
5  std::regex Pattern{"Hello There"};
6  std::smatch Match;
7  std::regex_search(Input, Match, Pattern);
8}

We covered the basics of std::regex_search() in our introductory lesson. Here, we’ll focus on its interaction with std::match_results objects and capture groups.

Each object in the match results is a std::sub_match. These objects contain some useful information about the sub-match that was found, as well as the matched string, which can be accessed using the str() method.

If the std::regex_search() call was successful, the std::smatch will contain at least one sub-match: the substring that matched the entire regex pattern we provided:

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{"Hello There"};
6  std::regex Pattern{"Hello There"};
7  std::smatch Match;
8
9  if (std::regex_search(Input, Match,
10                        Pattern)) {
11    std::cout << Match.size()
12              << " sub-match found!";
13    for (auto Submatch : Match) {
14      std::cout << "\nSubmatch: " << Submatch;
15    }
16  }
17}

11 sub-match found!
2Submatch: Hello There

When we’re not using capture groups, our std::match_results container will only contain one std::sub_match. However, when our regex contains capture groups, what was captured by those capture groups will be included in the std::smatch collection.

The overall match will be at index 0, what was captured by the first capture group will be at index 1, the second group at index 2, and so on

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{"Hi All"};
6  std::regex Pattern{"(Hello|Hi) (There|All)"};
7  std::smatch Match;
8
9  if (std::regex_search(Input, Match,
10                        Pattern)) {
11    std::cout << Match.size()
12              << " submatches found!";
13    for (auto Submatch : Match) {
14      std::cout << "\nSubmatch: " << Submatch;
15    }
16  }
17}

13 submatches found!
2Submatch: Hi All
3Submatch: Hi
4Submatch: All

The std::smatch has some additional properties and methods we may find useful. For example, the position() method accepts an integer parameter and will return the starting position of the corresponding sub-match within the input string.

Additionally, the std::sub_match objects have fields and properties we may find useful, including:

• length() - the length of the sub-match string
• first() - an iterator to the first character in the sub-match
• last() - an iterator to the last character in the sub-match

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{"Hello World"};
6  std::regex Pattern{"Hello (.*)"};
7  std::smatch Matches;
8
9  if (std::regex_search(Input, Matches,
10                        Pattern)) {
11    std::cout << Matches.size()
12              << " submatches found!";
13    for (size_t i{0}; i < Matches.size(); ++i) {
14      std::cout << "\n\nSubmatch " << i << ": "
15                << Matches[i] << "\n  Length: "
16                << Matches[i].length()
17                << "\n  First Character: "
18                << *Matches[i].first
19                << "\n  Position: "
20                << Matches.position(i);
21    }
22  }
23}

12 submatches found!
2
3Submatch 0: Hello World
4  Length: 11
5  First Character: H
6  Position: 0
7
8Submatch 1: World
9  Length: 5
10  First Character: W
11  Position: 6

Multiple Matches using std::regex_iterator

Calls to std::regex_search() will stop once a match is found. However, our regex may match multiple patterns in our input strings.

When we want to match all instances of a pattern within our string, we have some other options we can use. We could do it with multiple calls to std::regex_search(), but it’s typically easier and safer to use the standard library's dedicated regex iterators instead.

The std::regex_iterator is also a template class, but an alias has been provided if we're working with std::string objects. The alias is std::sregex_iterator.

We construct the starting iterator by passing a std::string iterator pair as the first two arguments, representing where we want the search to begin, and where we want it to end. Typically, we want to search the entire string, so we just pass the results of the begin() and end() methods of our input string.

The third argument we need to pass to the constructor is our regex pattern:

1std::string Input{
2    "Hello World, Goodbye World"};
3std::regex Pattern{
4    "(Hello|Goodbye) (World|Everyone)"};
5std::sregex_iterator Iterator{
6    Input.begin(), Input.end(), Pattern};

To create an end iterator to compare against, we can create a second std::sregex_iterator, passing no arguments.

With this setup, we can now use the std::sregex_iterator to iterate through all the matches found in our input string.

Similar to calls to std::regex_search(), each iteration will yield a std::match_results object. We can then access the std::sub_match objects within each container in the usual way:

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{
6      "Hello World, Goodbye World"};
7  std::regex Pattern{
8      "(Hello|Goodbye) (World|Everyone)"};
9  std::sregex_iterator Iterator{
10      Input.begin(), Input.end(), Pattern};
11  std::sregex_iterator End;
12
13  while (Iterator != End) {
14    std::cout << "Match";
15    for (auto Match : *Iterator) {
16      std::cout << "\n  Submatch: " << Match;
17    }
18    std::cout << "\n\n";
19    ++Iterator;
20  }
21}

1Match
2  Submatch: Hello World
3  Submatch: Hello
4  Submatch: World
5
6Match
7  Submatch: Goodbye World
8  Submatch: Goodbye
9  Submatch: World

std::regex_token_iterator

We have an alternative regex iterator we can use - the std::regex_token_iterator. A std::string version is available as std::sregex_token_iterator.

We construct and iterate over it in the same way we did std::sregex_iterator, but the matches are provided in a simpler form.

The token iterator skips the intermediate std::match_results containers - it instead just iterates directly through the std::sub_match objects.

By default, it provides us with the sub-match at index 0 of each match. That is, it gives us the sub-matches that matched the entire regex pattern, rather than any specific capture group:

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{
6      "Hello World, Goodbye Everyone"};
7  std::regex Pattern{
8      "(Hello|Goodbye) (World|Everyone)"};
9  std::sregex_token_iterator Iterator{
10      Input.begin(), Input.end(), Pattern};
11  std::sregex_token_iterator End;
12
13  while (Iterator != End) {
14    auto res = (*Iterator);
15    std::cout << "\nSubmatch: " << (*Iterator);
16    ++Iterator;
17  }
18}

1Submatch: Hello World
2Submatch: Goodbye Everyone

By passing a 4th argument to the std::sregex_token_iterator, we can specify which sub-match we want. Below, we specify index 1, ie, the sub-match that was captured by our first capture group:

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{
6      "Hello World, Goodbye Everyone"};
7  std::regex Pattern{
8      "(Hello|Goodbye) (World|Everyone)"};
9  std::sregex_token_iterator Iterator{
10      Input.begin(), Input.end(), Pattern, 1};
11  std::sregex_token_iterator End;
12
13  while (Iterator != End) {
14    auto res = (*Iterator);
15    std::cout << "\nSubmatch: " << (*Iterator);
16    ++Iterator;
17  }
18}

1Submatch: Hello
2Submatch: Goodbye

We can pass multiple indices to the 4th argument, using a std::vector, a C-style array, or an initializer list.

Below, we specify we want all the sub-matches of our regex. We know this pattern has two capture groups, so we expect three sub-matches per match. We want the overall match 0, and the capture groups 1 and 2:

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{
6      "Hello World, Goodbye Everyone"};
7  std::regex Pattern{
8      "(Hello|Goodbye) (World|Everyone)"};
9  std::sregex_token_iterator Iterator{
10      Input.begin(),
11      Input.end(),
12      Pattern,
13      {0, 1, 2}};
14  std::sregex_token_iterator End;
15
16  while (Iterator != End) {
17    auto res = (*Iterator);
18    std::cout << "\nSubmatch: " << (*Iterator);
19    ++Iterator;
20  }
21}

1Submatch: Hello World
2Submatch: Hello
3Submatch: World
4Submatch: Goodbye Everyone
5Submatch: Goodbye
6Submatch: Everyone

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::regex Pattern{
6      "(Hello|Goodbye) (World|Everyone)"};
7  std::cout << Pattern.mark_count();
8}

12

std::regex_replace()

The std::regex_replace() function allows us to make changes to a string, based on a regular expression. In the following example, we replace every instance of "World" with the string "Everyone":

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{
6      "Hello World, Goodbye World"};
7  std::regex Search{"World"};
8  std::string Replace{"Everyone"};
9
10  std::string Updated{std::regex_replace(
11      Input, Search, Replace)};
12
13  std::cout << "Before: " << Input
14            << "\n After: " << Updated;
15}

1Before: Hello World, Goodbye World
2 After: Hello Everyone, Goodbye Everyone

Below, we use a slightly more complicated regex pattern to replace anything that looks somewhat like an email address:

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{
6      "email me at bob@gmail.com or "
7      "bob@yahoo.com"};
8  std::regex Search{R"(\w*@[\w.]*)"};
9  std::string Replace{"[redacted]"};
10
11  std::string Updated{std::regex_replace(
12      Input, Search, Replace)};
13
14  std::cout << "Before: " << Input
15            << "\n After: " << Updated;
16}

1Before: email me at bob@gmail.com or bob@yahoo.com
2 After: email me at [redacted] or [redacted]

Note: A robust regex pattern for email addresses is significantly more complicated than this. The patterns used throughout this lesson have been simplified for clarity.

std::regex_replace() with Capture Groups

When using std::regex_replace() with capture groups, we can include what was captured within our replacement string. We do this using the $ symbol, followed by the number of our capture group within our regex pattern, starting from 1. For example, $1, $2, $3, and so on.

Below, we change how negative numbers are displayed. For example, -100 becomes (100) We do this by adding a capture group to our regex, and then referencing what was captured by that group using $1 in our replacement string:

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{
6      "The balances are 400, -100 and 250"};
7  std::regex Search{R"(-(\d*))"};
8  std::string Replace{"($1)"};
9
10  std::string Updated{std::regex_replace(
11      Input, Search, Replace)};
12
13  std::cout << "Before: " << Input
14            << "\n After: " << Updated;
15}

1Before: The balances are 400, -100 and 250
2 After: The balances are 400, (100) and 250

In this example, we use multiple capture groups to reorder and duplicate parts of our string:

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{"The name's James Bond"};
6  std::regex Search{"(The name's) (.*) (.*)"};
7  std::string Replace{"$1 $3, $2 $3"};
8
9  std::string Updated{std::regex_replace(
10      Input, Search, Replace)};
11
12  std::cout << "Before: " << Input
13            << "\n After: " << Updated;
14}

1Before: The name's James Bond
2 After: The name's Bond, James Bond

If we want our replacement string to include a literal dollar value, like "I can pay $3", we escape the capture group reference using an additional dollar sign. For example, to have $3 in our replacement string, we’d use $$3:

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{"The price is {price}"};
6  std::regex Search{R"(\{price\})"};
7  std::string Replace{"$$3.50"};
8
9  std::cout << std::regex_replace(Input, Search,
10                                  Replace);
11}

1The price is $3.50

We can access the full substring that was matched using the $& token. This is equivalent to the contents at index 0 of a std::match_results object:

1#include <regex>
2#include <iostream>
3
4int main() {
5  std::string Input{
6      "The hungry brown cat and the sleepy "
7      "black bear"};
8  std::regex Search{
9      ".*?(happy|hungry|sleepy) (brown|black) "
10      "(bear|fox|cat).*?"};
11  std::string Replace{
12      "Matched: $& \n  Animal: $3\n  Color: "
13      "$2\n  Mood: $1\n\n"};
14
15  std::cout << std::regex_replace(Input, Search,
16                                  Replace);
17}

1Matched: The hungry brown cat
2  Animal: cat
3  Color: brown
4  Mood: hungry
5
6Matched:  and the sleepy black bear
7  Animal: bear
8  Color: black
9  Mood: sleepy

Summary

In this lesson, we explored regex capture groups and their application within C++, enabling the manipulation and extraction of specific parts of strings for both analysis and transformation.

Main Points Learned

• The syntax and usage of capture groups in regular expressions.
• The distinction between capture groups and non-capture groups, and when to use each.
• How greedy and lazy quantifiers influence the matching behavior and how to apply them.
• The role of std::smatch and std::match_results in storing and accessing matched results from regex operations.
• Utilization of std::regex_search(), std::regex_replace(), std::regex_iterator, and std::sregex_token_iterator to perform complex regex operations in C++.
• Techniques for using capture groups in string replacement operations with std::regex_replace().
• How to programmatically determine the number of capture groups in a regex pattern using mark_count().

Next Lesson

String Views

A practical introduction to string views, and why they should be the main way we pass strings to functions

ContinueView Course