Python3 Regular Expressions

A regular expression is a special sequence of characters that helps you easily check whether a string matches a certain pattern.

In Python, the re module is used to handle regular expressions.

The re module provides a set of functions that allow you to perform pattern matching, searching, and replacement operations on strings.

The re module gives the Python language complete regular expression functionality.

This chapter mainly introduces commonly used regular expression processing functions in Python. If you are not familiar with regular expressions, you can check our Regular Expression - Tutorial.

re.match Function

re.match attempts to match a pattern from the starting position of a string. If the match is not successful at the starting position, match() returns None.

Function syntax:

re.match(pattern, string, flags=0)

Function parameter description:


Parameter	Description
pattern	The regular expression pattern to match
string	The string to match.
flags	Flags to control the matching behavior of the regular expression, such as case sensitivity, multiline matching, etc. See: Regular Expression Modifiers - Optional Flags

On successful match, re.match returns a match object; otherwise it returns None.

We can use the group(num) or groups() match object functions to retrieve matching expressions.


Match Object Method	Description
group(num=0)	The string of the entire matched expression. group() can accept multiple group numbers at once, in which case it returns a tuple containing the corresponding values.
groups()	Returns a tuple containing all subgroup strings, from 1 to the number of subgroups.

Example

#!/usr/bin/python

import re
print(re.match('www', 'www.runoob.com').span())  # Match at starting position
print(re.match('com', 'www.runoob.com'))         # Not at starting position

The output of the above example is:

(0, 3)
None

Example

#!/usr/bin/python3
import re

line = "Cats are smarter than dogs"
# .* matches any single or multiple characters except newlines (\n, \r)
# (.*?) means "non-greedy" mode, only saves the first matched substring
matchObj = re.match( r'(.*) are (.*?) .*', line, re.M|re.I)

if matchObj:
   print ("matchObj.group() : ", matchObj.group())
   print ("matchObj.group(1) : ", matchObj.group(1))
   print ("matchObj.group(2) : ", matchObj.group(2))
else:
   print ("No match!!")

The output of the above example is:

matchObj.group() :  Cats are smarter than dogs
matchObj.group(1) :  Cats
matchObj.group(2) :  smarter

re.search Method

re.search scans the entire string and returns the first successful match.

Function syntax:

re.search(pattern, string, flags=0)

Function parameter description:


Parameter	Description
pattern	The regular expression pattern to match
string	The string to match.
flags	Flags to control the matching behavior of the regular expression, such as case sensitivity, multiline matching, etc. See: Regular Expression Modifiers - Optional Flags

On successful match, re.search returns a match object; otherwise it returns None.

We can use the group(num) or groups() match object functions to retrieve matching expressions.


Match Object Method	Description
group(num=0)	The string of the entire matched expression. group() can accept multiple group numbers at once, in which case it returns a tuple containing the corresponding values.
groups()	Returns a tuple containing all subgroup strings, from 1 to the number of subgroups.

Example

#!/usr/bin/python3

import re

print(re.search('www', 'www.runoob.com').span())  # Match at starting position
print(re.search('com', 'www.runoob.com').span())         # Not at starting position

The output of the above example is:

(0, 3)
(11, 14)

Example

#!/usr/bin/python3

import re

line = "Cats are smarter than dogs"

searchObj = re.search( r'(.*) are (.*?) .*', line, re.M|re.I)

if searchObj:
   print ("searchObj.group() : ", searchObj.group())
   print ("searchObj.group(1) : ", searchObj.group(1))
   print ("searchObj.group(2) : ", searchObj.group(2))
else:
   print ("Nothing found!!")

The output of the above example is:

searchObj.group() :  Cats are smarter than dogs
searchObj.group(1) :  Cats
searchObj.group(2) :  smarter

Difference Between re.match and re.search

re.match only matches at the beginning of the string. If the beginning of the string does not match the regular expression, the match fails and the function returns None. re.search, on the other hand, matches the entire string until a match is found.

Example

#!/usr/bin/python3

import re

line = "Cats are smarter than dogs"

matchObj = re.match( r'dogs', line, re.M|re.I)
if matchObj:
   print ("match --> matchObj.group() : ", matchObj.group())
else:
   print ("No match!!")

matchObj = re.search( r'dogs', line, re.M|re.I)
if matchObj:
   print ("search --> matchObj.group() : ", matchObj.group())
else:
   print ("No match!!")

The output of the above example is:

No match!!
search --> matchObj.group() :  dogs

Search and Replace

Python’s re module provides re.sub for replacing matches in strings.

Syntax:

re.sub(pattern, repl, string, count=0, flags=0)

Parameters:

pattern : The pattern string in the regular expression.
repl : The replacement string, which can also be a function.
string : The original string to be searched and replaced.
count : The maximum number of replacements after pattern matching. Default 0 replaces all matches.
flags : The matching mode used during compilation, in numeric form.

The first three are required parameters; the last two are optional.

Example

#!/usr/bin/python3
import re

phone = "2004-959-559 # This is a phone number"

# Remove comments
num = re.sub(r'#.*$', "", phone)
print ("Phone number : ", num)

# Remove non-numeric content
num = re.sub(r'\D', "", phone)
print ("Phone number : ", num)

The output of the above example is:

Phone number :  2004-959-559
Phone number :  2004959559

repl Parameter as a Function

In the following example, matched numbers in the string are multiplied by 2:

Example

#!/usr/bin/python

import re

# Multiply the matched number by 2
def double(matched):
    value = int(matched.group('value'))
    return str(value * 2)

s = 'A23G4HFD567'
print(re.sub('(?P<value>\d+)', double, s))

Output:

A46G8HFD1134

compile Function

The compile function is used to compile a regular expression, generating a regular expression (Pattern) object for use by the match() and search() functions.

Syntax:

re.compile(pattern[, flags])

Parameters:

pattern : A regular expression in string form
flags : Optional, specifies the matching mode, such as case insensitivity, multiline mode, etc. Specific parameters are:
re.L Indicates that special character sets \w, \W, \b, \B, \s, \S depend on the current locale
re.MULTILINE or re.M - Multiline mode, changes the behavior of ^ and $ so they match the beginning and end of each line in the string.
re.DOTALL or re.S - Makes . match any character including newlines.
re.ASCII - Makes \w, \W, \b, \B, \d, \D, \s, \S only match ASCII characters.
re.VERBOSE or re.X - Ignores whitespace and comments, allowing for clearer organization of complex regular expressions.

These flags can be used individually or combined using bitwise OR (|). For example, re.IGNORECASE | re.MULTILINE enables both case insensitivity and multiline mode.

Example

>>>import re
>>> pattern = re.compile(r'\d+')                    # Used to match at least one digit
>>> m = pattern.match('one12twothree34four')        # Look at the beginning, no match
>>> print( m )
None
>>> m = pattern.match('one12twothree34four', 2, 10) # Start matching from 'e' position, no match
>>> print( m )
None
>>> m = pattern.match('one12twothree34four', 3, 10) # Start matching from '1' position, exact match
>>> print( m )                                        # Returns a Match object
<_sre.SRE_Match object at 0x10a42aac0>
>>> m.group(0)   # 0 can be omitted
'12'
>>> m.start(0)   # 0 can be omitted
3
>>> m.end(0)     # 0 can be omitted
5
>>> m.span(0)    # 0 can be omitted
(3, 5)

Above, when a match is successful, a Match object is returned, where:

group([group1, …]) method is used to obtain one or more grouped match strings. To get the entire matched substring, you can directly use group() or group(0);
start([group]) method is used to get the starting position of the grouped match substring in the entire string (the index of the first character of the substring), default parameter value is 0;
end([group]) method is used to get the ending position of the grouped match substring in the entire string (the index of the last character + 1), default parameter value is 0;
span([group]) method returns (start(group), end(group)).

Let’s look at another example:

Example

>>>import re
>>> pattern = re.compile(r'([a-z]+) ([a-z]+)', re.I)   # re.I means ignore case
>>> m = pattern.match('Hello World Wide Web')
>>> print( m )                            # Match successful, returns a Match object
<_sre.SRE_Match object at 0x10bea83e8>
>>> m.group(0)                            # Returns the entire matched substring
'Hello World'
>>> m.span(0)                             # Returns the index of the entire matched substring
(0, 11)
>>> m.group(1)                            # Returns the first grouped matched substring
'Hello'
>>> m.span(1)                             # Returns the index of the first grouped matched substring
(0, 5)
>>> m.group(2)                            # Returns the second grouped matched substring
'World'
>>> m.span(2)                             # Returns the index of the second grouped matched substring
(6, 11)
>>> m.groups()                            # Equivalent to (m.group(1), m.group(2), ...)
('Hello', 'World')
>>> m.group(3)                            # Third group does not exist
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: no such group

findall

Finds all substrings in a string that match the regular expression and returns a list. If there are multiple matching patterns, returns a list of tuples. If no matches are found, returns an empty list.

Note: match and search match once; findall matches all.

Syntax:

re.findall(pattern, string, flags=0)
or
pattern.findall(string[, pos[, endpos]])

Parameters:

pattern The matching pattern.
string The string to match.
pos Optional parameter, specifies the starting position of the string, default is 0.
endpos Optional parameter, specifies the ending position of the string, default is the length of the string.

Find all numbers in a string:

Example

import re

result1 = re.findall(r'\d+','runoob 123 google 456')

pattern = re.compile(r'\d+')   # Find numbers
result2 = pattern.findall('runoob 123 google 456')
result3 = pattern.findall('run88oob123google456', 0, 10)

print(result1)
print(result2)
print(result3)

Output:

['123', '456']
['123', '456']
['88', '12']

Multiple matching patterns, returns a list of tuples:

Example

import re

result = re.findall(r'(\w+)=(\d+)', 'set width=20 and height=10')
print(result)

[('width', '20'), ('height', '10')]

re.finditer

Similar to findall, it finds all substrings matching the regular expression in a string and returns them as an iterator.

re.finditer(pattern, string, flags=0)

Parameters:


Parameter	Description
pattern	The regular expression pattern to match
string	The string to match.
flags	Flags to control the matching behavior of the regular expression, such as case sensitivity, multiline matching, etc. See: Regular Expression Modifiers - Optional Flags

Example

import re

it = re.finditer(r"\d+","12a32bc43jf3")
for match in it:
    print (match.group() )

Output:

re.split

The split method splits a string according to matching substrings and returns a list. Its usage is as follows:

re.split(pattern, string[, maxsplit=0, flags=0])

Parameters:


Parameter	Description
pattern	The regular expression pattern to match
string	The string to match.
maxsplit	Number of splits; maxsplit=1 splits once. Default is 0, no limit on number of splits.
flags	Flags to control the matching behavior of the regular expression, such as case sensitivity, multiline matching, etc. See: Regular Expression Modifiers - Optional Flags

Example

>>>import re
>>> re.split('\W+', 'runoob, runoob, runoob.')
['runoob', 'runoob', 'runoob', '']
>>> re.split('(\W+)', ' runoob, runoob, runoob.')
['', ' ', 'runoob', ', ', 'runoob', ', ', 'runoob', '.', '']
>>> re.split('\W+', ' runoob, runoob, runoob.', 1)
['', 'runoob, runoob, runoob.']

>>> re.split('a*', 'hello world')   # For a string with no match, split will not split it
['hello world']

Regular Expression Objects

re.RegexObject

re.compile() returns a RegexObject object.

re.MatchObject

group() returns the string matched by the RE.

start() returns the starting position of the match
end() returns the ending position of the match
span() returns a tuple containing the (start, end) positions of the match

Regular Expression Modifiers - Optional Flags

Regular expressions can include optional flag modifiers to control matching modes.

The following flags can be used individually or combined using bitwise OR (|). For example, re.IGNORECASE | re.MULTILINE enables both case insensitivity and multiline mode.

Modifier	Description	Example
re.IGNORECASE or re.I	Makes matching case-insensitive	`import re pattern = re.compile(r'apple', flags=re.IGNORECASE) result = pattern.match('Apple') print(result.group()) # Output: 'Apple'`
re.MULTILINE or re.M	Multiline matching, affects ^ and $, making them match the beginning and end of each line in the string.	`import re pattern = re.compile(r'^\d+', flags=re.MULTILINE) text = '123\n456\n789' result = pattern.findall(text) print(result) # Output: ['123', '456', '789']`
re.DOTALL or re.S：	Makes . match any character including newlines.	`import re pattern = re.compile(r'a.b', flags=re.DOTALL) result = pattern.match('a\nb') print(result.group()) # Output: 'a\nb'`
re.ASCII	Makes \w, \W, \b, \B, \d, \D, \s, \S only match ASCII characters.	`import re pattern = re.compile(r'\w+', flags=re.ASCII) result = pattern.match('Hello123') print(result.group()) # Output: 'Hello123'`
re.VERBOSE or re.X	Ignores whitespace and comments, allowing for clearer organization of complex regular expressions.	`import re pattern = re.compile(r''' \d+ # Match digits [a-z]+ # Match lowercase letters ''', flags=re.VERBOSE) result = pattern.match('123abc') print(result.group()) # Output: '123abc'`

Regular Expression Patterns

Pattern strings use special syntax to represent a regular expression.

Letters and numbers represent themselves. Letters and numbers in a regular expression pattern match the same strings.

Most letters and numbers have different meanings when preceded by a backslash.

Punctuation marks only match themselves when escaped; otherwise they represent special meanings.

The backslash itself needs to be escaped with a backslash.

Since regular expressions often contain backslashes, it’s best to use raw strings to represent them. Pattern elements (like r’\t’, equivalent to \t) match the corresponding special characters.

The following table lists special elements in regular expression pattern syntax. If you provide optional flag parameters along with the pattern, the meaning of some pattern elements may change.


Pattern	Description
^	Matches the beginning of a string
$	Matches the end of a string.
.	Matches any character except newline. When the re.DOTALL flag is specified, it can match any character including newlines.
[…]	Used to match any single character within the brackets, e.g. [amk] matches ‘a’, ‘m’, or ‘k’
[^…]	Characters not in []: [^abc] matches characters other than a, b, c.
re*	Matches 0 or more occurrences of the expression.
re+	Matches 1 or more occurrences of the expression.
re?	Matches 0 or 1 occurrence of the fragment defined by the preceding regular expression, non-greedy mode
re{ n}	Matches exactly n occurrences of the preceding expression. For example, “o{2}” cannot match “o” in “Bob”, but can match both o’s in “food”.
re{ n,}	Matches n or more occurrences of the preceding expression. For example, “o{2,}” cannot match “o” in “Bob”, but can match all o’s in “foooood”. “o{1,}” is equivalent to “o+”. “o{0,}” is equivalent to “o*”.
re{ n, m}	Matches n to m occurrences of the fragment defined by the preceding regular expression, greedy mode
a\| b	Matches a or b
(re)	Matches the expression inside the parentheses and also represents a group
(?imx)	Regular expression includes three optional flags: i, m, or x. Only affects the area within the parentheses.
(?-imx)	Regular expression turns off i, m, or x optional flags. Only affects the area within the parentheses.
(?: re)	Similar to (…), but does not represent a group
(?imx: re)	Use i, m, or x optional flags within parentheses
(?-imx: re)	Do not use i, m, or x optional flags within parentheses
(?#…)	Comment.
(?= re)	Forward positive lookahead. Succeeds if the contained regular expression, represented by …, matches at the current position; otherwise fails. However, once the contained expression has been tried, the matching engine does not advance; the remaining part of the pattern still tries to match to the right of the lookahead.
(?! re)	Forward negative lookahead. Opposite of positive lookahead; succeeds when the contained expression cannot match at the current position in the string.
(?> re)	Independent pattern for matching, eliminating backtracking.
\w	Matches digits, letters, and underscore
\W	Matches non-digits, non-letters, and non-underscores
\s	Matches any whitespace character, equivalent to [\t\n\r\f].
\S	Matches any non-whitespace character
\d	Matches any digit, equivalent to [0-9].
\D	Matches any non-digit
\A	Matches the beginning of a string
\Z	Matches the end of a string. If a newline exists, only matches before the newline at the end of the string.
\z	Matches the end of a string
\G	Matches the position where the last match finished.
\b	Matches a word boundary, i.e. the position between a word and a space. For example, ‘er\b’ can match ‘er’ in “never”, but cannot match ‘er’ in “verb”.
\B	Matches a non-word boundary. ‘er\B’ can match ‘er’ in “verb”, but cannot match ‘er’ in “never”.
\n, \t, etc.	Matches a newline. Matches a tab, etc.
\1…\9	Matches the content of the nth group.
\10	Matches the content of the nth group, if it has been matched. Otherwise refers to an octal character code expression.

Regular Expression Examples

Character Matching


Example	Description
python	Matches “python”.

Character Classes


Example	Description
[Pp]ython	Matches “Python” or “python”
rub[ye]	Matches “ruby” or “rube”
[aeiou]	Matches any single letter within the brackets
[0-9]	Matches any digit. Similar to [0123456789]
[a-z]	Matches any lowercase letter
[A-Z]	Matches any uppercase letter
[a-zA-Z0-9]	Matches any letter or digit
[^aeiou]	All characters except aeiou letters
[^0-9]	Matches characters other than digits

Special Character Classes


Example	Description
.	Matches any single character except “\n”. To match any character including ‘\n’, use a pattern like ‘[.\n]’.
\d	Matches a digit character. Equivalent to [0-9].
\D	Matches a non-digit character. Equivalent to [^0-9].
\s	Matches any whitespace character, including spaces, tabs, form feeds, etc. Equivalent to [ \f\n\r\t\v].
\S	Matches any non-whitespace character. Equivalent to [^ \f\n\r\t\v].
\w	Matches any word character including underscore. Equivalent to ‘[A-Za-z0-9_]’.
\W	Matches any non-word character. Equivalent to ‘[^A-Za-z0-9_]’.