Python3 XML Parsing

---

What is XML?

XML stands for eXtensible Markup Language, a subset of the Standard Generalized Markup Language, and is a markup language used to mark electronic documents to make them structural. You can learn XML through this site’s XML Tutorial.

XML is designed to transmit and store data.

XML is a set of rules for defining semantic markup, which divides documents into many parts and identifies those parts.

It is also a meta-markup language, that is, a language that defines the syntactic language used to define other domain-specific, semantic, structural markup languages.

---

Python’s XML Parsing

Common XML programming interfaces include DOM and SAX. These two interfaces handle XML files differently and, of course, are used in different situations.

Python has three methods for parsing XML: ElementTree, SAX, and DOM.

1. ElementTree

xml.etree.ElementTree is a module in the Python standard library for processing XML. It provides a simple and efficient API for parsing and generating XML documents.

2. SAX (simple API for XML)

The Python standard library includes a SAX parser. SAX uses an event-driven model, processing XML files by triggering events one by one during XML parsing and calling user-defined callback functions.

3. DOM (Document Object Model)

Parses XML data into a tree in memory, and operates on XML by operating on the tree.

The XML example file movies.xml used in this chapter has the following content:

Example

<collection shelf="New Arrivals">
<movie title="Enemy Behind">
   <type>War, Thriller</type>
   <format>DVD</format>
   <year>2003</year>
   <rating>PG</rating>
   <stars>10</stars>
   <description>Talk about a US-Japan war</description>
</movie>
<movie title="Transformers">
   <type>Anime, Science Fiction</type>
   <format>DVD</format>
   <year>1989</year>
   <rating>R</rating>
   <stars>8</stars>
   <description>A schientific fiction</description>
</movie>
   <movie title="Trigun">
   <type>Anime, Action</type>
   <format>DVD</format>
   <episodes>4</episodes>
   <rating>PG</rating>
   <stars>10</stars>
   <description>Vash the Stampede!</description>
</movie>
<movie title="Ishtar">
   <type>Comedy</type>
   <format>VHS</format>
   <rating>PG</rating>
   <stars>2</stars>
   <description>Viewable boredom</description>
</movie>
</collection>

---

Python Using ElementTree to Parse XML

xml.etree.ElementTree is a module in the Python standard library for processing XML.

The following are some key concepts and usages of the xml.etree.ElementTree module:

ElementTree and Element Objects:

• ElementTree: The ElementTree class is a tree representation of an XML document. It contains one or more Element objects, representing the entire XML document.
• Element: The Element object is the representation of an element in the XML document. Each element has a tag, a set of attributes, and zero or more child elements.

Parsing XML

fromstring() method: Use the fromstring() method to convert a string containing XML data into an Element object:

Example

import xml.etree.ElementTree as ET

xml_string = '<root><element>Some data</element></root>'
root = ET.fromstring(xml_string)

parse() method: If the XML data is stored in a file, you can use the parse() method to parse the entire XML document:

tree = ET.parse('example.xml')
root = tree.getroot()

Traversing the XML Tree

find() method: Use the find() method to find the first child element with a specified tag:

title_element = root.find('title')

findall() method: Use the findall() method to find all child elements with a specified tag:

book_elements = root.findall('book')

Accessing Element Attributes and Text Content

attrib property: Access element attributes through the attrib property:

price = book_element.attrib['price']

text property: Access the text content of an element through the text property:

title_text = title_element.text

Creating XML

Element() constructor: Use the Element() constructor to create new elements:

new_element = ET.Element('new_element')

SubElement() function: Use the SubElement() function to create child elements with a specified tag:

new_sub_element = ET.SubElement(root, 'new_sub_element')

Modifying XML

Modifying element attributes and text content: Directly modify the element’s attrib and text properties.

Removing elements: Use the remove() method to remove elements:

root.remove(title_element)

Simple reading of XML content:

Example

import xml.etree.ElementTree as ET

# Define an XML string
xml_string = '''
<bookstore>
    <book>
        <title>Introduction to Python</title>
        <author>John Doe</author>
        <price>29.99</price>
    </book>
    <book>
        <title>Data Science with Python</title>
        <author>Jane Smith</author>
        <price>39.95</price>
    </book>
</bookstore>
'''

# Use ElementTree to parse the XML string
root = ET.fromstring(xml_string)

# Traverse the XML tree
for book in root.findall('book'):
    title = book.find('title').text
    author = book.find('author').text
    price = book.find('price').text
    print(f'Title: {title}, Author: {author}, Price: {price}')

The output of the above code is:

Title: Introduction to Python, Author: John Doe, Price: 29.99
Title: Data Science with Python, Author: Jane Smith, Price: 39.95

Next, we will create an XML document containing book information, then use the module for parsing and manipulation:

Example

import xml.etree.ElementTree as ET

# Create an XML document
root = ET.Element('bookstore')

# Add the first book
book1 = ET.SubElement(root, 'book')
title1 = ET.SubElement(book1, 'title')
title1.text = 'Introduction to Python'
author1 = ET.SubElement(book1, 'author')
author1.text = 'John Doe'
price1 = ET.SubElement(book1, 'price')
price1.text = '29.99'

# Add the second book
book2 = ET.SubElement(root, 'book')
title2 = ET.SubElement(book2, 'title')
title2.text = 'Data Science with Python'
author2 = ET.SubElement(book2, 'author')
author2.text = 'Jane Smith'
price2 = ET.SubElement(book2, 'price')
price2.text = '39.95'

# Save the XML document to a file
tree = ET.ElementTree(root)
tree.write('books.xml')

# Parse the XML document from the file
parsed_tree = ET.parse('books.xml')
parsed_root = parsed_tree.getroot()

# Traverse the XML tree and print book information
for book in parsed_root.findall('book'):
    title = book.find('title').text
    author = book.find('author').text
    price = book.find('price').text
    print(f'Title: {title}, Author: {author}, Price: {price}')

In the above example, we first create an XML document containing information about two books. Then we save this document to the file books.xml. Next, we use the ET.parse() method to parse the XML document from the file, traverse the document tree, and extract and print the title, author, and price information for each book.

---

Python Using SAX to Parse XML

SAX is an event-driven API.

Using SAX to parse XML documents involves two parts: the parser and the event handler.

The parser is responsible for reading the XML document and sending events to the event handler, such as element start and element end events.

The event handler is responsible for responding to the events and processing the transmitted XML data.

• 1. For processing large files;
• 2. When only part of the file’s content is needed, or only specific information from the file is needed.
• 3. When you want to build your own object model.

To use the SAX method to process XML in Python, you need to first import the parse function from xml.sax and the ContentHandler from xml.sax.handler.

ContentHandler Class Methods Introduction

characters(content) method

Call timing:

From the beginning of a line, before encountering a tag, if there are characters, the value of content is these strings.

From one tag, before encountering the next tag, if there are characters, the value of content is these strings.

From one tag, before encountering a line terminator, if there are characters, the value of content is these strings.

A tag can be either a start tag or an end tag.

startDocument() method

Called when the document starts.

endDocument() method

Called when the parser reaches the end of the document.

startElement(name, attrs) method

Called when an XML start tag is encountered. name is the tag name, and attrs is the tag’s attribute value dictionary.

endElement(name) method

Called when an XML end tag is encountered.

---

make_parser Method

The following method creates a new parser object and returns it.

xml.sax.make_parser( [parser_list] )

Parameter description:

• parser_list - Optional parameter, parser list

---

parser Method

The following method creates a SAX parser and parses an XML document:

xml.sax.parse( xmlfile, contenthandler[, errorhandler])

Parameter description:

• xmlfile - XML file name
• contenthandler - Must be a ContentHandler object
• errorhandler - If this parameter is specified, errorhandler must be a SAX ErrorHandler object

---

parseString Method

The parseString method creates an XML parser and parses an XML string:

xml.sax.parseString(xmlstring, contenthandler[, errorhandler])

Parameter description:

• xmlstring - XML string
• contenthandler - Must be a ContentHandler object
• errorhandler - If this parameter is specified, errorhandler must be a SAX ErrorHandler object

---

Python XML Parsing Example

Example

#!/usr/bin/python3

import xml.sax

class MovieHandler( xml.sax.ContentHandler ):
   def __init__(self):
      self.CurrentData = ""
      self.type = ""
      self.format = ""
      self.year = ""
      self.rating = ""
      self.stars = ""
      self.description = ""

   # Called when an element starts
   def startElement(self, tag, attributes):
      self.CurrentData = tag
      if tag == "movie":
         print ("*****Movie*****")
         title = attributes["title"]
         print ("Title:", title)

   # Called when an element ends
   def endElement(self, tag):
      if self.CurrentData == "type":
         print ("Type:", self.type)
      elif self.CurrentData == "format":
         print ("Format:", self.format)
      elif self.CurrentData == "year":
         print ("Year:", self.year)
      elif self.CurrentData == "rating":
         print ("Rating:", self.rating)
      elif self.CurrentData == "stars":
         print ("Stars:", self.stars)
      elif self.CurrentData == "description":
         print ("Description:", self.description)
      self.CurrentData = ""

   # Called when reading characters
   def characters(self, content):
      if self.CurrentData == "type":
         self.type = content
      elif self.CurrentData == "format":
         self.format = content
      elif self.CurrentData == "year":
         self.year = content
      elif self.CurrentData == "rating":
         self.rating = content
      elif self.CurrentData == "stars":
         self.stars = content
      elif self.CurrentData == "description":
         self.description = content

if ( __name__ == "__main__"):

   # Create an XMLReader
   parser = xml.sax.make_parser()
   # Disable namespaces
   parser.setFeature(xml.sax.handler.feature_namespaces, 0)

   # Override ContextHandler
   Handler = MovieHandler()
   parser.setContentHandler( Handler )

   parser.parse("movies.xml")

The output of the above code is:

*****Movie*****
Title: Enemy Behind
Type: War, Thriller
Format: DVD
Year: 2003
Rating: PG
Stars: 10
Description: Talk about a US-Japan war
*****Movie*****
Title: Transformers
Type: Anime, Science Fiction
Format: DVD
Year: 1989
Rating: R
Stars: 8
Description: A schientific fiction
*****Movie*****
Title: Trigun
Type: Anime, Action
Format: DVD
Rating: PG
Stars: 10
Description: Vash the Stampede!
*****Movie*****
Title: Ishtar
Type: Comedy
Format: VHS
Rating: PG
Stars: 2
Description: Viewable boredom

For the complete SAX API documentation, please refer to Python SAX APIs

---

Using xml.dom to Parse XML

The Document Object Model (DOM) is a standard programming interface recommended by the W3C for processing extensible markup languages.

When a DOM parser parses an XML document, it reads the entire document at once, saves all elements in the document in a tree structure in memory, and then you can use the different functions provided by DOM to read or modify the content and structure of the document, and can also write the modified content back to the XML file.

In Python, xml.dom.minidom is used to parse XML files. Examples are as follows:

Example

#!/usr/bin/python3

from xml.dom.minidom import parse
import xml.dom.minidom

# Use minidom parser to open the XML document
DOMTree = xml.dom.minidom.parse("movies.xml")
collection = DOMTree.documentElement
if collection.hasAttribute("shelf"):
   print ("Root element : %s" % collection.getAttribute("shelf"))

# Get all movies in the collection
movies = collection.getElementsByTagName("movie")

# Print detailed information for each movie
for movie in movies:
   print ("*****Movie*****")
   if movie.hasAttribute("title"):
      print ("Title: %s" % movie.getAttribute("title"))

   type = movie.getElementsByTagName('type')[0]
   print ("Type: %s" % type.childNodes[0].data)
   format = movie.getElementsByTagName('format')[0]
   print ("Format: %s" % format.childNodes[0].data)
   rating = movie.getElementsByTagName('rating')[0]
   print ("Rating: %s" % rating.childNodes[0].data)
   description = movie.getElementsByTagName('description')[0]
   print ("Description: %s" % description.childNodes[0].data)

The output of the above program is:

Root element : New Arrivals
*****Movie*****
Title: Enemy Behind
Type: War, Thriller
Format: DVD
Rating: PG
Description: Talk about a US-Japan war
*****Movie*****
Title: Transformers
Type: Anime, Science Fiction
Format: DVD
Rating: R
Description: A schientific fiction
*****Movie*****
Title: Trigun
Type: Anime, Action
Format: DVD
Rating: PG
Description: Vash the Stampede!
*****Movie*****
Title: Ishtar
Type: Comedy
Format: VHS
Rating: PG
Description: Viewable boredom

For the complete DOM API documentation, please refer to Python DOM APIs.