Python 101 – Intro to XML Parsing with ElementTree

If you have followed this blog for a while, you may remember that we’ve covered several XML parsing libraries that are included with Python. In this article, we’ll be continuing that series by taking a quick look at the ElementTree library. You will learn how to create an XML file, edit XML and parse the XML. For comparison’s sake, we’ll use the same XML we used in the previous minidom article to illustrate the differences between using minidom and ElementTree. Here is the original XML:

<?xml version="1.0" ?>
<zAppointments reminder="15">
    <appointment>
        <begin>1181251680</begin>        
        <uid>040000008200E000</uid>
        <alarmTime>1181572063</alarmTime>
        <state></state>
        <location></location>
        <duration>1800</duration>
        <subject>Bring pizza home</subject>
    </appointment>
</zAppointments>

Now let’s dig into the Python!

How to Create XML with ElementTree

Creating XML with ElementTree is very simple. In this section, we will attempt to create the XML above with Python. Here’s the code:

import xml.etree.ElementTree as xml
 
#----------------------------------------------------------------------
def createXML(filename):
    """
    Create an example XML file
    """
    root = xml.Element("zAppointments")
    appt = xml.Element("appointment")
    root.append(appt)
 
    # add appointment children
    begin = xml.SubElement(appt, "begin")
    begin.text = "1181251680"
 
    uid = xml.SubElement(appt, "uid")
    uid.text = "040000008200E000"
 
    alarmTime = xml.SubElement(appt, "alarmTime")
    alarmTime.text = "1181572063"
 
    state = xml.SubElement(appt, "state")
 
    location = xml.SubElement(appt, "location")
 
    duration = xml.SubElement(appt, "duration")
    duration.text = "1800"
 
    subject = xml.SubElement(appt, "subject")
 
    tree = xml.ElementTree(root)
    with open(filename, "w") as fh:
        tree.write(fh)
 
#----------------------------------------------------------------------
if __name__ == "__main__":
    createXML("appt.xml")

If you run this code, you should get something like the following (probably all on one line):

<zAppointments>
	<appointment>
		<begin>1181251680</begin>
		<uid>040000008200E000</uid>
		<alarmTime>1181572063</alarmTime>
		<state />
		<location />
		<duration>1800</duration>
		<subject />
	</appointment>
</zAppointments>

This is pretty close to the original and is certainly valid XML, but it’s not quite the same. However, it’s close enough. Let’s take a moment to review the code and make sure we understand it. First we create the root element by using ElementTree’s Elementfunction. Then we create an appointment element and append it to the root. Next we create SubElements by passing the appointment Element object (appt) to SubElement along with a name, like “begin”. Then for each SubElement, we set its text property to give it a value. At the end of the script, we create an ElementTree and use it to write the XML out to a file.

What’s annoying is that it write out the XML all on one line instead of in a nice readable format (i.e. “pretty print”). There’s a recipe on Effbot, but there doesn’t appear to be a way to do it internally. You may also want to take a look at some of the other solutions on StackOverflow. It should be noted that lxml supports “pretty print” out of the box.

Now we’re ready to learn how to edit the file!

How to Edit XML with ElementTree

Editing XML with ElementTree is also easy. To make things a little more interesting though, we’ll add another appointment block to the XML:

<?xml version="1.0" ?>
<zAppointments reminder="15">
    <appointment>
        <begin>1181251680</begin>        
        <uid>040000008200E000</uid>
        <alarmTime>1181572063</alarmTime>
        <state></state>
        <location></location>
        <duration>1800</duration>
        <subject>Bring pizza home</subject>
    </appointment>
        <appointment>
        <begin>1181253977</begin>        
        <uid>sdlkjlkadhdakhdfd</uid>
        <alarmTime>1181588888</alarmTime>
        <state>TX</state>
        <location>Dallas</location>
        <duration>1800</duration>
        <subject>Bring pizza home</subject>
    </appointment>
</zAppointments>

Now let’s write some code to change each of the begin tag’s values from seconds since the epoch to something a little more readable. We’ll use Python’s time module to facilitate this:

import time
import xml.etree.cElementTree as ET
 
#----------------------------------------------------------------------
def editXML(filename):
    """
    Edit an example XML file
    """
    tree = ET.ElementTree(file=filename)
    root = tree.getroot()
 
    for begin_time in root.iter("begin"):
        begin_time.text = time.ctime(int(begin_time.text))
 
    tree = ET.ElementTree(root)
    with open("updated.xml", "w") as f:
        tree.write(f)
 
#----------------------------------------------------------------------
if __name__ == "__main__":
    editXML("original_appt.xml")

Here we create an ElementTree object (tree) and we extract the root from it. Then we use ElementTree’s iter() method to find all the tags that are labeled “begin”. Note that the iter() method was added in Python 2.7. In our for loop, we set each item’s textproperty to a more human readable time format via time.ctime(). You’ll note that we had to convert the string to an integer when passing it to ctime. The output should look something like the following:

<zAppointments reminder="15">
    <appointment>
        <begin>Thu Jun 07 16:28:00 2007</begin>        
        <uid>040000008200E000</uid>
        <alarmTime>1181572063</alarmTime>
        <state />
        <location />
        <duration>1800</duration>
        <subject>Bring pizza home</subject>
    </appointment>
	<appointment>
        <begin>Thu Jun 07 17:06:17 2007</begin>        
        <uid>sdlkjlkadhdakhdfd</uid>
        <alarmTime>1181588888</alarmTime>
        <state>TX</state>
        <location>Dallas</location>
        <duration>1800</duration>
        <subject>Bring pizza home</subject>
    </appointment>
</zAppointments>

You can also use ElementTree’s find() or findall() methods to get search for specific tags in your XML. The find() method will just find the first instance whereas the findall() will find all the tags with the specified label. These are helpful for editing purposes or for parsing, which is our next topic!

How to Parse XML with ElementTree

Now we get to learn how to do some basic parsing with ElementTree. First we’ll read through the code and then we’ll go through bit by bit so we can understand it. Note that this code is based around the original example, but it should work on the second one as well.

import xml.etree.cElementTree as ET
 
#----------------------------------------------------------------------
def parseXML(xml_file):
    """
    Parse XML with ElementTree
    """
    tree = ET.ElementTree(file=xml_file)
    print tree.getroot()
    root = tree.getroot()
    print "tag=%s, attrib=%s" % (root.tag, root.attrib)
 
    for child in root:
        print child.tag, child.attrib
        if child.tag == "appointment":
            for step_child in child:
                print step_child.tag
 
    # iterate over the entire tree
    print "-" * 40
    print "Iterating using a tree iterator"
    print "-" * 40
    iter_ = tree.getiterator()
    for elem in iter_:
        print elem.tag
 
    # get the information via the children!
    print "-" * 40
    print "Iterating using getchildren()"
    print "-" * 40
    appointments = root.getchildren()
    for appointment in appointments:
        appt_children = appointment.getchildren()
        for appt_child in appt_children:
            print "%s=%s" % (appt_child.tag, appt_child.text)
 
#----------------------------------------------------------------------
if __name__ == "__main__":
    parseXML("appt.xml")

You may have already noticed this, but in this example and the last one, we’ve been importing cElementTree instead of the normal ElementTree. The main difference between the two is that cElementTree is C-based instead of Python-based, so it’s much faster. Anyway, once again we create an ElementTree object and extract the root from it. You’ll note that e print out the root and the root’s tag and attributes. Next we show several ways of iterating over the tags. The first loop just iterates over the XML child by child. This would only print out the top level child (appointment) though, so we added an if statement to check for that child and iterate over its children too.

Next we grab an iterator from the tree object itself and iterate over it that way. You get the same information, but without the extra steps in the first example. The third method uses the root’s getchildren() function. Here again we need an inner loop to grab all the children inside each appointment tag. The last example uses the root’s iter() method to just loop over any tags that match the string “begin”.

As mentioned in the last section, you could also use find() or findall() to help you find specific tags or sets of tags respectively. Also note that each Element object has a tag and a text property that you can use to acquire that exact information.

Wrapping Up

Now you know how to use ElementTree to create, edit and parse XML. You can add that information to your XML parsing toolkit and use it for fun or profit. You will find links to previous articles on some of the other XML parsing tools below as well as additional information about ElementTree itself.