The xmlstarlet utility for Linux

xmlstarlet is a powerful utility for Linux that lets you manipulate XML data from the command line and can be integrated into shell scripts. It is to XML what jq is to JSON.

Using xmlstarlet you can beautify XML output but also filter it like f.i. showing only a given node.

A wall of XML in, one word out​

Here is the file we'll play with, data.xml, exactly as a machine would have written it; everything on the same line, no format at all:

<?xml version="1.0" encoding="UTF-8"?><bookstore><book category="cooking"><title lang="en">Everyday Italian</title><author>Giada De Laurentiis</author><year>2005</year><price>30.00</price></book><book category="children"><title lang="en">Harry Potter</title><author>J K. Rowling</author><year>2005</year><price>29.99</price></book><book category="web"><title lang="en">XQuery Kick Start</title><author>James McGovern</author><author>Per Bothner</author><author>Kurt Cagle</author><author>James Linn</author><author>Vaidyanathan Nagarajan</author><year>2003</year><price>49.99</price></book><book category="web"><title lang="en">Learning XML</title><author>Erik T. Ray</author><year>2003</year><price>39.95</price></book></bookstore>?

Now let's ask it a question: give me the title of the book filed under the children category.

$ cat "data.xml" | xmlstarlet sel -t -v "//book[@category='children']/title"
Harry Potter▋

One command, one answer, and not a single line of parsing code.

Beautifying the file​

The second thing you'll use every day: making that wall readable, using the format action.

$ cat "data.xml" | xmlstarlet format --indent-spaces 4▋

<?xml version="1.0" encoding="UTF-8"?>?
<bookstore>
    <book category="cooking">?
        <title lang="en">Everyday Italian</title>?
        <author>Giada De Laurentiis</author>
        <year>2005</year>
        <price>30.00</price>
    </book>
    <book category="children">
        <title lang="en">Harry Potter</title>
        <author>J K. Rowling</author>
        <year>2005</year>
        <price>29.99</price>
    </book>
    <book category="web">
        <title lang="en">XQuery Kick Start</title>
        <author>James McGovern</author>
        <author>Per Bothner</author>?
        <author>Kurt Cagle</author>
        <author>James Linn</author>
        <author>Vaidyanathan Nagarajan</author>
        <year>2003</year>
        <price>49.99</price>
    </book>
    <book category="web">
        <title lang="en">Learning XML</title>
        <author>Erik T. Ray</author>
        <year>2003</year>
        <price>39.95</price>
    </book>
</bookstore>

Seeing It in Action with Docker​

Before installing anything, try both commands above in a throwaway container — data.xml, byte for byte the same file shown at the top of this article, is already waiting in the working directory.

Build and run it:

$ docker build -t xmlstarlet-demo .
[+] Building 9.4s (6/6) FINISHED
✔ exporting to image
$ docker run --rm -it xmlstarlet-demo
🐳 root ~/demo # cat data.xml | xmlstarlet sel -t -v "//book[@category='children']/title"
Harry Potter

Same file, same filter, same answer — no mkdir, no copy-pasting XML into a new file. Try cat data.xml | xmlstarlet format --indent-spaces 4 next to see the beautified version, or change the XPath expression to pull a different node.

Installing xmlstarlet​

To verify if xmlstarlet is already installed on your system, simply run which xmlstarlet. If you get xmlstarlet not found as an answer:

$ sudo apt-get update && sudo apt-get -y install xmlstarlet▋

$ xmlstarlet --version

1.6.1▋

Let's play​

To reproduce the two commands above on your machine, please start a Linux shell and run mkdir -p /tmp/xmlstarlet && cd $_, then create a new file called data.xml with the content displayed at the top of this article.

Understanding the XPath expression​

An XPath expression is just a path in the tree. Our root node is called bookstore, then we have one or more book and each book has a title:

<!-- highlight-next-line -->?
<bookstore>
    <!-- highlight-next-line -->
    <book category="cooking">?
        <!-- highlight-next-line -->
        <title lang="en">Everyday Italian</title>?
        [...]
    </book>
    [...]
</bookstore>

Walk that path from the root and you get every title of the file:

$ cat "data.xml" | xmlstarlet sel -t -v "/bookstore/book/title"

Everyday Italian
Harry Potter
XQuery Kick Start
Learning XML
▋

The expression used at the beginning of this article, //book[@category='children']/title, is the filtering version: give me each book; it doesn't matter where the book node is located (that's the // prefix); but only if it has an attribute named category and whose value is children. Then, if found, display its title.

Conclusion​

Two actions cover most of the day-to-day needs: format to make an XML file human-readable, and sel -t -v with an XPath expression to pull exactly the node you need out of it; both usable in a pipe, so both usable in a shell script.

Read the official documentation to learn more about xmlstarlet, and if your next file is a .json one, the jq utility for Linux is the very same story with a different syntax.