DOM benchmarks

Overview

The DOM benchmarks measure how fast different DOM implementations are when building and traversing documents:

• Css4j-DOM4J module (which subclasses DOM4J).
• Css4j's native DOM.
• Stand-alone DOM4J.
• JDK's bundled DOM implementation.

As a reference, some of the benchmarks were performed with Jsoup which does not implement DOM but is a popular HTML parser and pseudo-DOM library. It is only being benchmarked when the document's markup is (X)HTML.

The tables and charts were generated by Carte (look for the BenchmarkChartWriter class and the examples folder in the carte-jmh module). The grey lines at the top of the bars give the amplitude of the estimated error.

The computer has an Intel® Core™ i5-1035G7 CPU and 8GB of RAM. Unfortunately, being a laptop processor it is affected by thermal management, which may explain some relatively large error intervals.

(*) Note: one of the tested DOM implementations is the one that comes bundled with the JDK (identified as "JDK" in the graphics), and it has been observed that the version shipped with the Oracle JDK may be faster.

Build HTML documents

Measures the speed at which the validator.nu HTML parser can parse a small document (38 KB HTML) into a few DOM implementations. As a reference, they are compared to the same document parsed by Jsoup.

Numeric results (higher is better):

The standard DOM implementations are somewhat even in this test but Jsoup is much faster, about ten times the JDK and 12 times the speed of the others.

Build small XML documents

The SAX parser that comes bundled with the JDK is used to parse and build a document from a small XHTML file (38 KB).

Numeric results (higher is better):

Build XML documents

The SAX parser that comes bundled with the JDK is used to parse and build a document (1MB file).

Numeric results (higher is better):

DOM traversal: getFirstChild()/getNextSibling()

Count the nodes of an XML document, using a combination of getFirstChild()/getNextSibling() to traverse it.

Numeric results (higher is better):

Note: for unknown reasons, the usual procedure to build the JDK document with a DocumentBuilderFactory could not be used to initialize the document traversed in the benchmark, as that document is somehow left in an inconsistent state with no child nodes; this happened with the initialization code being executed in a Scope.Benchmark class and also when in a static initialization block. When that same code is executed in a JUnit test, the problem is not seen.

Because of this reason the JDK DOM document was built with css4j's XMLDocumentBuilder, although that process is not part of the timed benchmark.

DOM traversal: getLastChild()/getPreviousSibling()

Count the nodes of an XML document, using a combination of getLastChild()/getPreviousSibling() to traverse it.

Numeric results (higher is better):

The Css4j-DOM4J results are representative for DOM4J as well.

DOM traversal: NodeIterator (small file)

Count the elements of a 38kB XHTML document traversed by a NodeIterator.

Numeric results (higher is better):

DOM4J and Jsoup are not included as they lack a NodeIterator.

DOM traversal: NodeIterator

Count the elements of an XML document traversed by a NodeIterator.

Numeric results (higher is better):

DOM4J is not included as it lacks a NodeIterator.

Note: sometimes the NodeIterator created by the JDK is in an inconsistent state, and fails with an exception like:

# Warmup Iteration   1: <failure>

java.lang.ArrayIndexOutOfBoundsException: Index 34 out of bounds for length 33
        at java.base/java.util.ArrayList.add(ArrayList.java:455)
        at java.base/java.util.ArrayList.add(ArrayList.java:467)
        at java.xml/com.sun.org.apache.xerces.internal.dom.DocumentImpl.createNodeIterator(DocumentImpl.java:255)
        at io.sf.carte.mark.dom.DOMIteratorMark.markNodeIteratorJdk(DOMIteratorMark.java:46)
</failure>

But I have observed this only while benchmarking, and not in other cases.

DOM traversal: TreeWalker (small file)

Count the elements of a 38kB XHTML document traversed by a TreeWalker.

Numeric results (higher is better):

Neither DOM4J nor Jsoup provide a TreeWalker.

DOM traversal: TreeWalker

Count the elements of an XML document traversed by a TreeWalker.

Numeric results (higher is better):

As mentioned, neither DOM4J nor Jsoup provide a TreeWalker.

DOM traversal: iterator() (small file)

Traverse a 38kB XHTML document using native DOM's iterable getChildNodes(), DOM4J's nodeIterator() and Jsoup's iterable childNodes().

Numeric results (higher is better):

The JDK's DOM provides no iterable child collections.

DOM traversal: iterator()

Traverse an XML document using native DOM's iterable getChildNodes() and DOM4J's nodeIterator().

Numeric results (higher is better):

The JDK's DOM provides no iterable child collections, and Jsoup is not suitable for XML documents (neither was included).

DOM traversal: elementIterator() (small file)

Traverse a 38kB XHTML document using native DOM's elementIterator(), DOM4J's elementIterator() and Jsoup's iterable Elements.

Numeric results (higher is better):

The JDK's DOM provides no element iterator.

DOM traversal: elementIterator()

Traverse an XML document using native DOM's elementIterator() and DOM4J's elementIterator().

Numeric results (higher is better):

The JDK's DOM provides no element iterator, and Jsoup is not suitable for XML documents (neither was included).

DOM traversal: getElementsByTagName() (small file)

Traverse the list given by getElementsByTagName() from a 38kB document (it is an XHTML file so Jsoup is included in the comparison). In the case of css4j's native DOM, there are two results: one iterating the NodeList by the item() method, and another via the iterator (the returned NodeList implements Iterable).

Numeric results (higher is better):

DOM traversal: getElementsByTagName()

Traverse the list given by getElementsByTagName(), this time from a 1MB document. Again, for css4j's native DOM there are two results: one iterating the NodeList by the item() method, and another via the iterator (the returned NodeList implements Iterable).

Numeric results (higher is better):

Note: the iterator is documented as the recommended way to traverse the ElementList in css4j since version 3.2. Css4j versions prior to 3.2 perform much better in this benchmark, but the implementation was switched to one that is more lightweight (and the iterator performance is good enough).

DOM modification: appendChild()/removeChild()

Modify the nodes of an XML document by appending elements with appendChild() and later removing them with removeChild().

Numeric results (higher is better):

Analysis

Building a document with DOM4J (both plain DOM4J and the CSS-enabled subclasses that CSS4J provide) is fast, but has an important scalability problem. It uses a synchronized cache of QName objects, as shown by the benchmark profiler (e.g. java -jar build/benchmarks.jar XMLBuildBenchmark -prof stack:lines=5;top=3;detailLine=true;period=1):

Secondary result "io.sf.carte.mark.dom.XMLBuildBenchmark.markBuildDOM4J: stack":
Stack profiler:

....[Thread state distributions]....................................................................
 72,3%         BLOCKED
 27,6%         RUNNABLE

....[Thread state: BLOCKED].........................................................................
 72,3% 100,0% java.util.Collections$SynchronizedMap.get
              org.dom4j.tree.QNameCache.get
              org.dom4j.DocumentFactory.createQName
              org.dom4j.tree.NamespaceStack.createQName
              org.dom4j.tree.NamespaceStack.pushQName

So its usage is not recommended for multi-core systems.

Css4j's native DOM has more features than the other contenders (which implies a bit of overhead) but is still quite fast, although it is often slower than the JDK's DOM, which is the fastest standard DOM. The latter looks like a good choice for applications that do not require handling styles (or you could use the read-only DOM wrapper with it, if that fits your needs). Finally, DOM4J is lagging behind in performance for anything other than document build-up.

For users that do not need to handle CSS and do not mind dealing with a non-standard API, Jsoup excels at parsing and has good traversal speeds, although is several orders of magnitude slower in the document modification benchmark.