Main menu:

Site search



October 2014
« Sep    


Some notes on Nokogiri::XML::Reader

I’m a fan of Nokogiri, and I use it for all of my ruby XML processing needs. Recently, however, I needed to use the Nokogiri::XML::Reader class and had some trouble finding good examples so I’m putting some here.

Why might you want to use Nokogiri::XML::Reader? Probably because you have a very large XML file to process and reading the whole thing into memory, the way you do using the more standard Nokogiri::XML class, is going to be slow. For example, I started using Reader when one of our workflow robots needed to process a 2Mb XML file. Trying to process this file via DOM methods (i.e., the standard Nokogiri::XML method) was taking over a minute per file. Once we switched to using Nokogiri::XML::Reader, processing the same file took about 4 seconds.

The official docs are available at, but they don’t provide many examples beyond the most trivial ones. The crucial bit is this (adapted from the nokogiri docs):

reader = Nokogiri::XML::Reader(
reader.each do |node|

The idea is that the cursor moves forward through the document, and you have to get any info you want as the cursor moves past the part of the document you need. One pattern I encountered several times was this:

reader = Nokogiri::XML::Reader(
reader.each do |node|
when 'PREMIS:premis'
# do PREMIS stuff
when 'something_else'
# do something else

That lets you switch on the node name, which is pretty useful, but falls short in some important cases. First of all, it’s not obvious that with this method both <PREMIS:premis> AND </PREMIS:premis> are going to match that when statement. It also isn’t clear how to use this to match nested elements, for example if you only want PREMIS data when it occurs inside a specific element.

After some experimentation, here’s a method I like more. Consider this psudo code. I’m leaving out things that seem irrelevant to the Nokogiri::XML::Reader structure. Also note that node.node_type==1 will restrict to matching only opening elements. This processes each <repInfo> node, and then within that node you can use to move the cursor forward as needed.

def generate_jhove_hash
reader = Nokogiri::XML::Reader(
reader.each do |node|
if('repInfo' and node.node_type==1)
# do some stuff until'format' # to get to the inside of the format tag
jhove_hash[current_uri]["format"] = node.value until'status' # to get to the inside of the status tag
jhove_hash[current_uri]["status"] = node.value
if(current_uri =~ /jp2/) until'mix:imageWidth'
jhove_hash[current_uri]["width"] = node.value until'mix:imageHeight'
jhove_hash[current_uri]["height"] = node.value
return jhove_hash


Comment from Jason Ronallo
Time: December 11, 2010, 8:37 pm

You may also want to check out the SAX parser which is a similar kind of interface and is also very fast. In that case all you have is the ability to have a start tag, end tag and characters in between. Because this kind of event driven parsing is so different, something like gives a nice syntax for defining classes.

Write a comment