Some tiny typing mistakes in your XSLT stylesheet can cause major headaches. Why? Because they go undetected by the standard static and dynamic error checking mechanisms. Something as simple as writing /Filename where it should be /FileName will only show up if your own test/debug efforts find it and will not be trapped by the XSLT engine. Unless… you use the XSLT V2.0 mechanism for type checking.

What’s up?

Assume you have an XML document that looks like this:

<Things>
   <Thing thingid="12345FFD3">...</Thing>
   <Thing thingid="86779EAD0">...</Thing>
   ...
</Things>

In your XSLT stylesheet you have a named template that does something with a thing:

<xsl:template name="ProcessThing">
  <xsl:param name="ThingId"/>
  <xsl:for-each select="/Things/Thing[@id eq $ThingId]">
    <!-- Do something with the thing -->
  </xsl:for-each>
</xsl:template>

Spot the mistake: The author of this template forgot (or didn’t know) that the identifier attribute on a thing was spelled thingid and not id. If you run this, no error message will show up, the for-each loop will simply not execute. Maybe you notice, maybe you don’t (because it was embedded somewhere in a very complicated transformation…). The same for all spelling mistakes in element/attribute names in XPath expressions (like /Things/thing[@thingid eq $ThingId]).

Not nice at all: Your code went into production, the spelling mistake prevented an important part of some calculation from happening. Because of this the client paid too much taxes and now he/she can’t pay you…

Preventive measures

So what can you do? Double check, reread, ferocious debugging, keep fingers crossed, pray… certainly, do it all! But what you really (should) want is the bug to show up in a booming error message. Here is how to do this using the XSLT V2.0 type mechanism (this won’t work in V1.0).

XSLT V2.0 is type aware. You can specify what data type a variable (or parameter) must have for it to be valid. Although this certainly is no panacea for all mistakes, typing errors in XPath expressions can be caught by it.

For instance, if you rewrite the above example as this, the error will show up immediately:

<xsl:template name="ProcessThing">
  <xsl:param name="ThingId"/>
  <xsl:variable name="ThingToProcess" as="element(Thing)"
    select="/Things/Thing[@id eq $ThingId]"/>
  <xsl:for-each select="$ThingToProcess">
    <!-- Do something with the thing -->
  </xsl:for-each>
</xsl:template>

At runtime, the stylesheet processor will notice that /Things/Thing[@id eq $ThingId] does not return the expected <Thing> element but the empty set and will complain about it.

There is even a bonus: If the input document contains an error and has two things with the same id an error will show up also: The $ThingToProcess variable can only contain a single thing, not multiple!

To make this template even better, I would personally also type the parameter:

<xsl:param name="ThingId" as="xs:string" required="yes"/>

This traps forgetting the parameter (because of the required="yes") and also passing empty or multiple values.

Trapping mistakes in the input document

The “types against typos” mechanism can also be used in trapping input document mistakes. (This assumes that you haven’t validated your input document, which is of course the best way to do this, but you know how it is…).

Before using anything, put them in a typed variable first:

<xsl:variable name="Status" as="xs:string" select="@status"/>

No attribute means no value (which differs from the empty string!) and an error will stop the stylesheet.

Namespaces are inevitable when you work with XML. In the beginning they seem awkward and unnecessary, but after a while you find out that they have a purpose and are actually very useful.

When you start designing your own XML, sooner or later you’ll want your own namespace too. And there an important question arises: How to call your namespace, what name should your namespace have?

There are customs and, as usual, many opinions about this. Let me add mine…

Namespace basics

Before going to the actual subject, let’s do a quick recap on namespaces.

The main reason for namespaces is to differentiate between several types of XML in the same document. It’s handy for avoiding element and attribute name clashes and makes their meaning more clear. Here is a classic example: Assume you’re in an XML document about a book and the following element appears:

<name>Johnny Johnson</name>

Is this the name of a book? Or an author? Or a publisher? Adding a namespace clarifies this:

<name xmlns="authornamesnamespace">Johnny Johnson</name>

Another classic example is the envelope-content construction as used, for instance, in SOAP messaging:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns:msg="mymessagenamespace">
  <soapenv:Header>
   […]
  </soapenv:Header>
  <soapenv:Body>
    <msg:MyMessage>Hello world!</msg:MyMessage>
  </soapenv:Body>
</soapenv:Envelope>

Prefix and name

Never confuse the namespace prefix in an XML document with the namespace name. In the example above soapenv is the namespace prefix for the namespace with the name schemas.xmlsoap.org/soap/envelope/.

Changing the prefix is no problem. You can use anything you like (within the rules of a no-colon-name, an NCName). For instance, this means exactly the same:

<bettyboop:Envelope xmlns:bettyboop="http://schemas.xmlsoap.org/soap/envelope/" […]

However, changing the namespace name is not allowed. Your SOAP software will no longer recognize your messages if you do something like:

<soapenv:Envelope xmlns:soapenv="somethingilikemore"

It expects the envelope to be in the right namespace and so it should be!

So, what you choose for a prefix is not important (at least not for the meaning of your document, it can be important for its legibility), but choosing a correct namespace name is!

What’s in a name?

As long as a namespace name complies with the syntactic rules for a URI, you can name a namespace anything you like. It’s just a string of characters and nothing stops you from calling your namespace mumbledumbledoozydoe.

However, the reason domain names (like xmlsoap.org) are often used inside namespace names, is that it makes the namespace globally unique. This is piggy-backing on the existing mechanism for the registration of domain names: This ensures that nobody in the world can register a namespace already belonging to somebody else. Ergo, a domain name is globally unique and so a namespace using the domain name is globally unique also.

So its wise to use your domain name inside a namespace name when your XML is send out into the big wide world.

Naming conventions for namespaces

There are several conventions for naming namespaces, but the most widely used is one that looks like a web address or URL. Examples are everywhere:

http://schemas.xmlsoap.org/soap/envelope/

http://www.w3.org/1999/XSL/Transform

http://www.w3.org/1999/xhtml

http://www.w3.org/2001/XMLSchema

I know its fighting windmills, but I am very strongly opposed to this convention. Why? Well, because we are using things that look like web addresses for things that aren’t web addresses at all. A namespace is not a web address, it’s a namespace!

What do you think when you see a namespace formatted like this for the first time? Exactly, that it is a URL. It creates a mental barrier understanding what this is all about. I must admit that I have suffered from this myself. But I also train people in XML basics and the concept of namespaces is very hard to explain because of this URL convention.

So IMHO this is a silly convention, unnecessary complicating a very simple concept. We should stop using it.

An alternative

Ok, but what should we use instead? A good naming convention for namespaces must have two characteristics:

• It must be clear from the start that it is a namespace and not something else.
• It must be easy to make a namespace globally unique

To accomplish this, I propose the following convention (for which the general structure was stolen from the standard for a URN):

ns:{domain name}:{further information}

For instance:

ns:xatapult.nl:generic
ns:myowndomain.com:storage.text.specials
ns:html8.org:linking

There: simple, straight and it cannot be mistaken for something else.

Looking at the XML that is around, everybody seems to use different standards for element and attribute names. Some people write their elements in all uppercase, like <USERNAME>, some use UpperCamelCase, like <UserName>, or lowerCamelCase, like <userName>. Some standards, like XSLT, use all lowercase, like <xsl:value-of>. There is no uniformity, in contrast to, for instance, Java, where everybody more or less adheres to the same writing and coding rules.

So should we have a standard and what should it look like? Let’s explore.

Why coding standards?

Before writing this Blog I re-consulted the excellent book “C Style: Standards and guidelines” by David Straker, written in 1992. The C programming language might be a bit out of fashion, the general parts of the book still hold. David Straker thoroughly answers questions like: Why do we need standards and what should these look like.

I will not repeat all he has to say about it, but the baseline is: Coding standards are for humans. To make life as a designer/programmer easier. To make things more understandable for somebody else (or yourself when you revisit the code after a year or so).

A good standard takes human psychological limitations into account. For instance, computers don’t give a damn, but for humans <CustomerContactPersonUsernameCodeAbbreviation> is easier to read than <customercontactpersonusernamecodeabbreviation>. And a standard with just a few (good) rules is easier to follow than an extensive document with umpteen pages.

How exactly do (well designed!) coding standards help?

• As a designer/programmer, you no longer have to think about simple things. Thoughts like “Shall I write this in upper or lower case?”, “I created something like this a week ago; .I want to stay consistent, so what rules did I use to name it?”. In short, it frees parts of your mind to think about more important things.
• Reading somebody else’s designs or programs or whatever becomes much easier: You can recognize things in the way they are written or layed-out.

What shall we standardize?

Just a few lists with things we might (or might not) standardize:

Basic XML:

• Basic naming rules for elements and attributes
• For common types/usages prefixes or suffixes? For instance, I end the names of attributes that hold an identifier always with id (e.g. userid="…") and attributes that refer to an identifier with idref (e.g. currentuseridref="…")
• Namespace names (and prefix?)
• Maybe lay-out for comments…?

XML Schema (XSD):

• Type and group names (maybe something that distinguishes between simple and complex types, attribute and element groups)
• Versioning of schemas (a very tricky subject)
• Usage of container elements on repeating data (do you always need a <Customers> element around multiple <Customer> records?)

XML Transformations (XSLT):

• Variable/Parameter names
• Named template names (and for XSLT 2.0: functions)
• Modes

And so on, and so forth.

My standards

The way I design and write my XML is (after all these years) still developing. However, some things seem to have settled. For what its worth:

• Element names in UpperCamelCase (<UserName>, <CarModel>)
• Attributes in all lowercase (username, isvalid)
• An attribute that is an identifier ends with id (userid, carid)
• An attribute that is a reference to an identifier ends with idref (currentuseridref, modelidref)
• For XML schemas I use “Hungarian style” prefixes like t for a simple type, ct for a complex type, g for a group, etc.
• In XML Transformations elements (variables, templates, etc.) I use the UpperCamelCase convention. Modes always start with the word Mode.

To stay consistent, I have a document (which has the permanent status “working copy”, or “in progress”) in which I, now and then, write these kinds of things down. For me it’s the only way not to forget what I have decided about these things in the past.

Bottom line

IMHO:

A standard, even a simple one, is better than none at all.

Some standards are better than others.

Creating a good standard is balancing between too much and too little.

Its not necessary to have a “global” standard, but each company that is serious about its XML should have one.

In an XML document you can express the same semantics in syntactically very different ways. Which is a difficult way of saying: Watch out, XML documents may look different but can nonetheless mean the same! If you use the right tools to parse and interpret the XML that is not a problem. However, be aware for developers that do not…

This article explores the areas that cause most of the confusion in interpreting XML.

True stories

This really happened: One of the companies I worked with had to interpret SOAP XML messages coming from another company. Unfortunately they decided to use regular expressions (no, this is not a joke) to interpret the XML!

Of course in the beginning all went well. The XML messages came in, the regular expressions parsed them and miraculously produced the right results. Until… the company that sent the messages decided to upgrade their SOAP framework. After this the same messages were sent but the regular expression were no longer doing their job.

I can’t remember exactly what the new framework did, but it was a completely innocent thing with regards to the meaning of the messages. Something like an extra line break in between elements, a different namespace prefix, a different way of declaring namespaces or changing the order of the attributes. Nothing a true XML parser would have worried about, but the regular expressions stopped working.

Another example was an application that suddenly refused to handle a valid XML document because a namespace prefix was introduced (instead of using a default namespace). In this case a XML parser was used, but the handling of namespaces was definitely not correct.

Namespaces

Very often problems interpreting the XML are due to the handling of namespaces. Have a look at the examples below:

<data xmlns="basenamespace" xmlns:e="embeddednamespace">
   <e:content>bla bla bla</e:content>
</data>

<data xmlns="basenamespace" >
   <content xmlns="embeddednamespace">bla bla bla</content>
</data>

<b:data xmlns:b="basenamespace" xmlns:whatever="embeddednamespace">
   <whatever:content>bla bla bla</whatever:content>
</b:data>

As you might have guessed or seen, these examples all mean the same, albeit the different syntax. The following example however means something completely different, although at first glance it looks the same:

<data xmlns="basenamespace" xmlns:e="embeddednamespace">
   <content>bla bla bla</content>
</data>

Missed it? In the first three examples the inner <content> element was in the namespace embeddednamespace. In the last example it was in the basenamespace namespace. The last example did declare the embeddednamespace namespace but does not use it, something which might be confusing but is perfectly valid.

• Use an XML parser that can handle namespaces and all the variations in their declarations. Most can but some older or very simple ones do not (examples are the basic XML parsers in PHP and Perl).
• Always take the namespace of an element or attribute into account when interpreting the XML.
• Never ever use the namespace prefix (or the absence of it) to distinguish between namespaces. Always use the full namespace name.
• Do not get confused by the declaration of superfluous namespaces.

Text

Text is just text you might think. However it is not. Have a look at the following examples which all mean exactly the same (WATCH OUT: I inserted a space right before every entity ending semicolon. Otherwise WordPress will not display everything correct. Remove this space to try the examples):

<p>What are you looking at?</p>
<p><![CDATA[What are you looking at?]]></p>
<p>What are you looking at<![CDATA[?]]></p>
<p>What are you looking at&#x3F ;</p>
<p>What are you looking at&#x3f ;</p>
<p>&#x57 ;&#x68 ;&#x61 ;&#x74 ;&#x20 ;&#x61 ;&#x72 ;&#x65 ;&#x20 ;
&#x79 ;&#x6F ;&#x75 ;&#x20 ;&#x6C ;&#x6F ;&#x6F ;&#x6B ;&#x69 ;
&#x6E ;&#x67 ;&#x20 ;&#x61 ;&#x74 ;&#x3F ;</p>

So reading text is not straight forward. It might contain entities or CDATA sections and these must be handled well.

Whitespace

Handling whitespace is a very tricky area. Whether or not it is important relies on the nature of the XML. In data oriented XML, whitespace in between elements is usually not significant. Probably (but we cannot be absolutely sure just looking at the XML) the following two examples mean the same:

<codes><code>code1</code><code>code2</code></codes>

<codes>
   <code>code1</code>
   <code>code2</code>
</codes>

But what about these more text oriented examples?

<p><b>bold</b> <i>italic</i> <u>underline</u></p>

<p>
   <b>bold</b>
   <i>italic</i>
   <u>underline</u>
</p>

Again, it depends on how the XML is used.

• When processing data oriented XML, whitespace in between elements is usually not significant and can be discarded. This also allows pretty printers to make the XML more readable without changing its meaning.
• In more text oriented XML, things get more difficult. Be aware that whitespace in between elements might be significant.

Other pitfalls

• The order of attributes is not significant. <hello type="greeting" meaning="friendly"/> is the same as <hello meaning="friendly" type="greeting/>.
• XML is case sensitive. <Hello/> is not the same as <hello/> or <HELLO/>.
• Attributes can be enclosed in single or double quotes. <hello type="greeting"/> is the same as <hello type='greeting'/>

And probably numerous others. If you have anything to add I would like to hear from you.

Conclusion

Unless you are in the business of writing XML parsers (and not many are), don’t try to do it yourself. Most development environments have excellent software on board to do it for you. After the parsing you can access the XML using SAX or DOM or some custom interface.

XML looks simple but has an awful lot of nitty-gritty details you need to take care of interpreting it. And never rely on simple string parsing…

How do you associate a schema with an XML document? It’s easy but the details also easily forgotten. A summary and a how-to.

Why?

• You have an XML document and wants it validated against one or more W3C XML schemas. How do you tell the validator which schemas to use?
• Your XML editor can display useful hints and prevent creating invalid documents when it knows which schemas to use.

How?

• Associations between XML documents and W3C XML Schemas are created by adding specific attributes in the www.w3.org/2001/XMLSchema-instance namespace.
• To be able to do this, you have to specify this namespace in your XML document, usually on the root element:

<RootElement xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" …>
    …
</RootElement>

• The usual namespace prefix is xsi. Better stick to this to avoid confusion.
• Use the xsi:noNamespaceSchemaLocation to specify the location of a schema file for the part(s) of your document that have no namespace attached (or when you don’t use namespaces at all):

<RootElement xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:noNamespaceSchemaLocation="http://bogus.com/schemas/schemaforthisdocument.xsd"
    …
>
   …
</RootElement>

• The xsi:noNamespaceSchemaLocation attribute is put on the root element. Only in pathological circumstances (non-namespaced inside namespaced XML) you might have a reason to put it elsewhere.
• For namespaces, use the xsi:schemaLocation attribute. The content of this attribute consists of pairs of (namespace-name; schema-location-for-this-namespace), all separated by whitespace. For instance:

<RootElement xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="namespace1 schema-location-for-namespace1
                        namespace2 schema-location-for-namespace2
                        …"
   …
>
   …
</RootElement>

• There are several strategies where to put xsi:schemaLocation attribute(s):
• You can use a single xsi:schemaLocation attribute on the root element, specifying all namespaces and schema locations. Since namespaces and schema locations are usually long strings, this will soon become very unreadable.
• You can use an xsi:schemaLocation attribute on the first occurrence of an element/attribute in a specific namespace, specifying only the association for this particular namespace. This is cleaner but spreads the information on associated schemas all over your document.
• You can make a mess of it by mixing both approaches or by specifying locations on elements that have nothing to do with the namespace involved. The tools won’t mind but readers will.

Pitfalls and peculiarities?

• Values in the xsi:… attributes have the official status of hints to the schema processor. So they can be ignored, treated wrong or overridden by something else.
• Some tools have a tool-specific way to associate a document with its schemas. This usually overrides the values set in the xsi:… attributes, since these are only hints…
• If the schema locations contain relative path information, this is resolved against the location of the XML document. So be careful how you specify the file locations if you ever plan to move your documents elsewhere. Unless of course you always transfer your documents and schemas in one go.
• The contents of the xsi:schemaLocation attribute very soon becomes a hard to interpret visual mess. You can hand-edit it nicely (as in the examples) but an XML pretty printer will undo all your hard work very quickly. Not much we can do about it…

More information?

• Have a look at the rather incomprehensible official specification: www.w3.org/TR/xmlschema-1
• For a simple tutorial: www.w3schools.com/schema/schema_howto.asp

Once upon a time XML was born from SGML. And along with its birth came DTDs to define the document structure. Life was good. Everybody used to writing DTDs for SGML could keep doing so. And so they did…

But what happened? Newer and shinier methods for describing XML structures came along. W3C Schemas, Relax NG and others saw the light. Suddenly things that were impossible to do with DTDs became feasible: data typing, design modularization and many, many more. Wow! Suddenly you could really be strict about your document structure.

And so what happened in the ivory towers from which the gods send us their XML standards for us mere mortals to use? Where the standards accompanied by schemas in addition to the traditional DTDs? Yes, they were! Hurrah, a step forward. Now we can really and truly verify our documents.

But look closely. Are this schemas? Technically… yes. However they look an awful lot like DTDs. If I am not mistaken most of the DTDs are simply converted into schemas.

All right, so what? The standard developers (at least, most of them) seem to have chosen to keep on using DTDs. As a gesture to us humans they add schemas, but only schemas converted from their DTDs. No data typing, no modularization, nothing but elements and attributes.

And so we stay in the dark ages of XML design and miss all the opportunities to be more strict. But what’s even worse, it introduces an ambiguity: You can now validate documents with a completely different root element than intended…

Multiple root elements?

As an illustrative example, let’s assume we want to define the structure for this very complex XML document:

<Names>
   <Name>Erik</Name>
   <Name>John</Name>
   <!-- etc. -->
</Names>

A DTD for this could look like this:

<!ELEMENT Names ((Name+))>
<!ELEMENT Name (#PCDATA)>

This DTD defines that <Names> is the root element because it is not part of any other element’s definition. But if we convert this DTD directly into a schema (I use the build-in XML Spy convertor) something like this appears:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
   <xs:element name="Names">
       <xs:complexType>
           <xs:sequence>
               <xs:choice>
                   <xs:element ref="Name" maxOccurs="unbounded"/>
               </xs:choice>
           </xs:sequence>
       </xs:complexType>
   </xs:element>
   <xs:element name="Name">
       <xs:complexType mixed="true"/>
   </xs:element>
</xs:schema>

So, what’s wrong with this: The root element has become ambiguous! For instance, this is a perfectly valid XML document according to our brand new schema:

<Name>Erik</Name>

Not exactly what we would like, is it?

But is it a problem?

Yes. Definitely. Period.

DTDs were introduced to be able define the structure of our XML documents unambiguous. Schemas were introduced to make this even better. But now, by actually using schemas, we make it worse.

It could be the cause of all kinds of subtle and not so subtle errors when invalid XML, which a validator accepts undeserved, passes through systems. It might open backdoors for hackers because invalid data is accepted without raising alarms. And you could spent many unproductive hours debugging something that might have been detected very easily.

Oh, by the way: The schema feature to have more than one root elements is not a bug as such. There are situations you actually need it. For instance when you on purpose want to have a schema that allows multiple root elements. Or when you define a schema with elements for use inside another namespace.

What to do about it?

Do not blindly convert your DTDs into schemas and think you’re finished.

Create your schemas (or adapt the conversions) in such a way that only one root element is possible (unless of course you mean something else on purpose).

And to get the best results: Use the features a schema language possesses to define the structure of your XML as tight as possible. And that is a lot more tight than a DTD!