Deconstructing Deserialization, Part 1

Remote sensing services will be deployed on wireless devices.
Services will communicate in XML, and will be implemented in Java.

Deserialization is the process of transforming on-the-wire to in-memory representations of data, but we will here use the the term to specifically denote the process of transforming XML to Java objects.

Deserialization APIs are generally designed with a specific goal in mind, e.g. code generators, JAXB or JAXRPC runtime, legacy etc, and are not primarily intended to be used by application programmers. This might lead to the, on J2SE and upwards, acceptable consequence of having several deserialization APIs deployed in the same JVM. In a constrained environment such as J2ME, however, this consequence is definitely unwanted, probably unacceptable and possibly unfeasible.

We will here design a deserialization API that satisfies

Schema deserialization has traditionally been performed during compile time, but in order to make it possible to implement applications such as web service browsers and WFS (see http://www.opengeospatial.org) clients, support for this activity will have to be available at run time.

Assuming the presence some but not all of SAX (e.g. the JSR-172 JAXP subset), we immediately realize that the SAX event model on its own will not suffice for anything but rather simple XML - something on top of the SAX ContentHandler is needed.

The ContentContext Interface

What we need are contexts, that are created from the current (parent) context each time startElement is called, and re-established from the current (child) context each time endElement is called.

To accomplish this we define the ContentContext interface, its default implementation DefaultContext and the SAX event adapter ContextHandler (click to enlarge):

public interface ContentContext {
    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts) throws SAXException;

    public void characters(char[] ch, int start, int length) throws SAXException;

    public ContentContext endElement(String namespaceURI, String localName, String qName) throws SAXException;

    public Object getValue() throws SAXException;

    public void endChild(String namespaceURI, String localName, String qName, Object value) throws SAXException;
}


public class ContextHandler extends DefaultHandler {
    protected ContentContext context;
    
    public ContextHandler(ContentContext context) {
	this.context = context;
    }

    public void startElement(String namespaceURI, String localName, String qName, Attributes atts) throws SAXException {
	context = context.startElement(namespaceURI, localName, qName, atts);
    }

    public void characters(char[] ch, int start, int length) throws SAXException {
	context.characters(ch, start, length);
    }

    public void endElement(String namespaceURI, String localName, String qName) throws SAXException {
	context = context.endElement(namespaceURI, localName, qName);
    }
}


public class DefaultContext implements ContentContext {
    protected final ContentContext parent;

    public DefaultContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent) {
	this.parent = parent;
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts) throws SAXException {
	return new DefaultContext(namespaceURI, localName, qName, atts, this);
    }

    public void characters(char[] ch, int start, int length) throws SAXException{
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value) throws SAXException {
    }

    public ContentContext endElement(String namespaceURI, String localName, String qName) throws SAXException {
	parent.endChild(namespaceURI, localName, qName, getValue());
	return parent;
    }

    public Object getValue() throws SAXException {
	return this;
    }
}

Parsing the familiar Purchase Order Example (assuming the JSR-172 JAXP subset) http://www.w3.org/TR/xmlschema-0/#po.xml

	ContentContext ctx = new DefaultContext();
	SAXParserFactory factory = SAXParserFactory.newInstance()
	factory.newSAXParser().parse(Connector.openInputStream(getAppProperty("po.xml")), new ContextHandler(ctx));

results in the following sequence of events:

The Purchase Order

In order to find out if this design satisfies the needs of a JAXB runtime, we manually create classes for the types defined in the familiar example schema po1.xsd (all interfaces, some methods and strict JAXB compliance omitted for clarity).

class PurchaseOrderTypeImpl {
    USAddressImpl shipTo;
    USAddressImpl billTo;
    String comment;
    ItemsImpl items;
    String orderDate;

    public String toString() {
	return "Ship the following items to:\n" + shipTo + items;
    }
}


class USAddressImpl {
    String name;
    String street;
    String city;
    String state;
    String zip;
    String country;

    public String toString() {
	return "\t" + name + "\n\t" + street + "\n\t" + city + ", " + state + " " + zip + "\n\t" + country;
    }
}


class ItemsImpl {
    Vector items = new Vector();

    public String toString() {
	StringBuffer sb = new StringBuffer();
	for (int i = 0; i < items.size(); i++)
	    sb.append("\n").append(items.elementAt(i));
	return sb.toString();
    }
}


class itemTypeImpl {
    public String productName;
    public String quantity;
    public String USPrice;
    public String comment;
    public String shipDate;
    public String partNum;
    
    public String toString() {
	return "\t" + quantity + " copies of \"" + productName + "\"";
    }
}

Additionally, we have to implement contexts for each schema type plus the "bootstrap" context RootContext (which, incoincidently, bears a vague resemblance to a JAXB ObjectFactory). Note that schema type implementations must not be ContentContexts since these contain state relevant only during deserialization.

public class RootContext extends DefaultContext {
    Object root = null;

    RootContext() {
	super("", "#document", "#document", null, null, null);
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts) throws SAXException {
	if (localName.equals("purchaseOrder"))
	    return new PurchaseOrderTypeContext(namespaceURI, localName, qName, atts, this, new PurchaseOrderTypeImpl());

	else if (localName.equals("comment"))
	    return new StringContext(namespaceURI, localName, qName, atts, this);

	else
	    throw new SAXParseException(qName, null);
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value) throws SAXException {
	root = value;
    }

    public Object getValue() {
	return root;
    }
}


class StringContext extends DefaultContext {
    StringBuffer stringBuffer = new StringBuffer();

    StringContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent) {
	super(namespaceURI, localName, qName, atts, parent);
    }

    public void characters(char[] ch, int start, int length) throws SAXException {
	stringBuffer.append(ch, start, length);
    }

    public Object getValue() throws SAXException {
	return stringBuffer.toString();
    }
}


class PurchaseOrderTypeContext extends DefaultContext {
    protected PurchaseOrderTypeImpl purchaseOrderType;

    PurchaseOrderTypeContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent, PurchaseOrderTypeImpl purchaseOrderType) {
	super(namespaceURI, localName, qName, atts, parent);
	this.purchaseOrderType = purchaseOrderType;
	purchaseOrderType.orderDate = atts.getValue("orderDate");
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts) throws SAXException {
	if (localName.equals("shipTo"))
	    return new USAddressContext(namespaceURI, localName, qName, atts, this, new USAddressImpl());

	else if (localName.equals("billTo"))
	    return new USAddressContext(namespaceURI, localName, qName, atts, this, new USAddressImpl());

	else if (localName.equals("comment"))
	    return new StringContext(namespaceURI, localName, qName, atts, this);

	else if (localName.equals("items"))
	    return new ItemsContext(namespaceURI, localName, qName, atts, this, new ItemsImpl());

	else
	    throw new SAXParseException(qName, null);
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value) throws SAXException {
	if (localName.equals("shipTo"))
	    purchaseOrderType.shipTo = (USAddressImpl)value;

	else if (localName.equals("billTo"))
	    purchaseOrderType.billTo = (USAddressImpl)value;

	else if (localName.equals("comment"))
	    purchaseOrderType.comment = (String)value;

	else if (localName.equals("items"))
	    purchaseOrderType.items = (ItemsImpl)value;
    }

    public Object getValue() throws SAXException {
	return purchaseOrderType;
    }
}


class USAddressContext extends DefaultContext {
    protected USAddressImpl usAddress;

    USAddressContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent, USAddressImpl usAddress) {
	super(namespaceURI, localName, qName, atts, parent);
	this.usAddress = usAddress;
	this.usAddress.country = atts.getValue("country");
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts) throws SAXException {
	return new StringContext(namespaceURI, localName, qName, atts, this);
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value) throws SAXException {
	if (localName.equals("name"))
	    usAddress.name = (String)value;

	else if (localName.equals("street"))
	    usAddress.street = (String)value;

	else if (localName.equals("city"))
	    usAddress.city = (String)value;
	else if (localName.equals("state"))
	    usAddress.state = (String)value;

	else if (localName.equals("zip"))
	    usAddress.zip = (String)value;

	else
	    throw new SAXParseException(qName, null);
    }

    public Object getValue() throws SAXException {
	return usAddress;
    }
}


class ItemsContext extends DefaultContext {
    protected ItemsImpl items;

    ItemsContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent, ItemsImpl items) throws SAXException {
	super(namespaceURI, localName, qName, atts, parent);
	this.items = items;
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts) throws SAXException {
	if (localName.equals("item"))
	    return new itemTypeContext(namespaceURI, localName, qName, atts, this, new itemTypeImpl());

	else
	    throw new SAXParseException(qName, null);
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value) throws SAXException {
	items.items.addElement(value);
    }

    public Object getValue() throws SAXException {
	return items;
    }
}


class itemTypeContext extends DefaultContext {
    protected itemTypeImpl itemType;

    itemTypeContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent, itemTypeImpl itemType) {
	super(namespaceURI, localName, qName, atts, parent);
	this.itemType = itemType;
	this.itemType.partNum = atts.getValue("partNum");
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts) throws SAXException {
	return new StringContext(namespaceURI, localName, qName, atts, this);
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value) throws SAXException {
	if (localName.equals("productName"))
	    itemType.productName = (String)value;

	else if (localName.equals("quantity"))
	    itemType.quantity = (String)value;

	else if (localName.equals("USPrice"))
	    itemType.USPrice = (String)value;

	else if (localName.equals("comment"))
	    itemType.comment = (String)value;

	else if (localName.equals("shipDate"))
	    itemType.shipDate = (String)value;

	else
	    throw new SAXParseException(qName, null);
    }

    public Object getValue() throws SAXException {
	return itemType;
    }
}

A sequence diagram of parsing po1.xml looks like:

And indeed, we find that executing

	ContentContext ctx = new RootContext();
	SAXParserFactory factory = SAXParserFactory.newInstance();
	factory.setNamespaceAware(true);
	factory.newSAXParser().parse(Connector.openInputStream(getAppProperty("po1.xml")), new ContextHandler(ctx));
	System.err.println(ctx.getValue());

results in the correct output:

Ship the following items to:
	Alice Smith
	123 Maple Street
	Mill Valley, CA 90952
	US
	1 copies of "Lawnmower"
	1 copies of "Baby Monitor"

Moreover, the tediousness of the "Impl" and context classes makes it likely that these can be machine-generated.

But the code feels unnecessarily complex: Two classes have been manually created for each schema type, where only one ought to be necessary.

The Deserializer Interface

A closer look at the Purchase Order-specific ContentContext implementations reveals that very little (if any) of the overridden methods modify the state of the context itself. This brings us the idea of having one single context implementation that delegates its operations to an object that implements yet another interface.

This interface is the Deserializer interface. Since a Deserializer is not a context, the signatures of the methods in the interface will have to be basically the same as the context methods with an additional context argument, something that makes the deserializer stateless, at least from the point-of-view of the ContentContext implementation DeserializerContext.

public interface Deserializer {
    public ContentContext createContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent) throws SAXException;

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts, ContentContext context) throws SAXException;

    public void characters(char[] ch, int start, int length, ContentContext context) throws SAXException;

    public void endChild(String namespaceURI, String localName, String qName, Object value, ContentContext context) throws SAXException;

    public Object endElement(ContentContext context) throws SAXException;

    public Object getValue(ContentContext context) throws SAXException;
}


public class DeserializerContext implements ContentContext {
    protected final Deserializer deserializer;

    public DeserializerContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent, Deserializer deserializer) throws SAXException {
	super(namespaceURI, localName, qName, atts, parent);
	this.deserializer = deserializer;
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts) throws SAXException {
	return deserializer.startElement(namespaceURI, localName, qName, atts, this);
    }

    public void characters(char[] ch, int start, int length) throws SAXException{
	deserializer.characters(ch, start, length, this);
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value) throws SAXException {
	deserializer.endChild(namespaceURI, localName, qName, value, this);
    }

    public ContentContext endElement(String namespaceURI, String localName, String qName) throws SAXException {
	parent.endChild(namespaceURI, localName, qName, deserializer.endElement(this));
	return parent;
    }

    public Object getValue() throws SAXException {
	return deserializer.getValue(this);
    }
}


public class DefaultDeserializer implements Deserializer {
    public ContentContext createContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent) throws SAXException {
	return new DeserializerContext(namespaceURI, localName, qName, atts, parent, this);
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts, ContentContext context) throws SAXException {
	return createContext(namespaceURI, localName, qName, atts, context);
    }

    public void characters(char[] ch, int start, int length, ContentContext context) throws SAXException {
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value, ContentContext context) throws SAXException {
    }

    public Object endElement(ContentContext context) throws SAXException {
	return getValue(context);
    }

    public Object getValue(ContentContext context) throws SAXException {
	return this;
    }
}

This might at first not seem to decrease the number of classes at all: How is complexity decreased when we instead of having to implement ContentContext have to implement Deserializer?

The answer is of course that since a Deserializer implementation is stateless, the interface can be implemented by any class, in particular by a class derived from a schema type: The number of manually generated classes is hence decreased to five: four "Impl" classes plus the RootDeserializer (which, still incoincidently, bears a somewhat stronger resemblance to a JAXB ObjectFactory than the RootContext in previous example).

public class RootDeserializer extends DefaultDeserializer {
    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts, ContentContext context) throws SAXException {
	if (localName.equals("purchaseOrder"))
	    return new PurchaseOrderTypeImpl().createContext(namespaceURI, localName, qName, atts, context);

	else if (localName.equals("comment"))
	    return new StringDeserializer().createContext(namespaceURI, localName, qName, atts, context);

	else
	    throw new SAXParseException(localName, null);
    }
}


class DocumentContext extends DeserializerContext {
    Object root = null;

    DocumentContext(Deserializer deserializer) throws SAXException {
	super("", "#document", "#document", null, null, null, deserializer);
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value) throws SAXException {
	root = value;
    }

    public Object getValue() throws SAXException {
	return root;
    }
}


class StringDeserializer extends DefaultDeserializer {
    public ContentContext createContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent) throws SAXException {
	return new StringBufferContext(namespaceURI, localName, qName, atts, parent, this);
    }

    public Object getValue(ContentContext context) throws SAXException {
	return context.toString();
    }
}


class StringBufferContext extends DeserializerContext {
    StringBuffer stringBuffer = new StringBuffer();

    StringBufferContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent, Deserializer deserializer) throws SAXException {
	super(namespaceURI, localName, qName, atts, parent, deserializer);
    }

    public void characters(char[] ch, int start, int length) throws SAXException {
	stringBuffer.append(ch, start, length);
    }

    public String toString() {
	return stringBuffer.toString();
    }
}


class PurchaseOrderTypeImpl extends DefaultDeserializer {
    USAddressImpl shipTo;
    USAddressImpl billTo;
    String comment;
    ItemsImpl items;
    String orderDate;

    public String toString() {
	return "Ship the following items to:\n" + shipTo + items;
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts, ContentContext context) throws SAXException {
	Deserializer d;

	if (localName.equals("shipTo"))
	    d = new USAddressImpl();

	else if (localName.equals("billTo"))
	    d = new USAddressImpl();

	else if (localName.equals("comment"))
	    d = new StringDeserializer();

	else if (localName.equals("items"))
	    d = new ItemsImpl();

	else
	    throw new SAXParseException(localName, null);

	return d.createContext(namespaceURI, localName, qName, atts, context);
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value, ContentContext context) throws SAXException {
	if (localName.equals("shipTo"))
	    shipTo = (USAddressImpl)value;

	else if (localName.equals("billTo"))
	    billTo = (USAddressImpl)value;

	else if (localName.equals("comment"))
	    comment = (String)value;

	else if (localName.equals("items"))
	    items = (ItemsImpl)value;
    }
}


class USAddressImpl extends DefaultDeserializer {
    String name;
    String street;
    String city;
    String state;
    String zip;
    String country;

    public String toString() {
	return "\t" + name + "\n\t" + street + "\n\t" + city + ", " + state + " " + zip + "\n\t" + country;
    }

    public ContentContext createContext(String namespaceURI, String localName, String qName, Attributes atts, ContentContext parent) throws SAXException {
	country = atts.getValue("", "country");
	return super.createContext(namespaceURI, localName, qName, atts, parent);
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts, ContentContext context) throws SAXException {
	return new StringDeserializer().createContext(namespaceURI, localName, qName, atts, context);
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value, ContentContext context) throws SAXException {
	if (localName.equals("name"))
	    name = (String)value;

	else if (localName.equals("street"))
	    street = (String)value;

	else if (localName.equals("city"))
	    city = (String)value;

	else if (localName.equals("state"))
	    state = (String)value;

	else if (localName.equals("zip"))
	    zip = (String)value;

	else
	    throw new SAXParseException(localName, null);
    }
}


class ItemsImpl extends DefaultDeserializer {
    Vector items = new Vector();

    public String toString() {
	StringBuffer sb = new StringBuffer();
	for (int i = 0; i < items.size(); i++)
	    sb.append("\n").append(items.elementAt(i));
	return sb.toString();
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts, ContentContext context) throws SAXException {
	return new itemTypeImpl().createContext(namespaceURI, localName, qName, atts, context);
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value, ContentContext context) throws SAXException {
	items.addElement(value);
    }
}


class itemTypeImpl extends DefaultDeserializer {
    public String productName;
    public String quantity;
    public String USPrice;
    public String comment;
    public String shipDate;
    public String partNum;
    
    public String toString() {
	return "\t" + quantity + " copies of \"" + productName + "\"";
    }

    public ContentContext startElement(String namespaceURI, String localName, String qName, Attributes atts, ContentContext context) throws SAXException {
	return new StringDeserializer().createContext(namespaceURI, localName, qName, atts, context);
    }

    public void endChild(String namespaceURI, String localName, String qName, Object value, ContentContext context) throws SAXException {
	if (localName.equals("productName"))
	    productName = (String)value;

	else if (localName.equals("quantity"))
	    quantity = (String)value;

	else if (localName.equals("USPrice"))
	    USPrice = (String)value;

	else if (localName.equals("comment"))
	    comment = (String)value;

	else if (localName.equals("shipDate"))
	    shipDate = (String)value;

	else
	    throw new SAXParseException(localName, null);
    }
}

We find that changing the code to make use of the new classes results in a different sequence of events but produces the same output as before.

	ContentContext ctx = new RootContext()new DocumentContext(new RootDeserializer());
	SAXParserFactory factory = SAXParserFactory.newInstance();
	factory.setNamespaceAware(true);
	factory.newSAXParser().parse(Connector.openInputStream(getAppProperty("po1.xml")), new ContextHandler(ctx));
	System.err.println(ctx.getValue());

So far so good.

There are, however, a number of things that we have completely ignored. One of these things is the support for application-level namespace-awareness, i.e. support for colonized attribute values. This support is required by applications making use of e.g. xsi:type, but also by schema deserializers, which is the subject of our next example.

To be continued...

© Copyright 2004 by Johan Appelgren AB. All rights reserved.