XML serialization

Code for the example is under the cmajor/projects/xmlSerializationTests directory in the attr_model and attr_test projects.

1 Example

Suppose you are writing a traffic simulation application. A traffic simulation may contain a person class and classes for various vehicles:

A person has a name and it may own some vehicles. A vehicle has a color and a pointer to its owner. A car derives from a vehicle and has a registration number. A bicycle derives also from a vehicle and has a price.

In this example we build classes that can be serialized to XML and constructed from an XML representation.

1.1 XML Classes

Cmajor provides an automatic mechanism for serializing objects as XML. This works by declaring classes with an [xml] attribute. The compiler generates ToXml and FromXml member functions for classes having that attribute. They do the actual serialization to and from XML.

// person.cm:

using System;
using System.Collections;
using System.Xml.Serialization;

namespace model
{
    [xml]
    public class Person
    {
        // ...
        public string name;
        public List<UniqueXmlPtr<Vehicle>> vehicles;
    }
}

The Person class has an [xml] attribute, so the compiler generates XML serialization code for it. The member variables of an XML class are serialized automatically unless they have an [xml="false"] attribute, so name and vehicles are serialized when Person object is serialized.

In addition to ToXml and FromXml member functions, the compiler generates member variables for storing an integer class identifier for an XML serializable class and stroring an object ID for each XML serializable object. They can be accessed using the ClassId and ObjectId member functions.

The UniqueXmlPtr is a unique ownership pointer class that contains an object ID of the object it points to. A UniqueXmlPtr pointer deletes the pointed-to object when it goes out of scope.

// vehicle.cm:

using System;
using System.Collections;
using System.Xml.Serialization;

namespace model
{
    public enum VehicleColor
    {
        red = 0,
        green = 1,
        blue = 2,
        black = 3,
        white = 4
    }

    [xml]
    public abstract class Vehicle
    {
        // ...
        public VehicleColor color;
        public XmlPtr<Person> owner;
    }
}

Value of an enumerated type such as VehicleColor is serialized as a long value. This handles the color member variable of the Vehicle class. For the owner member variable, we have used the XmlPtr class that is like an ordinary pointer but in addition it contains an object ID of the object it points to.

The Car class derives from the Vehicle class, so the compiler overrides the ToXml and FromXml member functions. The overridden member functions call the base class member functions and in addition have serialization code for properties of the derived class:

// car.cm:

using System;
using System.Collections;
using System.Xml.Serialization;

namespace model
{
    [xml]
    public class Car : Vehicle
    {
        // ...
        public string registrationNumber;
    }
}

// bicycle.cm:

using System;
using System.Collections;
using System.Xml.Serialization;

namespace model
{
    [xml]
    public class Bicycle : Vehicle
    {
        // ...
        public float price;
    }
}

1.2 Saving

Now we are ready to create an example model consisting of XML classes and save the model to an XML file. Here's the code that does this:

// save.cm:

using System;
using System.IO;
using System.Collections;
using System.Xml.Serialization;

namespace model
{
    public void Save(const string& xmlFileName)
    {
        XmlContainer container; // XmlContainer does not own the objects

        UniqueXmlPtr<Car> car = new Car();
        car->SetRegistrationNumber("ABC-123");
        car->SetColor(VehicleColor.red);
        Add(car.Get(), &container);

        UniqueXmlPtr<Bicycle> bicycle = new Bicycle();
        bicycle->SetColor(VehicleColor.green);
        bicycle->SetPrice(1000.0f);
        Add(bicycle.Get(), &container);

        UniqueXmlPtr<Person> person(new Person());
        person->SetName("Joe Coder");
        Add(person.Get(), &container);
        person->AddVehicle(car.Release(), true);
        person->AddVehicle(bicycle.Release(), true);

        XmlBundle bundle = container.CreateBundle(person.Get(), XmlBundleKind.deep);

        string xmlStr = bundle.ToXmlString();
        StreamWriter xmlFile = File.CreateText(xmlFileName);
        xmlFile.WriteLine(xmlStr);
    }
}

We have created an instance of the XmlContainer class, and then created instances of Car, Bicycle and Person classes and added them to the container. The container does not own the objects added to it, so have we used UniqueXmlPtr for the Car, Bicycle and Person. The vehicles are released and added to the person object, because the person takes ownership of the vehicles added to it.

Then we have created an XmlBundle from the container by calling the CreateBundle function of the container. The XML bundle contains a reference to a root object, an object passed to the CreateBundle function, and also references to objects reachable from the root by traversing them using the XmlPtr and UniqueXmlPtr pointer objects. There are two kinds of bundles: shallow and deep. The shallow container does not contain other than the root object, but the deep container contains closure of objects reachable from the root up a depth. By default the depth is -1 which means the depth is unlimited.

Finally we have serialized the bundle to a string by calling the ToXmlString member function of the bundle, created a text file, and written the XML string to the file.

1.3 XML

The object IDs are string representations of the UUID class. For the XmlPtr and UniqueXmlPtr only the object ID of the pointed-to object is serialized. When reading an XmlBundle from an XML file, the bundle can recreate relations of the objects using the contained object IDs. The className attributes in the XML file are not used by the implementation in any way. They are generated so that classes can better be recognized by human readers.

1.4 Registration

For the system to be able to create XML serializable classes when reading their XML representation, the classes has to be registered. The registration is done by calling the compiler-generated static Register member function of an XML serializable class with an integer class identifier. The class identifiers should be unique within a program:

// register.cm:

using System;
using System.Xml.Serialization;

namespace model
{
    public const int carClassId = 0;
    public const int bicycleClassId = 1;
    public const int personClassId = 2;

    public void Register()
    {
        Car.Register(carClassId);
        Bicycle.Register(bicycleClassId);
        Person.Register(personClassId);
    }

    new class ConcreteXmlSerializableExtractor<Vehicle>;
    new class ConcreteXmlSerializableExtractor<Car>;
    new class ConcreteXmlSerializableExtractor<Bicycle>;
    new class ConcreteXmlSerializableExtractor<Person>;
}

In addition to registration code, we have added full instantiation requests for the ConcreteXmlSerializableExtractor class template specializations of Vehicle, Car, Bicycle and Person classes, because the compiler is not able to automatically generate instantiations for them.

1.5 Loading

Now that we have the classes registered, they can be read from an XML file. Here's the code that loads the model:

// load.cm:

using System;
using System.IO;
using System.Collections;
using System.Xml.Serialization;

namespace model
{
    public UniquePtr<Person> Load(const string& fileName)
    {
        string xmlStr = File.ReadAllText(fileName);
        UniquePtr<XmlBundle> bundle = ToXmlBundle(xmlStr, fileName);
        UniqueXmlPtr<Person> root = XmlCast<Person>(bundle.Root());
        if (root.IsNull())
        {
            throw Exception("Person expected");
        }
        return root;
    }
}

The ToXmlBundle function parses an XML document from the passed XML string, creates the XML classes using class identifiers contained by the object elements, and resolves the relations of objects using object IDs. By default the bundle returned from the ToXmlBundle function destroys objects that are not owned by UniqueXmlPtrs.

1.6 Main

// main.cm:

using System;
using System.Xml.Serialization;
using model;

public int main()
{
    try
    {
        Register();
        Save("traffic.xml");
        UniqueXmlPtr<Person> person = Load("traffic.xml");
        // use person...
    }
    catch (const Exception& ex)
    {
        Console.Error() << ex.ToString() << endl();
        return 1;
    }
    return 0;
}

2 Implementation

Implementation of the XML serialization scheme is done in half by the compiler and in half by the System.Xml.Serialization library.

The intf_model and intf_test projects under the cmajor/projects/xmlSerializationTests directory contain manual implementation of the XmlSerializable interface for the example classes Person, Vehicle, Car and Bicycle. The compiler implements this interface automatically for classes having an [xml] attribute.

2.1 Participants

2.1.1 XmlSerializable Interface

The XmlSerializable interface has a key role in the XML serialization scheme. The compiler implements the XmlSerializable interface for each class that has an [xml] attribute. In the picture above the MyXmlSerializableClass represents such a class. The MyXmlSerializableClasses do not have to derive from a common base class. We need an interface because Cmajor lacks multiple inheritance. This interface contains member function signatures for generating an XML representation, the ToXml method, and on the other hand reading member variable values from an XML representation, the FromXml method. It also enables putting of XML serializable classes to an XmlContainer or to an XmlBundle.

2.1.2 ToXml and FromXml

When the compiler generates an implementation of the ToXml member function for a MyXmlSerializableClass, it generates a call to one of the ToXml overloads contained by the System.Xml.Serialization library for each member variable of the MyXmlSerializableClass. Similarly when the compiler generates an implementation of the FromXml member function of a MyXmlSerializableClass, it generates a call to one of the FromXml overloads contained by the System.Xml.Serialization library for each member variable of the MyXmlSerializableClass. The overloads are selected by the compiler using concept-based overloading.

2.1.3 XmlSerializableProxy

The reason that the XmlContainer does not directly contain XmlSerializable interfaces is because interface objects cannot be directly put to containers in Cmajor. We need an intermediate class that is called XmlSerializableProxy that contains the interface object. Instances of XmlSerializableProxy class on the other hand can be put to containers,. So the Add member function of the XmlContainer creates a proxy object that holds the interface object and adds the proxy. The member functions of the XmlSerializableProxy delegate calls to the contained interface.

There's also an excuse why you can not directly put MyXmlSerializableClass* to an XmlContainer*, but you have to call free Add function template to do that. It's because Cmajor lacks member function templates.

2.1.4 XmlContainer

The XmlContainer class has two mappings. From an object IDs to an XmlSerializableProxy* and from MyXmlSerializableClass* (represented as void*) to an XmlSerializableProxy*. This means that you can get XmlSerializableProxy* by object ID or by object pointer. You can also remove object and XmlSerializableProxy* by object ID, and create an XmlBundle from an object pointer.

2.1.5 XmlBundle

An XmlBundle contains a root object and objects reachable from the root using those XmlPtrs and UniqueXmlPtrs.

When constructing an XmlBundle from an XML string, the ToXmlBundle function uses the XmlClassRegistry singleton to create a MyXmlSerializableClass associated with the class identifier read from an object XML element contained by the XML representation of the bundle.

2.1.6 XmlSerializableExtractor and ConcreteXmlSerializableExtractor

The XmlSerializableExtractor is an abstract class that contains an abstract member function for creating an XmlSerializable interface from an object pointer. The ConcreteXmlSerializableExtractor overrides that member function for some class registered to XmlClassRegistry by a class identifier.

2.1.7 XmlClassRegistry

The XmlClassRegistry is a singleton class that has two maps and can create a MyXmlSerializableClass using class identifier, and extract an XmlSerializable interface from that object pointer using an XmlSerializableExtractor. The first map associates a class identifier to a function pointer that creates an object and the second map associates a class identifier to an XmlSerializableExtractor.