Hello everybody.

This post has been on my to-do list for a while. It’s going to describe the tricks we use in our SWIG interfaces to export QuantLib to a handful of languages (for example, you might know that I’m fond of using it from Python).

I’m hoping that this post can enable and encourage others to contribute to the SWIG wrappers. There are quite a few features of the C++ library which are not exported, and I’d welcome warmly any pull requests aimed at fixing this state of affairs. If you pick up the gauntlet, you can open them at the QuantLib-SWIG GitHub repository.

In other news, last Friday I’ve released QuantLib 1.8.1. It’s a bug-fix release for version 1.8, mostly to prevent the test suite from failing with the upcoming Boost 1.62 release; you can find the (very short) list of changes here.

And in case you’re interested, why yes, there are still places available for my Introduction to QuantLib Development course: click the link for details.

Follow me on Twitter if you want to be notified of new posts, or add me to your Google+ circles, or subscribe via RSS: the buttons for that are in the footer. Also, make sure to check my Training page.

QuantLib and SWIG

One of the objectives of SWIG (I’m assuming you’re at least a bit familiar with what it does, so I’ll jump right into the matter) is to be able to parse C++ headers directly and generate wrapper code. Why, then, do we use a separate set of interface files which requires to be kept in sync with the C++ code?

One reason is that we often want to add features to the wrappers, and SWIG provides a number of directive for that; for instance, we can specify that a given function should use keyword arguments in Python, or we can rename methods returning a bool so that they end with a question mark in Ruby. This, however, could be done in a simpler interface file that would contain the desired SWIG directives and then include the relevant C++ header file.

Historically, the real reason was that the C++ library made pervasive use of smart pointers, and we didn’t want them to show up in the languages to which we were exporting our code; none of them had even the concept of pointer, and forcing users to wrap their objects in shared_ptr instances in order to pass them around would be against any idiomatic use.

The SWIG people are aware of this problem, and in more recent versions (you’ll pardon me if I don’t bother tracking down version numbers) most languages have built-in support for shared_ptr, so that objects in the host language can be passed transparently to underlying C++ functions and methods expecting a shared_ptr argument.

A side note: as a matter of fact, I’d very much like to reorganize our wrappers to use the built-in support, but unfortunately that’s not available for all the languages we export QuantLib to; namely, Perl support is missing. (Also Ocaml, MzScheme and Guile, but I suspect those wrappers haven’t been working for some time and I wouldn’t lose my sleep if I had to drop them.) If any of you has some interest in this, I’m sure the SWIG people would be glad to accept a contribution.

Anyway: we started using SWIG some 15 years ago, and built-in support wasn’t available. Therefore, we had to roll our own, which forced us to write custom interface files for most classes.

Let’s say we have a class that is passed around as a shared_ptr in the library; for instance, YieldTermStructure. Now, SWIG has had some support for smart pointers even then. If you declared a class with an operator->, say,

class Foo {
  public:
    void bar();
};

class Ptr {
  public:
    Foo* operator->();
};

SWIG would generate code that let you call the bar() method on Ptr instances, just like you can call ptr->bar() in C++; so, in the same way, you could export shared_ptr<T> and call methods on the underlying T instances. We took advantage of this, and added to our interfaces a minimal declaration of the shared_ptr class including the relevant operator->.

However, as I said, we didn’t want to expose pointers at all; so exporting, say, a YieldTermStructurePtr alongside YieldTermStructure wasn’t good enough. Instead, we masqueraded the pointer as the actual class and wrote:

%ignore YieldTermStructure;
class YieldTermStructure {
  public:
    DayCounter dayCounter() const;
    Calendar calendar() const;
    Date referenceDate() const;
    Date maxDate() const;
    Time maxTime() const;
    DiscountFactor discount(const Date&, bool extrapolate = false);
    DiscountFactor discount(Time, bool extrapolate = false);
    // ... other methods
};

%template(YieldTermStructure) boost::shared_ptr<YieldTermStructure>;

In the above, the %ignore directive tells SWIG not to export the YieldTermStructure class we’re declaring right after; and the %template directive tells SWIG to export the pointer and to call it YieldTermStructure in the target language. The exported fake YieldTermStructure class would still have all the methods that we declared in the real one, because of the operator-> in shared_ptr. Yes, sneaky.

However, it’s not as simple as this. When we pass around a class wrapped in a shared_ptr, it’s probably because we want to use it in a polymorphic way; and when inheritance enters the picture, it opens a whole can of worms.

The problem is that YieldTermStructure is just the base class, and an abstract one at that. You also need to export derived classes; let’s take FlatForward as an example. What do you do? Exporting shared_ptr<FlatForward> and calling it FlatForward in the target language won’t work: if you tried to pass it to a function taking a shared_ptr to YieldTermStructure, the typechecking code in the SWIG wrappers would stop you. Even if a class U inherits from T, shared_ptr<U> doesn’t inherit from shared_ptr<T> and thus the argument types don’t match (the conversion would work in C++, but SWIG isn’t aware of that).

What we came up with was the following:

%{
typedef shared_ptr<YieldTermStructure> FlatForwardPtr;
%}

%rename(FlatForward) FlatForwardPtr;
class FlatForwardPtr : public shared_ptr<YieldTermStructure> {
  public:
    %extend {
        FlatForwardPtr(const Date& referenceDate,
                       const Handle<Quote>& forward,
                       const DayCounter& dayCounter,
                       Compounding compounding = Continuous,
                       Frequency frequency = Annual) {
            return new FlatForwardPtr(
                new FlatForward(referenceDate,forward,dayCounter,
                                compounding,frequency));
        }
    }
};

The code between %{ ... %} braces is inserted in the wrapper code, and it tells the compiler (but not SWIG, that copies it without parsing) that the FlatForwardPtr type is actually just a shared_ptr to YieldTermStructure. Then, we tell SWIG that we’re exporting a FlatForwardPtr class which inherits from shared_ptr<YieldTermStructure>.

Now, let’s say that we pass a FlatForwardPtr instance to a function that takes a shared_ptr to YieldTermStructure. As far as SWIG is concerned, we’re passing a derived class to a function taking its base class, so the typechecking code won’t complain; and when the function is actually called, the compiler won’t complain either, because it sees that the type we’re passing is just a typedef to the expected shared_ptr.

We’re still missing a constructor for the class, though, because shared_ptr doesn’t provide the one declared in the true FlatForward class; so we attach it to the exported class by means of the %extend SWIG directive. The contructor code takes the passed arguments, builds a FlatForward instance, and stores it into a shared_ptr; thus, when we create an instance of FlatForward in the target language, we’re actually creating a shared_ptr to a YieldTermStructure in the underlying C++ library.

Dizzy? It’s not over yet. What if a derived class declares a few methods which are not in the base class, and we want to export them to the host language? For instance, let’s say that we have this hierarchy, using the same trick as above:

%ignore Index;
class Index {
  public:
    Calendar fixingCalendar() const;
    bool isValidFixingDate(const Date& fixingDate) const;
    Real fixing(const Date& fixingDate,
                bool forecastTodaysFixing = false) const;
    void addFixing(const Date& fixingDate, Rate fixing);
};

%template(Index) shared_ptr<Index>;

%{
typedef shared_ptr<Index> InterestRateIndexPtr;
%}

%rename(InterestRateIndex) InterestRateIndexPtr;
class InterestRateIndexPtr : public shared_ptr<Index> {
    ...
};

and we want to expose from the InterestRateIndex class the tenor method, which is not defined in the base Index class. Just declaring it in the interface wouldn’t work, because SWIG would try to call it on shared_ptr<Index> instances and fail. What we have to do is:

%rename(InterestRateIndex) InterestRateIndexPtr;
class InterestRateIndexPtr : public shared_ptr<Index> {
  public:
    %extend {
      Period tenor() {
          return dynamic_pointer_cast<InterestRateIndex>(*self)->tenor();
      }
    }
};

The above attaches the method to the class using the %extend directive; dereferences the self variable, which SWIG provides, to get the shared_ptr<Index> wrapped by the instance; downcasts it to a shared_ptr to InterestRateIndex; and finally calls the tenor method and returns the result. Just calling (*self)->tenor wouldn’t work, because self is a pointer to the base class. The downcast never fails, because we’ll only ever call this code with instances of InterestRateIndexPtr; thus, we don’t need to check for null pointers before or after the cast.

Another common scenario: what if we want to export a method that takes a pointer to a derived class? For instance, the constructor of the VanillaSwap class takes a shared_ptr to IborIndex, which inherits from InterestRateIndex and is exported to SWIG as usual:

%{
typedef boost::shared_ptr<Index> IborIndexPtr;
%}

%rename(IborIndex) IborIndexPtr;
class IborIndexPtr : public InterestRateIndexPtr {
  public:
    %extend {
        // constructors etc.
    }
};

If we exported the VanillaSwap constructor in the obvious way, we’d get

%{
typedef shared_ptr<Instrument> VanillaSwapPtr;
%}

%rename(VanillaSwap) VanillaSwapPtr;
class VanillaSwapPtr : public shared_ptr<Instrument> {
  public:
    %extend {
        VanillaSwapPtr(VanillaSwap::Type type, Real nominal,
                       const Schedule& fixedSchedule, Rate fixedRate,
                       const DayCounter& fixedDayCount,
                       const Schedule& floatSchedule,
                       const IborIndexPtr& index,
                       Spread spread,
                       const DayCounter& floatingDayCount) {
            return new VanillaSwapPtr(
                    new VanillaSwap(type, nominal,fixedSchedule,fixedRate,
                                    fixedDayCount,floatSchedule,index,
                                    spread, floatingDayCount));
        }
    }
};

but that wouldn’t work. The index parameter looks like an instance of a derived class, but for the compiler IborIndexPtr is just shared_ptr<Index> and thus can’t be passed to the VanillaSwap constructor. Instead, the correct code is:

%extend {
    VanillaSwapPtr(VanillaSwap::Type type, Real nominal,
                   const Schedule& fixedSchedule, Rate fixedRate,
                   const DayCounter& fixedDayCount,
                   const Schedule& floatSchedule,
                   const IborIndexPtr& index,
                   Spread spread,
                   const DayCounter& floatingDayCount) {
        shared_ptr<IborIndex> libor =
            dynamic_pointer_cast<IborIndex>(index);
        return new VanillaSwapPtr(
                new VanillaSwap(type, nominal,fixedSchedule,fixedRate,
                                fixedDayCount,floatSchedule,libor,
                                spread, floatingDayCount));
    }
}

in which we downcast the index variable before passing it to the VanillaSwap constructor.

One last trick. As you see above, the VanillaSwap class has an inner enumeration VanillaSwap::Type. That, too, can’t be just declared inside the VanillaSwapPtr interface, because the wrapper code would try to find it in the shared_ptr class and fail. The %extend directive can’t add enums, so we have to settle for data members instead:

class VanillaSwapPtr : public SwapPtr {
  public:
    %extend {
        static const VanillaSwap::Type Receiver = VanillaSwap::Receiver;
        static const VanillaSwap::Type Payer = VanillaSwap::Payer;
        // ...
    }
};

I guess that’s enough for one post. Hopefully, the above can explain most of the crazy that you’re going to find in our interfaces, so you can have a decent shot at adding more stuff to them (and sending me a pull request, of course, if you’re allowed to do so). But if I wasn’t clear enough, or if I skipped some important bit, please contact me and I’ll try again with a follow-up.