Pseudo closures in XS

This article describes pseudo closures in XS, a.k.a. XS code templates, which creates XSUBs dynamically without compilation. Because XSUBs are typically much faster than pure Perl subroutines, creating XSUBs without compilation will be useful for high performance applications.

Closure

What are closures? A closure is a subroutine that consists of binding (or external) data and code, which allows us to create subroutines dynamically without compilation, used especially in class builders. The following example is a simplest closure that create a counter, which consists of $i and the increment operator (++):

my $i       = 42;           # binding data
my $closure = sub { $i++ }; # data + code

Although this is a technique to make subroutines dynamically, the dynamic part is only the binding data, while the code part is statically defined. Thus, this technique can be applied to XSUBs.

An XSUB is a Perl subroutine which has a C function, and a subroutine is a type of SV. In Perl internal world, any SVs are able to have struct MAGIC, which can have other SVs. That is, we can bind SVs to XSUBs. However, Perl API doesn't support closures directly, so I call this technique pseudo closures.

Implementation

The actual implementation explained here is taken from Mouse, which is a Moose
compatible class builder with an XS implementation. The accessors Mouse provides are implemented in XS with this pseudo closure technique. Its basic idea is the same as Perl's: defining XSUBs statically, then binding Perl data to these XSUBs in run-time.

Take the rw accessor from Mouse/xs-src/MouseAccessors.xs for example:

/* snip */
#define dMOUSE_self  SV* const self = \
        mouse_accessor_get_self(aTHX_ ax, items, cv)
/* snip */
XS(XS_Mouse_accessor)
{
    /* setup XSUB args */
    dVAR; dXSARGS;
    dMOUSE_self; /* my $self = shift @_ */
    MAGIC* const mg = (MAGIC*)XSANY.any_ptr; /* fetch the bound data */

    SP -= items; /* PPCODE */
    PUTBACK;

    if(items == 1){ /* reader */
        mouse_attr_get(aTHX_ self, mg);
    }
    else if (items == 2){ /* writer */
        mouse_attr_set(aTHX_ self, mg, ST(1));
    }
    else{
        mouse_throw_error(MOUSE_mg_attribute(mg), NULL,
            "Expected exactly one or two argument for an accessor of %"SVf,
            MOUSE_mg_slot(mg));
    }
}

This is the prototype of all the rw accessor (and of course, other accessors are defined in the file). The data part is attached as a MAGIC, but available from XSANY.any_ptr for convenience.

The following function is an XSUB generator which creates an XSUB and bind data to the XSUB:

CV*
mouse_accessor_generate(pTHX_ SV* const attr, XSUBADDR_t const accessor_impl){
    AV* const xa = mouse_get_xa(aTHX_ attr);
    CV* xsub;
    MAGIC* mg;

    xsub = newXS(NULL, accessor_impl, __FILE__);
    sv_2mortal((SV*)xsub);

    mg = sv_magicext((SV*)xsub, MOUSE_xa_slot(xa),
        PERL_MAGIC_ext, &mouse_accessor_vtbl, (char*)xa, HEf_SVKEY);

    MOUSE_mg_flags(mg) = (U16)MOUSE_xa_flags(xa);

    /* NOTE:
     * although we use MAGIC for gc, we also store mg to
     * CvXSUBANY for efficiency (gfx)
     */
    CvXSUBANY(xsub).any_ptr = (void*)mg;

    return xsub;
}

XS_Mouse_accessor is passed to this function as an accessor_impl parameter. The attr parameter is an instance of Mouse::Meta::Attribute. The bound data are managed by the Perl interpreter, they are released in the same way as Perl's closures.

This technique might not be easy for everyone, but its power is fascinating. Enjoy metaprogramming in XS!

Closure

Implementation

See Also

Author