8. How bifrost fits together

(aka seeing C in Python pipelines)

Bifrost is based on two maxims: 1) by itself, C sucks. 2) by itself, Python sucks.

Bifrost bridges the two languages using some clever python libraries, a customized numpy array class, and enforcing some pipeline friendly design choices.

8.1. A trip to the C

Suppose you have some custom C code which you want to integrate into a bifrost pipeline. Here’s a nice and simple snippet that adds two arrays together:

for(int i=0; i < nelements; i+=1)
{
    x[i] = x[i] + y[i];
}

To get this into bifrost and use it in Python, you need to ‘bifrostify’ this code to accept bifrost’s special ndarray class.

8.1.1. Bifrost Python ndarrays

Bifrost has a special array class in python that plays nicely with C:

import bifrost as bf
a = bf.ndarray([1,2,3,4,5,6,7,8,9,10],  dtype='f32')
b = bf.ndarray([2,3,4,5,6,7,8,9,10,11], dtype='f32')

This ndarray object is very similar to the numpy.array, but it has a special method, .as_BFarray(). Essentially, as_BFarray returns a pointer to the numpy array’s memory address, along with some other useful stuff.

z = a.as_BFarray()

# Tab complete will show you this object has:
z.big_endian z.dtype      z.shape
z.conjugated z.immutable  z.space
z.data       z.ndim       z.strides

8.1.2. Bifrost C++ BFarray

In the bifrost C++ code, there is a matching BFarray that makes interfacing with Python straightforward. This is essentially a struct that provides the same info as the Python ndarray.

A usage example is worth a thousand words, so here is our simple snippet from before, after it’s been bifrostified:

BFstatus AddStuff(BFarray *xdata, BFarray *ydata)
{
    long nelements = num_contiguous_elements(xdata);

    float* x = (float *)xdata->data;
    float* y = (float *)ydata->data;

    for(int i=0; i < nelements; i +=1)
    {
       x[i] = x[i] + y[i];
    }

    return BF_STATUS_SUCCESS;
}

A full code example (with headers etc) can be found at the end.

8.1.3. Using your C++ code in Python

Now comes the cool part. You’ll need to edit three files:

  1. create add_stuff.cpp in src

  2. create add_stuff.h in src/bifrost

  3. edit src/Makefile and add add_stuff.o in the LIBBIFROST_OBJS part

Now, run make and make install from the root directory to rebuild the libbifrost.so library.

Once this is recompiled, open up iPython and try this:

import bifrost as bf
from bifrost.libbifrost import _bf

b = bf.ndarray([1,2,3,4,5,6,7,8,9,10], dtype='f32')
a = bf.ndarray([1,2,3,4,5,6,7,8,9,10], dtype='f32')
_bf.AddStuff(a.as_BFarray(), b.as_BFarray())
print a

That is: your AddStuff function is available in python via some ctypesgen magic. You can’t just pass a and b by themselves, but you can send their as_BFarray() output.

8.1.4. Wrapping up

Bravo, you’ve managed to run C++ code in bifrost! All the rest of the pipeline stuff is just python (to be continued…)

8.2. Example C++ codes

8.2.1. add_stuff.cpp

#include <bifrost/cpu_add.h>
#include <bifrost/array.h>
#include <bifrost/common.h>
#include <bifrost/ring.h>
#include <utils.hpp>
#include <stdlib.h>
#include <stdio.h>
#include <iostream>

extern "C" {
BFstatus AddStuff(BFarray *xdata, BFarray *ydata)
{
    long nelements = num_contiguous_elements(xdata);

    float* x = (float *)xdata->data;
    float* y = (float *)ydata->data;

    for(int i=0; i < nelements; i +=1)
    {
       x[i] = x[i] + y[i];
    }

    return BF_STATUS_SUCCESS;
}

}

8.2.2. bifrost/add_stuff.h

#include <bifrost/common.h>
#include <bifrost/array.h>

extern "C" {

BFstatus AddStuff(BFarray *xdata, BFarray *ydata);

}