3. Create a Pipeline¶

3.1. Blocks in this Example¶

This setup will require bifrost blocks which:

1. Read in the .wav file.

2. Copy the raw data to the GPU.

3. Split the time axis into chunks which we can FFT over.

4. FFT this new axis.

5. Take the modulus squared of these FFTs.

6. Transpose this data into a format compatible with the sigproc writer.

7. Copy the data back to the CPU.

8. Convert the data into integer data types.

9. Write this data to a filterbank file.

This file could then be used to do things like calculating the beats per minute of the song at different points of time, or could be used to just view the frequency components of the song with time.

First, ensure you have a working Bifrost installation. You should also have some CUDA-compatible GPUs to run this example.

3.2. Code Walkthrough¶

Now, let’s create the pipeline.

The first thing to do is to actually load in Bifrost. Load in the base library as bf:

import bifrost as bf

Next, let’s load in some function libraries. We want blocks, which is the block module in Bifrost, which is a collection of previously-written blocks for various functionality,and views, which is a library for manipulations of ring headers.

import bifrost.blocks as blocks
import bifrost.views as views

Now, let’s create our data “source,” our source block. This is the block that feeds our pipeline with data. In this example, we work with a .wav file. I assume that you have own some sort of audio file and can convert it, using, e.g., online-convert.

Now, I want to load this into Bifrost. The syntax for this instance is:

raw_data = blocks.read_wav(['heyjude.wav'], gulp_nframe=4096)

Where ['heyjude.wav'] is a list of .wav files, which is in this case, a sample of Hey Jude. gulp_nframe is an argument passed to this block which tells it how many frames of data to gulp at once.

Some terminology:

• frame: One chunk of data. In this case, it is a single sample of the audio file. By setting gulp_nframe=4096, we tell the block to read in 4096 samples at a time, and put these into the ring buffer at once.

• gulp: One read or write of the ring buffer. Imagine the block taking a gulp of data. Then gulp_nframe is how many frames are in that gulp.

Now, raw_data is now a reference to a block object, which implicitly points at the ring buffer which will hold the raw .wav data.

Next, we want to put this data onto the GPU. Bifrost makes this simple. Insert a copy block as follows:

gpu_raw_data = blocks.copy(raw_data, space='cuda')

In this line we are telling Bifrost to create a new block, a copy block, and set its input to be the raw_data variable which is the source block for our audio file. Then, by setting space='cuda', we tell Bifrost to create a ring in GPU memory, and copy all of the contents of raw_data into this new ring. With this GPU ring, we can connect more blocks and do GPU processing.

Now, since we only want to do a Fourier transform at different parts of the song, not the entirety of the song, we want to chunk up this audio file into segments over which we can Fourier transform. This lets us get a frequency view at various points of the song. Since our data comes as one long time stream, we want to break it up into parts. Bifrost lets you do this without extra processing. You simply manipulate the header of the ring, which stores all of the descriptions for the ring. These manipulations are accomplished with views:

chunked_data = views.split_axis(gpu_raw_data, 'time', 256, label='fine_time')

What have we done here? We took gpu_raw_data, which is a block on the GPU, and which implicitly points to its output ring buffer which sits on the GPU, and put it into the split_axis view. We said take the 'time' axis of this ring, and break it up into 256-size chunks. Create a new axis for this data, and call that axis 'fine_time'.

Note that views are special in that they do not actually modify the data. They just modify the metadata, which lets blocks interpret the data differently.

Now, let’s actually do the processing. We want to do an FFT along this 256-size axis, and we want to do it on the GPU. Bifrost knows which ring is on the GPU and CPU, so this is implicitly passed to the FFT block:

fft_output = blocks.fft(chunked_data, axes='fine_time', axis_labels='freq')

So, in this line, we create a new block, an FFT block, and send in the chunked data. We tell the FFT block to perform the Fourier transform along the 'fine_time' axis, which is the one of 256-size chunks, and then after Fourier transforming, to name the output axis 'freq'.

The Bifrost FFT block wraps cuFFT, the CUDA FFT package, which is heavily optimized. Inside the block, bifrost.ndarray’s are being created from the output ring of chunked_data, and this is being passed with ctypes into a C++ function.

Next, we want to take the square of these FFTs. In Bifrost, this can be done with the detect block:

squared = blocks.detect(fft_output, mode='scalar')

This block takes in the output of the FFT (we are still on the GPU!), squares each element (the 'scalar' mode), and then puts this in a new ring, implicitly stored in the squared block object.

Now, after carefully reading the documentation for the transpose block, we decide that we want to arrange the axes such that it goes in as ['time', 'pol', 'freq']. To do this, we need to transpose it. Bifrost has the transpose block to do this. We simply type the desired output axis layout as the only argument, and it is all performed on the GPU for us:

transposed = blocks.transpose(squared, ['time', 'pol', 'freq'])

This transposes the axes for us. Now we can ship it into sigproc writer. But first, we have to offload from the GPU:

host_transposed = blocks.copy(transposed, space='cuda_host')

Then, convert to an 8-bit integer data type for storage, with the quantize block ('i8' means an 8-bit i nteger).

quantized = bf.blocks.quantize(host_transposed, 'i8')

Finally, we pass the data stream into a sink block, which ends the pipeline and stores the data in a filterbank file:

blocks.write_sigproc(quantized)

In this case, the filename will be determined from the header information, which contains the name of the original .wav file. The output file is of type .fil, and is the channelized version of the original music file. It is the frequency decomposition of the audio.

So, what have we done? We:

1. Read in the .wav file.

2. Copied the raw data to the GPU.

3. Split the time axis into chunks which we could FFT over.

4. FFT’d along this new axis.

5. Took the modulus squared of these FFTs.

6. Transposed this data into a format compatible with the sigproc writer.

7. Copied the data back to the CPU.

8. Converted the data into integer data types.

9. Wrote this data to a filterbank file.

3.3. All the Code¶

For ease of reference, here is all the code at once:

import bifrost as bf
import bifrost.blocks as blocks
import bifrost.views as views

raw_data = blocks.read_wav(['heyjude.wav'], gulp_nframe=4096)
gpu_raw_data = blocks.copy(raw_data, space='cuda')
chunked_data = views.split_axis(gpu_raw_data, 'time', 256, label='fine_time')
fft_output = blocks.fft(chunked_data, axes='fine_time', axis_labels='freq')
squared = blocks.detect(fft_output, mode='scalar')
transposed = blocks.transpose(squared, ['time', 'pol', 'freq'])
host_transposed = blocks.copy(transposed, space='cuda_host')
quantized = bf.blocks.quantize(host_transposed, 'i8')
blocks.write_sigproc(quantized)

pipeline = bf.get_default_pipeline()
pipeline.shutdown_on_signals()
pipeline.run()