#include <stdio.h>
#include <stdlib.h>

int test_noRef(int value)
{
	return value + 1;
}

int test_ref(int &value){
	return value += 1;
}

int main(int argc, char *argv[]){

	int t = 1;

	for(int i = 0; i < 100000000; i++){
		//t = test_noRef(t);
		test_ref(t);
	}

	// final result = n loops + 1
	printf("%d\n", t);

}

This code is purpose built to thwart basic compiler optimizations. In particular the printf() call at the end prevents the compilers from simply skipping the entire function call block.

While simplistic, the basic idea is to see what optimizations each compiler performs and at what optimization level. Note that the LLVM/Clang suite can easily be installed on Fedora 15 by searching for LLVM. You’ll want to install LLVM and Clang:

No Optimizations

We start with a basic compile with no optimizations; in each case we’ll focus on test_ref() and test_noRef(). Please note I’ve removed the stack management code from these listings as they differ very little by compiler.

g++ main.cpp -S
clang main.cpp -S

[GCC]

_Z10test_noRefi:
.LFB0:
	movl	%edi, -4(%rbp)
	movl	-4(%rbp), %eax
	addl	$1, %eax
	popq	%rbp
	.cfi_def_cfa 7, 8
	ret

The noRef call in GCC moves our parameter to eax, adds 1, and cleans up the stack.

[LLVM]

_Z10test_noRefi:
.Leh_func_begin0:
	pushq	%rbp
.Ltmp0:
	movq	%rsp, %rbp
.Ltmp1:
	movl	%edi, -4(%rbp)
	movl	-4(%rbp), %edi
	addl	$1, %edi
	movl	%edi, %eax
	popq	%rbp
	ret

The LLMV code for the noRef() function is quite similar. The main difference is LLVM uses edi for the addition step, then pushes the result to eax before returning.

[GCC]

_Z8test_refRi:
.LFB1:
	movq	%rdi, -8(%rbp)
	movq	-8(%rbp), %rax
	movl	(%rax), %eax
	leal	1(%rax), %edx
	movq	-8(%rbp), %rax
	movl	%edx, (%rax)
	movq	-8(%rbp), %rax
	movl	(%rax), %eax
	popq	%rbp
	.cfi_def_cfa 7, 8
	ret

The reference using version performs the same basic logic, but instead of using movl we issue movq, which means our parameters are twice as large. That inefficiency aside we do use an optimization for adding the 1 to our input argument via the leal instruction.

[LLVM]

_Z8test_refRi:
.Leh_func_begin1:
	pushq	%rbp
.Ltmp3:
	movq	%rsp, %rbp
.Ltmp4:
	movq	%rdi, -8(%rbp)
	movq	-8(%rbp), %rdi
	movl	(%rdi), %eax
	addl	$1, %eax
	movl	%eax, (%rdi)
	popq	%rbp
	ret

LLVM’s code is a bit cleaner, focusing on pulling the value from the stack, performing the addl, and retuning the result.

[GCC]

main:
.L5:
	leaq	-8(%rbp), %rax
	movq	%rax, %rdi
	call	_Z8test_refRi
	addl	$1, -4(%rbp)
.L4:
	cmpl	$99999999, -4(%rbp)
	setle	%al
	testb	%al, %al
	jne	.L5
	movl	-8(%rbp), %eax
	movl	%eax, %esi
	movl	$.LC0, %edi
	movl	$0, %eax
	call	printf
	movl	$0, %eax
	leave

As with the function calls I’ve removed the activation record code. LLVM’s main function definition is essentially the same, so we won’t cover it here.

A quick benchmark between the two shows what we would expect from the extra fluff in GCC’s implementation:

GCC
real 0m0.407s
user 0m0.405s
sys 0m0.002s

LLVM
real 0m0.396s
user 0m0.394s
sys 0m0.000s

Level 1 Optimizations

It’s a bit unfair to draw comparisons without some optimizations turned on, so let’s do so now:

g++ main.cpp -O1 -S
clang main.cpp -O1 -S

[GCC]

_Z10test_noRefi:
.LFB19:
	leal	1(%rdi), %eax
	ret

Already we can see a huge difference in GCC’s output, which whittles this call down to a simple Load Effective Address and return.

[LLVM]

_Z10test_noRefi:
	leal	1(%rdi), %eax
	popq	%rbp
	ret

As expected LLVM performs the same lea optimization as GCC.

[GCC]

_Z8test_refRi:
.LFB20:
	movl	(%rdi), %eax
	addl	$1, %eax
	movl	%eax, (%rdi)
	ret

The reference-using version still has to contend with the indirection call via movl ($rdi), %eax, but still a good improvement.

[LLVM]

.Ltmp4:
	movl	(%rdi), %eax
	incl	%eax
	movl	%eax, (%rdi)
	popq	%rbp
	ret

LLVM is using incl to perform our addition call, GCC uses addl. A quick check on Google shows Intel recommends the GCC way.

[GCC]

	movl	$1, 12(%rsp)
	movl	$100000000, %ebx
.L4:
	leaq	12(%rsp), %rdi
	call	_Z8test_refRi
	subl	$1, %ebx
	jne	.L4

The main() function has been reduced to the basics, counting down from our loop initialization value and calling the test_ref() function.

[LLVM]

.Ltmp8:
	movl	$1, -20(%rbp)
	movl	$100000000, %ebx
	leaq	-20(%rbp), %r14
	.align	16, 0x90
.LBB2_1:
	movq	%r14, %rdi
	callq	_Z8test_refRi
	decl	%ebx
	jne	.LBB2_1

LLVM’s main function loop has also been optimized, though again we’re using decl instead of subl.

Although the two compilers perform similar optimizations here, lets examine how LLVM’s structured our reference value for the loop.

Right before the main loop it calls:

leaq    -20(%rbp), %r14

Which maps the address at that base-pointer location to r14. In essence its created an alias. Why it’s done this make sense when we see the rest of the loop.

The first call of every iteration is:

movq	%r14, %rdi

So we pass the address in %r14 to rdi and in the function call we have:

movl    (%rdi), %eax

Which essentially means dereference the value in rdi and push to eax for the incl call. That gets us our added value but it’s the next call where the magic happens, where we tie the whole thing together:

movl    %eax, (%rdi)

Now we’ve moved the value of eax to the address location in rdi. Since that address and the one in %r14 are one in the same, the next call in the main loop to:

movq    %r14, %rdi

Means that his freshly increased value from the function call being pointed to by %r14 is once again pushed to rdi, which in turn is dereferenced in the function call and incremented.

That’s a pretty neat bit of optimization so let’s see how it pans out in terms of performance:

GCC
real 0m0.165s
user 0m0.163s
sys 0m0.001s

LLVM
real 0m0.163s
user 0m0.161s
sys 0m0.001s

It’s much the same, as GCC performs a similar trick using %rdi and the stack.

Level 2 Optimizations

Finally, let’s enable the highest level of optimization:

g++ main.cpp -O2 -S
clang main.cpp -O2 -S

[GCC]

main:
.LFB21:
	.cfi_startproc
	subq	$8, %rsp
	.cfi_def_cfa_offset 16
	movl	$100000001, %esi
	movl	$.LC0, %edi
	xorl	%eax, %eax
	call	printf
	xorl	%eax, %eax
	addq	$8, %rsp
	.cfi_def_cfa_offset 8
	ret
	.cfi_endproc

[LLVM]

.Ltmp7:
	movl	$.L.str, %edi
	movl	$100000001, %esi
	xorb	%al, %al
	callq	printf
	xorl	%eax, %eax
	popq	%rbp
	ret

No need to show anything other than main() because it doesn’t matter.

Both LLVM and GCC have done what we’d expect with such a silly bit of code. Both compilers analyzed the code and came to the same conclusion: the final output value to prinf() can be calculated directly within the compiler, so no need to call anything. Just push 100000001 to %esi for printf() and call it a day : )

In Conclusion

While the code tested was simple, it’s interesting to see how the actual assembly differs. It’s clear that this simple test LLVM produces faster binaries, though not by much.

The other important bit is at least on Fedora, installing and using GCC and LLVM couldn’t be simpler.

Links

Installing and Using LLVM

LLVM Online Compiler Demo

The first three are of the fannkuch benchmark found on the sunspider test site, only ported over to C++ and PHP in addition to the JS version. The browser used for all tests was Firefox 5.

The HPHP listing is a compiled Hip Hop PHP version, which is interesting as it shows the relative difference between it and vanilla PHP.

The last item, Benchmark.php, is the same benchmark file you’ll find in a PHP source download.

See the full benchmarks after the jump!

access-fannkuch – n=9

JavaScript: 0.204

PHP: 1.624

HPHP: 0.538

C++ .20

access-fannkuch – n=10

JavaScript: 3.062

PHP: 19.058

HPHP: 6.395

C++ .189

access-fannkuch – n=11

JavaScript: 40.297

PHP: ~4 minutes

HPHP: 23.713

C++ 2.265

Benchmark.php

PHP: 2.542

HPHP: 0.547

The Qt documentation provides a short write-up on how to re-enable this driver, but it leaves out some important details.

To enable support for the QODBC driver follow these steps:

1. From the Qt Creator Application Launch Start Updater:

Continued after the jump…

2. Expand Qt SDK > Miscellaneous > Qt Sources > Qt 4.7.3 Sources and click Next to install the Qt Sources:

3. When that finishes from the Start Menu Launch:

4. In the new command prompt window navigate (using cd) to:

C:\QtSDK\QtSources\4.7.3\src\plugins\sqldrivers\odbc

Of course your path may be different, and in my case, I have used the offline installer. That said, the key is the online installer should be very similar, and in either case we want to hit the QtSources directory, and then within that, the plugins, sqldrivers, odbc folder.

Please note their is a similar set of files in:

C:\QtSDK\QtSources\4.7.3\src\sql\drivers\odbc

We do not want that. I say this as in an unfamiliar source tree it’s easy to get turned around if we’re not careful.

We should now have the following in the command prompt window:

5. Issue the command:

qmake

Nothing will change in the command-line window, but if you look at the folder we’re in, several files have been created:

This happened because the folder, at the start, contained a .pro file, which when run against qmake has created platform specific build files we’ll use in the next step.

6. Issue the commands:

nmake release

Followed by:

namke install release

If we check our mingw/plugins/sqldrivers folder we now have the newly created driver:

Now when we launch Qt Creator and build a project, SQL Server/ODBC support should be ready for use.

Of course it should be added in order to load the driver for a project we must add:

sql-plugins += odbc

…to that projects .pro file.

Caveats

Of course this instruction set assumes we have nmake available to us which may, to be honest, only be the case if we also have Visual Studio or a Windows SDK such as Windows 7 SDK installed and with a correct system PATH defined to nmake.

I say this as in theory what should happen when we launch the Qt command line shortcut is we should be able to just run make. I cannot get his to work however, which leads me to believe I must be missing something.

Thus, a good first check when running through this tutorial will be to check to make sure you can run nmake and get output other than a message saying nmake is not a recognized command. For example, I get:

I get this as again, I have the Windows 7 SDK and Visual Studio 2010 Express installed on my system, and also have added the PATH to nmake in my System Variables:

No matter, another popular target for extension management will no doubt be MySQL, to which this tutorial may be useful.

As great as the newest browsers are I though it would be interesting to take a random test and port it to C++ for the sake of comparison.

I decided the rather short fannkuch test was a good candidate. The results:

Not surprisingly the C++ version is 18x faster than Firefox and 8x faster than Chrome. For the curious, the ported C++ code is after the jump.

I should mention one thing changed from the stock Sunspider site (e.g., the link above), is the number of iterations was upped from 8 to 10. At 8 iterations the C++ version was sub-millisecond, meaning I’d have had to roll a custom assembly timer to get a benchmark value. That said, when we lower the iterations down to 9, the stack heavy algorithm starts to benefit the browsers more, with Chrome coming in only 3 times slower and Firefox 11.

Lower the iteration count to the “stock” 8 and Firefox actually catches up to Chrome, with both browsers reporting ~21ms. Interesting.

The C++ Code

int MainWindow::Fannkuch(int n)
{

    int check = 0;
    int perm[n];
    int perm1[n];
    int count[n];
    int maxPerm[n];
    int maxFlipsCount = 0;
    int m = n - 1;

    for (int i = 0; i < n; i++){
        perm1[i] = i;
    }

    int r = n;

    while (true) {

        // write-out the first 30 permutations
        if (check < 30){
            int s = 0;
            for(int i=0; i<n; i++) s += (perm1[i]+1);
            check++;
        }

        while (r != 1) { count[r - 1] = r; r--; }

        if (!(perm1[0] == 0 || perm1[m] == m)) {
            for (int i = 0; i < n; i++) perm[i] = perm1[i];
            int flipsCount = 0;
            int k;

            while (!((k = perm[0]) == 0)) {
                int k2 = (k + 1) >> 1;
                for (int i = 0; i < k2; i++) {
                    int temp = perm[i]; perm[i] = perm[k - i]; perm[k - i] = temp;
                }
                flipsCount++;
            }

            if (flipsCount > maxFlipsCount) {
                maxFlipsCount = flipsCount;
                for (int i = 0; i < n; i++) maxPerm[i] = perm1[i];
            }
        }

        while (true) {
            if (r == n) return maxFlipsCount;

            int perm0 = perm1[0];
            int i = 0;

            while (i < r) {
                int j = i + 1;
                perm1[i] = perm1[j];
                i = j;
            }

            perm1[r] = perm0;
            count[r] = count[r] - 1;

            if (count[r] > 0) break;
            r++;
        }
    }

}

In this post we’ll merge these concepts by creating a version of the CMYK to RGB conversion code strictly in raw SSE and assembly. The upshot is you’ll see how we can take existing, real-world C++ code and use GCC’s Extended Assembly syntax to interweave raw assembly code for potential performance gains.

This means this tutorial is not just about extended assembly or sse coding, it’s about using both in a real-world application. We’ll learn many concepts including data retrieval, loop processing, SSE processor instructions, floating point number representation, and much more!

Let’s start our tutorial by taking a quick look at the core algorithm logic we’ll be implementing (for a more in-depth refresher this is covered in some detail here ):

c = 1.0 - (bits / 255f)
m = 1.0 - (bits / 255f)
y = 1.0 - (bits / 255f)

k = the min of c/m/y

if k != 1
c = (c - k) / (1 - k)
m = (m - k) / (1 - k)
y = (y - k) / (1 - k)

c = c * 255
m = m * 255
y = y * 255
k = k * 255

The data source (the incoming image data) is a call to Qt’s QImage::bits() function, which returns a pointer to a uchar array containing the raw image data-stream.

The destination, that is, the RGB converted data, is a heap-based uchar array created via:

uchar *cmyk_temp = new uchar[(wt * ht) * 4];

In other words we already have a source and a destination set in C++. (To put it simply, the point of this tutorial is to write assembly to mange the data in-between these two places.) This means the first big task in creating an assembly version is knowing how we ‘hook’ into those existing data structures.

A key point in this regard is that while we’ll manage loops and data access in our assembly code, we do not want to think about managing the stack. Thus, we can think of these two arrays as places we’ll get the address to via pointers, but never create and manage in assembly on our own.

So how do we access this data from GCC Extended Assembly?

The answer is we, in extended assembly syntax, use these pointer variables as input and output parameters. (please see here for a brief primer on GCC inline assembly)

For example, lets say we have the following C++:

int a = 1;

__asm__ __volatile__(

"mov %0, %%ebx \n\t"

: /* output parameters */

: "m"(a)

: ebx /* clobbers */

);

This code says that we’ll take the int a variable and push it to ebx. The key is behind the scenes CGG will actually rewrite this mov instruction to something like:

mov -0x20(%rbx), %ebx

In other words, we let GCC manage the stack. Using Extended Assembly like this means we don’t care where on the stack int a comes from, we just want to make sure we have access to it. The extended assembly syntax allows for this easy manipulation of data.

For our code, this means the start of our implementation will actually be a series of C++ variable declarations that we’ll end up passing into the __asm__ call.

One item of note here is along with simple arrays pointers and ints, we also declare two __m128 objects for easy double quad-word storage of constants we’ll need for our calculations, those being vectors of 1.0f and 255.0f.

“Talking” to these __m128 items is accomplished in much the same as our example above, only now we use the movaps mnemonic as in:

"movaps %5, %%xmm14 \n\t" // 255.0

In other words, bytes, chars, floats, __m128′s — we can create whatever we need and pass it to the assembly routine, which means we don’t need to worry about the stack.

Register Pressure

This takes us to one of the main goals of this exercise: use as many CPU registers as possible during the conversion process. This means one of the explicit assumptions of this code is it only runs on 64-bit machines. That is, it’s hard-coded to use the full register set available to x86_64.

This means we have an extended range of 64-bit (quad-word) general purpose registers (r8-r15), as well as the full set of 128-bit (double quad-word) SSE registers, xmm0-xmm15.

Obviously the assumption here is the fewer trips to main memory we can make, the faster our code will be.

And so, the first ‘preparatory’ part of our assembly code is spent mapping various constants to known registers so we can refer back to them as often as needed without making expensive trips to main memory.

The interesting bit here is not all of our mov’s are for the same purpose. Some moves, as in above, are to store 128 bit values to an SSE register. Others are for setting up bit masks, and still others are set up for loops counters.

Of course key is that this logic happens outside of our main conversion loop. Once we enter the loop we do as much as we can to avoid reads and writes to main memory.

With that said, let’s jump strait into the code and see how it works!

Part 1 – Initialization.

In this section we perform the essential task of initializing the variables we’ll pass to the extended asm routine.

Note the *bits array, this is the source of the values being used. This is an example of a standard link to a variable, though we also create a few float arrays and zero out their initial values for easier debugging.

The most important bit in this block is the _pack_lookup_table float array. The creation of this item (a classic lookup table) is to reduce the possible overhead of the uchar to float conversions we must make.

The idea is simple: because we only have 256 possible input values, instead of converting each in turn at run-time, lets just create a lookup table and map the input values to the IEE floating point representation created in the loop (whose values are calculated at compile, not run-time). This has a huge potential benefit as each pixel will require 3 conversions, and when you’re dealing with 8 million pixels per image these int to float conversions can really add up.

To be clear though, converting ints to floats on modern hardware isn’t a huge deal (around 8-16 cycles on most CPU’S), but this is still a handy way to learn about assembly coding. We will see in Part 4 however, that our lookup table may not be the best possible solution because of the extra memory trips we end up making.

Alas, this is also a good excuse to learn a bit more about assembly level addressing modes, which we’ll see a good deal of in many different places.

As such, the code below is just standard C++.

int t = 1;
int l = (char)t;

uchar *bits = imin.bits();

int count = (wt * ht) * 4; // link with loop counter byte size

int k_min_value;

float * _pack_loop_array = new float[4];
// zero out array for easier debugging
_pack_loop_array[0] = 0;
_pack_loop_array[1] = 0;
_pack_loop_array[2] = 0;
_pack_loop_array[3] = 0;

float * _mask_array = new float[4];
// zero out array for easier debugging
_mask_array[0] = 0;
_mask_array[1] = 0;
_mask_array[2] = 0;
_mask_array[3] = 0;

float * _pack_lookup_table = new float[256];
for(int i = 0; i <= 256; i++){
    _pack_lookup_table[i] = (float)i;
}

// mmx constants - use m128 to ensure aligned data
__m128 m_1 = _mm_set_ps1(1.0f);
__m128 m_255 = _mm_set_ps1(255.0f);

Part 2 – Initial __asm__ and Constant Creation

In this section we dive into the heart of our routines assembly code.

A good deal of space is for initialising loop constants and mapping values to registers–pretty standard stuff.

We do have one interesting bit though, which is the creation of the 128-bit sign-flip mask.

We’ve done this because in IEE single-precision floating point representation, the HO bit (bit 31) is always the sign bit. When the bit’s 0 the value is positive, when it’s a 1 negative.

We can exploit this fact to easily flip the sign of a float, or in our case, the packed floats. Problem is, in order to create a 32-bit string in the form of the proper sign-flip mask (0×80000000) we would have to resort to all sorts of trickery in C++. That is, how do you create the bit pattern of our mask without the compiler trying to turn it into something else, such as a float, int, or char? It sounds like an easy problem to address, but it isn’t.

Thus, as the value needs to be created anyway, we’ll just do so in assembly, where defining and populating memory locations with arbitrary bit-strings is easy.

In the end we push this mask value to xmm4 where it remains constant throughout the life of the conversion process.

__asm__ __volatile__(

    // set sse constants

    "movaps %5, %%xmm14 \n\t" // 255.0
    "movaps %4, %%xmm15 \n\t" // 1.0

    // init sign SSE sign-flip mask
    "xorps %%xmm4, %%xmm4 \n\t"

    "xor %%rax, %%rax \n\t"
    "xor %%r11, %%r11 \n\t"
    "mov $0x80000000, %%r11 \n\t"
    "mov %9, %%rbx \n\t"

    // populate mask values

    "mov %%r11,(%%rbx, %%rax, 1) \n\t"
    "add $0x4, %%rax \n\t"
    "mov %%r11,(%%rbx, %%rax, 1) \n\t"
    "add $0x4, %%rax \n\t"
    "mov %%r11,(%%rbx, %%rax, 1) \n\t"
    "add $0x4, %%rax \n\t"
    "mov %%r11,(%%rbx, %%rax, 1) \n\t"

    // populate sse reg
    "movaps (%%rbx), %%xmm4 \n\t"

    // init loop constants

    "xor %%rax, %%rax \n\t" /* init i array counter */

    "xor %%rdx, %%rdx \n\t" /* set array counter upper bounds */
    "mov %2, %%edx \n\t"

    "xor %%rcx, %%rcx \n\t" /* get base address of source array */
    "mov %1, %%rcx \n\t"

    "xor %%rbx, %%rbx \n\t" /* get base address of dest array */
    "mov %3, %%rbx \n\t"

    "xor %%r14, %%r14 \n\t" /* get base address of _pack_loopup array */
    "mov %7, %%r14 \n\t"

    "xor %%r9, %%r9 \n\t" /* L_PACK_LOOP Bounds Check ($0x10) */
    "mov $0x10, %%r9 \n\t" // init with decimal 16 -  4 bytes x 4 loops

    "xor %%r10, %%r10 \n\t" /* get base address of _pack_loop_array */
    "mov %6, %%r10 \n\t"

    // init logic constants

    "xor %%r13, %%r13 \n\t"
    "mov $0x1, %%r13 \n\t" // 1 value for k_min compare

Part 3 – Initial Loop Logic

The following block of code mainly just resets r8 and r15 for loop counting and addressing purposes.

Again, the point of this exercise is to use as many registers as possible. Part of this means we also need to carefully manage how and where registers are used.

Moving (copying of course) rax to r15 for example, means r15 points to important locations we need to index from, but can be incremented without fear of “breaking” the main index register, which for our code is rax. We’ll see this type of activity in a few other places as well.

The point being of course that we’re not hitting main memory, just registers.

".L_MAIN_LOOP:" // convert each value to float, push to xmm0

    // subroutine - created packed data from single loop

    "xor %%r8, %%r8 \n\t" // inner loop counter
    "mov $0x0, %%r8 \n\t"

    "xor %%r11, %%r11 \n\t"
    "xor %%r12, %%r12 \n\t"

    "xor %%r15, %%r15 \n\t"
    "mov %%rax, %%r15 \n\t" // PACK_INDEX

Part 4 – Converting Ints To Floats – Two methods

With the main loop initialisation done we’re now free to start processing data. We start by grabbing the current CMYK value from the source array and placing it into r11, then masking it to leave only the LO byte.

This sets us up for the next step, which is converting the int values into proper floats for the xmm registers. Let’s look at two ways of accomplishing this step:

Method 1 – Memory Dependent And Slightly Slower – A Lookup Table In Action
Per above, the first step in both methods is to move and mask the raw int value to r11. Crucially this number will always be between 0-255. We exploit this fact in the mov call where r14 is the base address of the _pack_lookup_table array we created in C++, and r11, (also always between 0 and 255), acts as the index.

This is a common optimization known as a lookup table. In essence, instead of converting ints to floats (at a cost of around 8 cycles per conversion on my machine), we instead use simple memory moves to place a pre-computer floating point bit-string value created via the _pack_lookup_table array.

This is an intellectually pleasing way to handle the conversion step but it comes with an unfortunate side-effect. The problem is x86_64 provides no instruction for moving r32 values directly into xmm registers. We can however, move memory values to xmm, only this presents another problem: In order for our lookup table to work we access main memory twice: once for the lookup table value, then once again to store the floating point bit string in an aligned __m128 memory location (which is later used as the argument to movaps). Such repeated memory access can be devastatingly slow, and when compared to just doing a relatively speedy direct conversion, becomes tough to justify.

All told, in my tests the lookup table code would be a touch faster than GCC’s output if it were not for these mov instructions. Without those institutions though, we’d have no lookup table!

".L_PACK_LOOP:"

    "nop \n\t"

    "mov (%%rcx, %%r15, 1), %%r11 \n\t" // get source array value

    "and $0x00000000000000ff, %%r11 \n\t" // mask unwanted bits

    // use lookup table to push pre-converted float val
    "mov (%%r14, %%r11, 4), %%r12 \n\t" // indexed value

    // this is slow!
    "mov %%r12, (%%r10, %%r8, 1) \n\t" // push to _pack_loop_array

    "add $0x1, %%r15 \n\t" // 1 pixel per loop iteration

    "add $0x4, %%r8 \n\t"

    "cmp %%r8, %%r9 \n\t"

    "jne .L_PACK_LOOP \n\t"

"xor %%r8, %%r8 \n\t" // clear r8 index for push to xmm

"movaps (%%r10, %%r8, 1), %%xmm0 \n\t" /* move packed values into mmx */

You’ll notice the last instruction sets our 128-bit xmm0 register with the value of the pack_loop_array.

All told this works, and again, is somewhat intellectually pleasing. The question is can this be made faster? The answer is yes it can, and the secret is to avoid memory access at all costs. Unfortunately, this means we must rid ourselves of the lookup table, as described in Method 2…

Method 2 – Using Hardware Conversion & Avoiding Memory Access
The second attempt at this logic turns out to be better in terms of performance, even though we ditch the lookup table.

The basic idea is instead of worrying about the conversion costs, we embrace them and use an unrolled conversion block where each value is simply run through CVTSI2SS, then shuffled to make room for the next value. It should be said that the unroll logic provided no discernible speed-up–the improvements come from fewer costly memory access steps.

// c

"add $0x2, %%r15 \n\t" // index for c value

"mov (%%rcx, %%r15, 1), %%r11 \n\t" // get source array value

"and $0x00000000000000ff, %%r11 \n\t" // mask unwanted bits

"CVTSI2SS %%r11, %%xmm0 \n\t"

"shufps $0xC4, %%xmm0, %%xmm0 \n\t" // flip #2 and #4 11000110

// m

"sub $0x1, %%r15 \n\t" // index for m value

"mov (%%rcx, %%r15, 1), %%r11 \n\t" // get source array value

"and $0x00000000000000ff, %%r11 \n\t" // mask unwanted bits

"CVTSI2SS %%r11, %%xmm0 \n\t"

"shufps $0xE1, %%xmm0, %%xmm0 \n\t" // flip last two to get m 11100001

// y

"sub $0x1, %%r15 \n\t" // index for y value

"mov (%%rcx, %%r15, 1), %%r11 \n\t" // get source array value

"and $0x00000000000000ff, %%r11 \n\t" // mask unwanted bits

"CVTSI2SS %%r11, %%xmm0 \n\t"

The big difference here is memory access is kept to a bare minimum. With this simple change my implementation runs around the same speed as GCC’s -02.

Part 5 – CMY Conversion Step 1

Now that we have a series of 4 floating point numbers in xmm0 (our packed data) we can begin the conversion process.

The code starts with what has to be one of the more satisfying parts of SSE coding. As we now have a 128-bit packed string we can divide and subtract our constant values from the working value in two easy calls to divps and subps. It’s getting a lot of work done in two quick instructions, which is always a good feeling!

Things gets a bit more interesting after that though. At this point we most likely have a series of negative numbers, something that doesn’t play well with the next step of the algorithm.

This is because we need to find the lowest value of bunch to create our k value, but when we have the possibility of negative numbers this task becomes impossible.

Thus, before we can advance we need to flip the 31st bit of each packed float to 0 so the value comparison is valid and we find the ‘true’ lowest value. Think of this as the assembly equivalent of an abs() call.

This is where our mask comes in. The idea of masking is simple enough, though unfortunately it becomes more of a chore when dealing with packed values.

The basic problem is their is no ‘default’ mask we can create that will safely mask only the 31st bit of the 4 packed values without also knowing which values in the packed string are already positive. This is because XOR will flip negative to positive, but also positive to negative. We don’t want that of course, we only want the former.

Thus, we first need to modify our mask (known as a selection mask), to ignore positive values so they aren’t accidentally converted to negative values.

To do so means we must copy our mask and working xmm0 value to new registers. We then call psrad against the xmm0 copy to create a selection mask where values that have the 31st bit set (meaning it’s negative), become all 1′s ($0xffffffff). When then logically AND this value with the default selection mask copy to turn the double-words with positive values back to 0.

This new, modified mask can now be safely xor‘d with the original value to only flip negatives to positives.

// perform cmy conversion

// divide by 255
"divps %%xmm14, %%xmm0 \n\t"

// subtract 1
"subps %%xmm15, %%xmm0 \n\t"

// mask back to positive values for min processing

// copy mask for modifier change into xmm5 (xmm4 is mask constant)
"movaps %%xmm4, %%xmm5 \n\t"

// copy raw value for min to xmm6
"movaps %%xmm0, %%xmm6 \n\t"

// create selection mask modifier
"psrad $0x31, %%xmm6 \n\t" // for every negative value, makes item all $0xffffffff

// modify mask copy (xmm5) to mask only negative values
"andps %%xmm6, %%xmm5 \n\t"

// now mask sign bits to 0 with modified mask
"xorps %%xmm5, %%xmm0 \n\t"

Part 6 – CMY Conversion – Find Min

Finding the minimum value in assembly is very similar in practise to the last section where we manipulate masks, only now we’re also shuffling values around too.

Granted we also had to shuffle values around in the SSE intrinsics version as well, but here we need to think a bit harder about our bit-masks, as we no longer have the benefit of using Intel’s handy _MM_SHUFFLE helper. This type of coding is, I can safely say, a bit more complex.

After we find the minimum value we call cmpeqps and then store the LO result in k_min_value, which as we can see from the first code block is a simple int we pass into the asm block as a memory value. This may seem like a wasted step, but if we look back a bit we can see that movss can only write to a memory location.

We then use this value in a cmp block and if ‘true’, perform a bit of extra processing on our image data.

// find min value
"movaps %%xmm0, %%xmm1 \n\t"
"shufps $0x4E, %%xmm0, %%xmm1 \n\t" // reorder values (mask first)

"minps %%xmm0, %%xmm1 \n\t" // find first min, put in xmm1

"movaps %%xmm1, %%xmm2 \n\t"
"shufps $0xB1, %%xmm1, %%xmm2 \n\t"

"minps %%xmm1, %%xmm2 \n\t" // min (k) in xmm2

// process min logic (is min value == 1? - true = cmpeqps creates mask of all 1's)
"movaps %%xmm15, %%xmm12 \n\t" // move 1's to xmm12
"cmpeqps %%xmm2, %%xmm12 \n\t" // mask now in xmm12

"movss %%xmm12, %8 \n\t" // move mask to memory value for cmp

"cmp %8, %%r13 \n\t" // is k == 1?

"jne .L_MULTIPLY_ALL \n\t" // anything but 1, skip this block

// save 1 - k value
"movaps %%xmm15, %%xmm13 \n\t" // xmm13 is temp storage for 1.0f
"subps %%xmm2, %%xmm13 \n\t" // 1-k in xmm13

// subtract k from all
"subps %%xmm2, %%xmm0 \n\t"

// c-k / 1-k
"divps %%xmm13, %%xmm0 \n\t"

".L_MULTIPLY_ALL:"

"mulps %%xmm14, %%xmm0 \n\t" // cmy * 255
"mulps %%xmm14, %%xmm2 \n\t" // k * 255

Part 7 – Export Values

With the conversion process done we can now write our values back to the destination buffer.

At this point the question becomes: how do we move these packed floats back into an unsigned char array?

This is a surprisingly serious question, as their is (strangely) no native machine instruction for doing so.

Thus, we need to be a bit creative. To that end, the general solution is we extract the LO double-word from the source vector converting it to a truncated int in the process. We then grab the LO byte of that value and store it in an indexed position in the destination buffer. Next we shuffle the vector to place a new value in the LO double-word and repeat the process. In the end we store each value back in turn as opposed to sending all back to main memory in one shot (which would be far better in terms of performance).

As far as the exact implementation in our code: to start, the first instruction is for performance reasons, in that when we call CVTTSS2SI to covert our float values to truncated ints, they come back as double-words, but we only want bytes. As there are no such instructions for this type of conversion, it becomes apparent their must be some way to get byte values back. Sure enough, CVTTSS2SI stores the LO byte of the conversion result into the destination register’s 8 bit ‘shadow’ register (which is to say, just the LO single byte of the full ecx register–this is not a ‘different’ register by any means).

Thus, in our code we push our index register rcx to r11 so it can be restored, because if we use rcx as the destination register for CVTTSS2SI, the LO byte value of the result is saved to the ‘shadow register’ cl, which is exactly the value we need (an unsigned int between 0 and 255). We must do this in rcx as r11 has no such 8 bit equivalent.

The next instruction is to push the main loop counter, rax, to r15. Again, it’s all about saving registers, which means because rax contains the same offset address as our other arrays, we can use that along with rbx to create our indexed address for the output array population step.

Of course this means we need to manipulate this register with the proper byte offset our values need to be pushed to, which is the reason for the add and sub calls.

We do this as the LittleCMS engine process our output buffer is created filled for expects CMYK byte order, but our xmm0 values are currently in YMCK. Instead of issuing shuffles we simply increment and decrement our r15 register with the appropriate byte offsets depending on which color we’re writing to the destination array. This is how we rearrange YMCK into CMYK, which is simple and effective.

// conversion done, export values

"mov %%rcx, %%r11 \n\t" // save rcx so we can convert dwords to words using cl

"mov %%rax, %%r15 \n\t" // pointer for insert values (r15)

// y - truncate and export values to array
"CVTTSS2SI %%xmm0, %%rcx \n\t" // y

"add $0x2, %%r15 \n\t" // add 2 for y value insert
"mov %%cl, (%%rbx, %%r15, 1) \n\t"

// m
"shufps $0xE1, %%xmm0, %%xmm0 \n\t" // flip last two to get m
"CVTTSS2SI %%xmm0, %%rcx \n\t" // m

"sub $0x1, %%r15 \n\t" // subtract 1 for m value insert
"mov %%cl, (%%rbx, %%r15, 1) \n\t"

// c
"shufps $0x4E, %%xmm0, %%xmm0 \n\t" // flip #2 and #4
"CVTTSS2SI %%xmm0, %%rcx \n\t" // c

"mov %%cl, (%%rbx, %%rax, 1) \n\t" // rax is base address, no need to sub

// k
"CVTTSS2SI %%xmm2, %%rcx \n\t" // m

"add $0x2, %%r15 \n\t" // add 3 for k value insert
"mov %%cl, (%%rbx, %%r15, 1) \n\t"

"mov %%r11, %%rcx \n\t" // restore rcx

Part 8 – Closing The Loop and __asm__ Logic

The closing assembly bits are simply to increment our loop counter.

The last part is of course the extended assembly closing bit where we set input, output, and clobbers.

    "addq $0x4, %%rax \n\t" // increment main loop counter
    "cmpq %%rdx, %%rax \n\t"

    "jne .L_MAIN_LOOP \n\t"

    : "=m" (cmyk_temp)  /* destination */
    : "m" (bits), "m" (count), "m" (cmyk_temp), "m" (m_1), "m" (m_255), "m" (_pack_loop_array), "m" (_pack_lookup_table), "m" (k_min_value), "m"(_mask_array) /* source */
    : "ebx", "ecx", "edx", "memory" /* clobbers */

); // asm

delete _pack_lookup_table;
delete _pack_loop_array;
delete _mask_array;

Performance Notes

One of the main reasons I was interested in pursuing this task was to see if I could write code that had some parity with GCC’s -02 output. In all I’m relatively happy, with performance generally being only a few hundred milliseconds slower over the course of 120 images:

== || Total Process Time Elapsed (milliseconds): 21228 SSE 2 (unrolled intrinsics)
== || Total Process Time Elapsed (milliseconds): 23252 Raw Assembly
== || Total Process Time Elapsed (milliseconds): 21344 Raw Assembly
== || Total Process Time Elapsed (milliseconds): 21125 SSE 1 (intrinsics)
== || Total Process Time Elapsed (milliseconds): 20747 SSE 2 (unrolled intrinsics)

As fun as creating this code was, the lesson here is if you can use intrinsics do it. There is simply no reason to slave over this type of coding if you can possibly avoid it, as in the end you’ll have a very hard time besting your compiler anyway. Yes you may get lucky every now and then, but my guess would be most of the time, not so much. Intrinsics rock, use em!

The one thing I would say though is there are some holes in the current intrinsic line-up which make raw assembly attractive. One glaring example is the inability to deal with packed values in a horizontal fashion. That is, it should be very simple to find the min of four packed floats using one intrinsic–but no such intrinsic exists. There are plans for one, but nothing as of yet.

It would also be nice to have a native machine instruction to perform packed sign flips. If we did the above code could be a touch shorter.

Finally, it has to be said this algorithm is really not the best for SSE optimizations. This is because at it’s heart the vectors we create are non-uniform which means we end up treating each vector as 4 separate values instead of a single block. Actually, this severely limits us in terms of performance gains, which is why we’ll see non-SSE versions of GCC’s -O2 come very close if not best the SSE versions.

The good news is the techniques covered in this tutorial are still valid. Just keep in mind not all data structures are well suited for SSE.

General hints

Along the way I tried to keep note of the various idiosyncrasies of this type of coding.

Of course the best piece of advice is to just see what your own assembler is doing in a debug session.

The problem with this approach however, is that unlike Intel syntax, GCC’s Gas syntax adds many rules and formatting requirements that are not present in raw assembly output. For example, you’ll notice that Gas requires that we end each line with \n\t.

With that said, here are a few hints:

CGG Reversed Syntax
Ha ha. But seriously, GCC (AT&T syntax) reverses the order of operands in specific ways, meaning many pieces of documentation you’ll read won’t work unless we first reverse them. It’s important to note it’s not just the operands in two operand instructions, but also 3 operand ones too.

A good example is this guy from my code:

shufps $0xe4,%xmm1,%xmm0

In Intel documentation it looks like:

SHUFPS xmm1, xmm2/m128, imm8

Thus, not only are operands reversed (source/destination), so too is the immediate value.

Using 64-bit? Use 64-bit Register Names
This one sounds obvious, but it can be easy to forget that this won’t work:

"mov %%r15, %%eax \n\t"

Whereas this will:

"mov %%r15, %%rax \n\t"

Conclusion

And that’s it, hopefully you’ve learned a bit more about assembly coding : )

If you have any questions, please post ‘em below!

Full Code Listing

__asm__ __volatile__(
    "movaps %5, %%xmm14 \n\t"
    "movaps %4, %%xmm15 \n\t"
    "xorps %%xmm4, %%xmm4 \n\t"
    "xor %%rax, %%rax \n\t"
    "xor %%r11, %%r11 \n\t"
    "mov $0x80000000, %%r11 \n\t"
    "mov %9, %%rbx \n\t"
    "mov %%r11,(%%rbx, %%rax, 1) \n\t"
    "add $0x4, %%rax \n\t"
    "mov %%r11,(%%rbx, %%rax, 1) \n\t"
    "add $0x4, %%rax \n\t"
    "mov %%r11,(%%rbx, %%rax, 1) \n\t"
    "add $0x4, %%rax \n\t"
    "mov %%r11,(%%rbx, %%rax, 1) \n\t"
    "movaps (%%rbx), %%xmm4 \n\t"
    "xor %%rax, %%rax \n\t"
    "xor %%rdx, %%rdx \n\t"
    "mov %2, %%edx \n\t"
    "xor %%rcx, %%rcx \n\t"
    "mov %1, %%rcx \n\t"
    "xor %%rbx, %%rbx \n\t"
    "mov %3, %%rbx \n\t"
    "xor %%r14, %%r14 \n\t"
    "mov %7, %%r14 \n\t"
    "xor %%r9, %%r9 \n\t"
    "mov $0x10, %%r9 \n\t"
    "xor %%r10, %%r10 \n\t"
    "mov %6, %%r10 \n\t"
    "xor %%r13, %%r13 \n\t"
    "mov $0x1, %%r13 \n\t"
    ".L_MAIN_LOOP:"
    "xor %%r8, %%r8 \n\t"
    "mov $0x0, %%r8 \n\t"
    "xor %%r11, %%r11 \n\t"
    "xor %%r12, %%r12 \n\t"
    "xor %%r15, %%r15 \n\t"
    "mov %%rax, %%r15 \n\t"
    ".L_PACK_LOOP:"
    "mov (%%rcx, %%r15, 1), %%r11 \n\t"
    "and $0x00000000000000ff, %%r11 \n\t"
    "mov (%%r14, %%r11, 4), %%r12 \n\t"
    "mov %%r12, (%%r10, %%r8, 1) \n\t"
    "add $0x1, %%r15 \n\t"
    "add $0x4, %%r8 \n\t"
    "cmp %%r8, %%r9 \n\t"
    "jne .L_PACK_LOOP \n\t"
    "xor %%r8, %%r8 \n\t"
    "movaps (%%r10, %%r8, 1), %%xmm0 \n\t"
    "divps %%xmm14, %%xmm0 \n\t"
    "subps %%xmm15, %%xmm0 \n\t"
    "movaps %%xmm4, %%xmm5 \n\t"
    "movaps %%xmm0, %%xmm6 \n\t"
    "psrad $0x31, %%xmm6 \n\t"
    "andps %%xmm6, %%xmm5 \n\t"
    "xorps %%xmm5, %%xmm0 \n\t"
    "movaps %%xmm0, %%xmm1 \n\t"
    "shufps $0x4E, %%xmm0, %%xmm1 \n\t"
    "minps %%xmm0, %%xmm1 \n\t"
    "movups %%xmm1, %%xmm2 \n\t"
    "shufps $0xB1, %%xmm1, %%xmm2 \n\t"
    "minps %%xmm1, %%xmm2 \n\t"
    "movaps %%xmm15, %%xmm12 \n\t"
    "cmpeqps %%xmm2, %%xmm12 \n\t"
    "movss %%xmm12, %8 \n\t"
    "cmp %8, %%r13 \n\t"
    "jne .L_MULTIPLY_ALL \n\t"
    "movaps %%xmm15, %%xmm13 \n\t"
    "subps %%xmm2, %%xmm13 \n\t"
    "subps %%xmm2, %%xmm0 \n\t"
    "divps %%xmm13, %%xmm0 \n\t"
    ".L_MULTIPLY_ALL:"
    "mulps %%xmm14, %%xmm0 \n\t"
    "mulps %%xmm14, %%xmm2 \n\t"
    "mov %%rcx, %%r11 \n\t"
    "mov %%rax, %%r15 \n\t"
    "CVTTSS2SI %%xmm0, %%rcx \n\t"
    "add $0x2, %%r15 \n\t"
    "mov %%cl, (%%rbx, %%r15, 1) \n\t"
    "shufps $0xE1, %%xmm0, %%xmm0 \n\t"
    "CVTTSS2SI %%xmm0, %%rcx \n\t"
    "sub $0x1, %%r15 \n\t"
    "mov %%cl, (%%rbx, %%r15, 1) \n\t"
    "shufps $0x4E, %%xmm0, %%xmm0 \n\t"
    "CVTTSS2SI %%xmm0, %%rcx \n\t"
    "mov %%cl, (%%rbx, %%rax, 1) \n\t"
    "CVTTSS2SI %%xmm2, %%rcx \n\t"
    "add $0x2, %%r15 \n\t"
    "mov %%cl, (%%rbx, %%r15, 1) \n\t"
    "mov %%r11, %%rcx \n\t"
    "addq $0x4, %%rax \n\t" // increment main loop counter
    "cmpq %%rdx, %%rax \n\t"
    "jne .L_MAIN_LOOP \n\t"

    : "=m" (cmyk_temp)  /* destination */
    : "m" (bits), "m" (count), "m" (cmyk_temp), "m" (m_1), "m" (m_255), "m" (_pack_loop_array), "m" (_pack_lookup_table), "m" (k_min_value), "m"(_mask_array) /* source */
    : "rax", "rbx", "rcx", "rdx", "r9", "r10", "r11", "r12", "r13", "r14", "r15", "memory" /* clobbers */

); // asm

PHP is my favorite language, bare none. It’s simple, elegant, and fun to use. Problem is, for highly trafficked sites it’s a touch slow and can be quite memory hungry. If you’re Facebook this can lead to problems, which is why they invented Hip Hop PHP (HPHP), a collection of tools and technology that turns our slow and hungry PHP code into lean and mean C++.

Ok, So Just How Fast Is It?

As a quick comparison I created a simple FormBoss form and ran Apache Bench (ab from the command line), to get a sense of the speed difference between Apache 2.2 and HPHP.

The top two tests are when running our simple .php files, the bottom test is when serving a simple 62 byte .xml file with 100 concurrent users:

**It’s important to note these numbers will be lower when running though a network and calling a database. Also, while other servers like Cherokee can be twice as fast as Apache, HPHP is still nearly twice as fast again.

So yes, qualifications aside, HPHP is very fast indeed.

Sure these numbers are fantastic, but using HPHP means compiling the source from scratch and then using a series of command-line switches to run and manage the compiled PHP code.

No longer–In my spare time I’ve created and now released an open-source front-end UI to HPHP.

Read on to learn more, or just download the files!

A quick bit of history and why we need Aesop

Back in early 2010 Facebook announced a wonderful new technology called Hip Hop PHP (HPHP). The idea was simple: PHP is a fantastic scripting language, but its loosely typed roots and interpreted nature make it a good deal slower and memory hungry than other web technologies. As Facebook was already deeply invested in the PHP they didn’t want to give it up with out a fight. Thus, why not write a series of tools and applications that turn PHP into native C++?

That’s what they did of course, and apparently for an entity like FaceBook the savings are well worth it. That doesn’t mean that we can’t enjoy it as well, because along with the announcement that a large percentage of FaceBook’s traffic was already being server via HPHP, they were also releasing the whole mess as open-source software.

Bravo of course, what a fantastically nice thing to do. The only issue was that in order to get Hip Hop to run we need to compile it from source, then issue a series of command-line instructions to compile and run our applications. This isn’t the end of the world of course, but often times when trying to compile existing PHP source we’ll run into a bevy of errors and issues — not HPHP’s fault, but the existing PHP sources’. Thus, it’s a load of trial and error at first, which considerably diminishes the fun this type of work should be.

Nonsense I said, we can fix that. Lets create an application that removes the burden of working with the command line so we can instead focus on our code.

The Aesop UI

I was rightly excited when HPHP project was first released, and so even though it took me longer than I had wanted to start this project, I finally present a UI to HPHP.

The idea of course is to make the process of managing and using HPHP simple and possibly even fun, but above all useful.

To that end Aesop consists of three main functions and goals:

Pre-compile and package HPHP - The idea here is simple. To get us up and running as quickly as possible I’ve packaged the HPHP binaries in the full download version of Aesop. This means we no longer have to worry about compiling HPHP, which saves time and possible headache. We needn’t use this version though, we can still download a minimal, Aesop only version complete with full source code.

Compile – Compiling HPHP applications from the command line means well, using a command line, setting environmental variables, and so on. Thus, we remove this burden and instead focus on making sure our code-base is acceptable to HPHP.

Aesop builds the file lists HPHP requires for code-compilation, sets environment variables, displays errors, and also manages the results of HPHP for the other main function of Aesop…

Manage Servers – Compiling code is only half the story. Once compiled HPHP has literally created a server-in-a-box for us. It’s a single executable that contains your source converted to C++, a web server, and the full PHP run-time. Aesop manages these executable for us in a nice list format, allowing us to start, stop and delete them at will. Of course setting their properties using GUI controls or a more advanced interface to a so-called HDF file is included as well.

Who is this good for?

I would imagine the first group will be the tinker-types like me who are curious to see what all the fuss is about. My true hope however, is that after a while we can turn this into a true production-level appliance. This means anyone who needs the highest performance PHP stack possible.

To that end there are a few issues still on the table as of the first release:

• Even easier installation (e.g. packages for DEB and RPM)
• Static binding of the Qt libraries version.
• More robust process management for the compiler fork.
• A full command line interface for running headless.
• Support for HPHi
• Better support (more GUI options) for ./hphp
• Better support for Fedora

So yes I, (we) have work to do, but I think even as is the application provides an improved experience over the command line.

So what are the requirements, how do we get it, run it, and provide feedback?

Requirements

a) 64-bit Linux. HPHP requires a 64-bit OS, which in turn means Aesop was built for 64-bit as well. Also, HPHP requires several dependencies which are shown below in apt-get format. Fedora/Red Hat users will need to change the command to match your distro’s naming conventions and package names (e.g. yum install [...]).

b) The Qt 4 Libraries. Instructions for getting them are below.

Getting It

Download the files from the FormBoss Open Source page right here.

Please note we have two versions: The small one is just the Aesop source code, the large one is the Aesop source and a pre-compiled version of HPHP. Most users will want the full (large) version, as everything is packaged up and ready to be run. If however, you already have a compiled version of HPHP you may just want the smaller, source only download.

Running It

In the main download directory file we’ll find ‘Docs/Instructions.txt‘ file with full details. The short version however:

a) For Ubuntu we’ll run the following command to install HPHP’s dependencies:

sudo apt-get install git-core cmake g++ libboost-dev libmysqlclient-dev libxml2-dev libmcrypt-dev libicu-dev openssl binutils-dev libcap-dev libgd2-xpm-dev zlib1g-dev libtbb-dev libonig-dev libpcre3-dev autoconf libtool libcurl4-openssl-dev libboost-system-dev libboost-program-options-dev libboost-filesystem-dev wget memcached libreadline-dev libncurses-dev libmemcached-dev libicu-dev libbz2-dev libc-client2007e-dev php5-mcrypt php5-imagick

Fedora users need to install the same packages, but will need to use yum install and modify the names of some packages to match your distributions naming conventions. This link has some solid tips on doing so.

b) Open the Ubuntu Software Center and search for and install Qt Creator. This installs the Qt Libraries we need to run Aesop. Word is the Qt libs will be installed by default in Ubuntu 11.04. In any case, this also installs the most excellent Qt Creator and SDK, which means you may be that much more likely to contribute to this project in some way…

Fedora users will need to download Qt directly from Nokia’s site.

c) Extract the program files:

> For the full download (Aesop.tar.gz), simply extract the whole archive to your /Documents directory.
> Extract and place One-level up from where we’ve installed HPHP if you’ve downloaded the Aesop only archive (Aesop+Source.tar.gz).

d) Make sure Run-Aesop.sh is executable. Right click and under Permissions > Allow executing of file as program.

e) Double-click Run-Aesop.sh. The application should start, provided we’ve installed the Qt 4 Libraries.

f) We can also run Aesop in root mode by double clicking Run-Aesop-As-Root.sh and then selecting ‘Run in Terminal‘. This is for sites that require port 80 to run.

g) We can test out a sample strait away under the second tab: Run Compiled Code > Sample

Providing Feedback

This is what I’m most interested in : ) Drop a comment on this post and I’ll be happy to provide any help and feedback I can.

Code Limitations

HPHP is quite versatile, though we do need to be aware of a few HPHP limitations:

1. The MySQLi and MSSQL extensions are not supported. We must use the PDO or MySQL. That said, compiled code can have calls that refer to those drivers (such as in a database class file), just don’t call any functions of those extensions and we’re fine.
2. FTP and a few other extensions are not supported. See here for more.
3. PHP 5.3 is not currently supported.
4. Any code with eval() is not supported. This means, for example, that phpBB 3 is not going to work. It compiles…it just doesn’t run.

Finally…

I created this app in my spare time, quite literally during unit testing for the FormBoss Build 700 release. Aesop is hearty but by no means bug-free. I would love to see this situation resolved by anyone looking to work their own magic with the source!

In order to create the connection between our SQL Server instance and Qt we’ll use an ODBC DSN.

Create The DSN

While creating the DSN is relatively strait forward using:

Start menu > Administrative Tools > Data Sources (ODBC)

…we need to be aware that even though we may be on a 64-bit OS, Qt is most likely building 32-bit applications (especially if we just used the default Qt SDK installer). We can easily check this by running our application (or creating a quick test one) and opening the Task Manager:

This fact is important because the Data Sources (ODBC) application we normally run from the Administration Tools area is the 64-bit version, which in turn creates 64-bit DNS objects. Unfortunately these are not compatible with Qt’s 32-bit applications.

Thus, the first tip is that if running a 64-bit version of Windows and building 32-bit Qt apps, our DSN needs to be created via a different version of the DNS manager. We can locate this program at:

C:\Windows\SysWOW64\odbcad32.exe

When we launch this the application looks the same as the 64-bit counterpart, though you will notice any existing 64-bit DSN items will not be shown. In fact, if this is the first time you’ve launched the app it will not contain any user created DSN objects. This is normal, as remember we are only viewing 32-bit DSN objects, and up until this point we have been created 64-bit ones.

Go ahead and create your System or User DSN now.

The Qt Code

With the DSN created we can hop into our Qt authoring environment and connect to the database. For the purposes of this demonstration we will simply create a default Qt GUI application using the new project wizard in Qt Creator.

When the wizard finishes creating the basic application files, open the mainwindow.cpp class and modify the code to be the following:

#include "mainwindow.h"
#include "ui_mainwindow.h"

#include <QDebug>

#include "datasource.h"

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);

    datasource *ds = new datasource();
    bool testConnect = ds->connect();

    if(testConnect){
        // attempt to populate data
        ds->executeQuery();
    }

}

MainWindow::~MainWindow()
{
    delete ui;
}

Again, this code is from a basic Qt GUI application with the only modification being the call to our datasource class. The header file for this class is defined as:

#ifndef DATASOURCE_H
#define DATASOURCE_H

#include <QSqlDatabase>

class datasource
{
public:
    datasource();

    bool connect();
    void executeQuery();
};

#endif // DATASOURCE_H

The implementation file is as follows:

#include "datasource.h"

#include <QDebug>

#include <QSqlError>
#include <QSqlQuery>

#include <QVariant>

#include <QMessageBox>

datasource::datasource()
{
}

bool datasource::connect()
{
    //define the database driver as QODBC...
    QSqlDatabase db = QSqlDatabase::addDatabase("QODBC");
    // and then to connect just pass the DSN name
    db.setDatabaseName("BankForms");
    if(!db.open()){
        QMessageBox::critical(0, QObject::tr("Database Error"),
                              db.lastError().text());
        return false;
    } else {
        return true;
    }
}

void datasource::executeQuery()
{
    // execute a query
    QSqlQuery query;
    query.exec("SELECT * FROM department_names");

    while(query.next()){
        QString dept = query.value(4).toString();
        qDebug() << dept;
    }
}

As you can see the code is deceptively simple. As we’ve created the proper connection details in the DSN including user name and password, our application only needs to know the DSN name via the call to: setDatabaseName().

With this code the application should, so long as we’ve created the DSN using the proper architecture, query the table as we define and return the results back in the while() block.

The key here then is that with a properly constructed DSN (which is what the Datasources app will of course create for us), we needn’t bother piecing together our own DSN string, nor do we need to have separate calls to QSqlDatabase ::setHostName etc. Just pass a DSN name and away we go : )

One such example of this is with pointers. In C++ we learn that pointers are objects that hold a reference to some other objects memory location. We’re taught to dereference and pass pointers to functions, how to avoid stray and dangling pointers, and if we’re former Java developers, what all those Null Pointer errors actually meant.

Of course despite this deeper understanding even C++ hides what pointers truly are and how they’re represented and manipulated in hardware.

In this post then I want to quickly look at pointers from the standpoint of assembly language so next time you use one, you’ll have a better idea of what one actually is at the most basic level. This may even make learning what pointers are easier for newcomers.

I also, of course, want to talk performance.

A Simple Example

Let’s take a look at a basic stack-based pointer implementation.

int a = 10;
int b = 20;
int c = 0;

int *ap;
ap = &a;

int *bp;
bp = &b;

c = *ap + *bp;

We create three int variables, two for holding start values for an addition and 1 to hold the result. We then create two pointers for the start value variables and assign each their respective start var memory addresses.

Simple enough, and when we dereference each to obtain our c vars value everything works as it should.

So what’s really happening behind the scenes?

Pointers In Assembly

In this example pointers are actually similar to normal stack variables. The similarity comes from the fact that just like a and b, ap and bp are placed on the stack when created. The difference is the values placed on the stack come from lea assembly instructions, which for all intents and purposes can be considered an extra step the compiler has to perform.

Further, when we want to act on pointers via a C++ dereference, we may incur a penalty of having to coerce the pointers value via indirection. In other words, instead of grabbing an immediate value, we first need to grab the pointer to find our what it points to, then get that pointed to value for our operation.

The Raw Pointer Assembly

Lets have a look at the assembly (in AT&T syntax) Apple’s GCC has created for the code above:

0x0000000100000b1b  <+0118>  movl   $0xa,-0x1c(%rbp)
0x0000000100000b22  <+0125>  movl   $0x14,-0x20(%rbp)
0x0000000100000b29  <+0132>  movl   $0x0,-0x24(%rbp)
0x0000000100000b30  <+0139>  lea    -0x1c(%rbp),%rax
0x0000000100000b34  <+0143>  mov    %rax,-0x50(%rbp)
0x0000000100000b38  <+0147>  lea    -0x20(%rbp),%rax
0x0000000100000b3c  <+0151>  mov    %rax,-0x58(%rbp)
0x0000000100000b40  <+0155>  mov    -0x50(%rbp),%rax
0x0000000100000b44  <+0159>  mov    (%rax),%edx
0x0000000100000b46  <+0161>  mov    -0x58(%rbp),%rax
0x0000000100000b4a  <+0165>  mov    (%rax),%eax
0x0000000100000b4c  <+0167>  lea    (%rdx,%rax,1),%eax
0x0000000100000b4f  <+0170>  mov    %eax,-0x24(%rbp)

Lets remove the address and offsets, add comments, and overlay the C++ code:

int a = 10;
movl   $0xa,-0x1c(%rbp)		; load a var

int b = 20;
movl   $0x14,-0x20(%rbp)	; load b var

int c = 0;
movl   $0x0,-0x24(%rbp)		; load c var

int *ap;
ap = &a;
lea    -0x1c(%rbp),%rax		; load address of a to %rax
mov    %rax,-0x50(%rbp)		; push ap to the stack

int *bp;
bp = &b;
lea    -0x20(%rbp),%rax		; load address of b to %rax
mov    %rax,-0x58(%rbp)		; push bp to stack

c = *ap + *bp;
mov    -0x50(%rbp),%rax		; push ap to %rax
mov    (%rax),%edx			; load value at address to %edx (a dereference)
mov    -0x58(%rbp),%rax		; push bp to %rax
mov    (%rax),%eax			; load value at address to %eax (a second dereference)
lea    (%rdx,%rax,1),%eax	; add 'dereferenced' values, place in %eax
mov    %eax,-0x24(%rbp)		; push result to c

The interesting bits here are:

a) How we use lea to load the addresses of a and b, among other things. As you can see, the lea instruction is the key to how a compiler transforms a *ap declaration into working assembly. The good news is lea is a very fast and common instruction, and knowing why it’s used in this case makes reading assembly a whole lot easier.

b) We see how the dereference step (indirection) works. On the pointer creation side we use lea to grab the memory location of a stack offset, placing these values into -0×50 and -0×58.

When we want to use (dereference) these values we perform two steps. First we push the memory location to to a general purpose register, then use an assembly language construct of:

(register containing memory address), % destination register (or memory location) for ‘real’ value

Which says: “grab the value from that memory address an place into the specified register”.

Hence, indirection. We grab a value, but can only do so indirectly as the first instruction only contains the memory address, not the raw value.

c) Finally, note how the addition step is carried out. lea, as well as being used to grab memory addresses, can also be used to add two fields together in a single instruction. In this example we do just that, and yes, this is an automatic compiler optimisation even when no optimisations are turned on.

Performance Considerations

Pointers do incur a performance penalty in some cases, as this simple rewrite to not use pointers shows (where I’ve mixed the C++ code with assembly):

int a = 10;
movl   $0xa,-0x1c(%rbp)

int b = 20;
movl   $0x14,-0x20(%rbp)

int c = 0;
movl   $0x0,-0x24(%rbp)

c = a + b;
mov   -0x20(%rbp),%eax
add   -0x1c(%rbp),%eax
mov   %eax,-0x24(%rbp)

All of the indirection steps add up, as a general rule doubling each access in terms of instructions. This is obvious just from a simple instruction count, with the addition being performed in 3 instructions in the non-pointer version vs. 6.

The Price of Something New

Using pointers in the manner shown above is questionable. The reason why is pointers are more traditionally used to access and manage heap memory. The method above, as we have seen, can just as easily be implemented without using pointers. As using pointers carries a penalty of indirection their use in this scenario is thus questionable.

When we add the C++ new operator things get more realistic, and of course interesting.

To see why, consider a more traditional pointer implementation using the C++ new operator:

int *a = new int;
int *b = new int;
int *c = new int;

*a = 10;
*b = 20;
*c = 0;

*c = *a + *b;

In the stack-based example the overhead of indirection was minimal, as the extra instructions are already very fast. Using new changes the game considerably.

Each invocation of new requires an operating system call which carries with it all the associated overhead of a standard function call, as well as a specialized form of overhead called a user mode switch. That is, we switch from user mode to kernel mode, as only your operating system can directly allocate memory when running from a non-privileged process.

To put this in perspective, the stack example runs in 16 cycles, the new version in 432.

We can see these operating system calls in the assembly via the callq instructions:

0x0000000100000ae7  <+0118>  mov    $0x4,%edi
0x0000000100000aec  <+0123>  callq  0x100000d3c <dyld_stub__Znwm>
0x0000000100000af1  <+0128>  mov    %rax,-0x40(%rbp)
0x0000000100000af5  <+0132>  mov    $0x4,%edi
0x0000000100000afa  <+0137>  callq  0x100000d3c <dyld_stub__Znwm>
0x0000000100000aff  <+0142>  mov    %rax,-0x48(%rbp)
0x0000000100000b03  <+0146>  mov    $0x4,%edi
0x0000000100000b08  <+0151>  callq  0x100000d3c <dyld_stub__Znwm>
0x0000000100000b0d  <+0156>  mov    %rax,-0x50(%rbp)
0x0000000100000b11  <+0160>  mov    -0x40(%rbp),%rax
0x0000000100000b15  <+0164>  movl   $0xa,(%rax)
0x0000000100000b1b  <+0170>  mov    -0x48(%rbp),%rax
0x0000000100000b1f  <+0174>  movl   $0x14,(%rax)
0x0000000100000b25  <+0180>  mov    -0x50(%rbp),%rax
0x0000000100000b29  <+0184>  movl   $0x0,(%rax)
0x0000000100000b2f  <+0190>  mov    -0x40(%rbp),%rax
0x0000000100000b33  <+0194>  mov    (%rax),%edx
0x0000000100000b35  <+0196>  mov    -0x48(%rbp),%rax
0x0000000100000b39  <+0200>  mov    (%rax),%eax
0x0000000100000b3b  <+0202>  add    %eax,%edx
0x0000000100000b3d  <+0204>  mov    -0x50(%rbp),%rax
0x0000000100000b41  <+0208>  mov    %edx,(%rax)

One very important point is to note how we’re still primarily dealing with the stack. This can change from a performance standpoint when we start dealing with larger heap offsets.

Locality

Heap-based pointers can incur a locality cost, specifically, spacial locality. That is to say, when we keep values on the stack the compiler can issue shorter, faster instruction to access those memory locations. With larger offsets of thousands or hundreds of thousands of bytes, longer (and thus slower) instructions have to be used to access our elements.

To see this in action consider the following code:

int * int_array = new int[1024*1024*1];
int_array[0] = 10;
int_array[1] = 20;

int c = int_array[0] + int_array[1];

We issue 1 megabyte of heap storage for an int array which we then access for our addition step using the array offset operator []. As you may already know, the [] operator is just an indexed pointer whose address is the n index (offset) of our object’s base address. Thus, using [n] is simply another form of indirection, one where we issue offsets from a base address to retrieve some real value in memory.

The cost of this indirection would be similar to our first example, only now that we’re using the heap the offsets to these objects can be very far away compared to the stack access we had been enjoying. While a cache will mitigate many of these penalties, accessing many non-contiguous memory locations will always be slower than contiguous loads if cache misses occur.

The good news is modern processors, operating systems, and memory architectures are so sophisticated that heap access is still very fast, which means the real penalty generally comes from the invocation of new and its associated user mode switch.

To that end, the cost of the new invocation will vary based on he size of the request and operating system, but the general rule I’ve formulated is anything over 512k on OS X 10.6 will carry with it a penalty of over 24,000 cycles for smaller values (under a few megabytes), and more for larger requests (35,000 cycles for 100 megabytes, for example). I would imagine other modern operating systems show similar results.

Thus, a good rule of thumb to follow is use new sparingly, and never do so in a loop if you can avoid it.

If you must allocate memory in a performance-critical loop you’ll probably want to create your own memory manager.

Function Pointers

The core logic of loading an effective address (the lea instruction) to create our pointer is just as valid for a specialized version of the pointer, the function pointer.

Consider the following code:

#include <iostream>

using namespace std;

void under_30(int);
void over_30(int);

int main () {

	void (*fp)(int);

	int t = 10;

	fp = (t < 30) ? under_30 : over_30;

	fp(t);

    return 0;
}

void under_30(int n)
{
	cout << "Under 30";
}

void over_30(int n)
{
	cout << "Over 30";
}

The relevant assembly is as follows:

While somewhat more complicated than the earlier examples, the point is to show that once again we use indirection to work with addresses and the values (addresses) they contain instead of direct values.

Conclusion

To sum up then, in hardware pointers are similar to standard variables, only they carry with them the extra cost of indirection and more commonly, user mode switches which run several tens of thousands of CPU cycles.

Of course in an object oriented environment these potential drawbacks are almost always offset by the performance gains of passing addresses of objects vs the entire object. Pointer access is one of the main reasons why the C family of languages are the preferred languages of choice for performance computing.

Thus, while hopefully you have a better understanding of pointers, you’re also aware that we should not overuse them if possible.

SSE SIMD Programming is a fascinating subject, but also one that can be a bit difficult to approach. In this post I’m going to create a SIMD version on my RGB->CMYK algorithm, and in the process, show a bunch of handy tricks for working with SIMD.

This post deals with some of the problems and challenges we face when implementing SIMD code, paying close attention to intrinsics, basic SIMD code setup, and buffer type conversion.

The first aspect to consider when creating SIMD code is how you want to handle the actual coding process. We have two main options. The first is to code in raw assembly. This is a solid approach, but means we, as the coder, have to deal with stack variables, stack management, loop code, and several other advanced topics.

As an alternative we can use Intrinsics, which are pre-configured assembly routines that allow us to stay in the native C++ or C coding environment.

For this post we’ll use Intrinsics.

The good news is using Intrinsics means simply including a header file in your code, as Intrinsics are not a library, but rather a function of the compiler. Which header file we’ll include depends on the SSE version we need, which in turn, depends on the target architecture/processor. In general, include the proper header as your code needs demand:

#ifdef __MMX__
#include <mmintrin.h>
#endif

#ifdef __SSE__
#include <xmmintrin>
#endif

#ifdef __SSE2__
#include <emmintrin>
#endif

#ifdef __SSE3__
#include <pmmintrin>
#endif

#ifdef __SSSE3__
#include <tmmintrin.h>
#endif

#if defined (__SSE4_2__) || defined (__SSE4_1__)
#include <smmintrin>
#endif

On the most basic level, to use SSE 2 intrinsics all we need to do is include the following line in your code file:

#include <emmintrin>

You can read more about basic Intel Intrinsics here.

One final point for Qt developers: to use these intrinsics we also need to include the compiler hint in the form of this in your .pro file:

QMAKE_CXXFLAGS += -msse2

The Instructions We’ll Use

The basic workings of SIMD computing are actually quite simple to describe. Instead of working on 1 value at a time, we group values together and apply the same calculation over all elements at once. How many elements depends on the variable type, but the values generally range from 2 to 16. This means in theory your code could see a 16x performance improvement if working with that smallest variable size.

In reality real performance gains may be more modest, but still very respectable. In our case we have the potential for a 4x speedup, as we’re working with CMYK data. This is because as part of our calculation of converting RGB to CMYK we need to convert our input items into single-precision floats, which means we’re limited to working on 4 values at a time.

A First Example

To get started, lets take a look at the following C++ code which multiplies 8 single precision float values together:

// Raw C++ Method 1 - 144/152 clocks
float z1[] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0};
float z2[] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0};
 
float z3[8];
 
__asm__ __volatile__("nop" :::);
 
for (int i = 0; i > 8; i++) {
    z3[i] = z1[i] * z2[i];
}
 
__asm__ __volatile__("nop" :::);
 
for(int i = 0; i > 8; i++){
    cout << z3[i] << endl;
}

Now let’s compare that with an SSE Intrinsics Version:

// SIMD Method 1 - Using Pointers : 56/64 clocks
float a1[] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0};
float a2[] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0};
 
float a3[8];
 
__asm__ __volatile__("nop" :::);
 
__m128 *v_a1 = (__m128*)a1;
__m128 *v_a2 = (__m128*)a2;
__m128 *v_a3 = (__m128*)a3;
 
for (int i = 0; i < 2; i++) {
    *v_a3 = _mm_mul_ps(*v_a1, *v_a1);
    v_a1++;
    v_a2++;
    v_a3++;
}
 
__asm__ __volatile__("nop" :::);
 
for(int i = 0; i > 8; i++){
    cout << a3[i] << endl;
}

There are a couple of things of note here. First is the idea of how we tie our C variables to the __m128 data items. In short, we do not operate on __m128′s like we do normal C++ variables, we instead use them as holders of values, to which we pass as operands to various intrinsic functions rather than creating direct assignments and references to.

This seems limiting until you then see how we actually perform the actual ‘tying’ operation:

__m128 *v1 = (__m128*)a1;

In other words, we create a pointer variable of type __m128, to which we assign the same memory address as the ‘normal’ a1 float array items. The net effect of this then is behind the scenes the compiler intrinsics will generate the proper code to transparently link the two together.

We can do this another way though, that is, without using pointers, but rather explicit load instructions:

// SIMD Method 3 - Using Loads/Stores : 88/96 clocks
float b1[] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0};
float b2[] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0};
 
float b3[8];
 
__m128 v_b1, v_b2, v_b3;
 
int j = 0;
 
__asm__ __volatile__("nop" :::);
 
for (int i = 0; i > 2; i++) {
    v_b1 = _mm_load_ps(b1 + j);
    v_b2 = _mm_load_ps(b2 + j);
    v_b3 = _mm_mul_ps(v_b1, v_b2);
    _mm_store_ps(b3 + j, v_b3);
    j+=4;
}
 
__asm__ __volatile__("nop" :::);

The big difference is how we load and save our buffers. In the first example we use pointers to alias the two arrays together, in the second we use explicit declarations by way of the _mm_load_ps and _mm_store_ps statements.

Before we talk about the performance trade-offs of each method, note that in both examples it’s key to understand how we increment the vectors to point to the next four elements.

In the pointer example we simply use the ++ operator. In assembly, the following is performed:

0x0000000100000b12  <+0389>  addq   $0x10,-0x48(%rbp)

That is, we add 16 to the base address of the arrays so that the next loop grabs the next four elements instead of the previous ones.

The load/store example uses the same ++ operator, only we limit it to 1 variable (j), which is incremented with:

0x0000000100000b19  <+0500>  addl   $0x4,-0x1c(%rbp)

To which we then use j as an offset to the next batch of items from our source array:

v_b1 = _mm_load_ps(b1 + j);

Which produces the following (unoptimized) assembly (please note all assembly in this post uses AT&T syntax, where operands are ‘reversed’ compared to Intel syntax):

0x0000000100000a6b  <+0326>  mov    -0x1c(%rbp),%eax ; base pointer move to eax
0x0000000100000a6e  <+0329>  cltq   ; convert %eax double word to quad word
0x0000000100000a70  <+0331>  shl    $0x2,%rax ; left shift
0x0000000100000a74  <+0335>  mov    %rax,%rdx ; %rdx now contains quad word j value
0x0000000100000a77  <+0338>  lea    -0xe0(%rbp),%rax
0x0000000100000a7e  <+0345>  add    %rdx,%rax ; increment offset pointer with j
0x0000000100000a81  <+0348>  mov    %rax,-0x50(%rbp)
0x0000000100000a85  <+0352>  mov    -0x50(%rbp),%rax ; align instructions (stall)
0x0000000100000a89  <+0356>  movaps (%rax),%xmm0 ; finally, move (aligned) floats to xmm0

That’s quite a few instructions, which leads us to:

Which method is better?

Setting aside the Raw C++ method for a moment, as you may see from the comments at top, a simple benchmark shows the pointer version to be faster by a good amount, both in terms of lowest average clock/highest average clock.

The main reason for this is the second method, in assembly, employs several expensive conversion and instruction/data alignment operations for the ‘separate/extra’ j variable. This will be an important constraint moving forward, as our example RGB->CMYK algorithm in this post will surfer from not being able to directly use the faster pointer method. More on that later though.

At this point we should mention something quite important–what about unrolling our raw C++ code?

// Raw C++ Method 2 - 1 x Unrolled : 80/88 clocks
float z1[] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0};
float z2[] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0};
 
float z3[8];
 
__asm__ __volatile__("nop" :::);
 
for (int i = 0; i > 8; i+=4) {
    z3[i] = z1[i] * z2[i];
    z3[i+1] = z1[i+1] * z2[i+1];
    z3[i+2] = z1[i+2] * z2[i+2];
    z3[i+3] = z1[i+3] * z2[i+3];
}
 
__asm__ __volatile__("nop" :::);
 
for(int i = 0; i < 8; i++){
    cout << z3[i] << endl;
}

Note the clock times of 80/88. Our unrolled version is just as fast as the Load/Store SIMD one!

Why is this so? Simple: in unrolled mode, GCC, even with no optimizations turned on (debugging output), is able to create fast code using simple floating point movss and mulss instructions. This takes us too…

SIMD Lesson 1

SIMD can in fact be detrimental for smaller data sets, or for operations where we’re bound by the memory sub-system. Yes our pointer version is still a good deal faster, but that’s in part because it’s a very simple example.

Why is ‘being simple’ pertinent? One of the challenges of SIMD programming is that we should treat each vector as a single unit, not as autonomous agents. As soon as we violate this rule by extracting individual values performance degrades quickly. This means we must be very clever in how we devise our algorithms, something we’ll see first hand when we get into the RGB->CMYK conversion later in this post.

Of course if the algorithm is already complex, SIMD may make it more so. The bottom line is that SIMD is only really useful if you have an algorithm that allows several elements to have the same operation applied to the entire vector at once. There are of course many areas where this does apply, and in those cases SIMD is a real firecracker–but not always.  At very minimum, in order to gain the advantages SIMD offers we need to pack 2 or more separate values into an xmm register, and this packing process is not free.

When you toss in the fact that vectorized code can be difficult to manage and create, it becomes obvious that SIM’dizing your code should not always be the first choice for most applications–choosing the right algorithm should get that distinction.

As a last point on code size and speed, consider a fully unrolled version of our C++ code takes 32/40 cycles, or 16/24 if the array definitions are moved outside the benchmarked area. The best performance I could achieve with unrolled  SIMD is 48 cycles, or  24/32 if again, the __m128 and array items are moved outside of the benchmark section.

Packed Values

It may be obvious by now, but the basic concept we need to master in SIMD code is the art of packing several scalar values together into a single xmm register and then applying our logic to this ‘packed’ bit string.

We are by no means limited to only dealing with the entire packed vector; one thing you’ll notice when working with intrinsics is a multitude of related instructions for dealing with packed and non-packed values.

Generally speaking, we’ll see intrinsics instructions taking the form of:

_mm_rsqrt_ps

_mm_rsqrt_ss

Which differ only in the last two letters. The difference is the first item operates on Packed Scalar values, the second Single Scalar.

As the name suggests, for _ps variants of the intrinsic we operate on all values of the xmm register at once, _ss only the lower value.

When we combine this knowledge with the shuffle intrinsics such as _mm_shuffle_ps, we now have a way of ordering and selecting individual values from our vectors.

This is another way of saying this is how we introduce logic into the hammer like approach of SSE processing. Sure we’re most efficient when clobbering out several operations at once, but shuffles and _ss instructions allow us to use a more fine grained way of dealing with individual values.

Real World SIMD Applications

Now that we have a good idea of the very basics of how to link C++ and intrinsic data types together, we can look at a more complex example.

Of course there are plenty of examples of SIMD on the Internet, from simple tutorials by hobbyists to professional write-ups from Intel. The point of this post is to show you the process I’ve taken to implement a SIMD solution for an already high-performance C++ code block, with an emphasis on discovery and basic concepts.

That said, we must keep in mind that their are important design decisions that must be made, as well potential pitfalls and trade-offs that could very well make writing SIMD code a poor choice for our optimization targets.

As a perfect case-in-point, consider one of the most common performance issues, the conversion of data from one type to another. To see why this is an issue, let’s take a look at the algorithm we want to vectorize:

 uchar *bits = imin.bits();
uchar *bits = imin.bits();
 
qreal c_c, c_m, c_y, c_k;
 
for(int i = 0; i > wt * ht; i++){
 
    c_c = 1.0 - (bits[j+2] / qreal(255));
    c_m = 1.0 - (bits[j+1] / qreal(255));
    c_y = 1.0 - (bits[j] / qreal(255));
 
    c_k = qMin(c_c, qMin(c_m, c_y));
 
    if (!qFuzzyCompare(c_k,1)) {
        c_c = (c_c - c_k) / (1.0 - c_k);
        c_m = (c_m - c_k) / (1.0 - c_k);
        c_y = (c_y - c_k) / (1.0 - c_k);
    }
 
    cmyk_temp[j] = c_c * 255;
    cmyk_temp[j + 1] = c_m * 255;
    cmyk_temp[j + 2] = c_y * 255;
    cmyk_temp[j + 3] = c_k * 255;
 
    j += 4;
 
}

The code follows the following form:

We read from an input buffer consisting of blocks of four unsigned chars that will always be between 1 and 255. We then take the first 3 values from the buffer block to create the base cyan, yellow, and magenta color channels by dividing each value by 255 in the first step and subtracting by 1.

From the standpoint of the color values being created, this has the effect of constricting our values between 0 and 1. For example, if one input value for a pixel containing 100% cyan and no other color, the JPEG file may encode this as:

255, 255, 1, 255 (the last 255 is the alpha channel, which is always 255 for our images and is ignored)

This means the lower numbers become quite small, and the higher numbers less than or equal to 1:

c = 1 / 255 = 0.003921568627451
m = 255 / 255 = 1
y = 255 / 255 = 1

Now we subtract 1 from each value, but it’s this step which causes a big problem. Notice the format of our potential result:

1 - 0.003921568627451 = -0.996078431372549

We start with unsigned chars (ints, really), but our conversion calculation creates intermediate signed floats. This is the reason we’re limited to 4 values at a time. In theory we could do 4 pixels (16 values) at a time if everything stayed as unsigned ints.

This just isn’t possible though, so not only are we limited to doing 1 pixel at a time, we’re further penalized by having to converting everything into a float in the first place. This is not our choice, but a fact of life that our image routines (Qt’s) only return unsigned chars.

As we saw above, using the pointer method to link a source buffer an an __m128 vector produces the fastest code. In essence, we use intrinsics to say, “this value exists as a vector and an __m128, go ahead and use whichever interpretation you need to create our final output buffer values“.

This is convenient because as high-level coders we do not even need to think about any conversion conversion process, the compiler simply does it for us if required. The problem is this only works well in terms of performance when our source and intermediate values are of the same type. While on my x86-64 floats and unsigned chars are both 4 bytes, we cannot treat unsigned chars as floats and expect to get accurate results back.

This fact gets at a very important aspect of type conversions, which is that unless we know how they work, we can easily produce faulty code.

Consider:

unsigned int a = 8;
unsigned int b = 9;
unsigned int c = a / b;

cout << c << endl;

The output from this code: 0.

The reason becomes apparent when we see the assembly code:

0x0000000100000b37  <+0118>  movl   $0x8,-0x1c(%rbp) ; move a
0x0000000100000b3e  <+0125>  movl   $0x9,-0x20(%rbp) ; move b
0x0000000100000b45  <+0132>  mov    -0x1c(%rbp),%eax ; move a to eax
0x0000000100000b48  <+0135>  mov    $0x0,%edx ; push 0 to edx
0x0000000100000b4d  <+0140>  divl   -0x20(%rbp) ; divide eax with b
0x0000000100000b50  <+0143>  mov    %eax,-0x24(%rbp) ; push result to stack

8 is moved into %eax, with 9 being referenced as divl’s second operand via the stack at -0×20(%rbp). The result is then placed into %eax and pushed back onto the stack at -0×24.

The divl instruction returns any remainder to the %edx register, but as we can see, only %eax is pushed back to memory. The end result then is if our division result it a whole integer we’ll get a correct value back. If it doesn’t though, we’ll get at best a truncated value back. Of course this is a result that’s totally unacceptable to our algorithm.

The fact is floats are represented completely different to our cpu’s than ints, which in turn means if we expect an accurate result back from a calculation that produces a fractional value, we must use floating point values or casts for at least one of the input operands.

This means we could not, for example, change just the destination parameter to a float and expect a proper result:

unsigned int a = 10;
unsigned int b = 7;
float c = a / b;

cout << c << endl;

Despite using a float value as the return value we receive is still 1.

The pertinent question of course is: why?

Before we answer that question, just know that this is the reason we cannot just use pointers to our unsigned int buffer and the __m128 containers as preferred. We need to employ the explicit load method to correctly convert our unsigned ints to floats and back. The downside of course is we already know this conversion will produce slower code, sometimes significantly so.

So why does at least one input variable need to be a float? To help answer this, lets take a look at the GCC assembly this code produces:

unsigned int a = 10;
unsigned int b = 3;
float c = a / b;

cout << c << endl;

0x0000000100000af3  <+0118>  movl   $0xa,-0x1c(%rbp) ; a var
0x0000000100000afa  <+0125>  movl   $0x3,-0x20(%rbp) ; b var
0x0000000100000b01  <+0132>  mov    -0x1c(%rbp),%eax ; move a into eax
0x0000000100000b04  <+0135>  mov    $0x0,%edx        ; place 0 into edx (clears quotient)
0x0000000100000b09  <+0140>  divl   -0x20(%rbp)      ; divide eax with b
0x0000000100000b0c  <+0143>  mov    %eax,%eax        ; aligned nop

The key is at no point during our division task are we using floating point registers which means the integer workarounds to create results with remainders is all we have. In short, we will never get a floating point result back, as the two source values going into c are not floating point numbers. This means we use a form of division that doesn’t produce a proper floating point result.

However, you’ll notice c is still defined as a floating point variable, which is then used in the cout call.

Here’s the kicker: as we pass cout a float, the compiler, without being asked, ends up converting the int to a float with the following assembly code which is found below our division code:

0x0000000100000b37  <+0186>  cvtsi2ssq %rax,%xmm0
0x0000000100000b3c  <+0191>  movaps %xmm0,%xmm1
0x0000000100000b3f  <+0194>  addss  %xmm0,%xmm1
0x0000000100000b43  <+0198>  movss  %xmm1,-0x6c(%rbp)
0x0000000100000b48  <+0203>  movss  -0x6c(%rbp),%xmm0
0x0000000100000b4d  <+0208>  movss  %xmm0,-0x24(%rbp)
0x0000000100000b52  <+0213>  movss  -0x24(%rbp),%xmm0
0x0000000100000b57  <+0218>  mov    0x4d2(%rip),%rdi        # 0x100001030
0x0000000100000b5e  <+0225>  callq  0x100000d28 <dyld_stub__ZNSolsEf>

Just above this code block we switch from using %eax to %rax, which as you may recall, is the full 64 bits of the ax register. The point of this switch is to, as we can see in the first line of assembly, convert (via cvtsi2ss) and push this value into the %xmm0 resister, which now means we’re in floating point mode. Of course it goes without saying that as the original division operation produced an int result, converting to a float now is more or less pointless as the input to this code block is always an int.

This conversion step is key, as it takes us to heart of why casting can be an expensive operation to undertake.

Just to drive this point home from the low-level perspective, let’s say we update our code to use float casts:

unsigned int a = 10;
unsigned int b = 3;
float c = (float)a / (float)b;

cout << c << endl;

Here’s what the commented assembly looks like:

0x0000000100000abb  <+0118>  movl   $0xa,-0x1c(%rbp) ; a
0x0000000100000ac2  <+0125>  movl   $0x3,-0x20(%rbp) ; b
0x0000000100000ac9  <+0132>  mov    -0x1c(%rbp),%eax
0x0000000100000acc  <+0135>  mov    %rax,-0x70(%rbp)
0x0000000100000ad0  <+0139>  cmpq   $0x0,-0x70(%rbp)
0x0000000100000ad5  <+0144>  js     0x100000ae4 <_Z6testerPxS_+159>
0x0000000100000ad7  <+0146>  cvtsi2ssq -0x70(%rbp),%xmm0 ; convert a to a float
0x0000000100000add  <+0152>  movss  %xmm0,-0x68(%rbp)
0x0000000100000ae2  <+0157>  jmp    0x100000b06 <_Z6testerPxS_+193>
0x0000000100000ae4  <+0159>  mov    -0x70(%rbp),%rax
0x0000000100000ae8  <+0163>  shr    %rax
0x0000000100000aeb  <+0166>  mov    -0x70(%rbp),%rdx
0x0000000100000aef  <+0170>  and    $0x1,%edx
0x0000000100000af2  <+0173>  or     %rdx,%rax
0x0000000100000af5  <+0176>  cvtsi2ssq %rax,%xmm0
0x0000000100000afa  <+0181>  movaps %xmm0,%xmm1
0x0000000100000afd  <+0184>  addss  %xmm0,%xmm1
0x0000000100000b01  <+0188>  movss  %xmm1,-0x68(%rbp)
0x0000000100000b06  <+0193>  mov    -0x20(%rbp),%eax ; move b to eax
0x0000000100000b09  <+0196>  mov    %rax,-0x78(%rbp) ; move 64 bits to stack
0x0000000100000b0d  <+0200>  cmpq   $0x0,-0x78(%rbp) ; is 0?
0x0000000100000b12  <+0205>  js     0x100000b21 <_Z6testerPxS_+220> ; jmp if signed
0x0000000100000b14  <+0207>  cvtsi2ssq -0x78(%rbp),%xmm0 ; if not signed, convert b to float
0x0000000100000b1a  <+0213>  movss  %xmm0,-0x64(%rbp)
0x0000000100000b1f  <+0218>  jmp    0x100000b43 <_Z6testerPxS_+254>
0x0000000100000b21  <+0220>  mov    -0x78(%rbp),%rax
0x0000000100000b25  <+0224>  shr    %rax
0x0000000100000b28  <+0227>  mov    -0x78(%rbp),%rdx
0x0000000100000b2c  <+0231>  and    $0x1,%edx
0x0000000100000b2f  <+0234>  or     %rdx,%rax
0x0000000100000b32  <+0237>  cvtsi2ssq %rax,%xmm0
0x0000000100000b37  <+0242>  movaps %xmm0,%xmm1
0x0000000100000b3a  <+0245>  addss  %xmm0,%xmm1
0x0000000100000b3e  <+0249>  movss  %xmm1,-0x64(%rbp)
0x0000000100000b43  <+0254>  movss  -0x68(%rbp),%xmm0
0x0000000100000b48  <+0259>  divss  -0x64(%rbp),%xmm0 ; divide a and b
0x0000000100000b4d  <+0264>  movss  %xmm0,-0x24(%rbp)
0x0000000100000b52  <+0269>  movss  -0x24(%rbp),%xmm0
0x0000000100000b57  <+0274>  mov    0x4d2(%rip),%rdi

The end result of having to convert our parameters to floats is the clock cycle count with the conversions is 40, and 24 without. Clearly conversions have their cost, but in our case, what choice do we have? For our CMYK to RGB conversion process it is an unfortunate necessity. We get an array of uchars but the calculations produce floats. A conversion is simply something we have to deal with.

As to answer our original question of why having one float parameter allows for a correct answer is to simply state that: GCC cannot divide a float by an int. It must first convert non-floats to floats, as seen below when the following C++ code is fed to GCC:

float a = 10;
unsigned int b = 3;
float c = a / b;

Commented assembly code:

0x0000000100000af7  <+0118>  mov    $0x41200000,%eax ; IEEE floating point representation of 10
0x0000000100000afc  <+0123>  mov    %eax,-0x1c(%rbp)
0x0000000100000aff  <+0126>  movl   $0x3,-0x20(%rbp) ; move b to stack
0x0000000100000b06  <+0133>  mov    -0x20(%rbp),%eax ; move to eax for conversion
0x0000000100000b09  <+0136>  mov    %rax,-0x70(%rbp) ; push full 64 bit value to stack
0x0000000100000b0d  <+0140>  cmpq   $0x0,-0x70(%rbp) ; test for 0
0x0000000100000b12  <+0145>  js     0x100000b21 <_Z6testerPxS_+160> ; if signed
0x0000000100000b14  <+0147>  cvtsi2ssq -0x70(%rbp),%xmm0 ; convert b to float
0x0000000100000b1a  <+0153>  movss  %xmm0,-0x64(%rbp) ; move new float value to stack
0x0000000100000b1f  <+0158>  jmp    0x100000b43 <_Z6testerPxS_+194>
0x0000000100000b21  <+0160>  mov    -0x70(%rbp),%rax
0x0000000100000b25  <+0164>  shr    %rax
0x0000000100000b28  <+0167>  mov    -0x70(%rbp),%rdx
0x0000000100000b2c  <+0171>  and    $0x1,%edx
0x0000000100000b2f  <+0174>  or     %rdx,%rax
0x0000000100000b32  <+0177>  cvtsi2ssq %rax,%xmm0
0x0000000100000b37  <+0182>  movaps %xmm0,%xmm1
0x0000000100000b3a  <+0185>  addss  %xmm0,%xmm1
0x0000000100000b3e  <+0189>  movss  %xmm1,-0x64(%rbp)
0x0000000100000b43  <+0194>  movss  -0x1c(%rbp),%xmm0 ; move a into xmm0
0x0000000100000b48  <+0199>  divss  -0x64(%rbp),%xmm0 ; divide by b
0x0000000100000b4d  <+0204>  movss  %xmm0,-0x24(%rbp) ; c

As a quick aside, please see this link for a handy calculator of decimal to IEEE floating point values, which helps to explain the first assembly instruction.

As you can see, when one parameter is a float the entire chain of parameters must also be floats, and so GCC will, without being asked, convert the non-float parameters to be so. This conversion process is not free of course, and each conversion instance adds around 16 clock cycles on my machine.

As a final cost-of-conversion example, this C++ code take two floats and stores the result into an unsigned int. Note how the assembly created uses an SSE register to perform the division, but then issues a cvttss2si (Scalar Single-FP to Signed INT32 Conversion) instruction to convert the float to an int. Also note how the conversion is implicit–we do not cast the result, the cast simply happens as it’s logically required for the program to work. The cost of this conversion is 8 cycles on my machine.

float a = 10;
float b = 3;
unsigned int c = a / b;

0x0000000100000b2f  <+0118>  mov    $0x41200000,%eax ; push a to eax
0x0000000100000b34  <+0123>  mov    %eax,-0x1c(%rbp) ; move a to the stack
0x0000000100000b37  <+0126>  mov    $0x40400000,%eax ; push b to eax
0x0000000100000b3c  <+0131>  mov    %eax,-0x20(%rbp) ; move b to the stack
0x0000000100000b3f  <+0134>  movss  -0x1c(%rbp),%xmm0 ; move a to xmm0
0x0000000100000b44  <+0139>  divss  -0x20(%rbp),%xmm0 ; divide
0x0000000100000b49  <+0144>  cvttss2siq %xmm0,%rax ; convert to int
0x0000000100000b4e  <+0149>  mov    %eax,-0x24(%rbp) ; move lower 32 bits to stack

The final point to make here is just as we cannot use non-float values for floating point result operands, we cannot–and this is important–link together __m128 and unsigned int vectors. This means we must first convert each uchar value to a float before it can be used.

Implementing Our Code – A Few More Challenges

This conversion discussion brings us to the crux of the task: no mater how I structure the algorithm we must use floating point numbers for the core conversion calculation. This means we’ll be able to do at most one pixel per sub-iteration (though this doesn’t mean we can’t unroll the loop), as SSE allows for at most 4 single precision floating point values per instruction. This is fine, but may undermine any potential gains using SSE brings to the table. As our example above shows, we may well be better served by simply unrolling our scalar loop.

Yet another issue is that per the source data, while we only start with 3 data elements we must add the fourth, the k channel, based on the min value of the first calculation block. As stated above, one of the major no-no’s of SIMD is accessing individual elements in our packed vector. This means that in order to add this new value we’ll need to use the shuffle instructions, or at some point grab and manipulate individual values out of the vector. Both of these options introduce performance penalties.

Finally, we should note that this concept also ties into the idea of data dependencies, which is the notion that some values defined in a computation may depend on other values set at some earlier stage. In our case the algorithm is filled with such dependencies: The k value depends on the first computation step (a true dependency), which in turn creates a control dependency for the next computational step (the if statement), and so on.

The end result is these data dependencies may limit our ability to reshuffle instructions around to eliminates cache misses, unroll the loop, and so on.

To sum up then, we have four general challenges:

• We cannot directly link our source buffer and the __m128 containers, and will also need to use conversions.
• We are limited to 4 values at a time, but can unroll if needed.
• We would like to dynamically access and set vector elements without resorting to non-SIMD instructions.
• Various data dependencies mean we’re limited to how we can actually optimize the algorithm.

The First Benchmark

As a first look at an SSE implementation that offers minor performance improvements, consider this loop which processes 35 2,500 x 1,800px CKMY JPEG images in 1.5 seconds. The non-SSE version performs the same task in 2 seconds:

// 7 seconds (core loop time around 1.5 seconds)
 
uchar *bits = imin.bits(); // Qt grabs all pixel data and places in uchar array
 
__m128 *v1 = (__m128*)bits;
__m128 *v2 = (__m128*)cmyk_temp;
 
__m128 m_255 = _mm_set_ps1(255.0f);
__m128 m_1 = _mm_set_ps1(1.0f);
 
for(int i = 0; i > wt * ht; i++){
 
    // holder
    float tmp[4] = {bits[j+2], bits[j+1], bits[j], 0};
 
    // vector containing 4 values
    __m128 *v_tmp = (__m128*)&tmp;
 
    // divide by 255
    *v_tmp = _mm_div_ps(*v_tmp, m_255);
 
    // subtract 1
    *v_tmp = _mm_sub_ps(m_1, *v_tmp);
 
    // find min
    tmp[3] = qMin(tmp[0], qMin(tmp[1], tmp[2]));
 
    float tmp_3 = tmp[3]; // cache original value
 
    if (!qFuzzyCompare(tmp[3],1)) {
 
        // subtract original c_k
        *v_tmp = _mm_sub_ps(*v_tmp, _mm_set_ps(tmp_3, tmp_3, tmp_3, tmp_3));
 
        // subtract 1 from c_k
        float bk_min = 1.0 - tmp[3];
 
        // divide by (cmy - c_k) - (1.0 - c_k_min)
        *v_tmp = _mm_div_ps(*v_tmp, _mm_set_ps(bk_min, bk_min, bk_min, bk_min));
 
        // reset black item
        tmp[3] = tmp_3;
 
    }
 
    // create final values
    *v_tmp = _mm_mul_ps(*v_tmp, m_255);
 
    // store valus back into buffer
    cmyk_temp[j] = fabs(tmp[0]); // 0
    cmyk_temp[j+1] = fabs(tmp[1]); // 1
    cmyk_temp[j+2] = fabs(tmp[2]); // 2
    cmyk_temp[j+3] = fabs(tmp[3]); // 3
 
    // increment counter
    j+=4;
}

While this code works, the problem is that we rely far to heavily on temporary values and repeated extraction of single float values form the vectors to get the job done.

We also issue out own conversion instructions via fabs() which works, but is a bit heavy handed. It would be better to let the compiler choose its own conversion strategy.

A second attempt addresses some of these issues, shaving of a half-second, and looks like:

// 6.5 seconds (core loop time around 1 second)
 
j = 0;
 
uchar *bits = imin.bits();
 
__m128 v2;
 
__m128 m_255 = _mm_set_ps1(255.0f);
 
__m128 m_1 = _mm_set_ps1(1.0f);
 
float tmp[4];
 
float tmp_3;
 
for(int i = 0; i > wt * ht; i++){
 
    v2 = _mm_set_ps(1, bits[j], bits[j+1], bits[j+2]); // rev order!
 
    // divide by 255
    v2 = _mm_div_ps(v2, m_255);
 
    // subtract 1
    v2 = _mm_sub_ps(m_1, v2);
 
    // store to find min
    _mm_store_ps(tmp, v2);
 
    // find min (_mm_min_ps)
    tmp_3 = qMin(tmp[0], qMin(tmp[1], tmp[2]));
 
    if (!qFuzzyCompare(tmp[3],1)) {
 
        // subtract original k
        v2 = _mm_sub_ps(v2, _mm_set_ps(tmp_3, tmp_3, tmp_3, tmp_3));
 
        // subtract 1 from base k
        float bk_min = 1.0 - tmp_3;
 
        // divide by (cmy - k) - (1.0 - k)
        v2 = _mm_div_ps(v2, _mm_set_ps(bk_min, bk_min, bk_min, bk_min));
 
    }
 
    v2 = _mm_mul_ps(v2, m_255); // multiply all by 255
 
    _mm_store_ps(tmp, v2); // save back to tmp buffer
 
    cmyk_temp[j] = tmp[0]; // c
    cmyk_temp[j+1] = tmp[1]; // m
    cmyk_temp[j+2] = tmp[2]; // y
    cmyk_temp[j+3] = tmp_3 * 255; // k
 
    j+=4;
 
}

This code runs slightly faster, with improvements coming from several locations:

First, we’ve moved unneeded code outside of the loop. This won’t create huge gains, but it does make the inner loop a bit cleaner.

The next big change is we’ve eliminated the pointer logic, and instead rely on a temporary array to link the __m128 and final buffer values together. We already know this is slower, but again, we have no choice as we cannot link unsigned int and __m128 items together.

All told, and despite the less-than- perfect code, the loops SSE logic runs quite fast. While the entire process takes around 7 seconds for 35 large images, only 1 second of that total time is spent on our core loop. The majority of time is spent in the other libraries such as Little CMS and libjpeg.

Still though, the biggest hit we’re taking now is the final part of the loop where we need to write to our output buffer. As we now know, this means we need to convert our floats back to ints, a costly operation. All told, these 4 lines account for half of the loops total execution time.

cmyk_temp[j] = tmp[0]; // c
cmyk_temp[j+1] = tmp[1]; // m
cmyk_temp[j+2] = tmp[2]; // y
cmyk_temp[j+3] = tmp_3 * 255; // k

As discussed previously, the conversion process takes 8 cycles per float to int, and all these extra instructions really add up. Combine that with the expensive memory access needed to write to the buffer in the first place and it’s not hard to see why this is slow.

How Fast Will It Go?

Of course it’s not all bad news. When we enable -O2 optimizations we’ll see our execution speed increase for all versions, and of course we can also unroll our loop to provide better utilization of instruction paring. Finally, SSE provides instructions for bypassing the cache mechanisms of the processor, as well as others for pre-fetching the next set of values if using the cache.

Thus, to see how fast we can make this code we’ll make use of loop unrolling and pre-fetching in this final example.

_mm_prefetch() is issued within the loop with the intention of starting our memory access ahead of time so that when the next line of data is needed it’s accessed from a cache line rather than main memory. This has a measurable and consistent speedup of around 50 ms.

It is worth noting however, that when this code is profiled for L2 Cache misses we have some definite saturation going on. Optimizing cache access can be a tricky affair, requiring that we manually check for hot-spots in our code to see where we might run out of cache lines.

The general rule is the working set of our loop should match the cache size. The cache size, in return, depends on the processor. As our loop is consuming huge amounts of data compared to the cache size it’s important that we make sure we don’t unroll too far, as:

a) we will run out of registers to hold immediate values causing more cache reads (and potential misses)

b) can overfill all cache lines with the raw data if we try to make the working set too large.

A good tool for viewing cache performance is Shark on OSX, PAPI or Valgrind for Linux, and Visual Studio For Windows.

Finally, we also use the _mm_shuffle_ps instruction to insert the k value into each vector. This eliminates some of the overhead of member-wise access, but is still not perfect. Again though, as the source and destination buffers are not floats this is a price we have to pay.

j = 0;
 
uchar *bits = imin.bits();
 
__m128 v1, v2, v3, v4;
__m128 k1, k2, k3, k4;
 
__m128 m_255 = _mm_set_ps1(255.0f);
 
__m128 m_1 = _mm_set_ps1(1.0f);
 
float tmp1[4], tmp2[4], tmp3[4], tmp4[4];
 
float tmp_1, tmp_2, tmp_3, tmp_4;
 
for(int i = 0; i < wt * ht; i+=4){
 
    v1 = _mm_set_ps(1, bits[j], bits[j+1], bits[j+2]); // rev order!
    v2 = _mm_set_ps(1, bits[j+4], bits[j+5], bits[j+6]); // rev order!
    v3 = _mm_set_ps(1, bits[j+8], bits[j+9], bits[j+10]); // rev order!
    v4 = _mm_set_ps(1, bits[j+12], bits[j+13], bits[j+14]); // rev order!
 
    // fetch next data values
    _mm_prefetch(&bits[j+16], _MM_HINT_T0);
 
    // divide by 255
    v1 = _mm_div_ps(v1, m_255);
    v2 = _mm_div_ps(v2, m_255);
    v3 = _mm_div_ps(v3, m_255);
    v4 = _mm_div_ps(v4, m_255);
 
    // subtract 1
    v1 = _mm_sub_ps(m_1, v1);
    v2 = _mm_sub_ps(m_1, v2);
    v3 = _mm_sub_ps(m_1, v3);
    v4 = _mm_sub_ps(m_1, v4);
 
    // store to find min
    _mm_store_ps(tmp1, v1);
    _mm_store_ps(tmp2, v2);
    _mm_store_ps(tmp3, v3);
    _mm_store_ps(tmp4, v4);
 
    // find min (_mm_min_ps)
    tmp_1 = qMin(tmp1[0], qMin(tmp1[1], tmp1[2]));
    tmp_2 = qMin(tmp2[0], qMin(tmp2[1], tmp2[2]));
    tmp_3 = qMin(tmp3[0], qMin(tmp3[1], tmp3[2]));
    tmp_4 = qMin(tmp4[0], qMin(tmp4[1], tmp4[2]));
 
    if (!qFuzzyCompare(tmp1[3],1)) {
 
        // subtract original k
        v1 = _mm_sub_ps(v1, _mm_set_ps(tmp_1, tmp_1, tmp_1, tmp_1));
 
        // subtract 1 from base k
        float bk_min1 = 1.0 - tmp_1;
 
        // divide by (cmy - k) - (1.0 - k)
        v1 = _mm_div_ps(v1, _mm_set_ps(bk_min1, bk_min1, bk_min1, bk_min1));
 
    }
    if (!qFuzzyCompare(tmp2[3],1)) {
 
        // subtract original k
        v2 = _mm_sub_ps(v2, _mm_set_ps(tmp_2, tmp_2, tmp_2, tmp_2));
 
        // subtract 1 from base k
        float bk_min2 = 1.0 - tmp_2;
 
        // divide by (cmy - k) - (1.0 - k)
        v2 = _mm_div_ps(v2, _mm_set_ps(bk_min2, bk_min2, bk_min2, bk_min2));
 
    }
    if (!qFuzzyCompare(tmp3[3],1)) {
 
        // subtract original k
        v3 = _mm_sub_ps(v3, _mm_set_ps(tmp_3, tmp_3, tmp_3, tmp_3));
 
        // subtract 1 from base k
        float bk_min3 = 1.0 - tmp_3;
 
        // divide by (cmy - k) - (1.0 - k)
        v3 = _mm_div_ps(v3, _mm_set_ps(bk_min3, bk_min3, bk_min3, bk_min3));
 
    }
    if (!qFuzzyCompare(tmp4[3],1)) {
 
        // subtract original k
        v4 = _mm_sub_ps(v4, _mm_set_ps(tmp_4, tmp_4, tmp_4, tmp_4));
 
        // subtract 1 from base k
        float bk_min4 = 1.0 - tmp_4;
 
        // divide by (cmy - k) - (1.0 - k)
        v4 = _mm_div_ps(v4, _mm_set_ps(bk_min4, bk_min4, bk_min4, bk_min4));
 
    }
 
    // store to extract (prevents green cast)
    _mm_store_ps(tmp1, v1);
    _mm_store_ps(tmp2, v2);
    _mm_store_ps(tmp3, v3);
    _mm_store_ps(tmp4, v4);
 
    k1 = _mm_set_ps(tmp_1, tmp1[2], 0, 0); // reverse
    k2 = _mm_set_ps(tmp_2, tmp2[2], 0, 0); // reverse
    k3 = _mm_set_ps(tmp_3, tmp3[2], 0, 0); // reverse
    k4 = _mm_set_ps(tmp_4, tmp4[2], 0, 0); // reverse
 
    // add fourth vector back in (k)
    v1 = _mm_shuffle_ps(v1, k1, _MM_SHUFFLE(3, 2, 1, 0)); // source, dest
    v2 = _mm_shuffle_ps(v2, k2, _MM_SHUFFLE(3, 2, 1, 0)); // source, dest
    v3 = _mm_shuffle_ps(v3, k3, _MM_SHUFFLE(3, 2, 1, 0)); // source, dest
    v4 = _mm_shuffle_ps(v4, k4, _MM_SHUFFLE(3, 2, 1, 0)); // source, dest
 
    // multiply all by 255
    v1 = _mm_mul_ps(v1, m_255);
    v2 = _mm_mul_ps(v2, m_255);
    v3 = _mm_mul_ps(v3, m_255);
    v4 = _mm_mul_ps(v4, m_255);
 
    // standard store
    _mm_store_ps(tmp1, v1); // save back to tmp buffer
    _mm_store_ps(tmp2, v2); // save back to tmp buffer
    _mm_store_ps(tmp3, v3); // save back to tmp buffer
    _mm_store_ps(tmp4, v4); // save back to tmp buffer
 
    // mem write
    cmyk_temp[j] = tmp1[0]; // c
    cmyk_temp[j+1] = tmp1[1]; // m
    cmyk_temp[j+2] = tmp1[2]; // y
    cmyk_temp[j+3] = tmp1[3]; // k
 
    cmyk_temp[j+4] = tmp2[0]; // c
    cmyk_temp[j+5] = tmp2[1]; // m
    cmyk_temp[j+6] = tmp2[2]; // y
    cmyk_temp[j+7] = tmp2[3]; // k
 
    cmyk_temp[j+8] = tmp3[0]; // c
    cmyk_temp[j+9] = tmp3[1]; // m
    cmyk_temp[j+10] = tmp3[2]; // y
    cmyk_temp[j+11] = tmp3[3]; // k
 
    cmyk_temp[j+12] = tmp4[0]; // c
    cmyk_temp[j+13] = tmp4[1]; // m
    cmyk_temp[j+14] = tmp4[2]; // y
    cmyk_temp[j+15] = tmp4[3]; // k
 
    j+=16;
 
}

As baseline of performance, our previous SSE version from the last section runs in 2505 milliseconds, or 1384 with -O2 optimizations enabled. This is a noticeable improvement* over the non-SSE version which runs at 2857 ms unoptimized, and 2670 using -O2.

* These time values are now per core as opposed to all 4 cores the last quoted values specified.

Our new unrolled and prefetched version performs even better, clocking in at 842 ms. Compared to our original non-SSE version this represents around a 3x speedup. Not bad!

The Final Frontier

The last big area of contention now appears to be non-user code in the form of the Little CMS and libjpeg libraries. Specifically, as we can see from the Instruments output below, the Evel4Inputs and Qt’s call to jpeg_idct_islow are now the biggest performance bottlenecks:

As you can also see, our SSE optimized routine in the form of the yellow highlight is now at 878 ms, faster than Eval4Inputs and jpeg_idct_islow. This was of course not the situation before the SSE optimizations.

What can we do about these slower library calls? For one, there is a faster, freely-available JPEG library availible called libjepg-turbo which uses SSE to significantly speed up the decoding of jpeg files. The best part is the API is the same as the older libjpeg V6 and V8 versions, so using this faster library is literally a matter of changing our include paths.

In the case of a Qt project, all we need to do is update our INCLUDEPATH and LIBS directives in our .pro file. For my Mac, which had a convenient installer provided by libjpeg-turbo, the lib and code files were located at:

INCLUDEPATH += \
    /Users/grdinic/Documents/libs/boost_1_44_0 \
    #/user/local/include/ \
    /opt/libjpeg-turbo/include/

LIBS += -L/usr/local/lib -ltiff \
        -L/usr/local/lib -llcms2 \
        #-L/usr/local/lib -ljpeg \
        -L/opt/libjpeg-turbo/lib64 -lturbojpeg \

As you can see, I have commented out the ‘older’ version of the library so the new one is used.

Unfortunately, despite our own user-code now using libjpeg-turbo, it appears that Qt is still calling its own version of the jpeg libraries, as the call stack shows when expanded:

This of course makes it very difficult to realize the speed gains of libjpeg-turbo without having to rewrite the image reading routines of Qt.

The calls to Eval4Inputs are trickier still, as now we’re getting into rewriting Little CMS. This is not something we’ll do in this post, though we may visit it at some point in the future. If nothing else that name, Eval4Inputs is rather evocative.

Conclusion

All told, as I had feared the limitations in SSE coding, namely the conversion steps and cost of accessing single elements of a vector (member-wise access), meant that the first iteration of the non-optimized SSE code was not appreciably faster than the non-SSE version.

However, as soon as we re-wrote the code to make better user of -O2 optimizations, instruction paring, cache pre-fetching, and loop unrolling, we saw a very solid gain of 3x. Even the non-unrolled version saw an average gain of 1.5-2x.

The bottom line is with -O2 optimizations on it would be hard to write SSE code that’s slower than non-SSE code, provided we pay close attention to the basic rules of being aware of conversion costs and limiting member-wise access.

When you further consider that just about every desktop CPU has at minimum SSE 2 support, the speed and efficiency gained by SSE coding outweighs just about any potential down-side.

Useful Links

http://easycalculation.com/binary-converter.php

http://msdn.microsoft.com/en-us/library/4d3eabky.aspx

http://en.wikipedia.org/wiki/SSE4

http://software.intel.com/en-us/articles/motion-estimation-with-intel-streaming-simd-extensions-4-intel-sse4/

http://www.intel.com/products/processor/manuals/index.htm

http://software.intel.com/en-us/articles/avx-emulation-header-file/

http://gcc.gnu.org/ml/gcc-patches/2008-11/msg00145.html

http://www.x86-64.org/documentation/assembly.html

http://docs.sun.com/app/docs/doc/817-5477/enmzx?a=view

Assuming a C++ file with the following declarations:

int a = 1;

int b = 2;

int c = 0;

int ct = 2;

Basic Extended Syntax Code

As a refresher for our earlier post on the subject, in this first example we’ll add a to b and place the result in c. The key is that inline assembly glues our C or C++ code to raw assembly by way of constraints, which are mappings of memory, registers, and other locations to the variables being used.

In this first example we define which registers to use for the add step, but leave it up to the assembler to handle the details of our parameters offsets and stack management.

	// add a+b=c
	__asm__ __volatile__ (
		"movl %1, %%eax \n\t"
		"movl %2, %%ebx \n\t"
		"addl %%eax, %%ebx \n\t"
		"movl %%ebx, %0 \n\t"
	  : "=m" (c)
	  :	"m" (a), "m" (b)
	  : /* clobbers */

	);

Adding a Simple Loop

In this block we employ a label, along with a counting register (ecx), to loop though the code as many times as defined by ct.

	// add a+b=c
	__asm__ __volatile__ (
		"movl %3, %%ecx \n\t" // loop counter
		".LOOP1_START:"
		"movl %1, %%eax \n\t"
		"movl %2, %%ebx \n\t"
		"addl %%eax, %%ebx \n\t"
		"movl %%ebx, %0 \n\t"
		"dec %%ecx \n\t"
		"jnz .LOOP1_START"
	  : "=m" (c)
	  :	"m" (a), "m" (b), "m" (ct)
	  : /* clobbers */
	);

To effectively create loops and other flow control statements in assembly we must learn the Label and Symbol syntax.

At the top of the hierarchy is the general idea of a Symbols, with Labels and Local Symbols being implementations of specific types of symbols. This distinction is important because when creating assembly code we will use Labels and Local Symbols to create flow control.

Labels can be thought of as a way to create blocks of code that perform a related function. Within these blocks we can further create Local Symbols to create jump point in our code.

Naming conventions are important, and are defined as follows:

Labels should start (but do not have to) with a .L and end with a colon. For example:

.L_ADD_LOOP:

To refer to a label, for example, using the jmp op-code, we would use:

jmp .L_ADD_LOOP

Local Symbols are used to create jump point within a Label Block, and follow the form:

N:

With N being the unique positional number of that symbol within the larger, named Label block.

For example, we could have:

LReverseShort:
	movl    %ecx,%edx		// copy length
	shrl	$2,%ecx			// #words
	jz	3f
1:
	subl	$4,%esi
	movl	(%esi),%eax
	subl	$4,%edi
	movl	%eax,(%edi)
	dec	%ecx
	jnz	1b
3:
	andl	$3,%edx			// bytes?
	jz	5f
4:
	dec	%esi
	movb	(%esi),%al
	dec	%edi
	movb	%al,(%edi)
	dec	%edx
	jnz	4b
5:
        movl    8(%ebp),%eax		// get return value (dst ptr) for memcpy/memmove
        popl    %edi
        popl    %esi
	popl	%ebp
        ret

Which as you can see, allows the developer to use various jump instructions to navigate the code block.

The important part here is to note how the tail end of each call to the Local Symbols uses a f or b. For example, we have:

jnz 4b and jz 5f.

Why? We follow the convention of writing the raw numeric value of the local symbol, followed by either an f or b, for forward or backward with respect to where that label sits in relation to the calling jump instruction.

Thus, we have two main ways to refer to our Labels and Local Symbols:

We refer to Labels with the full name, and Local Symbols (optionally), with with the number of the Symbol followed by the direction the called area lies in relation to the Local Symbol.

To help drive this point home, see if you can follow the flow if this code segment:

	int a = 1;
	int b = 2;
	int c = 0;
	int ct = 1;

	__asm__ __volatile__ (
	  "movl %3, %%ecx \n\t" // loop counter

	  "movl %1, %%eax \n\t" // holds 1
	  "movl %2, %%ebx \n\t" // holds 2

	  "jmp .Multiply \n\t"

	  ".L_ADD_LOOP:"
	  "addl %%eax, %%ebx \n\t"

	  "cmp $0x1, %%ecx \n\t"
	  "je 1f \n\t"

	  "dec %%ecx \n\t"
	  "jnz .L_ADD_LOOP \n\t"

	  "jmp .EXIT \n\t"	

	  "1:"
	  "addl $0x10, %%ebx \n\t"
	  "dec %%ecx \n\t"
	  "cmp $0x0, %%ecx \n\t"
	  "je .EXIT \n\t"
	  "jmp .L_ADD_LOOP \n\t"

	  ".Multiply:"
	  "movl $0x2, %%eax \n\t"
	  "mull %%ebx \n\t"
	  "cmp $0x0, %%ecx \n\t"
	  "jne .L_ADD_LOOP \n\t"

	  ".EXIT:"
	  "movl %%ebx, %0 \n\t"

	  : "=m" (c)
	  :	"m" (a), "m" (b), "m" (ct)
	  : /* clobbers */

	  );

	cout << c << endl;

An interesting item of note in this example is that our eax register, because of the requirement the mull instruction places on the loading of the source operand into eax, actually corrupts our final calculation because eax is then used in our addl call in .L_ADD_LOOP.

We could avoid this by using a different register for the add, say edx, or more appropriately, by using a different control flow. No matter, the important point is that if we’re not careful such subtle bugs can creep up. Such is the nature of assembly coding.

For good measure, here is the c++ version of the same function:

	// c++ version
	int a1 = 1;
	int b1 = 2;
	int c1 = 0;
	int ct1 = 1;

	c1 = 2 * b1;
	for(int i = ct; i != 0; --i){
		c1 = c1 + b1;
		if(i == 1){
			c1 = 16 + c1;
		}
	}

A good source of these items in action can be found here.

Useful Links

http://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html

Local Labels and Symbols

http://tigcc.ticalc.org/doc/gnuasm.html#SEC18

http://tigcc.ticalc.org/doc/gnuasm.html#SEC48

Basic Math

http://en.wikibooks.org/wiki/X86_Assembly/Arithmetic