As other have pointed out, a rather large burden is placed on site owners, especially those who run aggregate sites like Reddit and YouTube. My understanding is content/copyright owners can request that sites be removed from the US Internet if they’re found to harbor, support, or link to unlicensed content.

On the face of it that seems like it makes sense — if you’re “linking” to stolen goods via a sign in the front yard that says “car stereos in garage” it’s only natural that, even if it’s not your house, the operation gets shut down.

Right now the Internet is as free as it gets. Virtual goods and ideas exchange “hands” with impunity. Be it a movie or your thoughts on the movie, the Internet and our government doesn’t make a distinction. It’s all just bits floating in space.

As a society then we have a simple decision to make: do we embrace the current system or do we throw it away?

If the answer is “embrace” content providers and producers will simply have to live with the Internet as it stands right now, pirates and all.

Me: I like to pay for things:

By my rough count I own 144 episodes at $2.99 each making for $432 in TV shows. I also own about 860 songs, which at .99 cents means I’ve spent close to a thousand dollars on music — in the past 4 years.

This isn’t counting the Blurays and DVD I’ve bought over the years, the movie tickets, and of course rentals.

In short I’ve still spent what I consider a pretty exorbitant amount of money on entertainment, an amount that in some countries would be the equivalent of a years salary or more.

I pay because feel anything else would be wrong. In fact, my beloved South Park episodes: I can watch those for free anytime I want at South Park Studios and yet I don’t. Why? I like to own stuff I’ve paid for I guess.

And yet as sure as you have me and my way of buying, you have others who feel entitled to whatever they want, when they want, and always for free.

Thus, the other option is we do not embrace openness and instead move to lock the Internet down. In this world the Internet will become what I’ve been predicting all along: the cable TV model but on the Net. Instead of an open model we’ll buy “Internet Content Packages” (channels) that contain blocks of content (that still contain ads, just like cable TV does today!).

So we’ll get the tech package that contains access to Ars and Vox properties, the sport package that contains ESPN, and so on. I fear smaller sites like mine will be relegated to what in cable TV today is the public access channel. A slower, less reliable data-stream that at best means fewer visitors and less traffic. And make no mistake, we’ll do this willingly. An effort will be made to make the Internet as dangerous (legally speaking) as can be for aggregates (fan sites, the aft-mentioned YouTube etc), which means the “good” content will be the curated stuff we “conveniently” pay for in the content packages.

And so as I said above the bill is supposed to help software guys like me (content producers), but in the long term I believe its potential to hurt my business is a real threat.

Put another way I feel the greatest failing of SOPA is its inability to stop piracy in general while fundamentally changing how the Internet works for us working folk. Same as I have my content channel (iTunes) pirates have theirs, disabling websites because of links won’t stop this at all. Thus, SOPA passes and I, a law-abiding consumer, pays, while the pirates quite literally never do. They’re already comfortable using illicit channels, what’s to stop that in the future?

The future of the Internet is not in question – it’s what it will look like that is. I personally side with those that argue so long as content is easy to access and comes with reasonable rights people will pay.

I believe their are more than enough of us that fit into this mold, why should we be punished for those that do not?

So let’s talk why: When building something like FormBoss you often run into a situation where you need to do cross-browser compatibility coding. With Google Chrome now in the #2 spot and IE variants still holding around 50% of the market, this means at minimum 3 browser windows open at once.

This also means the developer tools for each browser being open and ready for some heavy lifting as code checks are run and bug squashed. Problem is, somehow the IE developer tools are not pinned to the main window like Chrome and Firefox, which means I’ve got an extra window clogging an already tight workspace.

I need to pin them to the IE window, but do not know how. As I’m already in IE I Bing:

ping ie developer tools to main window

I know, I know : ) My spelling’s terrible, but that’s the whole point.

I miss my error at first, and my search results comes back with this:

Not to be a mean guy, but that’s utterly worthless as a search result. It’s not even close that what I need or want.

So I copy/paste my #erroneous# query into Google and get this:

Eureka. Three completely relevant links right up top.

And that’s the thing. I gave Google what to some humans would be an nonsensical question and yet Google still comes through.

Now to be fair when we change our query to:

pin ie developer tools to main window

Bing comes through in fine fashion, even providing a Microsoft link at top:

So it’s not all bad, but I’ve come to expect that my search engine is smarter than me.

Google is, Bing is not.

A quick note as we get started: I’ve never coded in jQuery before, so my apologies ahead of time if I miss something obvious!

A Button Clicking Example

In this first example let’s see what a typical event handler for a button click looks like when following the design pattern of separating design from implementation:

Prototype:

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<script src="prototype.js" type="text/javascript"></script>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Prototype</title>

<script type="text/javascript">

// bind each button
document.observe('dom:loaded', function(){

	syntax = 'quick';

	if(syntax == 'quick'){
		// quick syntax
		$$('.dynamic-element-wrapper > INPUT').invoke('observe', 'click', buttonClicked);
	}

	if(syntax == 'verbose'){
		// verbose
		$$('.dynamic-element-wrapper > INPUT').map(function(t){
			t.observe('click', buttonClicked);
		});
	}

});

function buttonClicked(e){
	$('console').update(e.element().id + ' Clicked');
};

</script>

</head>

<body>

<div class="dynamic-element-wrapper">
	<input type="button" id="button-1" name="button-1" value="Button 1 - Click Me" />
    <input type="button" id="button-2" name="button-2" value="Button 2 - Click Me" />
</div>

<div style="clear:both;" id="console"></div>

</body>
</html>

jQuery

$(document).ready(function() {

	syntax = 'verbose';

	if(syntax == 'quick'){
		$('.dynamic-element-wrapper > INPUT').click(buttonClicked);
	}

	if(syntax == 'verbose'){
		$('.dynamic-element-wrapper > INPUT').bind('click', buttonClicked);
	}

});

function buttonClicked(e){
	$('#console').html(e.target.id + ' Clicked');
};

Please note that as the HTML doesn’t change the jQuery code sample is limited to just the SCRIPT block.

So a couple items of note:

As a long-time Prototype guy jQuery’s automatic array return when using $() sits in stark contrast to Prototype’s use of $$(). I have to admit I’m torn on this one, as in a way the jQuery way makes more sense. The selector and knowledge of the document, combined with the function our event listeners call should be all that’s needed from a selector. In practical terms then differentiating between $() and $$() is admittedly more work without a huge gain in readability.

That aside, in Prototype we also get a bit more syntax heavy having to resort to .invoke() or .map(), whereas jQuery’s .click() makes short work of the job — providing a quick and concise way to perform such a common task is a huge win.

On the event processing side our buttonClicked() call is very similar. Once again jQuery’s love of shortcuts impresses me. .html() for getting and setting text is quite nice, and from what I can see, is duplicated via .val() of the form field side.

The this Reference Binding Issue

So here comes the main event. The example above in terms of design pattern is going to be just fine for most projects. Our design code is free from implementation code and when we click the element we get the info we need — everything just works.

But what happens when we follow a slightly more complex design pattern? Many times allowing UI implementation code to hover in the global script scope is perfectly acceptable. But their are times, either by virtue of space or code cleanliness, that wrapping our event code into objects is desirable or required.

An example of doing so in Prototype looks like this:

var buttonFactory = {

	_limit: 10,
	_secure: true,

	bindObjects: function(){
		$$('.dynamic-element-wrapper > INPUT').invoke('observe', 'click', this.buttonClicked);
	},

	buttonClicked: function(e){
		$('console').update(e.element().id + ' was clicked. ' + 'The limit value = ' + this._limit);
	}

};

// bind each button
document.observe('dom:loaded', function(){
	buttonFactory.bindObjects();

});

As you can see instead of binding our event listeners in the global scope we issue a simple call to buttonFactory.bindObjects(). If this was a much larger project it’s not hard to see why this would be a great way to work. Our document.observe code would simply be a series of calls to code that initialize other parts of the UI. The initialization code would then perform all sorts of long-winded tasks while leaving our document.observe call far from the down and dirty details.

But all’s not well. When coding this way — if not just to save space but to also create more complex event handling logic — we would expect that our buttonClicked() code would handle the reference to this._limit but it doesn’t. We’ll get this when we click a button:

button-2 Was Clicked. The limit value = undefined

That’s not at all what we want. You’d think this._limt should return with a value of 10 but it doesn’t. Why?

The problem is when we bound our event listener the this reference wasn’t part of the process. This means when we trigger a click event the this refers to the event object, not the buttons original parent object (buttonFactory).

Prototype has a solution for this via a call to bindAsEventListener(). bindAsEventListener() takes the object we wish to use as the this reference when invoked. To implement this fix in our example code we’d change our bindObjects() call to:

//
//
$$('.dynamic-element-wrapper > INPUT').invoke('observe', 'click', this.buttonClicked.bindAsEventListener(this));
//
//

Now when we run our code it returns the following when a button’s clicked:

button-1 was clicked. The limit value = 10

That’s more like it. Sure this is a synthetic example, but it’s absolutely essential to creating portable code.

Portable is the key word here, as yes, we could just do this to our buttonClicked() call:

//
//
$('console').update(e.element().id + ' was clicked. ' + 'The limit value = ' + buttonFactory._limit);
//
//

But this would be in bad form in my opinion, as now we’re making all sorts of assumptions of how this object will be consumed.

So the point is Prototype gives us a sleek, consistent way of holding objects together in event-driven code.

How does jQuery solve this issue?

jQuery

var buttonFactory = {

	_limit: 10,
	_secure: true,

	bindObjects: function(){
		$('.dynamic-element-wrapper > INPUT').bind('click', this, this.buttonClicked);
	},

	buttonClicked: function(e){
		$('#console').html(e.target.id + ' Clicked' + 'The limit value = ' + e.data._limit);
	}

};

$(document).ready(function() {
	buttonFactory.bindObjects();
});

The method signature of .bind() as we’re using it is:

.bind( eventType [, eventData], handler(eventObject) )

Thus, my solution is to simply pass our object reference as the eventData argument. We then pick up on the value via the event.data object in buttonClicked() and away we go.

Seems simple enough but again I do not have…well…any jQuery experience, so I would not be surprised is this is wrong in terms of implementation. What I can say though is it works just fine, and once again the syntax is pretty slick. For Prototype adding bindAsEventListener() is every bit as cumbersome as document.getElementById() if you catch my drift…

I say it works fine as for my part I did issue a test call of:

e.data._limit = 20;

under the #console update call and sure enough, the buttonFactory object was updated accordingly. From this we can tell this is passed as a reference, which means for my money this is an acceptable, working solution.

For the sake of completeness it appears we can also use .proxy().

When used our code changes to:

var buttonFactory = {

	_limit: 10,
	_secure: true,

	bindObjects: function(){
		var bProxy = $.proxy(this.buttonClicked, this);
		$('.dynamic-element-wrapper > INPUT').bind('click', bProxy);

	},

	buttonClicked: function(e){
		$('#console').html(e.target.id + ' Clicked' + 'The limit value = ' + this._limit);
		e.data._limit = 20;
	}

};

$(document).ready(function() {
	buttonFactory.bindObjects();
});

I’m not entirely sold on this method as now we’re once again in the mode of hand-wrangling specific behavior from our objects. It’s nice to know it can be done, however.

Conclusion

The title of this post has a vs. in it so who wins this round? Somewhat unsurprisingly I think it’s a draw, though if I had to pick I’d say jQuery. jQuery’s ability to pass this as an event parameter and it still works is fine by me and easier to code, and considering we can just as easily wrap this into a larger structure is better still.

Of course I’m ready and willing to have any and all ignorance corrected, so if you’ve seen something that’s not right with my solution(s) please let me know!

The bottom line is jQuery has some real time savers in its bucket of tricks; the fact that $() returns arrays is worth the price of admission alone.

Color me continually intrigued.

Prototype JS is…well…it’s another JavaScript library. But it’s the library I know and love. It’s what drives FormBoss and while not as exciting or popular as jQuery, I swear its day will come (again). Their are many reasons for this belief, but the first is due to what I believe is a fundamental misconception about jQuery vs. other libraries — its payload size.

In this first of a multi-part series we’ll take a look at the physical delivery size of each library. In the next post well dive into practical programming. First though, a bit of background.

Background

As a professional Web Developer I had a decision to make in mid-2008 as to what JavaScript library I would use for FormBoss. At that time I was looking for a library that fixed JavaScript’s core issues and had good AJAX support. (Little did I know that while AJAX support was important, far more relevant would be DOM handling and event binding, but more on that latter).

I checked a few libraries starting with Yahoo’s YUI and Dojo. Forget it. Far more complex than I needed, they seemed to make even simple tasks difficult (disclaimer: I haven’t looked into these libraries for some time now, they may be better, or perhaps my initial impressions were wrong!).

What I really wanted was a library like PHP. Provide a solid foundation that was not unto itself a framework. All I wanted was something that extends and smooths over vanilla JavaScript but also provides glue functions for doing more advanced stuff.

It wasn’t long before I narrowed my decision down to Prototype and jQuery (at versions 1.6.0.3 and at 1.2.6., respectively). Both seemed strong – jQuery on the count of its small size and simplicity, Prototype because of its more robust feature-set and AJAX support (again though DOM traversal and event binding played no part in this process, but would become crucial later on).

In the end I went with Prototype for a simple reason — it seemed more “complete” in terms of a programming construct. jQuery, always bragging about how small it was seemed almost too small, too limiting. I wasn’t looking for flash but utility, and Prototype fit the mold. For the most part I haven’t regretted this decision one bit–in the intervening years Prototype has been wonderful to me and always delights. In places where it needed improvement like upgrading to the Selectors API it’s done so in a reasonable time-frame, and for just about everything else, such as animation and event binding, it’s never let me down.

But always in the back of my mind has been jQuery, a nagging sense that I’m missing something. Why has it gotten so popular and does this mean I should be using it too?

Modern jQuery – The Popular Place To be

The base answer to our question of why so popular can be found right at the top of the jQuery home page:

Compare that with Prototype:

Ugh. It’s no wonder jQuery has more hype — seriously — by download link alone jQuery simply looks cooler and more advanced.

But it’s not just that…well not entirely : )

When you look at the jQuery download options you see that magic value: 31KB.

How brilliant can they be? Think about all of those large newspaper, e-commerce, and other highly trafficked sites. Bandwidth is money, and when it comes time to tell your boss you need to add more weight to a site it had better be as little as possible and worth it. More importantly, even as recently as 2009 broadband adoption in America was still only 67%. Sure that number continuies to rise but now consider our rising mobile Internet use. Truly two steps forward and one step back.

The point is size still matters, and Prototype’s stubborn lack of a minified version and frankly boring site and download page mean of course jQuery’s flashy site will draw more users in, both from a aesthetic and practical standpoint.

In Reality…

But of course we’re developers so we go deeper than that, and in preparation for this post I was–admittedly–floored by a simple test:

Had you asked me just five minutes ago I would have said “Of course jQuery is smaller, everyone knows that!”.

As of 1.6.4 jQuery has actually grown larger than Prototype in terms of raw source , 6084 lines vs. 9048 (and almost twice as large in terms of raw source code size in KB). More importantly though is while the minified version of Prototype is still larger, the minified + gzipped versions are identical. More importantly still: the minified jQuery is not 32KB but ~90KB. This is important because the jQuery website makes you feel like you’ll be getting all of that functionality for 32kb, but only if your server is gzipping content. The simple fact is almost all shared hosting plans won’t be doing do, which means your 32KB is actually closer to 90KB. And no, it’s not just the smaller plans, it happens with the big boys as well:

http://shop.nordstrom.com/

Run a Firebug NET session on that site and you’ll see what we’re about to cover below. They use a minified but not gzipped version of jQuery (version 1.5.1), which means payload size is 51KB.

In that light it’s instructive to see what happens with no server size gzipping with the latest versions of each library:

Prototype.js

jQuery

Here we can can very little server time spent on the requests, but with jQuery’s minified version definitely being smaller. Again though, the 32KB shown on the jQuery website, while true as it clearly states “Gzipped”, is not what we get without the server actually gzipping, as we would expect.

However, as soon as we enabling server size gzipping [via Apache’s mod_deflate using:

LoadModule deflate_module modules/mod_deflate.so

we get:

Prototype

jQuery

The minified version of Prototype is made via this minifier, and as we can see is essentially the same size as jQuery.

However, note that the non-minified version of Prototype is only ~5KB larger, and just as importantly, only take ~3 ms of extra server processing time (remember that gzipping in Apache is not free!). In other words we’re about even in that we do not need to create a minified version of Prototype and we’ll still deliver, in terms of processor load and payload size, the same thing.

In Conclusion

In this first post we’ve found that when properly delivered both libraries are essentially the same size. I would actually give Prototype the win however, as its smaller uncompressed size means we can simply plop the uncompressed version in to our delivery or ‘live’ section, and then in testing, instead of nonsensical minified error messages get actual line numbers we can refer back to. True this shouldn’t happen often, (it should be our code with the error, not Prototype or jQuery), but still handy in the larger scheme.

In non-gzipped environments however it’s clear that jQuery gets the win. This will be important for shared hosting plans of anywhere else we do not have access to server-side gzipping.

It’s also noteworthy that in order for Prototype to gain some of the more advanced animation framework goodness that (I assume) jQuery already has in the base library, we’d need to add Script.aculo.us, which would of course make the total payload larger.

Those considerations aside, what’s fascinating to me is that the perception of jQuery being “the” light-weight library. In the past 3 years I firmly believe this has been the primary driving force behind its mass adoption and popularity. I don’t know about you but I’ve been specifically told by bosses to not include Prototype because of its size — but at the same time given the go-ahead for jQuery. I’ve also made similar decisions for FormBoss based on the same reasoning (though not to instead use jQuery, just vanilla JavaScript).

Now that this is no longer true for the right environments (and truth be told, this has always been the case in the right environments), I think the time’s right for Prototype to take back some of the users it rightly deserves.

The irony of course is back in 2008 jQuery 1.4.2 was in fact smaller: 3549 lines of code vs. Prototype 1.6.0.1′s 4321. But success comes with a price, and jQuery’s potential bloat can only be a good thing for users looking to ‘trim’. How wonderful : )

Please understand though: I have nothing against jQuery, I simply feel that Prototype deserves more recognition and is in many ways the superior library for some specific types of work.

This will only happen though, and I cannot be more serious, if the Prototype folks get off their collective butts and do the same thing as jQuery –  advertise, as prominently as you can, the fact that when minified and gzipped your library is the same size.

Links

http://thinkrelevance.com/blog/2009/01/12/why-i-still-prefer-prototype-to-jquery.html

That’s not a typo, it’s a real feature of MySQL, and one I just learned about. The basic idea is to SELECT rows from one or more tables to populate another. It does this with a temp table, and for the most part respects things like auto-increment fields and foreign key restraints.

It’s a neat little trick that should help with those times where we’ve been populating temp tables with incremental updates, and now need to create one final INSERT into a master table.

#include <stdio.h>
#include <stdlib.h>

int test_noRef(int value)
{
	return value + 1;
}

int test_ref(int &value){
	return value += 1;
}

int main(int argc, char *argv[]){

	int t = 1;

	for(int i = 0; i < 100000000; i++){
		//t = test_noRef(t);
		test_ref(t);
	}

	// final result = n loops + 1
	printf("%d\n", t);

}

This code is purpose built to thwart basic compiler optimizations. In particular the printf() call at the end prevents the compilers from simply skipping the entire function call block.

While simplistic, the basic idea is to see what optimizations each compiler performs and at what optimization level. Note that the LLVM/Clang suite can easily be installed on Fedora 15 by searching for LLVM. You’ll want to install LLVM and Clang:

No Optimizations

We start with a basic compile with no optimizations; in each case we’ll focus on test_ref() and test_noRef(). Please note I’ve removed the stack management code from these listings as they differ very little by compiler.

g++ main.cpp -S
clang main.cpp -S

[GCC]

_Z10test_noRefi:
.LFB0:
	movl	%edi, -4(%rbp)
	movl	-4(%rbp), %eax
	addl	$1, %eax
	popq	%rbp
	.cfi_def_cfa 7, 8
	ret

The noRef call in GCC moves our parameter to eax, adds 1, and cleans up the stack.

[LLVM]

_Z10test_noRefi:
.Leh_func_begin0:
	pushq	%rbp
.Ltmp0:
	movq	%rsp, %rbp
.Ltmp1:
	movl	%edi, -4(%rbp)
	movl	-4(%rbp), %edi
	addl	$1, %edi
	movl	%edi, %eax
	popq	%rbp
	ret

The LLMV code for the noRef() function is quite similar. The main difference is LLVM uses edi for the addition step, then pushes the result to eax before returning.

[GCC]

_Z8test_refRi:
.LFB1:
	movq	%rdi, -8(%rbp)
	movq	-8(%rbp), %rax
	movl	(%rax), %eax
	leal	1(%rax), %edx
	movq	-8(%rbp), %rax
	movl	%edx, (%rax)
	movq	-8(%rbp), %rax
	movl	(%rax), %eax
	popq	%rbp
	.cfi_def_cfa 7, 8
	ret

The reference using version performs the same basic logic, but instead of using movl we issue movq, which means our parameters are twice as large. That inefficiency aside we do use an optimization for adding the 1 to our input argument via the leal instruction.

[LLVM]

_Z8test_refRi:
.Leh_func_begin1:
	pushq	%rbp
.Ltmp3:
	movq	%rsp, %rbp
.Ltmp4:
	movq	%rdi, -8(%rbp)
	movq	-8(%rbp), %rdi
	movl	(%rdi), %eax
	addl	$1, %eax
	movl	%eax, (%rdi)
	popq	%rbp
	ret

LLVM’s code is a bit cleaner, focusing on pulling the value from the stack, performing the addl, and retuning the result.

[GCC]

main:
.L5:
	leaq	-8(%rbp), %rax
	movq	%rax, %rdi
	call	_Z8test_refRi
	addl	$1, -4(%rbp)
.L4:
	cmpl	$99999999, -4(%rbp)
	setle	%al
	testb	%al, %al
	jne	.L5
	movl	-8(%rbp), %eax
	movl	%eax, %esi
	movl	$.LC0, %edi
	movl	$0, %eax
	call	printf
	movl	$0, %eax
	leave

As with the function calls I’ve removed the activation record code. LLVM’s main function definition is essentially the same, so we won’t cover it here.

A quick benchmark between the two shows what we would expect from the extra fluff in GCC’s implementation:

GCC
real 0m0.407s
user 0m0.405s
sys 0m0.002s

LLVM
real 0m0.396s
user 0m0.394s
sys 0m0.000s

Level 1 Optimizations

It’s a bit unfair to draw comparisons without some optimizations turned on, so let’s do so now:

g++ main.cpp -O1 -S
clang main.cpp -O1 -S

[GCC]

_Z10test_noRefi:
.LFB19:
	leal	1(%rdi), %eax
	ret

Already we can see a huge difference in GCC’s output, which whittles this call down to a simple Load Effective Address and return.

[LLVM]

_Z10test_noRefi:
	leal	1(%rdi), %eax
	popq	%rbp
	ret

As expected LLVM performs the same lea optimization as GCC.

[GCC]

_Z8test_refRi:
.LFB20:
	movl	(%rdi), %eax
	addl	$1, %eax
	movl	%eax, (%rdi)
	ret

The reference-using version still has to contend with the indirection call via movl ($rdi), %eax, but still a good improvement.

[LLVM]

.Ltmp4:
	movl	(%rdi), %eax
	incl	%eax
	movl	%eax, (%rdi)
	popq	%rbp
	ret

LLVM is using incl to perform our addition call, GCC uses addl. A quick check on Google shows Intel recommends the GCC way.

[GCC]

	movl	$1, 12(%rsp)
	movl	$100000000, %ebx
.L4:
	leaq	12(%rsp), %rdi
	call	_Z8test_refRi
	subl	$1, %ebx
	jne	.L4

The main() function has been reduced to the basics, counting down from our loop initialization value and calling the test_ref() function.

[LLVM]

.Ltmp8:
	movl	$1, -20(%rbp)
	movl	$100000000, %ebx
	leaq	-20(%rbp), %r14
	.align	16, 0x90
.LBB2_1:
	movq	%r14, %rdi
	callq	_Z8test_refRi
	decl	%ebx
	jne	.LBB2_1

LLVM’s main function loop has also been optimized, though again we’re using decl instead of subl.

Although the two compilers perform similar optimizations here, lets examine how LLVM’s structured our reference value for the loop.

Right before the main loop it calls:

leaq    -20(%rbp), %r14

Which maps the address at that base-pointer location to r14. In essence its created an alias. Why it’s done this make sense when we see the rest of the loop.

The first call of every iteration is:

movq	%r14, %rdi

So we pass the address in %r14 to rdi and in the function call we have:

movl    (%rdi), %eax

Which essentially means dereference the value in rdi and push to eax for the incl call. That gets us our added value but it’s the next call where the magic happens, where we tie the whole thing together:

movl    %eax, (%rdi)

Now we’ve moved the value of eax to the address location in rdi. Since that address and the one in %r14 are one in the same, the next call in the main loop to:

movq    %r14, %rdi

Means that his freshly increased value from the function call being pointed to by %r14 is once again pushed to rdi, which in turn is dereferenced in the function call and incremented.

That’s a pretty neat bit of optimization so let’s see how it pans out in terms of performance:

GCC
real 0m0.165s
user 0m0.163s
sys 0m0.001s

LLVM
real 0m0.163s
user 0m0.161s
sys 0m0.001s

It’s much the same, as GCC performs a similar trick using %rdi and the stack.

Level 2 Optimizations

Finally, let’s enable the highest level of optimization:

g++ main.cpp -O2 -S
clang main.cpp -O2 -S

[GCC]

main:
.LFB21:
	.cfi_startproc
	subq	$8, %rsp
	.cfi_def_cfa_offset 16
	movl	$100000001, %esi
	movl	$.LC0, %edi
	xorl	%eax, %eax
	call	printf
	xorl	%eax, %eax
	addq	$8, %rsp
	.cfi_def_cfa_offset 8
	ret
	.cfi_endproc

[LLVM]

.Ltmp7:
	movl	$.L.str, %edi
	movl	$100000001, %esi
	xorb	%al, %al
	callq	printf
	xorl	%eax, %eax
	popq	%rbp
	ret

No need to show anything other than main() because it doesn’t matter.

Both LLVM and GCC have done what we’d expect with such a silly bit of code. Both compilers analyzed the code and came to the same conclusion: the final output value to prinf() can be calculated directly within the compiler, so no need to call anything. Just push 100000001 to %esi for printf() and call it a day : )

In Conclusion

While the code tested was simple, it’s interesting to see how the actual assembly differs. It’s clear that this simple test LLVM produces faster binaries, though not by much.

The other important bit is at least on Fedora, installing and using GCC and LLVM couldn’t be simpler.

Links

Installing and Using LLVM

LLVM Online Compiler Demo

Improved: Improved Zend Engine memory usage and performance

Sounds good to me, and after my previous post on references I thought I’d run the same test with this new build (all times are in seconds):

I’d say that’s a decent improvement, though one that follows the same trend from the first post of using references with objects is slower than not using them.

As one final test, here’s the result of loading the FormBoss.net homepage several times in succession:

So about the same when performing standard page interpreting, if not a touch slower.

Links

http://qa.php.net/

What are references?

At the most simple level a reference acts like a pointer (a memory address) of a variable in memory. To understand what this means a short example is in order. Let’s take this block of code:

<?php

  $test = 1;

  function noReference($value){

    $value = 2;

  }

  noReference($test);

  echo $test; // 1

?>

Now compare with a reference-using version:

<?php

  $test = 1;

  function usingReference(&$value){

    $value = 2;

  }

  usingReference($test);

  echo $test; // 2

?>

If we run both functions on the same page the output will be:

1 Now compare with a reference using version: 2

Note the only difference between the two versions is the addition of the ampersand to the function parameter $value.

The ampersand is the reference operator, and its effect is to tell the function, in English:

“Do not create a local copy of this parameter, use the one I’m giving to you and make any and all changes to that guy.”

In other words, when not using a reference passing a parameter to a function means creating a local copy of that parameter, with any action taking place on that copy instead of the original item/value passed in. That’s why in the first code item (noReference), setting $test = 2 had no effect on the variable outside of the function call.

So why use references?

As mentioned above references are an integral part of C languages, and this is for two good reasons, among others: performance and function return limitations.

Performance because if you think about it, making a copy of a local variable takes time and uses more space when compared to just changing the original. References can thus be handy in situations where performance is key.

The reason why is In C using a reference means the compiler skips that item for inclusion in the activation record (e.g., the stack). For simple data types like ints and chars this doesn’t make a huge difference, but may if passing large arrays, objects, and structures, or if passing lots of variables such that the activation record grows very large (and hence the offsets needed to access the activation record items grows).

Thing is, In PHP all variables (generally speaking), become zvals* which are themselves pointers to items in a hash table. Thus, the performance gains from using references will be a touch less impaction. but the gains are real in some cases, though not all as we’ll see shortly.

As a quick test I ran the following code in command line php (CLI) using the Linux command of: time php test.php:

<?php

function noRef($v){
	$v = 2;
}

function ref(&$v){
	$v = 2;
}

$test = 0;

for($i = 0; $i < 1000000; $i++){

	//noRef($test);
	ref($test);

}

?>

The results:

No References:
0.461

References:
0.362

So a difference for sure, though truth be told, not a big one. This is explained by the fact that standard variables are small to begin with, the 32-bits to hold an int is the same as the 32-bits needed to hold that items address. In fact, references add an extra step of indirection to the process, in theory.

Again though, as PHP wraps all variables into zvals* the standard C theory on references doesn’t hold. This becomes abundantly clear when we change our code to:

<?php

function noRef($v){
	$v = 2;
}

function ref(&$v){
	$v[7] = 2;
}

$test = range(1,100000);

for($i = 0; $i < 1000000; $i++){

	noRef($test);
	//ref($test);
}

?>

No References:
0.517

References:
0.485

Now we’re passing a much larger object to our function, but the time difference when running the code has been mostly erased. This is understood in terms of all variables are basically references in PHP to start with (when it comes to accessing them as we always need to use indirection to access their values), so using them will not bring any appreciable speedups for larger objects.

In fact, when using objects references are actually slower. So this test code:

<?php

class tester{

	private $test;

	public function setValue($value){
		$test = $value;
	}
}

function noRef($testObject, $value){
	$testObject->setValue($value);
}

function ref(&$testObject, &$value){
	$testObject->setValue($value);
}

$testObject = new tester();
$value = 2;

for($i = 0; $i < 1000000; $i++){

	//noRef($testObject, $value);
	ref($testObject, $value);
}

?>

Produces:

No References:
0.940

References:
1.25

Performance considerations aside, the second reason we use references comes form the fact that in C functions are limited to one return value. This means if we wish to write code that affects a longer lasting value that what’s possible in function scope, passing references is a handy way of doing so. Again though this is not really an issue for most users, as we can always return arrays/hashes that contain all the values we need. This goes for PHP and C of course.

Conclusions

The simple fact is references, while nice to have, are simply not a feature most of us need or will in fact want to use. As many PHP developers will not be versed in the use of references, including them in your own code can actually make portability and maintainability an issue.

I personally never use references if I can avoid it, in the 30,000+ lines of FormBoss codebase I use exactly one reference, and only because it was the most elegant solution for that particular task. I could have gotten away with not using a reference however, and perhaps I should have.

Perhaps the best way to put it is for raw variables like int and floats it makes sense to use references from a performance standpoint, though only if you’re sure they wont throw off other who may want to contribute to your code. For larger object types I would avoid references if possible.

Links

http://php.net/manual/en/language.references.pass.php

Their is one area where PHP falls flat however, and that’s intensive data processing. As PHP internally stores all user-space variables in a hash-table, algorithms that require heavy manipulation and lookups of these items will always be slow.

Of course PHP will also fall flat for us if we need some capabilities that no extension provides.

In this post we’ll learn about the solution to both of these issues: building custom PHP extensions. The reason why this post even exists is because of an excellent PHP tool called: ext_skel, as with it we can quickly create our own PHP extensions that would otherwise take many laborious hours of trial and error.

ext_skel

So what’s ext_skel? ext_skel is a tool located in a source download of PHP (snaps or regular downloads). Its job is to create the basic skeleton a PHP extension needs to run. As this post will show, provided we supply a header file with function definitions, for basic tasks ext_skel will create an almost complete project. This is a good thing as the single greatest barrier when trying to create our first extension is knowing the rather complex layout of the needed files.

Setup & Build Environment

Before we can get started we’ll need PHP sources (snaps or regular downloads).

We’ll also need GCC, which can be had on the command line with:

Fedora

yum install gcc

Ubuntu

apt-get install gcc

We also need a program called autoconf, which while supplied with most distributions, is for the most recent Fedora and Ubuntu too new.

Thus, we need to grab an older version from:

http://ftp.gnu.org/gnu/autoconf/

Look for the file:

autoconf-2.13.tar.gz

To avoid overwriting any existing installation of autoconf we’ll use a prefix to install this guy to a custom location, then export an environment variable to tell PHP where to find it.

Before we do that though, let’s create a common build environment.

On my Fedora 15 instance I have a folder called:

/home/matthew/Documents/server

Which is the master location for our work environment. Any sources for items, including the PHP and autoconf downloads, are placed in:

/home/matthew/Documents/server/src

When we build our software we install the results once level up using the ./configure PREFIX command, back in:

/home/matthew/Documents/server

Visually then, we have the source folder which contains the PHP and autoconf sources:

And once level up from that, the location where all compiled results will be saved too:

If you haven’t used the common ./configure : make : make install chain before, let’s use our freshly downloaded autoconf as an example.

Open a terminal and navigate to the directory where the autoconf source is:

The key here is we need to tell the configure step where we want the final output placed. We do so by including the argument: –prefix as in:

./configure --prefix=/home/matthew/Documents/server/autoconf

Of course your path will be different in terms of user name (where mine says matthew), but so long as we get that path right, we can issue, in order:

./configure --prefix=/home/matthew/Documents/server/autoconf

make

make install

This will, as you can check for yourself, install the autoconf binaries in:

/home/matthew/Documents/server/autoconf

That’s all there is too it.

While we’re at it, we also need to compile PHP.

PHP compiles with almost the same ease, but to make sure extension building goes as smoothly as possible, we need a few more options defined.

For my instance, and yours as well, cd to the PHP source directory we downloaded previously and issue:

./configure --enable-debug --enable-cli --disable-xml --without-mysql --disable-sqlite --disable-cgi --disable-pear --prefix=/home/matthew/Documents/server/php-snap

The important part, again, is the –prefix path. Make sure to change yours to match your user name. To that end you’ll notice my path is:

/home/matthew/Documents/server/php-snap

I’ll be using this path in the code examples below, and as you may suspect, the snap comes from the fact that I’ve downloaded a PHP snapshot. Non-snapshot downloads should work just as well, though please note I had an issue with running ./buildconf on the non-snaps version. This may sound like nonsense right now, so if you’re interested in getting up and running as quickly as possible, I would suggest downloading the snaps version to match my environment exactly.

Just as before, after running the configure command issue make, and make install.

The final result is we now have a folder that should have the full PHP installation:

As the final step, we need to issue a shell command to tell the PHP build process where autoconf is. Again, this is necessary because the autoconf version we’ll have in the most recent distributions like Fedora 15 and Ubuntu 11.04 is too new for PHP. The command is simple however, and looks like:

export PHP_AUTOCONF=/home/matthew/Documents/server/autoconf/bin/autoconf

As always, please be sure to change the path to match your actual file system.

We can  check to see if this path took by running:

env

With all of that done we should be ready to start building our extension!

Creating A Header File

As mentioned in the introduction, ext_skel is an incredibly handy tool for many reasons, but my favorite is that if we supply a .h header file ext_skel will go ahead and create all of the most common glue code a c function needs to run in the PHP userspace.

This is, again, a good thing, as you’re probably not going to guess that a function call for a PHP extension may start with:

int argc = ZEND_NUM_ARGS();
long n;

if (zend_parse_parameters(argc TSRMLS_CC, "l", n) == FAILURE) // ampersand before n omitted because of WordPress

return;

Of course it needs to be said if we’re truly serious about PHP extension development we’ll need to learn this stuff regardless, and more importantly, our example today is simple in that we’re only going to deal with simple parameters and returns. Anything more complex will require that we truly dive in and learn the down and dirty syntax ext_skel is kindly creating for us.

Regardless, we want to make sure we pass the ext_skel program a parameter which is the location of a very simple .h file.

The file, called fannkuch.h, has one line:

int fannkuch(int n)

This is a typical C method signature, and is all ext_skel needs to create our function code for a PHP extension. Thus, create this file on your desktop, Documents folder, or wherever you like — we need it for the next step.

Running ext_skel

So at this point we have a PHP installation and source directory, autoconf built and installed, a environment variable set, and a .h file with a simple method signature.

The next steps come down, basically, to us running a few simple commands to build our extension and compile it into the PHP core. Thus, it’s of utmost importance that we know where we should be at all times.

For the first step then, we need to navigate to our snaps download ext directory:

cd /home/matthew/Documents/server/src/php5.3-201107100630/ext/

Again, your path will different, and in this case both in name and PHP version number.

Once in the ext directory we’ll issue the magic command:

./ext_skel --extname=fannkuch  --proto=/home/matthew/Desktop/fannkuch.h

This command does several things. First, it creates a folder in our ext directory called fannkuch. The –proto bit is where we point to our .h file, created previously. Taking that file then, it creates several files in the newly created fannkuch directory, everything in fact, for a working extension…almost.

As you’ll notice after running the previous command, ext_skel was nice enough to print some instructions for us:

The key here is we need to integrate our new extension into PHP. As it stands this will not happen until we edit the config.m4 file (the # 2 step above), also created by ext_skel:

Editing config.m4

So what changes need to be made? The config.m4 file, a little too long to be shown here, is used by the main PHP build process. Specifically, as PHP is being built via ./configure it checks all directories in the ext directory resolved by the ./buildconf step (which we’ll do shortly, and is also, you’ll notice, step 3 on the list). If a match is found it attempts to parse the m4 file and uses it to produce the final build file, which is in turn how the extension is actually compiled.

ext_skel cannot, for what it’s worth, do all of the work for us, simply because in the world of PHP extensions their are so many variables: threaded or not, module or shared object, external dependencies or not, and so on.

Thus, the m4 file created by ext_skel contains all of the code we need to produce a working extension, it just comments most of it out and lets us decide what to expose to ./buildconf.

The good news is the changes we need to make are quite minimal. In fact, we just need to uncomment three lines.

At the top of the file we start with:

Uncomment lines 16 and 18 for:

Then, at the bottom of the file we need to uncomment just one line:

Line 60 becomes:

Make sure to save the file when done–we’re almost ready to get our first extension up and running.

Building Our Extension

Right–so now we have a properly configured m4 file, one that will be picked up by ./buildconf in this next step, read, and then presented to the ./configure step.

First, we need to bounce up one directory so that we’re in the main PHP source directory, e.g.:

/home/matthew/Documents/server/src/php5.3-201107100630

This is the big one. Provided we’re installed autoconf and supplied the proper PHP_AUTOCONF environment variable, we should be able to run:

./buildconf

If all goes well the command line will spit out a few lines of warnings and notices–none of these are an issue.

If not, and this was eluded to earlier, I also had issues with ‘production’ versions of PHP not taking too kindly to ./buildconf. Their is a –force flag that can be set which should override these issues, though I did not have any luck with that either. At this point, as we’re just getting started, it is well worth using a debug (snaps) version of PHP, as it’s much easier to debug anyway.

One other quick point: running ./buildconf is what processes any and all directives we’ve set in the extensions config.m4 file. Thus, if we change the config.m4 file, we must rerun ./buildconf!

Provided the previous ./buildconf step worked, we can now instruct the PHP build process to include our new extension with:

./configure --enable-fannkuch -prefix=/home/matthew/Documents/server/php-snap

Note that we’re saying –enable-fannkuch: The use of –enable as opposed to –with means the extension doesn’t have any external dependencies.

The prefix bit, like it has been in all cases, means that we have PHP installed to a custom location, and that’s where all files need to be delivered when we run make install. Thus, it should be said that the directory we’re in right now is just the source code version of our extension, when we run make install we’re sending our extension to the php-snap folder. Yes, it’s a real, official extension we’re building here: )

Finally, we now need to run:

make

make install

Running The Extension

At this point we’re ready to try out our new extension.

As you may have noticed in step 6 after running ext_skel, we can simply run:

../../php-snap/bin/php ext/fannkuch/fannkuch.php

That is, we bounce up two directories to hit our actual php binary, then dive into the ext/fannkuch directory to run the test file ext_skel was nice enough to create for us.

If all goes well we’ll see:

At this point we can also test the actual function call we defined, fannkuch(int).

To do so I created a new file called test.php in the raw extension source folder:

…with the following contents:

<?php
 echo fannkuch(10);
?>

We can now call this guy with:

../../php-snap/bin/php ext/fannkuch/test.php

To which we’ll get:

Warning: fannkuch: not yet implemented in /home/matthew/Documents/server/src/php5.3-201107100630/ext/fannkuch/test.php on line 2

That’s no fun, but fear not, as next we’ll add to this extension to do something interesting.

Adding To The Extension

A PHP extension is far more interesting when it does something useful. While we’re not going to get too complex here, we will implement a function that accepts a parameter and returns a value.

To do so we need to modify the part of the fannkuch.c file ext_skel created for us which defines the userspace PHP function, fannkuch().

The good news is ext_skel has done just about all of the hard work for us. By hard work I mean creating PHP extensions from scratch means learning a whole lot of C Macros and Zend calling conventions, and if you’re like me, limitations of C as opposed to C++.

The first step then is to open our fannkuch.c file and scroll down to the line:

PHP_FUNCTION(fannkuch)

This macro, again, created for us by ext_skel, is how user-space functions are defined by PHP. That is, in a PHP source file, provided this extension has been loaded, I can call fannkuch() and our extension function will be called. As you’ll notice, the error/warning we got when runnng test.php comes from the default implementation thereof:

PHP_FUNCTION(fannkuch)
{
 int argc = ZEND_NUM_ARGS();
 long n;

 if (zend_parse_parameters(argc TSRMLS_CC, "l", &n) == FAILURE)
 return;

 php_error(E_WARNING, "fannkuch: not yet implemented");
}
/* }}} */

Thus, to “do something interesting” means we need to remove that php_error line and implement our own C code.

Custom Code Implementation

Again, we’re not going to get too deep here, but implementing code is only possible if we know a few important points about PHP extension writing. Most important of these topics for us: input parameters and returning values.

Input Parameters

Passing parameters in Zend extensions is a large topic as you would expect, but once again ext_skel comes through by defining long n for us. Of course as programmers it’s odd to call and define a function where we don’t pass parameters directly, but with a heavy helping of macros like zend_parse_parameters that’s exactly what happens.

That is, in our original fannkuch.h file we defined our function as:

int fannkuch(int n)

ext_skel thus went in and said: right, we need a parameter called n, I’ll create one for us and also add the code to implement the actual variable assignment:

if (zend_parse_parameters(argc TSRMLS_CC, "l", &n) == FAILURE)
 return;

This macro grabs the passed value from the internal parameter stack and make it available to our function via the n parameter. To add more parameters to this function we would need to add more entries to the zend_parse_parameters macro, which is unfortunately outside of the scope of this post.

Returning Values

Just like input parameters, returning values in PHP extensions is unique. Whereas in C we can use a simple return statement, in PHP extensions we once again implement macros. Different versions exist for the type of variable we return, but in the case of our function, we’ll return a long with:

RETVAL_LONG()
return;

In other words, in PHP extensions we need to push any return values into a larger hash table structure, which is then, at function termination, possibly accessed via other structures and functions in the larger context of the core PHP Zend engine (as in our example, we need to access the return value of our function for the echo call) .

Critically, we still need to terminate our function which means we still issue a return as normal (we can however, use the RETURN_LONG() macro to do both).

Coincidentally, this is also why PHP is so darn slow for intensive data processing where user space code dictates the parameter space. It’s also, of course, why such tasks benefit so greatly from custom extensions. If we have code that requires many variable accesses of any sort, moving such code into an extension and only passing a few parameters provides positively massive gains, as we’ll see shortly.

The Code

So the code that’s we use as a test extension is none other than our Sunspider fannkuch test code.

I’ve added a few comments along the way with some important points, such as how we cannot use bool as a type but rather zend_bool, and how we cannot declare for() loop variables inline.

In all these are minor changes, and most importantly, after a few hiccups of learning Zend syntax for how parameters and returns work, this code, originally JavaScript, then C++, and now C, was almost a copy/paste affair.

PHP_FUNCTION(fannkuch)
{
    int argc = ZEND_NUM_ARGS();
    long n;

    if (zend_parse_parameters(argc TSRMLS_CC, "l", &n) == FAILURE)
        return;

    //php_error(E_WARNING, "fannkuch: not yet implemented");

    // my code
    int check = 0;
    int perm[n];
    int perm1[n];
    int count[n];
    int maxPerm[n];
    int maxFlipsCount = 0;
    int m = n - 1;

    // cannot declare loop values in non-c99 mode
    int i;
    for (i = 0; i < n; i++){
        perm1[i] = i;
    }

    int r = n;

    zend_bool t = 1; // zend/c cannot use bool, but zend_bool
    while (1) { // no matter, just use 1 until return

        // write-out the first 30 permutations
        if (check < 30){
            int s = 0;
            int i;
            for(i=0; i<n; i++) s += (perm1[i]+1);
            check++;
        }

        while (r != 1) { count[r - 1] = r; r--; }

        if (!(perm1[0] == 0 || perm1[m] == m)) {
            int i;
            for (i = 0; i < n; i++) perm[i] = perm1[i];
            int flipsCount = 0;
            int k;

            while (!((k = perm[0]) == 0)) {
                int k2 = (k + 1) >> 1;
                int i;
                for (i = 0; i < k2; i++) {
                    int temp = perm[i]; perm[i] = perm[k - i]; perm[k - i] = temp;
                }
                flipsCount++;
            }

            if (flipsCount > maxFlipsCount) {
                maxFlipsCount = flipsCount;
                int i;
                for (i = 0; i < n; i++) maxPerm[i] = perm1[i];
            }
        }

        zend_bool g = 1;
        while (1) {
            if (r == n) {
                // return maxFlipsCount;
                // zend return method - can also use RETURN_LONG
                RETVAL_LONG(maxFlipsCount);
                return;
            }
            int perm0 = perm1[0];
            int i = 0;

            while (i < r) {
                int j = i + 1;
                perm1[i] = perm1[j];
                i = j;
            }

            perm1[r] = perm0;
            count[r] = count[r] - 1;

            if (count[r] > 0) break;
            r++;
        }
    }

    // my code end

}

How do we test the updated extension code? Simple: save the .c file and, back in the command line, which should still point to:

/home/matthew/Documents/server/src/php5.3-201107100630

We need only run:

make

and:

make install

This will completely rebuild our extension with the new code, which we can then instantly try out via our previous command to the test file:

../../php-snap/bin/php ext/fannkuch/test.php

If all went well, we should now get the result of our fannkuch() call, which should be 38.

Closing Remarks & Performance

Building more complex PHP extensions is not exactly an easy affair, as here we’ve only scratched the surface. Their are resource types, hashes, reference type variables, Objects, on and on…the key though, is by utilizing the excellent ext_skel tool we get our first extension up and running very quickly. We can then, provided we’ve given ext_skel a header file to work with, even implement our own code with very little effort. In many cases this may be all that’s required–just a simple addition to protect your code-base, add needed functionality, and so on.

PHP, from the user or developer space, never ceases to surprise and delight me.

Of course as a final point it should be said that implementing fannkuch as an extension is, by no exaggeration, simply stunning in terms of raw performance when compared to standard PHP.

From a few quick tests via:

time ../../php-snap/bin/php ext/fannkuch/test.php

Show that where:

n = 9:

30
real	0m0.025s
user	0m0.018s
sys	0m0.006s

n = 10

38
real	0m0.189s
user	0m0.187s
sys	0m0.001s

n = 11

51
real	0m2.246s
user	0m2.239s
sys	0m0.003s

Compare that to fully optimized C++
n = 9

0m0.019s

n = 10

0m0.188s

n = 11

0m2.261s

And vanilla PHP:
n = 9

0m1.624s

n = 10

0m19.058s

n = 11

> 4 minutes

In short: It’s as fast as C++, and simply destroys vanilla PHP.

The first three are of the fannkuch benchmark found on the sunspider test site, only ported over to C++ and PHP in addition to the JS version. The browser used for all tests was Firefox 5.

The HPHP listing is a compiled Hip Hop PHP version, which is interesting as it shows the relative difference between it and vanilla PHP.

The last item, Benchmark.php, is the same benchmark file you’ll find in a PHP source download.

See the full benchmarks after the jump!

access-fannkuch – n=9

JavaScript: 0.204

PHP: 1.624

HPHP: 0.538

C++ .20

access-fannkuch – n=10

JavaScript: 3.062

PHP: 19.058

HPHP: 6.395

C++ .189

access-fannkuch – n=11

JavaScript: 40.297

PHP: ~4 minutes

HPHP: 23.713

C++ 2.265

Benchmark.php

PHP: 2.542

HPHP: 0.547