Anonymous functions, also known as closures, allow the creation of functions which have no specified name. They are most useful as the value of callback parameters, but they have many other uses. Closures can also be used as the values of variables; PHP automatically converts such expressions into instances of the Closure internal class.

PHP has very few predefined classes that are part of the core language, so naturally I was intrigued by the Closure class. The PHP Manual has this to say about the class:

The predefined final class Closure was introduced in PHP 5.3.0. It is used for internal implementation of anonymous functions. The class has a constructor forbidding the manual creation of the object (issues E_RECOVERABLE_ERROR) and the __invoke() method with the calling magic.

The invoke magic method is also a new feature in PHP 5.3. It is called when an object is used in the context of a function (e.g. $object($parameter);). Since Closure objects will be used like functions, this is a critical feature of the Closure object. The Closure class may be perfectly equipped to act like an anonymous function, but it does not provide any extra utility beyond that. A var_dump() of a closure will reveal the functions parameters, but there is no way to get any other information about the Closure (like the actual code of the function). Trying to serialize the Closure throws an Exception and json_encode() just returns an empty JSON string. To make matters worse, the Closure class is final, so there is no way to extend it.

That simply wasn’t going to cut it for me. I wanted to make my own Closure class that was at least able to do the following:

1. Invoke the Closure (with __invoke()) just like the PHP Closure class
2. Retrieve the actual code of the Closure
3. Retrieve detailed data about the Closure’s parameters
4. Retrieve the names and values of any variables inherited from the Closure’s parent’s scope (with the use construct)
5. Serialize and unserialize the Closure

I decided to play around with the Reflection API to see what kind of information I could get from the Closure. A Closure is a function, so using the ReflectionFunction class provides the same information about Closures as it does about any other functions. PHP 5.3 also added the isClosure() method to the ReflectionFunction (in case you weren’t convinced that reflection would be helpful). After some tinkering, it became apparent that I would be able to accomplish all of my desires. I will walk you through the construction of my “SuperClosure” class and explain how the creative use of Reflection allows us to find ways around the problems normally blocking the ability to have my desired features.

1. Grant the ability to invoke the Closure

The first goal was to create the basic SuperClosure class that both encapsulates and allows invocation of the Closure. To do this I wrote a simple class with a constructor and the magic __invoke() method. I also included an accessor method (getter) for the actual Closure. In the constructor I created an instance of the ReflectionFunction class and stored it as a class member that helped allow invocation of the closure with a variable number of arguments. It also helped me accomplish other things later. The following code shows the basic class upon which I will be building. The complete will be shown at the end of the article.

class SuperClosure {

	protected $closure = NULL;
	protected $reflection = NULL;

	public function __construct($function)
	{
		if ( ! $function instanceOf Closure)
			throw new InvalidArgumentException();

		$this->closure = $function;
		$this->reflection = new ReflectionFunction($function);
	}

	public function __invoke()
	{
		$args = func_get_args();
		return $this->reflection->invokeArgs($args);
	}

	public function getClosure()
	{
		return $this->closure;
	}
}

The __invoke() method allows the SuperClosure object to be used exactly as if it was a Closure object. I had to recreate this functionality for the SuperClosure class since I was not able to extend the Closure to begin with. In order to pass arguments through to the real Closure, I used a combination of the func_get_args() function and the invokeArgs() method of ReflectionFunction to ensure that any variable number of arguments could be used. I could have also used call_user_func_array() function, but I prefer Reflection, and since I already had an instance of the ReflectionFunction, the invokeArgs() method seemed like a better choice.

2. Retrieve the actual code of the function

Secondly, I added code that allowed the SuperClosure class to find and store the actual code defining the closure. This feature is actually the key to doing the serialization later and is the most complicated part of the program. To accomplish this portion of the program, I used the instance of ReflectionFunction from Step 1 and the SplFileObject class, an SPL class which provides a nice object-oriented interface for dealing with files. The getFileName(), getStartLine(), and getEndLine() methods of the ReflectionFunction class allowed me to retrieve all the information I needed to find the source code of the Closure function. Using the SplFileObject to open and read from the source file and in combination with some string manipulation, I was able to obtain the closure’s source code. The following code shows the protected _fetchCode() method used to parse the closure’s code out of its source file:

protected function _fetchCode()
{
	// Open file and seek to the first line of the closure
	$file = new SplFileObject($this->reflection->getFileName());
	$file->seek($this->reflection->getStartLine()-1);

	// Retrieve all of the lines that contain code for the closure
	$code = '';
	while ($file->key() < $this->reflection->getEndLine())
	{
		$code .= $file->current();
		$file->next();
	}

	// Only keep the code defining that closure
	$begin = strpos($code, 'function');
	$end = strrpos($code, '}');
	$code = substr($code, $begin, $end - $begin + 1);

	return $code;
}

The only limitations with the current version of this function are that you cannot have multiple closures on a single line and you cannot use the word “function” anywhere besides the actual closure’s declaration.

3. Retrieve detailed data about the Closure’s parameters

Retrieving information about the closure’s parameters was as simple as adding a method that simply returns the result of the getParameters() method the SuperClosure’s instance of ReflectionFunction. That’s all. The getParameters() method returns ReflectionParameter objects which have several methods for getting information about the parameters including their names, values, default values, and more.

4. Retrieve the names and values of any variables inherited from the Closure’s parent’s scope

One of the biggest problems I had was retrieving the names and values of the variables added to the Closure’s scope with the use construct. There isn’t a documented way of doing this as far as I know, but I was able to figure out a way to do it after playing around some more with the ReflectionFunction class (it is just so helpful). These variables are actually included in the results of the getStaticVariables() method. If the closure declares any variables with the static keyword, these will also be included in the results. To get around this I added the _fetchUsedVariables() method that uses a combination of the getStaticVariables() method and some string manipulation of the Closure’s code (from Step 2) to find only the variables that were inherited from the parent’s scope. The following code shows the _fetchUsedVariables() method:

protected function _fetchUsedVariables()
{
	// Make sure the use construct is actually used
	$use_index = stripos($this->code, 'use');
	if ( ! $use_index)
		return array();

	// Get the names of the variables inside the use statement
	$begin = strpos($this->code, '(', $use_index) + 1;
	$end = strpos($this->code, ')', $begin);
	$vars = explode(',', substr($this->code, $begin, $end - $begin));

	// Get the static variables of the function via reflection
	$static_vars = $this->reflection->getStaticVariables();

	// Only keep the variables that appeared in both sets
	$used_vars = array();
	foreach ($vars as $var)
	{
		$var = trim($var, ' $&');
		$used_vars[$var] = $static_vars[$var];
	}

	return $used_vars;
}

5. Grant the ability to serialize and unserialize the Closure

Finally, I implemented the serialization capabilities. Since an actual closure object cannot be serialized (it gives a fatal error), I had to be creative to come up with a way to do my own serialization. The code I wrote for Step 2 and Step 4 made this possible. The sleep magic method allows you to hook into the serialization process, so I used this method to prepare my SuperClosure class for serialization. In order for my class to be serialized, I needed to first stop the Closure object and the ReflectionFunction object representing the Closure from being serialized. The __sleep() method should return an array of the class members you are serializing, so I simply left those variables out of the list. Because I was serializing the Closure’s code and static variables, it made possible in the __wakeup() magic method to recreate the Closure object using the code. To do this I had to use the extract() function on my array of used variables to import them back into the scope. Then I performed an eval() operation on the Closure’s code in order to recreate the Closure. eval() is always a risky operation, but in this case it makes sense. If you are planning to use this code for any of your own projects, I urge you to take precautions. The following code shows the complete SuperClosure class with the additions of the __sleep() and __wakeup() methods.

class SuperClosure {

	protected $closure = NULL;
	protected $reflection = NULL;
	protected $code = NULL;
	protected $used_variables = array();

	public function __construct($function)
	{
		if ( ! $function instanceOf Closure)
			throw new InvalidArgumentException();

		$this->closure = $function;
		$this->reflection = new ReflectionFunction($function);
		$this->code = $this->_fetchCode();
		$this->used_variables = $this->_fetchUsedVariables();
	}

	public function __invoke()
	{
		$args = func_get_args();
		return $this->reflection->invokeArgs($args);
	}

	public function getClosure()
	{
		return $this->closure;
	}

	protected function _fetchCode()
	{
		// Open file and seek to the first line of the closure
		$file = new SplFileObject($this->reflection->getFileName());
		$file->seek($this->reflection->getStartLine()-1);

		// Retrieve all of the lines that contain code for the closure
		$code = '';
		while ($file->key() < $this->reflection->getEndLine())
		{
			$code .= $file->current();
			$file->next();
		}

		// Only keep the code defining that closure
		$begin = strpos($code, 'function');
		$end = strrpos($code, '}');
		$code = substr($code, $begin, $end - $begin + 1);

		return $code;
	}

	public function getCode()
	{
		return $this->code;
	}

	public function getParameters()
	{
		return $this->reflection->getParameters();
	}

	protected function _fetchUsedVariables()
	{
		// Make sure the use construct is actually used
		$use_index = stripos($this->code, 'use');
		if ( ! $use_index)
			return array();

		// Get the names of the variables inside the use statement
		$begin = strpos($this->code, '(', $use_index) + 1;
		$end = strpos($this->code, ')', $begin);
		$vars = explode(',', substr($this->code, $begin, $end - $begin));

		// Get the static variables of the function via reflection
		$static_vars = $this->reflection->getStaticVariables();

		// Only keep the variables that appeared in both sets
		$used_vars = array();
		foreach ($vars as $var)
		{
			$var = trim($var, ' $&amp;');
			$used_vars[$var] = $static_vars[$var];
		}

		return $used_vars;
	}

	public function getUsedVariables()
	{
		return $this->used_variables;
	}

	public function __sleep()
	{
		return array('code', 'used_variables');
	}

	public function __wakeup()
	{
		extract($this->used_variables);

		eval('$_function = '.$this->code.';');
		if (isset($_function) AND $_function instanceOf Closure)
		{
			$this->closure = $_function;
			$this->reflection = new ReflectionFunction($_function);
		}
		else
			throw new Exception();
	}
}

Conclusion

With some cleverness and reflection, I was able to create the SuperClosure class and extend the Closure’s normal capabilities. Although my class does not enhance the typical use of a PHP Closure it does make it possible to serialize and transport a closure and provides an interface for examining the parameters and inherited variables. This means that we can do:

$closure = new SuperClosure(
	function($num1, $num2) {return $num1 + $num2;}
);
$serialized_closure = serialize($closure);
$unserialized_closure = unserialize($serialized_closure);
echo $unserialized_closure(1, 5);

without worrying about errors. Also, it would technically be possible to send closures through a remote service.

From what I have read in the PHP Manual, this class may have to be changed in the future. The Manual states this regarding anonymous functions:

Anonymous functions are currently implemented using the Closure class. This is an implementation detail and should not be relied upon.

So according to this, future versions of PHP might implement closures differently, and my class would have to be rewritten.

If you would like a copy of the code from this article (and an example of its use) you can find it on my Github account.

While managing your social networking presence on Twitter and Facebook, it can be difficult to quantify the impact of each medium. While I’m a huge fan of Twitter, traffic results from earlier today on one of my sites confirmed for me what may sound like common sense: Facebook fans drive far more traffic per-user than Twitter followers for a given promotional message.

I’m currently running a contest in association with T-shirt company Threadless. (It’s called Threadknits, and it’s based on knitting and crocheting their t-shirt designs into crafts.) Today, Threadless posted a message on their Facebook page and  Twitter, both with essentially the same content: an invitation to check out Threadknits. They were both posted at nearly the same time.

The numbers are what might surprise you. Threadless has almost 1,500,000 followers on Twitter, and “only” 102,000 fans on Facebook. With the posts made within an hour of each other, my traffic on the site shot up, with a couple thousand visitors hitting by day’s end. Here’s the breakdown of traffic driven from each:

 
The difference is absolutely staggering. Whereas Facebook generated an approximate 1.08% clickthrough rate, Twitter’s was closer to, well, 0%. 232 visitors came from Twitter or related sites directly and 450 additional clicks landed on the home page without a referrer, which I’m chalking up to clicks from Twitter clients. (Though, to be fair, this could easily overstate Twitter’s influence.)

On a previous contest, Threadless would tweet and I’d see between 1,000-2,000 clicks on their roughly 1.4 million followers, so while it may be a bit low today, I think the point stands: Even at its best, Twitter for large audiences generates clickthrough rates dramatically lower than Facebook. For 2,000 clicks, the rate at 1.4M followers stood at 0.14%. A quick look at the bit.ly stats on a few links from Ashton Kutcher (the #1 Twitter personality by followers) shows they typically net about 20,000-30,000 clickthroughs, on 4.3M followers, gaining a decent amount on the Threadless best-case scenario all the way up to 0.48% ~ 0.60%. (This accounts somewhat for the viral nature of Twitter as bit.ly clicks are counted for retweets as well.) Naturally, clickthrough rates will vary dramatically even amongst popular Twitter personalities for a variety of reasons. I’d like to focus more on the significant difference between the Facebook and Twitter rates I witnessed today.

There are likely several possible reasons for this:

• The audience may be slightly different—people willing to consider themselves “fans” on Facebook may be more picky with their allegiance than those willing to follow an account on Twitter.
• The phrasing and formatting of the message were slightly different—not exactly apples-to-apples as Facebook includes the logo and a text clip from the website, but I imagine this had a negligible effect.
• My mileage may vary—this is an admittedly small sample size, but I think the evidence and logic around these results indicate they’re not anomalous.
• Most importantly, Facebook lingers while Twitter sails by. Users are probably more likely to follow links during their Facebook time than from a passing Twitter notification unless it’s of particular interest to them.

That last point is particularly important. Facebook, having reconfigured their News Feed yet again, no longer sorts things there chronologically. They’ve merged the Highlights functionality back into the News Feed which they now use to keep certain posts “stickier” than others based on what they believe you might be interested in. (It manages to do a strikingly horrible job at this compared to how it used to perform, but that’s a conversation for a different post.)

With Twitter, the very nature of real-time can be summed up: blink and you miss it. While you can use a Twitter client to review tweets over the past day or two, it’s still less likely your tweet was as visible over Twitter as a post would be on Facebook’s News Feed. I’d like to see some more statistics on total audience reach. The clickthrough rate surely only tells part of the story—I’d be far more interested to learn what percentage of each audience even saw the post, and determine true clickthrough rates from that.

In the end, it’s important to consider the overall spirit of the findings here. Twitter is great for growing virally and interacting with customers, but your message on Facebook may have a far more lasting impression and generate greater returns, even if fans are more of a fight to procure. Engage on both, but recognize the differences between them and leverage each of their strengths. I’ll likely post about the best way to do that for each site in the near future.

(The above graphic represents the total clickthrough breakdown by medium assuming a linear progression of Threadless’ Facebook audience to match their Twitter audience, maintaining the same clickthrough rates from today’s traffic. It’s likely the Facebook clickthrough rate could in fact fall some as their audience grew, but it’s my belief that it would still beat Twitter, user for user.)

PHP is a very easy language to learn and use because it has a syntax that is a simplified sampling from C++/Java and Perl. People who use these languages pick up PHP fast, and people who haven’t used any languages often learn PHP with as much ease as they would learn Perl.

A bit about namespaces

Most languages have namespaces in one form or another. A namespace is a lot like a class: it is a named entity that houses functions and classes, whereas a class is a named entity that houses properties and methods. Namespaces are found, in one form or another, in most popular languages. In C++ there are explicit namespaces and the namespace keyword. In Java, since you are allowed to have classes nested within classes, classes act as namespaces. Namespaces make it easier to organize and call your code. If you have a bunch of classes that work together to do something you might name them in a similar way. For example, we here at the Studios sometimes use the following classes:

Canopy_Xml_Document
Canopy_Xml_Document_CDataSection
Canopy_Xml_Document_NodeList
Canopy_Xml_IWritable
Canopy_Xml_Exception

With namespaces, we could define a namespace Canopy, add to it a namespace Xml, and then add classes Document, Exception and interface IWritable. We could also add a Document namespace and add classes CDataSection and NodeList. Then we could just tell PHP that we want to refer to things in a certain namespace. We would simply write:

use Canopy::Xml;
$D = new Document(); //$D is equivalent to a Canopy_Xml_Document


In the beta builds of PHP 5.3, the terminal “::” called the scope resolution operator (SRO) is used to denote namespaces. So, in the same way that one would write Class::staticMethod() one could write NamespaceA::NamespaceB::functionInNamespace(). This is the same operator used in C++, and it is intuitive since it refines a name to a scope where that name belongs.

PHPs SRO problem

Recently, PHP has had to abandon using the SRO, forego the syntax of other languages and has had to make up their own way of denoting a namespace. One reason for this is that bare words in PHP can have multiple meanings. For example, in PHP you can do:

const('A','constant');
function A() { return 'function'; }
class A(){ public _toString(){ return 'class'; } }


such that

echo A; // constant
echo A(); // function
echo new A(); // class


all do what you’d expect. But in namespaces, there’s some ambiguity introduced. Consider rather A::B::C() is namespace A, class B, static method C, or namespace A::B, function C. In C++ you’re not allowed to have a symbol like A, B, or C represent multiple types of things, so using the SRO does not arrive at ambiguity there. (Instead, you get an error.) But since PHP does allow for that type of thing, the internals team had to choose between:

1. Not allowing namespaces to have functions, but rather only classes, thus raising the ire of the community.
2. Using some terrible terrible other operator to indicate namespaces, thus raising the ire of the community. (Quoth internals team member: <@CelloG> % is also possible but holy god almighty it is fugly
3. Put off namespaces until (or if) something decent can be figured out, thus raising the ire of the community.

The team decided on 2, going with the \ character.

What’s that mean for the language?

The discussion was concerned with preventing unanticipated behavior from just making a rule to disambiguate rather a function or static method is called. I feel that throwing an error (or warning and choosing one) whenever that situation arises would be a perfectly acceptable solution. I would also have been happy to see them further follow Java’s lead and allow classes to be nested. Then there is no ambiguity between a static method and a function because they are the same. This would probably require much more work and you would have to allow class definitions across multiple files, but I feel this is a better option than ‘\’. To that end I would have enjoyed option 1 over option 2.

The most cited complaints about PHP is how there is little consistency in function names and in argument order. This new ‘\’ operator adds to the “weird-factor” which makes it more difficult for people to read and use the language. It further marginalizes PHP, adding to a series of minor but obnoxious issues proponents are embarrassed by.

The language is adrift with little direction, trying to address the wants of its developers, and adding a terminal is payment for a lack of planning and a general stubbornness of the internals list to consider namespaces back when the 5 changes were initially being discussed. On one hand I feel like this has ominous portents for PHP, but on the other, I’m really happy to have namespaces. For our domain, it still doesn’t make sense to jump ship to Groovy or Python, and it would take a lot of new terminals to push us over the edge. And while it’d be nice to see PHP not give more ammo to detractors, however menial, the discussion log makes it clear what issues they were struggling with.

Frederik Holmström points out another flaw with the backslash, as well, but many commenters note that the internals team knew there was no perfect solution here.

When I began my career at the medium sized company, I initially saw my project managers as a pain in the ass. All I was interested in doing was coding. I had my own ideas of how the project or feature was going to be done and I thought that I could handle deadlines and project requirements better than they could. Was it really necessary to ask me multiple times a day what I was doing and what percentage of the project was still left to be done? PM’s gave new meaning to the phrase “avoid like the plague.” In all honesty, I wondered why in the hell these people were even hired to begin with. I just could not see the role of a project manager as being all that important.

Of course, after a bit of development work and experience doing projects with and without them, I found myself having moments of… “OMFG…where is my project manager?” As developers, void of project managers, we often get carried away with the art that is code. We plan for one thing but, weeks post-deadline, end up creating something entirely different. Does it work? Usually. Is it beautiful? Maybe. Is it what was promised to the customer? Well, no.

I’m not just talking about instances where a simple login feature ballooned into a grand hierarchical authentication scheme. I’m also talking about those projects, that for some reason or another, we just despise working on and push off completion; sometimes even writing substandard code just to get it over with. And don’t forget those projects that are just so complex that we’re not even sure if we are starting at the right place, much less dedicated to a deadline of any sort.

It seems that the problem here is not one of competence—most of us could, if we really tried to, stick within initial specs and scope. To me the problem lies in the fact that without the role of the project manager on the team, we lack the time and task management designed to keep us on track. We are trained in how to solve problems of logic through good coding principals and design. We are artists. Our canvas is digital and we sometimes let our creativity and desire for compounding beauty interfere with the job we were hired to do. We are not trained in restraint; the project manager is, at least in theory.

Depending on your experience, I suppose this next statement may be gospel or blasphemy, but to me, it is gospel all the way. The successful completion of any project hinges upon a solid development methodology based in clearly defined project goals, block specifications, complete use cases, and the division and distribution of task level development. A good project manager will be able to take high level design specs and goals from a product manager or customer contact and break them out into clearly defined and manageable building blocks. Each block can then be broken down into feature/requirement sets which can finally be tasked out to the developer/s for completion. This is the job of the project manager, not the developer. Developers should only be given spec’d out tasks with detailed use cases and an explanation of exactly what should be done. It is then up to the developer to determine how the task will be done.

Now, this is not to say that developers are not to be involved in planning or design phases of projects; by nature, they have to be. However, design and planning meetings involving developers should be focused on the technical goals of the project or block, not on management and product/customer level concerns. These meetings must be narrow in focus and should have a clearly defined result. For instance, determining the database schema for the project or settling on a development framework. The layer of abstraction here is critical in keeping the project within scope and to prevent developers from expanding on or reducing the project deliverables. Once the meeting objectives have been met, project managers should then be able to chunk and task out the technical specs to the team of developers.

Now, up until a little while ago, I hated the word “task.” Every morning I would check my email and see, “You have been assigned a new task from…” after which my head would sink low and I would seriously contemplate my career path yet again. I would constantly wonder why everything had to be a task: a task to change a misspelled word, a task to add 5 pixels of spacing to the end of the page. Wouldn’t it just be easier to send me an email or IM or for goodness sake, just turn around and tell me? Surely this couldn’t be the Zen that is programming? However, recent experience has taught me that, by God, we all need tasks just like we all need competent project managers!

Task assignment, and by extension, a good task management system (note that a task/project management system is very different from a strict bug tracking system) is an absolute necessity for project management. Proper task etiquette enabled by a solid and complete project management system serves many purposes. First and foremost to developers, it serves as our grocery list. It tells us EXACTLY what we have to do, when it needs to be done by, and contains a history of the problem or feature (most PM system tasks have comments/histories and file attachment capabilities). From the management side of things, a PM system will track the number of hours developers spend on each task which can then be used for billing or other performance metrics. It also can serve as a yardstick in determining the level of project completion by showing a variety of statistics based on ticket assignment, time used, percentage of tasks complete for a given project, and developer performance.

Not to sound too preachy (or take too much of an aside), but I have come across a number of good techniques gathered from the various project managers and lead developers I have worked with in the past that I would like to share and recommend that people use.

Quote Tasks

Make use of quote tasks. It may take a little more time and to some it may seem really useless, but in the end it will serve as a good measure of your time estimation abilities and speak to the thoroughness of your work. The way it works is that for every task you are assigned, unless fixing the problem will take less than 10 min., in the comments section you should list the files and line numbers that you will need to make changes to in addition to a brief one liner explaining the expected change. You then assign the ticket back to your project manager to quickly review. If your time estimate falls within the allowable project time, then you will get the task back and no matter how many days or weeks may pass, you will know immediately what needs to be fixed and where.

Notate Completed Tasks

When tasks are complete, you should list each file and line number range that you made modifications to and provide a one or two sentence description of changes (in addition your regular brief outlining the totality of changes). This makes tracking changes easier for those who may follow you and serves as a written history in a developers words of exactly what was done for the task.

The Six-Hour Work Day

One small tip on time management and allocation I learned from my last PM was to only allot 6 hours of estimated work in a day. He said that the remaining 2 hours are spent on bathroom breaks, water cooler conversations, task switching ramp up, nerf fights, etc. At first I thought this was strange. Could bathroom breaks really take up 2 hours of my working day? But as I was tasked on this schedule, it seemed to work out pretty well. While I didn’t exactly lose 2 hours doing random things, what did happen was that I was rarely under pressure to squeeze out tasks leaving me less stressed and able to produce better quality and more thoroughly tested code and documentation. Funny thing, comparing a “6 hour” work day with the regular, “get as much work done in the 8 hours you are here” approach, I felt more productive and competent under the 6 hour routine. Give it a try, you may be surprised.

Developers as Project Managers: Warning

Developers should never be placed in the role of a project manager; creating and managing their own tasks. However, sometimes there is little choice (little company resources, sole operation, etc). If this is the case, expect that as a developer you will be spending probably 1/4-1/3 of your day planning and managing tasks; not working on them. Be sure your boss knows this if they are not willing or able to task out projects properly to begin with. This is really for your sanity and for the thoroughness that comes with being a competent developer. Now, in the rare case where you as the developer are tasked with actually taking customer specs and being told to simply get it done, placing the entire development life cycle on you or your team’s shoulders, I strongly suggest that you ask management to hire a project manager. If that is not possible then maybe you should be dusting off that resume because project specing, chunking, and higher level project management duties like those are not your job. There is a good reason developers are not PM’s and PM’s are not developers.

The Devil’s in the Details

Nag, nag, nag. If you get assigned tasks that are too general in scope and you have any questions about exactly what it is that your PM or boss is asking for, send the ticket back and ask for further explanation. It usually only takes a few times doing this for your boss to get the idea of what information you need to get your work done right. Remember, training goes both ways; you learn from your boss as much as your boss learns from you. It’s better to take the time upfront and get accurate and detailed specs than for you to guess and have to go back and rewrite something down the road.

All that being said, there are a number of development methodologies out there to help you and your team develop software in an efficient and smart manner. Now, as I’ve made crystal clear, I’m not a PM so I don’t claim to be an expert on any development methodology, but I take particular affinity to Agile development methods like Scrum. Any methodology where you as a developer are protected from business processes (changes in specs/customer expectations, etc) external to your core job function is a winner in my book. Avoid the traditional waterfall and cowboy coding approaches as they are highly inflexible and in the former case, plain stupid as a development methodology for any business.

So there! I hope I’ve been able to give a little personal experience to you developers out there (and you managers) who may be a little too code-centric. Don’t ignore the management side of your business. It can save your sanity and make you a better developer in the process!

As developers, once we start separating our code into abstract ontological typologies, we make use of the human mind’s phenomenal ability to work with types.  Our code becomes less about jump tables and registers and more about users, email messages and images.  What once was a problem of allocating resources and operations within the computer becomes an abstract, logical problem within a collection of objects.  Like children awe-struck by stories of magicians of old, speaking incantations and pressing their wishes into reality by the power of their mind alone, we become drunk with the sense of awe and possibility.

We *really* dive into things after the jump, so go ahead and hit it.

The magician’s spell is our codebase, and as they toil in their laboratories collecting newts and dragon finger-nails, so we assemble libraries or components and frameworks.  We write ever more diverse and sprawling codebases to power our behemoth, monolithic applications and we grow increasingly unhappy with the amount of time we spend writing the same code over and over.  As we come up with more magic to shorten our tasks, we outpace ourselves by envisaging larger and larger projects.

As we write better, more reusable code, early on in this process we experience the satisfaction of leveraging our components to achieve results.  So developers (those of us who aren’t terrible) get into this groove of reducing the size of the logical elements we write, reusing those elements, or recycling them through extension or currying. Over time, by constantly working to reuse our own code, we choose practices that work well for ourselves and discard practices that don’t work as well or slow down our workflow. For developers flying solo or those working on small projects, this evolutionary process is a sufficient way of going about things.

But there’s trouble when we add other players into the mix—other developers, a user interface person, a database person, a sysadmin, a project mana-jerk: as a developer, they don’t have access to our “experience” of the code and we don’t have access to theirs.  So the practices, workflows and logic which made sense to us when we built a component for personal reuse may not make nearly as much sense as we’d hope to another developer who tries to work with it. Worse, we don’t learn from these mistakes because we may not even know they’ve been made.  Even if we suspect there’s trouble—perhaps a dead fish in your computer case or threatening instant messages—we still may not know enough about the mistake to be able to affect positive changes on our behavior.

We’ve been doing some extra work on our libraries at the Studios, and we’ve been arguing about the why and how some our many programmes for code re-use have failed, miserably so, and the surprising things that are still in use today.

Recycling Aluminum, Plastic and Code

I envoked the slogan “reduce, reuse and recycle” earlier because refuse recycling serves as a good metaphor for code reuse, and frankly, because you can take the magician allegory only so far. Recycling is expensive: It has all of the costs of trash collection (save the landfill/incinerator/New Jersey [Hey!—Editor from NJ]) and adds the cost of sorting and additional transportation costs. The idea is that value is recuperated from the actual recycled materials, but that’s not always the case. (See the parallel yet?) From an about.com article on recycling:

Michael Shapiro, director of the U.S. Environmental Protection Agency’s Office of Solid Waste, also weighed in on the benefits of recycling:

“A well-run curbside recycling program can cost anywhere from $50 to more than $150 per ton trash collection and disposal programs, on the other hand, cost anywhere from $70 to more than $200 per ton. This demonstrates that, while there’s still room for improvements, recycling can be cost-effective.”

But in 2002, New York City, an early municipal recycling pioneer, found that its much-lauded recycling program was losing money, so it eliminated glass and plastic recycling. According to Mayor Michael Bloomberg, the benefits of recycling plastic and glass were outweighed by the price — recycling cost twice as much as disposal. Meanwhile, low demand for the materials meant that much of it was ending up in landfills anyway, despite best intentions.

We can draw parallels between the things that drive the cost of some municipal recycling programs up past those of others and the things that reduce our ability to reuse software.

Consider the following factors:

• Volume of recyclables / Volume of components to reuse
• Volume of non-recyclables /  How many components are NOT reused
• Adoption of the recycling program / Adoption of libraries and frameworks by developers (consistently high usage rates are necessary for effectiveness)
• Choices as to what to recycle and what not to recycle / Clearly defined boundaries on which components are reusable and which ones are not
• Time it takes to sort through the material to separate the recyclables / Developer’s ability to know what’s already done and in the library or framework, versus what they have to build, versus what they want to build just for the hell of it
  
• Expense to transport material /  Overhead associated with libraries and frameworks and autoloaders and oh my

In software development as in refuse recycling, we naturally want to reuse as much as possible. But the return on recycling can vary greatly from item to item. And when you factor in the cost of setting up a component for reuse (introducing it into the environment, building with proper levels of flexibility and extensibility) and the non-trivial, yet oft-overlooked cost of the time a developer spends familiarizing themself with the way you’ve designed the component, it may not make economic sense to recycle certain components.

It’s up to the savvy developer to decide what will be recycled and what will not be recycled. Sometimes we make poor choices and have components that are stuck into frameworks that we never use. This is a double-whammy, since we spend more time building the component for reuse and designing for scenarios which will never occur. How do we minimize this and select the best candidates for reuse? Let’s get into that.

Choosing What To Recycle

My alma mater, Arizona State University (ASU, go Devils!), has separate recycling bins for plastic and aluminum all around campus.  ASU doesn’t recycle only aluminum and plastic bottles, but these are the most prominent containers around campus. This decision makes a lot of sense because students go through a lot of soft drink cans and water bottles daily, while their paper waste output in public thoroughfares on campus is considerably less. (You normally sort and toss your flyers, notes or other paper propoganda once you get back to your dorm, for instance.)

Arizona State makes up a significant portion of the city of Tempe.  Tempe’s residents output a larger variety of refuse so their recycling effort has tried to be more inclusive of other materials; almost something of a long-tail of recycling, to a point.

It’s too easy to hang oneself in trying to make ever increasingly complex and elaborate artifices in the name of reuse. Deciding what components to build for reusability is an important part of making code reuse feasible. I wont enumerate the number of projects that we took extra time on so that we could reuse them only to have them languish in a corner of our code repository, never to see the light of day again. Nor will I tell you the dollar amount associated with the number. I will say that it’s significant.

But worse still is the number of hours we’ve spent coding the same component over and over again, each time a new iteration for a new project, when one well-developed go at it would’ve delivered a highly reusable, stable piece of brilliant cost-savings and elegant code. (Realistically, we develop a “reusable” component each time, making it “better” each time.)

The best way to know what components are good candidates for reuse is to know from experience.  If you don’t have experience I suggest you develop a time portal to the future and observe the results of your work before doing it. My preferred type of “time portals” are books on design patterns:

Martin Fowler’s “Patterns of Enterprise Application Architecture“, Alur et al’s “Core J2EE Patterns“, Gamma et al’s “Design Patterns” and Matt Zandstra’s “PHP Objects, Patterns, and Practice” are all great resources for problems that come up a lot and solutions that have worked, the latter being written in PHP. Beyond that, researching how things are done in other channels of software development (or even, other industries) can provide inspiration on what can be used and reused.

Once you’ve identified what types of components you’d like to reuse you must ensure that your components are, first and foremost, usable. It sounds simple, but a lot of people overlook this important requirement.

What is Actually Recyclable

I wrote a JavaScript form validation framework for a project some time back that failed horribly over the course of three projects due to my comfort with its complexity. And I was in no way a “newb” programmer at the time I developed the code. Form validation essentially came down to three components:

First, there was a messaging bus which allowed me to drop JavaScript message objects onto a function and have them organized and displayed to the user with pretty colors and effects.
Second, I wrote a component to highlight form fields in different ways, and to present contextual messages near the elements.
Third, I wrote a component to attach handlers to a form and validate the data, cancelling the form submission, running its own handler when validation failed.
I was intent on keeping these parts separate so that I could switch them out as different projects required. This was not a bad thing in and of itself, but it turned bad like prom night with the quickness.

As it turned out, it was too complex.  The three components had to be tied together each time, and that sucked.  I wanted to be able to switch out components with minimal incremental effort. Unfortunately, the significant effort involved in tying everything together was too much to ask from our developers.

In the middle of developing a component that we wish to reuse we are actually handicapped by the knowledge of what the component does and how it does it. When we get feedback from other developers, their impressions are often not as descriptive as we need them to be to be able to refine our code to be more reusable. What seemed like perfectly acceptable amounts of coding to me (heck, I just wrote the thing) was a bear-like chore for others, so I ended up implementing all of the validation—a chore that was horrible itself because, as it turned out, they were right about it requiring too much coding.

Validation required much of the fine-grained control that I had written, so I decided that my failure had to be one of documentation. I assembled some pages that defined the API and had example code for common use cases, but that didn’t fix anything. After watching Chris and David not use it, I left it for a while, came back, forgot how it worked, and realized how awful it was. I looked at what I was doing and realized that I was essentially rewriting much of the same code over and over. So I built a new type of validator, the SimpleValidator.

The SimpleValidator allows you to assign a function to a named element in a form, and if that function returns a string, the validator fails and gives the user all the string collected in this method.  Everything else happens magically, behind the scenes.  To create a SimpleValidator, you called one of the factory methods on the constructor that tailors things to your concerns.

The last thing I did was make a library of functions for common validation “schemes”, like email addresses, names, numbers, zip codes, et cetera, to streamline the process further. I avoided coupling the components by simply defining the factories for the different object in a SimpleValidator settings object.

That worked well 98% of the time. For the other 2% we could still dig through my documentation and figure out how to add the element via the older interfaces so that the required behavior could be accomplished at the cost of making the coupling concrete—a tradeoff we accepted happily.

Our final version was reusable because it had factories that corresponded to use cases. Adding a simple form validation script required little more than a list of what elements were of what type. Extending this behavior was easy as well, as the developer could simply wrap the existing validation function in another function which had further behavior defined. And since the low level mechanics were still all there and documented, when times got rough we simply reverted to that API and were able to achieve our desired behavior. I’ve been referring to this property of a software component as having a “degrading API”.

If we did not have this low level access to the component, the product would be abandoned when times got tough, new solutions would be introduced into the code base, and along with them we’d be adding confusion as to which component to use where. This happened with our security component some time back, and it was horrible. During testing no one knew what was going on when a security fault occurred. We just ran around flailing and screaming until someone would hug us down from our panic. [It's true, we prefer to be hugged down from our many, many panics.—Ed]

With form validation, we did not just create a degrading API for developers to work with, we created the right API for them to work with. In short, we read their freaking minds. You’re a bit spooked by this thought, I can sense that. Calm down—deep breaths. I’ll show you how.

Ensuring Recyclability – Communication

One way for those unfortunate souls among us who cannot read minds to arrive at acceptable component boundaries and API is to talk to the other developers in the team and get their input.

For another project we needed to build a star rating system. This means we needed an input for users to select a number of stars, and an ouput that would show the number of stars an item has received. After some time with Photoshop, I decided to use one image of stars in different states. I’d reuse this image for each of the 5 stars and have hidden radio buttons storing the state of the object. After talking with our JavaScript guy (me) about if I should do this in PHP or in JS, we decided JS was the way to go. You’d be surprised the number of times an internal dialogue-turned-external has solved problems AND frightened everyone else in the office to their very core.

Next I talked to the developers who would be working with this object and the CSS guy who would be styling this site. The developers liked the idea of just using an input element and having everything happen magically, but the stylists and JavaScripters were concerned about having low level access to the internals. We ended up doing the following:

• Created a settings singleton that can contain much of the complexity of defining each Stars object (This allows developers to use the default settings, modify it to suit their own needs, and forget it, Ron Popeil style.)
• Created a constructor in JavaScript that allows for an arguments object where we can override the settings singleton. This way developers can keep their settings in a convenient, easy-to-use object.
• Created a static factory method which converts an input element of type=”star” into our model, copying the required attributes (such as class, style and id) as necessary.
• Created a static method that trawls (getElementsByTagName) the DOM’s input elements and replaces the type=”star” ones with Stars objects.
• Allowed setting a “disable” attribute in the arguments of the constructor, or in the settings, or searched for a disabled attribute on the input element, which caused the dynamic behavior not to be added making it a presentation only element.

This allowed our HTML guys to write things like <input type="star" name="rating" value="4.5" size="5" /> and <input type="star" value="4.5" size="5" disabled="true" /> and have it automagically created. The code is semantic—you understand what it is doing by reading it.   The JavaScript people (that’s me again) demanded error reporting through window.console if it exists, so we added that as well. Then we documented the crizazzle out of it. In this way, the API also degrades across our levels of concern for customization.

Using <input type="star" name="{element name}" value="{float}" size="{max}" /> seemed to encapsulate the required logic for incorporating star-ratings into a web form to such a degree that it became almost memetic. Using any other way to define the star-ratings seemed unnaturally complex and offputting. It was easy to integrate the feature all across the project and into other projects. That was the sure-fire sign that we had something good.

Ensuring Recyclability – Composition

Whenever I hear “API,” my mind automatically goes to a dark place of interface definitions and return type declarations for some class. This is fine for when we’re addessing a concern unique to a specific class, but if we wish to add functionality across many classes we run into trouble with that dark visage.

One place in which we’ve seen this play out is in the Observer pattern. The Standard PHP Library (SPL) has interfaces for SplSubject and SplObserver, and this leads people down the natural road of inheritence. Indeed, many projects use a God class from which all other objects extend. This sucks for three reasons I’ll explain and a few more that you can discover for yourself.

First, since late static binding is still due in PHP 5.3, there is difficulty in knowing what class you’re in if you’re not in an instance.
Second, it sucks because everything gets that logic whether you want it to or not.
Third, it sucks if you want to use two such projects since they each demand their own “one true God” class. It is quite the jihad to get them to work together, ha ha, no I kid, I kid.  But seriously, if you make the God classes extend another class, then updating your libraries will overwrite the changes and you’re stuck going through re-applying the changes, and it’s nothing short of a crusade. [Way to diversify your religious warfare references.—Ed]

The alternative is to implement the interface each time. The book “PHP Objects, Patterns, and Practice” by Matt Zandstra (which is an excellent book for which I have both the first and second editions) suggests doing just that (pg 204, 2nd ed.) But such an approach is so unpalatable that there is talk of adding mixins to the PHP language proper so that you can define functions once and have them mixed into your object definitions as a kind of pseudo-multiple-inheritence. We couldn’t wait for the language to change, so we had to come up with a solution then and there.

Martin Fowler talks about the often underused power of composition in PEAA, and we decided that his books had enough pages to be right. We decided to create an Event object that would handle all of the event interactions and be composited within our existing objects. (Later on we discovered that Symfony uses a similar approach.)  This made adding event capabilities to existing classes very simple. We could even decorate (via extension) most classes that we had no control over and give them event support.  In the worst case situation, we could have an event registry that dispatches the proper event object for each object given as argument. 

Our composition programme took about four hours with unit tests. First, we made the constructor protected and created a factory function that took a string or object, and an array of event types and returned an event manager. The factory always returns the same object (===) for whatever string or object is passed in as an argument and registers the event types with the event manager. When it registers the event types it adds new ones as they appear in the array. This means that if a parent constructor creates an event manager, and then our constructor “remakes” it, there will still only be one event manager linked to our object and it will have a valid list of all the available event types as well.

We created the event manager with an array access interface. The interface maps the name of the event to an event host that was contained in the manager. The host implements SplSubject and is the point of contact for the event. Then we added the SplObserver interface to our function reference class and demanded that event registrars use SplObserver.

The end result is that we can add event handling capabilities to a class by simply making a decorator with an Event_Manager $events property, and we can cause things to happen as simply as $this->events['onToString']->fire(); and $Obj->events['beforeSave']->attach( $FuncRef );.  We also created an event class that is passed to the listeners. This event allows the listeners to cancel the event, perform actions on the firing object, or modify the values being output (like Intercepting “Filter” from Alur et al).

The Carrying Cost

The cost I most overlook when it comes to software reuse is the carrying cost—that is, how much it costs to bring your reusable components into your workspace and make use of them.  On the web, this problem is significantly larger because we’re often straddling multiple languages that may lack an engineered solution for ensuring the proper resources are made available to the browser.  PHP’s autoload functionality is nice, but don’t ask it for an image referenced in a JavaScript include.  You’ll just be disapointed (like junior prom).  A further problem for PHP is that we’re building up and tearing down everything each time, so we need methods of importing components to be shallow and cachable.

The problem I kept having was that I needed to pass information from PHP’s environment to JavaScript imported into an XHTML document, and in that JavaScript document, I needed to reference other resources that were relative to the script file.  CSS gets this right in that references to images in external CSS documents are read relative to the CSS document path.  JavaScript, surprise, does no such thing.  Plus, some scripts need to occur at certain locations in the document (which can be surmounted by the event stack in IE or the event queue in Firefox; if you feel the urge to cut yourself, you’re doing it right.) Finally, some scripts must be included before others, but I didn’t want to throw errors when someone imported one component before another. 

We built something called Axon, which, for lack of a better description, allows for something like Java’s import statements.  A package definition file (there can be any number of them) describes resource names like “Synapse.Forms.Validation” and maps them to required resource names (dependencies) and resource paths associated to channels.  A channel was the end resource type, such as “CSS” or “JavaScript”, and mapped onto the Loader to manage which things are imported.  Further, Axon recursively met the dependencies to ensure things entered the environment in the proper order.  Lastly, we added a service definition so that a resource could describe itself as providing, say, “sendEmail”, and allowed the loader to be queried for a service.

Axon “worked” in that it made using our messaging bus system and form validation code easy. We used it everywhere.  It failed in that we never used services, and no one remembers if caching is on or off, causing comically bad situations where bug fixes don’t work or, alternatively, performance tanks.  Both great options.

Also, I was pretty much the only person to ever write things into Axon, and I never documented it well.  We ended up using the PEAR style name-to-directory convention convention for PHP objects and directly modifying our templates for most code for what I am told are “performance issues.”

The toe-hold that Axon had with the bus and form validation meant that it stuck around, and we slowly began to add more things to it that were useful (although the issues with caching persisted.)  But the down side of sticking around is that we’re stuck with it.  No mana-jerks want to budget time to give it an overhaul, and we developers couldn’t agree on how to fix it.    Currently we’re rotating out a portion of our Canopy framework in favor of Zend’s framework so we might abandon Axon altogether (although we’ll keep the name for something else), but if we weren’t in such turmoil and tumult we might well be stuck with Axon ex infinitum.

Hard Choices About Recycling

Some people (probably from the Drupal project) will note that composition has more performance overhead than directly defining the observers and subjects using the SPL interfaces. Yes, that’s true, especially in PHP. But that’s nothing compared to the performance overhead of PHP itself—you can make Java do some wicked fast things if you’ve got the time to write it. But we write in interpreted languages like PHP because it speeds development and deployment. That leaves us with some hard choices to make. For code recycling to work, like real recycling, it has to save money.

For projects like Drupal and WordPress, performance is an issue since people will be adding to them, and if the project is a performance hog to begin with, these additions will bog down performance further and the product will become unusable.  But performance is neither paramount nor sacrosanct.  You can spend real cheese on optimizing a system where adding more hardware may be a much cheaper solution.  Many projects we work on are fluid—that is to say, they require the ability to quickly and cheaply add new features and change according to their communities needs.  The added time for developers to work through the mess of optimization often is more expensive than just adding hardware to compensate for less-than-stellar code.

Now I’m not advocating solving all performance issues by throwing hardware at it.  That’s a terrible idea.  Terribly terrible.  But look at the cost of resources and where your project is going.  It may make sense (apologies for what I’m about to say) to prototype and start with PHP and then create a Java-based solution when the project is mature (maybe a few years later).

As PHP developers we find ourselves in a community that prides itself on open source. That pride means that every single one of us has downloaded WordPress, Drupal and Magento, looked at the source, and wondered “What the F double hockey sticks is this about?” With open source, the reuse ballgame is much wider. We’re targeting and being the targets of a much larger community of developers.

In open source projects, your code has to make sense to users from China to Oklahoma and let me tell you, there are some cultural divides there.  The documentation has to be written and translated.  And the scale of the larger open source projects usually means that reuse is limited to building your own items into their pre-existing frameworks. I always hope to find a tier separated from display so that I can envoke and operate on an application’s data object, but it rarely ever works out that way. Perhaps it’s a case of cyber-myopia, but the successful large OSS projects seem to emphasize ease of deployment and customization over individual component reuse. This is not the case for many of the projects self-identifying as “frameworks”.

There is an explosion of PHP frameworks and JavaScript frameworks, and the ones that have made themselves available as toolkits as well as frameworks (or, as David likes to call them, “component frameworks”) have been able to weather the storm (and garner more attention).  Similar to my form validation class, I think success in the open source world requires a low cost to deployment, degrading API’s and proper separation of concerns.

By making your code open source and available you can get feedback worth it’s weight (if printed on gold paper) in gold. Further, people who use the project will also make improvements. I posted a REST interface to Amazon’s S3 in response to a request on the Zend frameworks list some time ago, and the requester kindly made it function with streams (which I knew nothing about at the time). In relying on the (all too) true fact that there are people way smarter than me on the internet, we both got something very useful.

Conclusion

My psychic abilities tell me you’re wondering why this wall of text was worth your time. It probably wasn’t. (This is especially true if you’re Martin Fowler, man am I ever wasting your time, Martin, and I’m sorry.) For PHP developers especially, notice that as we make our code more meaningful ad face, we may be sacrificing speed on the processor but we’re gaining huge amounts of development time, bug tracking time and fees for busted keyboards and restraining order hearings. That’s the hard sell—as a developer we have to deal with costs, and as PHP developers we have the advantage of being on an already “slow” interpreted language rather than writing assembly into our C++ for speed.

By focusing on reuse we gain savings directly and indirectly, as reusable components serve as a vocabulary to discuss a project.  We all feel the need for it, but it’s difficult to secure a way to ensure reuse. In an effort to sum up, I am suggesting requiring the following in your own projects:

• Require that concerns be addressed one domain at a time, and composite those solutions as needed
• Focus on writing semantic code (for which the meaning is obvious from its structure)
• Document your code and include example use cases
• Documentation should always include a description of what you are trying to achieve as well as what you are actually achieving with your design.  Be honest
• Use composition over inheritence when adding features to multiple classes from the past, present and future (mainly when spanning different codebases)
• Allow for high level uses via factories
• Allow for low level uses via constructors and factories.
  (When PHP 5.3 lands, don’t let your constructors be public)
• When something will be constant for a long time, provide a place to put that data once and reuse it, don’t demand it each time
• Talk to people who will be developing with this piece of code, or pretend you are them, or actually be them and learn what concerns they have
• Imagine refactoring an older project to use this code. Ask yourself what problems you forsee, and how can you make design decisions now to prevent these problems?
• Release your code to a community that can use it (and ensure they can use it by releasing it at GPL3 or better.) Encourage feedback and participation
• Drink a lot of vodka and red bull and black out when you code so that you can approach it with a fresh mind the following day
• Request comments at every stage,  examine the gap between what they expect and what you’re planning on building, but ignore feature requests that would cause you to drift off target.  Do not try to complete components all at once, rather let it be something to which you come back.  Don’t rush it, just let it happen, baby
  
• Do not fear trading small amounts of performance for semantic structure. You’re not developing mainframes or nuclear stockpile simulators, you’re building web apps in PHP and more often than not the most expensive thing is you, so pamper yourself
• Be able to say that something is not a good candidate for re-usability.  If, over the course of time, you begin to notice yourself re-writing it again and again in a way that can be abstracted, then go for it, but it’s okay if not everything is part of a library
• Be willing to use (or willing to consider using) existing libraries for reusable components.  It will get you in the mindset of your community and provides you with a bunch of tools you don’t have to build yourself

If you do these things, you’ll get rich quick working from home.  No, not really, you won’t, at least you probably won’t, because the results I’ve cited are not typical.

In fact, in the spirit of this article I hope any reader who makes it to this point writes a post on their own experience—what’s worked and what’s failed.  My psychic abilities tell me that several of you read this and said “whoa, that’s a recipie for a performance disaster,”  and when projects explode in popularity this criticism is most certainly true.  My “Highly Available Enterprise Application Architecture with PHP” post is twice as long and three times as ornery.  Also it is entirely written in Tamarian allegory regarding Darmok and Jalad.  But with Thrift, Smarty-like templates, EC2, a smart caching strategy, APC, memcached and a little shuffling of resources you can rebound.

Now go hit the vodka red bulls and let us know your thoughts.

Towns, Buildings, Construction

So let’s talk about patterns. Specifically, I’d like to talk about the MVC architecture pattern, and some thoughts I’ve had recently regarding its application. Let’s first clear the air a bit, make sure everyone is limber, and define our terms. It may be unknown to some that the seminal work in patterns, A Pattern Language: Towns, Buildings, Construction, was actually about architecture (the kind with buildings) and urban planning, not software design. One of the key observations the author makes in the book is that many cities which are laid out in a practical and attractive manner follow a similar pattern, which was varied depending on the exact situation. This is exactly how patterns in software architecture work; Patterns are elegant solutions to common problems that are flexible enough to be applied to differing scenarios.

The Venerable MVC

MVC is one of the most common patterns, especially in web development. MVC is an architecture pattern, which means that it defines several distinct responsibilities in an application, and describes a set of objects, each of which fulfills one of those responsibilities. Specifically the MVC in MVC stands for model, view and controller. Different implementations define the responsibilities differently, but roughly they are defined as follows: (that’s right, I’m so semantic I’m about to use the <dl> tag)

Model

Represents the data used by the application, tends to also hold some amount of business logic. Also facilitates persistence (often by using or encapsulating some kind of data abstraction layer).

Controller

Handles events, may make changes to the model.

View

Generates an appropriate interface (such as an HTML page).

This isn’t quite the whole story, and I’m not really interested in telling the whole story, but I’d like to add a little bit more. Most MVC applications will use a front controller to dispatch incoming requests. The front controller will itself use a variety of helpers to actually invoke Controllers and Views, but the diagram below represents a simplified lifecycle diagram for a typical MVC application implemented in PHP.

Fear and Doubt

A question that ought to arise after a brief period of thought and maybe a few sips of your coding-beverage of choice (mine is usually a latte or Dr. Pepper) is, “Where in the HELL am I supposed to put my business and application logic?” There has been a fair bit of discussion and ensuing misinformation about this one on the interwebs.

Some recommend that business logic be placed in the controllers. Controllers (both Front and Application), however, exist to control program flow; they do things like delegating routing, creating views, and invoking models. Furthermore, if a developer implements application logic in their controllers, they have essentially coupled the method of accessing a feature to the implementation of that feature. This means that changing URL patterns, or allowing different URLs for accessing the data in different ways requires code duplication, or ugly hacks to the routing system.

From there, then, it would appear to make sense to implement business logic on the model itself. Recall, however, that the model is really about encapsulating the data. Putting business logic in the model allows direct access to that data, rather than access to an interface. This results in a tight-coupling between the model and application logic. Reuse is hurt, because functionality specific to an application gets built into the model. Later users who require a different set of functionality but wish to access the same model are forced to either rewrite the models, or live with a bunch of crufty code written for the previous project. Logic which works on multiple objects clearly does not belong in any of the objects it uses. It becomes clear that we must take a different tack.

A Different Tack

Since we want to decouple application features from program flow, as well as from the underlying data they access, the solution is to implement some fourth class that allows access to those features, rather than to program flow, or to data. In Core J2EE Patterns, which is a great read and the inspiration for this post, this is referred to as the application service pattern. An application service object allows access to application features, hiding the underlying logic and abstracting the data access further. Note that use of an application service does not exclude direct access to models when it makes sense; it merely provides another option when logic should be decoupled from the model. We now modify the above sequence diagram to demonstrate the application service pattern in action.

Note that addition of the ApplicationService, which sits between the Controller and the Models. When the Controller needs access to a feature provided by an application service, it accesses that ApplicationService. On the other hand, when more direct access to the data is required, the option to access the model directly still exists.

Handling Online Payments: An Example

By now, I’m sure you’re thinking one thing: “What is all this hibbity-jibbity and gobbledygook?” Let’s do an example. Suppose you’re handling payments on your website. You have a transactions table which stores the payment status, the email addresses of the parties involved, and transaction id received from the gateway, and an invoices table, which stores the amount due, invoice due dates, and so on. You also have the corresponding models, which encapsulate simple CRUD operations on this data.

When the payment status changes, the server receives an XML formatted request. When this happens, a script should verify the data received by posting it back to the gateway server. If the data received is valid, it should store that information in the transactions table, and update the invoices table to reflect the new information. It makes sense to implement the validation functionality on the Transactions model, because that functionality operates only on the data encapsulated by that model, and because there isn’t a case where you would want to use that model without using the validation functionality.

It doesn’t make sense, however, to implement the logic that decides what to do once the data has been validated on the Transactions model. A payment gateway is something very likely to be reused, but the invoice\billing module will differ from project to project. It also doesn’t make sense to implement this logic on the Invoices model either, because that would couple the invoices to the payment gateway, and violate separation of concerns.

It therefore makes sense to implement an application service that handles this logic. The sequence diagram below represents the program flow using the application service pattern as described. Note that the controller still uses the Transaction model directly to access its data, before sending it to the view.

Blog Post Requirement: Fulfilled

So, we’ve fulfilled my blog post requi—er, defined a more elegant way of providing access to application services. To summarize the factors that lead us to the application service, we wanted to provide complex application features that operate on multiple objects without polluting those objects or increasing tight coupling between them. By abstracting these features away from the model, we enhanced reusability of the models and preserved separation of concerns. Similarly, by removing functionality from the controller, we decoupled application functionality from program flow.

Please contribute your comments and experiences with MVC and other patterns in developing for the web.