source code bean

Two new features I really like in PHP 5.3

Peter — Sun, 06 Jun 2010 16:40:09 +0000

Namespaces
Finally PHP has support for namespaces, not a day too late! Before the days of object oriented PHP name clashes for functions was usually prevented by adding a prefix to your function names. When object oriented PHP was introduced function name clashes was less likely, but class names still had to be unique, ie you could only have one class named for example “User”. This led to the need of prefixing class names. From PHP 5.3 we no longer have to do this!

Namespaces in PHP introduced two new keywords:

namespace – define a namespace
use – Use a namespace
use … as – Use a namespace but give it an alias name

Late static binding

Lambda functions and closures
Lambda functions, or anonymous functions as they are called in PHP, is really something I have missed in PHP. Lambda functions are very convenient to use when you are dealing with callbacks, for example:

echo preg_replace_callback(‘~-([a-z])~’, function ($match) {
return strtoupper($match[1]);
}, ‘hello-world’);
// outputs helloWorld
?>

Closures are also a very welcome addition to the language. A closure is not the same thing as an anonymous method, which seem to be a widespread misunderstanding. This is the definition from wikipedia of what a closure is:

“In computer science, a closure is a first-class function with free variables that are bound in the lexical environment. Such a function is said to be “closed over” its free variables. A closure is defined within the scope of its free variables, and the extent of those variables is at least as long as the lifetime of the closure itself.” - Wikipedia

In PHP closures will allow us to define closures like this:

$sayhito= function($name)
{
printf("Hi %s\r\n", $name);
};
$greet(‘Foo’);
$greet(‘Bar’);
// This will print
// Hi Foo
// Hi Bar
?>

Pretty sweet!

Yahoo Pipes is still very cool

Peter — Thu, 06 May 2010 21:12:04 +0000

I know that Yahoo Pipes is not a new thing, it came out in 2007 so in web years it is almost a 100 years now, but it is still darn cool! If you somehow have managed to miss what it is, this is how wikipedia describes it:

“Yahoo! Pipes is a web application from Yahoo! that provides a graphical user interface for building data mashups that aggregate web feeds, web pages, and other services, creating Web-based apps from various sources, and publishing those apps”

So why I am blogging about this in 2010? Well, it is still really useful. For many years I have been following the CodeSOD section on the blog “The daily WTF”, the rest of the stuff on there is funny as well, but I enjoy the CodeSOD the most. On the blog there was no option of subscribing to only one section, so i deiced to create a Pipe to feed me the CodeSOD only.

The user interface is super easy, just drag items from the list on the left and connect them together:

My CodeSOD pipe can be found here if you want to subscribe to it!

First steps toward EPiServer and ASP.NET MVC

Peter — Mon, 19 Apr 2010 20:33:38 +0000

Recently I started a new assignment at a customer where we are going to build their new public web site and their intranet, both of them based on EPiServer CMS 6. The person leading the project is very found of ASP.NET MVC, so my first task was to experiment with EPiServer and ASP.NET MVC, to try to get a running prototype up.

I did some research online to see what others had done. I found two great posts by Joel Abrahamsson and one from Fabio Fabrizio, who based his solution on Joels experiments.

Joels first attempt was to create a base class from which all Controllers inherited. However, this made the controller do more than it should and made it very hard to test. His second attempt was to create a custom model binder and make every action receive currentPage as a parameter. This approach makes it much easier to create unit tests for the code, since it is now possible to moc the currentPage parameter.

Fabio took a different approach and decided to implement a custom MVC handler and a controller factory. This approach fits very well with MVC and felt like the right way to go, so I decided to create a prototype based on Fabios code. Fabios code was based on EPiServer CMS 5 and ASP.NET MVC 1.0, but I wanted to use EPiServer 6 and ASP.NET MVC 2.0, so I started porting the solution.

Porting the solution to EPiServer 6 and ASP.NET MVC 2.0 required some changes to both code and configuration, which not all of them was trivial. At the moment I have a prototype up running, and it works remarkably well! I will continue my experiments and post my results.

EPiServer friendly URLs for paginated pages (and why the asp:LinkButton must die)

Peter — Mon, 15 Mar 2010 06:05:37 +0000

Recently I got a request from a customer to perform some search engine optimizing for an old EPiServer site we are maintaining. One of the optimizations was to fix the paging on their product page. The products are fetched from an external data source and are not stored as pages in EPiServer, thus normal EPiServer paging controls can not be used. Image a normal paging control like this:

Prev 1, 2 , 3, …, 99 Next

A fairly normal approach to this would have been to use a query parameter for handling the paging. Like this:

By using this approach each page would have a unique entry point, using query strings is however not optimal for SEO, but even worse is using asp:LinkButtons. Whoever created the site had decided to use asp:LinkButtons. LinkButtons are very convenient to work with in ASP.NET, but the HTML code they generate is not very SEO friendly. This is the HTML that was generated:

href="javascript:__doPostBack(‘ctl00$MainContent$rptPages$ctl00$lbPage’,”)">1

href="javascript:__doPostBack(‘ctl00$MainContent$rptPages$ctl01$lbPage’,”)">2

href="javascript:__doPostBack(‘ctl00$MainContent$rptPages$ctl02$lbPage’,”)">3

href="javascript:__doPostBack(‘ctl00$MainContent$lbNext’,”)">Next

We can see that:

LinkButtons generates a javascript that is run when the link is clicked, instead of using a normal link
Paging is handled using a postback, the pages will no longer have unique entry points.

This is disastrous from a SEO perspective.

So lets fix it. First step is to replace the LinkButtons with normal links (asp:Hyperlink) – and then we are going to do some URL rewrite magic to create SEO-friendly URLs. The code behind was changed to assign links like /products/?page=4.

Now, to get URLs like /products/page/4, we can create a custom url rewrite module in EPiServer. First we create a new class, Rewrite, that inherits from EPiServer.Web.FriendlyUrlRewriteProvider. In this class we override three methods (don’t ask we why they have such confusing names, someone at EPiServer must have been under the influsene of something when nameing them):

ConvertToInternalInternal - Used to convert from /product/page/4/ to an internal EPiServer page with the page id as a query parameter
ConvertToExternalInternal - Used to convert from an internal EPiServer URL (/PageType.aspx?lotsofqueryparameters=values) to an external (/products/page/4/)
ConvertToInternal - Needed to work around URL rewrite caching behaviour in EPiServer, will get into more detail on this later.

This is the class:

namespace Utils
{
public class Rewrite : FriendlyUrlRewriteProvider {
// The regexp to match a paged url
string _regexpPaging = @"(.+)/page/([0-9]+)/$";
protected override bool ConvertToInternalInternal(
UrlBuilder url, ref object internalObject){…}
protected override bool ConvertToExternalInternal(
UrlBuilder url, object internalObject, Encoding toEncoding){…}
public override bool ConvertToInternal(
UrlBuilder url, out object internalObject) {…}
}

The first method we implement is the ConvertToInternalInternal which will convert the URL to an internal EPiServer URL.

protected override bool ConvertToInternalInternal(UrlBuilder url, ref object internalObject)
{
if (url == null)
{
return false;
}
// Regexp to match if the URL ends with /page/{Id}/
Match match = Regex.Match(url.Path, _regexpPaging);
// If we have a match, remove the /page/{Id}/ from the end of the URL
// and add a querystring to the internal URL that is dqcPagingId
// (should have a unique name to not clash with some other querystring)
if (match.Length > 0)
{
url.Path = match.Groups[1].Value + "/";
url.QueryCollection["dqcPagingId"] = match.Groups[2].Value;
base.ConvertToInternalInternal(url, ref internalObject);
return true;
}
// Now when the /page/{Id}/ is removed from the URL, and the querystring is added, we can let EPiServer do its normal URL-rewriting.
return base.ConvertToInternalInternal(url, ref internalObject);
}

While not nessesary, we should override the ConvertToExternalInternal as well. This will let EPiServer automatically convert internal urls containing the dqcPagingId querystring to a external url ending with /page/{Id}/.

protected override bool ConvertToExternalInternal(UrlBuilder url, object internalObject, Encoding toEncoding)
{
// First let EPiServer convert the internal URL to an external. This will give us a URL like:
// /Products/?dqcPagingId=5 (if it is a paged page)
bool isRewritten = base.ConvertToExternalInternal(url, internalObject, toEncoding);
// Check if the URLs query string contains dqcPagingId
// If it does we add /page/{Id} to the URL and removes the query string
if (url.Query.Contains("dqcPagingId"))
{
url.Path = string.Concat(url.Path, "page/", url.QueryCollection["dqcPagingId"], "/");
url.QueryCollection.Remove("dqcPagingId");
}
return isRewritten;
}

Now one can think we would be done, but there is one more method we need to implement. EPiServer uses a cache to cache URL rewrites from external URLs to internal URLs, but it will only cache the querystrings used by EPiServer, not the dqcPagingId we added. This gives a rather unexpected result. The first time the page is loaded everything loads fine, from code behind we can access the dqcPagingId and show the requested page. If you wait 10 senconds (default cache time) or more and reloads the page, it works fine. But if you reload the page sooner, the dqcPagingId will not be set when the page loads. This is because the querystring dqcPagingId is not cached by EPiServer. This took me quite some time to figure out.

The solution to the problem is to override ConvertToInternal to bypass the default caching:

public override bool ConvertToInternal(UrlBuilder url, out object internalObject)
{
// If the URL end on /page/{id}/, bypass cache by calling ConvertToInternalInternal
// A more optimal solution would be to perform some kind of caching here
if (Regex.IsMatch(url.Path, _regexpPaging))
{
internalObject = null;
ConvertToInternalInternal(url, ref internalObject);
return true;
}
// Else, it is ok to use the cached result
return base.ConvertToInternal(url, out internalObject);
}

The last step is to add our Rewrite module to the web.config:

name="MyUrlRewriter"
enableSimpleAddress="true"
friendlyUrlCacheAbsoluteExpiration="0:0:10"
type="Utils.Rewrite, Utils"
description="URL rewriter for paging" />

Now we should be all done! Enjoy!

Trying out CouchDB for the first time

Peter — Mon, 08 Mar 2010 06:05:59 +0000

There are several good libraries that will abstract accessing CouchDB. However, in order to understand what goes on in the libraries, I think it is important to first understand what is going on on a lower lever, so that is what I will show you.

Step 1, install CouchDB. CouchDB for OSX can be downloaded here, the bundle contains bouth the Erlang runtime and CouchDB – no compiling or installing needed, just download the .dmg, mount it and run the application! If you are on Linux it is quite likely that CouchDB is in the repository of your distribution. In Ubuntu CouchDB is found in “Universe” and can easily be installed using apt. I haven’t tried CouchDB on Windows so I can’t give you any guidance here, if you try it out, please leave a comment.

Now when CouchDB is installed lets explore it using use one of my favorite tools, good old cURL. Port 5984 is the default port for CouchDB.

$ curl -X GET http://localhost:5984/
{"couchdb":"Welcome","version":"0.10.0"}

CouchDB is up running! Lets create a new database:

$ curl -X PUT http://localhost:5984/testdb
{"ok":true}

CouchDB’s responses are also in JSON form. Lets inspect the database we just created:

$ curl -X GET http://localhost:5984/testdb
{
"db_name":"testdb",
"doc_count":0,
"doc_del_count":0,
"update_seq":0,
"purge_seq":0,
"compact_running":false,
"disk_size":79,
"instance_start_time":"1266963717052501",
"disk_format_version":4
}

CouchDB returns some statistics of the database, we can see that it contains 0 documents. Lets store an empty document in the database:

$ curl -X POST http://localhost:5984/testdb/ -H "Content-Type: application/json" -d {}
{
"ok":true,
"id":"ef40feff87010a6ef3a45a16df5af977",
"rev":"1-967a00dff5e02add41819138abb3284d"
}

CouchDB returns the unique id for the document and the version number (did i mention that all documents are version controlled?:)). Next step is to fetch all documents:

$ curl -X GET http://localhost:5984/testdb/_all_docs
{
"total_rows":1,
"offset":0,
"rows":[{
"id":"ef40feff87010a6ef3a45a16df5af977",
"key":"ef40feff87010a6ef3a45a16df5af977",
"value":{"rev":"1-967a00dff5e02add41819138abb3284d"}
}]
}

Yes! it is there! Now you should also be able to see your database “testdb” and the document you created in the CouchDB GUI.

CouchDB a NoSQL database

Peter — Mon, 01 Mar 2010 20:44:29 +0000

Traditionally relational databases has been the primary way of storing, sorting and searching data, and for most purposes they are very good at it. However, in the last few years – with the growth of cloud computing and sites such as Facebook pushing relational databases to its limits, people have started to look for other alternatives. The problems with relational databases is that it has been hard to scale them in a vertical way (scale them over several servers with linear, or close to linear, performance increase). A few databases in a cluster is no problem, but when it comes to several terabytes of data, which should be searched in real time, they are simply not enough. CouchDB and the other NoSQL databases aims to provide a truly horizontally scalable database.

NoSQL is a umbrella term for a wide variety of data stores, which all have in common that they do not store data in a relational way. Some examples of NoSQL databases are CouchDB, MongoDB, Amazon SimpleDB and Google BigTable. This is some of the properties they have in common:

1. No schema
The data is stored in one big hashtable like data structure. No schema is needed.

2. No more joins
Joins are slow in general, when they are spread out over several servers it gets even worse. In CouchDB there is no join, instead data should be duplicated. This might sound odd if you like me was taucht back in collage that normalization is of highest importance, and that you should really really avoid to duplicate data in your database. This is still true for most relational databases, but keep in mind that when relational databases was invented, disk space was expensive and normalization was a great way to save a few bytes. Storage is everything but expensive today, this is why NoSQL empathizes that data should instead be duplicated.

3. Eventual consistency
When you update the database there is no longer a quarantine that all subsequent queries will get the updated value immediately, it might take some time depending on the system load. For some systems such as banking systems, this kind of behavior would be a big no no. But for most large web sites, this is no problem, the data is going to be cached in one way or another anyway.

CouchDB

CouchDB is a document database, accessible via a RESTful JSON API. Everything stored in the database is a “document” and is stored in a flat address space.There are no schemas, the documents are stored and retrieved as JSON objects.

To address this problem of adding structure back to semi-structured data, CouchDB has something called “views”. A view is as close as we gets to a SQL query. The views are expressed in Javascript and consist of map and filter functions. They are built dynamically and will not affect the underlying data.

CouchDB is written in Erlang and runs on all systems that the Erlang runtime supports (Linux, Windows, OSX and other unix systems). I have tested CouchDB on Linux and OSX. This is a screenshot of the CouchDBX GUI running on OSX:

The JSON representation of the same document:

In my next post I will show you some hands on action with CouchDB.

Articles coming up on CouchDB and ExtJS

Peter — Sun, 14 Feb 2010 23:46:24 +0000

It has been a bit too long since my last post so I wanted to give you an short update on what I have been up to lately. Last year I built a web service using Amazon SimpleDB, which got me interested in document based databases. Since then I have been reading and following the progress of different open source document based databases, such as CouchDB and MongoDB.

During the time, I have been thinking about what other kind of projects a document based database would be suitable for. Because my lack of imagination (or just because I was too eager to get started:)), i decided to build a simple Web CMS system using PHP and CoachDB. Pages in a CMS system are perfect for storing as documents in a document database, so it seemed like a good choice.

Building the CMS backend actually turned out to be what took the shortest time. I wanted to create a nice looking Javascript based GUI for the editors and administrators, and decided to build it using the javascript library ExtJS. I had never worked with ExtJS before (and I am more of a backend guy than a javascript frontend guy), so it took me some time to get my mind around it, but the result is sweet – ExtJS is really powerful.

The CMS is far from complete, but so far it has given me quite a few ideas for new blog posts, so stay tuned for the upcoming posts on CoachDB and ExtJS!

Creating a JSON-RPC service using Zend Json Server

Peter — Thu, 07 Jan 2010 08:11:15 +0000

In this post I am going to show how easy it is to create a JSON-RPC web service using the built in support in Zend Framework.

First we need to create the php-file that will handle the incoming RPC calls. It is not advised to put this inside the MVC structure of a Zend web application, since that will lead to unessesary complexity and overhead. The Zend people recommend that we create the JSON-RPC under /public/api/vX/, so lets create the file /public/api/v1/jsonrpc.php (if you haven’t setup your Zend MVC structure, read my blog post Getting started with the zend framework to get started).

We will have to do the regular bootstrapping to get our application up and running:

// Define path to application directory
defined(‘APPLICATION_PATH’)
|| define(‘APPLICATION_PATH’, realpath(dirname(__FILE__) . ‘/../../../application’));
// Define application environment
defined(‘APPLICATION_ENV’)
|| define(‘APPLICATION_ENV’, (getenv(‘APPLICATION_ENV’) ? getenv(‘APPLICATION_ENV’) : ‘production’));
// Ensure library/ is on include_path
set_include_path(implode(PATH_SEPARATOR, array(
realpath(‘../../../library’),
get_include_path(),
)));
/** Zend_Application */
require_once ‘Zend/Application.php’;
// Create application, bootstrap, and run
$application = new Zend_Application(
APPLICATION_ENV,
APPLICATION_PATH . ‘/configs/application.ini’
);
$application->bootstrap();

The next step is to create the class that will be exposed through the service. I will create a very simple class that will simply perform an addition of two ints. It is very important to describe the input parameters using the @param directive in the comment. This information is used by the Json Server when creating the SMD (Service Mapping Description).

/**
* Simple – sample class to expose via JSON-RPC
*/
class Simple
{
/**
* Return sum of two variables
*
* @param int $x
* @param int $y
* @return array
*/
public function add($x, $y)
{
return array(‘result’ => $x + $y);
}
}

The last step to get the JSON-RPC server running:

// Instantiate server, etc.
$server = new Zend_Json_Server();
$server->setClass(‘Simple’);
if (‘GET’ == $_SERVER[‘REQUEST_METHOD’]) {
// Indicate the URL endpoint, and the JSON-RPC version used:
$server->setTarget(‘/api/v1/jsonrpc.php’)
->setEnvelope(Zend_Json_Server_Smd::ENV_JSONRPC_2);
// Grab the SMD
$smd = $server->getServiceMap();
// Return the SMD to the client
header(‘Content-Type: application/json’);
echo $smd;
return;
}
$server->handle();

Now your JSON-RPC Server should be up running. Browsing http://{your web server}/api/v1/jsonrpc.php should result in the following SMD:

{
"transport":"POST",
"envelope":"JSON-RPC-2.0",
"contentType":"application\/json",
"SMDVersion":"2.0",
"target":"\/api\/v1\/jsonrpc.php",
"services": {
"add":{
"envelope":"JSON-RPC-2.0",
"transport":"POST",
"parameters":[
{"type":"integer","name":"x","optional":false},
{"type":"integer","name":"y","optional":false}],
"returns":"array"
}
},
"methods":{
"add":{
"envelope":"JSON-RPC-2.0",
"transport":"POST",
"parameters":[
{"type":"integer","name":"x","optional":false},
{"type":"integer","name":"y","optional":false}],
"returns":"array"}
}
}

jQuery does not support calling JSON-RPC services out of the box, but fourtenly there is plenty of plugins for jquery that fixes this. One is the JSON-RPC client found here. Download the client and put the javascript files into your /js folder. Then create a new file test.html and add the following html:

We are all done! Browsing test.html should result in an alert box containg the result.

Congratulations! You have created a JSON-RPC service!

Upgrading from Zend Framework 1.7 to 1.9 – The autoloader

Peter — Wed, 30 Dec 2009 16:54:56 +0000

Today I upgraded a project we started working on last year from Zend Framework 1.7 to Zend Framework 1.9. I excepted to run into several API incompatibilities, but the only problem I got was the autoloader.

In Zend 1.7, and earlier versions, the autoloader was registered like this:

require_once "Zend/Loader.php";
Zend_Loader::registerAutoload();

In Zend Framwork 1.9 this has changed slightly, you now have to register the namespaces you want to autoload:

require_once ‘Zend/Loader/Autoloader.php’;
$loader = Zend_Loader_Autoloader::getInstance();
$loader->registerNamespace(‘Dqc_’);

This was the only change we needed to do to upgrade from 1.7 to 1.9, quite impressive!

Getting started with the Zend Framework

Peter — Sun, 13 Dec 2009 21:53:00 +0000

It used to be a bit tricky to get started with the Zend Framework. The Zend Framework is very flexible and allows you to set it up in almost any way that fits your needs. This means for example that the directory structure and location of files is up to you, however – there is a recommended layout. When I first started using Zend I had to figure out this by looking at examples and reading the (at that time) rather poor documentation available. In Zend Framework 1.9 a tool, zf.sh, was introduced. It simplifies creating a new site a lot. In this blogpost I will guide you though the process of setting up a Zend development environment in OS X. The only part that is OS X specific is MAMP. Zend Framework runs just fine under Windows and Linux as well.

Step 1 – Getting a *AMP setup (LAMP, MAMP, WAMP)
The AMP (Apache, Mysql, PHP) stack is available for almost all modern operating systems. I am writing this on my Macbook, so in this tutorial I will use MAMP. When I develop PHP in Windows I usually use WAMPServer and in Linux you can install the LAMP-stack using the packaging system in most distributions.

Setting up MAMP is pretty straightforward, download the MAMP .dmg-file and drag the MAMP folder to your Applications folder.

Start the application and press the “Open start page” button to make sure everything works. On the start page you will find information about your site, phpInfo, phpMyAdmin and SqLiteManager. We will get back to configuring the http root directory later.

Step 2 – Download and install Zend Framework
Go to the framwork download page and download the minimal distribution. In this tutorial I am using 1.9.6, but the instructions will probably apply to all 1.9 versions.

Extract the downloaded file and move the folder to a shared location, for example /usr/local/. Next step is to create an alias for the zf.sh tool. Edit your ~/.bash_profile and add the following line (change the path to where you moved the extracted files):

zf=/usr/local/ZendFramework1.9/bin/zf.sh

This will allow you execute the zf tool without using the full path. Try it out by executing zf show version, it should return the version number of the file you downloaded.

$ zf show version Zend Framework Version: 1.9.6

Step 3 – Create your project
Go to the folder where you want to create your new project, in my case ~/Development/. Run zf create project zf-tutorial. This will setup the default directory structure and create the necessary files.

The application/ folder is where the source code for your website lives. It contains separate folder for models, views and controllers. The public/ folder is the folder that is going to be your document root.

Now you need to copy the Zend library (in my case /usr/local/ZendFramework1.9/library/Zend) or create a symlink for it so your site can find the Zend files. I prefer using a symlink:

$ cd ~/Development/zf-tutorial/library $ ln -s /usr/local/ZendFramework1.9/library/Zend/ Zend

Step 4 – Run the project
The last step before we will have running site up is to configure the step that we skipped in Step 1, configuring the Apache document root.

Open up MAMP and click the preferences button, under the apache tab you will find the document root. Click select and navigate to the “public” folder in your zend project. Apply the settings and restart the server.

Now open your browser and direct it to http://localhost and you should see the default welcomescreen:

Now open your application/controllers/IndexController.php and start hacking your code!

If you want to learn more about the Zend Framework i have some posts on some more advanced topics: