I wanted to add a live notification function to the chat, as soon as someone is typing, three dots should appear next to the user’s name. This means that as soon as the user starts we need to send a message to the server, this message is then broadcasted to all the connected clients. To not flood the server and the users, maximum one message per second should be sent.

Keydown – time since last message >= 1 second, or this is the first keydown in the sequence => send message
Keydown – time since last message < 1 second => do nothing

There is a built in method called Throttle that does almost this, we could use it, but then the first keystroke would get delayed and I did not want this. So being a beginner at RX I struggled for a while trying to get to where I wanted. I came up with a solution that did what it I wanted, but it was not so elegant. Then I found this post on StackOverflow, with a solution by Sergey Aldoukhov that solved the problem in a very elegant way, so I just decided to use it instead. Sergey had created an extension method called OneInTime, this is the code:

1. Rx.Observable.prototype.OneInTime = function (delay) {
2.     return this
3.         .Take(1)
4.         .Merge(Rx.Observable.Empty().Delay(delay))
5.         .Repeat();
6. };

So what does this mean? Take(1) is the simple part, we take 1 event from the stream. The merge expression is used to merge multiple IObservable streams into single IObservable stream. Each value on the source streams is projected into the result stream until all source streams complete. The Repeat method with no argument will repeatedly sub­scribe to the same observable collection indefinitely. So by taking one, merging with the delayed empty stream and then calling Repeat(), we take one event from the stream, and then skips all the events during the next second. After one second we repeat. Exactly what we wanted to achieve!

And this is how I would use it in the chat (#msg is the text input):

1.  
2. $("#msg").toObservable("keypress") // Create an observable from the keypress event
3.   .OneInTime(1000)
4.   .Subscribe(function () { chat.userTyping(); });
5.  

Pretty neat right?

The code is live at http://signalrchat.apphb.com/ and you can find the source on GitHub.

• SignalR.Server – .NET code running on IIS
• SignalR.Js – A SignalR client in javascript
• SignalR.Client – A SignalR client in .NET. Useful for communicating with a SignalR server from, for example, a Windows Phone application.

SignalR is using long polling as a method to keep its http connections open. In the future support for WebSockets is going to be added as well (a guess would be when .NET 4.5/IIS8 is released, it will have support for WebSockets).

Let me show you how simple it is to create a chat server. First start Visual Studio and create a ASP.NET MVC3 application. When you have created your project you want to add SignalR from NuGet, so open up the NuGet manager and search for SignalR. You need the SignalR.Server and SignalR.Js packages.

When SignalR has installed we need to create the actual server, so lets add a new class, MyConnection. This class will handle the asynchronous communication with the clients:

To route the traffic to MyConnection we need to modify Global.asax slightly:

And the last part, the UI. Put this into your Home/Index.cshtml view:

Alright, that should be all. Compile and run it! You can test out my chat live over at AppHarbor:

http://signalrchat.apphb.com/

https://chrome.google.com/webstore/detail/lffoakndkbfaoglgbnaagjglhhlapdnk?hl=sv&hc=search&hcp=main

Comet – using browser connections

Comet is a web application model in which a long-held HTTP request allows a web server to push data to client browsers, without the browser explicitly requesting it. It works the way that the web browser requests an url from the server, and the server deliberately keeps it open, allowing it to work as a stream of data between the server and the client web browser. In this stream the web server can push content to the client.

Keeping connections open for such a long time is not how HTTP requests was intended to be used, so a lot of people have been describing this as a hack. The biggest problem with Comet has been that the HTTP 1.1 specification states that a web browser should not have more than two simultaneous connections to a web server. By using Comet, one of these connections would always be held open for push notifications. Only having one available connection could limit the browser. An example could be that the browser may be blocked from sending a new request while waiting for the results of a previous request while loading a series of images. Most sites solved this by creating a separate sub domain for the real time communication. Also, getting Comet to work properly in all web browsers isn’t at all trivial. So what other options are there?

Comet – using Flash sockets

Flash sockets is based on the same idea as Comet. On the server side it is very similar to the regular use of Comet, but on the client side the connection to the server is kept open by using a Flash Socket running in a flash application. Since the HTTP connection is opened by Flash it will work the same way in all web browsers, but the downside is of course that it requires Flash, so many mobile browsers will not be able to use it. The Flash socket implementation is considered as more stable and not as much as a hack as the browser Comet implementation. However, it is considered a temporary solution until most web browsers have implemented support for WebSockets.

WebSockets – The replacement for Comet

WebSockets is a technology for providing bi-directional, full-duplex communications channels, over a single TCP socket and is supposed to be able to replace the different Comet solutions that exists today. The API is being standardized by the W3C and is currently implemented in Chrome 4+ and Safari 5+. It is also implemented in Firefox and Opera, but disabled by default. As usual, Internet Explorer is lagging behind and no support for WebSockets exists as of today. Maybe in Internet Explorer 10, who knows… So it will be a while until WebSockets are so widely accepted that we can only rely on WebSockets.

On the server side

To allow for Comet connections the web server needs to support long lived HTTP connections. Most web servers were not built for this, so implementing Comet on a web server such as IIS can be a bit of a challenge, especially when scaling to lots of concurrent users.

Holding lots of concurrent connections open is something that Node.js is very good at (I blogged about Node.js a few weeks ago), and not supprisingly there are many Comet notification servers built on it. The two most popular implementations seems to be Juggernaut and Faye. Lets get into a crash course in how to start using Juggernaut.

Juggernaut

Juggernaut is a push notification server for WebSockets with a easy fallback to Flash sockets if the clients web browser does not support it. Getting started with Juggernaut is very easy, I got my server up running in just a few minutes. Let me show you how to do it on OSX.

First, if you don’t already have Homebrew (a Unix package manager for OSX) installed, go ahead and install it:

$ curl -LsSf https://github.com/mxcl/homebrew/tarball/master | sudo /usr/bin/tar xvz -C/usr/local --strip 1


Now install Node.js, Redis (key/value data strore server, the Juggernaut server and the Juggernaut gem (if you want the Ruby support). Follow the instructions for each package.

$ brew install node
$ brew install redis
$ npm install juggernaut
$ gem install juggernaut


Now start Redis and Juggernaut:

$ redis-server &
$ node ~/node_modules/juggernau/server.js &


Juggernaut is by default hosted on port 8080 and it should now be possible to browse to http://localhost:8080/application.js. This is the javascript that Juggernaut clients should load in order to communicate with the server.

Now lets create an HTML file, jugtest.html, and host it on a local web server:

Also, let’s create a simple Ruby script, test.rb, to send notifications:

Everything should be setup now, let’s browse jugtest.html, my webserver runs on port 3000, so for me that is http://localhost:3000/jugtest.html. Nothing shows up in the web browser, but in the terminal running Juggernaut we should be able to see that one client has connected. Lets run the ruby script:

$ ruby test.rb


The message shows up in the web browser, it works!

As you can see getting started with push notifications using Juggernaut is very easy. If you have any experience with other push notification servers, please leave a comment, it would be interesting to hear your experiences!

The fact is that Javascript is a very powerful language, unfortunately it has had a bad reputation for many years. I admit, for a long time I also looked at JavaScript as a second-class citizen in the programming world. A programming language used only to glue together HTML pages, while the heavy lifting was done on the server side in C#, Java, Ruby or in any other “real” language. However, my view of Javascript has drastically changed during the last few years. The language have “grown up”, and by that I don’t really mean the language itself, instead I am thinking about the libraries and tools available for Javascript, such as jQuery, JSON, Node.js, and HTML5. jQuery was a major game changer on the client side and now we have Node on the server side.

Node is a low level framework for building high performance web applications. It contains of an interpreter based on Google’s V8 and a framework library. HTTP is a first class protocol in Node and it is very easy to get a web server up running. The main difference from traditional frameworks is that Node is event driven (asynchronous). While most webservers use synchronous thread pool based request handling, Node server will only run one event loop which will handle all incoming requests using callbacks. Asynchronus web servers are proved to be very fast, for example the Nginx and Lighthttpd web servers, and Node is not an exception.

Let me show you what the simplest web server in Node looks like:

Running the program will start a web server listening on port 1234 and visiting http://localhost:1234 will show a “Hellow World!” message to the visitor.

The main difference when writing code for Node is that you have to understand is that blocking IO should be avoided. Everything runs in parallell except your code, writing blocking code will prevent Node from serving other requests. This is an example of writing blocking code:

The above example listens to two urls http://localhost:1234 and http://localhost:1234/longrun. When first visiting / it loads fast. Loading the /longrun takes 10 seconds as expected (think of this as database query or some other IO operation that takes long time), but it also causes Node to block and not serve new requests until /longrun has completed. To avoid this you should use non blocking IO at all times. By doing this, Node will be able to process other requests while waiting for the request to complete.

For a more in depth introduction to Node I can recommend the short book “The Node Beginner Book” and also the blog post “Understanding the node.js event loop”

To get started with Node I suggest you head over to Github and follow the intallation instructions to install Node and npm (the package manager for Node) https://github.com/joyent/node/wiki/Installation.

To install it on ubuntu type:

sudo apt-get install apache2-utils

Once installed we can use it to create an htaccess file.

htpasswd -c -b /path/to/htpasswd NewUser NewPassword

Now we need to add one block to the nginx config. Bellow is the config for this site, the part in bold is what needs to be added.

server {
    listen   80;
    server_name sourcecodebean.com;

    index index.php index.html;
    root /path/to/web/root;

    location ^~ /secret-dir/ {
        auth_basic            "Restricted";
        auth_basic_user_file  /path/to/htpasswd;
    }

    location / {
        try_files $uri $uri/ /index.php?q=$uri&$args;
    }

    location ~ \.php$ {
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        include /etc/nginx/fastcgi_params;
        fastcgi_pass  127.0.0.1:9200;
        fastcgi_index index.php;
        fastcgi_param  SCRIPT_FILENAME  /path/to/web/root$fastcgi_script_name;
    }
}

Woho! All done!

When I first read about AppHarbor I got the impression that is was running ontop of Microsoft Azure, which turned out to be all wrong. AppHarbor is running in the Amazon Cloud on EC2 instances. When it comes to databases AppHarbor provides access to MySQL and MSSQL. Interestingly they recommend users to use MySQL:

“we expect to be able to offer MySQL on faster hardware and with more redundancy than SQL Server at the same price point. If you are planning to build an app on top of AppHarbor and have the option, we recommend you go with MySQL.”

Another interesting choice is that they have chosen Nginx as their proxy/load balancer.

Ok, let me show you how easy it is to create a simple ASP.NET MVC 3 application and deploy it on AppHarbour The first step is to go to appharbor.com and register an account. Once created you will be able to create your first app:

Give your application and press create. You have done all the configuring you need to do at AppHarbor.

Lets create a MVC 3 project with unit tests:

Make sure your project builds and that the tests passes. Also make sure you have Git installed, if not you can fetch it from here. Now open a shell and cd into the folder where you created your application.

First we need to configure git with your name, the email you used to register at AppHarbor and last the git postBuffer size:

PS C:\dev\appharbor\MyCoolApp> git config --global user.name SourceCodeBean
PS C:\dev\appharbor\MyCoolApp> git config --global user.email mail@sourcecodebean.com
PS C:\dev\appharbor\MyCoolApp> git config --global http.postBuffer 52428800

After this we initiate the git repository:

PS C:\dev\appharbor\MyCoolApp> git init
PS C:\dev\appharbor\MyCoolApp> git add .
PS C:\dev\appharbor\MyCoolApp> git commit -m "initial commit"

And when this is done we add AppHarbor as a remote repository:

PS C:\dev\appharbor\MyCoolApp> git remote add appharbor https://moberg@appharbor.com/mycoolapp-1.git

We are now set to push our code onto AppHarbor. To push it just type:

PS C:\dev\appharbor\MyCoolApp> git push appharbor master

We should now be able to see the project building:

Hopefully it built successfully. A problem I ran into was that I had ASP.NET MVC 3 RC installed and AppHarbor had the RTM version installed, this caused my first attempt to build my project to fail. After upgrading my local version and fixing the project I pushed a new version that built without problems.

We can now browse to the site (yay! that was way easier than setting up IIS manually):

Failing tests

Ok, lets change something in our application that will make a test fail:

Now we are going to be bad and not run our unit tests before commiting and pushing the code

PS C:\dev\appharbor\MyCoolApp> git commit -a -m "a failing test"
PS C:\dev\appharbor\MyCoolApp> git push appharbor master

.. and we are building again

Woot! The build has failed!

We can easily see that it is the test that verifies the text on the start page that has failed

Ok, lets fix it!

Commit and push it again. And wait for the build to complete.

Yay! Our new version built successfully!

Lets check it out:

Rolling back to a previous version

Being able to rollback an update is sometimes necessary and AppHarbor makes it ridiculously simple, all you need to do is to press the “Deploy” link next to an old version. Clicking it will build and deploy the selected version.

We can now see that it is the first version that is active

I love the idea behind AppHarbor, this is really how sites should be deployed! However, using AppHarbor also means giving up some control over your environment. It will be interesting to see how they handle, for example, upgrades between different versions of IIS, installation of service packs, etc. Two features i miss is the ability to run background jobs and a distributed cache to use, but hopefully it will be coming to AppHarbor. AppHarbor still a very young startup, and so far what I have seen is really impressive!

Installing Nginx and configuring the site
The first step is to install the Nginx webserver and PHP.

apt-get install nginx php5-cli php5-cgi spawn-fcgi
update-rc.d nginx defaults
/etc/init.d/nginx start

Then we need to create the configuration for our site.
/etc/nginx/sites-enabled/sourcecodebean:

server {
        listen 80;
        server_name www.sourcecodebean.com;

        location / {
                rewrite ^/(.*) http://sourcecodebean.com/$1 permanent;
        }
}

server {
    listen   80;
    server_name sourcecodebean.com;

    index index.php;
    root /var/www/sourcecodebean.com/public_html;

    location / {
        try_files $uri $uri/ /index.php?q=$uri&$args;
    }

    location ~ \.php$ {
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        include /etc/nginx/fastcgi_params;
        fastcgi_pass  127.0.0.1:9000;
        fastcgi_index index.php;
        fastcgi_param  SCRIPT_FILENAME  /var/www/sourcecodebean.com/public_html$fastcgi_script_name;
    }
}

The first server section is only there to redirect www.sourcecodebean.com to sourcecodebean.com. It would be nice if Nginx supported conditional location blocks, then this extra server block would not be needed. But from what I understand this is not supported.

The second bock has two important parts

    location / {
        try_files $uri $uri/ /index.php?q=$uri&$args;
    }

This part handles the URL rewrite and is specific to WordPress. If don’t need URL rewrite you can leave this part out.

 location ~ \.php$ {
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        include /etc/nginx/fastcgi_params;
        fastcgi_pass  127.0.0.1:9000;
        fastcgi_index index.php;
        fastcgi_param  SCRIPT_FILENAME  /var/www/sourcecodebean.com/public_html$fastcgi_script_name;
    }

This section tells Nginx that all requests to files ending with .php should be routed to the PHP-FastCGI binary running on localhost.

Configuring PHP-FastCGI
The second step is to get PHP-FastCGI working. First lets create a script to start it. Lets place it in /usr/bin/php-fastcgi. The script should contain the IP, port and what user the fastcgi process should run as. This is what my script looks like:

  #!/bin/sh
  /usr/bin/spawn-fcgi -a 127.0.0.1 -p 9000 -C 6 -u www-data -f /usr/bin/php5-cgi

The -C controls how many child processes that should be spawned. Make sure to make this script executable.

Now we need to configure so it automatically starts when the system starts. I found a simple init script in the linode library, so I am going to use that.

wget https://library.linode.com/web-servers/nginx/php-fastcgi/reference/php-fastcgi-init-deb.sh
mv php-fastcgi-init-deb.sh /etc/init.d/php-fastcgi
chmod +x /etc/init.d/php-fastcgi
update-rc.d php-fastcgi defaults
/etc/init.d/php-fastcgi start

Everything should now be up working. Lets test it by creating a simple php file, test.php, and placing it in your web root.

<?php phpinfo(); ?>

Now browse http://yourdomain.com/test.php and hope that you get the phpinfo page!

• The build process should be totally automated
• Building the site should result in a release package
• The build server should run unit tests, code analysis and test code coverage every time the build is run
• The deployment process should be totally automatic (for the test environment and for the production environment)
• It should always be possible to rollback an update (files, database, IIS-settings, etc.). Rollback should also be fully automated.

I am very happy to say that we have fulfilled all our above goals, let me tell you have we did it.

Build process and the release package

We are using TeamCity as build server. TeamCity is great build server which support numerous different build agents and it hooks up very easily with our Mercurial version control system. This is what a typical check in/build/package scenario looks like:

1. 1. A developer checks in new code
2. 2. TeamCity picks up the change from Mercurial and starts a build
3. 3. Once built TeamCity runs all the units tests, code analysis and code coverage
4. 4. If the unit tests are successful the TeamCity creates a deploy package. The package contains a MSDeploy zip file, configuration files and our deploy tool chain (Powershell scripts and a few executables) – everything you need to deploy the site.

By downloading and extracting this file you get a file structure that looks like this:

Scripts – contains various scripts needed to perform deployment, the most important being Deploy.ps1 and Deploy-Local.ps1.

SiteConfig – contains shared settings and one directory per unique environment (AT = Acceptance Test, ST = System Test, Dev-PM = My development environment). In the SiteConfig directory we also find the following files:

• settings.xml
• episerver-config.master.xml
• episerver-config-site1.xml
• episerver-config-site2.xml

The settings.xml is the base of all non-episerver settings. It contains connection strings to databases, paths to where the web site should be installed, where logs should be placed, where VPP files are placed, integration settings to external systems and much more.

In each environment specific directory, such as “AT” it is possible to override these settings by creating a settings.xml. The new settings.xml does not replace the one in the parent directory, it is merged with it and the environment specific file takes precedence over the general.

A simple example let pretend this is our \SiteSettings\settings.xml:

<settings>
  <webDir>D:\WebSites\</webDir>
  <logDir>D:\Logs</logDir>
</settings>

Say that I would like to override where the logs are stored in my ST environment. To do this I would create \SiteSettings\ST\settings.xml, containing:

<settings>
  <logDir>C:\Temp\Logs</logDir>
</settings>

The merged config would be:

<settings>
  <webDir>D:\WebSites\</webDir>
  <logDir>C:\Temp\Logs</logDir>
</settings>

We use this for both development and deployment. When a developer checks out the source code, there is a “Init.ps1” script in the Scripts folder (not included in the release package). This script will setup the website according to the specified settings (in my case the Dev-PM folder). The script configures the local IIS server, creates app pools,  creates the sites, sets permissions on folders, creates local MSMQ queues, configures NServiceBus and so on. Thanks to this script all the developers have a development environment that is nearly identical! Super sweet!

Ok, back to deployment. As you might have figured out, the episerver-config.master.xml serves as base for all episerver sites. Since this is an enterprise installation we can have several sites and each –siteX represents such a site.

All the settings in the files are overridable in the same way as with settings.xml. We use this to, for example, provide the appropriate license file for the specific environment.

SiteWebDeployPackage

This might be the least exciting directory, it simply contains a MS WebDeploy package created by MSBuild. This package contains everything needed to run the sites, except for the site specific configuration files that Deploy.ps1/Deploy-Local.ps1 will put in place.

Tools
This directory contains our “deploy tool chain”. The tool chain includes, MSDeploy.exe (WebDeploy), SimpleScriptRunner.exe (DB migration), 7za.exe (zip) and a bunch of other tools needed to setup and install Msmq and NserviceBus.

The deploy process

When we started implementing the automated deployment we were hoping to be able to do most of the heavy lifting in WebDeploy. But after a lot of research and testing we came to the conslusion that it was probably possible to do what we wanted to do in WebDeploy (WebDeploy is very powerfull), but we could not figure out how. The WebDeploy documentation at that time was almost non-existent and had few advanced examples. Instead we made the decision to use PowerShell for most of the work and use WebDeploy only as a method to package transfer files.

This is the steps needed to perform a deployment:

1. 1. Download the latest release package from TeamCity
2. 2. Extract it, start PowerShell and cd into the Scripts folder
3. 3. Run “Deploy.ps1 <environment>”. The addresses of the servers are confgured in the environment settings

This is what Deploy.ps1 does:

1. 1. Remotes to the service bus host and takes it offline
2. 2. Remotes to the webserver and takes the site offline
3. 3. Creates a backup of the entire IIS containing all its settings and sites using WebDeploy (used if rollback is carried out)
4. 4. Creates a backup of all databases (used for rollback)
5. 5. Uses SimpleScriptRunner to connect to our database server and apply database schema updates on our databases (not the EPiServer database)
6. 6. Copies the deploy package to the webserver
7. 7. Remotes to the webserver and runs Deploy-Local.ps1. The script installs the files, configures IIS, copies license files, etc.
8. 8. Starts the site and the service bus and runs runs integration tests
9. 9. Print result and exit!

If the update of some reason is not working as it should, the update can be easily rolled back by executing “Deploy.ps1 <environment> -Rollback”. This will restore all affected system to exactly the same point as they were before the update.

Conclusion

Automating the build and the deployment process is absolutly worth the effort it takes to implement it. By automating the process we have been able to catch more bugs, develop and release faster and at the same time make the entire process less dependent on certain people in the group – our knowledge is documented in the build and deployment scripts.

A very nice bonus from this is that we can version control everything related to the site, IIS settings, EPiServer settings, EPiServer Licenses and much more. Also, installing the release on a “empty” server is as easy as updating an existing installation.

The tools that made all this possible:

Microsoft WebDeploy – Deployment tool from Microsoft.

Psake- Build automation tool written in PowerShell (inspired by rake in Ruby).

SimpleScriptRunner - Tool to run a directory of numbered sql scripts against an MSSQL database.

OpenDBDiff - Tool/library to generate SQL-diff scripts.

PageTypeBuilder – Automatic synchronization of pagetypes in EPiServer (page types are defined in code)

The first thing you need to do is to install WebDeploy. It can be installed using Web Platform Installer or by downloading the .msi file. I prefer to download the .msi. It can be found here. Download the installer and run this (you can change the port and the address to whatever you want):

msiexec /I WebDeploy_x64_en-US.msi /passive ADDLOCAL=ALL LISTENURL=http://+:80/MsDeployAgentService/


Then start the service:

net start msdepsvc

To check if the service is up running execute:


c:\program files\iis\microsoft web deploy\msdeploy.exe -verb:sync
-source:dirPath='d:\some\local\path'
-dest:dirPath='f:\some\remote\path',
computerName=http://xxx.xxx.xxx.xxx:1234/MSDeploy,
userName=remoteUser,password=remotePassword -verbose


If the service is up running a lot of information will be printed to the screen, if not, MSDeploy will complain on that the remote service did not respond with “v1″ of MSDeploy.

The sync script

Bellow is a simple PowerShell script that has two functions.

Send-Files – Synchronizes a local folder to a remote server.

Execute-RemoteCommand – Executes a batch file on a remote server. The file must already be in place on the server.