Code a real-time survey with HTML5 WebSockets

This article first appeared in issue 217 of .net magazine - the world's best-selling magazine for web designers and developers.

WebSockets are a part of the HTML5 specification, and the simplest way of looking at them is that they set a standard for creating long-running bi-directional connections between a browser and a server. With this connection in place, browsers can send messages back to the server (like quicker Ajax) and, even more crucially, the server can push new information to connected clients as it becomes available. WebSockets allow extremely low latency communication with little overhead.

There have been techniques for pushing data to connected clients for many years, and these have been grouped under the term Comet (see en.wikipedia.org/wiki/Comet_(programming) for details). However, WebSockets create a far simpler and standardised way of doing this, and are likely to revolutionise the way that people build real-time interfaces. At this stage, WebSockets are compatible with several modern browsers and there are Flash fallback options available for those that have yet to jump on board.

Why use them? WebSockets enable you to build truly real-time user experiences much more easily. People have already become accustomed to a feeling of immediacy in the applications they use regularly, and want to feel connected to other people using a service. The rise of Facebook chat, multiplayer online games, Twitter’s real-time updates and collaborative editing via Google Docs have paved the way for the applications of the next few years. These types of user experiences are going to grow in popularity, and WebSockets will be the technology behind them. It’s going to become increasingly important to know how to build these sorts of interfaces.

How to use them

So how do we use this awesome new technology? Unlike some of the other parts of the HTML5 spec, WebSockets need both a client and a server component. They’re therefore not as trivial to get up and running as with things like video, canvas, geolocation or form controls. However, don’t let that you put you off, as there are loads of options to get started.

The first method available is to install and maintain your own socket server. There are open source projects such as Socket IO, which are fairly easy to get started with. However, it’s not always simple to install these kinds of socket servers in your average hosting environment because of imposed hosting security restrictions and reduced resource allocations for the long-running connections that WebSockets require.

Another option is to use a hosted service that integrates with your existing website infrastructure. This is what my company, Pusher, provides.

For the tutorials, I’ll use our service because the WebSocket API is a bit ‘bare bones’ and an abstraction layer can make the communication between client and server easier to understand.

The abstraction layers can also manage common scenarios such as handling connections being dropped and reconnection, and can add richer features and functionality too. For example, a library that wraps WebSockets can be considered similar to the way jQuery wraps access to basic DOM manipulation; think document.getElementById(“myDiv”) vs. jQuery(“#myDiv”).

What do they look like?

When we say that WebSockets are a part of the HTML5 specification, it isn’t necessarily clear what that means. In practice, this doesn’t mean a change to the HTML of the page, but rather the functionality is exposed via some new JavaScript APIs. The first thing that is new is a WebSocket object that is initialised with the URL it needs to connect to:

var ws = new WebSocket('ws://example.com');

This WebSocket object has several events that you can use to hook into the rest of your code. The main events occur when a connection is opened, when it closes, and when new messages arrive.

socket.onopen = function(evt) { alert("Connection established") } socket.onclose = function(evt){ alert("Connection closed") } socket.onmessage = function(evt){ alert("I got data: " + evt.data) }

You can send data over the connection too:

socket.send( "Have some data" );

From the limited code above, hopefully you can see how Twitter could, for example, establish a WebSocket connection that would receive new tweets as they happen, so they could be displayed on the page.

Creating a real-time survey

To demonstrate a more detailed use, I’ve taken an example of a real-time survey. If this was a full application, there would probably be several things I’d do differently, but it will allow me to show how WebSockets are useful.

In this example, our application asks the simple question ‘What is the best language?’ and provides buttons for the user to make their choice. A graph is displayed above the buttons that tracks the popularity of the various answers. What makes this more interesting is that you can see the results change in real- time as people in other browsers vote.

I'll be using Ruby on the server, and will try to keep things pretty simple so we can focus on the WebSocket interaction. I’ll also use Pusher for the WebSocket server, since it adds fallback support for older browsers. The source code can be found on Github. For those who’d like to start by building the basic application, have a look at the Getting Started section. If you’d rather jump into adding real-time functionality, download the basic application.

Getting started

First, we need to create our basic survey Rails application. We need a database, a SurveyEntry model, a SurveyEntries controller and a view to display results. It’s easy to do this in Ruby on Rails. Open a command prompt or terminal window and enter the following commands:

rails new realtime_survey – create the new application
cd realtime_survey – navigate into the new application directory
rake db:create – create the database
rails generate model SurveyEntry choice:string – create the SurveyEntry model
rails generate controller SurveyEntries index create – create the SurveyEntries controller with actions called index and create
rake db:migrate – create the survey_entries table in the database for the SurveyEntry model

An index.html.erb file should have been created within the /app/views/ survey_entries folder. This represents the view for the index action of the SurveyEntries controller. If present, delete the index.html file found within the app/public directory and update the /config/routes.rb so that the index action within the SurveyController is called by default.

RealtimeSurvey::Application.routes.draw doroot :to => "survey_entries#index" resources :survey_entries end

If you run the rails server command from the realtime_survey directory and go to http://localhost:3000 you’ll see the contents of the index.html.erb view. Let’s also add some code that fetches the number of votes per language from the database by adding a method to the SurveyEntry model (survey_entry.rb). We’ll define the programming language options in the model too.

class SurveyEntry < ActiveRecord::Base FORM_OPTIONS = ['ruby', 'php', 'python', 'java', 'other'] def self.get_results counts = SurveyEntry count({ :group => :choice }) FORM_OPTIONS.map do |o| { 'title' => o, 'votes' => counts[o]||0 } end end end

Then we should call this code from our controller and make the survey entry data available to our view.

class SurveyEntriesController < ApplicationController def index @survey_entries = SurveyEntry.get_results end def create SurveyEntry.create(:choice => params[:choice]) redirect_to '/' endend

Finally, we should update our survey entries index view to display the survey entry results and the programming language options.

<table id="results"> <% @survey_entries.each do |o| %> <tr> <td><%= o['title'] %></td> <td><%= o['votes'] %></td> </tr> <% end %></table> <form action="/survey_entries" method="POST" id="surveyEntries"> <% SurveyEntry::FORM_OPTIONS.each do |o| %> <p><button name="choice" value="<%= o %>"><%= o %></button></p> <% end %></form>

Once we have a table within our HTML page, we can use progressive enhancement and apply CSS to make things look nicer and improve the page layout. We can also make the results data more visually appealing using a jQuery plug-in to convert the tabular data into a chart.

To convert the table to a chart we need to add the jQuery library (jquery. com), the jQuery chart plug-in and make a simple call to create the chart to the HTML page:

<script src="/javascripts/jquery-1.6.1.min.js"></script><script src="/javascripts/jquery.charts.js"></script><script>$(function(){ $(“#results”).chart();}); </script>

In the form, we’ve a button for each choice, and when clicked, the value of the button will be sent to our server. In Rails, the controller code to receive the vote looks like the following:

def create SurveyEntry.create(:choice => params[:choice]) redirect_to '/' end

The SurveyEntry.create method is inherited from the ActiveRecord class (ar.rubyonrails.org/) and persists the choice to the database. ActiveRecord can also handle our form validation for us but we won’t go into that here.

Once the record has been saved, the user is redirected to the originating page and the index action. The form submission could also be progressively enhanced using an Ajax request, but standard form submission will do for now. We won’t stop people from voting more than once, but a real application probably would.

Adding real-time updates

We add real-time updates to the application by introducing a WebSocket connection so we can push the survey results from the server to all the connected browsers whenever they change. In this example, we’ll use Pusher for the WebSocket server, because configuring a socket server is rather beyond the scope of this tutorial.

Pusher also provides a REST API that allows you to send messages to connected clients from your existing language of choice. However, many of the principles from Pusher can easily be applied to a more generic WebSocket installation. Pusher has free ‘sandbox’ accounts to trial the functionality, so if you feel like trying this example you’ll need to sign up there.

There will be two parts to this WebSocket integration:

• In the client-side code we’ll have to establish a WebSocket connection and listen for messages coming out of it.
• On the server, we’ll have to broadcast new versions of the data when someone makes a change.

Client-side integration

The first thing we need to do is establish a WebSocket connection with the Pusher server. Pusher has a JavaScript library that handles this for you. You just need to include some JavaScript on your page, and create a new Pusher object with your public API key:

<script src="http://js.pusherapp.com/1.8/pusher.min.js"></script><script> var pusher = new Pusher('YOUR_API_KEY');</script>

Pusher also uses the concept of channels so a single connection can receive many different filtered streams of data. In our case, we need to subscribe to only one channel for our survey, which we will call survey-channel:

var myChannel = pusher.subscribe('survey-channel');

If our application were to include multiple surveys, we might name the channels after them. Behind the scenes, a WebSocket connection is being established to Pusher, and our application is now listening for messages.

Dealing with messages

Pusher wraps messages in a simple JSON protocol that makes them very easy to deal with on the client-side. As we saw earlier, a WebSocket object generally has only one event listener: onMessage and anything beyond a simple application becomes difficult to manage. The following example demonstrates this using a hypothetical scenario where items are added to a list:

var ws = new WebSocket('ws://ws.example.com')ws.onMessage(function(data){ $('#myList').append(data);});

If you wanted to be able to cater for items that had to be removed from the list or that had updated, the data object would need to be inspected to find out whether the event indicated an item had been added, updated or removed and your code could start to become more complicated.

We call messages sent to Pusher ‘events’, and they are similar to the evented UI programming you may be familiar with in JavaScript. For example, when a mouse moves it fires a mousemove event with data about the position of the cursor.

With Pusher, when an event happens on your server (eg someone registers a survey entry), this event is broadcast to the connected client (we’ll add this server code later).

You can use your own naming conventions to handle different events. In our example, we’ll be binding to an event called data-changed, which will supply us with new survey results data. It will look like this in our JavaScript:

myChannel.bind('data-changed', function(data){ updateResults(data) }); function updateResults(data){ var tbody = jQuery("#results tbody"); var html = ""; for (var i=0; i < data.length; i++) { html += "<tr><td>" + data[i].title + "</td>" + "<td>" + data[i].votes + "</td></tr>"; } tbody.html(html); jQuery(".chartscontainer").remove(); // remove old chart jQuery("#results").charts(); // redraw }

This code will update the table body HTML, remove the current graph and create a new one for the data we’ve received. We can also use a jQuery notice plug-in (code.google.com/p/jquery-notice/) within the updateResults function to double-check that the user sees the results have updated.

function updateResults(data){ jQuery.noticeAdd({ text: 'Results updated', stay: false }); var tbody = jQuery("#results tbody"); var html = ""; for (var i=0; i < data.length; i++) { html += "<tr><td>" + data[i].title + "</td>" + "<td>" + data[i].votes + "</td></tr>"; } tbody.html(html); jQuery(".chartscontainer").remove(); // remove old chart jQuery("#results").charts(); // show updated chart }

Triggering ‘events’

We now have a way of dealing with new data as it comes in, but we also need to write a little bit of code on our server application that sends it out. Because our application is written in Ruby, we can install the Pusher RubyGem, which will help us with this. There are many libraries in various languages that you can use to communicate with Pusher. In our case, we just need to install the gem using gem install pusher and require it in our code. An easy place to include this is in config/environment.rb, where you can also put in your Pusher credentials:

require 'pusher' Pusher.app_id = 'YOUR APP ID' Pusher.key = 'YOUR KEY' Pusher.secret = 'YOUR SECRET'

Ensure that the Gemfile has been updated to also include pusher:

gem ‘pusher’

In the controller, where we previously wrote the results to the database,we will now add a line that broadcasts the updated survey data to all connected clients by triggering an event. We’ve already established the channel we’ll be broadcasting this to, and the event name we’ll be using:

def createSurveyEntry.create(:choice => params[:choice])Pusher['survey-channel'].trigger('data-changed',SurveyEntry.get_results.to_json) redirect_to '/' end

Now comes the fun bit. Open the survey application in two browsers and register some votes in the first window. You will see a notice in the second that a vote has been received and the graph will redraw. This is a pretty crude example, but it demonstrates some of the power of this technology.

Conclusion

In this tutorial we first created a simple Ruby survey application. We enhanced the basic look and feel of the application through progressive enhancement by applying CSS and a jQuery charting visualisation. We then bettered the user experience further by adding real-time functionality to the application using HTML5 WebSockets and Pusher.

I’m sure this tutorial has shown the power of HTML5 WebSockets and how easy it can be to add real-time functionality to any existing app or have real- time enabled in your app from day one. In this tutorial we demonstrated server to client communication and it’s important to remember that WebSockets allow bi-directional communication. I’m really excited about this technology and have already seen loads of great examples of how it can be used (pusher.com/demos), but the technology is still young and I’m convinced that we are going to start to see a new generation of web apps and games powered by HTML5 WebSockets and its bi-direction communication capabilities.