Five things you didn't know the web could do

A few weeks ago, I presented "The Web Can Do That!?" at Google I/O 2012:

Unlike some of the other bleeding edge HTML5 talks I’ve given in the past, this year’s focus was to demonstrate practical uses of the latest web tech and show it in action. After all, flashy demos are the bees knees. No one believes a bunch of bulleted lists and hand waving!

In this article, I’m going to dive into the top five most amazing things from “The Web Can Do That!?” If you’re antsy, the source for everything is up on my GitHub and the presentation itself can be found at htmlfivecan.com.

Heads up: Some of this stuff is still very new. Your best bet is to try the slide deck and demos in Chrome 21+.

1. CSS for web apps

CSS has brought us to some magical places, but unfortunately many of its layout and presentation capabilities (I’m looking at you, absolute positioning and floats) fail miserably in the age of modern web development. Problem is, we’re not building websites any more – we’re building apps. The requirements are very different. Many of CSS’s original constructs do a poor job in situations like responsive design.

Lucky for us the CSS Working Group is three steps ahead of us! It's proposed a number of new specs to directly address app layout and design issues. CSS Grid Layout, Hierarchies, Regions, and Flexible Box Layout Module to name a few.

Let’s cover one of these: CSS Flexbox.

Alignment

I’m particularly stoked about Flexbox because it enables me to centre content both horizontally and vertically with three lines of CSS. No more nasty top/left 50% and negative margin tricks.

Here's how it works:

.box { display: +flex; +justify-content: center; +align-items: center; }<section class="box"> <div>A</div><div>B</div><div>C</div></section>

Note: I’m using “+” through this article to indicate a vendor prefix (eg “+flex” should be -webkit-flex, -ms-flex, etc).

This example produces the following layout:

Try it!

display: flex tells a parent container to become a 'flex container'. In the illustration above, Mr. Red <section> is our flex container and its content (the three blue children) are 'flex items'. To pack A, B, and C to the centre of the main axis and cross axis, we justify-content: center and align-items: center, respectively.

Ordering and orientation

It doesn’t stop there. Another fantastic property of Flexbox is that we can fully separate the order of our markup from the way it’s presented. This is achieved through two new CSS properties, order and flex-direction. Order controls the sequence in which sibling items appear. Flex-direction modifies their orientation (row vs column).

Let’s say that instead of a row, we wanted to arrange A, B, and C into a column ... no problem! flex-direction: column arranges the child along the cross axis instead of the main axis. Note, by default items render in the order they’re defined in the markup, but we can easily override that. By giving B a lower order value than its siblings, it will precede the others children.

.box { +flex-direction: column;}.box > :nth-child(2) { +order: -1;}

Produces:

Try it!

Keep in mind, we didn’t change the source at all. It’s still A, B, then C. Flexbox gives us the ability to arrange content independent of how our source is defined.

Flexibility

The bread and butter of Flexbox is its 'flexibility' feature. In addition to alignment, orientation, and ordering, one can also instruct items to grow/shrink to fill the available space around them. This is done through the flex property.

The flex property takes three values. The first is a positive flex value: how much the element can grow compared to its siblings. The second is a the negative flex: how much it’s allowed to shrink. The third value is the desired width of the item.

Modifying our previous example, we can use the flex property to make B three times bigger than its buddies:

.box > * { +flex: 1 0 auto;}.box > :nth-child(2) { +flex: 3 0 auto;}

The result is exactly what you'd expect:

Try it!

Demo

Requires: Chrome 21

The demo illustrates how easy it is to create that 'Holy Grail' layout (header, 3 column, footer) with CSS Flexbox. Best of all, the entire app is responsive. Try resizing your browser window.

Support

The new Flexbox spec is implemented in Chrome 21 and IE 10. I’m told Mozilla is actively working on an implementation as well.

2. One-way data binding

Databinding is something every modern web app needs. Until Web Components’ MDV is a ready, we’ll have to rely on JavaScript frameworks like Angular or Ember to fill in the gaps for us. Or do we?

Angular is one of my favourite MVC frameworks at the moment because it’s simple. I don’t need to learn new APIs or templating syntax to use it. It relies on raw HTML as its templating language and pure JS for controller logic. That, I like!

One-way data binding in Angular is a cinch with a little markup:

<div ng-app ng-init="val=25"> Volume: <input type="range" min="0" max="100" ng-model="val"> {{val}}/100</div>

As the user moves the slider, the model named val is updated and its corresponding template variable is automagically recalculated. Angular does the heavy lifting for us by setting up hidden event listeners and re-rendering our view as the model changes.

Using HTML5 data-* attributes

The very same thing can be done using HTML5 features and a clever trick with :before/:after pseudo elements. However, for the HTML5 case we don’t have the luxury of automatic binding. The process looks like this:

• Model: data-* attribute. Use CSS attr() to get the value.
• View: render generated content to :before/:after pseudo elements.
• How to bind?: Hook up an event listener to watch for changes in the data model.

Implementing this idea, we get the exact same slider as the Angular example, but sans framework!

<style>input::after { content: attr(data-value) '/' attr(max); position: relative; left: 135px; top: -20px;}</style><input type="range" min="0" max="100" value="25"><script> var input = document.querySelector('input'); input.dataset.value = input.value; // Set an initial value. input.addEventListener('change', function(e) { this.dataset.value = this.value; // Update the view. });</script>

Try it!

Using <datalist>

Perhaps a better (and more semantic) way to do one-way databinding in HTML5 is to use the new <datalist> element. Opera and FF have had <datalist> for ages, but it’s only recently landed in WebKit.

The binding process for this one looks like:

• Model: <option> values specified in the <datalist> element.
• View: a regular Joe <input> element.
• How to bind?: reference the id of the data list with the list attribute.

The code is equally simple:

Browsers: <input list="browsers"><datalist id="browsers"> <option value="Chrome"> <option value="Firefox"> <option value="Internet Explorer"> <option value="Opera"> <option value="Safari"></datalist>

Try it!

<datalist> is a great way to specify pre-defined list options for an <input> element. Think an autocomplete form populated from an IndexedDB database.

Support

Believe it or not, all of these framework-free data binding techniques are supported in all modern browsers!

3. Access a filesystem

Most decent apps need file I/O at some point in their lifecycle. The HTML5 Filesystem API brings a proper filesystem to the web. No plug-ins, no fuss. With it, users can persist data to files and folders to a filesystem sandboxed to your web app’s origin.

To open the filesystem, call window.requestFilesystem (vendor prefixed of course):

window.webkitRequestFileSystem( TEMPORARY, // Storage type: PERSISTENT or TEMPORARY 1024 * 1024, // size (bytes) of needed space initFs, // success callback opt_errorHandler // opt. error callback, denial of access);

To be clear, we’re not reading/writing data to the user’s My Documents or My Pictures folders on the native OS. The HTML5 Filesystem can only interact with the sandboxed filesystem that’s created for your app. This also means you cannot modify data in another web app’s filesystem.

A common use case for this API is an AppCache replacement. As an example, dynamically caching an image file is a breeze:

var xhr = new XMLHttpRequest();xhr.open('GET', '/path/to/image.png', true);xhr.responseType = 'arraybuffer'; // We want a byte array, not a string.xhr.onload = function(e) { window.requestFileSystem(TEMPORARY, 1024 * 1024, function(fs) { // fs.root is the root DirectoryEntry for the filesystem. fs.root.getFile('image.png', {create: true}, function(fileEntry) { fileEntry.createWriter(function(writer) { writer.onwriteend = function(e) { ... }; writer.onerror = function(e) { ... }; writer.write(new Blob([xhr.response], {type: 'image/png'})); }, onError); }, onError); }, onError);};xhr.send();

Demos

Requires: Chrome

The Filesystem Playground is a visual UI on top of the HTML5 Filesystem API. The entire app is client-side. Some of the functionality includes dragging in files and folders from the desktop to import them into the web app, create empty folders, preview files, rename items, download them and so on.

The HTML5 Terminal emulates an old-school command line. It too sits on top of the Filesystem API.

After you’ve created a few files, I dare you to try the 3d command!

Support

THe HTML5 Filesystem API is Chrome-only at the moment. It also appears Mozilla is steadfast on keeping IndexedDB as the sole solution for File I/O. In my experience, IndexedDB is far too difficult to use for stashing files and creating any kind of a folder hierarchy. It’s an abuse of the API. In the interim, I’ve created idb.filesystem.js, which polyfills the Filesystem API into browsers that only support IndexedDB. That means any browser that supports IDB should also (theoretically) support the Filesystem API. You’re welcome!

If you want to take a deeper dive into the Filesystem API, check out my book, Using the HTML5 Filesystem API. In addition, give filer.js a look. It’s a handy wrapper library that abstracts the API into UNIX calls such as (cp, mv, mkdir).

4. Access native hardware

You’re probably saying, “Whaaaat!? The web can’t access native hardware”. For the most part I have to agree. Sadly, accessing things like USB, Bluetooth, and UDP are just not possible on today’s web. We’ve seen frameworks such as PhoneGap pave the way here – but the fact remains, the drive-by web doesn’t have all of the APIs developers are foaming at the mouth for.

Device access is hot topic these days; so much so that the W3C formed the Device APIs Working Group in August 2011 to tackle the issue head-on. Recent projects such as Chrome Apps and Firefox OS have also started to explore such endeavours.

What can we do today?

The ability for web apps to leverage high level JS APIs that sit on top of underlying hardware isn’t foreign web. The last few years have brought us a bunch of this kind of stuff:

• Geolocation API (positional GPS)
• Device Orientation API (accelerometer)
• WebGL (GPU)
• HTML5 Filesystem API (sandboxed file I/O)
• navigator.onLine / navigator.connection (network connectivity)
• Battery API
• Gamepad API (USB access to a specific device)
• WebRTC (voice/video processing) / Web Audio API (core system audio)

...the list goes on. Let’s focus on the last one.

Microphone access

Since the dawn of mankind, one of the coveted asks of the web has been proper camera and microphone access (without a plug-in). A first step was the x-webkit-speech property implemented in Chrome:

<input type="text" x-webkit-speech>

This attribute was exciting. With one attribute, we got minimal access to the user’s microphone and a totally new way for users to interact with our app.

Aside: It’s worth noting browsers are expanding on this feature with a more robust Speech JavaScript API in the coming months.

Today we can do better thanks to the recent work in the WebRTC space. At the core of WebRTC is getUserMedia, an API by which enables an app to request access to the mic/camera:

<video autoplay controls></video>window.URL = window.URL || window.webkitURL;navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia;navigator.getUserMedia({audio: true, video: true}, function(stream) { document.querySelector('video').src = window.URL.createObjectURL(stream);}, function(e) { console.log(e);});

What I really like about getUserMedia is that it’s reusing older parts of the platform, namely HTML5 <video>. Instead of setting the video.src to a movie file, we set it to a blob URL created from the camera stream. A live feed. This kind of integration is a great example of older APIs co-existing with newer ones.

Recording video

One thing I’m looking forward to (which is not quite ready yet) is being able to record a LocalMediaStream as it happens:

<input type="button" value="" onclick="record(this)"><input type="button" value="--" onclick="stop(this)">var localMediaStream;var recorder;function record(button) { recorder = localMediaStream.record();}function stop(button) { localMediaStream.stop(); recorder.getRecordedData(function(blob) { // Upload blob using XHR2. });}

Support

Basic support for getUserMedia is in Chrome 20 (enabled in about:flags), Opera 12, and Firefox 17 (currently nightly). The flag is going away in Chrome 21.

5. Stream multimedia

Something I hear a lot from web developers is: “Can HTML5 stream audio?” Turns out, it absolutely can. The secret sauce is a blending of binary WebSockets and the Web Audio API.

For many moons, one could only send string data through a WebSocket. That limitation alone prevented some crazy-cool apps from being built using WebSockets. We have JS Typed Arrays and binary data in the web platform now. Why shouldn’t I be able to stream a file? The spec authors and browser vendors eventually caught onto this little idea and now, implementations include a newer send() method that supports sending binary data.

The concept is similar to XHR2. You simply set the format of the data you’re exchanging. For example, to send a blob/file, set the .binaryType property to blob:

var socket = new WebSocket('ws://example.com/sock', ['dumby-protocol']);socket.binaryType = 'blob'; // or 'arraybuffer'socket.onopen = function(e) { window.setInterval(function() { // Send off data when nothing is buffered. if (socket.bufferedAmount == 0) { socket.send(new Blob([blob1, blob2])); // presumably image data. } }, 50); // rate limit us.};socket.onmessage = function(e) { document.querySelector('img').src = window.URL.createObjectURL(e.data);};

On the receiving end (eg the onmessage handler), we can use the image file (e.data) directly by creating a blob URL from it. No more needing to Base64 encode/decode data on either end! Performance boosts ... me likey!

Streaming audio

Binary websockets allow for some pretty interesting use cases, including streaming audio.

Instead of covering the technique in great detail, you can check out my audio_streamer demo to see how the code works. However, the process is straightforward:

1. On the DJ machine: a) Use the Web Audio API to decodeAudioData() on an entire .mp3 file. b) Once the file is decoded, slice the entire AudioBuffer into smaller chunks. We don’t want to send the whole thing at once. c) Use a simple NodeJS server to send each audio chunk (as an ArrayBuffer) over a binary WebSocket.
2. On the listener’s machine: a) Use the Web Audio API to load + schedule each audio chunk at the precise time when it’s supposed to play. This seamlessly 'recreates' the audio for the listener as if it were a single file.

The result of this workflow is essentially streaming audio ... something that wouldn’t be possible without two new features in the web platform: binary WebSockets and the Web Audio API.

WebRTC data channels

Perhaps the future of file sharing is the DataChannel API from the WebRTC effort. Unfortunately, it’s still being implemented in Chrome and FF. That API aims to enable true peer-to-peer data exchange in a real time fashion.

Support

Chrome, FF, IE 10, and Safari support binary WebSockets. Chrome and Safari are the only browsers supporting the Web Audio API.

Conclusion

It pains me that the whole 'native vs web' debate is still a thing. I’m a web developer. I couldn't care less about what native can do but I care everything about what the web can do! Sure, there are holes in our platform, but we’re addressing those gaps with an ever ever-increasing number of APIs. Things such as Web Components are going to change the way we build web apps. In this spirit, I’d like to see us web folk stop squaring off with the other guys and shift the conversation more around what the web platform can do. A lot of people don’t even realise what’s possible.

Hopefully this article, htmlfivecan.com, and htmlfivewow.com have proven the web can often do more than meets the eye.