Skip to main content

HTML5 Server-Push Technologies, Part 2

April 26, 2010

{cs.r.title}







The upcoming HTML5 specification includes a lot of powerful and exiting features which turn web browsers into a fully capable ich internet application (RIA) client platform. Part 1 of this article series presented an overview of the history of the web, and investigated the new HTML5 Server-Sent Events communication standard.

2.2 WebSockets

The upcoming HTML5 standard also includes WebSockets. WebSockets enables establishing a bidirectional communication channel. In contrast to Server-Sent Events, the WebSocket protocol is not build on top of HTTP. However, the WebSocket protocol defines the HTTP handshake behaviour to switch an existing HTTP connection to a lower level WebSocket connection. WebSockets does not try to simulate a server push channel over HTTP. It just defines a framing protocol on top of TCP. In this way WebSockets enables two-way communication natively.

Like the Server-Sent Events specification, WebSockets specifies an API as well as a wire protocol. The WebSockets API specification includes a new HTML element, WebSocket.

Listing 5. Example JavaScript using the WebSocket interface

<html>
   <head>
     <script type='text/javascript'>
        var ws = new WebSocket('ws://localhost:8876/Channel', 'mySubprotocol.example.org');
        ws.onmessage = function (message) {
          var messages = document.getElementById('messages');
          messages.innerHTML += "<br>[in] " + message.data;
        };
       
        sendmsg = function() {
          var message = document.getElementById('message_to_send').value
          document.getElementById('message_to_send').value = ''
          ws.send(message);
          var messages = document.getElementById('messages');
          messages.innerHTML += "<br>[out] " + message;
        };
     </script>
  </head>
  <body>
     <form>
       <input type="text" id="message_to_send" name="msg"/>
       <input type="button" name="btn" id="sendMsg" value="Send" onclick="javascript:sendmsg();">
       <div id="messages"></div>
     </form>
  </body>
</html>

A WebSocket will be established by creating a new WebSocket instance. The constructor takes one or two arguments. The WebSocketURL argument specifies the URL to connect. A WebSocketURL starts with the new scheme type ws for a plain WebSocket connection or wss for secured WebSocket connection. Optionally, a second parameter protocol can be set which defines the sub-protocol to be used (over the WebSocket protocol). As with the EventSource element, an onmessage handler can be assigned to a WebSocket, which will be called each time a message is received. Data will be sent by calling the send() method.

If a new WebSocket is created, first the underlying user agent will establish an ordinary HTTP(S) connection to the host of the URL. Based on this new HTTP connection, an HTTP upgrade will be performed. The HTTP specification defines the upgrade header field to do this. The upgrade header is intended to provide a simple mechanism for transition from HTTP protocol to other, incompatible protocols. This capability of the HTTP protocol is used by the WebSocket specification to switch the newly created HTTP connection to a WebSocket connection. By adding the optional WebSocket-Protocol header, a specific sub-protocol is requested.

REQUEST:
GET /Channel HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: myServer:8876
Origin: http://myServer:8876
WebSocket-Protocol:
mySubprotocol.example.org


RESPONSE:
HTTP/1.1 101 Web Socket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
WebSocket-Origin: http://myServer:8876
WebSocket-Location:
ws://myServer:8876/Channel
WebSocket-Protocol: mySubprotocol.example.org

Figure 2. WebSocket upgrade handshake

After receiving the response HTTP header, data will be transmitted according to the WebSocket protocol. This means at this point only WebSocket frames will be transferred over the wire. A frame can be sent at each time in each direction. The WebSocket protocol defines two types of frames: a text frame and a binary frame. Each text frame starts with a 0x00 byte and ends with a 0xFF byte. The text will be transferred UTF8-encoded between the start and the end byte. A text frame requires only 2 additional bytes for packaging purposes. Figure 3 shows a text frame for the string "GetDate" and the string "Sat Mar 13 14:00:25 CET 2010".

Text frame of "GetDate":
0x00 0x47 0x65 0x74 0x44 0x61 0x74 0x65 0xFF

Text frame of "Sat Mar 13 14:00:25 CET 2010":
0x00 0x53 0x61 0x74 0x20 0x4D 0x61 0x72 0x20 0x31
0x33 0x20 0x31 0x34 0x3A 0x30 0x30 0x3A 0x32 0x35
0x20 0x43 0x45 0x54 0x20 0x32 0x30 0x31 0x30 0xFF

Figure 3. Text frame example

Binary data can be transferred by using a binary frame. A binary frame starts with 0x80. In contrast to the text frame the binary frame does not use a terminator. The start byte of a binary frame is followed by the length bytes. The number of length bytes is given by the required bytes to encode the length. Figure 4 shows the binary frame for a small number of bytes to transfer which requires one length byte as well as a larger binary frame which requires two length bytes.

binary frame of 0x00 0x44:
0x80 0x02 0x00 0x44

binary frame of 0x30 0x31 0x32 0x33  0x34  0x35 […] 0x39 (1000 bytes):
0x80 0x87 0x68 0x30 0x31 0x32 0x33 0x34 0x35 […] 0x39

Figure 4. Binary frame example

Because JavaScript cannot operate with binary data represented as a byte array, the binary frame type is limited to be used for languages other than JavaScript. In addition the binary frame and text frame, new frames types can be introduced by future releases of the WebSocket protocol specification. WebSocket's framing is designed to support new frame types. A connection can be closed at any time. No extra end-of-connection byte or frame exists.

The overhead involved managing a WebSocket is very minimal. Comet protocols such as Bayeux and BOSH, for instance, uses some "hacks" to break the HTTP Request-Response barrier. This forces such protocols to implement a complex session and connection management. Due the fact that WebSockets is not implemented on the top of HTTP it will not run into trouble caused by HTTP protocol limitations.

On the other hand WebSockets, does almost nothing for reliability. It does not include reconnect handling or support guaranteed message delivery like Server-Sent Event does. Further more, as a non-HTTP based protocol, WebSocket cannot make use of the built-in reliability features of HTTP. For instance HTTP supports auto-retry execution strategies in case of network errors. Based on the fact that a GET method can be executed at any time without side effects, a GET method will be re-executed by browsers and modern HttpClients automatically, if a network error occurs. A GET method must not change the server-side resource state by definition and is therefore safe.

This means reliability has to be implemented in the application (sub-protocol level) when using WebSockets. The same is true for the "keep alive" message approach to avoid a proxy server dropping the connection after a small period of inactivity. Additionally, sharing a WebSocket between different pages often causes trouble. In contrast to Server-Sent Events, a WebSocket also includes an upstream channel which is difficult to share. For instance, concurrent writes and reads have to be synchronized, which is not a simple task. This general challenge also affects bidirectional Comet protocols such as Bayeux or BOSH. When using WebSockets, the per-server connection limitation has to be considered carefully.

As with the origin policy used by web browsers to restrict browser-side programming languages from contacting the server, a WebSocket server will only be contacted if the web page is loaded from the same domain. This is not true for stand-alone WebSocket clients such as shown in Listing 6. Such clients contact the WebSocket server in a direct way without same origin policy limitations.

Listing 6. Example Java Client

class MyWebSocketHandler implements IWebSocketHandler {

  public void onConnect(IWebSocketConnection wsCon) throws IOException {
               
  }

  public void onMessage(IWebSocketConnection wsCon) throws IOException {
    IWebMessage msg = wsCon.readMessage();
    System.out.println(msg.toString());
  }
           
  public void onDisconnect(IWebSocketConnection wsCon) throws IOException {
               
  }
}

       
MyWebSocketHandler hdl = new MyWebSocketHandler ();
IWebSocketConnection wsCon = httpClient.openWebSocketConnection(
    "ws://myServer:8876/WebSocketsExample",
    "mySubprotocol.example.org", hdl);
       
wsCon.writeMessage(new TextMessage("GetDate"));
// ...

Listing 7 shows an example WebSocket Server implementation. The server handler implements two interfaces: the IHttpRequestHandler handles ordinary HTTP requests and the IWebSocketHandler handles WebSocket connections. In the case of a standard HTTP request (without the upgrade request) the IHttpRequestHandler's onRequest() method will be called. If the client opens a WebSocket, the server will handle the HTTP upgrade and call the IWebSocketHandler's onConnect() method. Each time a WebSocket message is received, the IWebSocketHandler's onMessage() method is called.

Within the onConnect() method some preconditions can be checked. For instance if a required sub-protocol is not supported, the example server will return an error status. Furthermore the origin header will be checked. As is the case with the referer header, the origin header will be set by the browser automatically. The origin header is defined by the HTTP Origin Header RFC, which is in draft. In contrast to the referer header, the origin header includes the domain name of the page's source only. Each time embedded code makes a request, the browser adds the origin header which contains the origin page's domain. Web Servers can block requests that send invalid origin headers.

Listing 7. Example Java WebSocket Server

class ServerHandler implements IHttpRequestHandler, IWebSocketHandler {
           
           
  // IHttpRequestHandler method
  public void onRequest(IHttpExchange exchange) throws IOException {
  String requestURI = exchange.getRequest().getRequestURI();
               
  if (requestURI.equals("/WebSocketsExample")) {
    sendWebSocketPage(exchange, requestURI);
                   
  } else {
    exchange.sendError(404);
  }
}
           
           
  private void sendWebSocketPage(IHttpExchange exchange, String uri)
          throws IOException {
    String page = "<html>\r\n " +
                  "  <head>\r\n" +
                  "     <script type='text/javascript'>\r\n" +
                  "        var ws = new WebSocket('ws://" +
                           exchange.getRequest().getHost() + "/Channel',
                           'mySubprotocol.example.org');\r\n" +
                  "        ws.onmessage = function (message) {\r\n" +
                  "          var messages = document.getElementById('messages');\r\n" +
                  "          messages.innerHTML += "<br>[in] " +
                             message.data;\r\n"+
                  "        };\r\n" +
                  "        \r\n" +
                  "        sendmsg = function() {\r\n" +
                  "          var message = document.getElementById
                             ('message_to_send').value\r\n" +
                  "          document.getElementById('message_to_send').value = ''\r\n" +
                  "          ws.send(message);\r\n" +
                  "          var messages = document.getElementById('messages');\r\n" +
                  "          messages.innerHTML += "<br>[out] " + message;\r\n"+
                  "        };\r\n" +
                  "     </script>\r\n" +
                  "  </head>\r\n" +
                  "  <body>\r\n" +
                  "     <form>\r\n" +
                  "       <input type="text" id="message_to_send"
                          name="msg"/>\r\n" +
                  "       <input type="button" name="btn" id="sendMsg"
                          value="Send" onclick="javascript:sendmsg();">\r\n" +
                  "       <div id="messages"></div>\r\n" +
                  "     </form>\r\n" +
                  "  </body>\r\n" +
                  "</html>\r\n ";
               
    exchange.send(new HttpResponse(200, "text/html", page));
  }

           
           
  // IWebSocketHandler method
  public void onConnect(IWebSocketConnection webStream) throws IOException, BadMessageException {
    IHttpRequestHeader header = webStream.getUpgradeRequestHeader();

    // check origin header
    String origin = header.getHeader("Origin");
    if (!isAllowed(origin)) {
      throw new BadMessageException(403);
    }
               
    // check the subprotocol 
    String subprotocol = header.getHeader("WebSocket-Protocol", "");
    if (!subprotocol.equalsIgnoreCase("mySubprotocol.example.org")) {
      throw new BadMessageException(403);
    }
  }

  private boolean isAllowed(String origin) {
    // check the origin
    // ...
    return true;
  }
           
           
  // IWebSocketHandler
  public void onMessage(IWebSocketConnection webStream) throws IOException {
    WebSocketMessage msg = webStream.readMessage();
    if (msg.toString().equalsIgnoreCase("GetDate")) {
      webStream.writeMessage(new TextMessage(new Date().toString()));
    } else {
      webStream.writeMessage(new TextMessage(
              "unknown command (supported: GetDate)"));
    }
  }
           
  // IWebSocketHandler
  public void onDisconnect(IWebSocketConnection webStream)
          throws IOException {  }

}
       
XHttpServer server = new XHttpServer(8876, new ServerHandler());
server.start();class ServerHandler implements IHttpRequestHandler,
        IWebSocketHandler {
           
           
  // IHttpRequestHandler method
  public void onRequest(IHttpExchange exchange) throws IOException {
  String requestURI = exchange.getRequest().getRequestURI();
               
  if (requestURI.equals("/WebSocketsExample")) {
    sendWebSocketPage(exchange, requestURI);
                   
  } else {
    exchange.sendError(404);
  }
}
           
           
  private void sendWebSocketPage(IHttpExchange exchange, String uri) throws
          IOException {
    String page = "<html>\r\n " +
                  "  <head>\r\n" +
                  "     <script type='text/javascript'>\r\n" +
                  "        var ws = new WebSocket('ws://" +
                           exchange.getRequest().getHost() + "/Channel',
                           'mySubprotocol.example.org');\r\n" +
                  "        ws.onmessage = function (message) {\r\n" +
                  "          var messages = document.getElementById('messages');\r\n" +
                  "          messages.innerHTML += "<br>[in] " +
                             message.data;\r\n"+
                  "        };\r\n" +
                  "        \r\n" +
                  "        sendmsg = function() {\r\n" +
                  "          var message = document.getElementById(
                                   'message_to_send').value\r\n" +
                  "          document.getElementById('message_to_send').value = ''\r\n" +
                  "          ws.send(message);\r\n" +
                  "          var messages = document.getElementById('messages');\r\n" +
                  "          messages.innerHTML += "<br>[out] " + message;\r\n"+
                  "        };\r\n" +
                  "     </script>\r\n" +
                  "  </head>\r\n" +
                  "  <body>\r\n" +
                  "     <form>\r\n" +
                  "       <input type="text" id="message_to_send"
                          name="msg"/>\r\n" +
                  "       <input type="button" name="btn" id="sendMsg"
                          value="Send" onclick="javascript:sendmsg();">\r\n" +
                  "       <div id="messages"></div>\r\n" +
                  "     </form>\r\n" +
                  "  </body>\r\n" +
                  "</html>\r\n ";
               
    exchange.send(new HttpResponse(200, "text/html", page));
  }

           
           
  // IWebSocketHandler method
  public void onConnect(IWebSocketConnection webStream) throws IOException, BadMessageException {
    IHttpRequestHeader header = webStream.getUpgradeRequestHeader();

    // check origin header
    String origin = header.getHeader("Origin");
    if (!isAllowed(origin)) {
      throw new BadMessageException(403);
    }
               
    // check the subprotocol 
    String subprotocol = header.getHeader("WebSocket-Protocol", "");
    if (!subprotocol.equalsIgnoreCase("mySubprotocol.example.org")) {
      throw new BadMessageException(403);
    }
  }

  private boolean isAllowed(String origin) {
    // check the origin
    // ...
    return true;
  }
           
           
  // IWebSocketHandler
  public void onMessage(IWebSocketConnection webStream) throws IOException {
    WebSocketMessage msg = webStream.readMessage();
    if (msg.toString().equalsIgnoreCase("GetDate")) {
      webStream.writeMessage(new TextMessage(new Date().toString()));
    } else {
      webStream.writeMessage(new TextMessage("unknown command (supported: GetDate)"));
    }
  }
           
  // IWebSocketHandler
  public void onDisconnect(IWebSocketConnection webStream) throws IOException {  }

}
       
XHttpServer server = new XHttpServer(8876, new ServerHandler());
server.start();

As shown in Listing 7, the origin header is checked against an internal whitelist to reject unwanted requests. This technique avoids the situation where an attacker copies a java script fragment from a publicly available page and embeds this code fragment into his page. In this case the browser would set the origin header with the domain of the attacker’s page and the upgrade request could be rejected. This technique helps to defend against Cross-Site Request Forgery attacks. The origin header specification is independent of the WebSocket protocol specification. However, the WebSocket protocol defines a WebSocket-Origin header which has to be included in the WebSocket upgrade response.

Due the fact that a WebSocket connection will be established over an HTTP connection, the WebSocket protocol also works with HTTP proxy servers. When using a visible proxy server, the browser always communicates with the proxy server, which forwards the HTTP requests and responses. If the browser is configured to use an HTTP proxy and a WebSocket is opened, first the browser opens a tunnel to the proxy server. By sending an HTTP/1.1 connect request, as shown in Figure 5, the browser asks the HTTP proxy to make a TCP connection to a dedicated (WebSocket) server. Once this connection has been established, the role of the HTTP proxy is “downsized” to act as a simple TCP proxy to the WebSocket server. Using this proxied connection, the browser sends the WebSocket upgrade request to WebSocket server.

1. REQUEST: 
CONNECT myServer:8876 HTTP/1.1
Host: myServer:8876
User-Agent: xLightweb/2.12-HTML5Preview6
Proxy-Connection: keep-alive


1. RESPONSE:
HTTP/1.1 200 Connection established
Proxy-agent: myProxy


2. REQUEST:
GET /Channel HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: myServer:8876
Origin: http://myServer:8876
WebSocket-Protocol:
mySubprotocol.example.org


2. RESPONSE:
HTTP/1.1 101 Web Socket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
WebSocket-Origin: http://myServer:8876
WebSocket-Location:
ws://myServer:8876/Channel
WebSocket-Protocol: mySubprotocol.example.org

Figure 5. WebSocket upgrade handshake based on a tunnel

Even though a browser does not explicitly configure an HTTP proxy, transparent HTTP proxies can be passed through invisibly by calling the WebSocket server. This depends on the current network infrastructure. Under some circumstances, such transparent HTTP proxies cause trouble for WebSockets. The Connection and Upgrade header are hop-by-hop headers by definition. The HTTP specification says that hop-by-hop headers have to be removed by an intermediary if a request is forwarded to the next hop. In the case of the WebSocket upgrade request, a transparent HTTP proxy will remove the Connection: upgrade header, which will result in the WebSocket server receiving a corrupt WebSocket upgrade request. Today, most HTTP proxies are not familiar with the WebSocket protocol.

Using secured WebSockets can avoid this effect. In creating a secured WebSocket connection, the browser opens an SSL connection to the WebSocket server. In this case intermediaries will not be able to interpret or modify data.

Conclusion

With WebSockets, writing highly interactive real-time web applications becomes a simple task. The WebSocket API is very easy to understand and to use. The underlying WebSocket protocol is high efficient: there is a minimal overhead involved in managing a WebSocket. Due the fact that the WebSocket protocol runs on the top of TCP, the WebSocket protocol does not have to deal with "hacks" as do popular Comet protocols like Bayeux or BOSH. Simulating a bidirectional channel over HTTP leads to complex and less efficient protocols. Especially if only a small amount of data will be transferred, such as tiny notification events, the overhead of the classic Comet protocols is very high. This is not true for WebSockets.

Furthermore, WebSockets fit well into the existing Web infrastructure. For instance WebSockets, use the same ports that standard HTTP connections use. To establish a new WebSocket connection, the WebSocket protocol makes use of the connection management capabilities of the HTTP protocol. WebSockets support highly efficient bi-directional communication by using the existing Web infrastructure without adding new requirements or components.

On the other hand, WebSockets do less for reliability. This has to be done on the application (sub-protocol) level. In contrast to Server-Sent events, the WebSocket protocol does not include reconnect handling or guarantee message delivery. The current WebSocket protocol represents a low-level communication channel only.

In contrast to WebSockets, the Server-Sent Events protocol includes powerful features to reconnect and synchronize messages. High reliability is a built-in feature of Server-Sent Events. Furthermore, as with WebSockets, the overhead involved in managing a Server-Sent Event stream is very low. However, Server-Sent Events support a unidirectional server push channel only. By creating a Server-Sent Event, a server-to-client server-push event stream will be opened. Often Server-Sent Events will satisfy the requirements of a server-push situation, but this depends on the concrete use cases.

What do WebSockets and Server-Sent Events mean for popular Comet protocols such as Bayeux and BOSH? The HTML5 communication standards have the potential to substitute for the classic Comet protocols and become the dominant server-push technology, at least for new applications. On the other side, for instance, the cometd community started implementing cometd 2.0 which will support the WebSockets protocol as a new transport type. cometd is the most popular Bayeux implementation.

Resources


width="1" height="1" border="0" alt=" " />
Gregor Roth works as a software architect at United Internet group, a leading European Internet Service Provider to which GMX, 1&1, and Web.de belong. His areas of interest include software and system architecture, enterprise architecture management, object-oriented design, distributed computing, and development methodologies.
Related Topics >> Web Applications   |   Featured Article   |   

Comments

Please consider that the

Please consider that the WebSocket protocol has been updated. The article is based on draft75 (February 2010). Meanwhile draft 76 (April 2010) has been released (http://www.whatwg.org/specs/web-socket-protocol/).

The updated draft does not longer allow binary frames. Further more a close frame has been introduced which consist of a 0xFF byte followed by a 0x00 byte.

Additionally, a header prefix Sec- has been introduced for the WebSocket protocol headers. For instance the header WebSocket-Protocol has been replaced by the header Sec-WebSocket-Protocol.
Further more two new request header Sec-WebSocket-Key1 and Sec-WebSocket-Key2 have been introduced as well as an eight byte field which has to be sent after the request header fields. This makes the WebSocket protocol more secure against attackers.

Thanks to Simon for his feedback