A Brief History of the Web
Once upon a time, Tim Berners-Lee invented the World Wide Web. With it came Messenger, eMule, eBay, Facebook, Youtube, Twitter, cat videos, memes and probably happiness.
In the early days of the web, the HTTP protocol was developed to serve images, sound, and other content, in addition to HTML web pages. These pages were sent as static HTML that could not be changed. Very soon, people realized the limitations of this technology that allowed no interaction of the user with the web page.
The web evolved very quickly to a request-response paradigm, where the client makes a request and the server responds with information. REST or RESTful API design (Representational State Transfer) was developed under this methodology of request-response.
REST is the most common architectural design for creating APIs under HTTP protocol and became an important player in the development of the modern web because it facilitates communications between different systems. REST defines a common abstraction layer that allows applications written in different languages (Python, Java, JS, etc) to communicate with each other. In REST APIs, the server can manage different types of requests from the client: GET, POST, PUT and DELETE, each of them with a different purpose.
When working with web clients, in the early days of web development, the typical workflow started with the user making a request to the server. The server would receive the information, process it and return a response. The new information that the client receives had to be added to the static HTML code by refreshing the entire web page. However, normally, not all the elements of the HTML code were changed, so refreshing the entire web page was a wasteful operation. To address this issue, AJAX technology was developed.
Asynchronous JavaScript and XML (AJAX) is a client-side technique for updating the HTML code of a web page without refreshing the entire web page. It will make a request, wait for the response to be received and then call a callback function that will process the response. All these operations are done asynchronously, so the user can keep interacting with the web page (i.e. it’s non-blocking).
Even though it was a great advancement years ago, it has some drawbacks. First, AJAX can only send data as strings, which means that you have to cast all data to the correct datatype with server-side code. But its main disadvantage is that AJAX is a one-way communication technology. It always has to poll the server for data rather than receive it via push directly from the server. Additionally, since it works over HTTP protocol, each time a request is made, a new connection has to be created. For real-time applications, this is problematic. Ideally, we need a low latency technology that allows bidirectional communications, client-to-server and server-to-client. To address these issues, WebSockets were developed.
WebSockets
A WebSocket is a protocol intended for real-time communications developed for the HTML5 specification. It can create a persistent, low latency connection that can support transactions initiated by either the client or server. Instead of using the HTTP protocol, it uses TCP protocol creating a persistent connection that can be closed by either the client or the server.
The communication is performed via messages and can be triggered by the client or the server, this is usually referred to as full-duplex communication. The advantage of this over AJAX is that it improves the system latency since now the client doesn't have to pull data from the server every time it needs to add new content.
To create a WebSocket connection, the first step is to let both client and server know that they are going to use that protocol. For this, WebSockets need to perform a handshake over TCP, which effectively upgrades from HTTP protocol to WebSocket protocol. An upgrade header is added in the request to inform the server that the client wishes to establish a WebSocket connection.
Once the connection is established, messages can be sent bi-directionally. In simple terms, the client and server can perform two types of actions: they can emit a message or they can manage a received message. To emit a message (depending on the language) you would need to write something like socketio.emit('my_event', msg), where my_event is the identifier that the receiver handles. To handle a sent message you would need to write something like socket.on('my_event', function (msg) { ... });.
In terms of data IO, WebSockets require less data transfer than other technologies. The headers are passed only when the connection is created. In contrast, in the HTTP request-response paradigm, the headers have to be passed in every request. In this interesting post there is a comparison between HTTP/1.1, HTTP/2 and WebSockets.
WebSocket technology is widely accepted and can be developed using many different languages. In JavaScript, one can use the libraries Socket.io and ws; in Python, tornado and Flask-SocketIO; in Java, Jetty and in C++, uWebSockets. In my codebase, I have a simple example of how to use WebSockets with Python.