At over thirty years old, HTTP is still the foundation of the web and one of the Internetâs most popular protocolsânot just for browsing, watching videos and listening to music, but also for apps, machine-to-machine communication, and even as a basis for building other protocols, forming what some refer to as a âsecond waistâ in the classic Internet hourglass diagram.
What makes HTTP so successful? One answer is that it hits a âsweet spotâ for most applications that need an application protocol. â Building Protocols with HTTPâ (published in 2022 as a Best Current Practice RFC by the HTTP Working Group) argues that HTTPâs success can be attributed to factors like:
â familiarity by implementers, specifiers, administrators, developers, and users;
â availability of a variety of client, server, and proxy implementations;
â ease of use;
â availability of web browsers;
â reuse of existing mechanisms like authentication and encryption;
â presence of HTTP servers and clients in target deployments; and
â its ability to traverse firewalls.
Another important factor is the community of people using, implementing, and standardising HTTP. We work together to maintain and develop the protocol actively, to assure that itâs interoperable and meets todayâs needs. If HTTP stagnates, another protocol will (justifiably) replace it, and weâll lose all the communityâs investment, shared understanding and interoperability.
Cloudflare and many others do this by sending engineers to participate in the IETF, where most Internet protocols are discussed and standardised. We also attend and sponsor community events like the HTTP Workshop to have conversations about what problems people have, what they need, and understand what changes might help them.
So what happened at all of those working group meetings, specification documents, and side events in 2022? What are implementers and deployers of the webâs protocol doing? And whatâs coming next?
New Specification: HTTP/3
Specification-wise, the biggest thing to happen in 2022 was the publication of HTTP/3, because it was an enormous step towards keeping up with the requirements of modern applications and sites by using the network more efficiently to unblock web performance.
Way back in the 90s, HTTP/0.9 and HTTP/1.0 used a new TCP connection for each requestâan astoundingly inefficient use of the network. HTTP/1.1 introduced persistent connections (which were backported to HTTP/1.0 with the `Connection: Keep-Alive` header). This was an improvement that helped servers and the network cope with the explosive popularity of the web, but even back then, the community knew it had significant limitationsâin particular, head-of-line blocking (where one outstanding request on a connection blocks others from completing).
That didnât matter so much in the 90s and early 2000s, but todayâs web pages and applications place demands on the network that make these limitations performance-critical. Pages often have hundreds of assets that all compete for network resources, and HTTP/1.1 wasnât up to the task. After some false starts, the community finally addressed these issues with HTTP/2 in 2015.
However, removing head-of-line blocking in HTTP exposed the same problem one layer lower, in TCP. Because TCP is an in-order, reliable delivery protocol, loss of a single packet in a flow can block access to those after itâeven if theyâre sitting in the operating systemâs buffers. This turns out to be a real issue for HTTP/2 deployment, especially on less-than-optimal networks.
The answer, of course, was to replace TCPâthe venerable transport protocol that so much of the Internet is built upon. After much discussion and many drafts in the QUIC Working Group, QUIC version 1 was published as that replacement in 2021.
HTTP/3 is the version of HTTP that uses QUIC. While the working group effectively finished it in 2021 along with QUIC, its publication was held until 2022 to synchronise with the publication of other documents (see below). 2022 was also a milestone year for HTTP/3 deployment; Cloudflare saw increasing adoption and confidence in the new protocol.
While there was only a brief gap of a few years between HTTP/2 and HTTP/3, there isnât much appetite for working on HTTP/4 in the community soon. QUIC and HTTP/3 are both new, and the world is still learning how best to implement the protocols, operate them, and build sites and applications using them. While we canât rule out a limitation that will force a new version in the future, the IETF built these protocols based upon broad industry experience with modern networks, and have significant extensibility available to ease any necessary changes.
New specifications: HTTP âcoreâ
The other headline event for HTTP specs in 2022 was the publication of its âcoreâ documents â the heart of HTTPâs specification. The core comprises: HTTP Semantics â things like methods, headers, status codes, and the message format; HTTP Caching â how HTTP caches work; HTTP/1.1 â mapping semantics to the wire, using the format everyone knows and loves.
Additionally, HTTP/2 was republished to properly integrate with the Semantics document, and to fix a few outstanding issues.
This is the latest in a long series of revisions for these documentsâin the past, weâve had the RFC 723x series, the (perhaps most well-known) RFC 2616, RFC 2068, and the grandparent of them all, RFC 1945. Each revision has aimed to improve readability, fix errors, explain concepts better, and clarify protocol operation. Poorly specified (or implemented) features are deprecated; new features that improve protocol operation are added. See the âChanges fromâŠâ appendix in each document for the details. And, importantly, always refer to the latest revisions linked above; the older RFCs are now obsolete.
Deploying Early Hints
HTTP/2 included server push, a feature designed to allow servers to âpushâ a request/response pair to clients when they knew the client was going to need something, so it could avoid the latency penalty of making a request and waiting for the response.
After HTTP/2 was finalised in 2015, Cloudflare and many other HTTP implementations soon rolled out server push in anticipation of big performance wins. Unfortunately, it turned out thatâs harder than it looks; server push effectively requires the server to predict the futureânot only what requests the client will send but also what the network conditions will be. And, when the server gets it wrong (âover-pushingâ), the pushed requests directly compete with the real requests that the browser is making, representing a significant opportunity cost with real potential for harming performance, rather than helping it. The impact is even worse when the browser already has a copy in cache, so it doesnât need the push at all.
As a result, Chrome removed HTTP/2 server push in 2022. Other browsers and servers might still support it, but the community seems to agree that itâs only suitable for specialised uses currently, like the browser notification-specific Web Push Protocol.
That doesnât mean that weâre giving up, however. The 103 (Early Hints) status code was published as an Experimental RFC by the HTTP Working Group in 2017. It allows a server to send hints to the browser in a non-final response, before the ârealâ final response. Thatâs useful if you know that the content is going to include some links to resources that the browser will fetch, but need more time to get the response to the client (because it will take more time to generate, or because the server needs to fetch it from somewhere else, like a CDN does).
Early Hints can be used in many situations that server push was designed for â for example, when you have CSS and JavaScript that a page is going to need to load. In theory, theyâre not as optimal as server push, because they only allow hints to be sent when thereâs an outstanding request, and because getting the hints to the client and acted upon adds some latency.
In practice, however, Cloudflare and our partners (like Shopify and Google) spent 2022 experimenting with Early Hints and finding them much safer to use, with promising performance benefits that include significant reductions in key web performance metrics.
Weâre excited about the potential that Early Hints show; so excited that weâve integrated them into Cloudflare Pages. Weâre also evaluating new ways to improve performance using this new capability in the protocol.
Privacy-focused intermediation
For many, the most exciting HTTP protocol extensions in 2022 focused on intermediationâthe ability to insert proxies, gateways, and similar components into the protocol to achieve specific goals, often focused on improving privacy.
The MASQUE Working Group, for example, is an effort to add new tunneling capabilities to HTTP, so that an intermediary can pass the tunneled traffic along to another server.
While CONNECT has enabled TCP tunnels for a long time, MASQUE enabled UDP tunnels, allowing more protocols to be tunneled more efficientlyâincluding QUIC and HTTP/3.
At Cloudflare, weâre enthusiastic to be working with Apple to use MASQUE to implement iCloud Private Relay and enhance their customersâ privacy without relying solely on one company. Weâre also very interested in the Working Groupâs future work, including IP tunneling that will enable MASQUE-based VPNs.
Another intermediation-focused specification is Oblivious HTTP (or OHTTP). OHTTP uses sets of intermediaries to prevent the server from using connections or IP addresses to track clients, giving greater privacy assurances for things like collecting telemetry or other sensitive data. This specification is just finishing the standards process, and weâre using it to build an important new product, Privacy Gateway, to protect the privacy of our customersâ customers.
We and many others in the Internet community believe that this is just the start, because intermediation can partition communication, a valuable tool for improving privacy.
Protocol security
Finally, 2022 saw a lot of work on security-related aspects of HTTP. The Digest Fields specification is an update to the now-ancient `Digest` header field, allowing integrity digests to be added to messages. The HTTP Message Signatures specification enables cryptographic signatures on requests and responses â something that has widespread ad hoc deployment, but until now has lacked a standard. Both specifications are in the final stages of standardisation.
A revision of the Cookie specification also saw a lot of progress in 2022, and should be final soon. Since itâs not possible to get rid of them completely soon, much work has taken place to limit how they operate to improve privacy and security, including a new `SameSite` attribute.
Another set of security-related specifications that Cloudflare has invested in for many years is Privacy Pass also known as âPrivate Access Tokens.â These are cryptographic tokens that can assure clients are real people, not bots, without using an intrusive CAPTCHA, and without tracking the userâs activity online. In HTTP, they take the form of a new authentication scheme.
While Privacy Pass is still not quite through the standards process, 2022 saw its broad deployment by Apple, a huge step forward. And since Cloudflare uses it in Turnstile, our CAPTCHA alternative, your users can have a better experience today.
What about 2023?
So, whatâs next? Besides, the specifications above that arenât quite finished, the HTTP Working Group has a few other works in progress, including a QUERY method (think GET but with a body), Resumable Uploads (based on tus), Variants (an improved Vary header for caching), improvements to Structured Fields (including a new Date type), and a way to retrofit existing headers into Structured Fields. Weâll write more about these as they progress in 2023.
At the 2022 HTTP Workshop, the community also talked about what new work we can take on to improve the protocol. Some ideas discussed included improving our shared protocol testing infrastructure (right now we have a few resources, but it could be much better), improving (or replacing) Alternative Services to allow more intelligent and correct connection management, and more radical changes, like alternative, binary serialisations of headers.
Thereâs also a continuing discussion in the community about whether HTTP should accommodate pub/sub, or whether it should be standardised to work over WebSockets (and soon, WebTransport). Although itâs hard to say now, adjacent work on Media over QUIC that just started might provide an opportunity to push this forward.
Of course, thatâs not everything, and what happens to HTTP in 2023 (and beyond) remains to be seen. HTTP is still evolving, even as it stays compatible with the largest distributed hypertext system ever conceivedâthe World Wide Web.
Source: cloudflare.com