If you navigate to whatwebcando.today, you will see which Web APIs are currently natively supported by the browser you are using. Although there is some variability between browsers, the important thing to realize is the number and richness of the web APIs that are standard today. With them we can implement awesome and complex web experiences running natively on web technologies: from accessing device features such as vibration signals, the gyroscope and others, to providing native-app-like features such as offline access and add-to-homescreen functionality, to you name it. Even Virtual and Augmented Reality is a thing in the web today.
One drawback of this richness in web capabilities is how complex it has become to develop web experiences that truly take advantage of what the platform has to offer. As the capabilities and complexity of the web platform have increased, there has been as well an expansion in what we call the Capability/Usage Gap, or C/U Gap for short: the difference between what can be done in the web vs. what is actually done in the Web.
In the early stages of the history of web development, say circa 1999, arguably this gap was not too big; we basically needed to master HTML and CSS, we had to develop only for a single target form factor (Desktop!), and the communication networks, albeit slow, were stable and more predictable. Now here we are in 2018: the web platform has evolved significantly; there has been an explosion both on the number of development tools, frameworks, and other technologies; there seems to be a myriad of target device types to develop for; and the underlying networks servicing our web sites and apps are challenged and unpredictable. Sustainably mastering and leveraging this contemporary development landscape is really challenging.
The Matthew Effect of Accumulated Advantage
The existence of a large C/U gap has serious collateral effects that hinder the mission and success of the web ecosystem. In essence, without exploiting the capabilities of the web platform it is very difficult to sustainably develop and maintain web sites and apps that fulfill the pillars of delightful user experiences. We see quite a bit of evidence of this in the results of studies showing the prevalence of web experiences that exhibit poor load time performance (e.g. 19 seconds to load on 3G; 75% of mobile sites take 10+ seconds to load), and/or poor runtime performance (e.g. unresponsive pages, content shifting, and other misbehaviors). And the bad news is that users don’t like these behaviors at all, as evidenced by the results of studies showing an impact on the revenue potential of sites (e.g. high abandonment rates for slow sites, and inverse correlations between conversions and load time, as well as between revenue and load time).
But perhaps the worst collateral effect of a wide C/U gap is what is known in Sociology as the Matthew Effect of Accumulated Advantage, which in our context essentially means that given the complexity of the development landscape in the web ecosystem, it is implicitly entailed that developers that are super stars, or organizations that can afford large and high-quality engineering teams, are the only ones who can really take advantage of what the web platform has to offer; and that is not what the web is all about.
Closing the C/U Gap
The bottom line is that we must strive to close the C/U gap in every way we can to make our users happy, support developers’ ability to monetize their sites; and democratize the building of delightful user experiences on the web. There seem to be two main roads towards achieving that goal: the all-developers-do-the-right-thing road; and (2) the progressive-web-ecosystem road.
In a way, the former is the road most traveled: the web capabilities, tools and frameworks, and coding/performance best practices, are all there for us to take advantage of them, and there are lots of advice and guidance to do so; in principle we all could just “do the right thing”. However, that is way easier said than done. This road is loaded with tons of power and flexibility, but also with tons of complexity and cumbersomeness which are hard to master sustainably. Therefore there should be alternatives in the web ecosystem where doing the right thing and providing delightful experiences is easy.
Progressive Web Ecosystem
The second road to closing the C/U gap is enabled by what we call a progressive web ecosystem; that is, a web ecosystem encompassing the following aspects:
- An evolving Web platform, where existing Web APIs are progressively improved, and new APIs are integrated into the platform over time
- Effective tooling to help developers to automate complex tasks; tools like workbox which make it easy to leverage the power of service workers
- Development frameworks which provide layers of abstraction to facilitate the development of complex experiences
- A concerted effort for taking the learnings gained from developing web experiences on the current web, with the current tools and frameworks, and integrate them into the web platform itself
In a progressive web ecosystem, doing the right thing is easy because the underlying platform is advanced and ever evolving, there are tools and frameworks that pack a punch of capabilities with high usability and efficiency, and we, as a community, can do what humans do best, learn and improve our platform, tools, and frameworks. After that, repeat.
A Simple Model of the Web
The Web is a complex system composed of many interrelated and interacting parts. To reason about where to focus our efforts to close the C/U gap, it is useful to think about the web ecosystem in terms of a simple model consisting of three layers.
The top layer is composed of a federation of content ecosystems, all interrelated and interacting, giving us the perception of “the whole world wide web” as a thing. The middle layer is composed by Content Management Systems, or CMSes, which are software platforms designed to simplify the creation and management of sites and their content. Notice that the model’s top layer is divided into two equally sized sets because, nowadays, about 50% of sites are powered by some sort of CMS platform, and the other 50% is powered by a variety of configurations of tools and frameworks. And the bottom layer, usually referred to as the web platform, encompasses all the capabilities of the web exposed via browser implementations of Web APIs.
Efforts to close the C/U gap must be pursued across all these layers and at Google we are indeed pursuing significant efforts along all dimensions of the web ecosystem. One of the specific efforts we are pursuing focuses on closing the C/U gap on the CMS part of this model. All CMSes are important and we want all of them to become progressive platforms. In this post we focus specifically on WordPress.
The Progressive WordPress Road
A progressive web ecosystem encompasses the notion of Progressive Web Development: the development of user-first web experiences using modern web capabilities, and coding and performance best practices. The key aspect of this definition is the “user first” part; the core of progressive web development is all about User eXperience; it is all about how our users feel when navigating and engaging with the content and web experiences we create.
When we talk about progressive WordPress we are referring to a WordPress platform where: modern development workflows, coding and performance best practices, and the use of modern Web APIs are commonplace, standard things; that is, a WordPress platform where doing the right thing, and creating user-first experiences is easy.
Now, given the size and scope of the WordPress ecosystem, how can we advance quickly along the Progressive WordPress road? Here is where the Accelerated Mobile Pages project, or AMP for short, enters the game. AMP is an open source library which makes it easy to build web pages and full sites that offer compelling experiences and load near instantaneously. In a nutshell, what AMP does is to provide, out of the box, a powerful set of capabilities and optimizations that address three essential aspects in the development of delightful experiences: load time performance (i.e. near-instant first visit load times); runtime performance (i.e. no un-responsive content); and usability (i.e. no content shifting). Check the documentation to learn more about the AMP project.
If we could integrate AMP into the WordPress platform, we would put at the fingertips of all WordPress developers the ability to take advantage of the efficiency and speed of AMP with minimal effort. That is, AMP brings home the coding and performance best practices part of the progressive WordPress equation. And this is exactly what we have been working on.
AMP in WordPress
In WordPress everything happens in one of three places: in the WordPress core platform; in Themes, which have to do with the look-and-feel of sites; and in Plugins, which have to do with extending the functionality of the WordPress platform. When we talk about AMP in WordPress, we are referring to the capabilities of an AMP Plugin, and the interaction of that plugin with the other components composing a site, such as the underlying theme and the other plugins installed.
The official WordPress plugin enabling AMP in WordPress was pioneered by an open source AMP Plugin initially developed by Automattic. This plugin was created at a very early stage of AMP, as WordPress was one of the earliest adopters of the project. It provided what we call a paired-mode, where for a given content post there was also an AMP version of it based on simple template parts. This mode did a great job enabling AMP on a large number of sites, but it had the main limitation that without significant engineering efforts, the resulting experience suffered from significant visual gaps between the AMP and non-AMP versions of the content, and significant functional gaps between the two versions (e.g. no hamburger menu, no ‘Load updates” button, etc.).
As the result of a joint effort between Automattic, XWP, and Google, the plugin has evolved significantly and steadily and now there is no need for a paired mode because the plugin generates experiences without neither functional nor visual gaps between the AMP and non-AMP versions. The latest version of the plugin, 1.0, effectively brings the power and efficiency of AMP to WordPress via a seamless integration with the standard WordPress content creation flow.
The AMP plugin will reach version 1.0 with an ETA of July 2018, and this version includes:
- Automatic conversion of WordPress built-in widgets, embeds, short-codes, etc.
- Tools to give users the power to control the actions of the plugin during the content creation workflow if needed
- Integration with Gutenberg, the new and revolutionary content creation workflow in WordPress
- Automatic optimization of CSS usage via CSS tree-shaking
With v1.0, WordPress developers get significant support making it easier to create native AMP experiences without sacrificing neither the fidelity and functionality of their content, nor the flexibility of the WordPress platform.
Native AMP Themes
Also, in our Google I/O talk you can get a glimpse of an all-AMP WordPress theme, built leveraging the capabilities of the new Gutenberg editing workflow:
And also, we described a summary of the process required to convert the 2015 WordPress theme to become all AMP. In this case, the plugin’s capabilities did most of the work, the whole process entailed only three steps:
- Use the add_theme_support( ‘amp’ ) function to register theme support for AMP, effectively telling the plugin to apply its magic and attempt to convert the theme to native AMP
- Use the capabilities of AMP bind to fix the menu toggle, which uses custom JS and can’t be converted automatically by the plugin
- Use the validation tool in the plugin to ensure the safety of the conversions made automatically by the plugin. This process gives control to the user on the behavior of the plugin, and clearly points developers to the parts of their sites that may need their attention to make them fully AMP compatible.
Without trying hard to tune the performance of this site, we observe good Lighthouse scores for the native AMP News WordPress site, influenced by the coding and performance best practices enabled by AMP.
And here you can see the demonstration of the Adventures Travel site, showing a native AMP theme fully developed using Gutenberg.
Further Along the Progressive WordPress Road
We have made significant progress along the progressive WordPress road with the advances in the plugin and its integration with the platform. Still, there is quite a bit of work ahead of us to further advance along the progressive WordPress road. The main areas we are focusing our efforts are: enabling the AMP Shell Architecture in WordPress to enhance the user experience beyond first visits, on the user’s onwards journey; providing a Progressive Theme Boilerplate/starter kit, to facilitate building progressive themes from scratch; and adding quality incentives to the ecosystem, to empower developers and site owners to develop and choose themes and plugins based on user-centric metrics, rather that just public’s opinion. A little more specifically:
App Shell Architecture in WordPress
The App Shell navigation model is a pattern for architecting sites in a way where the static elements in a page can be cached easily, and therefore loaded reliably and instantly on navigations; and the non-static components of the page can be loaded dynamically.
Enabling the App Shell Architecture in WordPress would enable several benefits and capabilities to sites, such as: reliable performance that is consistently fast, native-like interactions, and a reduction in data consumption. And this work involves the integration of the Service Worker API into WordPress core. Very excited about this effort!
Building Progressive Themes from Scratch
One of the key elements in a progressive web is tooling to help developers do the right thing in an easier and faster way. An important tool to have in the toolbox of WordPress developers is one that facilitates the development of Progressive WordPress themes. Our colleagues at LinkedIn Learning have been busy developing a Progressive WordPress Theme Boilerplate, which is a theme development starter kit that makes it easier for WordPress theme developers to get from 0 to a fully progressive theme. The boilerplate comes out of the box with:
- A modern development workflow, which is highly-configurable, and uses tools such as Composer, Gulp, and browser sync
- A range of coding and performance best practices baked in, such as asynchronous loading of resources, and lazy loading of images
- And it has opt-in integration with the AMP plugin, so that if the developer so choses, a fully AMP compatible WordPress starter theme can be generated from the get go
The main developer of this project, Morten Rand-Hendricksen, who is also a Senior Staff Instructor at LinkedIn learning, will be publishing soon a course on the LinkedIn Learning platform which will teach all about this boilerplate; and that particular course is going to be free. Don’t miss it!
Adding Incentive Mechanisms to the Ecosystem
One of WordPress major virtues is the flexibility of the platform. As a developer you can accomplish basically anything you can imagine; this is demonstrated by the large and ever expanding ecosystem of plugins available to add functionality and customize your website in a myriad of ways. However, this same virtue can also turn into one of the major challenges of the platform. Even taking into account plugins that are very good (i.e. follow coding/performance best practices and coexist nicely with other site components), the likelihood of having plugins colliding in their interactions (or impacting performance) when jointly activated in a site, is definitely not negligible.
We can deal with this challenge by introducing to the WordPress ecosystem tools that facilitate the verification of well-defined quality standards to allow developers to measure how good their products are in terms of UX-related metrics, and empower WordPress site owners to make choices about which plugins and themes to use, based on actual quality as opposed to only on popularity. We are pursuing efforts in this area, for example in the context of projects such as XWP’s Tide and Lighthouse.
Success on the web is all about building user-first experiences: it is all about UX. WordPress is an amazing platform and we are striving to close the Capabilities/Usage gap and enable content creation and consumption flows satisfying the pillars of delightful experiences. v1.0, the latest version of the official AMP plugin (currently in alpha), brings the power and efficiency of AMP to WordPress via a seamless integration with the standard WordPress content creation flow effectively enabling native AMP experiences in WordPress without compromising content fidelity or surrendering the flexibility of the WordPress platform.
Stay tuned for new developments in the areas we mention in this post. We will participating in WordCamp EU in Belgrade this year where we will talk and show the latest progress on our efforts. Hope to see you there!