Common sense is good; enough of it is genius

Category: web (page 1 of 1)

Progressive Content Management Systems

Tl;DR

This post summarizes a recent talk Weston Ruter and myself gave at Chrome Dev Summit , 2018:  Content Management Systems (CMS) are software platforms designed to simplify the creation and management of websites and their content. Nowadays, about 50% of websites are powered by some sort of CMS platform. Below, we discuss our experience moving a complex CMS (i.e. WordPress) forward along the Progressive Web Road by integrating modern capabilities (e.g. Service Worker API) and progressive technologies (e.g. AMP) into its platform and ecosystem, and how the lessons learned can be applied to other CMSes.

It is all about UX

As web developers our goal is to maximize the joy our users feel when engaging with the websites we build or the web content we create and publish on the web.  And although User Experience is a complex matter, there are common elements in web experiences that users love.  Specifically,  users love…

  • Sites that load quickly consistently and offer good runtime performance
  • Sites where they feel safe sharing their data and doing transactions on them
  • Sites that are accessible and allow them to take advantage of the full capabilities of their devices, and
  • Sites that offer great content quality

The good news is that we can build experiences that fulfill these pillars if we take advantage of modern capabilities the web has to offer. Doing this is what we call Progressive Web Development: developing user-first experiences using modern development workflows and APIs, following coding and performance best practices, and taking advantage of incentive mechanisms (e.g. engagement increases) and validation structures (e.g. performance budgets and others) which are available to us.

The way we go about taking advantage of Progressive Web Development depends on how we go about our creative process in the web.

Web Development Process

The things we create on the web fall in a general sense into two areas. On one hand, we can build websites or web apps from scratch, in which case we have full control of the creative journey: from the build process, to the functionality, to the look-and-feel, of our website or web app. On the other hand, we can use some kind of Content Management System, or CMS for short, which are Software platforms providing layers of abstraction and capabilities to make easier to create, publish, and consume digital content on the web.

The first route gives us a lot of power and flexibility, but it is also it requires quite a bit of expertise and resources. The second route, allows us to take advantage of the functionality provided by a CMS, but the experience we provide to our users depends a great deal on the specifics of the CMS we choose.

Contemporary Content Creation and Publishing

Nowadays the majority of experiences in the web are created, published, and consumed via some kind of CMS: More than 50% of the web is powered by some form of CMS; and this year there was an 11% YoY growth! [bit.ly/w3tech-stats]. And there are many other platforms that do not consider themselves CMSes but also provide similar capabilities that allow users to create and publish content on the web easily. Therefore, this number is actually much higher in reality.

In practical terms, this means that the CMS space is highly populated and complex. There are many CMS options out there for users to choose from and they come in many shapes and forms, varying across dimensions such as: Static vs. Dynamic, Proprietary vs. Open Source, Special Purpose vs. General Purpose, and others.

But although there are differences between them, they also share lots of commonalities, and many of them face common challenges when it comes to integrating progressive web capabilities into their platforms. To reason about how to tackle such challenges across the CMS space, it is useful to abstract our thinking about these platforms in terms of the fundamental components that make them.

CMS Ecosystems

Each CMS has in one way or another a set of components that form a unique ecosystem. There are usually the following components:

A platform Core: which encompasses the main functionality provided by the platform; a mechanism for extending the core functionality of the platform. Such extensions are known as modules, plugins, and themes; a community of developers who are in charge of implementing and evolving both the core and the extensions for the platform; and a user base who either create content on the platform, or consume content from the platform, or both.

When we talk about progressive CMSes, we are referring to bringing the pillars of PWD to the components of a CMS Ecosystem; specifically, things such as:

  • Bringing coding and performance best practices to the development of both the platform core and the extensions
  • Introducing incentive mechanisms for extension developers to do the right thing
  • Providing tooling to developers to make it easy for them to do the right thing

In order for a CMS to fulfill its potential, all these components need to walk together along the progressive road.

In this post we will describe specific results of efforts our team has been pursuing in the context of a specific CMS, and we will touch upon some of the great work being done by other  players in the CMS space.

Progressive WordPress

Our team has been pursuing efforts to bring progressive web technologies to the WordPress ecosystem. WordPress is certainly one of the most influential CMSes out there, as reflected by the fact that it powers about ? of the web today.  Given the size and complexity of this ecosystem, addressing its challenges will not only have an impact on a very large swath of the web, but it will also help us to shed light on how to address similar challenges in other CMSes.

Our goals are two fold: to tackle the challenges for the WordPress ecosystem, and to distill and generalize the learnings and solutions we gather along the way and bring them over to other CMSes in the form of objective guidance, tutorials, and technology transfersOur approach to achieve these goals consists of two parts:

  • Integrating modern Web APIs into the WordPress ecosystem, either via the core of the platform or via plugin-enabled capabilities.

AMP: A Well-lit Path to Modern Web Development

AMP is a web component library, together with a runtime system and a validation framework. AMP provides, out of the box, a set of capabilities and optimizations addressing essential aspects of high-quality web experiences. These include:

  • Coding and performance best practices, which enable great runtime performance (i.e. no un-responsive content) and great usability (i.e. no content shifting)
  • Performance incentives, such as the ability to achieve near-instant loading times via pre-rendering when AMP content is served from an AMP cache
  • validation framework that guides the development process towards highly performant sites, and makes it easy for developers to keep their sites in that state over time

Enabling AMP content creation in WordPress means that WordPress developers can take advantage of these capabilities without deviating from their standard content creation workflow in the platform. Such integration has been enabled via the official AMP plugin for WordPress, which is currently at version 1.0-RC2 (available on Github), and version 1.0-stable will be released at the end of this month (November, 2018).

The Official AMP Plugin

To enable WordPress developers to take advantage of AMP’s capabilities, AMP content creation must be seamlessly integrated into the standard content creation workflow in WordPress. We can envision a path where AMP-compatible markup is generated by WordPress core itself, as an option, similarly to what happened when HTML5 became a thing. Alternatively, such seamless integration can be achieved by a plugin.

The official AMP Plugin for WordPress is a product designed to fulfill two main roles. First, to enhance the development process of AMP experiences in WordPress, such as AMP-compatible plugins and themes. And second, to ensure that AMP compatibility is maintained as a site evolves, so that no invalid AMP is ever served.

The plugin is not a turn-key solution to the “how to AMPlify my site” problem; instead, the AMP plugin functionality substitutes AMP components for their corresponding HTML ones where required and possible (img => amp-img, video => amp-video, etc.), and it aids the development process by exposing sources of validation errors, providing tools to understand the context of such validation errors (e.g. preview integration), and ensuring that only AMP-valid content is ever served for a given WordPress site. These capabilities make it easier for developers to build high-quality AMP experiences (e.g. full sites) and AMP-related solutions (e.g. editors); and enable the ecosystem of developers making themes (e.g. Genesis) and plugins (e.g. Jetpack), to tackle AMP development in a WordPress way, that is, in a way that is integrated with the regular WordPress development workflow and theme/plugin ecosystem.

In its second role, the plugin functions as key enabling component ensuring a site states AMP compatible by keeping developers, implementers, and site owners, always aware of the compatibility status of their site.

The plugin provides all its functionality in an opt-in manner. And users that do opt-in have the option of enabling AMP only on specific sections of their site, and also disable AMP at a granular level (e.g. on a single post).

Integrating Modern Web APIs into the WordPress

The second part of our approach to bringing Progressive Web Development into WordPress consists of integrating modern web APIs and capabilities into the WordPress core platform. We want the WordPress platform to support natively things such as: Service Workers, Offline access, Streaming, background sync, etc. The goal of such integration is to provide a consistent and unified approach for WordPress developers to easily access these APIs and capabilities as they develop core functionality, as well as themes or plugins.

In addition to our work on AMP for WordPress, our team focused this year on devising and implementing the integration of some of these building blocks into the WordPress platform, and we did this with an emphasis on doing it in a way that is seamlessly integrated to how things work in the WordPress platform.

Service Workers in WordPress, the WordPress Way

The Service Worker API is at the center of many of the advanced capabilities we want to enable natively in WordPress. Nowadays, it is certainly possible to use Service Workers in WordPress, but doing so poses several fundamental challenges that need to be addressed.

First, there needs to be a unified API access for creating and manipulating service workers in WordPress. Without it, if multiple plugins and themes want to leverage the capabilities of this API, they would have to do all the work by themselves, not able to take advantage of any reusability, and running the risk of having collisions among multiple service workers installed on a site, which is not allowed by the API.

Second, providing access to the Service Worker API is just one part of the solution needed; programming the logic of Service Workers is by far not a trivial task. If we want all developers to take advantage of the power offered by Service Workers, an abstraction layer must be added in WordPress that makes doing so easier for developers.

Finally, one of the virtues of WordPress is the flexibility and versatility of its extension mechanism: plugins and themes. The challenge is that these extensions are not sandboxed and consequently, it is very difficult to implement advanced capabilities such as the App Shell model in WordPress because down the road additional plugins can be installed and they can alter/modify the content of a site in ways that breaks these capabilities.

To address these challenges, we have implemented abstractions in WordPress to integrate the Service Worker API into WordPress in a way that is applicable not only to WordPress but also to other CMSes. These abstractions cover three main aspects:

  • A low-level API in WordPress that provides direct access to Service Workers, so that all themes and plugins can each include their own logic in a single installed service worker.
  • An integration of Workbox, which is a library providing additional layers of abstraction to  facilitate developing capabilities on top of the service worker API, eliminating the need of writing of complex JavaScript code that would otherwise need to be written from scratch  over and over again.
  • Since WordPress developers have historically been PHP-first developers, we have implemented an abstraction layer in WordPress that translates PHP API calls into the corresponding Workbox JavaScript code which is then included in the service worker.

Progressive Capabilities in WordPress

Once service workers are accessible in a unified way, many advanced capabilities  are enabled. Some of them are the App Shell model, and offline access capabilities. 

The App Shell Model in WordPress

One of such capabilities is the app shell model. An app shell facilitates features such as seamless page transitions and offline interactions, similar to the experiences provided by native applications. But implementing an app shell is normally very challenging to do in WordPress due to the lack of sandboxing between plugins and themes.

This is where AMP plugin plays a key role because it enables the enforcing of constraints to ensure that the content on a site behaves in a way that doesn’t break the app shell model, while also ensuring the content loads in a performant way. Furthermore, AMP pages are also specifically designed to be embedded in this way via the Shadow DOM API.

In this demo site we have a WordPress site using the default theme and implementing the App Shell model. Pay attention to the fun spinning logo. Notice what happens when we click to navigate to the “About” page and other pages in the nav menu:

  • The logo keeps spinning without resetting.
  • And the nav menu remains expanded with the current page being highlighted.
  • There is no flash of white in the header.
  • Only the actual content of the page is changing, and it does so with fading animation.

To give you a glimpse of how this is accomplished on the user side, here are some snippets of code any WordPress developer would use in order to take advantage of the capabilities provided by the integration of Service Workers into WordPress core.

First of all, to address the problem of no sandboxing, we leveraged the AMP plugin to make all pages of a site available in AMP. This can be done with a standard add_theme_support() call.

Second, we introduced a way of specifying the dynamic part of the template which changes as you navigate the site. Specifically you can see this via these two function calls placed before and after the content.

And lastly, we extended the AMP plugin to add logic to the page as well as the service worker to implement the app shell. All a theme has to do is to opt-in with another add_theme_support() call like you see here.

The bottom line from these examples is that the complexities of dealing with Service Workers are abstracted away from the theme developer and absorbed directly by the platform.

Offline Access

Beyond the app shell model, the service worker is useful for many other advanced capabilities. As mentioned previously, the service worker also enables offline access.

WordPress themes include a 404 template which is served when the site can’t locate the requested content. We’ve extended the WordPress template system to introduce an offline template which is served when the user’s browser can’t access the site. Thus, in addition to a 404.php template, WordPress theme developers just have to add an offline.php like to their site in order to customize the user experience when facing the loss of connectivity.

The service worker is also useful to cache content the user has visited (or may visit) so that if connectivity is lost, they can continue accessing that part of the site’s content when they are offline. In the snippet above, you can see how the stale-while-revalidate serving strategy is added for navigation requests, so that anything previously visited will continue to be available.

When the user wants to provide a comment on an article but are offline when they try to submit it, the service worker can intercept the request and replay it later via background sync when the user is back online. The developer doesn’t have to do anything as all this happens by default.

Progressive CMSes: Beyond WordPress

The CMS space is moving forward steadily along the progressive web road. Google is working and supporting many of the platforms in this space to help them accelerate their pace, and we are seeing great advances by several of them. Including…

  • Drupal is another popular open source CMS. Their developer community is driving the development of AMP and PWA modules for the Drupal platform. Both modules are available and used by Drupal sites today and they are actively working on enhancing them.
  • Magento is one of the leading e-commerce platforms and they have been strong advocates for PWAs on their platform for a while now. They will be releasing Magento 2.3 in November and that release will include the launch of PWA Studio, a set of developer tools and guidelines, to make it easier to build great web experiences on their platform.
  • Adobe Experience Manager is a popular enterprise CMS; they have made it possible to create and publish Single Page Applications directly in the Experience Manager Page Editor and they’re working to provide direct support for Service Workers in the near future.
  • EC-CUBE is a leading eCommerce platform in Japan; they are working, together with a development agency called Sunday Systems, on an AMP plugin, which will enable the creation of AMP content on their platform.

It is exciting to see platforms like these becoming more progressive, and we are looking forward to continued progress by all of them.

The Road Ahead

We have made significant progress on our efforts in the WordPress ecosystem, but there is still some way to go to get to where we really want to be. We are continuing our efforts described in this post, including the following aspects.

The AMP plugin for WordPress is available on Github today. If you are WordPress developer, the plugin gives you the tools and support you need to make your themes and plugins fully AMP compatible. We are working on expanding the integration of AMP content creation in WordPress, especially in the context of Gutenberg, which is the new editing experience in WordPress.

The integration of the Service Worker API and other PWA capabilities into WordPress core is well underway. These capabilities are currently available as a Feature Plugin, which you can download and experiment with today.  We are continuing this effort, and we are working with Automattic and others in the WordPress ecosystem to get the integration merged into core some time not too far in 2019

Finally, we are continuing working with the WordPress ecosystem to help them take advantage of progressive technologies at all levels of their ecosystem.

Web Content Ecosystems @ Google

I have written before about Google’s Progressive Web vision, the work being pursued by the Web Content Ecosystems team, and our commitment to contribute to the advancement of WordPress along the Progressive web road. Our team has spent a lot of time learning about WordPress internals and architecture over the last year, and we are growing to expand our efforts. I am super excited to announce something you may have already heard: Weston Ruter and Thierry Muller have joined the Web Content Ecosystems team at Google!

Weston and Thierry bring many years of combined hands-on experience across different areas of WordPress, including aspects such as managing core releases, extensive experience with core development, building popular frameworks and plugins, and both are strong advocates of the open web. WordPress has been steadily  advancing in long strides along the progressive web path and I’m definitely excited for what we’ll be able to contribute in the year(s) ahead, with Weston and Thierry on board. Having them as part of our team, will accelerate the pace of advancing WordPress in particular, and the Web content ecosystem in general, along the progressive web development road. Weston, Thierry, welcome onboard!

WordPress @ Google: Part II

Screen Shot 2018-05-13 at 8.36.05 AM
A Simple Model of the Web

At Google we are pursuing efforts that champion awesome user experiences across all platforms in the Content Management Systems (CMS) space. One of those efforts focuses specifically on the WordPress platform. As part of Google’s commitment to these efforts, we are growing a team focused on WordPress Developer Relations. The goal of this team is to pursue  scalable WordPress development efforts laser-focused on two main areas:

  1. Advancing the WordPress platform along the progressive road: accelerate the adoption of coding and performance best practices, integrate relevant modern Web APIs with WordPress core (e.g. Service Workers API), development of tools (e.g. plugins, progressive themes starter kits), work closely with the WordPress community, etc.
  2. Making the experience of Google products in WordPress (i.e. Analytics, Search console, etc.) awesome: develop tools and plugins, that enable the WordPress community to have free high-quality options for leveraging Google technologies such as Analytics, Search Console, and others.

If you are a strong WordPress developer, you love the web in general and WordPress in particular, and you feel passionate about contributing to making those platforms better, you may be interested in joining our team. That would be awesome! And if you decide to do that, you must go through the interview process at Google.

DevRel Interviews @ Google

The WordPress Developer Relations team we are building is part of our Web Content Ecosystems initiative, which itself is part of the Developer Relations (aka DevRel) part of Google.

developers_logo

DevRel at Google is engineering-driven, and therefore the interviews for the positions related to (WordPress) Developer Relations at Google are technical interviews. In this post I offer my perspective into what these interviews are all about, what do they seek to achieve, and how to go about preparing for one. This guidance is applicable to most technical DevRel interviews settings at Google, not only the ones for our roles.

Every interview and interviewer at Google is different, but the interview process is in general well defined. In a nutshell, the process is a technical interview with the majority of questions focused on the basics and fundamentals of the skills associated with a given role. The interview process also seeks to get a glimpse of how you think, how you approach problems and unknowns, and how you interact with your interviewer.

An interview for a DevRel position has two main components, with varying degrees of weight and volume depending on the specific role: Developer Advocacy and Engineering Skills. Once you engage with the interview process, you will be guided to which is the right balance according to the position you are interviewing for.

In the context of developer advocacy, there will be some questions related to areas such as communication skills, community engagement experience, and other questions related to open source projects that you have been involved with, and the development of APIs to be used by the developer’s community. We are looking for strong communicators: engineers that like to engage with community efforts, ideally have a track record of collaborative work, and have a voice in the community — past talks, presentations, etc., are all strong assets. Furthermore, you need to feel comfortable in this role, as you’ll be expected to represent Google around the world — that’s a key requirement.

And in the context of engineering skills there will be a number of technical coding questions related to software development knowledge (algorithms, coding, design); this is where an understanding of fundamental algorithms and data structures is important.

How to Prepare

Dilbert-Three-Masters

Succeeding in an interview depends on many factors and I know many extraordinary people that have failed interviews at Google and elsewhere. So, failure is not an option, but it is a possibility. And preparing well is the best we can do to maximize the chance of success.

If you have been involved in the WordPress developer’s community, or have been developing on the WordPress platform for a while, you likely understand the advocacy aspects entailed in working with such an open, large, and vibrant community. But in any case, once you engage with the interview process, your recruiter will give you details on what to expect.

Preparing for the engineering part of the interview is in certain ways harder because it involves bringing to your head’s cache memory the fundamentals of Computer Science. This approach to interviews is effective but it is not perfect. Several of the best engineers I know have no formal CS background, and in spite of being able to build a castle on a browser, they can too be vulnerable to this part of the interview process. Nevertheless, knowing this, and no matter what your background is, you can prepare well for an interview anywhere. If you are a good developer, you certainly know the fundamentals, even if the names you use for them are different. Let’s roll up our sleeves, and prepare well.

The Fundamentals

You should know at least one programming language very well; if you are a WordPress developer, that language would likely be PHP. Since WordPress Developer Relations is part of Web Content Ecosystems, candidates are expected to understand at a good level of depth the fundamentals of the web: the DOM, HTML, CSS, and JavaScript. For example, an understanding of what kind of data structure should be used to implement a DOM abstraction is expected; as well as why having async and await  language structures in a language for the web (e.g. JS) would be necessary and important.

coding-tag-cloud

In most interviews you will be expected to write code in one way or another. In a phone interview you may write the code on a Google doc; whereas in an onsite interview  you will write code on a whiteboard. For many people it is often challenging to write code on a medium other than an editor/IDE. It is very important that you practice writing code on paper or on a whiteboard as you study.

Regarding the code you will be asked to write, it will be expected that you know a fair amount of details about your favorite programming language. Your code should be well written, consistently styled, and as close to compilable as possible. This does not mean that you will fail if you forget a semicolon or an indentation, but coding style and approach are important signals of how you work.

Depending on the role you are interviewing for you will be expected to know more or less about API’s, OOD/OOP, how to test your code, as well as coming up with corner cases and edge cases for your and other people’s code. My approach to this is: prepare as if I was interviewing for the highest level I can possibly seek; doing that forces me to do my best and be as strongly prepared as possible. I am totally fine being over-prepared for an interview because that mitigates the anxiety and pressure I (and others) experience during interviews

Understanding Interviews

Interview-Dilbert

I like to think of engineering jobs as jobs in the construction industry; I mean the jobs of the folks that actually build things. All the things we know: algorithms, data structures, design strategies, etc., are tools at our disposal in a toolbox. When you are asked a technical problem your task is two-fold:

  1. Extract the essence of the problem asked and bring it to its fundamental pieces that enable you to propose a solution.
  2. Determine which tools in your toolbox you could use to solve the problem elements. Many actual Google problems are solved in terms of essential components of such a toolbox, such as Sorting (plus Searching and Binary Search), Divide-and-Conquer, Dynamic Programming / Memoization, Greediness, Recursion or algorithms linked to specific data structures.

Part of your goal in preparing for a technical interview is to make sure you understand the fundamental tools that should be part of the engineering toolbox to fulfill the role you are seeking. If you want to build a wooden house, you better have some nails and a hammer in that toolbox.

You should study up on as many data structures as possible, including those such as Arrays, Stacks, Queues, Hashing, Dictionary, TreesBinary Trees, Heaps, and Graphs. Ideally you should know these data structures well, the runtime complexity for most their operations, and what algorithms tend to go along with each of them. Knowing this enables you to identify patterns in problems and map them to a domain where the application of a given technique is easier. For example, many problems become easier to solve once their input has been sorted, effectively enabling the application of sub-linear searching strategies (i.e. Binary Search); where easier means, for example, bringing down a quadratic, O(n^2)), time complexity to a more manageable (i.e. O(nlogn))  one for very large input datasets.

Keep in mind that the interviewer will be evaluating not only your technical abilities but also how you approach questions, how you derive the answers, how you communicate, and how you interact with them. We don’t expect you to know the answer to every problem thrown at you in an interview setting; that’s not how the real-world works. We are trying to assess whether we would be willing to sit with you side-by-side in the future and work on a problem that neither of us knows the answer to: we want to see what tools you have in your toolbox, how you use them, and how you handle uncertainty. And at the same time, the interview is an opportunity for you to get a sense of how it would be to work with a Googler.

During the Interview

Interview-Dilbert-3


During the interview make sure to ask questions any time you need clarity. Don’t worry if you don’t know something at first. In such cases, the technique I apply is to bring the problem specification to its fundamental components and engage with the interviewer to clarify what is needed. Often interview questions are deliberately underspecified because interviewers are looking to see how you engage the problem and find a solution. The goal is to show that your answers are organized, demonstrate that you follow a process, and that you take risks and edge cases into account, and can clearly communicate your ideas.

When you are developing an answer to any question it is very important to talk aloud through your thought process. The way I have done it in the past, is to keep the reasoning dialogue in my head flowing with the speakers on. I ask questions aloud to myself (i.e. with constraint X, then I may have options Y and Z; let’s see what happens if…).

Notice that more often than not, a perfect solution is not the ultimate goal. It is very valuable to arrive to a suboptimal solution following a solid reasoning path, and then understand the parts of the solution that could be improved. Therefore, do not hesitate to build upon your answer/solution if you think of ways it can be improved or more efficient. In many cases, the first answer that springs to mind may not be the best or the most elegant solution but it opens the path to arrive at a better solution; refine your answers as you feel necessary. Also, often the interviewer points out with questions towards aspects that may require attention; reasoning about those cases with the interviewer is very useful as well.

Ready, Set, Go!

It is true that the interview process for tech  roles in general is often a grueling experience for many; it has certainly been that way for me sometimes. Nevertheless,  understanding what the interview process is all about, and preparing very well prior to tackling it, makes the chances of succeeding much higher and, at the very least, guarantees that you gain the most towards your career capital no matter the outcome.

Progressive WordPress

If you navigate to whatwebcando.today, you will see which Web APIs are currently natively supported by the browser you are using. Although there is some variability between browsers, the important thing to realize is the number and richness of the web APIs that are standard today. With them we can implement awesome and complex web experiences running natively on web technologies: from accessing device features such as vibration signals, the gyroscope and others, to providing native-app-like features such as offline access and add-to-homescreen functionality, to you name it. Even Virtual and Augmented Reality is a thing in the web today.

One drawback of this richness in web capabilities is how complex it has become to  develop web experiences that truly take advantage of what the platform has to offer. As the capabilities and complexity of the web platform have increased, there has been as well an expansion in what we call the Capability/Usage Gap, or C/U Gap for short: the difference between what can be done in the web vs. what is actually done in the Web.

Screen Shot 2018-05-12 at 12.07.26 PM

In the early stages of the history of web development, say circa 1999, arguably this gap was not too big;  we basically needed to master HTML and CSS, we had to develop only for a single target  form factor (Desktop!), and the communication networks, albeit slow, were stable and more predictable. Now here we are in 2018: the web platform has evolved significantly; there has been an explosion on the number of development tools, frameworks, and other technologies; there seems to be a myriad of target device types to develop for; and the underlying networks servicing our web sites and apps are challenged and unpredictable. Sustainably mastering and leveraging this contemporary development landscape is really challenging. 

The Matthew Effect of Accumulated Advantage

The existence of a large C/U gap has serious collateral effects that hinder the mission and success of the web ecosystem. In essence, without exploiting the capabilities of the web platform it is very difficult to sustainably develop and maintain web sites and apps that fulfill the pillars of delightful user experiences.  We see quite a bit of evidence of this in the results of studies showing the prevalence of web experiences that exhibit poor load time performance (e.g. 19 seconds to load on 3G; 75% of mobile sites take 10+ seconds to load), and/or poor runtime performance (e.g. unresponsive pages, content shifting, and other misbehaviors).  And the bad news is that users don’t like these behaviors at all, as evidenced by the results of studies showing an impact on the revenue potential of sites (e.g. high abandonment rates for slow sites, and inverse correlations between conversions and load time, as well as between revenue and load time).

matthew-effect

But perhaps the worst collateral effect of a wide C/U gap is what is known in Sociology as the Matthew Effect of Accumulated Advantage, which in our context essentially means that given the complexity of the development landscape in the web ecosystem, it is implicitly entailed that developers that are super stars, or organizations that can afford large and high-quality engineering teams, are the only ones who can really take advantage of what the web platform has to offer; and that is not what the web is all about.

Closing the C/U Gap

The bottom line is that we must strive to close the C/U gap in every way we can to make our users happy, support developers’ ability to monetize their sites; and democratize the building of delightful user experiences on the web. There seem to be two main roads towards achieving that goal:  the all-developers-do-the-right-thing road; and (2) the progressive-web-ecosystem road.

In a way, the former is the road most traveled:  the web capabilities, tools and frameworks, and coding/performance best practices, are all there for us to take advantage of them, and there are lots of advice and guidance to do so; in principle we all could just “do the right thing”. However, that is way easier said than done. This road is loaded with tons of power and flexibility, but also with tons of complexity and cumbersomeness which are hard to master sustainably. Therefore there should be alternatives in the web ecosystem where doing the right thing and providing delightful experiences is easy.

Progressive Web Ecosystem

The second road to closing the C/U gap is enabled by what we call a progressive web ecosystem; that is, a web ecosystem encompassing the following aspects:

  • An evolving Web platform, where existing Web APIs are progressively improved, and new APIs are integrated into the platform over time
  • Effective tooling to help developers to automate complex tasks; tools like workbox which make it easy to leverage the power of service workers
  • Development frameworks which provide layers of abstraction to facilitate the development of complex experiences
  • A concerted effort for taking the learnings gained from developing web experiences on the current web, with the current tools and frameworks, and integrate them into the web platform itself

In a progressive web ecosystem, doing the right thing is easy because the underlying platform is advanced and ever evolving, there are tools and frameworks that pack a punch of capabilities with high usability and efficiency, and we, as a community, can do what humans do best, learn and improve our platform, tools, and frameworks. After that, repeat.

A Simple Model of the Web

The Web is a complex system composed of many interrelated and interacting parts. To reason about where to focus our efforts to close the C/U gap, it is useful to think about the web ecosystem in terms of a simple model consisting of three layers.

The top layer is composed of a federation of content ecosystems, all interrelated and interacting,  giving us the perception of “the whole world wide web” as a thing. The middle layer is composed by Content Management Systems, or CMSes, which are software platforms designed to simplify the creation and management of sites and their content. Notice that the model’s top layer is divided into two equally sized sets; this is  because, nowadays, about 50% of sites are powered by some sort of CMS platform, and the other 50% is powered by a variety of configurations of tools and frameworks. And the bottom layer, usually referred to as the web platform, encompasses all the capabilities of the web exposed via browser implementations of Web APIs.

Efforts to close the C/U gap must be pursued across all these layers and at Google we are indeed pursuing significant efforts along all dimensions of the web ecosystem. One of the specific efforts we are pursuing focuses on closing the C/U gap on the CMS part of this model.  All CMSes are important and we want all of them to become progressive platforms. In this post we focus specifically on WordPress.

The Progressive WordPress Road

A progressive web ecosystem encompasses the notion of Progressive Web Development: the development of user-first web experiences using modern web capabilities, and coding and performance best practices. The key aspect of this definition is the “user first” part;  the core of progressive web development is all about User eXperience; it is all about how our users feel when navigating and engaging with the content and web experiences we create.

When we talk about progressive WordPress we are referring to a WordPress platform where: modern development workflows, coding and performance best practices, and the use of modern Web APIs are commonplace, standard things; that is, a WordPress platform where doing the right thing, and creating user-first experiences is easy.

Now, given the size and scope of the WordPress ecosystem, how can we advance quickly along the Progressive WordPress road? Here is where the Accelerated Mobile Pages project, or AMP for short, enters the game. AMP is an open source library which makes it easy to build web pages and full sites that offer compelling experiences and load near instantaneously. In a nutshell, what AMP does is to provide, out of the box, a powerful set of capabilities and optimizations that address three essential aspects in the development of delightful experiences: load time performance (i.e. near-instant first visit load times); runtime performance (i.e. no un-responsive content); and usability (i.e. no content shifting). Check the documentation to learn more about the AMP project.

If we could integrate AMP into the WordPress platform, we would put at the fingertips of all WordPress developers the ability to take advantage of the efficiency and speed of AMP with minimal effort. That is, AMP brings home the coding and performance best practices part of the progressive WordPress equation. And this is exactly what we have been working on.

AMP in WordPress

In WordPress everything happens in one of three places: in the WordPress core platform; in Themes, which have to do with the look-and-feel of sites; and in Plugins, which have to do with extending the functionality of the WordPress platform. When we talk about AMP in WordPress, we are referring to the capabilities of an AMP Plugin, and the interaction of that plugin with the other components composing a site, such as the underlying theme and the other plugins installed.

The official WordPress plugin enabling AMP in WordPress was pioneered by an open source AMP Plugin initially developed by Automattic. This plugin was created at a very early stage of AMP, as WordPress was one of the earliest adopters of the project. It provided what we call a paired-mode, where for a given content post there was also an AMP version of it based on simple template parts. This mode did a great job enabling AMP on a large number of sites, but it had the main limitation that without significant engineering efforts, the resulting experience suffered from significant visual gaps between the AMP and non-AMP versions of the content, and significant functional gaps between the two versions (e.g. no hamburger menu, no ‘Load updates” button, etc.).

As the result of a joint effort between Automattic, XWP, and Google, the plugin has evolved significantly and steadily and now there is no need for a paired mode because the plugin generates experiences without neither functional nor visual gaps between the AMP and non-AMP versions. The latest version of the plugin, 1.0, effectively brings the power and efficiency of AMP to WordPress via a seamless integration with the standard WordPress content creation flow.

The AMP plugin will reach version 1.0 with an ETA of December 2018, and this version includes:

  • Automatic conversion of WordPress built-in widgets, embeds, short-codes, etc.
  • Tools to give users the power to control the actions of the plugin during the content creation workflow if needed
  • Integration with Gutenberg, the new and revolutionary content creation workflow in WordPress
  • Automatic optimization of CSS usage via CSS tree-shaking

With v1.0, WordPress developers get significant support making it easier to create native AMP experiences without sacrificing neither the fidelity and functionality of their content, nor the flexibility of the WordPress platform.

Native AMP Themes

The first step towards native AMP WordPress sites is a fully AMP compatible WordPress theme. In our AMP Conference talk, we showed such a theme for a native AMP News site:

Also, in our Google I/O talk you can get a glimpse of an all-AMP WordPress theme, built leveraging the capabilities of the new Gutenberg editing workflow:

And also, we described a summary of the process required to convert the 2015 WordPress theme to become all AMP. In this case, the plugin’s capabilities did most of the work, the whole process entailed only three steps:

  1. Use the add_theme_support( ‘amp’ ) function to register theme support for AMP, effectively telling the plugin to apply its magic and attempt to convert the theme to native AMP
  2. Use the capabilities of AMP bind to fix the menu toggle, which uses custom JS and can’t be converted automatically by the plugin
  3. Use the validation tool in the plugin to ensure the safety of the conversions made automatically by the plugin. This process gives control to the user on the behavior of the plugin, and clearly points developers to the parts of their sites that may need their attention to make them fully AMP compatible.

Without trying hard to tune the performance of this site, we observe good Lighthouse scores for the native AMP News WordPress site, influenced by the coding and performance best practices enabled by AMP.

And here you can see the demonstration of the Adventures Travel site, showing a native AMP theme fully developed using Gutenberg.

Further Along the Progressive WordPress Road

We have made significant progress along the progressive WordPress road with the advances in the plugin and its integration with the platform. Still, there is quite a bit of work ahead of us to further advance along the progressive WordPress road. The main areas we are focusing our efforts are: enabling the AMP Shell Architecture in WordPress to enhance the user experience beyond first visits, on the user’s onwards journey; providing a Progressive Theme Boilerplate/starter kit, to facilitate building progressive themes from scratch; and adding quality incentives to the ecosystem, to empower developers and site owners to develop and choose themes and plugins based on user-centric metrics, rather that just public’s opinion. A little more specifically:

App Shell Architecture in WordPress

The App Shell navigation model is a pattern for architecting sites in a way where the static elements in a page can be cached easily, and therefore loaded reliably and instantly on navigations; and the non-static components of the page can be loaded dynamically.

Enabling the App Shell Architecture in WordPress would enable several benefits and capabilities to sites, such as: reliable performance that is consistently fast, native-like interactions, and a reduction in data consumptionAnd this work involves the integration of the Service Worker API into WordPress core. Very excited about this effort!

Building Progressive Themes from Scratch

One of the key elements in a progressive web is tooling to help developers do the right thing in an easier and faster way. An important tool to have in the toolbox of WordPress developers is one that facilitates the development of Progressive WordPress themes.  Our colleagues at LinkedIn Learning have been busy developing a Progressive WordPress Theme Boilerplate, which is a theme development starter kit that makes it easier for WordPress theme developers to get from 0 to a fully progressive theme. The boilerplate comes out of the box with:

  • A modern development workflow, which is highly-configurable, and uses tools such as Composer, Gulp, and browser sync
  • A range of coding and performance best practices baked in, such as asynchronous loading of resources, and lazy loading of images
  • And it has opt-in integration with the AMP plugin, so that if the developer so choses, a fully AMP compatible WordPress starter theme can be generated from the get go

The main developer of this project, Morten Rand-Hendricksen, who is also a Senior Staff Instructor at LinkedIn learning, will be publishing soon a course on the LinkedIn Learning platform which will teach all about this boilerplate; and that particular course is going to be free. Don’t miss it!

Adding Incentive Mechanisms to the Ecosystem

One of WordPress major virtues is the flexibility of the platform. As a developer you can accomplish basically anything you can imagine; this is demonstrated by the large and ever expanding ecosystem of plugins available to add functionality and customize your website in a myriad of ways. However, this same virtue can also turn into one of the major challenges of the platform. Even taking into account plugins that are very good (i.e. follow coding/performance best practices and coexist nicely with other site components), the likelihood of having plugins colliding in their interactions (or impacting performance) when jointly activated in a site, is definitely not negligible.

We can deal with this challenge by introducing to the WordPress ecosystem tools that facilitate the verification of well-defined quality standards to allow developers to measure how good their products are in terms of UX-related metrics, and empower WordPress site owners to make choices about which plugins and themes to use, based on actual quality as opposed to only on popularity. We are pursuing efforts in this area, for example in the context of projects such as XWP’s Tide and Lighthouse.

Summary

Success on the web is all about building user-first experiences: it is all about UX. WordPress is an amazing platform and we are striving to close the Capabilities/Usage gap and enable content creation and consumption flows satisfying the pillars of delightful experiences.  v1.0, the latest version of the official AMP plugin (currently in alpha), brings the power and efficiency of AMP to WordPress via a seamless integration with the standard WordPress content creation flow effectively enabling native AMP experiences in WordPress without compromising content fidelity or surrendering the flexibility of the WordPress platform.

Stay tuned for new developments in the areas we mention in this post. We will participating in WordCamp EU in Belgrade this year where we will talk and show the latest progress on our efforts. Hope to see you there!

AMP in WordPress

If you only have one minute to read this post, here is the gist of it: WordPress users can get a first-class AMP experience without compromising the fidelity of their content or surrendering the flexibility of the WordPress platform. Get a glimpse of what is possible with AMP in WordPress by watching our AMP Conf 2018 talk! https://youtu.be/GGS-tKTXw4Y If you have a few extra minutes, follow me to the next paragraphs to learn more about the state of AMP in WordPress.

The WordPress Ecosystem

Screen Shot 2018-02-11 at 8.14.19 AM In a nutshell, WordPress is: a Content Management System (CMS) for websites; the largest player in the CMS space with ~59% of market share, powering about ? of the web; and a thriving ecosystem of plugins, themes, hosting providers, agencies, freelancers, all working together on the creation of WordPress-powered content. But when I think about WordPress I think about one thing: there are many users publishing and consuming content via WordPress-powered sites, and all of them should get an awesome UX on the platform.

And since AMP is all about enabling an awesome (fast) User eXperience to all users, in this post we discuss two things: (1) the case for AMP in WordPress; and (2) the state of the art of AMP in the WordPress Ecosystem. Curious? Read on!

Why AMP in WordPress

One of WordPress major virtues is the flexibility of the platform. You can accomplish basically anything you can imagine, as demonstrated by the large and ever expanding ecosystem of plugins available to add functionality and customize your website in a myriad of ways. However, this same virtue can also turn into one of the major challenges of the platform. Even taking into account plugins that are very good (i.e. follow coding/performance best practices and coexist nicely with other site components), the likelihood of having plugins colliding in their interactions (or impacting performance) when jointly activated in a site, is definitely not negligible.

This virtue/challenge duality of the WordPress ecosystem has its parallel in the open-web web publishing ecosystem. In the latter, there is nothing restricting a web developer to hamper the performance of a site by following counterproductive coding practices (e.g. loading scripts synchronously, render-blocking CSS). And even if the developer builds a site impeccably, adding a third-party JS library could spell a performance disaster.

AMP emerged as a response to that reality, bringing forward effective design principles and control mechanisms that enable sites to offer predictable performance. The effectiveness of AMP and its impact on UX is demonstrated by plenty of data and examples in the wild. And WordPress publishers would also benefit greatly from having more control of the UX of their sites. This could be achieved by adopting coding and performance best practices in WordPress development, and integrating AMP with the WordPress ecosystem to make the WordPress platform more performant.

The plugin work for enabling AMP in WordPress described in this post, was pioneered by the AMP Plugin initially developed by Automattic. Subsequently Google and XWP joint the open source development effort to help evolve the plugin significantly. Want to know what is possible with AMP and WordPress nowadays? Let’s see!

The AMP Plugin

The initial goal of the AMP plugin was to enable AMP for WordPress.com VIP sitesIt certainly met its goals for VIP, and it was also adopted at some level by the general WordPress community either directly, or through other plugins built on top of it. In the context of its initial objectives, the main characteristics of the early versions of the AMP plugin make sense:

  • Developer oriented. The plugin provided a minimal set of core abstractions to enable AMP in WordPress, and allowed developers (i.e. agencies, plugin and theme makers) to modify it and extend it to tailor its capabilities to the needs of their individual sites.
  • Pair mode. The plugin adopted a simple-theme approach bypassing the WordPress template hierarchy and generating a separate AMP version of published content based on very basic theme template parts.

These characteristics worked well for sites that have good engineering support to tailor the AMP content to their needs. However, at a larger scale, the adoption of the plugin was more challenging because the baseline capabilities did not offer enough functional and visual parity between the two versions.

Google and Automattic started working together last year with the goal of bringing the AMP plugin to its maximum expression, and enable a rich AMP experience for all WordPress sites. A few months after,  XWP engineering team also joined the project to push for the same goals with the expanded scope of integrating modern web technologies in general into the WordPress ecosystem. And since everything is open source, the WordPress community has also been part of this work through many contributions. In the past few months versions 0.5 and 0.6 of the plugin were released, including bug fixes, expanded support for embeds, support for pages, parity with AMP library specification, better configuration capabilities, and much more. And after these, we have focused on one of our main initial goals; since the beginning we have had our sight set on enabling an organic AMP experience in WordPress; that is, an experience where there is no need for a pair mode (unless applied by choice) because there are no gaps, either functional or visually, between the AMP and non-AMP versions of content published in WordPress. Without such a gap, WordPress publishers are able to publish a single version of their content which is beautiful, feature-rich, all-around WordPress, and runs at the speed of AMP! We call this an all-AMP experience.

An all-AMP UX in WordPress

The aforementioned pair mode approach makes sense when there are functional and visual gaps between the AMP and non-AMP versions of a given content. In such a case the pair mode keeps a “version” of the original content and an AMP’d version of  it. By enhancing the AMP plugin and devising the right mechanisms for theme/plugin integration, we can eliminate these functional and visual gaps an enable WordPress sites to offer a single, all-AMP experience.

Screen-Shot-2018-02-10-at-4.59.38-PM.pngAn all AMP experience in a WordPress site requires: (1) a plugin that enables a seamless integration of AMP into WordPress; (2) a theme that integrates nicely with AMP (adds AMP theme support); and (3) the seamless coexistence of third-party plugins in an all-AMP site.

We have made good progress on the plugin work and by the time you are reading this post, many goodies will be available in version 0.7 of the plugin (release). We have also made progress on defining an approach for building progressive themes; that is, themes built following the latest coding and performance best practices, leveraging as much as possible progressive technologies in the web platform, and integrated seamlessly with the AMP plugin. The result on this work was presented at AMP Conf (video), and a prototype of an AMP-Compatible theme is available, and you can experience it live as well.  And the cherry on top of the cake, is the work by Morten Rand-Hendricksen on the development of a Progressive Theme Boilerplate project, which enables theme developers to go from 0 to starter progressive theme in an easy way. Today, a native AMP experience in WordPress is possible! On the theme side of the story, native AMP support is enabled by:

  • Integrating with the standard WordPress templating system, as opposed to the AMP plugin’s initial paired mode bundled theme for post templates
  • Supporting standard site elements such as the Widget sidebar, native comments, and many others are fully supported
  • Providing opt-in choice for enabling the experience with fine-grain control: the theme works with or without AMP

And on the AMP plugin side of the story, the seamless integration of AMP into the WordPress workflow is enabled by:

  • Supporting all standard/default widgets; non-core widgets will also work, but anything they add which is not AMP-compatible will be stripped. And we encourage the plugin community to work with us on their integration
  • Providing AMP compatibility for all relevant WordPress actions / filters
  • Preventing WordPress from printing  scripts/styles in the header and footer
  • Having an efficient “AMP preprocessor” ensuring AMP validity with maximum fidelity
  • Providing functionality in the AMP preprocessor to identify and flag incompatible plugins at runtime. Besides helping the publisher to keep their AMP content valid, this also will help guiding the community to work with us to make every relevant plugin AMP-compatible
  • Allowing three “use modes”: (1) legacy pair mode, with the pre-0.6 ad hoc post templating system; (2) new (enhanced) pair mode, integrated with the standard WordPress templating system; and (2) AMP Canonical, with a single AMPed version of the content and also integrated with standard templating system.
  • Supporting all default embeds; other embeds are also allowed but the AMP preprocessor may not make them AMP compatible in the best way (i.e. an may be used)

The combination of these features and capabilities allows WordPress publishers to run a pristine WordPress site that is all-AMP! The following screenshot shows a demo of an all-AMP WordPress site; there is no pair mode, and look at the console! AMP-Canonical-WordPess-Demo

The Road Ahead

AMP integration with the WordPress ecosystem has come a long way. But there is still quite a bit of work ahead of us; very exciting work indeed! Some of the main areas currently capturing our focus include:

  • Continue expanding the capabilities and scope of the AMP plugin to fully enable an awesome AMP experience in WordPress at scale, and collaborate with publishers, WordPress users, and the WordPress developer community, to make all relevant plugins/themes AMP-compatible
  • Continue our work on Progressive Themes aimed at defining a systematic approach to building themes that provide AMP support, follow the latest coding and performance best practices, and integrate the latest advances in the web platform.
  • Align our efforts with the evolution of WordPress via contributions and integrations with Gutenberg, the new editing experience which is set to revolutionize content creation in WordPress. This includes supporting Gutenberg blocks, including blocks for AMP Components. For example, we’re looking to augment Gutenberg blocks with AMP-specific attributes, like the layout and media attributes, which  would then inform the AMP preprocessor on how best transform an element into AMP.
  • Contribute our collaboration with the Tide project, to empower WordPress developers and site owners with the ability to choose themes and plugins based on concrete quality metrics
  • Enable the systematic tracking of the performance of the WordPress ecosystem by leveraging extensive datasets and the power of Google’s data processing infrastructure

Advances in the web platform all aim at improving the user experience along different dimensions. AMP design premises focus on the performance of the mobile web and the associated quality of the perceived user experience. With the AMP plugin and a modern approach to building themes, WordPress-powered publishers are able to leverage the performance and ecosystem advantages to be gained from using AMP. There is still quite a bit of work to be done. These are very exciting times to be a web developer, and also a WordPress developer! We are certainly excited as we continue advancing towards our goals and unlock the even greater potential of the WordPress platform. And remember: everything we are working on is open source, including AMP! All the work described in this post is the result of a great collaboration between Automattic, XWP, Google, and the WordPress community. Want to help? Join the effort and participate with your contributions, feedback, and engagement!

WordPress @ Google: Part I

WordPress-Google

At Google we care deeply about the long-term success of the open web: we work on the core technologies and standards that power the web platform, we develop and maintain the Google Chrome browser, and we build tools and frameworks to help developers build amazing experiences on the web — just to name a few. As a result, it shouldn’t come as a surprise that we are also huge fans of WordPress, which enables publishers to easily create and deliver rich web content to users all around the world.

Taking a guess, I would say the above is not a big shock or surprise. However, did you know that Google is also actively engaged and looking for expert WordPress developers to support and accelerate success of the WordPress ecosystem? It’s true, and it could be you!

Google’s WordPress Vision

The web platform has been evolving rapidly in recent years and it continues to evolve as we speak; the bar for web experiences has been raised significantly. This is reflected by all the buzz around the term Progressive Web Apps, which essentially refers to websites (and web apps) built using the latest capabilities supported by the Web platform; things like Service Workers, Push Notifications, identity management, security, etc. Similarly, the evolution of the platform defined by the WordPress ecosystem has being evolving as well at an accelerated pace.

The goal of our team is to help bring these two worlds together and work with both the WordPress community and core developers, and the web platform developers to enable amazing user experiences on the web — powered by WordPress.

WordPress Experts Wanted

Our WordPress vision covers a lot of exciting work across the full breadth of the WordPress ecosystem: collaborating on WP core, developing themes, plugins, and WP-tailored tools and infrastructure, and engaging with the broader WordPress community of publishers.

Sounds interesting? Come join our amazing Google Developer Relations team. We’re looking for passionate WordPress experts, developers, and advocates that are willing to roll up their sleeves and help us with:

  • Contributing to Gutenberg development to enable rich user experiences on the web
  • Building of progressive themes to accelerate integration of modern web technologies
  • Expanding capabilities of the AMP Plugin to enable delightful user experiences
  • Contributing to the Tide project to empower WordPress developers and site owners to develop and find high quality themes and plugins
  • Develop tools and reports to track the health of WordPress performance
  • And much more…

Some examples tasks that are part of our daily job are…

  • Working with engineering teams at Google to develop relevant product integrations. This includes reviewing API designs, testing of new features, and guiding internal teams to success within the WordPress ecosystem.
  • Collaborating with external WordPress developers to deliver production code to WordPress core, plugins, and themes
  • Working closely with the WordPress community to drive awareness, and adoption of new technologies, as well as coding and performance best practices — e.g. attend and speak at WordCamp events, create written guides and videos, and so on.
  • Participating in technical and design discussions with both technical leads and executives at strategic partners to speed up adoption and ensure best practices during implementation.

Hopefully this description gives you a good idea of the scope and vision of the WordPress-related work we are pursuing in the Web Content Ecosystems Team at Google. Aren’t you super curious and excited? I am!

Join us!

Are you passionate about WordPress and its ecosystem? Do you have strong expertise developing with WordPress? Are you interested in getting involved in projects aimed at making WordPress better at all levels? If your answers to these questions are on the affirmative quadrant you may be interested in joining Google to work with us. Looking forward to hearing from you!

Reasoning about Speed

rail

One challenging aspect of web performance is that it is often addressed once it has become a problem already; this can be referred to as a “reactive approach to handling performance”. A reactive approach is challenging because the symptoms of the problem emerge way after they were introduced early on during the initial design/implementation phases of the a project; and it is often very difficult, if not impossible, to change design/implementation decisions that have pervasively percolated through the whole system.

Another challenging aspect is that we are trying to achieve globally-optimal results (i.e. good overall performance), from a collection of local optimization decisions (e.g. defer the evaluation of a JS file, etc).

Performance models are one approach to mitigate these challenges and enable us to reason about performance. The use of performance models during the process of designing and implementing applications provides guidance to our decision making process; they provide a framework for reasoning about how design/implementation decisions affect performance and user experience and to strike the proper tradeoffs at every step along the way.

Another benefit of defining and using performance models is that they provide a Lingua Franca, a common language for teams to talk about performance and ensure that they are all looking at the reality of the app from the same perspective.

Two specific performance models/patterns relevant for designing/implementing websites are: (1) the Response, Animation, Idle Load (RAIL) model; and (2) the Push, Render, Pre-cache, Lazy-load (PRPL) pattern.

The RAIL Performance Model

The RAIL model seeks to capture the four major areas of a web app life cycle; it stands for: Response, Animation, Idle, Load. If we look at them in chronological order, we would read the acronym backwards:

  • Load. We want our initial load and rendering to happen in about one second.
  • Idle. An app is usually idle while waiting for the user to interact; idle time is divided into chunks of fifty ms each, and they offer a great opportunity to squeeze work that the app needs to do.
  • Animation. This part of the model relates to one of the most challenging aspects of the performance of web apps: the need to achieve animations that render at 60fps (frames per second). This means that we have about 16 ms to render one frame; or even less since in reality, browsers have to squeeze other tasks in that time budget, leaving us with an actual budget of  about 10 ms. Check this reference for more detailed information.
  • Response. This responsiveness corresponds to how quickly the app reacts to user interactions; that is, how quickly the app becomes interactive. Research has shown that there is a limit of about 100 ms from when a user interacts with the application (e.g. tap a button, scroll) and when she gets the desired response.

In practical terms, the RAIL model translates into this :

  • Focus on the user; the ultimate goal is to make users happy
  • Respond to users immediately; acknowledge user input in under 100ms.
  • Animate smoothly. When animating or scrolling, produce a frame in under 10ms.
  • Be efficient. Maximize main thread idle time.
  • Load fast (keep users engaged). Deliver interactive content in under 1000ms.

The PRPL Pattern

The acronym PRLP refers to a pattern for structuring and serving Progressive Web Apps (PWAs), with an emphasis on the performance of app delivery and launching. The pattern comprises the following elements:

  • Push critical resources for the initial route.
  • Render initial route.
  • Pre-cache remaining routes.
  • Lazy-load and create remaining routes on demand.

In a nutshell, PRPL seeks to optimize three aspects of the development, launching, and performance of a web app:

  • Minimize the Time-to-Interactive (TTI) metric: especially on first use (regardless of entry point), and on real-world mobile devices
  • Maximize caching efficiency: Simplify app development and deployment

The PRPL pattern is less about specific technologies or techniques that can be used, and more about a mindset and a long-term vision for improving the performance of the mobile web applications. The goal is to keep the PRPL pattern in mind when planning for performance.

So…

If there is one takeaway from this post, is this: web performance is more than technique; it is a mindset. A mindset that should encompass every aspect involved in the development of our products: designers, developers, decision makers. And for that mindset to be effective, we need to be able to reason about performance. Performance models are your tools to do that and harness control of the performance of your sites and apps.

Progressive Web in LATAM

[Post written with Demian Renzulli, my colleague @ Google]

Almost ten years have passed since the mobile revolution started with the emergence of iOS and Android; nowadays a large percentage of web traffic is generated from mobile devices. In LATAM, this reality is prevalent. A recent study on the penetration of mobile technologies in the region shows incredible stats such as:

  • By 2020, a smartphone adoption rate of 71%, ahead of the global average of 66%
  • An additional 171 million new smartphone users across the region
  • By 2020, 42% of 4G connections, close to the global average of 44%
  • Brazil, in particular, is seeing a strong 4G growth spurt

These stats are staggering and organizations in the region have been rapidly realizing they must provide great user experiences across all platforms, both mobile and desktop.

To achieve an awesome experience on mobile platforms, an organization can choose to develop a native application (iOS or Android). This path can indeed provide a great user experience, but still leaves two aspects in need of solutions: the distribution problem, which is one of the hardest problems in software; and matching the mobile experience on desktop to ensure an awesome UX across all platforms.

The distribution challenges of native apps are apparent from studies like the 2017 US Mobile App Report, which indicates that 51% of users download zero apps per month. Although these numbers may actually be better when based on larger datasets from the Play Store, for example, it still is true that there exist a high friction for users getting to native apps, and that users tend to download and keep only a handful of top-ranked apps. Furthermore, the limitations of the devices predominantly used in the region compound on the distribution challenges for native apps; one of most sold phones in 2017 in LATAM is the Samsung Galaxy J5; this device may be considered above average, but it is still fairly limited in terms of storage and capabilities.

The other option an organization may choose is to develop a web application tapping into the great reach of the web, reducing the storage capacity needed on user devices, and avoiding the high install friction of native apps.

Despite the appealing advantages offered by web applications, native apps have been perceived, rightfully so, as more prevalent than web applications. Historically, there are several reasons for the disparity between the use of native apps vs. web apps: limitations in the mobile platform, which do not play nicely with the traditional web access model; failing to adapt the web development workflow and strategies when moving from desktop to mobile platforms;  and limitations of the web platform, which until not long ago used to lack most of the capabilities needed to provide a user experience on web at par with that provided on native apps.

Of these challenges, achieving high performance on mobile web apps emerges as one of the main ones and certainly the one of most concern and frustrations for users.  But why is it so hard to achieve high performance on web apps? The reasons are two-fold:

  1. Mobile-platform Specific: differences in CPU architectures (e.g. non-symmetric multi processors), differences in storage characteristics (e.g. multi-layer memories), and differences in the quality and operational characteristics of the cellular networks used to connect mobile devices to the Internet (e.g. latency and error rates)
  2. Website specific: page bloat (size and number of resources used), page structure and rendering strategies, and things like intensive animations and other coding anti-patterns.

This is a challenge faced by users and developers alike at a worldwide scale, but it’s exacerbated in the LATAM region where OS fragmentation is the norm, low-end devices represent a high percentage of the market, and connectivities are far from ideal.

Performance is User eXperience

Users have high expectations for the performance of websites; and when such expectations are not met, users react in ways that clearly impact the outcome of key business metrics. There have been a variety of studies aimed and showing this reality. For example, a 2016 DoubleClick study reported the following results:

User expectations

  • 46% of consumers indicate that waiting for pages to load is what they dislike the most when browsing the mobile web
  • 50% of people expects a page to load in less than 2 seconds
  • About 53% of all mobile visits are abandoned if the page loads in more than 3 seconds

Load times

  • 77% of mobile sites take longer than 10 seconds to load
  • On 3G networks, the average time to load is 19 seconds
  • On 4G networks, the average time to load is 10 seconds
Read more