Preamble

We have been looking for ways to expand our cacheable content beyond anonymous requests for a long time. Once a user is logged in, a number of personalizations primarily in the chrome around the content (user name, tabs, links) make it hard to reuse a cached copy of an entire page. Initial trials to perform those personalizations with ESI were done as early as 2004, but even with Varnish testing we have seen performance and stability issues. Server-side composition technologies like ESI or SSI also introduce a second code path, which makes it harder to test and develop front-end code without intimate knowledge of a complex part of our stack.

An alternative is to use JavaScript for the composition. This opens up the possibility of running the same JS code

• on the client, in service workers (essentially caching HTTP proxies running in the browser), or
• on the server, behind edge caches, in a JS runtime like Node.js with an implementation of the ServiceWorkers API, processing both cache misses and authenticated views.

By using JavaScript, we get to use familiar and mature HTML templating systems that have support for pre-compilation. This simplifies the development and testing process. While Varnish performance drops significantly with each ESI include (we measured 50% with five includes), pre-compiled JS templates can potentially perform fairly fine-grained customizations with moderate overhead.

In browsers that support it (like current Chrome, about 40% of the market), we can preload templates and styles for specific end points and speed up performance by fetching the raw content only. By working as a proxy and producing an HTML string, we also avoid changes to the regular page JavaScript. In contrast to single-page applications, we don't incur routing complexity and heavy first-load penalties.

An interesting possibility is to prototype this in a Service Worker targeting regular page views (/wiki/{title}) only, while letting all other requests fall through to the regular request flow.

Proposal

This task is about implementing a minimal service that essentially composes an HTTP response based on multiple other resources that are themselves more cacheable and less variable. Initial design requirements:

• High-throughput. Suitable for handling traffic at the edge, essentially doing only HTTP and string manipulation.
• Request routing.
• HTML Templating. Fetch, precompile and cache templates - presumably Mustache.
• Streamable. Must implement template handling so flushing starts early and continues progressively.
• Export as ServiceWorker. Add an endpoint that exports a JavaScript program compatible with a ServiceWorker that contains all the utilities (elematch, mustache etc.), Router, RequestHandler, and the current install's router configuration and custom RequestHandlers.

Related:

• https://jakearchibald.com/2016/streaming-template-literals/
• https://github.com/digitaldesignlabs/talisman

The Node.js service itself should probably use wikimedia/service-runner and not be specific to MediaWiki in anyway. The service would only know which domain names it serves, and from where it can fetch the sw.js executable.

Dependencies

Making Service Workers work for wiki page views, is impossible without first resolving a substantial amount of technical debt.

• T111588#2416137 at T111588: RFC: API-driven web front-end
• T140664: Achieve predictable MediaWiki routing and cacheable skin data
• more?

I suggest the initial implementation is used for a less complicated use case. For example, the Wikipedia.org portal, which can justify the Page Composition Service to improve their localisation workflow, which is currently not very performant due to client-side XHR, causing FOUCs.

Related work by @GWicke and @Krinkle:

• node-serviceworker-server: Node library that runs an HTTP service. It can be configured to map a domain and request scope to a service worker url. The service will use the node-serviceworker library to turn the service worker script into something that we can instantiate and make a request to on the server-side.
  • Status: Work in progress (https://github.com/gwicke/node-serviceworker-proxy)
  • Task: T116126: Provide server-side ServiceWorker interfaces
• node-serviceworker: Node library that provides a Node sandbox with several browser APIs available in its scope (such as Fetch, ServiceWorker Cache, and more).
  • Status: Work in progress (https://github.com/gwicke/node-serviceworker)
  • Task: T116126: Provide server-side ServiceWorker interfaces
• sw-wikimedia-helpers: Collection of utilities we expect most of our ServiceWorker clients to need. Such as request routing, view abstraction, and a command-line script to generate a compact sw.js file. This utility library will likely make use of:
  • browserify
  • mixmaster: Produce a readable stream from an array of string literals, functions, promises, and other streams. With the option to pass through one or more transforms. This allows progressively streaming to the client with the ability to dynamically substitute portions, and to precompile any templates.
  • elematch: Efficient matching of elements in a stream of HTML. To be used with Mixmaster. This would allow to progressively stream to the client with the ability to dynamically substitute portions.
  • musti: Streamable Mustache renderer. Uses Mixmaster.

See also

• T34618: MediaWiki should support partial page caching (edge side includes)
• ServiceWorkers and Streams for the win by Jake Archibald, showing Chrome 50's experimental streaming response composition support using a Wikipedia frontend.
• Fast and resilient web apps: Tools and techniques - Ilya Grigorik at Google I/O 2016 - 10 % of navigations fail on 2G, and this rate is similar in India & the UK
• Offline Wikipedia demo by Jake Archibald
• Making Netflix.com Faster: Netflix on its move to a JS-only frontend
• Using service workers to adapt to network conditions
• T101731: Leverage Service Workers for performance improvements
• Reflections on 10 years of ESI by Mark Nottingham (2011), including a good discussion on ESI vs. client-side composition with Illya Grigorik
• MDN: Functions / classes available to workers