documentation for the more resilient config.json loading routine (ref. #48)

docs/ARCHITECTURE.md

@@ -179,3 +179,19 @@ To enable content-based MIME type guessing, set the `useMimeSniffingLibrary` to
 By default, content-based MIME guessing is *disabled*, because it is somewhat slower (content needs to be read, inspected, and compared to a lot of signatures), and relies on an external library that needs to be distributed along with LibResilient, while not being needed for most plugins, nor necessary for those that do actually need to guess content MIME types.
 
 When enabled, content-based MIME guessing is attempted first for any given piece of content that requires it. If it fails, extension-based MIME guessing is then used.
+
+## How configuration file is processed
+
+When loading the configuration file, `config.json`, LibResilient's service worker attempts to make sure that a broken config file is never used.
+
+The file's syntax is checked before any attempt at loading it is performed. When loading the file, its contents are merged with built-in defaults, which provide a sane baseline. Once a config file is successfully loaded, it is also cached in a way that provides fall-back in case loading a newer version later on fails for whatever reason.
+
+There are two levels of cache of the `config.json` file employed here: "*regular*" and "*verified*". When attempting to load the configuration:
+1. first the "*regular*" cache is used;
+2. if it is empty or the configuration file is invalid for whatever reason (fails the syntax check, or loading it errors out), the "*verified*" cache is used;
+3. if that in turn is empty or broken, a regular `fetch()` is attempted to the original website;
+4. and finally if that fails for whatever reason, the built-in defaults are used.
+
+Whenever a configuration file is successfully loaded and applied, it gets saved to the "*verified*" cache, so that it is available as a known-good fall-back in the future. After the `config.json` file is loaded and applied, if it was loaded from any of the caches it is checked for staleness. If it is stale (older than 24h), a an attempt is made to retrieve a newer version through the currently configured plugins. If it succeeds and the retrieved `config.json` passes verification, it is cached in the "*regular*" cache, to be used next time the service worker i initialized.
+
+This verification involves checking the syntax of the file and if it contains all the necessary fields. If the file was retrieved using means other than regular `fetch()`, it is *also* checked in case it requires any plugins whose code has not been loaded in the currently deployed service worker. If it does, it is discarded — the Service Workers API specifies that code loaded by the service worker can *only* come from the original domain; if the config file was loaded using some other means, it might not be possible to load the necessary plugin code when initializing the service worker later.

docs/FAQ.md

@@ -75,24 +75,28 @@ When debugging code on a developer machine, you can disable cache in your browse
 
 ### Can LibResilient's service worker be updated when the original domain is not accessible?
 
-No, it cannot. This is clearly specified in the API. Until the website on the original domain becomes available again, the code associated with the service worker cannot be updated or modified.
+LibResilient's service worker **code** (including plugin code) cannot be updated through alternative transports. However, the **configuration** of LibResilient's service worker and plugins **can** be updated even if the original website is down.
 
-In the context of LibResilient, this means that even when LibResilient is deployed on a website, and works as expected (pulling content from alternative endpoints or through alternative transports), as long as the original domain is down it is *impossible* to update the service worker or load new plugins.
+The Service Workers API clearly specifies that any code that is run in a service worker context can *only* be updated from the original website, through a secure connection. Until the website on the original domain becomes available again, the *code* associated with the service worker cannot be updated or modified.
 
-Please note: if the original domain is hijacked or otehrwise taken over, whoever controls it can deploy their own service worker code that will be loaded by visitor's clients, as long as they use the same path and file name of the service worker script. In a situation where the original domain has been taken over or can be expected to be taken over, it is strongly advised to deploy configuration changes that redirect visitors to a new domain (for example, by using the [`redirect` plugin](../plugins/redirect/)).
+In the context of LibResilient, this means that even when LibResilient is deployed on a website, and works as expected (pulling content from alternative endpoints or through alternative transports), but the original domain is down, it is *impossible* to update the service worker code, load new plugins, or update the code of existing plugins.
 
-It is possible, with some caveats (see below), to change LibResilient's configuration (including configuration of already loaded plugins) even if the original domain is down, as `config.json` is not a JavaScript file and therefore it is *not* treated as code by browsers. You can read more on updating LibResilient's configuration during disruption [here](./UPDATING_DURING_DISRUPTION.md).
+*Please note: if the original domain is hijacked or otherwise taken over, whoever controls it can deploy their own service worker code that will be loaded by visitor's clients, as long as they use the same path and file name of the service worker script. In a situation where the original domain has been taken over or can be expected to be taken over, it is strongly advised to deploy configuration changes that redirect visitors to a new domain (for example, by using the [`redirect` plugin](../plugins/redirect/)).*
+
+However, It is possible — with some caveats (see below) — to change LibResilient's *configuration* (including configuration of already loaded plugins) even if the original domain is down, as `config.json` is not a JavaScript file and therefore it is *not* treated as code by browsers. You can read more on updating LibResilient's configuration during disruption [here](./UPDATING_DURING_DISRUPTION.md).
 
 ### Can LibResilient's plugins be updated, or new plugins loaded, when the original domain is not accessible?
 
 No. As discusse above, the Service Worker API does not allow any code to be loaded otherwise than via direct request to the original website.
 
-On the other hand, configuration changes that *do not* require loading new (previously not loaded) plugins or loading new versions of already loaded plugins, [are possible](./UPDATING_DURING_DISRUPTION.md) even if the original website is down, as long as content is available through any of alternative transport plugins.
+On the other hand, configuration changes that *do not* require loading new (previously not loaded) plugin code or loading new versions of already loaded plugins, [are possible](./UPDATING_DURING_DISRUPTION.md) even if the original website is down, as long as content is available through any of alternative transport plugins.
 
 ### Can LibResilient's configuration be updated while the original domain is not accessible?
 
 Yes, configuration changes that *do not* require loading new (previously not loaded) plugins or loading new versions of already loaded plugins, [are possible](./UPDATING_DURING_DISRUPTION.md) even if the original website is down, as long as content is available through any of alternative transport plugins.
 
+LibResilient's service worker also goes to great lengths to ensure that a broken configuration is not accidentally loaded. This means that LibResilient's service worker will not load a configuration file that requires loading new plugin code, unless the original website is available.
+
 ### Can a buggy service worker be removed from clients?
 
 Yes, as long as the website on the original domain is up. The Service Worker API only allows installing, updating, and removing service workers for a given website based on requests made to that website directly.

docs/UPDATING_DURING_DISRUPTION.md

@@ -1,14 +1,16 @@
 # Updating the configuration during disruption
 
-If a website is experiencing an outage, but LibResilient is deployed on it and is working correctly for the visitors whose browsers were able to install and activate it before the disruption started, it is possible to update LibResilient configuration using any alternative transport plugins available.
+If a website is experiencing an outage, but LibResilient is deployed on it and is working correctly for the visitors whose browsers were able to install and activate it before the disruption started, it is possible to update LibResilient configuration using any alternative transport plugins that were configured originally.
 
-It is not, however, possible to load plugins that have not been previously loaded based on the previous `config.json`, as this would require a call to `importScripts()` and thus: a direct HTTPS request to the original domain.
+It is not, however, possible to load new plugins that have not been previously loaded based on the previous `config.json`, as this would require a call to `importScripts()` and thus a direct HTTPS request to the original domain.
+
+When loading the configuration file that has been retrieved using alternative transports, LibResilient will reject it if it specifies plugins which have not been loaded previously.
 
 ## ServiceWorker API and `importScripts()`
 
 The [Service Worker API](https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API/Using_Service_Workers) allows for updating the service worker script itself and any imported scripts (via [`importScripts()`](https://developer.mozilla.org/en-US/docs/Web/API/WorkerGlobalScope/importScripts)) only via a direct HTTPS request to the original domain.
 
-LibResilient expects configuration in a `config.json` file, not in a script, so that it can be retrieved and used without a call to `importScripts()`. This allows updating the configuration during disruptions. The `importScripts()` limitation still affects plugin scripts, however: it is not possible to load new plugin scripts while the original website is experiencing disruption or outage.
+LibResilient expects configuration in a `config.json` file, not in a script, so that it can be retrieved and used without a call to `importScripts()`. This allows updating the configuration (but not code) during disruptions. The `importScripts()` limitation still affects plugin scripts, however: it is not possible to load new plugin scripts while the original website is experiencing disruption or outage.
 
 ## LibResilient's Service Worker lifecycle
 
@@ -26,7 +28,7 @@ When installing or handling any kind of event, the LibResilient Service Worker s
 
 If there is no `config.json` available in cache, the only way to get it at this stage (as plugins are not configured yet) is a regular HTTPS `fetch()` request to the original domain. If that works, the retrieved `config.json` is cached for future use (that is, next time when the Service Worker has been stopped and is being started again) and used to configure and initialize the plugins.
 
-If there is no `config.json` available in cache, and the `fetch()` request does not produce a valid `config.json` either, LibResilient falls back to the default config (that is, a pretty trivial set-up using `fetch` and `cache`). At this stage LibResilient Service Worker can start handling requests generated by the open tabs.
+If there is no `config.json` available in cache, and the `fetch()` request does not produce a valid `config.json` either, LibResilient falls back to the default config (that is, a pretty trivial set-up using `fetch` and `cache`). At this stage LibResilient Service Worker can start handling requests generated by any open tabs in visitor's browser.
 
 ## Updating the config file
 
@@ -35,3 +37,31 @@ After configuring and initializing the plugins, LibResilient checks if the cache
 However, due to the `importScripts()` limitation, if a `config.json` file is retrieved using alternative transport plugins (not via the `fetch` plugin), it is expected *not to* introduce plugins that have not been used before — only to change the configuration of already-loaded plugins. If a plugin that has not been loaded beforehand is present, that `config.json` file is ignored and discarded.
 
 The reasoning behind this is that at this stage we do have a working configuration, as we were able to retrieve `config.json` even though the original website is not available. So it's better to keep using that proven configuration, rather than risk having to call `importScripts()` for a new plugin, which might fail due to original website apparently not being available.
+
+## Pre-loading of plugins in case of disruption
+
+Plugins can be configured in the `config.json` file as `enabled: false`. For example:
+
+```json
+{
+    "plugins": [{
+        "name": "fetch"
+    },{
+        "name": "alt-fetch",
+        "endpoints": [
+            "https://example.com/"
+        ]
+    },{
+        "name": "redirect",
+        "enabled": "false"
+    }],
+    "loggedComponents": ["service-worker", "fetch", "alt-fetch", "redirect"],
+    "defaultPluginTimeout": 1000
+}
+```
+
+This is a pretty simple `config.json` that configures LibResilient to retrieve any website content using regular `fetch()`, but with a fall-back to an `alt-fetch` alternative transport with one endpoint in case the original website is inaccessible.
+
+Note that the `redirect` plugin is configured as disabled. What this means is that its *code* of this plugin will be loaded (using a call to `importScripts()`) into the service worker (as long as the original website is available), but it will not be instantiated nor used for handling any requests.
+
+If the website becomes unavailable for whatever reason, the `alt-fetch` plugin will allow the service worker to pull an updated `config.json` file from and alternative endpoint. That `config.json` file can then activate the (already loaded, but inactive) `redirect` plugin to redirect any and all requests to a completely new domain.