WebdriverIO with ChromeDriver headless without Selenium

I was excited to discover that WebdriverIO supports the use of ChromeDriver running in headless mode without the use of Selenium (because ChromeDriver supports the WebDriver protocol). Here’s a starter guide (assuming Node.js is installed):

  1. Install Chrome Web Browser
  2. Download ChromeDriver
  3. Run ChromeDriver
  4. Install WebdriverIO by running npm i webdriverio
  5. Run the following file (based on the WebdriverIO example)

const webdriverio = require('webdriverio')
const client = webdriverio.remote({
  host: "localhost",
  port: 9515,
  path: "/",
  desiredCapabilities: {
    browserName: "chrome"
  }
})
client
  .init()
  .url('https://duckduckgo.com/')
  .setValue('#search_form_input_homepage', 'WebdriverIO')
  .click('#search_button_homepage')
  .getTitle().then(function(title) {
    console.log('Title is: ' + title);
  })
  .end()

To run Chrome in headless, use the following config:

const client = webdriverio.remote({
  host: "localhost",
  port: 9515,
  path: "/",
  desiredCapabilities: {
    browserName: "chrome",
    chromeOptions: {
      args: ["headless", "disable-gpu"]
    }
  }
})

Websockets are easy

I was interested in getting a minimal example working for Websockets and it was surprisingly easy to get a demo working between Node.js and a browser.

First install the ws library:

npm install ws

Create an index.js file with the contents:

const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 8080 });

wss.on('connection', function(ws) {
  console.log('Starting connection');
  ws.on('message', function(message) {
    console.log('Received message: ' + message);
    ws.send('You sent ' + message);
  });
});

Start the Node.js process:

node index.js

In a browser console, send a message to the Node.js process:

var ws = new WebSocket("ws://localhost:8080");
ws.onmessage = function(e) {
    console.log(e.data);
}
ws.send('Hello World');

Re-rendering map layers

I recently was optimising the performance of a Leaflet-based map that rendered TopoJSON layers via Omnivore. The layers were a visualisation using the ABS’s Postal Areas, and while there was only a single TopoJSON file, this resulted in a number of feature layers being displayed, with each bound to their own data. The data for each feature layer could change depending on what filters were set (these filters were displayed in a left panel).

In the original implementation, whenever a change was made to a filter, the entire TopoJSON layer was removed, and then re-joined to the data set and rendered:

var dataLayer;
function renderDataLayer() {
    var prevLayer = dataLayer;
    dataLayer = L.geoJson(null, {
        filter: filter,
        style: style,
        onEachFeature: onEachFeature
    });
    omnivore
        .topojson('postal_areas.topojson', null, dataLayer)
        .on('ready', function() {
            // Remove previous layer when current layer is ready to avoid flickering
            if (prevLayer) {
                prevLayer.remove();
            }
        })
        .addTo(map);
}

Using Chrome’s “Record JavaScript CPU Profile”, it clearly showed the code invoking Omnivore was the problem:

Changing the render function to (1) iterate over the existing layers and (2) change the style of the layer, made the map feel a lot more responsive.

var dataLayer = omnivore.topojson('postal_areas.topojson', null, dataLayer);
function renderDataLayer() {
    dataLayer.eachLayer(function(featureLayer) {
        featureLayer.setStyle(style(featureLayer.feature));
    });
}

One of the disadvantages was that we can’t use the filter function on the geoJson object anymore, since in order for a style change to occur, the feature layer must be present (although it can be hidden). This potentially can lead to slowness when dragging or zooming a map.

Another related disadvantage is hiding layers both visually and from mouse events. Setting the opacity and fillOpacity of the layer to 0 will take care of the first, while this CSS will prevent the mouse cursor from changing when hovering over a hidden layer:

/* Don't show pointer on hidden layers */
.leaflet-pane > svg path.leaflet-interactive[stroke-opacity="0"][fill-opacity="0"] {
    pointer-events: none;
}

Leaflet and Google Maps

I’ve recently been developing an application that uses Leaflet to interactivity with a geographic map. One of the business requirements was to use Google Maps as a basemap, since it is pervasively used by our customers. A naive implementation used Leaflet’s tileLayer to render the tiles directly:

var map = L.map('map').setView([-29, 133], 4);
L.tileLayer('https://maps.googleapis.com/maps/vt?pb=!1m5!1m4!1i{z}!2i{x}!3i{y}!4i256!2m3!1e0!2sm!3i349018013!3m9!2sen-US!3sUS!5e18!12m1!1e47!12m3!1e37!2m1!1ssmartmaps!4e0').addTo(map);

This approach, while simple, does not conform to the Google Maps API Terms of Service. Section 10.1.a indicates that:

No access to APIs or Content except through the Service. You will not access the Maps API(s) or the Content except through the Service. For example, you must not access map tiles or imagery through interfaces or channels (including undocumented Google interfaces) other than the Maps API(s).

To conform with the Terms of Service, I initially tried using leaflet-plugins, which creates a layer using the Google Maps Javascript API. However, when panning the map, the Google Map tiles lag compared to vector layers (e.g. polylines, polygons). This is a known, but unresolved issue and occurs because the Google Map setCenter method is asynchronous and thus will get out of sync with the Leaflet controlled elements.

I next used a wrapper around leaflet-plugins. The wrapper (1) hides the leaflet-plugins Google layer and (2) finds the right tile image in the DOM (rendered by leaflet-plugins) and renders it when it becomes available. Here is a simplified example:

var GoogleGridLayer = L.GridLayer.extend({
  // googleLayer is a leaflet-plugins Google object
  initialize: function(googleLayer) {
    this.googleLayer = googleLayer;
  },
  onAdd: function(map) {
    L.GridLayer.prototype.onAdd.call(this, map);
    map.addLayer(this.googleLayer);
    // Hide the leaflet-plugins layer
    $(this.googleLayer._container).css("visibility", "hidden");
  },
  onRemove: function(map) {
    L.GridLayer.prototype.onRemove.call(this, map);
    map.removeLayer(this.googleLayer);
    $(map._container).find("#" + this.googleLayer._container.id).remove();
  },
  createTile: function(coords, done) {
    var img = L.DomUtil.create("img");
    var googleLayer = this.googleLayer;
    var interval = setInterval(function() {
      var src;
      var id = "#" + googleLayer._container.id;
      var googleImg = $(id + " .gm-style img").filter(function(i, el) {
          var src = $(el).attr("src");
          return src.indexOf("!1i" + coords.z + "!") > 0
                 && src.indexOf("!2i" + coords.x + "!") > 0
                 && src.indexOf("!3i" + coords.y + "!") > 0;
        });
        if (googleImg.length) {
          googleImg = googleImg.first();
          src = googleImg.attr("src");
        }
        if (src) {
          clearInterval(interval);
          img.src = src;
          done(null, img);
        }
    });
    return img;
  }
});

This approach seems to work reasonably well and is as responsive as the original tile layer implementation.

Other enhancements I added later were:

  • Caching the image src rather than having to poll the DOM on every create tile request
  • Adding support for Satellite and Hybrid maps (the latter requires a div with two nested images – one for the Satellite tile and one for the labels/roads tile)
  • Adding a timeout to polling – there are certain tile requests by Leaflet that aren’t fulfilled by the Google tile layer. It may be that Leaflet tries to load tiles past the viewport
  • Adding attribution (including logo and copyright) by moving some of the .gm-style elements in front of the GoogleGridLayer. The Google Map stores a reference to these elements and appropriately changes their content.

One of the disadvantages at this stage is that there is no easy method to access Street View coverage tiles without simulating a drag of the Pegman. It would be nice if the Google Maps Javascript API supports a function to turn on/off the Street View coverage tiles.

Combining DataTable’s Ajax and Javascript sources

I often use DataTables as it provides a lot of out-of-the-box functionality for searching, ordering, paginating and use of Ajax data sources. However, using the server-side processing example means the HTML page will load, and then there is a short wait until DataTables fetches the data from the server using Ajax. This can make an application feel slower, since the user has to wait for the page to load, and then wait again for the data to arrive.

While DataTables can load data from a Javascript source, it seems this option is ignored if we want to use server-side processing.

We can eliminate the second Ajax call by embedding the Javascript object into the HTML that would result from the second call. The object should have exactly the same structure that would normally be returned by the server-side processing source, including the recordsFiltered and recordsTotal attributes to give correct counts:

var intialData = {
  "draw": 1,
  "recordsTotal": 57,
  "recordsFiltered": 57,
  "data": [
    [
      "Airi",
      "Satou",
      "Accountant",
      "Tokyo",
      "28th Nov 08",
      "$162,700"
    ],
    [
      "Angelica",
      "Ramos",
      "Chief Executive Officer (CEO)",
      "London",
      "9th Oct 09",
      "$1,200,000"
    ],
...
}

We then need to use the Ajax option as a function and check if we are performing the first or a subsequent render.

$('#example').dataTable({
  processing: true,
  serverSide: true,
  ajax: function(data, callback, settings) {
    if (typeof initialData.done === 'undefined') {
      callback(initialData);
      indexData.done = true;
    } else {
      $.ajax({
          url: "../server_side/scripts/server_processing.php"
        })
        .done(function(data) {
          callback(data);
        });
    }
  }
});

HTML5 presentation with motion background

I was recently looking for a way to present our church’s worship lyrics on top of a motion background. I naturally turned to PowerPoint, but quickly found out that the only way to achieve a seamless transition between slides is to combine all the lyrics on a single slide and use animations to show and hide the lyric text.

This was a clunky solution, and there is a software market specifically for this use case. However, instead of splurging on new software, I wondered if my simple requirements could be met using a HTML5 web page. The only requirements were that:

  • A full screen, edge-to-edge video could be set as the background. It should automatically start and loop.
  • The keyboard arrow and space keys can be used for navigation between slides.
  • Multi-line text must be centered vertically and horizontally on the screen.

I initially looked into using reveal.js and impress.js. It was unclear if the former already supported video backgrounds or if it was still in development only. The latter seemed extremely complicated for my simple use case.

Looking into browser presentations using jQuery, I came across this article which provided a way to navigate between slides represented as divs. There was additional code to support visual navigation buttons in a footer which I didn’t need, so consequently stripped out. It also used a old style way of adding a jQuery keyboard event handler, so I updated that. I also replaced a simple jQuery hide/show with a fadeOut/fadeIn for a little jazz.

I then looked into the video requirement and found this article which had exactly what I needed to create an edge-to-edge video with autoplay and looping. Relevant code:

<video autoplay loop id="bgvid">
  <source src="motion-background.mp4" type="video/mp4">
</video>

#bgvid {
  position: fixed;
  right: 0; bottom: 0;
  min-width: 100%; min-height: 100%;
  width: auto; height: auto;
  z-index: -100;
  background-size: cover;
}

I then needed a way to both vertically and horizontally center text, and I investigated what Flexbox could provide since I had full control of what browser was used (although Chrome has supported Flexbox for a while). While the CSS was extremely simple, I wasn’t sure why my paragraphs on each slide were displayed inline. It turned out that I needed an extra div, so instead of this structure:

<div class="slide">
  <p>Line 1</p>
  <p>Line 2</p>
  <p>Line 3</p>
</div>

I needed this structure:

<div class="slide">
  <div class="content">
    <p>Line 1</p>
    <p>Line 2</p>
    <p>Line 3</p>
  </div>
</div>

Not having any motion backgrounds on hand, I found that www.motionbackgrounds.co had some decent free videos that I could use to test out the concept.

Combining the above, it all looked pretty good in full screen mode. The only deficiency is the lack of a presenter view, where the presentation can be started full screen on a separate monitor. There appears to be an solution using Firefox, but it didn’t seem robust when I tried it.

The source code and example presentation is available on GitHub. I’ve included a motion background created using Premiere tutorial to avoid copyright issues.

Adding CSRF token to jQuery AJAX requests

When using a jQuery-supported framework such as Backbone, underlying jQuery AJAX requests are typically abstracted at the model layer. To insert Cross-Site Request Forgery (CSRF) tokens or other session data into the request, one method is to proxy a method in the call stack and add the token via an option (example). This does have a disadvantage if you need to call $.ajax directly as you’ll need to again insert the CSRF token as a header option.

The DRY way? Use jQuery’s ajaxPrefilter API:

$.ajaxPrefilter(function(options, originalOptions, jqXHR) {
  var token;
  if (!options.crossDomain) {
    token = $('meta[name="csrf-token"]').attr('content');
    if (token) {
      return jqXHR.setRequestHeader('X-CSRF-Token', token);
    }
  }
});