First Test Suite Complete

The first version of the test suite is completed now and everything is passing, testing 6 levels of child worker nesting, and some core functionalities such as importScripts, self alias for the worker global scope, the lack of "window" and hiding the worker global scope constructors.

The results of the test suite are this:

worker received: worker
worker: JSUNITY: Running unnamed test suite
worker: JSUNITY: 4 tests found
worker: JSUNITY: [PASSED] testImport
worker: JSUNITY: [PASSED] testWindow
worker: JSUNITY: [PASSED] testSelf
worker: JSUNITY: [PASSED] testWGSVisibility
worker: JSUNITY: 4 tests passed
worker: JSUNITY: 0 tests failed
worker: JSUNITY: 2 milliseconds elapsed
[object Object]
worker: JSUNITY: Running unnamed test suite
worker: JSUNITY: 1 test found
worker: JSUNITY: [PASSED] testInnerWorkerExists
worker: JSUNITY: 1 test passed
worker: JSUNITY: 0 tests failed
worker: JSUNITY: 0 milliseconds elapsed
[object Object]
worker-child queue: worker-child
worker received: another message from parent
worker received: worker-child received: worker-child
worker received: worker-child: JSUNITY: Running unnamed test suite
worker received: worker-child: JSUNITY: 4 tests found
worker received: worker-child: JSUNITY: [PASSED] testImport
worker received: worker-child: JSUNITY: [PASSED] testWindow
worker received: worker-child: JSUNITY: [PASSED] testSelf
worker received: worker-child: JSUNITY: [PASSED] testWGSVisibility
worker received: worker-child: JSUNITY: 4 tests passed
worker received: worker-child: JSUNITY: 0 tests failed
worker received: worker-child: JSUNITY: 3 milliseconds elapsed
worker received: [object Object]
worker received: worker-child: JSUNITY: Running unnamed test suite
worker received: worker-child: JSUNITY: 1 test found
worker received: worker-child: JSUNITY: [PASSED] testInnerWorkerExists
worker received: worker-child: JSUNITY: 1 test passed
worker received: worker-child: JSUNITY: 0 tests failed
worker received: worker-child: JSUNITY: 0 milliseconds elapsed
...

I need to add tests for the other portions of the API that are completed as well as the portions which are not. Once I do that, I will start working on using a MessagePort implementation for the communication between the worker and worker global scope. This should allow me to plug in communication based on local storage or web database, which is necessary for implementing SharedWorker.

The work I have been doing on the postMessage and onmessage events and how they queue when the objects are not ready should make it simpler to implement the MessagePort / MessageChannel structure.

Right now the most glaring problem is that there is significant duplication of the onmessage code between DedicatedWorker and WorkerGlobalScope. I was allowing this because the code in WorkerGlobalScope uses a closure to remember the scope inside the workerPool onmessage method and the DedicatedWorker didn't need it, but there shouldn't be any reason not to just store the scope reference for DedicatedWorker, too.

I need to experiment with where I can store that function for both objects to use, however. I am finding that the structure of the current objects and how they are passed around is proving quite limiting in terms of allowing for shared functions. Solving this will be even more important as I design a larger framework that includes other parts of the HTML5 and related APIs, available inside and outside of the worker scope.

I will try to create an object called html5shims and attach all of the shim API implementations to the global scope from within the closure surrounding html5shims. The current implementation of Worker uses a style of function assignment which is not compatible, so I will have to check again which browser was requiring that and see if I can find a different way.

Here's that style if you're curious:

Worker = (function InitWorker(window,navigator,wgsSource) {
                  function DedicatedWorker (url) {
...
                  }
              return DedicatedWorker;
          })(this,...);

with (foo) { function bar() {} } fubared

The last two days I've been working on a test suite for my webworker shim and found a couple of wrinkles.

The first was that messages were being lost when they were sent immediately after the worker creation.

For example:

var w = new Worker("foo.js");
w.onmessage = function (event) {
    alert(event.data);
};
w.postMessage("foo");

The message "foo" would never reach the worker. This was solved simply enough by queueing messages when the communication channel is not ready. Which brought me back to looking at the spec for MessagePort and MessageChannel, but that's not important right now.

The really wrinkly thing that I'm seeing now is based on this test:

function testImportScripts () {
   importScripts("../scripts/import.js"); // declares function importedFunction
   assertNotUndefined(importedFunction,"imported function is defined");
   ...
}

First I was getting reference errors telling me that importScripts was undefined. I was able to get around that by surrounding the call to importScripts in a with (this) {} block. That worried me. And importedFunction was still undefined.

So I write a simple worker script:

onmessage = function (event) {
 postMessage("received: " + event.data);
};

importScripts("../scripts/import.js");
importedFunction();

No importedFunction. However, with importedFunction assigned instead of declared -- viola! function declarations inside of with blocks are not supported according to ECMA-262 3rd edition. It makes sense, but it puts a bit of a crimp in my plans. (I found this post which confirmed my suspicion, I didn't actually dig through the spec to find out!

The current version of the WorkerGlobalScope uses this code inside of it's _executed method for all internal executions:

Function("with (this) { " + source + " }").call(this);

I strongly prefer to maintain the use of with so for now I am modifying the code passed to the worker to make function declarations function assigments. The latest version of the code and the tests so far are checked in to SVN.

Web Worker API Shim Demo Posted

I finally got the Web Worker API shim demo posted on Google Code tonight.

There is an interesting trick to posting the demo HTML page where you need to set the svn:mime-type property to "text/html" or it won't be served as HTML. Makes sense but confused me for a while and made me sidetrack looking at cleaning up my old portfolio site.

Not that that wouldn't be a great idea, too.

Web Worker Demo

To see the shim, visit that URL with a non-supporting browser that has Google Gears installed. You may also be interested in comparing it against browser implementing web workers natively such as FF 3.5.

The shim is implemented via two classes. Probably the simplest way to understand how they work is from the inside out. Implementing the Worker object and the worker-to-worker-thread messaging mechanism on top of Gears is pretty simple. But the API doesn't just cover instantiating a worker, giving it some work to do and talking to it. The API defines a window-like environment that the worker thread operates in, where other APIs are available:

WorkerGlobalScope

Once the WorkerGlobalScope is created, the Worker itself is a dispatcher, sending and receiving messages. So what does the WorkerGlobalScope look like and how do we make one?

The HTML5 Web Workers API spec covers the WorkerGlobalScope in detail. WorkerGlobalScope.js is my implementation. In it, we have constructor functions for each of the WorkerGlobalScopes (Dedicated, Shared and the "base") which handle setting instance variables both private and public. We also have DedicatedWorkerGlobalScope onmessage prototype and a prototype object which handles all of the methods available in any WGS, including empty Worker and SharedWorker constructor methods.

But the WorkerGlobalScope established by the constructor and prototype is incomplete -- not only because I haven't finished everything! It is missing the very crucial piece of providing a working Worker implementation.

Of course, we can't have a WorkerGlobalScope without having already created a Worker. The implementation of Worker is provided not directly by WorkerGlobalScope but by the calling Worker. The DedicatedWorker.js file bootstraps the Worker into the originating window environment with the InitWorker method that returns the DedicatedWorker constructor.

Simplified:

 Worker = (function InitWorker(window) {
  function DedicatedWorker(url) {
  ...
  }
  return DedicatedWorker;
 })(this,navigator,WorkerGlobalScopeSource);

The InitiWorker method provides closure around the bootstrap parameters. "this", the global scope for workers. "navigator", the navigator object passed through from the global scope. The source which can be used to create the WGS. Because it is named, it can be easily passed as a string via function decompilation into the Gears worker pool thread. It is my understanding that because it is defined as part of an assignment operation, it does not pollute the namespace (I think there are some finer points there, but I don't know them as well as I would like).

When we create the code to execute our worker's payload in the workerPool, which must instantiate the worker global scope and make all of the APIs available to the worker, we basically repeat the exact same construction. There it is text and not source code, but it is still the same thing. In this way, we share the source code for Workers and WorkerGlobalScope with all child instances even though we cannot directly pass the objects.

The code which is executed by the Gears workerPool thread:

this._source = [
 wgsSource,
 "var wgs = new DedicatedWorkerGlobalScope(\""+url+"\");",
 "wgs.navigator = " + toSource.call(navigator)+";",
 "wgs.navigator.online=true;",
 "Worker=wgs.Worker=("+InitWorker+")(wgs,"+ toSource.call(navigator) +",'"+ escapeQuotes(wgsSource)+"');",
 "wgs._loadSource({url: ['" + url + "'], "+
 "callback: function(scripts){ wgs._scripts = scripts; wgs._execute('importScripts(\""+url+"\")'); } });"
].join("\n");

Matters are complicated by the fact that the HttpRequest object (analog to XHR) available in Gears does not allow for synchronous operations but that is a story for another day.

As the credits on the demo say, thanks to Andrea Giammarchi for pointing me at the need for this shim.

HTML5 Shims, Shivs, Fallbacks and Compatibility Layers

I am working on my implementation of HTML5 WebWorker shim via Google Gears plugin (currently). Part of what is interesting is that the WebWorker has a lot of the other HTML5 APIs available internally, so I am also looking to bring in other shims / write my own if I want the project to be complete and provide an actual fully-functioning Worker-alike and SharedWorker-alike.

As a result, I've decided to start a project to collect the available shims. Rather than just the Web Worker, I will try to bring together as much of the HTML5 js apis as possible. The project is called "html5-shims" and is hosted on Google Code.

I have added a page to the wiki for my worker shim/html5 shims project which lists other shims available, the relevant specs, etc.

HTML5 Shims : Links & Resources