RunJS to RequireJS?

There was a thread that started on the CommonJS list about a transport format, something that works well in the browser via script injection. I sketched out a proposal, Transport/C, that builds on the Transport/B and Transport/A specs.

Transport/C is very similar to some basic mechanics of RunJS but uses require() as the top level function, and supports the special "module" and "exports" free variables used in the normal CommonJS module spec.

In order to prove the concept for Transport/C, I made a branch of the RunJS code, calling it RequireJS, that implements Transport/C.

It seems like it fits with the existing CommonJS module spec, but is something that works well in the browser. I also made a simple conversion script that converts traditional CommonJS modules to this format.

I am tempted to convert from RunJS to this RequireJS branch, and to start evangelizing that approach for browser toolkits. It would be great if Transport/C would also be approved as the transport format for CommonJS too.

Kris Kowal has some concerns about the *very* long-term effects of the approach. I read his comments as possibly pointing out some things that would be done differently if the primordials and e-maker type of modules were ever accepted as part of an ECMAScript standard.

As I read the primordials and e-maker strawman proposals, I think the only difference is Transport/C functions are only expected to be called once, but e-maker style would favor calling the function on every require() call. As I say in my response, I believe e-maker support, would affect regular CommonJS modules in the same way as the transport format, and it is assuming the strawmans make it in to the spec at some point, as they are specified now.

I also believe how it works in how I coded Transport/C as part of the RequireJS branch is what a normal developer would expect, and I think fits better with existing browser/script behavior, and the assumptions that go along with coding CommonJS modules today.

So I am tempted to rename the RunJS project to RequireJS and proceed with that. If you have any feedback to the contrary, please let me know. Otherwise, I will likely do the change early this week.

Script async, Raindrop and Firefox 3.6

In honor of the Firefox 3.6 release, I upgraded Raindrop to use the new async attribute for script tags.

Why is async neat? It does not block the rest of the page, and will just evaluate the script once it is retrieved. More information is in the HTML5 spec. Note that the script you add async to should NOT use document.write(), as doc.write will likely destroy your page.

Also, be aware that async is a boolean attribute, but that does not mean you should use async="true" to turn it on. The HTML5 spec on boolean attributes says that a value of empty string or a string that matches the attribute name should only be used. To avoid async, just do not include the attribute. For Raindrop, I used async="async" since that looks better to me than an empty string.

Raindrop uses RunJS for the module loader, and RunJS uses dynamically added script tags via head.appendChild(), so the modules loaded by RunJS already behave in an async manner.

However apps that want to use the raindrop front end libraries normally include a script called rdconfig.js as their own script file, and that config file does a document.write to write out the Dojo+RunJS and jQuery tags. Those Dojo+RunJS and jQuery tags now use the async attribute.

RunJS Dependency API

In my last post, I talked about a suggestion from David Ascher to try to improve the syntax of specifying dependencies when declaring a module. Here is an example of that suggestion, using run.def(), a possibly new API dedicated to just defining a module called "rdw/Message":

run.def("rdw/Message", {
rd: "rd",
dojo: "dojo",
Base: "rdw/_Base",
friendly: "rd/friendly",
hyperlink: "rd/hyperlink",
api: "rd/api",
template: "text!rdw/templates/Message!html"
}, function(R) {
//Module definition function.
//Use things like R.api and R.Base in here that map to the modules up above.
...
});

While that does make it clear what each module name's variable will be inside the function that defines the module, it has the following drawbacks:

1) It hurts minification -- having properties off the R object passed to the module function means that minification tools will not be able to minify those references as easy.

2) All the modules dependencies must be referenced via a prefix "R.". For example R.api. It is a small bit of typing and an extra property lookup. That might be seen as an advantage too -- it is clear that a symbol is a dependency because it is off the R. function.

3) Makes it hard to find typos for properties on the R. function. With the current runjs format, JSLint can actually find typos for a dependency's variable name.

4) This syntax is more verbose for JS files that do not care about scope encapsulation. For JavaScript libraries like jQuery, MooTools and Prototype, they pretty much operate in the same global scope. jQuery does have a noConflict(), but that just helps if there is only one other thing called $ in the page. MooTools and Prototype add things to global prototypes.

So for these libraries, specifying dependencies is more like just specifying script tags, and they do not need a local-scoped variable in the function module to get things defined. They would just need to do the following:

run.def("rdw/Message",
["some/module", "something/else", ...]
function() {
//This function does not need some locally scoped variables, and most
//likely, the script dependencies above may not call run.def() and define
//an object, just add things to the global space
});

So for these libraries, it would be onerous to be forced to create variable names for each of those dependencies.

This last point seems to be enough to tip the scales back to using the current model used by run. However I am aware that it is possible for the developer to not get the order or number correct, matching the dependency string with the correct variable for the function.

I am hoping using a coding standard like the following will help:

run.def("rdw/Message",
["rd", "dojo", "rdw/Base"], function(
 rd,   dojo,   Base) {
 //Define the module for rdw/Message and return it.
});

Basically, make sure all dependencies are on one line, along with the function keyword, then put the variable that matches each dependency aligned directly under the depdendency name (the example above may not be correctly aligned depending on the font or format you are viewing this message).

I purposely trimmed the list of dependencies, so I can get this example to show up in this blog. That is the down-side with this sort of code style: it can have a very long line for the dependency names.

For Raindrop, I want to try to keep local scope encapsulation, particularly since I expect some slicker extensions to it that may introduce other code into the page that may conflict. However, if I did not care to do that, I could shorten up the above example quite a bit.

So at this point, I am favoring making it easy to use RunJS for other toolkits that do not care about local scope encapsulation and detecting bad references to variables inside the module definition over avoiding errors with a mismatched function variable name to a dependency name.

It is a hard choice to make. Neither path is perfect. If you have an opinion, feel free to share it.

Raindrop + JSLint + JSDoc + RunJS

I just pushed some changes into the front-end code for Raindrop. Besides using JSLint as the code style, and JSDoc for the format of the files, it now uses a modified version of Dojo 1.4 that uses the RunJS code loader.

I am fortunate enough to work with people that will tolerate this sort of brief experimentation to help improve the state of browser code loading. Do you know Mozilla Messaging is hiring? Great people, great technology, and a great mission. OK, end of the sales speech.

The code conversion to JSLint, JSDoc and a new loader all at one time took longer than I would have liked, but I am glad I did it.

For one thing, the code looks a lot more uniform thanks to using JSLint. While I do not care for all of the rules mandated by JSLint, no one ever likes a coding standard 100%, and JSLint is widely known and a programmatic code checker.

Converting Raindrop to RunJS was really beneficial for RunJS. While RunJS has unit tests, nothing helps shake out the kinks like a large project. RunJS is now more robust because of it.

If you want to peruse the Raindrop code, you can hop to the Raindrop Mercurial source web view. That link will take you to Raindrop's client/lib/rdw directory, where we keep the UI widgets. You can click on some files to see how they look with the RunJS loader and the new formatting.

One thing that became apparent, particularly when talking to David Ascher about the code: the way the dependencies in RunJS are specified as an array, and having a separate list of function arguments that must match that array might be prone to errors.

Here is an example from rdw/Message:

run("rdw/Message",
["run", "rd", "dojo", "rdw/_Base", "rd/friendly", "rd/hyperlink", "rd/api",
"text!rdw/templates/Message!html"],
function (run, rd, dojo, Base, friendly, hyperlink, api, template) {
 ...
});

I condensed the run boilerplate to the top to three lines, otherwise, it would have been quite long vertically to express it all. But you can see the problem. Looks scary.

David and I talked about it. I talked about how YUI uses just a Y object as the only argument to the function, but I did not like how they use odd module names like "yui-anim", and then you have to know that creates a Y.anim property that you can use in your function.

So David suggested the following:

run("rdw/Message", {
 rd: "rd",
 dojo: "dojo",
 Base: "rdw/_Base",
 friendly: "rd/friendly",
 hyperlink: "rd/hyperlink",
 api: "rd/api",
 template: "text!rdw/templates/Message!html"
}, function(R) {
 //Use things like R.api and R.Base in here that map to the modules up above.
 ...
});

I like this better. It is much clearer what module gets assigned to what variable, no "off by one" errors.

The downside: it does not minify as nicely as the existing run format. However, in this case, avoiding developer errors probably trumps the minification cost, and the extra cost to type R. in front of the dependent module references. You can call it r. if you want. Hmm, I might prefer the lower-case version. Easier to type. Although the capital stands out better.

So I am very tempted to convert RunJS to use this format for listing dependencies and passing them to the function callback. I will likely do the work this weekend.

Update: I decided not to do this, favoring the original format since it has other advantages.

Another thing we talked about was the multiple invocations of run(): run() can be called with a starting string argument to define a module, but then can also be called without that starting string argument if you just want to run some code that has some dependencies.

David felt it would be clearer if those two actions were two different function names. I can see that being clearer. So what about the following for the API:

• run(), for just running code that does not define a module.
• run.def() for defining a module.
• run.mod() for a module modifier (the existing run.modify() API call)
• run.get() for getting a defined module after initial module evaluation (needed for some some dynamic cases where you do not know the module name before-hand and in circular dependency cases). This is an existing API.

So the changes are renaming run.modify() to run.mod() and moving the run() call style that defined a module to run.def().

Thanks to David for the feedback, and for the rest of the Raindrop team for being patient with me as I did some experimentation in browser module loading.

Speaking of feedback on RunJS, Rawld Gill (one of the authors of Mastering Dojo) was already working on a loader, and he did some work to convert it to use the same API as RunJS. I am still processing the results, but it is great to see alternate implementations. I am hoping we can take the best of both and make a great loader. I already changed the module evaluation algorithm to use the recursive style he suggested.

RunJS takes its roots from the cross-domain Dojo loader, and that loader was constructed when Dojo had to support Safari 2. Safari 2 has a ridiculously small call stack, so a recursive module evaluator blew up very easily. I switched to a model that traced the dependencies to work out an array of sequenced modules, then a separate loop to then call them into being. That allowed the loader to work in Safari 2, but now that we do not need to support that browser, the more natural, straight-forward recursive model can be used.

Really neat stuff! Feels like we are getting close to a robust but compact loader.