AMD support for Underscore and Backbone

As mentioned previously, at the moment there is no support for AMD in Underscore, a decision that also affects Backbone.

However, there have been enough developers interested in using Underscore and Backbone with AMD that forks were set up under the amdjs organization that have optional define() calls in them:

https://github.com/amdjs/underscore

https://github.com/amdjs/backbone

Ryan Florence suggested the AMD community maintain these forks, and if you would like to see AMD support as part of the main repos, click on the links above and click the Watch button. But only do it if you actually use the libraries above in an AMD context.

Hopefully this will give the DocumentCloud folks some indication of how many people might benefit from having support directly in the main repos, and maybe over time AMD support can be added to the main repos.

Both forks optionally register as AMD modules but also export global objects. This is to address the difficulty Underscore saw when only doing the AMD call, which led to the AMD removal from Underscore.

If you want an easy way to fetch these AMD versions from the command line, or to have a tool to make it easy to AMD-wrap older releases of those libraries, use volo. The README's Usage section has specific instructions on how to fetch these libraries.

If you would like to try an AMD+Backbone option that does to not modify the source files, check out Tim Branyen's explorations.

AMD, the question, and a JavaScript sugar analogy

I want to expand more on the feedback from Tom Dale and Patrick Mueller (see part 2 too), who do not think AMD is the answer, and Dave Geddes asking at the end of his post what was the question?

More on the question, and an analogy:

How can we get to a future where we can easily share JS code, one that works on the server and in the browser? 

I have been talking about the advantages of AMD because it answers that question very well, and I have been reading people's responses to suggest CommonJS (CJS) modules as the format as impractical because it cannot work everywhere. Maybe what we can do is agree on some basics that still allow for sugar.

In other words, consider AMD the "base format" for distributing JS code, because it can work anywhere. That was its design goal. However, it is fine to use sugar on top of that, and even distribute source in that sugared form. But it would be great if the runtimes/tools you use have the capability to ingest AMD (if you take in outside code) and the tools that output your code should be be able to register as AMD modules.

An analogy:

JavaScript is winning because it is widely deployed. However, not everyone wants to program in plain JS. Some folks like to use sugar, like CoffeeScript and Stratified JS. In the end though, those sugared forms reduce to JS so that it can run in the most widely deployed environment.

AMD is to JavaScript as CJS modules/browser globals are to CoffeeScript/Stratified JS

It is fine to use sugar, it just helps if that sugar can take input/generate output that works everywhere. A bit more on why AMD can work everywhere:

Wrap It Up

Any complete JS module solution needs to have a wrapped format to allow:

1) serving optimized code, where multiple modules are combined into one file.
2) dynamically loading code from CDNs.

AMD solves this need, by allowing string IDs and a function wrapper, the shortest form is here:

define('module/id/here', function (require) {

    //Module code in here.

});

The current ES harmony module proposal may be able to get around a wrapped format for case #2 (cross domain script fetching), but they need a "wrapped" format to allow #1, and it looks like:

module moduleIdHere {

    //Module code in here.

}

I think it is better to use string IDs for the harmony case, and harmony is by no means done, but still, the need is the same: when combining multiple modules together. they need to be demarcated and named.

Given old browsers and old code that cannot participate in harmony, I believe any harmony module format should be designed to be translatable to AMD since AMD works everywhere. At the same time, we should make converters from AMD to harmony. I can appreciate these conversions may not allow 100% conversion, but it should be a very large percentage. I see any harmony proposal as sugar given it will not be ubiquitous. Maybe in 2-5 years after ES.next comes out the story will be different.

Give Me Some Sugar

Tom and Patrick object to having to write too much boilerplate, or ceremony, to code a module, and that is great goal. It is the same reason why some people like CoffeeScript or Stratified JS. But, we need to have a common wrapped format if we want to share code in one file bundles and from CDNs.

The Middle Way

Viewed in this light, I'm hoping we can all code in our preferred sugar, but still be able to share modular code at the base level. Here are some suggestions for us to get to that goal, some for browser library authors, node library authors, JS environment authors, and AMD Toolmakers:

Browser Library Authors

Not all of these may work for you. If you cannot do the first or second one, at least consider the third option.

Consider writing your modules using AMD. The define(function(require){}) wrapper is really light on boilerplate/ceremony, particularly if you are already using a CJS-style of code. There are AMD tools you can use to help with the building of your library, and it allows others with AMD tools to easily use your code in source form.

You get a free "custom builder" capability with this approach. Consumers of your source, particularly if it consists of multiple modules, can just drop your library code in their project, and when they do a build with AMD tools, it will only pull out the modular pieces of your library that they actually use. That is awesome.

You will be using a common idiom that people understand. It will be easier to translate skills.

CJS modules have this quality too, so feel free to use them if you prefer that style. In that case, I hope you will still consider the rest of these suggestions.

Consider using AMD if you need to bundle more than one module together. Almond can be used to provide the AMD API shim. Ace's mini-require might be another option.

If you limit yourself to just define() declarations, no loader plugins or callback-style require, then you can easily make your own wrapper. If you just need a way to concatenate the modules together, but do not need the require calls after a build because you are using globals (I think Tom fits in this group) then you might be able to get by with plain anonymous function wrappings, so this item may not apply to you.

For the top-level, public API for your module, feature detect for define.amd, and register your code as an AMD module. This is particularly helpful when you have dependencies. You can also get away from using globals and allowing better named dependencies.

You can still export a global for now, even in the AMD case, because AMD is still new to some people, and as underscore discovered, it can lead to problems. I expect this type of problem will go away after AMD support has been around a while and the global export can be removed. Since modules are still a new thing to many JS programmers, there will be a transition period while developers get used to not relying completely on globals. This will be true even if/when harmony modules ship.

By calling define() an AMD consumer now has the option now to get direct references for your code under a name of their choosing, and properly state and wait for its dependencies before running with them. The umdjs/umd project has some boilerplate examples to help you get started with this item.

Node Library Authors

If you author user-land node modules for others to use, in other words, do not develop Node core, and you think your library may be usable in the browser, then:

Stick with declarative dependencies. Avoid using calculated values for dependency names:

var a = require(someCondition ? 'a' : 'a1');

Ideally node would support a callback-style require(), as AMD does, to help when you do have a calculated dependency (see JS environment authors section below), but in the meantime, the module will translate very easily to AMD with just declarative dependencies:

var a = require('a'),

    b = require('b');

These type of calculated dependencies will not work in harmony modules either, so it is good to get used to this now.

Avoid __dirname and __filename. Or at least, be aware that in AMD loaders, you have access to module.uri which is like __filename, and a __dirname can be calculated from that value. So this type of detection is a good way to go: var uri = module.uri || __filename.

Use amdefine to create a define() method in your module, and call define() to define your module. This will make it easier for browser authors to directly consume your code, and the RequireJS optimizer removes the if(){} block that uses amdefine, so browser developers will not take the hit of that code or that module.

Consider using AMD as the bundling format, if you make a bundling tool, like browserify. If you need help with this or have questions on how best to do it, feel free to contact me.

The wrapper can be as simple as define('id', function(require, exports, module){});. You may need to provide an option to wrap your output in a function and use an internal define inside that function, but call the global define to expose the public API for the bundled set of modules.

By using the amdefine module as a dependency in your package.json, this will help test a real implementation of define(), and help core Node contributors get a feel for how many modules would like to have define() as an API option directly in Node.

JS Environment Authors

Do you make an environment that can run JavaScript? Like Node, Rhino, Mozilla's Jetpack?

Support AMD APIs for easily ingesting code. You can support the sugared CJS modules as a default, but allow JS library authors to reduce their feature detection boilerplate and allow easier code transfer by allowing define()'d modules.

This does not mean that you must load modules asynchronously. The nice thing about the basic define() API is that it works well in synchronous environments too, particularly with the declarative dependencies.

For Node in particular, I know this has been a contentious thing to consider, and it can be done in phases. Not all the phases need to be done, particularly loader plugins, but each one allows easier code sharing:

• Support module.uri and encourage that instead of __filename. In AMD, module.id is a module ID that is not a full path, so there is explicit module.uri for getting the module's path.
• Support a simple define() call, like the one Isaac had in previously. It can still operate synchronously.
• Support callback-style require([], function(){}) inside define() calls, one that fires the callback on nextTick. This is important for calculated dependencies.
• Consider loader plugin support. It could be limited to loader plugins that can only be run synchronously. This one might also help with the sort of transpiler support you have in Node now.

The previously mentioned amdefine package for Node, which consists of one file, does all of the above, and I am happy to continue work on it, build up unit tests for it, do whatever code changes you think are needed that would allow something like its define() implementation to land in core.

I think Node's synchronous module behavior can be maintained, but still open up easier code sharing pathways with code written by browser-based developers.

I hope my previous posts show why Node's module system as-is is not the right general solution for module-based development in the browser. The ES Harmony module proposal set has similar high level features as AMD, like a module in a block, the equivalent of a callback-style require. The need for those things are not going away.

Plus, the more browser developers (the biggest JS audience there is) can easily transfer their knowledge/idioms/code to run under Node, the more it helps ensure Node's longevity.

Support in other JS environments is also encouraged. It can take a similar path to the one outlined for Node. Mozilla's Jetpack right now only supports a simple define() call, and that is great. It still allows for some basic code sharing. Even getting a little bit of support, doing small steps is fine.

AMD Toolmakers

For those of us that make AMD tools, there are some things we can do that can help people share their code easier. Here is what I am doing:

Create translation tools. I'm working on volo, which can do this, among other things. With volo, you can download your favorite libraries from GitHub and optionally specify AMD wrapping for the library. Right now it is set up to wrap a library that uses globals. I want to add support for converting CJS/Node modules to AMD. CJS conversion is cleaner than the globals wrapping -- the globals wrapping needs more information from the developer to work.

Consider configuration of global import and export of module values. In a runtime loader, allow the developer to configure something like "when underscore loads, grab the global _ and use it as the value for the module". And conversely, "when this AMD module loads, also assign the module export value to a global named 'foo'". This may be better served by the translation tool -- it can be more direct, precise -- but this might help for cases in which the developer does not want a tool to modify the library source. It would only help with libraries that have no dependencies though, so it may not be useful enough on its own.

It is still best for library authors to call define() themselves since they can be more precise on the dependencies and allow getting direct references instead of using globals. However, the AMD toolmakers can help bridge the gap.

Globalize All The Things

All of the above assumes you think getting local handles on other modules/scripts is better than using globals. Not all JS developers hold that position though. In particular, I think Thomas Fuchs, Jeremy Ashkenas and to some extent Scott Gonzalez have all expressed a desire for a system that does not require changing the source files from using globals.

I'm open to hearing how that might work, but I think it is unrealistic as a general approach because:

We already have a problem with globals today, where multiple third party libraries want to use the same global but at different versions. For anything larger than a simple app, it is not scalable to try to do a "noConflict()" approach on nested dependencies.

You could try to get around that by executing module sets in a "sandbox", but the tech to do that with today's browsers, iframes, is riddled with land mines and it has trouble sharing things across frames.

It is bad practice for modules to name themselves. Case in point: Zepto. Zepto is targeted as a jQuery replacement. However, to use it as-is, it means libraries like Backbone have to do manual detection for it: var $ = root.jQuery || root.Zepto. This is not scalable, and it would be to Zepto's advantage to register as an anonymous module, and then allow the end developer, not every library author, to map require('jquery') to a path that resolves to Zepto instead.

It does not map well to other environments like Node and Jetpack, which have gone away from globals.

To be clear here though -- I want a module format that allows for the use of globals, and AMD does allow that. It is just not good to enforce use of globals as the path to code sharing/use. There needs to be a way to get local references via string names that can be mapped to other providers of that functionality.

In the end, I see globals as another kind of sugar. The tricky part is that it is hard to auto-convert given the variability between what a file is called, what global(s) it can export and what dependencies it uses, and mapping those dependencies effectively to files. Systems like AMD/CJS have a much more consistent approach to this, which leads to cleaner code and more effective sharing and tooling.

Summary

Hopefully this gives some concrete ways we can work together even if you prefer some sort of sugar over plain AMD.

Supporting AMD, even just on the output/input edges, is not because you think dynamic loading of scripts is the way to go, or that you have to throw away your favorite sugar. It is about getting a base level of code understanding that allows effective sharing, tooling and reuse across all JS environments.

Reply to Tom on AMD

Tom Dale had a good post about why he dislikes AMD. Please go check it out. We talked about it a bit while he was writing it. It is great he took the time to write it out. We need more of that. We are all trying to find the best path, and it is great we are talking this out.

While I appreciate the critique, what is missing is a solid alternative to AMD. In particular, "use build tools to wrap up CommonJS modules" is not a generic solution. It does not allow for generic dynamic loading of resources, particularly from CDNs. Dynamic loading is needed for big sites that still do builds but want to stage loading of the site as the user uses it.

On his concern about too much ceremony -- this is the minimum amount of ceremony:

define(function(require) {

    //Module code in here

});

By doing that, it is easier for other people to consume your module since module authors do not have to provide "built" versions of their module to use on CDNs. And if they did have to do builds, what format would they publish it in? It would need a function wrapper. Hey, AMD has that!

That simple function wrapper allows moving away from globals, although you still can use globals if you want. It allows for dynamic loading. Recall I'm talking about built layers for staged loading/performance, not "load each module with a separate HTTP request". It allows very simple module sharing now. It allows easier sharing of built files too, since AMD specifies the built file format.

With AMD, you can still use build tools if that is how you like to roll in dev. Go for it. Almond is a great AMD one for that purpose. Here is an example of using the RequireJS optimizer for runtime HTTP-served builds. And since there are multiple AMD implementations, you can choose how big of an AMD lib you want.

I'm sure Tom could adjust his 50 line loader to process the define calls, since his built code probably has function wrappers. He could even adapt his "hydrate from strings" loader to use the AMD syntax.

By using that function wrapper, others could consume his individual modules and use their own choice of AMD loader to deal with their particular loading needs.

On the "hydrate from strings" build approach: I would prefer using appcache and using built layers of AMD code instead. This gives much better overall mobile app performance since more than just JS can be cached. But that is more of a side point. Feel free to use the "hydrate from strings" approach, AMD could be used in that fashion too.


My summary: it sounds like he just wants to avoid a small function wrapping and a level of indent. And to be fair, many simple AMD introductions use the array of dependencies syntax, not the sugared form above, so I can appreciate the confusion.

But trying to avoid that function wrap+indent comes with some code sharing costs, and the alternative he mentions does not handle the breadth of JS module consumption.

Simplicity and JavaScript modules

All of us are looking for simplicity, but there are different levels to simplification. This is a story of what could be considered simple for modules in JavaScript. This post was prompted by the removal the optional AMD define() call in underscore.

For a post on simplicity, it is a bit long, but I'm not a great writer, and I find I normally edit myself so much as to lose interest in posting, so then I end up not communicating. Better to start communicating even if imperfect.

I want to lay out why AMD modules are the simplest overall module solution for JavaScript at the moment, and where other approaches are not as simple as they may appear.

Script Tags

JavaScript does not have syntax for modules, but most programming languages do. "import", "require", "include" seem like popular choices.

JavaScript on the web has meant using script tags and manually ordering those script tags so that dependencies on global objects are worked out correctly. It is also important that those dependencies execute in order.

That sucks for the following reasons:

• You use a separate language, HTML, to specify your JS dependencies and their order.
• A script's dependencies can be unclear.
• If you want to later load some code on demand, it is very difficult to work it out so that the scripts execute in order. This has gotten better over time -- newer browsers that support the async="false" attribute help with this. But older browsers still suck.

So what happens?

• You constrain yourself to only creating simpler applications that can get by with concatenating all the scripts in a certain order, and depend on server tools to help you write out your HTML with the script tags and the rewrite the page after a build to not have those script tags. Think Ruby on Rails or Django.
• Toolkits start building APIs to do this work for you. Dojo and YUI were some of the very first to do so. The Closure library does too, although it was more of a cousin to Dojo.

Option 1 is clearly not appropriate for the full spectrum of web development. There are HTML game engines, webmail/office suites, and then browser-based "no server" development like Phonegap-backed mobile apps.

So it is best to have some API or syntax in JavaScript to get units of code. For it to work well, we should all be using the same API. Otherwise things get messy real fast.

CommonJS

The CommonJS group tried to work out a system for doing this. It is pretty simple too:

• require('some/id') to reference a bit of code. Since a string literal is used, it allows for easy static analysis of dependencies -- no more manually ordering script tags! The ID also has a convention of mapping to a partial path. So you can get by without having to specify full URLs and it opens up for ways to map an ID to a path does not fit the ID-to-path convention. This is really helpful most immediately for mock testing, but it has many other uses.
• Modules do not give themselves a name, they are anonymous.  This is great -- if you load require('jquery') then you do not have to have special knowledge to access some global with different spelling, like jQuery. Same for 'backbone.js' loading an object called Backbone. The end user is in control of the name.
• exports is handy for circular dependencies. Yes you should avoid circular dependencies. If you have one, there is a good chance you are doing it wrong. But there are valid circular dependencies, and a script referencing system needs to support them.
• module is important because modules are not named. Sometimes though you need to know the name they ended up with, and sometimes from what path, because you may have some non-JS assets you need to reference. module.id and module.uri give you that ability.

There were some wrinkles though:

• They did not formalize a way to export a module value that was not just a plain object with properties. It is extremely common and useful to want to export a function as the module value. jQuery is probably the most known example. Constructor functions are another set of useful export values.
• It was not so simple to get it to work in the browser.

AMD

The original participants on the CommonJS list thought it was better to blue-sky the development of a module syntax, not be held back by what might work in the browser, just what might work with existing JS syntax. The group also started off as ServerJS, so they were also in the mindset of what would work best on the server, where file I/O is cheaper/easier.

The hope was that if they worked out a module system that worked well with the existing JS syntax but had problems in the browser, hopefully they could convince browser makers to plug the holes to make it possible. In the meantime, since they were developing servers that could run JS -- they could just bundle up/transform the JS using server tools, or offer a compile step, while browsers caught up.

However, for those of us who came from Dojo, requiring a server tool or compile step to just develop in JS was a complication. I'm going to mangle Alex Russell's quote on this, but "the web already has a compile step. It's called Reload".

Why force the use of a tool to just start developing? It should be simple: just a browser and a text editor.

This was accomplished by taking the CommonJS format and allowing a function wrapper around the code:

define(function (require){

    var dep = require('dep');

    return value;

});

Or, the shorter form::

define(['dep'], function (dep) {

    return value;

});

Since a function wrapper was used, it meant:

• the scripts could load in any order they want, easy to parallelize even on old browsers.
• "return" could be used to return the module value, even functions.
• No special tooling was needed to convert source to an acceptable browser format.
• It worked in browsers, today, and would in the future.

So the story around modules gets very simple, no special tooling to start, no worrying about "I need to run a converter to share this module", no special sets of instructions for your server of choice to do conversions.

Yes, a loader library is needed, but one is needed in any case, there is no native JS syntax. That is the basic ante for any module system that scales up beyond a simple JS concatenation for a Rails application.

For some people, a function wrapper with a level of indent was not seen as simple enough. However, designing a system without it meant a bunch of complexity once you stopped looking at an individual file. A miscalculation of the overall complexity cost.

Loader Plugins

Another simplicity in AMD systems: loader plugins. Loader plugins would not be considered if you had copious, synchronous IO capabilities. However, by fully embracing the network/async loading on the web, you start to see how creating a loader plugin that treats a dependency as a simple string as useful.

Some dependencies are not static scripts, but could have more complex loading (Google Maps code) or simple HTML templates that need to be loaded for the module to be useful.

In AMD, this can look like so:

define(function (require) {

    var maps = require('googleapi!maps?sensor=false'),

        template = require('text!form.html');

    return function (data) {

        //You can  synchronously return a value

        //based on the template.

        return template.replace(...);

    });

});

Compare that with a non-plugin approach. Do you want to load those two resources in parallel? That would be ideal, but then to do that, you probably need something like a promise library to help out. And now you have two problems. I mean that partly in jest -- I use a promise library for some types of code -- but it is definitely a complexity hurdle to jump.

The async networking also means your module is not completely ready until those resources are available, so your public module API now must be a callback API. For "simplicity" assume you just load the dependencies serially, to avoid some of the promise-isms.

define(function (require) {

    var googleapi = require('googleapi'),

        text = require('text');

        googleapi.fetch('maps?sensor=false', function (map) {

            text.fetch('form.html'), function (text) {

                //dependencies are now loaded.

            });

        });

    return function (data, callback) {

        waitForDependenciesToLoad(function (data, callback) {

            callback(template.replace(...));

        });

    }

});

Loader plugins simplify async development, and some of their resources, like the text templates, can be inlined in a build file with other JS modules. Loader plugins give you simplicity and speed.

If you like transpiling other languages into JS, they are great for that purpose too.

ECMAScript

The ECMAScript group wants to get modules in for the harmony effort, the next ECMAScript release. They chose to not go with the CommonJS syntax, but what did they did choose looks fairly similar on the surface. It favors the introduction of new syntax to get some advantages with compile time checks. You can check out the following for more information:

• harmony:modules
• harmony:module_examples
• harmony:module_loaders

Those links have changed over time, so the rest of this feedback may not be valid in the future. As I read it today, I do not believe harmony modules give much benefit over what AMD can do now, but harmony modules do introduce more complexity, and has some unanswered questions:

• It uses module Foo {} syntax for declaring an inline module, where the module ID, Foo, is a JS identifier. However dependencies can string names/URLs. How do you optimize this code for delivery in a web browser? In AMD, string names are used both for references and for the module names. This is better because it allows for loader plugin IDs that can have their output inlined in build output, and it ties module IDs more directly to the string names used in dependency names.
• It does not support a loader plugin API out of the box. You can construct one by using the module_loaders API, but at that point you need to ship a .js file for the "loader", so now the developer has the same complexity cost as AMD today.
• New syntax means the module support cannot be shimmed in older browser via a runtime library. It requires a compile step/server transform to work in older browsers. This is one the same problems the CommonJS format had.
• It is unclear how a JS library that works in browsers without ES module support "opts in" to register as an an ES module since ES modules use new syntax. Maybe the suggestion is to use the module_loader's loader.createModule syntax? I cannot see how that fits with the compile-time export checking though. If a runtime capability check cannot be used to opt in to ES module registration, it makes it very hard to upgrade web libraries to ES module syntax. We're back to the problems that made it hard to use CommonJS syntax on the web.

The new syntax in harmony modules is used to enable some compile time checking of export names and if they are referenced correctly.

However, compile time checking to see if an export name is use correctly is a very small benefit for the end developer given the other costs above. Furthermore, I want more than just an export name/type check. I want intellisense on the arguments that can be passed to functions, general data type information and comments on usage.

Working out a comment-based system that can reflect this info into text editors will provide much more value. Since it is comment based it fits in with old browsers. I know it is harder to agree on that comment syntax (mostly because it is easy to bikeshed), but it will simplify developers' lives more, and lead to faster turn-around time on development.

If something like loader plugins are not natively supported, and the optimization story is not sorted out, it does not have an advantage for AMD today, and AMD is simpler.

However, the harmony module_loaders API is really useful. I'm not sure it needs to be as big as it currently is. I would be fine with something like Node's vm module API as a start. Basically, some container or vat I can load scripts into without interfering with other scripts. This is one area that is very hard to do on the web today.

So for me, the "simple" solution for any ES6 module-related work:

• Make it a module API, not new syntax. Then we can shim it easier for older browsers, and existing libraries and capability check it and opt in. If you need a suggested API, I hear AMD is quite nice. It has a few implementations and has been used in the real world.
• Support loader plugins natively. It really helps with async programming, and optimizations. If I need to ship a library to deliver the benefits that loader plugins give for async programming, then there is very little motivation for developers to switch away from AMD.
• Provide something like Node's VM API, maybe with a little bit of the intrinsics stuff in the current module_loaders API.
• Do not bother with compile time checking of export names. Instead put effort into a comment based system that can reflect more information for use in editors. That will save developer more time. Since it is comment based, it will work in older browsers and optimizes out cleanly.

I will post this feedback to the es-discuss group. I wanted to hone my feedback before giving it to the es-discuss group, but this will have to do, otherwise I may never give it.

Underscore and AMD

This brings us to the recent code change to remove the AMD block from underscore. The arguments for removing it are listed here, but I think the main argument is really about what is simple for Jeremy: He does not need it for the kinds of sites he builds, and by adding it, he has gotten reports of problems.

Those problem reports go away if he also exports underscore as a global, but he does not think he should have to do that if AMD is available. I do not think that is a fair bar to hold for AMD, since he would have to do the same for a harmony syntax, so his lib could be used in cases where ES loading and old style global loading is still in play.

Some feedback on his specific reasons:

Folks requesting other module formats for other loaders.

No other module format comes close to the level of support AMD has: AMD has multiple implementations, better support in other libraries (Dojo, jQuery, libraries friendly to Ender-bundling, MooTools), it is used in real sites, and has a thriving amd-implement list and thriving implementations. What else can claim all of those things? What else is being asked for inclusion?

I can appreciate it is easy for someone to come up with a loader syntax and want people to use it. I think AMD has gone the extra steps though to be considered more of a standard. But it depends on what you want to use as for standards of legitimacy.

If any library that depends on Underscore (and there are many of them: http://search.npmjs.org/) does not yet 'support' AMD, but Underscore does, things get royally screwed up.

npm is for node modules, not browser modules, so they would not have the error that was being reported for Underscore.

That said, I do believe Underscore is a common dependency for browser-based code. But for the browser, as mentioned, registering a global is perfectly acceptable for this transition period, something I believe will need to happen anyway no matter what modular format is chosen. There will always be a transition period.

In an ideal situation, libraries do not have to be modified to support a particular script loader (or group of loaders).

The point is to make developer's lives simpler. A script loader like that does nothing to help order the dependency tree correctly, or use a naming convention that can be parsed easily by tools like optimizers.

The browser globals with implicit dependencies just do not scale well past maybe 15 dependencies? Not everything fits as "lets concat all the scripts" Rails app. But I'm open to seeing a design that might scale up.

JavaScript's upcoming native module support is entirely incompatible with AMD.

They are very compatible as far as base semantics. As mentioned above, ES harmony modules are still baking, not ready for prime time. But even with that, it would be very easy to convert AMD code to harmony module code -- since the dependencies are all string names it fits in. Loader plugins would be a problem for sure, but basic modules are easy.

It is much more compatible than the current "browser globals and implicit dependencies" approach.

Loading individual modules piecemeal is a terrifically inefficient way to built a website. Because of this, there's the great RequireJS optimizer, which will turn your modules into ordinary packages.

The browser globals approach already depends on build tools, so I'm not sure why this is a knock against using AMD. A bunch of manually typed HTML script tags perform just as poorly.

Fortunately, since dependencies can be easily statically determined with AMD calls, the developer no longer has to worry about manually figuring out the load order, and there can be (and are!) many different build tools built on top of the standardized AMD API.

Summary

In the end, though I just think it boils down to Jeremy not needing this personally based on the scope of his work, and he has other things he would rather work on. It is hard to get a standard of legitimacy in this area, so it is easier to just wait it out. For the particular issue in Underscore, the simple export of a global even in the AMD case would have solved the issue, but such is life.

I'm a bit sad because Backbone and RequireJS have been a very popular combination. They fit very well together. The thought of maintaining a fork/branch is distasteful, particularly since the AMD patch for Backbone was smaller than the code it replaced.

Auto-wrapping tools are difficult to do generically given how scripts want to grab globals. The dependency name can have weirder names that do not match the file name, so it loses the nice, simple dependency parsing of AMD module IDs. It means creating a centralized list of global names that map to IDs/paths. Not very webby/distributed. Not very simple.

Oh well. It probably means AMD just needs a bit more time out in the wild, even more adoption, and we'll see how people feel in 6 months.

Another Approach?

I have heard complaints from folks on the internet about AMD from time to time, but they have not offered anything better, particularly given the simplicity tradeoffs mentioned above. I think it is just an NIH thing most of the time, or getting it mixed up with generic browser script loader.

Here is a survey of things I know about:

Ender is fine if you want to just build a file that you will not change often and use it in place of jQuery, but it really does not help with the larger site structure and loading issues, being able to dynamically load. It is not a general module system for the web. It basically is just like the CommonJS "do a build before starting development" approach. Same as SproutCore/Ember. Same complexity problems as mentioned above, and complicates individual module debugging.

Dan Webb started something with loadrunner. It supports part of AMD, and the "native format" it supported do not seem better than AMD, maybe just another function nesting and different API names.

Ext has something similar to AMD, but mixes in a particular class declaration syntax into the format, so that will not fly as a general solution.

YUI is close, but obscuring what a dependency name loads on a Y instance makes it hard to associate what came from what module, and the API to add a module relies on naming the module and putting named fields on the Y instance. This will lead to name collisions.

What else am I missing?

One thing these approaches all have in common: they get away from the browser globals and implicit dependency approach used by traditional browser scripts. If you think something using that traditional pattern is the future, please describe how it might work. Remember, requiring build step to just start developing is a complication. That does not scale well across all the types of JS development mentioned above.

While Jeremy and I were talking in the documentcloud room, he mentioned that Ryan Dahl, if he could do it over again, would prefer not to use CommonJS modules system for Node, but do something closer to how browsers load scripts in script tags.

I think I have heard that comment too, but in the context I heard it, it seemed like an off-hand comment. I'm not sure how much that was just about having to wade through CommonJS discussions or actual problems with the module API. I would love to hear more about it though. Ryan or someone how has more info on this, if you happen to see this post, please clue me in. I'm on github, the amd-implement list, or on gmail as jrburke.

I want to get to a workable, simple solution that works well in web browser too. I only do AMD because I need it and I think it is the simplest overall path for end developers. But I want to solve other higher level problems. I want something to good to win. It does not have to be AMD, but I do think it hits the right simplicity goals particularly given browser use. It will not be brain-dead simple because upgrading the web is not simple. But it does a pretty good job considering.

I feel like the folks doing AMD have done their due diligence on the matter. ES harmony may be able to do something a bit different, but the basic mechanisms will be the same: get a handle on a piece of code via a string ID and export a value. The rest starts to look like arguing paint colors.

Anyway, enough of that. Back to making things to make things better.

RequireJS 1.0.4 released

RequireJS 1.0.4 is available.

The big fix for this release is making sure the config options are passed correctly to uglifyjs. There was a problem with 1.0.3 that would result in some files not being parsed correctly.

The small set of bug fixes for:

• require.js
• r.js optimizer

volo: A JS tool for JS projects

Over the last month I have been playing with a tool to help me make JS-based projects. It is still very early in its development, but it can do a trick or two now, so it might be fun for you to try out too.

volo is a command line tool. It is written as a set of AMD modules that are built into one JS file that runs in node.

Check out the README for more info, but what I like about this project:

• Just one JS file to install.
• It uses GitHub as the code registry.
• You can create new commands for it (using volo) and if you put your commands on GitHub it is easy for others install them.

Still lots to do and there are lots of sharp edges to it, but it already feels pretty sweet. If you would like to help out, check out the design goals and working the code. Also feel free to give some feedback on some of the more important issues I would like to sort out:

• Windows support
• "Short name" dependency lookups, possibly using some sort of GitHub search API.

On a side note: I have been on vacation and not checking email. If you are waiting on a reply from me, I should get to it this week.

RequireJS 1.0.3 released

RequireJS 1.0.3 is available for download.

Small fixes, mostly for the optimizer. See the 1.0.3 fix list for:

• require.js
• optimizer

almond 0.0.3 released

almond is an AMD API implementation that does not do any dynamic loading -- it is best used as part of a deployment wrapper for AMD modules that are combined into one file. Version 0.0.3 is just 857 bytes when minified with Closure Compiler and gzipped.

Changes in 0.0.3:

• Patrick Mueller had some suggestions on how to make it more friendly to CommonJS modules that are simply wrapped with define(function (require, exports, module){}) calls, and to allow out of order module listings in the built file.
• Works properly with jQuery 1.7+ versions that optionally call define().

RequireJS 1.0.2 released

RequireJS 1.0.2 is available for download.

Small fixes, see the 1.0.2 fix list for:

• require.js
• optimizer

Why not AMD?

This started as a private response to some tweets by Jeremy Ashkenas that indicated some concern around optionally registering code as an AMD JavaScript module, but since his comments were in a public space, it makes sense to post my response in a public space.

His concerns as mentioned in the tweets:

• Isn't canonical in any JS runtime
• Inefficient by default
• Incompatible with JS.next
• Known temporary solution

Twitter is bad for this kind of conversation, and Jeremy is busy with work, but hopefully he can expand on his concerns at some point if my responses below do not address them.

Isn't canonical in any JS runtime

The goal is to create a grassroots effort based on real, multiple implementations that makes it canonical. I believe it is following the way to get something standardized. There are multiple implementations (requirejs, curl, dojo and mootools), and there are multiple higher level toolkits that register with it: jQuery, Dojo and MooTools.

It is also based on real world experience supporting modular code in the browser. It is informed by Dojo's previous modular experience, and jQuery's needs for modular loading. I believe this places it in a better position than how modules in ES harmony are being designed, particularly since it can be opt-in that works with ES3 grammar.

Node adopted a variation of CommonJS modules, but Node's implementers have been very explicit about not picking up anything else from CommonJS, and they support their own non-conformant extensions, like module.exports. Node is a monoculture and very inward focused, and since it has sync IO, it is not concerned with browser needs.

Even if CommonJS were supported well in the browser today, there still needs to be a way to embed multiple CommonJS modules in a file. AMD is a great candidate for that format -- CommonJS never standardized on a "transport format" for this need.

I expand on this a bit more in the Why AMD document.

Inefficient by default

I assume this means that it allows separate modules to be loaded async. FWIW, the current state of ES harmony modules will allow separate file loading for each module. This works better for debugging.

For AMD, you can get the "one script file at the bottom of the page" loading with the 750 byte almond AMD shim and runtime http loading.

I cannot see a better solution to this issue. If you have ideas I would like to hear them.

Incompatible with JS.next

Since JS.next/ES harmony modules do not have an implementation yet, I am not sure what this means. It still needs at least one good implementation to prove out the API. I think AMD can better inform the ES harmony effort since it has real deployment by web-based JS users, the harder environment to get right.

Also, given that harmony modules uses new syntax and by default there is no global access, I think they are making it very hard to allow existing code to opt in to being a harmony module. I have given David Herman feedback to this effect.

I want to help make any harmony module implementation better based on AMD's real deployment. I go into it more in the aforementioned Why AMD page, but being able to set the exported value to a function, and the use of loader plugins to avoid nested, async callbacks for simple resources are of great value in any module system.

Known temporary solution

As you might gather from the above, I do not see it as a temporary solution. It has more legs than CommonJS modules because it works well in browsers and it can be used as the "transport format" for CommonJS modules.

In addition, unless someone can get Microsoft to adopt rapid IE updates, even for old IEs, AMD will be around for a long time. It is a great transpile target for ES harmony module code that wants to run in older browsers. I am prototyping that effort, although I need to update to a real parser instead of regexps. I have a branch of narcissus that has the core parser/lexer/definitions that will run in ES3 browsers as part of that work.

All things considered, I would rather not work on JS module formats. I want to get on to building useful things for end users. But the CommonJS effort was deficient (but a great first effort given their design goals -- AMD uses a bunch of their work), and I do not think ES harmony modules are there yet. Instead of just complaining, I worked with others to work out something better and got implementations and actual, real adoption.

I'm open to something else that has done the above, but I have not seen it yet. Feel free to offer alternatives and feedback though. I just want to get to a good solution, but I'm also tired of waiting for something magical to fix the problem.

RequireJS 1.0.1 released

RequireJS 1.0.1 is available for download.

This just has three small bug fixes in it (one in require.js, two in r.js). It was prompted by the release of jQuery 1.7 and wanting to make sure people could use the "namespace" optimizer option with it, now that jQuery 1.7 registers as an AMD module.

The bug fixes were related to:

• allowing full URLs for simplified CommonJS wrapped modules
• AST parsing of dependencies for modules that use a variable for the factory function
• catching more cases that should have the "namespace" optimizer option applied

Detailed list of changes for require.js and the r.js optimizer:

• require.js 1.0.1 changes
• r.js 1.0.1 changes

RequireJS 1.0 released

Hell yeah. Get it now.

This release is basically the same as the 0.27.1 release. There was just one change, to a regexp used by the optimizer when it converts CommonJS modules to AMD modules via its -convert flag.

This release has been two years in the making. The code and APIs have changed over time based on feedback from the community, and now there are some very strong AMD loaders besides RequireJS. The AMD API and RequireJS have been proven useful and used in real projects.

I created a new Why AMD document if you want to learn more about why the AMD API exists and how you can use it.

I still plan on doing RequireJS releases and pushing modular JavaScript forward. This is not the end, but it is an important milestone. Thank you to the RequireJS and AMD communities for making this release.

Go use it already!

• Use it to build apps!
• Upgrade your library code to optionally call define()!
• Implement AMD in your favorite JS environment!

Let's make modular JS a reality on the web today.

RequireJS 0.27.1 released

RequireJS 0.27.1 is available for download.

Just a small update with some bug fixes for 0.27.0. See the 0.27.0 release for the major functionality changes over previous releases.

I will wait about a week to see if this release seems stable for a 1.0 release.

Fixes:

• define(id, function () {}) where the function has require('') dependencies will now be scanned for dependencies. Allows for smaller universal module adapters.
• Loader plugin that depends on a different plugin's loaded resource works as it did in 0.26.0.
• Optimizer: update UglifyJS to 1.1.
• Optimizer: semicolons are inserted between files if concatenating would cause errors.
• Optimizer: always strip BOM files on all platforms when file transforms/concatenations are done.
• Optimizer: allow override of modules used in optimizer. Example.
• Optimizer: allow copying of .directories via build config option.
• Optimizer: Resolving paths for .js dependencies might fail if an appDir was not part of the config.

RequireJS 0.27.0 released, 1.0 release candidate

RequireJS 0.27.0 is available for download.

This is the 1.0 release candidate. Unless things go horribly awry, I expect to do the 1.0 release in about a week. So please be sure to try it out soon if you have concerns. If you need more time to evaluate it, let me know.

From the release notes:

• require.ready() has been removed. In its place, use the domReady plugin. This allows better interoperability with other AMD loaders and better separation of concerns.
• A new wrap config option for the optimizer is available, for wrapping built code in a function. Allows for better API hiding and tiny builds with the almond API shim.
• The order plugin is improved for IE.
• Loader plugins can now have dependencies and they will work in the optimizer, as long as the dependencies work in the optimizer environment (Node, Rhino).
• The namespace config option for the optimizer is more robust.
• Removed require.def(), use define() instead.
• Removed module.setExports, use module.exports instead.

almond: a small AMD shim loader

I just put up release 0.0.1 of almond, a tiny shim loader for the AMD JavaScript module API. It can be used as a replacement for RequireJS after an optimized build. It is great for standalone libs or tiny apps that want a very minimal footprint with all the benefits of AMD.

From the README:

Some developers like to use the AMD API to code modular JavaScript, but after doing an optimized build, they do not want to include a full AMD loader like RequireJS, since they do not need all that functionality. Some use cases, like mobile, are very sensitive to file sizes.

By including almond in the built file, there is no need for RequireJS. almond is 948 bytes when minified with Closure Compiler and gzipped.

Since it can support certain types of loader plugin-optimized resources, it is a great fit for a library that wants to use text templates or CoffeeScript as part of their project, but get a tiny download in one file after using the RequireJS Optimizer.

If you are building a library, the wrap=true support in the RequireJS optimizer will wrap the optimized file in a closure, so the define/require AMD API does not escape the file. Users of your optimized file will only see the global API you decide to export, not the AMD API. See the usage section below for more details.

So, you get great code cleanliness with AMD and the use of powerful loader plugins in a tiny wrapper that makes it easy for others to use your code even if they do not use AMD.

RequireJS 0.26.0 released, npm install requirejs

RequireJS 0.26.0 is available for download.

The big feature is being able to npm install requirejs to allow require("requirejs"). This allows you to:

1) Load AMD modules inside node without running a bootstrap script. It also fixes some path issues using traditional Node modules alongside AMD modules. So, now you can use requirejs inside Node to load AMD modules and use loader plugins:

var requirejs = require("requirejs");



requirejs.config({

    nodeRequire: require

});



requirejs(["module/a", "module/b", "text!templates/one.html"],

 function (a,          b,          template) {

     //use a and b with the text template

 });

2) Exposes the optimizer as require("requirejs").optimize() to allow dynamic server builds for people who like to do "only one script tag before end of body tag" development. With the "excludeShallow" optimizer config, you can still debug a single module/script while having the rest combined into one script.

Pretty sweet -- using JavaScript to run a server to optimize JavaScript on the fly but still get fine-grained JavaScript debugging. JavaScript turtles all the way down (until you hit the C turtle).

You can still use the r.js script to do command-line build optimizations. If you npm install -g requirejs, then you can use r.js as an executable (the requirejs package replaces the old requirejs-r package).

More information on the Use with Node page.

The requirejs npm package includes require.js for the browser. So, for web pages, load node_modules/requirejs/require.js to use RequireJS in the browser. As always, it is easy to get require.js for the browser without using npm.

Also notable in the release: UglifyJS in the minifier is updated to 1.0.6. The upside: has() branch trimming now works with the default minifier.

Other items from the release notes:

• Fixes for running under Node on Windows using the native node.exe builds that are available in the Node 0.5.x series. Now there is less of a need to use Java to drive the RequireJS Optimizer!
• Configuration is now done via a require.config({}) call, to get in line with the amdjs require API. The old require({}) method works on the global require() for backwards compatibility, but the suggested API going forward is require.config({}). The API doc has been updated to show proper usage.
• There is a namespace option now for builds, to allow moving require() and define() calls under a different namespace. This allows you to build an optimized file that uses RequireJS but does not interfere with any other AMD loader on the page, and you can make sure only your modules are loaded in that namespaced object.
• The default error behavior when a define() factory function throws an error is to not catch it. The catching done in 0.25.0 made it more difficult to debug. However, there are some situations where catching the errors is preferred. Setting the config value catchError.define = true will switch to catching the errors and allow processing via require.onError()
• Closure Compiler in the optimizer was updated. As a result, the code to invoke Closure Compiler changed, and will likely only work with the latest Closure Compiler release. You can grab a version known to work with the optimizer in the optimizer's lib/closure directory.
• There is now a pragmasOnSave build option, which is used in the require-cs CoffeeScript loader plugin build profile to strip out the CoffeeScript compiler after a build. The end result: tiny build layers of converted CoffeeScript code.

RequireJS node adapter now in npm

[Update August 17, 2011: This post is out of date, see the 0.26.0 release for a better way to use RequireJS with npm.]

[Update July 21, 2011: GitHub users arlolra and JasonGiedymin clued me in to setting up r.js as a bin file that can be installed globally, making it much easier to use. The instructions below have been updated to reflect the changes.]

As asked for on the requirejs list, the RequireJS Node adapter + optimizer is available via npm now:

npm install -g requirejs-r

This will install the r.js file as a global bin/executable that is available for any of your Node projects.

To run main.js via RequireJS in Node:

r.js main.js

To optimize a web project using RequireJS, assuming app.build.js contains your optimization profile:

r.js -o app.build.js

"requirejs-r" is a bit of a funny package name, but it comes out of the requirements to use a directory for publishing to NPM, and the r.js name. If it seems useful, I may publish other RequireJS-friendly loader plugins under the "requirejs-" prefix. I may even push require.js to npm, but I am not sure it makes sense yet. If you think it would be useful to you, feel free to leave a comment on this issue.

I'll update the requirejs.org docs if this installation option seems to work out for people.

RequireJS 0.25.0 released, AMD advancing

RequireJS 0.25.0 is available for download.

It has been a few months since the last release, longer than normal. I have had some life changes (for good reasons) and I have busy at the day job. The interest in AMD loaders has picked up, and I have done some groundwork related to that interest. More information later in this post.

I wanted RequireJS 1.0 to be this release, but decided to name it 0.25.0 since I did some changes in the interest of better compatibility with other AMD loaders. Most applications should not notice the changes, but I want to have a release to shake out any problems on the edges.

Release Highlights

The awesome part: the optimizer is now just one JS file, r.js! It also doubles as a bootstrap script that supports the full capability of AMD modules and loader plugins in Node and in Rhino.

To use the optimizer, pass the "-o" option to r.js:

    node r.js -o app.build.js

To run your AMD-based project in Node (assuming server.js is your top-level AMD module):

    node r.js server.js

Running AMD modules in Node has more information. The optimizer docs have been updated to the new optimizer syntax, and the r.js script has its own project now, to allow releases that are decoupled from require.js.

Running r.js under Rhino is still supported, but you need to fetch a couple of JAR files first.

Other highlights:

• The loader plugin API changed to allow plugins to create cross-domain-accessible resources. The main use case: you use the text plugin to dynamically load text resources, but you want to deploy your scripts and text resources to a CDN. See the text plugin's implementation of writeFile() as an example.
  
• There is a global requirejs() function object that is the same as the global require() function object. This should allow RequireJS to work better in environments like Mozilla Chromeless, which already have a built-in require() function that does not have full AMD/loader plugin capabilities.
  
• It is now possible to specify the precise version of jQuery to allow in a RequireJS context. This is useful if you know of other scripts that load different versions of jQuery on a page.

Some changes in the name of compatibility with other AMD module loaders and Node:

• The "lib" directory configuration in package support was removed. It was always very awkward to support. Node no longer supports it, and that was the extra justification I needed to remove it.
• Relative module IDs are now relative to the related module ID, not the related module's resolved path.
• includeRequire in the optimizer config was removed, Use a paths config to include require.js instead. See the optimization docs for more details.

A small change to the context-specific require() passed to a loader plugin's load() call: require.isDefined() is now require.defined() and there is require. specified().

A New Home for AMD

Part of the RequireJS release delay was because the AMD API was moved off the CommonJS wiki and a set of compatibility tests were created.

While it was useful to hash out some of the basic goals and API on the CommonJS list, support for AMD on that list did not reach consensus. However, there has been continued and increasing interest in others making AMD-compatible loaders and tools. To avoid bothering people that would rather talk about other non-module things on the CommonJS list , a new AMD-focused discussion list and wiki has been set up and there is a set of compatibility tests.

AMD is advancing

More evidence that AMD is advancing:

• Alternate AMD loaders are available. Curl continues to advance, as well as the Dojo's 1.7 loader. Loadrunner has grown some basic support.
• There are at least two "AMD lite" projects, one in Ace, and one called DefineJS, that focus on basic AMD API support for when all the modules have been optimized into one file. The goal is to use a smaller bootstrap in optimized applications instead of a full standalone loader, like RequireJS.
• Node 0.5.0 supports the simplified CommonJS wrapper version of define(). While this support is very limited (no dependency array or loader plugins), it is a start to allowing some basic low-level module sharing between Node and browser apps better. See Kris Kowal's excellent Q library as an example of a module that works in both Node and in full AMD loaders.
• Look for better opt-in support for AMD in a couple of browser-focused libraries and toolkits later this year.
  

Path to RequireJS 1.0

For RequireJS, I want to get to a 1.0 when:

• 0.25.0 has been shown to work well in existing projects.
• Do some work around require.ready: possibly using a domReady plugin/module with optional loader hooks, to help improve cross-loader compatibility.
• Try to finalize more of the loader plugin API with other AMD implementers, to make sure more code can be shared across loaders. The loader plugin API is a separate API from basic AMD support, so I may push out a 1.0 before that API is completely final, and go to a 2.0 branch for any backwards incompatible changes to the loader API.

On inventing JS module formats and script loaders

There was some recent twitter tweets over the last couple months that indicated some folks want to experiment with some JavaScript module formats that work in the browser. In addition, making a script loader is almost as popular as making your own template engine. This is my response to both of those endeavors.

AMD is Winning

If you are one of those people that think they want to invent a module format or a script loader that works well in today's browsers, let me save you some trouble: a module format has already been worked out. Check out the Asynchronous Module Definition (AMD). There is still some room for innovation though, see near the end of this post.

If your script loader does not support AMD, it will be of very limited usefulness and in a sea of competitors that will make it hard for you to gain adoption.

So, for the module syntax, that problem is solved, and the solution is AMD.

A Solved Problem

Why is it solved?

• It has been discussed for more than a year and a half by multiple people, in the Dojo world, and on the CommonJS list.
• It has been implemented more than 5 times by different people.
• It has been deployed in real projects, the BBC iPlayer being a very visible one.
• It works in the browser, web workers, Node, Rhino, Mozilla Add-On SDK-based add-ons, and can be used in traditional Mozilla add-ons (Firebug will be using it).
  
• Many AMD implementations support loader plugins. You can add modules that know how to load other things as dependencies. While not strictly part of AMD, plugin systems greatly enhance a base technology.
  

You will be facing an uphill battle trying to get your format accepted. AMD has at least a half-year head start and has been seriously battle-tested. The ECMAScript folks are considering separate, different language changes to support modules, so you have that on the event horizon. Time is really running out pushing for something new.

I do not want to harsh your mellow, and I definitely want to follow your bliss, but just know the actual cost involved, and do your research first. Make sure you understand why AMD is constructed how it is, and what it really is.

It is not the traditional CommonJS Module format, but it can support similar concepts. However, it is different, take the time to figure out why.

Here is a handy gist with a summary of the AMD API, loader plugins, and some thoughts as to how it relates to modules implemented in Node, which are more like traditional CommonJS modules. In addition, you can read the RequireJS API docs, and the AMD wrapper format for traditional CommonJS modules.

If you think something was missed, feel free to ask on the RequireJS list, or on the CommonJS list, and enlighten us all. Or just send me an email or GitHub message if you want to talk privately. However, I believe most questions will be in the minor bikeshed/personal preference arena that do not ultimately matter for adoption. In particular, do not propose anything that requires the developer to do more typing that what is in AMD already. The format needs to be usable and easy to type. Discoverability is not a goal, daily usability is, and the AMD format is easy to explain.

Requirements

This leads to a discussion of other requirements. Here is what I believe are the minimum requirements for a module format in the browser, and they well-met by AMD:

• No globals: The module format needs a way for the developer to avoid creating global variables for each module they create. The format needs to be web-scalable, which means that some sites need the ability to embed two different versions of a module on a page. This normally means embedding in different "contexts" (the different versions do not need to talk to each other, just co-exist on the page), but it is a web-scale need. Seriously. We have seen it in Dojo. jQuery has seen it, it is the reason they have noConflict(). While noConflict() is nice, there are edge cases where it falls down. Know why that happens (in particular, dynamic script loading in IE: more than one script can execute before the first script's onload event fires).
  
• Allow for globals: at the same time, you want developers to be able to do want they want, in particular, some like to modify global object prototypes. Prototype and MooTools are valid ways to build web sites. So allow for it. This is one of my concerns with the ECMAScript harmony modules experimentation, but I think they know they need to allow for this, even if within a module loader instance.
• Do not require server transforms or think that using XHR+text transform+eval() is usable across all of web development. It is not. Here are some reasons why. They are fine for your boutique rails projects, but do not assume that translates well to the web at large, for instance, to serving JS files from disk for mobile widgets.
• You will need to use a function wrapper and a way to specify dependencies before that function wrapper is executed. You will be using script elements to load content (seriously, don't use eval, that is an evolutionary dead path), and dynamic script elements will load scripts async and out of order. A function wrapper will need to be part of the format.
  
• You need a way for modules to have a name. Otherwise you cannot combine them all together for optimization purposes.
• At the same time, it is best to allow source modules (the non-optimized ones) to not be named. This makes it much easier to move code around. This is not a hard requirement though, and I think one of the easiest to try to not support, see next section.

Places for Innovation

So you are not going to come up with a better module format. I know that sounds harsh, and for some of you that will be just the thing you need to hear to try something different. But I really am trying to save you some work. Again though, go for it if it really is your bliss, just know the amount of work you are taking on.

A better use of your time might be doing the best AMD implementation/script loader. In particular, you can choose some limiting design choices to help make it easier to meet/more compact:

• Do not support the simplified wrapper for traditional CommonJS Modules.
• Only allow named modules.
• Do not support loader plugins.
• Do not worry about supporting more than one version of a module in a page. The module format allows for it, which is the most important aspect, so module authors can code for it if they wish. However, most sites can get away without needing a loader that can actually support loading multiple versions.
  

If you do those things, you still can consume the built/optimized AMD modules that someone made using another implementation, and any modules created while working with your implementation will work with other AMD implementations. And with that, you are part of a larger ecosystem.

If you make a script loader that does not understand AMD, it will be a boutique loader, and not something that fits all of front-end development, and limits the ability to share code with other JS environments. Which is fine, boutique businesses can be really satisfying, just realize you are in a boutique, and do not over-sell the loader.

FAQ

These are odds and ends that do not fit above.

Why use modules or a script loader at all? Just server-side concat all your scripts!

The goal is to establish a larger ecosystem of shared code that works without puking globals all over the place. JavaScript is also not just jQuery (although I greatly appreciate the project), so it cannot rely on a jQuery implementation.

Not all environments can use server-side script concatenation (file-served mobile widgets), and it does not allow for more nuanced optimization like loading built layers on demand after initial page load. On-demand loading is really needed for large web apps like a webmail UI.

With tools like the RequireJS optimizer, you can still get the same effect of "concat all files". If you do not want to take the hit of downloading a script loader in that scenario, there is work underway to allow a simplified stub that removes the need for a full script loader. The Ace project uses a "mini-require" after they build the code to avoid loading the RequireJS script loader.

What about YUI 3 modules?

YUI 3 got a lot of things right with their modules, in particular designing for async up front. However, I believe the way they implemented the module name-to-JS-variable-name mapping is not right. It requires knowledge of two names that allows for easy typos and creates the opportunity for a naming conflict on the Y object. For instance:

YUI().use("io-base", "my-module", function(Y) {

//Where does .on() come from? Did 'my-module' add it?
//Did 'io-base'? What if both tried to add it?
Y.on(...);

//'io-base' creates the 'io' property on Y.
//How is that known? Should it be 'ioBase'?
//What else does 'io-base' add to Y?
var request = Y.io(...);
});

In short, too much magic that requires extra documentation. YUI 3 has very nice documentation so that helps, but I do not believe that approach scales as well as AMD.

Is AMD a CommonJS specification?

No, it is a proposal, but with lots of implementations and real use. Not all participants on the CommonJS list believe it should be a elevated to "endorsed spec" status. Those participants wish browsers would grow better base technology and/or they want to maintain stricter compatibility with traditional CommonJS modules, which were not designed primarily for async loading environments like the browser.

Any browser technology change will likely be for harmony module support which operate differently than traditional CommonJS modules, and AMD works well today in all JS environments. AMD is proven, and it can work on the server side. The RequireJS Node adapter even allows using NPM-installed modules in Node.

How does AMD relate to ECMAScript harmony modules?

They are unrelated, although hopefully the AMD work and the traditional CommonJS module work will help inform the harmony effort. I believe AMD modules will be easy to convert to harmony modules, but as the harmony work is still in progress it is hard to know for sure. The return value options in AMD may be more flexible than harmony modules, and it is unclear if loader plugins will work in a harmony module world.

AMD has the benefit of working in all the JS environments now, and it will work in the future, and I do not believe the syntax gains in harmony modules are a vast improvement over the typing cost in AMD, particularly if sharp functions are supported. That said, I hope to inform the harmony effort where I can to help make it the best it can be.

Who are you?

I'm James Burke (GitHub, Twitter). I maintained the original Dojo XHR-based script loader, maintained Dojo Core, created the Dojo xdomain script element-based loader, created RequireJS (formerly RunJS), and rewrote RequireJS about three times. I have strongly advocated for browser-friendly modules on the Dojo and CommonJS lists. I like Pina Coladas and getting caught in the rain.

RequireJS+jQuery project moved, updated to jQuery 1.5.2

The sample project that demonstrates how jQuery and RequireJS can be used together has been moved to its own repository. This should make it easier to update. As proof, it now has jQuery 1.5.2, and can be downloaded here.

Also, the sample project has gone back to using an integrated RequireJS+jQuery file as the basis for the example. Keeping RequireJS and jQuery separate and using the "priority" configuration option in jQuery has some edge case considerations that required more explanation than what most people need. However, the README in the sample project repository explains how to use the priority configuration if you prefer to use that option.