tag:blogger.com,1999:blog-190027232024-03-07T19:18:10.805-08:00TagnetoNotes on JavaScript, <a href="http://requirejs.org/">RequireJS</a>, and <a href="https://mozillalabs.com/">Mozilla Labs</a>.Jameshttp://www.blogger.com/profile/12067100302830600925noreply@blogger.comBlogger13125tag:blogger.com,1999:blog-19002723.post-51391145862907165582012-09-05T16:23:00.000-07:002012-09-05T16:23:12.795-07:00Moving to a new blogAny new posts I do will be at <a href="http://jrburke.com/">jrburke.com</a>. There is a post over there <a href="http://jrburke.com/2012/09/05/new-blog/">about the new blog</a>.<br />
<br />
I will keep this blog for historical purposes, but any new posts will be at the new location.<br />
<br />James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com2tag:blogger.com,1999:blog-19002723.post-60116118766487212582012-08-29T20:15:00.000-07:002012-08-29T20:15:05.297-07:00volo 0.2.3: semver and a web site<a href="http://volojs.org/">volo</a>, a command line tool to create web projects, add front end dependencies and automate tasks, is now at 0.2.3. Get it via npm:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> npm install -g volo</span><br />
<br />
The <a href="https://github.com/volojs/volo/issues?milestone=6&state=closed">complete set of changes are here</a>. The notable ones:<br />
<br />
1. <a href="https://github.com/isaacs/node-semver">semver</a> ranges can be used with dependencies now. For example:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> volo add requirejs/~2 </span><br />
<br />
will download the latest 2.x version of jrburke/requirejs from GitHub.<br />
<br />
2. <a href="http://volojs.org/#add">volo add</a> mentions possible incompatibilities after doing a nested dependency install, and gives commands for manually choosing one of the other versions.<br />
<br />
3. volo now has <a href="http://volojs.org/">a simple web site</a> that hopefully gives a better idea intro to volo. Many thanks go to <a href="http://jlongster.com/">James Long</a> for starting the structure and helping to refine its message. There is more to do for the site, but it is already much better than what was there before.<br />
James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com0tag:blogger.com,1999:blog-19002723.post-50139716557429190792012-07-25T14:34:00.000-07:002012-07-25T14:34:42.394-07:00On client components for web appsThis is a response to <a href="http://tjholowaychuk.com/post/27984551477/components">a blog post by TJ Holowaychuk</a> about browser-based components for web applications, and <a href="http://blog.izs.me/post/27987129912/tj-holowaychuk-components">Isaac's notes on TJ's post</a>.<br />
<br />
I am going to try to make this brief because I get tired of people in the Node community wanting to apply the same patterns from Node in the browser. I feel like I say these things on a periodic basis, but human communication is hard, and I certainly could do better. But I also want to get back to just making things. So this will be terser than I normally would like.<br />
<br />
<b><span style="font-size: x-large;">Web components</span></b><br />
<br />
I suggest TJ look at <a href="https://github.com/volojs/volo">volo</a>, my attempt in this space. It does lots of what he describes already, and it can even be used as <a href="https://github.com/volojs/ghvolo">a module in another command line tool</a>. We use volo for some things in Mozilla already.<br />
<br />
volo uses GitHub as the component registry. It does so without the downsides that TJ mentions.<br />
<br />
Specifically, volo uses the <a href="http://developer.github.com/v3/">GitHub HTTP API</a> to get version tags, do registry searches. I grabs .zip snapshots for a given version/github commit/branch, so the command line tool (the consumer) does not need use git. Git is not necessary on the client side.<br />
<br />
This means the downloaded code is smaller -- no need to get a full repo and all of its commits.<br />
<br />
volo also understands dependencies via the shorter "owner/repo/tag" IDs instead of the full github URLs.<br />
<br />
It has <a href="https://github.com/volojs/repos">a "shim" repo</a> that means it can support installing components without needing the author of the component to conform to some new publishing system. Since it allows <a href="https://github.com/volojs/volo/wiki/Library-best-practices#wiki-packagejsontagsgithub">{version} replacement in URLs</a>, the registry setup just needs to be done once. From then on, normal best-practice versioning via git tags is enough.<br />
<br />
Some <a href="http://tagneto.blogspot.ca/2012/05/package-management-for-front-end.html">other notes in this post</a>.<br />
<br />
<span style="font-size: x-large;"><b>Base module format</b></span><br />
<br />
Node bros, the AMD trolling is getting tiresome. Node's module system is woefully under-specified for web-based loading. While you can limp along with browserify, there are still these issues:<br />
<br />
* For builds you need a wrapped format. For CDN deployment you need a wrapped format. browserify uses a wrapped format. AMD anyone? For that reason alone, AMD will never go away. Get used to it already.<br />
* Web code needs <a href="https://github.com/amdjs/amdjs-api/wiki/require">a callback-style require</a> for on-demand loading.<br />
* Browserify's uses of file paths for IDs is awful for mixed local and CDN-based loading. Module IDs need to stay in ID format, not translated to a specific file path.<br />
* <a href="https://github.com/jrburke/requirejs/wiki/Plugins">Loader plugins</a> reduce the need for callback-style APIs, and callback pyramid of doom, or inside-out callback hell, or the need for promise based programs. This more than makes up for the extra level of indent in AMD.<br />
<br />
Loader plugins solve the translation issues TJ talks about, and they can participate in optimization builds, meaning templating engines can inline the JS function form of the template. Ditto for language transpilers like CoffeeScript.<br />
<br />
By doing this:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">define(function (require) {<br /> //node module code in here.</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> //Return module value instead </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> //of needing `module.exports`<br />});</span><br />
<br />
you have an AMD module.<br />
<br />
Quit dismissing AMD for surface issues. AMD avoids mandating translation layers that lead to more things for the developer to understand and fix, and more process for the user to go through to deploy code. It is a net win when the source file works when deployed anywhere, without requiring specialized builds/converters.<br />
<br />
Even if you want to personally use Node style and always do builds before loading in the browser, AMD is a great target for the built, wrapped format. You can even use the requirejs optimizer to do this, with <a href="https://github.com/jrburke/r.js/blob/master/build/example.build.js#L412">the cjsTranslate option</a>.<br />
<br />
The universal module boilerplate gets simpler when Node supports AMD's define along with Node's existing module format. If you want to help improve the ugliness, start there.<br />
<br />
AMD comes from real world exprience in Dojo with trying to deploy an unwrapped module format that depended on XHR+eval in dev and a wrapped format for builds. Yes, you can something to to work but the second order translation and support costs are not worth it. Some environments disallow eval. CORS configuration is awkward, and potentially hazardous if your API is on the same domain and CORS is done incorrectly.<br />
<br />
The simplicity of the complete module lifecycle is worth the function wrapping. Quit looking just at what you type once, and consider the complete code lifecycle, and how much time could be wasted there.<br />
<br />
<b><span style="font-size: x-large;">npm's registry as the component registry</span></b><br />
<br />
The implied rules with npm and node's module behavior are not good fits for front end web development:<br />
<ul>
<li>Forcing a directory structure is complicating project layout and loading for web-based projects. It should be possible to publish and install single JS libraries as single files. <a href="https://github.com/volojs/volo/wiki/Library-best-practices#wiki-single">volo can do this</a>.</li>
<li>Related: the "index.js" convention is awful for web development and debugging. Debugging 'jquery.js' instead of trying to find 'jquery/index.js' in the web tools? No thank you.</li>
<li>npm's registry namespace is already polluted. Check searches for 'jquery'. Maybe that just means having a separate npm registry-based registry for client code. But if there needs to be a separate repo, might as well use one that can adapt better to front end development. Like single JS/CSS file installs without extra Java-esque directory structures and metadata debris on the file system.</li>
</ul>
Ender is not a success story for using npm. Ask the Ender folks how well
that worked out. It only works because it is small scale.<br />
<br />
By using GitHub, it comes with user auth handled, private repos, and robust social tools that will not be matched by a something like npm because the financial incentives are not there. Plus developers already use it. For simple open source sharing, make it easy without introducing more things in the middle.<br />
<br />
Github as the registry is not perfect, and we still need some standalone servers that can be run inside corporations/for mirrors, but I would model those standalone servers on the github API. At least the default case of a public repo can be bootstrapped very quickly.James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com5tag:blogger.com,1999:blog-19002723.post-86237398721191830972012-06-26T11:23:00.000-07:002012-06-26T12:11:01.300-07:00Comments on Isaac's ES modules postIsaac Schlueter <a href="http://blog.izs.me/post/25906678790/on-es-6-modules">posted some thoughts on the ES modules proposal</a>. He works on Node and NPM, so it is great to see things from his perspective vs. my browser-based perspective.<br />
<br />
I believe his post an my <a href="http://tagneto.blogspot.ca/2012/06/es-modules-suggestions-for-improvement.html">previous ES modules post</a> fit together well. Here is some feedback on Isaac's post to point out where we align and what may still need more discussion. Section titles and numbered points below match the ones in Isaac's post.<br />
<br />
<b><span class="Apple-style-span" style="font-size: x-large;">Problems with the Current Spec</span></b><br />
<br />
1. <strong>It seems to be based on the assumption that nesting module systems is
a thing that people want.</strong><br />
<br />
As Isaac says, "no one wants to write a module system". Agreed. The default behavior of a built in loader should be enough for most, nearly all people. Others can build their own systems with existing tech, and if they catch on, consider them later.<br />
<br />
I prefer to have support for loader plugins that have a constrained API. I'm not sure how that fits with Isaac's view. My previous post outlines uses cases where AMD folks have found them useful, and they help reduce the "pyramid of doom" of callbacks for resources that are important for a module to do its work, and therefore reduces the complexity of the module's external API. If Isaac has a different way to solve those issues, it would be good to know.<br />
<br />
2. <strong>It puts too many things in JavaScript (as either API or syntax)
which belong in the host (browser/node.js).</strong><br />
<br />
I think Isaac means that the browser implements a default Loader.resolve(), and Node does its own thing, but the language does not have a built default.<br />
<br />
I think there is value in using the browser model as the default language one, with Node having the ability to override as it sees fit. The browser model of single file IO lookup based on a declarative config can work in "everything is available locally" model.<br />
<br />
But for me this is a small distinction. I do not relish trying to convince a separate standards body that is even more unwieldy than es-discuss to code the default browser resolver, but if that is how it must go, so be it.<br />
<br />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
My main desire: the front end developer should not have to ship a library file to do imperative configuration of a loader.</div>
<div>
<br /></div>
<br />
3. <strong>It borrows syntax from Python that many Python users do not even
recommend using.</strong><br />
<strong><br /></strong><br />
Agreed. I would not include import * if it were just up to me. The "evaluate dependencies first before evaluating the current module" means that just normal let/var destructuring could be used, no more need for import at all. All that is needed is a way to call out a dependency either via a keyword like "from" or via an API like "from()" or "require()". Or reusing "import" to be simpler, as Isaac does later in the post.<br />
<br />
4. <strong>It favors the “object bag of exported members” approach, rather
than the “single user-defined export” approach.</strong><br />
<strong><br /></strong><br />
I too prefer a way to set the exported value (<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">return</span> in AMD parlance, <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">module.exports =</span> in Node). There needs to be a way to allow just the "exports" property assignment for circular dependencies, but for everything else (the vast majority of modules) assignment is nice.<br />
<br />
I believe that exports approach in the ES design was chosen so that they can do the type checking of export values, supports circular dependencies, and with destructuring, it was argued that it was more palatable for getting the export value with this model:<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">import {jQuery} from 'jquery';</span><br />
<br />
Not my favorite, but I could live with it.<br />
<br />
The <a href="http://tagneto.blogspot.com/2012/06/es-modules-suggestions-for-improvement.html">"middle way" evaluation approach</a> would support module value assignment.<br />
<br />
<br />
<b><span class="Apple-style-span" style="font-size: x-large;">A Simpler Proposal</span></b><br />
<div>
<br /></div>
<br />
1. <b>A <code>Loader</code> built-in object, with a few methods that must be
specified before modules can be used.</b><br />
<br />
See feedback above. If a front end developer needs to ship a JS library to bootstrap the Loader, that is a failure condition for me.<br />
<br />
I can understand exposing a Loader API so that for example someone could make an AMD API shim, but the ideal, future case should be no need for imperative setup. Simple declarative setup only, and just for how to deal with module IDs-to-paths, and possibly a declarative shim config for legacy scripts. Note that this config is about specific modules, not implementing the general module/resolution API.<br />
<br />
2. <b>Within a module, the <code>import pathstring</code> syntax that can be easily
detected statically in program text before evaluation, and returns a
module’s exported object.</b><br />
<br />
I was thinking in terms of "from", but "import" is fine if it is used like this. I do not care about the name.<br />
<br />
I also prefer an API that could be used/detected by browser scripts that need to live in the on-ES modules world and the new one.<br />
<br />
This is what AMD loaders do today for <a href="http://requirejs.org/docs/whyamd.html#sugar">the sugared form</a>, except it looks for <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">require('string literal')</span>.<br />
<br />
3. <b><code>Loader.define(path, program text)</code> defines a module at the
specified <code>path</code>, with the <code>program text</code> contents.</b><br />
<br />
I suggest using "module ID" instead of path here. Once modules are combined, it is best to refer to logical IDs, like 'jquery' instead of 'my/specific/path/to/jquery.js', because it makes it hard to combine sets of built modules together.<br />
<br />
AMD loaders use module IDs, and it works out well for us.<br />
<br />
This Loader.define call is similar to the capabilities in AMD. AMD uses a <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">function (require, exports, module) {}</span> wrapper for <program text="">, but I can see where perhaps an ES loader could just get by with <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">{}</span>.<br />
<br />
No comments on 4,5,6. Those are about the resolver API, which I do not care too much as long as what is used in the browser means an app developer does not have to ship another library to use it.<br />
<br />
7. <b>Within a module, the <code>export expression</code> statement marks the result
of <code>expression</code> as the exported value from the module.</b><br />
<br />
</program><br />
<div>
As Isaac alluded, I think circular dependencies will kill this one. Or at least, there needs to be a way to do what is possible in Node/AMD where an exports object can be created for a module before it executes, and that exports is available for other dependencies in a cycle.</div>
<div>
<br /></div>
<div>
8. <b>The global object within a module context is equivalent to
<code>Object.create(global)</code> from the main global context.</b> and:</div>
<div>
9. <b>If a module does not contain an <code>export</code> statement, then its global
object is its export.</b> </div>
<div>
<br /></div>
<div>
This is tricky since some libs may export more than one global. I would probably still favor allowing all scripts loaded within a container to share the same "global" space.</div>
<div>
<br /></div>
<div>
With a <a href="http://requirejs.org/docs/api.html#config-shim">shim config</a>, this would allow consuming scripts that use browser globals, and for using library plugins that use a browser global to attach functionality (jQuery or Backbone plugins for example).</div>
<div>
<br /></div>
<div>
I wouldn't mind a declarative loader config that allowed for #8, not sure if it should default to on or off. Maybe on, and if using legacy scripts, you have to do a bit of work by configuring it to off.</div>
<div>
<br /></div>
<div>
This area needs more thought, but my general goal is not needing a library.js to set up imperative loader APIs to do loading in the browser. As long as that works out, I will probably be fine with it.</div>
<div>
<br /></div>
<div>
<b><span class="Apple-style-span" style="font-size: x-large;">In Web Browsers</span></b><br />
<br />
As mentioned above, I believe it is better to use module ID instead of <path> for the name of a module via Loader.define(), but to use module IDs instead. That would probably mean transforming the script tag to be script suggestion to <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">script module="moduleId" src=""</span>.
<br />
<br />
But really, I'm fine with just a JS API for this, so just doing <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">System.load('id'); or Loader.load('id')</span> in an inline script tag is fine by me.<br />
<br />
<b><span class="Apple-style-span" style="font-size: x-large;">What’s Missing from this Proposal</span></b><br />
<br />
For me,<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> Loader.define('id', {})</span> is just another way to say <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">module 'id' {}</span>. I like having an API instead of/in addition to a keyword, for shimming something that could work in today's browsers, although in this case shimming may not be important. So I'm neutral on that. <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">module 'id' {}</span> is slightly shorter.<br />
<br />
As for sourcemaps: maybe what was meant there was sourceURL, and I agree, it would be good if the Loader.define() method or whatever it is automatically got the same script debugger treatment similar to what sourceURL gets today.<br />
<br /></div>James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com2tag:blogger.com,1999:blog-19002723.post-87012191182418207622012-06-25T21:17:00.000-07:002012-06-25T21:17:21.599-07:00ES Modules: suggestions for improvementThere has been a recent bout of comments about ECMAScript (ES) harmony modules on twitter and elsewhere. Here is my attempt to explain parts of it, some of the design tradeoffs, and perhaps a middle ground that would open up some options that may bridge some gaps.<br />
<br />
Modules are one of those things that seem very simple, but involve quite a lot of decisions and tradeoffs. This post is mostly just about module linking and module ID resolution, and even with that, it is quite long.<br />
<br />
If ES Modules do not come up with different ways to work (or maybe explain where I have it wrong), they are not competing well with what can be done with a combination of CommonJS/Node and AMD.<br />
<br />
My background: I work on <a href="http://requirejs.org/">RequireJS</a> and <a href="https://github.com/amdjs/amdjs-api/wiki/AMD">AMD</a>.<br />
<br />
<b><span style="font-size: x-large;">What is it</span></b><br />
<br />
First, some links to the specs. The "harmony" moniker means it is in process for the next version of the ECMAScript (JavaScript) language:<br />
<br />
<ul>
<li><a href="http://wiki.ecmascript.org/doku.php?id=harmony:modules">Harmony Modules</a>: The basic module spec.</li>
<li><a href="http://wiki.ecmascript.org/doku.php?id=harmony:module_loaders">Harmony Module Loaders</a>: Specifies how you can create isolated loaders for modules. Very useful.</li>
</ul>
<br />
The <a href="http://wiki.ecmascript.org/doku.php?id=harmony:modules_examples">module examples page</a> is suggested if you want to get a feel for it, but it is good to read the other docs too. It can be a bit daunting though, unless you speak the spec language.<br />
<br />
<span class="Apple-style-span" style="font-size: x-large;"><b>Points of reference</b></span><br />
<br />
One way to evaluate how ES Modules works is to compare it to something you may already know:<br />
<br />
<ul>
<li><a href="http://wiki.commonjs.org/wiki/Modules">CommonJS</a> / <a href="http://nodejs.org/docs/latest/api/modules.html">Node</a>. Node implements a version of the CommonJS module API.</li>
<li><a href="https://github.com/amdjs/amdjs-api/wiki/AMD">AMD</a> / <a href="http://www.requirejs.org/docs/api.html">RequireJS</a>. RequireJS implements the AMD module API.</li>
</ul>
<br />
<span class="Apple-style-span" style="font-size: x-large;"><b>Run time vs compile time</b></span><br />
<br />
ES is a "compile time" approach where the formats mentioned above are "run time" approaches. Maybe not precise terms, but here is a definition of what is meant by those terms for the purposes of this post:<br />
<br />
<b>"Compile time" means:</b><br />
<br />
<ul>
<li>JS text is parsed, and the "module" "import", and "export" syntax is found.</li>
<li>Any dependencies are fetched and parsed.</li>
<li>Once the dependency tree has been all fetched, the ES module loader will wire up the exports from a dependency to a module's "module" or "import" use, and do type checking on that export type and how it is referenced in the module</li>
<li>The module code is then evaluated/executed.</li>
</ul>
<br />
<br />
<b>"Run time" means</b>: there is usually no pre-parse stage. The JS text is evaluated, and any module API that is encountered is run as it is encountered.<br />
<br />
AMD will actually do a small parse step if the module looks like:<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">define(function (require) {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> var a = require('a');</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">});</span><br />
<br />
In that case, it will parse the function to look for require() dependencies, and then <b>load and execute</b> the dependencies first before running the function above.<br />
<br />
"Compile time" was chosen for ES because:<br />
<br />
<ul>
<li>it is familiar from other scripting languages</li>
<li>sets the way for other possible static features, like macros</li>
<li>ensures that "import *" are static, *not* dynamic bindings</li>
<li>allows some type checking on the values that are explicitly "export"ed.</li>
<li>generally seen as safer and easier to reason about that run time.</li>
</ul>
<br />
The CommonJS/Node style of pure runtime, no parse, was hard to get to work with some edge cases as I understand it, but I heard that second-hand, I did not see that discussion.<br />
<br />
<b><span class="Apple-style-span" style="font-size: x-large;">Impact of compile time</span></b><br />
<br />
For compile time to work well, it should use new keywords in the language, to have clear markers on what is participating in the module system.<br />
<br />
Although, it could work with a module API instead of new syntax, by only recognizing literal use of that API, and do not support variable assignment of the API or dependencies to other names. This is what AMD does for the "<a href="http://requirejs.org/docs/whyamd.html#sugar">sugared CommonJS form</a>" mentioned above.<br />
<br />
For "import *", static binding is critical because anything that is a runtime scope lookup gets into "with" territory, and "with" has been seen as a mistake by the committee. ES5's 'use strict' <a href="https://developer.mozilla.org/en/JavaScript/Strict_mode#Simplifying_variable_uses">bars its use</a>.<br />
<br />
Since new syntax is involved, ES Modules cannot be "shimmed" into existing JS libraries. There is a <a href="http://wiki.ecmascript.org/doku.php?id=harmony:module_loaders#module_objects">Module Loader "runtime" registration</a> call that can used for a module to register its, but it means those libraries cannot participate in the compile time linking stage, so they need to be pre-loaded by a script loader before an ES Module can effectively reference it with module syntax.<br />
<br />
<br />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span class="Apple-style-span" style="font-size: x-large;"><b>Module ID resolution</b></span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
One other consideration, one that is usually overlooked when talking about modules, is how a module ID like "jquery" is resolved to a file path and loaded.</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
Both CommonJS/Node and AMD/RequireJS support "short, logical names" for dependencies. So, you can say require('jquery') and that jquery gets converted to a path using some algorithm. Node uses multiple paths to find jquery.js, and AMD in the browser relies on a declarative configuration to do so.</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
ES modules do not really have support for this, unless you also implement <a href="http://wiki.ecmascript.org/doku.php?id=harmony:module_loaders#options.resolve_relurl_baseurl">an imperative resolver</a>. They support full URLs, like:</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">module foo at 'http://example.com/scripts/foo.js'</span></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
but we have found in AMD that it is useful to be able to say <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">require('jquery')</span>, but then declaratively map that to zepto.js. </div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
So, an individual module specifies a dependency on an API provider, but how that provider is satisfied is resolved using the declarative configuration.</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
If there is only an imperative resolve API, no simple declarative API to resolve short names, it will mean shipping a userland "loader library" to effectively use modules. This opens the door to balkanization in module ID resolution since there is not built in support.</div>
<br />
<br />
<span class="Apple-style-span" style="font-size: x-large;"><b>Special factors in JavaScript</b></span><br />
<br />
There are a few special factors with JavaScript that are not usually in other programming languages, and they have an impact on the design:<br />
<br />
<ul>
<li>The largest deployed use case of JavaScript, the browser, is async, network IO. File size and number of requests are very important to performance. So combining modules together into one file, and minifying/transforming the source for smaller delivery is common.</li>
<li>There is a large legacy codebase of browser-based JavaScript that just use browser globals, and no real module format. Some small uses of JavaScript do not need modules, and browsers will support those use cases indefinitely.</li>
</ul>
<br />
<b><span class="Apple-style-span" style="font-size: x-large;">My goals</span></b><br />
<br />
I want AMD and RequireJS to go away.<br />
<br />
They solve a real problem, but ideally the language and runtime should have similar capabilities built in.<br />
<br />
Native support should be able to cover the 80% case of RequireJS usage, to the point that no userland "module loader" library should be needed for those use cases, at least in the browser.<br />
<br />
If the ES module format requires a web developer to use a script loader to use existing, non-AMD/CommonJS, non-ES JS in a project for those 80% use cases, it is a failure.<br />
<br />
Example: If I cannot use jquery and backbone in an ES 6 module without needing another library to preload or prep those libraries for ES 6 module use, then existing JS users will not see much advantage over using AMD.<br />
<br />
If the web developer needs to code any imperative logic to wire up the ES Module Loader, that will result in a loader library. That is a failure condition.<br />
<br />
As compared to AMD: if the ES approach cannot do the above without a helper loader library and the ES approach does not allow something like loader plugins, then there is no contest -- AMD will still be more useful to a developer than the built in system. Small savings in the amount to type and a thin layer of type checking is not enough.<br />
<br />
This may very well not be the goal of ES modules, and it would be great if the specs or some background material acknowledge that, and list out the mitigation strategies developers are expected to use.<br />
<br />
<b><span class="Apple-style-span" style="font-size: x-large;">Shortcomings of ES modules</span></b><br />
<br />
Right now, ES harmony modules do not improve an AMD user's workflow because of the following:<br />
<br />
<b><span class="Apple-style-span" style="font-size: large;">New syntax makes it very hard to optionally upgrade</span></b><br />
<br />
If I am the author of something like jQuery or Backbone, I cannot optionally add in a way to register as an ES module because ES modules use new syntax. However, there are many uses of those libraries which will not be in ES module-capable browsers.<br />
<br />
The Node and AMD communities have found optional opt-in via a runtime API very useful for adoption of code that works with their module systems, but still work in older "use plain script tags with browser globals" approach.<br />
<br />
There is a runtime API in the ES module loader proposal that would allow a legacy script to <a href="http://wiki.ecmascript.org/doku.php?id=harmony:module_loaders#module_objects">register something as a module</a>, but that requires the end developer to use another script loader library to load that library so it can do that runtime call, then start loading ES module code.<br />
<br />
The developer may as well just stick with AMD. Complexity has not been reduced.<br />
<br />
<b><span class="Apple-style-span" style="font-size: large;">Register module and a global</span></b><br />
<br />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
Backbone originally had trouble adopting AMD because if it called define() to register a module, they found other libraries, like Backbone plugins, would break. The plugins were expecting to find a Backbone global variable but when Backbone called define() it was not also exporting a global.</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
This same problem will exist in ES-mixed code. Any dynamic registration also needs to allow an export of a global so that downstream libraries will work until they are also converted to optional module registration.</div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<br /></div>
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
There should be a migration path, one that allows gradual rollout of modules without requiring a project to go whole hog on module syntax.</div>
<div>
<br /></div>
<br />
<span class="Apple-style-span" style="font-size: large;"><b>Declarative module ID resolution</b></span><br />
<br />
While I have made tools to allow a developer to "convert" an existing library to AMD, there are many developers that did not want to touch existing libraries. It makes it difficult to compare against new versions and there is a concern that the conversion introduces breaking scope changes (rightly so).<br />
<br />
So the <a href="http://requirejs.org/docs/api.html#config-shim">"shim" configuration</a> for requirejs was introduced to allow specifying dependencies and an export value for JS code that does not call a module API. This has been well received in the community. <a href="https://github.com/jrburke/requirejs/wiki/Upgrading-to-RequireJS-2.0#wiki-shim">More background on shim here</a>.<br />
<br />
"shim" with "<a href="http://requirejs.org/docs/api.html#config-paths">paths</a>" and "<a href="http://requirejs.org/docs/api.html#config-map">map</a>" make it possible to declaratively set up a configuration that allows for one file IO lookup per module ID, an easy way to "shim" old libraries, and to load more than one version of a module for use by different modules. That covers the 80% case for using old and new code with a module loader.<br />
<br />
By using a declarative configuration that is supported by the "default" module ID resolution mechanism in the language, then it avoids having to ship a userland loader library for browser use. This is a <b>big win</b> because it will help kill AMD.<br />
<br />
It is fine if the Module Loaders API still has an imperative API to set up different module ID resolution logic. That would allow Node to maintain its current multiple IO, nested directory lookup logic. However, the default should favor the harsher browser environment in such a way that an extra loader library is not needed.<br />
<br />
<span class="Apple-style-span" style="font-size: large;"><b>Loader plugins</b></span><br />
<br />
I can appreciate that supporting Loader plugins may seem out of scope for the default module loader, but they have been incredibly useful for AMD. Node has seen a use for them, even though they are done in a different way via <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">require.extensions</span>. They effectively allow use of transpiled languages.<br />
<br />
I find the AMD loader plugins better than Node's approach because:<br />
<br />
<ul>
<li><b>load behavior vs. file format</b>: since a prefix is used on the resource ID instead of just using a file extension suffix, it allows multiple plugins to deal with the same type of file extension. For example, "text!index.html" and "template!index.html" can be used in the same app, the first one just giving the raw text, the second one "compiling" some text for use as a template. The developer, not the plugin provider, chooses the right use. It still allows "single extension" plugins too, and for those, they can omit the file extension in the ID, so no increase in ID length.</li>
<li><b>one IO lookup</b>: For a resource ID "foo", node may do a lookup for "foo.js", "foo.coffee" and "foo.node". By specifying the loading mechanism via the prefix, it avoids multiple IO lookups, which are important for browser use. It also makes it clear what handles the loading.</li>
</ul>
<br />
AMD loader plugins can participate in build steps, so the "text!" plugin can inline the text as module in a built file:<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">define('text!index.txt', function() {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> return 'hello world';</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">});</span><br />
<br />
That is incredibly useful for getting good network performance in the browser.<br />
<br />
Even for local file environments like Node, being able to combine all the assets for a program into one file is really great for distribution. It is conceptually simpler to reason about tracking one file vs. "nested directory of directory" installs. Think of it as a way to easily share shell scripts.<br />
<br />
<b><span class="Apple-style-span" style="font-size: x-large;">The middle way</span></b><br />
<br />
For developer workflow, right now AMD is a better alternative than ES harmony modules, given the choices around compile time linking, new syntax, and the use of imperative ID resolution.<br />
<br />
Here are some suggestions on how to allow some of the benefits of the compile time approach with the run time ones used by AMD. The goals are reuse non-module code in modular systems, allow for a way to get some static version of import *, and perhaps even macros.<br />
<br />
<b><span class="Apple-style-span" style="font-size: large;">Fetch dependencies, execute, modify, execute</span></b><br />
<br />
The core of the middle way for compile time vs run time:<br />
<br />
<ul>
<li>do not force compile time operations to be all up front, before any evaluation.</li>
<li>evaluate dependencies before executing the current module.</li>
<li>provide an API for modules, not just new syntax</li>
</ul>
<br />
These are effectively what AMD does today, except it does not have a way to alter the AST before final execution. Well, an AMD loader could do that, but AMD loaders have traditionally avoided it. However, the <a href="https://github.com/jrburke/require-hm">harmony loader plugin</a> I made effectively does this to support "import *". More below:<br />
<br />
The ES module loader would operate like so:<br />
<br />
<ul>
<li>Load JS text. Parse out dependency references.</li>
<li>Load dependencies, parse out their dependencies, load them, etc...</li>
<li>Before executing a given module, <b>execute</b> its dependencies, and wait for the dependencies to finish exporting their module values.</li>
<li>Take that exported value and if there is an "import *" in the current module, modify the AST of the current module such that it gets a locally bound variable to any of the hasOwnProperties of the dependency that are known <b>at that time</b>. So, any properties added to the module after this point are not visible. This should avoid concerns about dynamic scope.</li>
<li>Once the module AST has been fixed up for any import *, then evaluate it.</li>
</ul>
<br />
When parsing out dependency references, look for any new keywords, but also any API that corresponds to that keyword. So, look for <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">at('moduleID')</span> for dependency references in addition to <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">at 'moduleID'</span>.<br />
<br />
The runtime API for the module should be something like<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> at('moduleID')</span> for dependencies and <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">exports.propertyName</span> for specifying export properties. <b>I am not arguing for that specific API</b>, just mentioning that there would be an API alternative to the new syntax. The API alternative <b>does not</b> need an import alternative though.<br />
<br />
Since a module is executed before giving the exports to a module that depends on it, and since there is a runtime API for modules, then that allows existing JS code to opt-in to ES modules without getting bitten by new syntax.<br />
<br />
Since all dependencies are executed before executing the current module, an "import *" can be supported, and I believe that would allow for macros later.<br />
<br />
There are some limitations around circular dependencies, but they are still possible, and the restrictions are minor in comparison to allowing existing code to opt in to ES modules and still work in non-ES module environments.<br />
<br />
<b><span class="Apple-style-span" style="font-size: large;">Declarative configuration</span></b><br />
<br />
Support something like the "paths", "map" and "shim" config as used in RequireJS. This allows easier use of old code, and scales up to very large code without requiring a developer to ship a library that sets up an imperative resolution API.<br />
<br />
<b><span class="Apple-style-span" style="font-size: large;">Support loader plugins</span></b><br />
<br />
Now that all dependencies are executed before executing the current module, then it is easier to support loader plugins, as the loader will have the exported value for that plugin resource before running the current module.<br />
<br />
These environment-based loading, like an "env!" plugin that can load a module for Node and a different API-compatible one for the browser. See also a "has!" plugin for feature detection-based loading, and plugins to enable transpilers.<br />
<br />
Yes, it is more to sort out, but they provide a lot of benefit. AMD has already primed this pump. It even works with a build/optimization step for inlining resources.<br />
<br />
<b><span class="Apple-style-span" style="font-size: large;">Use string IDs for module identifiers</span></b><br />
<br />
This allows the module references in dependencies to be the same as the ID that is inlined when modules are combined and named in built files. Right now it is weird to use a JS identifier, like <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">module Foo {}</span> to name a module, but then see <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">module Foo at "Foo"</span>. It is hard to match up <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">at "Foo"</span> with <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">module Foo</span>.<br />
<br />
<b><span class="Apple-style-span" style="font-size: x-large;">The extreme positions</span></b><br />
<br />
The following is based on my limited experience. I am not a language designer. I am but a simple plumber that uses the pipes that available to build things. I may not have the right long term thinking involved, but I think the following make the ES module proposal simpler.<br />
<br />
To be clear though, I think the middle way above is enough to bridge the gap. <b>Please</b>, do not read the following and then discount the middle way. The middle way is separate from these more extreme measures.<br />
<br />
<span class="Apple-style-span" style="font-size: large;"><b>No new syntax</b></span><br />
<br />
If there is a runtime API available to allow existing code to opt in, just shed the new syntax. Just have one way to do it via API that can also then be shimmed.<br />
<br />
<b><span class="Apple-style-span" style="font-size: large;">no import *</span></b><br />
<br />
import * makes it difficult to determine where code comes from. If<br />
this type of construct is allowed:<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">module foo {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> var sin = function () {};</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> module bar {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> import * from "Math";</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> sin();</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">}</span><br />
<br />
for a minifier, it now needs access to Math to do its work correctly. This has not been the case in the past. It would suck to need all of the code for all the modules used in a system just to complete a minifier pass.<br />
<br />
For developers, if you have two modules that do import * it can be difficult to know where something comes from.<br />
<br />
Destructuring provides enough benefit for these use cases, just do the<br />
comma separated list for things you really use:<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> import {sin, cos} from "Math";</span><br />
<br />
import * is a bad pattern and it does not save much.<br />
<br />
If you get rid of import * then with the "middle way" of evaluating modules, then regular var/let-based destructuring is enough, there is no need for an import keyword.<br />
<br />
<b><span class="Apple-style-span" style="font-size: large;">No macros</span></b><br />
<br />
Similarly, rethink the need for macros long term. They suffer from the same "where did this come from" problem as import * does. The function capabilities in JavaScript are good enough to get the job done for the "don't repeat yourself" task.<br />
<br />
<b><span class="Apple-style-span" style="font-size: x-large;">A way forward for today's code</span></b><br />
<br />
The nice thing is that we can prototype this new world by combining what CommonJS/Node does today with AMD. So we can just use the require() and define() as used today to get there. The ES committee does not have to ratify it, and we get the benefit of real world implementation and use before committing to default language support.<br />
<br />
<a href="https://github.com/requirejs/cajon">Cajon</a> is my attempt from the AMD side to bridge the gap with plain Node code and a runtime browser loader. <a href="https://github.com/linkedin/inject">LinkedIn's Inject</a> is another AMD loader that uses a similar approach. So, just use CommonJS/Node modules in the browser in dev, use the r.js optimizer to compile down to AMD for final deployment.<br />
<br />
The <a href="https://github.com/jrburke/r.js/blob/master/build/example.build.js#L412">cjsTranslate</a> capability in the r.js optimizer allows a developer that always likes to do builds, even in dev, can code in Node syntax but output to AMD and load it in the browser either by the small <a href="https://github.com/jrburke/almond">Almond AMD shim</a>, or the full dynamic loader via RequireJS. Or choose <a href="http://dojotoolkit.org/">Dojo</a> or <a href="https://github.com/cujojs/curl">curl.js</a>.<br />
<br />
<br />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;">
<a href="https://github.com/substack/node-browserify">browserify</a> can be updated to use AMD as its transport format instead of its home-grown require.define() API, and then not have to ship a loader, but use one of the AMD loaders/API shims. browserify is nice in that, unlike the r.js optimizer+cjsTranslate, it provides browser module shims for the native node modules. It would be great to break those out as a separate project that could be consumed by a project just using the r.js optimizer.</div>
<div>
<br /></div>
<br />
If Node adds define() support, callback-require for use within a module for dynamically calculated dependencies, and supports at least a limited form of loader plugins, then we're done. The <a href="https://github.com/jrburke/amdefine">amdefine</a> project is an implementation proof of that support. There are details to sort out, but it is doable. Any node committers are interested, give me holler. We can work out the details.<br />
<br />
<b><span class="Apple-style-span" style="font-size: x-large;">Summary</span></b><br />
<br />
For developer workflow, the current ES module spec is not competing well with a combination of CommonJS/Node and AMD with loader plugins. Or even just AMD with loader plugins.<br />
<br />
Using the middle way for module execution and getting a good declarative module ID to path configuration in the ES spec will level the playing field. Add loader plugins to get language transpiler support and environment/feature detection loading that is efficient for the browser.<br />
<br />
I have given some of this feedback to the es-discuss list, but I think some of it, in particular the "middle way" module evaluation flow, got lost in my poor communication where it seemed like I was proposing a dynamically scoped import *. Hopefully this post clarifies what I was trying to achieve with that earlier feedback.<br />
<br />
Finally, I appreciate working on the ES committee is very difficult work. I do not envy them. I do not mean for this feedback to come across harshly, but the committee is running out of time, and I do not feel like it has made the case very well for how what is being proposed is better than what we have cobbled together with existing technology. To be clear, I want an ES Modules proposal to succeed because I do not want to do AMD or RequireJS forever. Hopefully this feedback can be viewed as loyal opposition, and as a challenge to do better, or at least to do it in a way that is explained more clearly.<br />
<br />James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com7tag:blogger.com,1999:blog-19002723.post-73716990827326162012-06-20T23:26:00.000-07:002012-06-20T23:26:18.181-07:00volo 0.2.1 released, using volo as a libraryvolo 0.2.1 has been released. This is a bug fix release, if you are using 0.2.0, you are encouraged to upgrade. Here is the <a href="https://github.com/volojs/volo/issues?milestone=4&page=1&state=closed">list of fixes</a>. To install:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> npm install -g volo</span><br />
<br />
Also new: the <a href="https://github.com/volojs/ghvolo">ghvolo project</a>. It shows how to use volo as a library to just resolve dependency IDs to github-hosted resources. This is useful if you are building a command line interface for fetching front end dependencies and you want to use the same github resolution logic as volo, but without using volo to do the actual installation of dependencies.<br />
<br />
ghvolo is a command line tool that supports "search" and "resolve". Nothing is installed by using ghvolo. It just uses volo as library to resolve IDs to some JSON data. <a href="https://github.com/volojs/ghvolo">See the README</a> for more information and some sample commands.James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com2tag:blogger.com,1999:blog-19002723.post-82972942170287567292012-06-13T11:39:00.002-07:002012-06-13T11:39:36.276-07:00Cajon: a browser module loader for CommonJS/node/AMD modulesI just released the first version of <a href="https://github.com/requirejs/cajon">Cajon</a>. From the README:<br />
<br />
<a href="http://en.wikipedia.org/wiki/Caj%C3%B3n">Cajon</a> is a JavaScript module loader
for the browser that can load CommonJS/node and AMD modules. It is built
on top of <a href="https://github.com/jrburke/requirejs">RequireJS</a>.<br />
<br />
You can use it to code modules for your project in CommonJS/node style, then
use the <a href="http://requirejs.org/docs/optimizer.html">RequireJS Optimizer</a> to
build all the modules into an AMD-compliant bundle. This allows you to
then use a small AMD API shim, like
<a href="https://github.com/jrburke/almond">almond</a>, to get nicely optimized code
without needing a full runtime loader.<br />
<br />
See the <a href="https://github.com/requirejs/cajon">README for more information and restrictions</a>.<br />
<br />
Why do this?<br />
<br />
It is an experiment, to see what people like to build with. The attributes of AMD are needed for any comprehensive JavaScript module solution, but some people really like sugar, and <a href="http://requirejs.org/docs/whyamd.html#sugar">the sugared form of AMD</a> may not be enough for them. They also may want to use existing CommonJS/node modules as-is but still want to get a good, optimized/built story, and something that works well cross-domain in the browser.<br />
<br />
This experiment goes along with the latest requirejs 2.0.2 optimizer setting, <a href="https://github.com/jrburke/r.js/blob/master/build/example.build.js#L412">cjsTranslate</a>, which will automatically convert CommonJS/node modules to have a define() wrapper, so they can be consumed by the optimizer. This would allow you, for example, to build a node command that watched your js lib folder as you did changes to modules in the CommonJS/node format, and build them into an optimized AMD bundle.<br />
<br />
End result, if you cannot bring yourself to use AMD:<br />
<br />
If you do not want to do builds during your CommonJS/node-based module development, use Cajon. If you like doing builds, you can now use the RequireJS optimizer (with the <a href="https://github.com/jrburke/almond">almond</a> AMD shim) to do that.<br />
<br />
To be clear: CommonJS/node modules as-is are not enough for a comprehensive JS module solution. These tools allow you to use them though and fill in the gaps by "compiling down" the code to AMD.James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com0tag:blogger.com,1999:blog-19002723.post-27650400866111560222012-05-03T23:58:00.000-07:002012-05-03T23:58:16.705-07:00Web app template update, now with GitHub authThis is a quick update about the latest version of the <a href="https://github.com/volojs/create-responsive-template">web app template</a> I have been working on.<br />
<br />
You can check out some background on this app template from <a href="http://tagneto.blogspot.ca/2012/04/web-apps-on-github-pages-that-install.html">the previous post</a>.<br />
<br />
The big change since that last post: <a href="http://redpuma.net/blog/">Dan Mosedale</a> had some good feedback about treating <a href="http://help.github.com/pages/">GitHub Pages</a> as a deploy target, so I changed around the template to do do that.<br />
<br />
As part of these changes, <a href="https://github.com/volojs/volo">volo has a new release 0.1.2</a> that has support for doing GitHub authorization to do OAuth-based API calls with GitHub. The app template uses this to create a GitHub repo on the fly for GitHub pages deployment.<br />
<br />
The nice thing about this setup is that the app creation is simpler (no more questions), and you can decide later to deploy to GitHub pages without needing to make the decision up front:<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="281" mozallowfullscreen="" src="http://player.vimeo.com/video/41541859" webkitallowfullscreen="" width="500"></iframe>
<br />
See the video for a more complete walkthrough, but basically to get started now it just looks like this:<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
volo create myproject volojs/create-responsive-template</div>
<div style="font-family: "Courier New",Courier,monospace;">
cd myproject</div>
<span style="font-family: "Courier New",Courier,monospace;">volo appcache</span> (generates the appcache-enabled build)<br />
<span style="font-family: "Courier New",Courier,monospace;">volo ghdeploy</span> (copies the built contents and pushes them to GitHub Pages)<br />
<br />
Useful links from the video:<br />
<ul>
<li><a href="https://github.com/volojs/create-responsive-template">create-responsive-template</a> - the web app template.</li>
<li><a href="https://github.com/jrburke/gaia-devserver">jrburke/gaia-devserver</a> - easy way to serve the Boot to Gecko Gaia UI in Nightly Firefox on your Desktop.</li>
<li><a href="https://developer.mozilla.org/en/Apps/Getting_Started">Getting Started with the Mozilla Web Apps API</a> - for installing apps into Gaia.</li>
</ul>
<br />James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com0tag:blogger.com,1999:blog-19002723.post-4839233159510234582012-04-12T19:01:00.000-07:002012-04-12T19:01:06.572-07:00Making apps with the webThe pieces are coming together: you can can use the web (HTTP, HTML, JS, CSS) to package focused pieces of functionality as apps, with the possibility to make money. <br />
<br />
This post outlines how it is coming together, and how you can get involved. While I work in the Mozilla Labs group and I am sure this post will have a Mozilla slant, this is my personal outlook.<br />
<br />
<span style="font-size: x-large;"><b>What is a web app</b></span><br />
<br />
I am old school, so when I hear "web app" I associate that with the "more of an app style than document style" web page. Something more like GMail instead of the New York Times.<br />
<br />
However, it seems to be getting a more focused definition, something that implies characteristics around the actual usage of the app. Here is a fuzzy definition of it:<br />
<br />
A web app is a focused piece of functionality implemented using web technologies. These pieces of functionality are grouped in an "app dashboard".<br />
<br />
<br />
In the ancient web times, the app dashboard was just your browser bookmarks. However, the newer dashboards have a richer relationship to the apps. There are app icons/sections that can change state, and with Windows Metro, even provide some data display services. There are other associated concepts like an app being purchased and working offline.<br />
<br />
So, in a way, everything old is new again, but better and richer. The "native app" success on mobile devices has set the stage and help define what should be possible.<br />
<br />
While web apps may not be completely equivalent with the capabilities of native apps yet, the stars are moving into position.<br />
<br />
<span style="font-size: x-large;"><b>Alignment of the stars</b></span><br />
<br />
Using web apps to deliver functionality is coming together because to the following forces:<br />
<ul>
<li>Platforms and money</li>
<li>Design practices</li>
<li>Development tools</li>
</ul>
<span style="font-size: x-large;"><b>Platforms and money</b></span><br />
<br />
Here are the platforms where you can use the web to make apps, and how you can make money doing so.<br />
<br />
<span style="font-size: large;"><b>Today's mobile: PhoneGap/Cordova</b></span><br />
<br />
The <a href="http://phonegap.com/">PhoneGap</a> approach works now, today. Apps are made with just plain HTML/JS/CSS and wrapped up in an platform-specific binary that gives the code access to the device capabilities.<br />
<br />
<a href="http://incubator.apache.org/cordova/">Cordova</a> is the Apache-housed, open source parts that PhoneGap is built on. Work in Cordova feeds into web standards discussions. The hope is to not need Cordova at all, but just have all the capabilities built into the web platform.<br />
<br />
Make money the usual ways via the platform-specific app stores: charge up front, do freemium, in-app purchases, use ads.<br />
<br />
<span style="font-size: large;"><b>Future mobile: Boot to Gecko (B2G)</b></span><br />
<br />
Mozilla is working on a web-only mobile OS called <a href="https://wiki.mozilla.org/B2G">Boot to Gecko</a> (B2G). The great thing about this platform: it is just plain web technologies. All apps are fetched from URLs, and they can work offline through web technologies like AppCache.<br />
<br />
The tagline for B2G is "the web is the platform". The "native" in B2G is the web. If the web is not sufficient in some way, Mozilla is putting real effort into improving it. Just like Cordova, the development is all out in the open. You can be the change you want to see by participating.<br />
<br />
The B2G effort includes a set of default web apps, including a web app "home screen". This set of web apps is called <a href="https://wiki.mozilla.org/Gaia">Gaia</a>. You can "install" your app into the home screen using the <a href="https://developer.mozilla.org/en/Apps">web app APIs</a>, and Mozilla is working on <a href="https://www.mozilla.org/en-US/apps/">a marketplace</a> and APIs to allow other marketplaces for apps. There is an identity API via <a href="http://www.mozilla.org/en-US/persona/">Persona</a>, and efforts to work out other details like digital receipts. The <a href="https://wiki.mozilla.org/Apps/Roadmap">apps roadmap page</a> will help you get acquainted with some of the moving pieces if you want to get involved.<br />
<br />
Right now the B2G code is still really early in development. It is a bit rough to get going. Think of it as getting access to iOS a year before the first iPhone shipped.<br />
<br />
<span style="font-size: large;"><b>Desktop</b></span><br />
<br />
Both Firefox and Chrome will know how to install web apps. The hope is that all the browser vendors can converge on some common APIs and get those uplifted into all browsers.<br />
<br />
Mozilla is also working out a way to <a href="https://developer.mozilla.org/en/Apps/Apps_architecture">"install" a web app from your browser into your desktop operating system</a>. A small OS app shell will be wrapped around a web renderer that is tied to that specific web app. For the end user, it just looks like any other app installed on their desktop. <br />
<br />
You will be able to use <a href="http://msdn.microsoft.com/library/windows/apps/hh465037">web technologies</a> for <a href="http://msdn.microsoft.com/en-us/windows/apps/">Windows 8 Metro apps</a> and in Windows Phone. Microsoft is making the web stack a first class development option for their platform.<br />
<br />
On the money side:<br />
<br />
There is the <a href="https://chrome.google.com/webstore">Chrome Web Store</a> for Chrome browsers, and Mozilla is working on a <a href="https://www.mozilla.org/en-US/apps/">Marketplace</a>. The hope is to work out open, standardized APIs that would allow other marketplaces and integration with any web browser. There is a <a href="http://www.windowsphone.com/en-US/marketplace">Windows Phone Marketplace</a>, and I fully expect a marketplace for Metro apps for Windows 8.<br />
<br />
While marketplaces by themselves do not translate directly into money, they establish ways for users to buy functionality and discover new apps. They set up a fertile environment. Using ads and providing apps for free with money coming in from other channels will still be possible.<br />
<br />
<span style="font-size: x-large;"><b>Design practices</b></span><br />
<br />
There are a few design practices that fit well with web app development:<br />
<br />
<span style="font-size: large;"><b>Mobile first</b></span><br />
<br />
The number of mobile, non-desktop PC devices are growing like crazy. The "app" phase was brought on by these mobile devices. By <a href="http://www.lukew.com/resources/mobile_first.asp">focusing on these mobile experiences first</a>, it sets you up to reach a very big market.<br />
<br />
<span style="font-size: large;"><b>Focused execution</b></span><br />
<br />
This falls out of the Mobile First design approach. When you start with mobile, the interfaces are usually more focused. This is a great way to do any app design -- make sure to distill the design problem to the smallest user goal possible, and then build up from there as necessary.<br />
<br />
<span style="font-size: large;"><b>Responsive design</b></span><br />
<br />
Different kinds of devices that have different resolutions and input methods are best served with some custom work done for each device. However, there is a lot of code that can be shared by moving away from a "pixel perfect" and device-specific look to more of a
"web aesthetic" using <a href="http://en.wikipedia.org/wiki/Responsive_Web_Design">responsive design.</a> <br />
<br />
<span style="font-size: large;"><b>URL-based design</b></span><br />
<br />
"<a href="http://designmind.frogdesign.com/blog/mobile-apps-must-die.html">Mobile Apps Must Die</a>" brings up some interesting points when apps are addressable via URLs. It opens up new
discovery possibilities and different ways to "organize" your apps.<br />
<br />
<span style="font-size: x-large;"><b>Developer tools</b></span><br />
<br />
Here are some APIs that can help with web app plumbing:<br />
<ul>
<li><a href="https://developer.mozilla.org/en/Using_Application_Cache">Application Cache</a>: learn it. Yes, it has its quirks, but for serving offline UI, it is the state of the art.</li>
<li><a href="https://developer.mozilla.org/en/DOM/Storage#localStorage">localStorage</a>: for storing smaller name/value paires of data offline. </li>
<li><a href="https://developer.mozilla.org/en/IndexedDB">IndexedDB</a>: for storing larger amounts of data offline. Right now iOS/Android devices do not support IndexedDB, just the older, deprecated Web SQL. However, that should change over time. <a href="http://westcoastlogic.com/lawnchair/">Lawnchair</a> can provide an adapter layer if you need to target platforms with uneven storage support today.</li>
</ul>
The best part: you can start using some of these technologies today, even for apps that target traditional desktop browser deployment.<br />
<br />
I hope to share more tools and approaches as I discover them.<br />
<br />
<span style="font-size: x-large;"><b>Under construction</b></span><br />
<br />
As you jump into making web apps you will discover some rough edges. You may get frustrated. This is OK. The future is under construction.<br />
<br />
If you want to see change, get involved. <a href="http://movethewebforward.org/">Move the Web Forward</a> is a great resource that can help you figure out how you want to engage.<br />
<br />I am a web hacker and my current interest is the B2G platform, so I will be checking out <a href="https://wiki.mozilla.org/Gaia">Gaia</a>. I will share what I find as I go along. If you want to engage on a lower level, check out the Gonk and Gecko layers in <a href="https://wiki.mozilla.org/B2G">B2G</a>. <br />
<br />
<br />
I am really excited to see web apps come into their own for many different platforms that include ways to make money. It is great to see some smart designers figure out how to design for the web app world.<br />
<br />
<br />James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com2tag:blogger.com,1999:blog-19002723.post-26229084572890443202012-03-09T18:03:00.000-08:002012-03-09T18:03:25.872-08:00A dependency manager for web projects<span style="font-size: x-large;"><b>The problem</b></span><br />
<br />
Effective, modular development usually comes with a dependency management tool. Think easy_install for python, ruby gems for ruby, npm for Node.<br />
<br />Front-end web development does not have this kind of tool. Traditionally it has been ad-hoc script management done by the developer. However, now that modular development is <a href="http://requirejs.org/">available now</a>, and will become more prevalent as ECMAScript committee solidifies native module support, front end developers should have a dependency management tool.<br />
<br />
<span style="font-size: x-large;"><b>volo, a solution</b></span><br />
<br />
<a href="https://github.com/volojs/volo">volo</a> is a concrete attempt at this solution. While I do not believe it is fully formed yet, it is already useful, and by having something real to start from, it will better inform any coordinated effort.<br />
<br />
volo does not have to be the only dependency management tool. Hopefully we can use it as way to discuss common approaches that we can all share.<br />
<br />
There have been some efforts in this space, in particular npm, cpm and bpm. I go into why those were not used for volo in a <a href="https://github.com/volojs/volo/wiki/Prior-art">Prior art page</a>.<br />
<br />
The main point of departure is the choice of registry. volo uses GitHub.<br />
<br />
<span style="font-size: x-large;"><b>GitHub as the registry</b></span><br />
<br />
One of the difficulties with running a dependency management tool is having an effective way to find dependencies. The JavaScript community has really embraced GitHub, and I think it has all of the right pieces to serve as a registry:<br />
<ul>
<li>Allows forks. Namespacing is possible via the use of the owner's name, not just a project name.</li>
<li>Has a search function that can give owner/repo values for a given dependency name.</li>
<li>Encourages open source and social communities around the source. </li>
<li>Supports tags for versions.</li>
<li>GitHub has a great simple API for getting versions/repo info.</li>
<li>Provides zipballs/tarballs of tags/branches.</li>
<li>Dependencies can be pulled not only by version tag but by branch name.</li>
<li>Provides a Downloads area for built code.</li>
</ul>
Most importantly, it means volo is already useful without having to stand up a registry or require developers to push code to new places. Many projects can be fetched as-is, and some just need a package.json property to be usable.<br />
<br />
<span style="font-size: x-large;"><b>Conventions</b></span><br />
<br />
volo uses some conventions that are used by some JS projects already, to make it easier to bootstrap. For other projects, usually one package.json property is enough for it to be used by volo.<br />
<br />
Check out the <a href="https://github.com/volojs/volo/wiki/Library-best-practices">Library best practices</a> document for the conventions and configuration used by volo. volo has some <a href="https://github.com/volojs/volo/blob/master/vololib/volo/config.js#L35">project-specific overrides</a> it uses during this bootstrap phase, to simulate how those projects would behave if they used the package.json property that mapped to their development style.<br />
<br />
<span style="font-size: x-large;"><b>Try it Out</b></span><br />
<br />
<a href="https://github.com/volojs/volo/blob/master/README.md">So try it out</a>. It is just one JS file that runs in Node, so it is easy to get rid of it if you do not like it.<br />
<br />
volo's 0.1.0 release has search built in, so it is fun just to try the following kinds of commands to see what it finds on GitHub:<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
> volo search jquery</div>
<div style="font-family: "Courier New",Courier,monospace;">
> volo search backbone</div>
<div style="font-family: "Courier New",Courier,monospace;">
<br /></div>
<div style="font-family: "Courier New",Courier,monospace;">
> volo add jquery</div>
<div style="font-family: "Courier New",Courier,monospace;">
> volo add dojo</div>
<div style="font-family: "Courier New",Courier,monospace;">
> volo add dijit</div>
<div style="font-family: "Courier New",Courier,monospace;">
> volo add ember</div>
<br />
<span style="font-size: x-large;"><b>Next Steps</b></span><br />
<br />
While volo is useful already, it is certainly not done. In particular,<br />
<ul>
<li>Do the <a href="https://github.com/volojs/volo/wiki/Library-best-practices">library best practices</a> make sense? Also, there is info <a href="https://github.com/volojs/volo/wiki/add-dependency-rules">the rules "volo add" uses</a>. I am hopeful volo allows for a reasonable default convention but still give enough flexibility via a couple <a href="https://github.com/volojs/volo/wiki/Library-best-practices#wiki-packagejson">package.json properties</a>, including a way to embed a /*package.json*/ comment for single file JS libraries.</li>
<li>volo does not install nested dependencies yet. It is doable, just needs a bit more coding.</li>
<li>A private/local server that responds to the GitHub APIs used by volo would be useful. Not all code can be placed on GitHub or a public URL, and sometimes even for a developer, having a local cache server is useful. I'm hoping this server can proxy to GitHub for common public libraries and cache them, and then provide a mechanism for private libraries to live in that server.</li>
<li>volo does not support .tar.gz files yet, but it might be nice if it did. zip files were just easier to get working cross-platform. </li>
<li>Allow access to private GitHub repos by using OAuth with GitHub.</li>
<li>Allow a "volo.versions" section in a package.json, to allow listing versions without having to use GitHub tags. This could be useful for libraries that <a href="https://github.com/volojs/volo/wiki/Library-best-practices#wiki-minimizinggithub">do not want to host their code in GitHub</a>.</li>
</ul>
I would like to discuss these topics with other interested parties. In the absence of a common list for it, I suggest using the <a href="http://groups.google.com/group/volojs">volo list</a>, but I am more than happy to have the discussion take place in a new common space. As mentioned above -- volo does not have to be the only tool in this space, but it would be great to work out common conventions and configuration that would allow interop.<br />
If you have a specific code issue, feel free to <a href="https://github.com/volojs/volo/issues">open a volo issue</a>.<br />
<br />
<br />
<br />James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com0tag:blogger.com,1999:blog-19002723.post-89689876219480085492012-02-20T10:26:00.000-08:002012-02-20T10:26:15.431-08:00Template for responsive mobile, offline web appsIt can be difficult to create a web app that is responsive to different screen resolutions and works well in mobile environments or environments with spotty network availability.<br />
<br />
In order to make it easier, <a href="https://github.com/jstrimpel">Jason Strimpel</a> and I created a project template, <a href="https://github.com/volojs/create-responsive-template"><b>create-responsive-template</b></a>, that takes care of a few things for you:<br />
<ul>
<li>Uses <a href="http://twitter.github.com/bootstrap/">Twitter Bootstrap</a> for a baseline style and responsive UI</li>
<li>A couple of small JS helpers for <a href="https://developer.mozilla.org/en/Using_Application_Cache">AppCache</a> and <a href="https://developer.mozilla.org/en/DOM/window.navigator.onLine">network events</a></li>
<li>A build step that optimizes JS and CSS and generates the AppCache manifest for you</li>
</ul>
A bonus: the tooling is in JavaScript, run in Node. <a href="http://tagneto.blogspot.com/2012/01/web-dev-with-two-turntables-and.html">Some background on that choice</a>.<br />
<br />
Here is a video that shows how you can use the template:<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="281" mozallowfullscreen="" src="http://player.vimeo.com/video/36997211" webkitallowfullscreen="" width="500"></iframe><br />
<br />
The next steps I would like to see for this project template:<br />
<ul>
<li><b>An IndexedDB data laye</b>r: A JS helper library that looks like the IndexedDB API, but will use an API shim over Web SQL if IndexedDB is not available. The shim does not have to implement 100% of the IndexedDB API, but something that allows a good start to local data storage. I am looking at <a href="http://westcoastlogic.com/lawnchair/">Lawnchair</a> at the moment, but open to other ideas.</li>
<li>An easy way to <b>generate app store manifests</b> for <a href="https://developer.mozilla.org/en-US/apps">Mozilla Web Apps</a> and <a href="https://chrome.google.com/webstore/category/home">Google Chrome Store</a>.</li>
</ul>
If you have any thoughts on how best to accomplish those steps, <a href="https://github.com/volojs/create-responsive-template/issues">open an issue with details</a>, or leave a comment here on the blog post. <br />
<br />James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com0tag:blogger.com,1999:blog-19002723.post-54043096424408828472010-11-12T14:29:00.000-08:002010-11-12T19:47:12.305-08:00The Open Source of Mozilla F1I am a front end developer on <a href="http://f1.mozillamessaging.com/">Mozilla F1</a>, a sharing extension in Firefox, and this is some technical background on it. Note that any opinions in here are from my personal perspective, you should not treat this as official Mozilla policy, etc...<br /><br />F1 consists of a few components:<br /><br />1) Firefox browser extension<br />2) HTML/JS/CSS share UI served from from the F1 web server<br />3) API server written in Python that lives on the F1 server<br /><br />The extension (#1) is responsible for the share button in the toolbar, injecting <a href="https://github.com/mozilla/f1/wiki/navigator-share-api">navigator.mozilla.labs.share()</a>, and for managing a browser instance that is inserted above the main web browser area. This browser area makes a call to the F1 server to server up the main share UI (#2).<br /><br />The share UI is written in pure HTML/CSS/JS. They are static files that use <a href="http://requirejs.org/">RequireJS</a> to build modular JavaScript which is then optimized via the <a href="http://requirejs.org/docs/optimization.html">RequireJS optimizer</a> into a single application-level file. It does not yet use <a href="https://developer.mozilla.org/En/Offline_resources_in_Firefox">appcache</a>, but it is something I want to add. It should be straightforward. Making sure we properly version files with long cache times will also help.<br /><br />The share UI uses the API server (#3) to do the actual sharing. The API server also manages the complications of the OAuth dance.<br /><br />If you are curious about the implementation, all the code is open. You can download it/fork it at the <a href="https://github.com/mozilla/f1">Mozilla Labs F1 GitHub repository</a>. There is a README.md file with setup instructions.<br /><br />The great thing about this? You can run your own share server that you control. If you want the F1 browser extension to use your server for the share UI and API server, set the following F1 configuration in the about:config Firefox preferences:<br /><br /><span style="font-family:courier new;">extensions.ffshare@mozilla.org.share_url = 'http://your.server.com/share/'</span><br /><br />Ideally that would be an https URL, but if you are setting up your own server, hopefully you know the risks involved. Be sure to restart the browser so the extension will use this new config value.<br /><br />If you think this is really cool and want to help contribute code/ideas/submit pull requests, be aware that we are still in the early stages of F1. We expect to make many changes as we refine it. Mozilla is also new to GitHub, so we appreciate your patience as we try out different work flows.<br /><br />I will close out this post with a couple developer bikeshed questions and answers. Again, these are my personal perspectives, and other people on the team may have different ones.<br /><br /><span style="font-size:130%;"><span style="font-weight: bold;">Why is the share UI served from a web server and not from the extension?</span></span><br /><br />This is an experiment. More things are moving to the cloud/server, and some of the auth APIs, like OAuth, are better suited to server use. While Mozilla already runs some non-trivial server-based services, it is good to get experience with varying types of server-based services.<br /><br />I like the flexibility and ease of updates a server solution provides. Also, it is easier to debug web code that is served from content space vs. browser chrome space.<br /><br />It may allow us to support mobile and even other browsers easier. <a href="https://github.com/jrburke/f1/commit/2247cd0487026585a93283eb37cf49d9f55ac55e">I played around with doing a Google Chrome extension</a> that served up the share UI. Unfortunately, the Google Chrome extension model is not as flexible as what we can do in Firefox, so it does not really work.<br /><br />In particular, I was looking at a Chrome extension <a href="http://code.google.com/chrome/extensions/browserAction.html">Browser Action</a> that served up the UI in an iframe in a <a href="http://code.google.com/chrome/extensions/browserAction.html#popups">Popup</a>, but I could not get the popup wide enough to show the UI, and it closes as soon as an OAuth window jump occurs. We would have to do some rethinking on UI to support that, and it is not clear how beneficial that is at this early development stage.<br /><br />All that said, the UI could be burned in to the extension at some point. It is still too early to tell where this will go.<br /><br /><span style="font-weight: bold;font-size:130%;" >Why did you use Python for the server?</span><br /><br />Ease of setup, combined with some knowledge of real services built with it, and the mix of developer skills of the F1 team.<br /><br />Ideally I would like to see the server running JavaScript via Node, but that is because I am an irrational JavaScript zealot. However, the JS libraries for Node are still young, and at the end of the day we want to solve real user problems, not debug other people's code.<br /><br />That could change in the future. But for now, it is more important to get something up quickly that works well so we can test our designer's usability theories for solving real user problems.James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com2tag:blogger.com,1999:blog-19002723.post-53582012524189464912010-07-13T11:33:00.000-07:002010-07-23T15:37:03.543-07:00Simple Modules FeedbackExecutive summary, with apologies to Jay-Z: "I got 99 problems but lack of lexical scoping ain't one".<br /><br />While at the Mozilla Summit, I saw Dave Herman's presentation on a <a href="http://wiki.ecmascript.org/doku.php?id=strawman:simple_modules">Simple Modules proposal</a> for JavaScript/ECMAScript. <a href="http://blog.mozilla.com/dherman/2010/07/08/javascript-needs-modules/">Dave posted the slides</a>, and be sure to read <a href="http://blog.mozilla.com/dherman/2010/07/09/javascript-needs-modules-ctd/">his follow-up post</a>. I suggest you read his slides and blog posts first for some background.<br /><br />[Sidenote: I'm going to use JavaScript instead of ECMAScript in this post -- JavaScript and I go way back, before it got its colonial, skin disease-inspired name.]<br /><br />Simple Modules is a strawman proposal at the moment, it is still a work in progress. Some of the more interesting parts for me, the dynamic loading, are still very rough and in a separate proposal. It sounded like Dave wants to focus on prototyping the lexical scoping and static loading bits first before proceeding further on the dynamic bits. Great idea on actual prototyping, sounds like they will leverage <a href="http://en.wikipedia.org/wiki/Narcissus_%28JavaScript_engine%29">Narcissus</a> for doing the prototyping.<br /><br />So some of my feedback may be a bit premature, but some of it gets to why there are modules and what should be allowed as a module, so hopefully that might be useful even at this early stage.<br /><br />First, some perspective on where my feedback comes from: I am a front-end developer, I do web apps in the browser. I love JavaScript, I want to use it everywhere, and I believe that it is the only language that has the potential to be used effectively anywhere.<br /><br />However, that is only because JavaScript is available and works well in the browser. Any new solutions for modules should *work well* in the browser to be considered a solution. The browser environment should be treated as a first class citizen, keeping in mind the browser performance implications on any approach. This is one of my <a href="http://tagneto.blogspot.com/2010/03/commonjs-module-trade-offs.html">main criticisms of CommonJS modules</a>, and the reason I write <a href="http://requirejs.org/">RequireJS</a>, a module loader that works well in the browser. I also maintain <a href="http://www.dojotoolkit.org/">Dojo</a>'s module loader and build system.<br /><br /><span style="font-weight: bold;font-size:180%;" >Why</span><br /><br />Why have modules? What are they? Modules are smaller units of code that help build up larger code structures. They make <a href="http://en.wikipedia.org/wiki/Programming_in_the_large_and_programming_in_the_small">programming in the large</a> easier. They usually have specific scope, and avoid dumping properties into the global scope. Otherwise the likelihood of a name collision between two modules is very high and errors occur. So a module system has syntax to avoid polluting the global space.<br /><br />There also needs to be a way for modules to reference other modules.<br /><br /><span style="font-size:180%;"><span style="font-weight: bold;">Simple Modules</span></span><br /><br />The Simple Modules proposal outlines a Module {} block to define what looks like a JavaScript object as far as inspection (for .. in notation, dot property referencing), but is something more nuanced underneath.<br /><br />Anything inside the Module {} block is not allowed to use a global object, and you cannot add/change a Module after its definition. Here is a sample module, called M, that demonstrates some of the syntax and scoped variable implications:<br /><pre>module M {<br /> //In normal code this would define a global,<br /> //but not inside the module declaration. This<br /> //is likely to be an error(?) in Simple Modules.<br /> foo = "bar";<br /><br /> //color is only visible within module M's block<br /> var color = "blue";<br /><br /> //Creates a publicly visible property called<br /> //"name" on the module.<br /> export name = "Module M";<br /><br /> //setColor is only visible within module M's block<br /> function setColor() {}<br /><br /> //Creates a publicly visible property called<br /> //"reverseName" on the module whose value<br /> //is a function<br /> export function reverseName() {}<br />}</pre>You can reference/statically load other modules via <span style="font-weight: bold;">load</span> (syntax is just a placeholder, not set in stone):<br /><pre>module jQuery = load “jquery.js”;<br /></pre>The goals with this approach:<br /><ol><li>Stronger lexical scoping: no <span style="font-weight: bold;">eval</span> or <span style="font-weight: bold;">with</span> allowed in the modules, and a loaded module shares some lexical scope with the module that loaded it.</li><li>No access to a global object by default.</li><li>Hopefully better syntax for declaring modules over the existing <a href="http://yuiblog.com/blog/2007/06/12/module-pattern/">function-based module pattern</a>.</li></ol>To the extent that modules help programming in the large, the simple modules approach do not give me anything more than I have now, and the proposal has some specific weaknesses. I can appreciate there are some juicy things in the proposal for JavaScript engine developers, but as a user of modules, I do not see it as a net advantage. Here is why:<br /><br /><span style="font-size:180%;"><span style="font-weight: bold;">Lexical scoping/Global access</span></span><br /><br />In addition to using the <a href="http://www.yuiblog.com/blog/2007/06/12/module-pattern/">function-based module pattern</a> to avoid leaking globals, I use <a href="http://www.dojotoolkit.org/">JSLint</a> to avoid accessing globals and the use of eval/with. For programming in the large, JSLint helps even more because it enforces a code style that produces much more uniform code. It is built into many editors and easy to run as part of build processes.<br /><br />JSLint is not perfect (I would like to see JavaScript 1.7/1.8 idioms supported, like let and for each), and you may not like some of the style choices. However, reproducible, consistent style that can be checked automatically is more important than bikeshed-based style choices. It warns of global usage, eval and with, and even helps you find unused local variables.<br /><br />What is even nicer is that you can opt out of some of the JSLint choices, you can use some globals if you need to. There is some flexibility in the choices.<br /><br /><span style="font-size:180%;"><span style="font-weight: bold;">Functions as Modules</span></span><br /><br />For #3, better syntax for module definitions, I do not see it as a net win over the function(){} module pattern, particularly how it is used for <a href="http://requirejs.org/docs/api.html#define">modules in RequireJS</a> where it encourages not defining global objects.<br /><br />The Simple Modules syntax does not allow exporting a function as the module definition. This is a big wart to me. Functions are first class entities in JavaScript, one of its strongest features. It is really ugly to me that I have to create a property on a module object to export a constructor function, or some module that lends itself to being a function:<br /><pre>module jQuery {<br /> export jQuery = function () {};<br />}<br /><br />/*** In some other file ***/<br />module jQuery = load “jquery.js”;<br />//ugly<br />var selection = jQuery.jQuery();<br /><br />//less ugly, but more typing, so still ugly<br />var $ = jQuery.jQuery;<br />var selection = $();</pre>Again, ugly. It should be possible to set the module value to be a function. I know this makes some circular dependency cases harder to deal with, but as I outlined in <a href="http://tagneto.blogspot.com/2010/03/commonjs-module-trade-offs.html">the CommonJS trade-offs post</a>, it is possible to still have circular dependencies. Even in CommonJS environments now, it is seen as useful. Node supports setting the exported value to a function via module.exports, and there is a more general CommonJS proposal for a <a href="http://wiki.commonjs.org/wiki/Modules/SetExports">module.setExports</a>.<br /><br />It means the developer that codes a circular dependency case needs to take some care, but it works. Coding a circular dependencies is a much rarer event than wanting to use a module that exports just a constructor function or function. The majority use case should not be punished to make a minority use case a little easier, particularly since you can still trigger errors in the minority use case. Coding a circular dependency will always require special care.<br /><br />This particular point makes it hard for me to get on board with Simple Modules even in its basic lexical scoping/static loading form. I strongly urge any module proposal to make sure functions can be treated as the exported value. We have that capability today with existing module implementations, and it fits with JavaScript and the importance it places on functions.<br /><br />Given the extra typing that would be needed to access functions that are exported as modules, I do not see the Simple Modules syntax a net win over the function-based module pattern, particularly as used in RequireJS.<br /><br /><span style="font-size:180%;"><span style="font-weight: bold;">Beyond Lexical Scoping</span></span><br /><br />For programming in the large, what is really needed are more capabilities than what has been outlined so far for Simple Modules. However, making modules useful needs attention in these areas. This is the "99 problems" part:<br /><br /><span style="font-size:130%;"><span style="font-weight: bold;">Dynamic loading</span></span><br /><br />Dynamic loading is harder to work out than static loading. If there is dynamic loading, it is unclear I would need static loading . The goal of modules is to allow programming in the large, and even for a smaller project, why do I need to learn two ways to load modules (static vs. dynamic), when one (dynamic) will do? Dynamic loading is also necessary to enable all the performance options we have today to load scripts in the browser.<br /><br />There is a <a href="http://wiki.ecmascript.org/doku.php?id=strawman:module_loaders">module loader strawman proposal</a> that would tie into Simple Modules, but I understand it will not be nailed down more until the basic Simple Modules with static loading is worked out/prototyped.<br /><br /><span style="font-size:130%;"><span style="font-weight: bold;">Referring to other modules</span></span><br /><br />It is unclear how a Module Resource Locator (MRL) is translated to a path to find a module. In CommonJS/RequireJS, an MRL looks like <span style="font-weight: bold;">"some/module"</span>, and that MRL is used in require() calls to refer to other modules. <span style="font-weight: bold;">require("some/module")</span> translates the MRL string "some/module" to some path, <span style="font-weight: bold;">"a/directory/that/has/some/module.js"</span>. That path is used to find and load the referenced module.<br /><br />Looking at the Simple Modules examples, it looks like just plain URLs are used as the MRL, and those do not scale well for programming in the large. You will want to use a symbolic name for the MRL, and allow some environment config to map those symbolic names to paths. Otherwise it places too many constraints on how the code is stored. It may not even be a disk -- apparently CouchDB uses design docs to store modules.<br /><br />I have seen some comments about using more symbolic names for MRLs in some of the notes around the proposals, so maybe it is planned.<br /><br />In RequireJS, the symbolic name is also used in the module definition. However, since symbolic names can be mapped, they do not have to be the reverse DNS symbolic names, like "org/mozilla/foo". In fact it is encouraged to not use long names.<br /><br /><span style="font-size:130%;"><span style="font-weight: bold;">Distributing and sharing modules/module groups (packages)</span></span><br /><br />This issue can be treated separately from a module spec, but it could affect how MRLs are mapped via a module loader. And this issue really is important for programming in the large. The solution may just be "use packages as outlined by CommonJS". While there are still some gray areas in the package-related specs for CommonJS, that could be a fine answer to the problem.<br /><br /><span style="font-size:130%;"><span style="font-weight: bold;">Performance in the browser</span><br /></span><br />This is getting even further away from the basic Simple Modules spec, but a solution to this issue should be considered for any module solution. The browser needs to be able to deliver many modules at once to the browser in an efficient way. I have heard that Alexander Limi's <a href="http://limi.net/articles/resource-packages/">Resource Packages proposal</a> may be a way to solve this that may work with the Simple Modules approach.<br /><br />A common loading pattern for web apps will be to load some base scripts from a Content Delivery Network (CDN), then have some domain-specific scripts to load. As long as this still works well with the bundling solution that is great. We already have tools today to help bundling, minifying and gzipping scripts. Any solution will have to be better than what we can do today. Resource Packages could be since it allows other things like images to be effectively bundled.<br /><br /><span style="font-size:180%;"><span style="font-weight: bold;">Summary</span></span><br /><br />I do not feel like Simple Modules are an improvement over what can be done today. In particular, I feel RequireJS when used alongside JSLint is a compelling existing solution, and it works well, and fast, in the browser.<br /><br />For the more immediate goals of Simple Modules:<br /><ul><li>the expanded, stricter lexical scoping is nice, but for a web developer, it is a slight incremental benefit if JSLint is already in use.</li><li>Not being able to set a function as the module value means the syntax is not a net win over the function-based module pattern.</li></ul>The larger issues of module addressing and bundling/distribution are understandably hazy in this early stage of the strawman proposals, but they will need to be addressed as well as or better than existing solutions to gain traction.<br /><br />I do not want to contribute stop energy around the proposals, I am just hoping to provide feedback to indicate what problems need to be solved better from my web developer viewpoint. I appreciate I could be wrong on some things too. I may be missing something grander or larger, but hopefully if that is the case, this feedback can indicate how to explain the proposals better.James Burkehttp://www.blogger.com/profile/00451746837849321739noreply@blogger.com3