Wednesday, November 16, 2005

Using On-Demand JavaScript for Everything

I really like using On-Demand JavaScript (dynamically creating a SCRIPT tag and attaching to the HEAD) for loading new data or UI, particularly since it gets around the domain restrictions of XMLHTTPRequest. There is also no need to deal with messy XML to JavaScript transformations.

The challenges:
  1. GET URL length restrictions.
  2. Data has to be JavaScript.
  3. Knowing when script load is complete.
Tagneto can help deal with these issues. The approach and issues are slightly different for retrieving UI (View) vs. Data (Model):

On Demand UI

1. GET URL length restrictions

This normally shouldn't be an issue for retrieving UI, particularly if the UI does not depend on particular request or user data (see Why document). If it is an issue, please see the approach mapped out below in On Demand Data.

2. Data has to be JavaScript

It is particularly tricky to encode HTML as JavaScript. Tagneto's ctrl: tags make this easier. An example, to create a JavaScript function that returns an HTML string based on some input parameters.

<ctrl:function name="getHelloWorld" parameters="name, planet">
<b>Hello &ctrl:out-value:planet;, this is <ctrl:out value="name/>.</b>
After running this file through Tagneto's view assembly tool, Sample.js will look something like this (the actual result is a little more complex and efficient than string concatenations::
function getHelloWorld(name, planet)
return '<b>Hello ' + planet + ', this is ' + name + .</b>';

Now Sample.js is ready to be dynamically included in the page.

3. Knowing when script load is complete

(Update 1/21/2006: the original post referred to a Srvr JS library. That library was renamed/changed and now is the Dsr library. This post was changed to reference the new library.)

The script can be dynamically added to the page using a method in Dsr.js:
Dsr.sendAndPoll('Sample.js', 'getHelloWorld', sampleLoadedListener);
where sampleLoadedListener is an object that could look like this:
var sampleLoadedListener = {
onLoad: function()
document.getElementById('outDiv').innerHTML = getHelloWorld();
onError: function(status, statusText, response)
alert('status: ' + status + ', error: ' + response);
onTimeout: function() { alert('timeout'); },
timeout: 30
sampleLoadedListener.onLoad will be called once getHelloWorld is detected as being defined.

On Demand Data

This section is bit more experimental. It is just a first thought, probably needs more work. I think I'll need to make changes to Dsr to have it play nice with this model, perhaps even provide convenience method to handle multi part URLs.

1. GET URL length restrictions

(UPDATE 1/21/2006: The API described below has been significantly changed. See the Dynamic Script Request API for the new version. The Dsr.js library now implements the DSR API.)

It seems like URLs can only be at maximum around 1KB, so this can be problematic if you need to send a large amount back to the server. What about using multiple requests (parts) to post the data back? The server would collect the parts, and on the final part, do the action. Some spec is needed for GET parameters on the part URLs:
[&actual data to give to server]
An example URL (broken into multiple lines for readability):


1.4 means this is part 1 of a 4 part request.

myDataLoaded is a JavaScript function already defined in the page, and it takes one parameter, the actual JavaScript response data from the server after the server finishes collecting all the parts and processing the requests.

is a JavaScript function already defined in the page that would take perhaps an error object/message as the only parameter.

is a JavaScript function already defined in the page that would take the part number that was just completed by the server. The server's response would call this method as the very last line in the response.

As the JavaScript gets notification of a successful part being processed
(via the partComplete function), it would attach the next part as a SCRIPT element to the page.

The method callback parameters only need to be sent up as part of the first part of the total request.

2. Data has to be JavaScript

Frameworks like DWR or JSON (in the Java world) may be able to help with dynamic data (based on request or user data). If it is static data, custom tags using Tagneto's org.tagnetic.core.tags.define.DefineInclude tag handler or the view:xmldatasource tag could be used to transform XML data into JavaScript.

3. Knowing when script load is complete

Completion is known by the calling of the success/partComplete/error functions. Srvr.Script should be changed to allow the checkString to be optional, and to provide a wrapper around this approach.


Animal said...

I used this technique before I discovered XMLHttpRequest.

I still have my servlets generate the correct JavaScript to do all the DOM manipulation rather that send some form of encoded parameters back which then have to be postprocessed.

On the issue of knowing when the script is loaded, I used a technique where the function sending the request added a "?requestID=nnnn" parameter to the end of the URL, and created a response handler function in responseHandler[nnnn].

A call to that function is added to the output stream at the end. Servlets which generate JavaScript use a special JavascriptWriter as a wrapper for the servlet OutputStream which handles this.

James said...

The approach you used before is what I ended up using for the spec I made here:

As I'm sure you noticed, the big trick is getting reliable notification about which request just got loaded, so having a defined callback and an ID for the request is important. In the DSR API described above, I was hoping that be defining a well-known callback, onscriptload, and defining a standard set of properties to receive in the callback (like an id and status), it would make it easier for people to use other people's APIs.

It also sounds like the DSR JAR implementation uses a similar approach to what you used before: it defines a Servlet Filter that creates a specialized HttpServletResponseWrapper. The wrapper exposes a specialized Writer to capture the output from other servlets.

paulo said...

I have used this kind of technique in a simple (but efficient) "chat" before discover the "XMLHttpRequest" and it works really fine.

paulo said...

I have used this kind of technique in a simple (but efficient) "chat" before discover the "XMLHttpRequest" and it works really fine.

Shay said...

You are my hero !
I've been working around this javascript issue for ages... didnt know there is a GET URL length restrictions.
Thank you very much for giving me the knowledge to solve this problem, our wedding planner site (hebrew encoded) can now run again without errors.

תכנון חתונה