Microsoft was kind enough to look into the issue, and here is more information.
The "Navigate sub-frames across different domains" preference is designed to address this security report:
http://secunia.com/advisories/11978/
IE 7 has shipped with automatically turning this preference on.
There is a workaround to the issue: the two iframes that do the communication must both have a parent frame from their domain loaded.
So, the original setup I use in the XHR IFrame Proxy situation looks like the following. The dashes indicate the level of nestedness (Top Window contains IFrame 1, IFrame 1 contains IFrame 2):
Top window (Domain A)
--IFrame 1 (Domain A)
----IFrame 2 (Domain B)
Where IFrame 1 and IFrame 2 communicate by setting each other's fragment identifiers (location.hash in JavaScript). When the communication is done, IFrame 1 tells the top window what the result is.
What broke was IFrame 2 (on Domain B) setting IFrame 1's location (via parent.location).
The following iframe structure works in IE7:
Top window (Domain A)
-- IFrame W (Domain B)
---- IFrame 1 (Domain A)
------ IFrame 2 (Domain B)
Notice the introduction of IFrame W. Now, apparently since IFrame W (on Domain B) owns IFrame 1, it is OK for IFrame 2 to change the location of IFrame 1.
Here is a test of this structure (thanks in large part of Microsoft).
This test uses something I personally have not done before: using window.open() to grab named iframes. Interesting technique. I'm curious to see if it is required when I patch IFrame XHR for IE 7.
So, the end result is that *another* IFrame is needed to accomplish the task. I'll be looking at modifying XHR IFrame Proxy code to use this extra frame, but only for IE 7. The down side is that now, in addition to the remote server placing xip_server.html on their server, they will have to place another HTML file too (the file for IFrame W). If the code bloat is not too much, I might consider making xip_server.html smart enough to act as IFrame W content as well as being the content for IFrame 2.
So, while it works, I still ask Microsoft to consider the following:
1) It seems like this change to address the security issue (http://secunia.com/advisories/11978/) seems like it should not affect setting parent/child frames. Firefox seems to have addressed the security issue without breaking subframe communication. I know that is not entirely a fair comparison, but it seems like the code change in IE to protect against a specific issue has too broad of an effect.
2) Even with this preference on, allow changing of fragment identifiers. It is a non-destructive change to the parent frame, and the parent frame can choose to ignore it. It is also a very solid, useful tool for cross-domain communication that works for today's browsers.
If you want to get fancy, consider supporting a security setting on the iframe, that allows Domain A to say it trusts content from Domain B to change its location. Ideally this sort of configuration of an iframe would be some sort of standardized access control scheme that other browser makers could implement if they so desired.
Tuesday, October 24, 2006
Wednesday, October 11, 2006
IE 7 Breaks IFrame APIs that use parent.location
Update (Nov. 7, 2006): Microsoft provided more information. See Part 2.
Original post (Oct. 11, 2006):
One of my co-workers was recently testing IE 7 RC1 and found that using fragment identifier messaging (FIM) with IFrames did not work. When the content in the nested IFrame tried to set parent.location, IE 7 popped a new window instead. If you have an IFrame-based API that uses parent.location, then you might be affected by this change.
I would like Microsoft to reconsider the preference change in IE 7 that causes this issue, and this post describes why. Feel free to contact Microsoft if you feel likewise. More information below.
When I investigated the issue more, I found that IE 7 only popped a new window if there was a nested IFrame. If there was just an IFrame in the top level document/browser window, it did not pop a window.
More information on why I think IE 7 should keep the preference as "Enabled":
No public documentation of discussion
Our Microsoft contact did not provide any public info indicating this was documented or discussed publicly. When I asked for a description of the change, there was no documentation that was ready for public consumption yet. There might have been some public disclosure about it, but I am not aware of any. I apologize if there was something posted. I have been subscribed to the IEBlog for a while now, but none of the previous posts triggered any warning bells for me. But I could very well have missed it.
This page describes that security preference, but reading does not seem to be very helpful (and seems to discuss talking to sub-frames and not parent frames).
I do not mean to chastise Microsoft on this point. I can appreciate with such a large project as a browser, it might not always be evident what changes can have an adverse effect on people using your product. And even if they did note it somewhere, there are lots of changes, and it could very easily get lost. It has happened to me on my software projects.
I can also see that their answer could be "why don't you just test with the IE 7 betas and release candidates? If there is an issue, let us know". That is a fair and valid position to take. But just as it might be hard to document and message all the changes for a large product like IE 7, it can be hard for web developers to stay on top of everything. I can see many not taking IE 7 testing seriously until RC 1.
And given that the parent.location works as long as you are communicating with a browser window and not an IFrame, I can see for some developers using IFrame APIs, there might not appear to be an issue.
For my case, with the XHR IFrame Proxy, I just worked out that solution July 31st. I haven't done IE 7 testing because I have been busy, and I thought the big things for that release were CSS-related changes (from a web developer POV). There did not seem to be that many JavaScript changes (besides an apparent increase in the JS engine performance). But still, I feel bad getting around to this issue this late in the IE 7 cycle.
So, ideally both of us could have caught the issue sooner and talked about it, but it is understandable that it did not happen. And I hope that since it is a preference change, that changing the IE 7 behavior back to "Enable" is hopefully an easy thing to do even this late in the IE 7 cycle. This would give more time to discuss the issue.
Behavior is buggy and could use refinement
I am sure there is a good use case to justify having this preference, and perhaps it even describes the current behavior. However, from my outsider's point of view, it seems buggy.
Why is Test Case 1 allowed (setting parent.location on the top browser window), but setting the parent.location on an IFrame is not allowed? They both seem to be "cross frame" communication, and violate whatever security issue the preference tries to address.
In addition to addressing the apparent bug with the behavior, I think setting parent.location should be allowed if the new value for parent.location is the same protocol, domain, port and path, but just differ by fragment identifier. In those cases, the parent document is not destroyed or modified, so perhaps whatever security issue is being addressed with this preference does not apply? Allowing fragment identifier changes is important for the next point...
FIM is the best thing we have for allowing pure browser, cross-domain web APIs
There are other ways to do cross-domain communication in a browser, but they all suffer from some limitation that does not make them a good general purpose, pure browser solution.
I also believe Fragement Identifier Messaging (FIM) that is used as part of the XHR IFrame Proxy is the most secure cross-domain communication option, since it requires explicit action for a server to allow the cross-domain action, and it has very fine grain control on the types
of requests that are allowed. FIM also does not automatically "execute" the response as using dynamic script tags might. The receiving frame can choose to just ignore messages if it wants.
And with the exception of the current behavior in IE 7 RC 1, FIM works in all other (major) browsers: IE 6, Firefox 1.5 and 2.0, Safari 2.0, and Opera 9.
The real solution would be a native XMLHttpRequest that can do cross-domain requests, but even when that becomes available, it will take a couple of years at the least to be widespread enough that it is actually be usable by web developers for cross-browser solutions.
In Conclusion
Microsoft, please do not change the preference to "Disable" with IE 7. Keep the default "Enable" behavior of IE 6. If at some point you want to change the preference to "Disable" by default, consider a public discourse of the implementation issues discussed here and the impact on FIM.
Original post (Oct. 11, 2006):
One of my co-workers was recently testing IE 7 RC1 and found that using fragment identifier messaging (FIM) with IFrames did not work. When the content in the nested IFrame tried to set parent.location, IE 7 popped a new window instead. If you have an IFrame-based API that uses parent.location, then you might be affected by this change.
I would like Microsoft to reconsider the preference change in IE 7 that causes this issue, and this post describes why. Feel free to contact Microsoft if you feel likewise. More information below.
When I investigated the issue more, I found that IE 7 only popped a new window if there was a nested IFrame. If there was just an IFrame in the top level document/browser window, it did not pop a window.
- Test Case 1 (OK: Top browser window -> IFrame)
- Test Case 2 (FAIL: Top browser window -> IFrame -> IFrame)
- There is a preference that was introduced in Windows XP SP2 that controls this behavior. You can find it by looking in the Internet Settings -> Security Tab -> Click on Custom Level button -> "Navigate sub-frames across different domains".
- With IE 6, this is set to "Enable" by default.
- When IE 7 installs, IE 7 automatically sets the preference to "Disable" by default.
- There is no documentation about the change, and the change was done without checking with the web developer community.
- I think the behavior is buggy and could use refinement.
- Fragment identifier messaging (FIM) across IFrames is the best thing we have for allowing pure browser, cross-domain web APIs.
More information on why I think IE 7 should keep the preference as "Enabled":
No public documentation of discussion
Our Microsoft contact did not provide any public info indicating this was documented or discussed publicly. When I asked for a description of the change, there was no documentation that was ready for public consumption yet. There might have been some public disclosure about it, but I am not aware of any. I apologize if there was something posted. I have been subscribed to the IEBlog for a while now, but none of the previous posts triggered any warning bells for me. But I could very well have missed it.
This page describes that security preference, but reading does not seem to be very helpful (and seems to discuss talking to sub-frames and not parent frames).
I do not mean to chastise Microsoft on this point. I can appreciate with such a large project as a browser, it might not always be evident what changes can have an adverse effect on people using your product. And even if they did note it somewhere, there are lots of changes, and it could very easily get lost. It has happened to me on my software projects.
I can also see that their answer could be "why don't you just test with the IE 7 betas and release candidates? If there is an issue, let us know". That is a fair and valid position to take. But just as it might be hard to document and message all the changes for a large product like IE 7, it can be hard for web developers to stay on top of everything. I can see many not taking IE 7 testing seriously until RC 1.
And given that the parent.location works as long as you are communicating with a browser window and not an IFrame, I can see for some developers using IFrame APIs, there might not appear to be an issue.
For my case, with the XHR IFrame Proxy, I just worked out that solution July 31st. I haven't done IE 7 testing because I have been busy, and I thought the big things for that release were CSS-related changes (from a web developer POV). There did not seem to be that many JavaScript changes (besides an apparent increase in the JS engine performance). But still, I feel bad getting around to this issue this late in the IE 7 cycle.
So, ideally both of us could have caught the issue sooner and talked about it, but it is understandable that it did not happen. And I hope that since it is a preference change, that changing the IE 7 behavior back to "Enable" is hopefully an easy thing to do even this late in the IE 7 cycle. This would give more time to discuss the issue.
Behavior is buggy and could use refinement
I am sure there is a good use case to justify having this preference, and perhaps it even describes the current behavior. However, from my outsider's point of view, it seems buggy.
Why is Test Case 1 allowed (setting parent.location on the top browser window), but setting the parent.location on an IFrame is not allowed? They both seem to be "cross frame" communication, and violate whatever security issue the preference tries to address.
In addition to addressing the apparent bug with the behavior, I think setting parent.location should be allowed if the new value for parent.location is the same protocol, domain, port and path, but just differ by fragment identifier. In those cases, the parent document is not destroyed or modified, so perhaps whatever security issue is being addressed with this preference does not apply? Allowing fragment identifier changes is important for the next point...
FIM is the best thing we have for allowing pure browser, cross-domain web APIs
There are other ways to do cross-domain communication in a browser, but they all suffer from some limitation that does not make them a good general purpose, pure browser solution.
I also believe Fragement Identifier Messaging (FIM) that is used as part of the XHR IFrame Proxy is the most secure cross-domain communication option, since it requires explicit action for a server to allow the cross-domain action, and it has very fine grain control on the types
of requests that are allowed. FIM also does not automatically "execute" the response as using dynamic script tags might. The receiving frame can choose to just ignore messages if it wants.
And with the exception of the current behavior in IE 7 RC 1, FIM works in all other (major) browsers: IE 6, Firefox 1.5 and 2.0, Safari 2.0, and Opera 9.
The real solution would be a native XMLHttpRequest that can do cross-domain requests, but even when that becomes available, it will take a couple of years at the least to be widespread enough that it is actually be usable by web developers for cross-browser solutions.
In Conclusion
Microsoft, please do not change the preference to "Disable" with IE 7. Keep the default "Enable" behavior of IE 6. If at some point you want to change the preference to "Disable" by default, consider a public discourse of the implementation issues discussed here and the impact on FIM.
Wednesday, August 30, 2006
How I want to build web applications
Use a web server and browser technology (HTML, JavaScript, CSS, etc...) to build the application. Do not use an application server, or any kind of server that is more than a simple web server. Ideally, I could serve the application from the local file system and it would work.
Data service APIs should be usable by JavaScript running in the browser. The data services can be implemented in any choice of server, but the API should not indicate the server implementation. For instance, if I'm using a getItems service, it should not have getItems.jsp/getItems.php/getItems.aspx as part of the API URL.
The web application should be runnable from any domain on any web server. This means the data services must support some sort of cross-domain access from a web browser. My preferences:
I believe this approach gives the best overall value with scalability and perceived performance when compared to server cost. Scalable because the code can live on any web server, and well-performing because the entire application can be heavily cached by the browser. There is a first time load cost to fetch the files, but after that, the files should have very long cache times. This gives great perceived performance to the user -- things start to visibly happen in the browser window as soon as the user goes to the application URL.
There are solutions that might give you better scalability or better performance, but this approach gives the best overall value when considering server cost. Only simple web servers are needed, and with the high cacheability of the application, the web servers will not need to do work for every invocation of the application. The user's computer and browser are doing more work, but since many (most?) of the applications and services are free, or nearly free, I believe this is an acceptable tradeoff. Of course there are exceptions to this rule, in particular cell phone web applications. However, I do desktop/laptop web application development at the moment.
A very important point about this development approach -- *anyone* can run the web application. Every ISP gives you some amount of disk space on their servers to serve files. This development approach allows you to use that ISP space to run a web application. And I believe that is a key driver to allowing widespread "mashup" usage. Anyone with a text editor, access to the internet, access to cross-domain data service APIs, and a web browser can make a web application usable by the whole world. I am not talking about just serving some web pages, I mean serving web applications: email, instant messaging, calendar, a map service mashup. That is beautiful.
Tools I want to use
A web server
Preferably Apache, but any solid web server that allows for setting cache controls on files will do.
A solid JavaScript toolkit
I prefer Dojo, because I contribute to it, and it offers a nice breadth of libraries with an include system, dojo.require(), that allow you to do progressive optimizations on the code you use without having to recode a bunch of your JavaScript. Once the accessibility and internationalization support is fully integrated, it will be a hard toolkit to beat.
A non-server, i18n-friendly templating system
JSP, ASP, PHP and Rails enable some nice templating systems, but they all require server infrastructure. Tagneto is my attempt at a templating system that does not need server infrastructure. It requires a compile-time step by the developer, but the output is just plain HTML/JavaScript/CSS runnable from any web server (even local disk).
Dojo has some support for a type of templating via widgets and the i18n work going in now, but I don't feel it is as performant as something that you can do with a compile step. Most of the templating work (and in particular i18n string work) could be done once in a compile time step instead of downloading all the code to do the template transformation and doing the transformation every time the code is loaded.
The downside with a compile step, is that the developer needs to run a bunch of compiles while developing. One thing I would like to do is enable development using Dojo for the templating needs, but then apply Tagneto during a build process. Or expand Dojo to do the work as part of its build process. Or just use Tagneto, but allow using it from something like Jetty on the local computer, and have that do the transform on the fly. Once the developer is finished, run the compile step as part of the build to generate the final static files that can be served from a web server.
But for me, I'm fine with doing the compile step as part of the development process. At least for now, until I commit more time to solving the problem. I want to help get Dojo 0.4 out the door first.
Data service APIs should be usable by JavaScript running in the browser. The data services can be implemented in any choice of server, but the API should not indicate the server implementation. For instance, if I'm using a getItems service, it should not have getItems.jsp/getItems.php/getItems.aspx as part of the API URL.
The web application should be runnable from any domain on any web server. This means the data services must support some sort of cross-domain access from a web browser. My preferences:
- Cross Domain XMLHttpRequest using IFrame Proxies (IFP XHR). The data service should offer returning JSON or XML. Ideally JSON but XML will work too.
- Using SCRIPT tags to fetch JSON data (ScriptSrcIO).
I believe this approach gives the best overall value with scalability and perceived performance when compared to server cost. Scalable because the code can live on any web server, and well-performing because the entire application can be heavily cached by the browser. There is a first time load cost to fetch the files, but after that, the files should have very long cache times. This gives great perceived performance to the user -- things start to visibly happen in the browser window as soon as the user goes to the application URL.
There are solutions that might give you better scalability or better performance, but this approach gives the best overall value when considering server cost. Only simple web servers are needed, and with the high cacheability of the application, the web servers will not need to do work for every invocation of the application. The user's computer and browser are doing more work, but since many (most?) of the applications and services are free, or nearly free, I believe this is an acceptable tradeoff. Of course there are exceptions to this rule, in particular cell phone web applications. However, I do desktop/laptop web application development at the moment.
A very important point about this development approach -- *anyone* can run the web application. Every ISP gives you some amount of disk space on their servers to serve files. This development approach allows you to use that ISP space to run a web application. And I believe that is a key driver to allowing widespread "mashup" usage. Anyone with a text editor, access to the internet, access to cross-domain data service APIs, and a web browser can make a web application usable by the whole world. I am not talking about just serving some web pages, I mean serving web applications: email, instant messaging, calendar, a map service mashup. That is beautiful.
Tools I want to use
A web server
Preferably Apache, but any solid web server that allows for setting cache controls on files will do.
A solid JavaScript toolkit
I prefer Dojo, because I contribute to it, and it offers a nice breadth of libraries with an include system, dojo.require(), that allow you to do progressive optimizations on the code you use without having to recode a bunch of your JavaScript. Once the accessibility and internationalization support is fully integrated, it will be a hard toolkit to beat.
A non-server, i18n-friendly templating system
JSP, ASP, PHP and Rails enable some nice templating systems, but they all require server infrastructure. Tagneto is my attempt at a templating system that does not need server infrastructure. It requires a compile-time step by the developer, but the output is just plain HTML/JavaScript/CSS runnable from any web server (even local disk).
Dojo has some support for a type of templating via widgets and the i18n work going in now, but I don't feel it is as performant as something that you can do with a compile step. Most of the templating work (and in particular i18n string work) could be done once in a compile time step instead of downloading all the code to do the template transformation and doing the transformation every time the code is loaded.
The downside with a compile step, is that the developer needs to run a bunch of compiles while developing. One thing I would like to do is enable development using Dojo for the templating needs, but then apply Tagneto during a build process. Or expand Dojo to do the work as part of its build process. Or just use Tagneto, but allow using it from something like Jetty on the local computer, and have that do the transform on the fly. Once the developer is finished, run the compile step as part of the build to generate the final static files that can be served from a web server.
But for me, I'm fine with doing the compile step as part of the development process. At least for now, until I commit more time to solving the problem. I want to help get Dojo 0.4 out the door first.
Tuesday, June 06, 2006
Cross Domain Frame Communication with Fragment Identifiers (for Comet?)
A page is served from a different domain than the URL for an iframe in that page. Normally cross domain, cross frame communication is prohibited for security reasons. However, the two frames can communicate with each other by using fragment identifiers (the hash part of an URL, like http://some.domain.com/path/to/page.html#fragmentIdentifier).
Since fragment identifier changes don't reload the page, state can be maintained in each of the frames.
This could be used to allow cross domain usage of an API that uses Comet as its communication with the server. Or for UI that a third party wants to embed it, but still allow some stateful communication with the hosting page.
The limitations:
Since fragment identifier changes don't reload the page, state can be maintained in each of the frames.
This could be used to allow cross domain usage of an API that uses Comet as its communication with the server. Or for UI that a third party wants to embed it, but still allow some stateful communication with the hosting page.
The limitations:
- Communication is limited to the size of fragment identifiers. I'm not sure on the max size for all browsers, but I would think the same limitation on the size of GET URLs probably hold here too. So the max size for the full URL should probably be kept under around 1KB.
- Using the iframe may cause issues with the back button.
Monday, May 22, 2006
dojo.io.ScriptSrcIO and Dojo 0.2.2
dojo.io.ScriptSrcIO allows you to use dojo.io.bind() but use script tags instead, effectively giving you cross-domain data access (but the data returned as to be JavaScript).
It was introduced for Dojo 0.3, but here is a version of that IO transport that works with Dojo 0.2.2:
http://tagneto.org/dojo/0.2.2compat/ScriptSrcIO.js
In order for it to work with 0.2.2. the back button integration was commented out. Also, since this is not a package that is included with the Dojo 0.2.2 release, you can either place it in the src/io directory of your Dojo install, or if you don't have access to the dojo install, just do a script src tag for this file, right after the dojo.js script tag:
<script src="/path/to/your/dojo/dojo.js" type="text/javascript"></script>
<script src="/path/to/your/copy/of/ScriptSrcIO.js" type="text/javascript"></script>
Then you can use this transport as described in the test page:
http://archive.dojotoolkit.org/nightly/tests/io/test_ScriptSrcIO.html
When you upgrade to Dojo 0.3, be sure to remove the extra script tag if you have it (since ScriptSrcIO.js ships with 0.3).
It was introduced for Dojo 0.3, but here is a version of that IO transport that works with Dojo 0.2.2:
http://tagneto.org/dojo/0.2.2compat/ScriptSrcIO.js
In order for it to work with 0.2.2. the back button integration was commented out. Also, since this is not a package that is included with the Dojo 0.2.2 release, you can either place it in the src/io directory of your Dojo install, or if you don't have access to the dojo install, just do a script src tag for this file, right after the dojo.js script tag:
<script src="/path/to/your/dojo/dojo.js" type="text/javascript"></script>
<script src="/path/to/your/copy/of/ScriptSrcIO.js" type="text/javascript"></script>
Then you can use this transport as described in the test page:
http://archive.dojotoolkit.org/nightly/tests/io/test_ScriptSrcIO.html
When you upgrade to Dojo 0.3, be sure to remove the extra script tag if you have it (since ScriptSrcIO.js ships with 0.3).
Wednesday, March 29, 2006
JSONRequest, Part 2 (Cross Domain Policy for All)
As mentioned in part 1, instead of introducing a new request object, JSONRequest, I would prefer to fix the current available pathways. However, I can appreciate that it may be hard to do that. If JSONRequest does go forward, here are some suggestions. The goal of these suggestions are to get JSONRequest to behave in a safe, useful way that could be ported to the other pathways, like script src or XMLHTTPRequest, if it all seemed to work out (and maybe one day merge JSONRequest with XMLHTTPRequest into a generic request object).
The main goal of these suggestions are to protect legacy server systems that did not expect cross domain requests. These suggestions require a server to do explicit actions to allow cross domain behavior. This should address most of the security concerns while laying the groundwork for a general cross-domain request model for the browser.
These suggestions are only for cross-domain requests made with JSONRequest. XMLHTTPRequest's same origin behavior can be used for JSONRequests to the same origin server.
I do not have any particular attachment to the names of the attributes and headers used below. They are just used for illustration of the concept.
The suggestions:
To alleviate the concerns about data stealing and impacts on legacy systems, introduce a new attribute for cookies, xdallow. New server services could use this new attribute to indicate that it wants to allow cookies to be sent in JSONRequests that originate from a page that is outside of the server's domain. Like today's cookies, the cross domain page cannot actually see the value of these cookies, but if it makes a JSONRequest to a server that set these cookies, they would be sent (as long as the request conformed to the policy set by this new cookie attribute):
Using an example from the cookie spec, current cookies can look like this:
The spec uses a Domain header to specify the server that is making the request, and the value is derived from document.domain. I would prefer not to rely on document.domain, since it is a script property that can be changed to a limited degree.
I would prefer to use the page's real domain and include the path of the page making the request. Effectively like the Referer header. I would like to just use the Referer header, but I can see for cross-domain requests only, a new header name should be used. In particular, if these changes were ever merged with the XMLHTTPRequest capability, a header that cannot be modified by a setHeader command should be used. So, perhaps use a new header name like:
Cross-Domain Policy for Pages
This is a mechanism for checking if a web page wants to allow cross domain requests via JSONRequest. This is to mitigate the case when the web page is compromised by malicious script. If JSONRequest is asked to make a cross-domain request, it first checks with the server that the page originates from for a xdallow policy file. This policy file could be similar to the one mentioned in the cross-domain cookies section.
If the example page is at the following URL:
http://some.domain.com/path/to/web/page.html
JSONRequest would check the following paths in this order:
The spec does not allow for caching of responses or GET requests. If the cookies are not allowed for JSONRequest, I believe the majority use case for JSONRequest will be for non-authenticated APIs, and a majority of those cases could benefit from cached responses. I don't believe POST requests can be effectively cached by browsers or proxy caches, so it is important in that respect to allow GET requests. Respecting cache headers would also be required of JSONRequest.
If all of the above recommendations are used, it helps protect GET requests. Legacy services would still be protected since they don't support JSON responses. As servers roll out support for JSONRequest, they will be aware of the above requirements to allow cross-domain usage.
In general, these additions force newer services to do explicit actions to allow the cross-domain requests and the additions give those services the information to make intelligent, secure decisions. Legacy systems are protected if they don't opt-in to these requirements. They would have very strong protection by just disallowing any request with the Request-Origin header.
The main goal of these suggestions are to protect legacy server systems that did not expect cross domain requests. These suggestions require a server to do explicit actions to allow cross domain behavior. This should address most of the security concerns while laying the groundwork for a general cross-domain request model for the browser.
These suggestions are only for cross-domain requests made with JSONRequest. XMLHTTPRequest's same origin behavior can be used for JSONRequests to the same origin server.
I do not have any particular attachment to the names of the attributes and headers used below. They are just used for illustration of the concept.
The suggestions:
- Allow Cookies with a xdallow Attribute
- Request Origin Header
- Cross-Domain Policy for Pages
- Allow GET requests to Allow Caching
To alleviate the concerns about data stealing and impacts on legacy systems, introduce a new attribute for cookies, xdallow. New server services could use this new attribute to indicate that it wants to allow cookies to be sent in JSONRequests that originate from a page that is outside of the server's domain. Like today's cookies, the cross domain page cannot actually see the value of these cookies, but if it makes a JSONRequest to a server that set these cookies, they would be sent (as long as the request conformed to the policy set by this new cookie attribute):
Using an example from the cookie spec, current cookies can look like this:
Set-Cookie: CUSTOMER=WILE_E_COYOTE; path=/; domain=anvil.acme.com; expires=Wednesday, 09-Nov-99 23:12:40 GMTThe new attribute would look like this:
Set-Cookie: CUSTOMER=WILE_E_COYOTE; xdallow=XDALLOWVALUE; path=/; domain=anvil.acme.com; expires=Wednesday, 09-Nov-99 23:12:40 GMT
where XDALLOWVALUE could be:- xdallow=all - all pages are allowed to use a request to the origin server with this cookie.
- xdallow=domain.one.com,domain.two.com - a list of comma-separated domains
- xdallow=http://a.domain.com/with/path.html - a full URL to allow. If the server wanted to allow multiple domains, send multiple Set-Cookie headers with different xdallow values.
- xdallow=/path/to/policy.xml - A path to the server's cross-domain cookie policy. It would be in XML format and could allow specifying a whitelist and black list of domains and paths that are allowed.
The spec uses a Domain header to specify the server that is making the request, and the value is derived from document.domain. I would prefer not to rely on document.domain, since it is a script property that can be changed to a limited degree.
I would prefer to use the page's real domain and include the path of the page making the request. Effectively like the Referer header. I would like to just use the Referer header, but I can see for cross-domain requests only, a new header name should be used. In particular, if these changes were ever merged with the XMLHTTPRequest capability, a header that cannot be modified by a setHeader command should be used. So, perhaps use a new header name like:
Request-Origin: http://some.domain.com/path/to/page/making/request.htmlThis header would even be sent for https requests. If this is problematic, then don't allow https pages to make cross domain requests.
Cross-Domain Policy for Pages
This is a mechanism for checking if a web page wants to allow cross domain requests via JSONRequest. This is to mitigate the case when the web page is compromised by malicious script. If JSONRequest is asked to make a cross-domain request, it first checks with the server that the page originates from for a xdallow policy file. This policy file could be similar to the one mentioned in the cross-domain cookies section.
If the example page is at the following URL:
http://some.domain.com/path/to/web/page.html
JSONRequest would check the following paths in this order:
- http://some.domain.com/path/to/web/.xdallow/page.html.xml
- http://some.domain.com/path/to/web/.xdallow/
- http://some.domain.com/.xdallow/
The spec does not allow for caching of responses or GET requests. If the cookies are not allowed for JSONRequest, I believe the majority use case for JSONRequest will be for non-authenticated APIs, and a majority of those cases could benefit from cached responses. I don't believe POST requests can be effectively cached by browsers or proxy caches, so it is important in that respect to allow GET requests. Respecting cache headers would also be required of JSONRequest.
If all of the above recommendations are used, it helps protect GET requests. Legacy services would still be protected since they don't support JSON responses. As servers roll out support for JSONRequest, they will be aware of the above requirements to allow cross-domain usage.
In general, these additions force newer services to do explicit actions to allow the cross-domain requests and the additions give those services the information to make intelligent, secure decisions. Legacy systems are protected if they don't opt-in to these requirements. They would have very strong protection by just disallowing any request with the Request-Origin header.
Tuesday, March 14, 2006
JSONRequest Thoughts
This is feedback related to JSONRequest.
Specific detail comments
I'm curious why it is designed as a static method invocation and request ID vs. XMLHTTPRequest's instance-based approach. I would prefer to match XHR's instance-based approach, unless there is a good reason to avoid it. It would be good to match the XHR idiom as much as possible, since developers are familiar with it.
I also like the idea of request limiting on failure count. It would probably be good even for scenarios when it is not an attack, but the server is just having problems. No need to pound the server relentlessly.
General Problem Comments
In general, it feels too limiting, and I'm not sure the limits do much in the end to make it a viable alternative to just dynamically adding SCRIPT tags to the HTML head to load cross-domain scripts and/or JS data. With the dynamically added SCRIPTS, you get cookies sent to the domain, and have the option to get more than just a JSON object.
It is dangerous, but the point is it is possible to do it today. The only thing missing is being able to do a POST and getting a standard callback to know the script has loaded. I tried solving those things with the Dynamic Script Request (DSR) API.
Even though it can be dangerous, developers can use it today, and if it solves their problem, then they will use it. If it is possible to get browser makers to do some work in the area of cross domain script usage, I would prefer to see that effort used to strengthen the existing method and to provide the ability to specify a page-level script policy concerning cross domain scripts.
Strengthen Existing Pathway
I would like to see the following things added to to script tags, and/or have a ScriptRequest object to encapsulate the behavior (I don't really care about the names of these attributes, just what capabilities they allow):
The HTTP request headers could not be modified by the script request object (except indirectly, in the case of the cookie header). In addition, I would require that a Referer HTTP header be sent with the request. There was a comment on the WHATWG mailing list that Referer is not sent for HTTPS requests. I think something needs to be sent to indicate the calling domain and page path in the HTTP headers so that the server handling the request can make a decision on whether to allow the request.
Page Level Cross Domain Script Policy
Cross domain XML policy files on the server (like the ones used by Flash) do not seem sufficient. As far as I know, those only allow setting domain-to-domain trusts, but there could be many types of web applications on a domain (one HTML page can be a whole application). Sites can host pages from many users.
It should be possible to specify a cross domain script policy in a web page. Something that would set the permissions for scripts that come from other domains than the current domain. It could even be used for any exterior scripts that are loaded, even ones on the same domain. I'm not too particular on that, but it might be a good idea.
The policy would be specified by an HTML element, a child of HEAD. I do not care what the tag is called, but for the purposes of illustration, I'll use a META tag. But again, I do not have any special feelings about using META:
<meta name="ExternalScriptPolicy" content="dom=yes,cookie=no,request=no">
Different script capabilities can be allowed/disallowed, like whether to allow:
Conclusion
JSONRequest has some nice ideas, but since there is a gaping hole with SCRIPT tags anyway, I would prefer to use the browser makers time to improve the SCRIPT pathway, make it better to identify request origins, and provide page-level script security policies.
Specific detail comments
I'm curious why it is designed as a static method invocation and request ID vs. XMLHTTPRequest's instance-based approach. I would prefer to match XHR's instance-based approach, unless there is a good reason to avoid it. It would be good to match the XHR idiom as much as possible, since developers are familiar with it.
I also like the idea of request limiting on failure count. It would probably be good even for scenarios when it is not an attack, but the server is just having problems. No need to pound the server relentlessly.
General Problem Comments
In general, it feels too limiting, and I'm not sure the limits do much in the end to make it a viable alternative to just dynamically adding SCRIPT tags to the HTML head to load cross-domain scripts and/or JS data. With the dynamically added SCRIPTS, you get cookies sent to the domain, and have the option to get more than just a JSON object.
It is dangerous, but the point is it is possible to do it today. The only thing missing is being able to do a POST and getting a standard callback to know the script has loaded. I tried solving those things with the Dynamic Script Request (DSR) API.
Even though it can be dangerous, developers can use it today, and if it solves their problem, then they will use it. If it is possible to get browser makers to do some work in the area of cross domain script usage, I would prefer to see that effort used to strengthen the existing method and to provide the ability to specify a page-level script policy concerning cross domain scripts.
Strengthen Existing Pathway
I would like to see the following things added to to script tags, and/or have a ScriptRequest object to encapsulate the behavior (I don't really care about the names of these attributes, just what capabilities they allow):
- method="POST"
- data="url encoded data sent as the POST body"
- sync="true" (to get blocking requests)
- onload="event callback that really worked and was called when the script was loaded"
- createScope="true" (or something that says "don't treat read the data and do the equivalent of a window.eval () on it. Instead, treat the data as specific scope or as instance object).
The HTTP request headers could not be modified by the script request object (except indirectly, in the case of the cookie header). In addition, I would require that a Referer HTTP header be sent with the request. There was a comment on the WHATWG mailing list that Referer is not sent for HTTPS requests. I think something needs to be sent to indicate the calling domain and page path in the HTTP headers so that the server handling the request can make a decision on whether to allow the request.
Page Level Cross Domain Script Policy
Cross domain XML policy files on the server (like the ones used by Flash) do not seem sufficient. As far as I know, those only allow setting domain-to-domain trusts, but there could be many types of web applications on a domain (one HTML page can be a whole application). Sites can host pages from many users.
It should be possible to specify a cross domain script policy in a web page. Something that would set the permissions for scripts that come from other domains than the current domain. It could even be used for any exterior scripts that are loaded, even ones on the same domain. I'm not too particular on that, but it might be a good idea.
The policy would be specified by an HTML element, a child of HEAD. I do not care what the tag is called, but for the purposes of illustration, I'll use a META tag. But again, I do not have any special feelings about using META:
<meta name="ExternalScriptPolicy" content="dom=yes,cookie=no,request=no">
Different script capabilities can be allowed/disallowed, like whether to allow:
- DOM access
- Cookie access (maybe break it out by read and write)
- Request access (whether the external script can use XMLHTTPRequest or ScriptRequest)
- Other restrictions/privileges?
Conclusion
JSONRequest has some nice ideas, but since there is a gaping hole with SCRIPT tags anyway, I would prefer to use the browser makers time to improve the SCRIPT pathway, make it better to identify request origins, and provide page-level script security policies.
Wednesday, January 25, 2006
Clarifying Tagneto's mission
I think the description of Tagneto in the Why and What documents may be a little vague or too generic. I'm trying to clarify the mission and the message of the mission. Here goes:
Provide tools to build fast, rich web applications that don't require application server infrastructure.
Creating rich user interfaces in HTML should not require a server infrastructure. A web server is needed to serve the files and to distribute updates, but server infrastructure like an application server (whether it is in Java, Python, Ruby, Perl, PHP, .NET, whatever) should not be needed. Server infrastructure is needed for data APIs, but for the user interface, it should not be required.
For some larger applications, it is desirable to have the ability to create the user interface from component HTML pieces, and assemble them together in the page. Normally, an application server technology like JSP, ASP, PHP would be used to accomplish this task. However, since the desire is to avoid application server infrastructure, a View Assembly tool is needed that would provide that capability. Tagneto's View Assembly tool does just that, and it uses tools the UI developer already needs to know, JavaScript, XML, HTML and Entities.
The View Assembly tool also implements advanced features such as
Avoiding application server infrastructure opens up the possibility to deliver the user interface from anywhere, a specialized web farm like a Content Delivery Network (CDN), or even the local disk.
However, to successfully access the data APIs, cross-domain access to the APIs via JavaScript is required. Something that is not possible with XMLHTTPRequest. Ideally, future advances of the XMLHTTPRequest object will provide that functionality, but for the browsers of today, The Dynamic Script Request API is an attempt to solve that issue.
All that said, application server infrastructure is a critical component of delivering a complete application. Hopefully it is just used for the data APIs, but the View Assembly tools should play nice with those technologies if they also need to be used to generate UI. Tagneto's View Assembly tool does play nice with the technologies, since it can be run before deploying the pages to the application server, and it is very flexible in choosing which tags to match and process.
The main goal of Tagneto is not necessarily to provide a "JavaScript library framework". Some JavaScript libraries are available with Tagneto, but in the service of the mission above. In fact, recently I have been considering providing View Assembly tool integration with some of the more well-known JavaScript frameworks, like Dojo or Prototype, with Dojo integration probably first.
Perhaps for Dojo, "View Assembly tool integration" means:
I'll see how it goes, but right now I like this clarification. I'll gradually work on updating the site docs to reflect it. Feedback or suggestions are welcome.
Provide tools to build fast, rich web applications that don't require application server infrastructure.
Creating rich user interfaces in HTML should not require a server infrastructure. A web server is needed to serve the files and to distribute updates, but server infrastructure like an application server (whether it is in Java, Python, Ruby, Perl, PHP, .NET, whatever) should not be needed. Server infrastructure is needed for data APIs, but for the user interface, it should not be required.
For some larger applications, it is desirable to have the ability to create the user interface from component HTML pieces, and assemble them together in the page. Normally, an application server technology like JSP, ASP, PHP would be used to accomplish this task. However, since the desire is to avoid application server infrastructure, a View Assembly tool is needed that would provide that capability. Tagneto's View Assembly tool does just that, and it uses tools the UI developer already needs to know, JavaScript, XML, HTML and Entities.
The View Assembly tool also implements advanced features such as
- "Aspected Oriented HTML" via overlays, allowing the matching and processing of tags across pages, and allowing processing before, after, as the first child or as the last child of the matching tag.
- Chained tag handlers, for combining many small, focused tags together to accomplish something big.
- The architecture is also designed to allow other scripting languages (Python, Ruby, Perl) to be used instead of JavaScript for the tag scripting (but right now only JavaScript is implemented).
- The source does not have to completely valid XML.
- Internationalization/Localization support.
Avoiding application server infrastructure opens up the possibility to deliver the user interface from anywhere, a specialized web farm like a Content Delivery Network (CDN), or even the local disk.
However, to successfully access the data APIs, cross-domain access to the APIs via JavaScript is required. Something that is not possible with XMLHTTPRequest. Ideally, future advances of the XMLHTTPRequest object will provide that functionality, but for the browsers of today, The Dynamic Script Request API is an attempt to solve that issue.
All that said, application server infrastructure is a critical component of delivering a complete application. Hopefully it is just used for the data APIs, but the View Assembly tools should play nice with those technologies if they also need to be used to generate UI. Tagneto's View Assembly tool does play nice with the technologies, since it can be run before deploying the pages to the application server, and it is very flexible in choosing which tags to match and process.
The main goal of Tagneto is not necessarily to provide a "JavaScript library framework". Some JavaScript libraries are available with Tagneto, but in the service of the mission above. In fact, recently I have been considering providing View Assembly tool integration with some of the more well-known JavaScript frameworks, like Dojo or Prototype, with Dojo integration probably first.
Perhaps for Dojo, "View Assembly tool integration" means:
- Using Dojo's string building utilties to build the JS-escaped strings for the ctrl tags. Right now the Array.append/join is used.
- Supporting Dojo's connect() methods for event binding via the ctrl:listen tag.
- Possibly do some compile-time "preprocessing" on Dojo widget tags to convert them to the HTML/JavaScript that will actually be used at runtime. This would avoid having to scan the HTML at runtime to find and bind the widgets.
- Integrating the DSR with dojo.io.bind.
I'll see how it goes, but right now I like this clarification. I'll gradually work on updating the site docs to reflect it. Feedback or suggestions are welcome.
Tuesday, January 17, 2006
Using DSR to allow On-Demand JavaScript for RSS/ATOM feeds
It would be great to see data services provide a JavaScript version of their APIs to allow web developers that don't have (or want) server resources to implement data mashups. JavaScript APIs that can be used via HTML script elements allow for cross-domain data access without server intermediaries.
With the release of Tagneto 0.4.0 there is an update to the Dynamic Script Request (DSR) API, an API that makes On-Demand JavaScript via HTML script elements reliable and easy to use.
The API now uses a reserved callback function, window.onscriptload, and an event object passed to that method to communicate the success/error codes and the data associated with the response. Other browser events and the XMLHTTPRequest object were used to guide the names of the event properties.
These changes allow for data services that provide RSS/ATOM feeds (or any type of XML data feed) to provide a JavaScript response that is equivalent to the normal XML response without too much work (basically encode the XML as a JavaScript string, and put it in an event object that is passed to window.onscriptload).
The API document has more detail on how a RSS/ATOM feed might be provided as JavaScript in the Possible Applications section.
Tagneto 0.4.0 provides a Dsr.js helper script that implements the DSR API for use in a web page, and there is a DSR jar file that can be used in Java Servlet applications to send DSR responses.
Tagneto's home page uses the DSR JavaScript library and the DSR jar for the "Recent Blog Entries" section as a demonstration of the API and the implementation.
With the release of Tagneto 0.4.0 there is an update to the Dynamic Script Request (DSR) API, an API that makes On-Demand JavaScript via HTML script elements reliable and easy to use.
The API now uses a reserved callback function, window.onscriptload, and an event object passed to that method to communicate the success/error codes and the data associated with the response. Other browser events and the XMLHTTPRequest object were used to guide the names of the event properties.
These changes allow for data services that provide RSS/ATOM feeds (or any type of XML data feed) to provide a JavaScript response that is equivalent to the normal XML response without too much work (basically encode the XML as a JavaScript string, and put it in an event object that is passed to window.onscriptload).
The API document has more detail on how a RSS/ATOM feed might be provided as JavaScript in the Possible Applications section.
Tagneto 0.4.0 provides a Dsr.js helper script that implements the DSR API for use in a web page, and there is a DSR jar file that can be used in Java Servlet applications to send DSR responses.
Tagneto's home page uses the DSR JavaScript library and the DSR jar for the "Recent Blog Entries" section as a demonstration of the API and the implementation.
Monday, January 16, 2006
Tagneto 0.4.0 Released
The focus on 0.4.0 was on improving and simplifying the Dynamic
Script Request (DSR) API. From the Release Notes:
Script Request (DSR) API. From the Release Notes:
- Significant changes to the DSR API. Decided to go with a reserved global callback handler, onscriptload, to simplify things. Other browser events and the XMLHttpRequest object were used to guide the format and names of the methods and parameters. The result is a much simpler API. A sample usage of the API for RSS and ATOM feeds is described.
- The Tagneto distribution includes a server/dsr directory. This directory contains a DSR jar that helps servers implement the DSR API for use in web applications that use Java Servlets. A sample WAR file that uses the jar is also included. The JAR and WAR file should run with JRE 1.4+.
- The Dsr.js file has been updated to the new API.
- The Tagneto distribution includes the JavaScript libraries in a js directory at the top of the distribution.
- Tagneto's home page has been updated to use the new DSR js and jar libraries for the "Recent Blog Entries" section.
Sunday, January 01, 2006
Tagneto 0.3.2 Released
The main focus of the release was the JavaScript libraries that support doing dynamic script requests.
See http://tagneto.org/ReleaseNotes.html for more information.
See http://tagneto.org/ReleaseNotes.html for more information.