On the other hand, the rewrite engine can create higher server load than responding to the 304 headers. 304 If-Modified-Since was designed to reduce bandwidth for the server and for the client by not having to send the file if it has not been modified. The bandwidth and load used by a 304 is less than the load caused by using the rewrite engine in most cases (from my own studies, YMMV).

Yes I suppose this could happen, but if you set it up the way I outline then the only way is if no-one ever visited your site twice, i.e. no 304’s ever being sent. Otherwise mod_rewrite is very friendly for simple rewrites like this. But keep in mind you don’t have to use mod_rewrite, this method would work perfectly if you renamed your files for real. The rewriting just adds the ability to not have to rename the actual files. I could see a server using lighthttpd or something similar benefitting from that. 304’s will always produce more server load, but the biggest problem is the network hogging.

Validation Model

When a cache has a stale entry that it would like to use as a response to a client’s request, it first has to check with the origin server (or possibly an intermediate cache with a fresh response) to see if its cached entry is still usable. We call this “validating” the cache entry. Since we do not want to have to pay the overhead of retransmitting the full response if the cached entry is good, and we do not want to pay the overhead of an extra round trip if the cached entry is invalid, the HTTP/1.1 protocol supports the use of conditional methods.

Basically Ryan you remove validation information (last-modified, etags, content-md5) for your files you want to cache in this way, and then you specify a far-futures-expires cache-control header. Like for a month. Because your entity is cached client-side with no validation information, no 304’s of any kind will ever be sent to your server until the expires runs out, OR if you rename the entity.

Instead your files (like my apache.js and apache.css) can be cached on the clients browser in the purest sense, meaning you are almost creating your files in an offline mode similar to google gears. Of course this would be worthless for most people unless their was a way to force user-agents and clients to refresh their caches whenever we want, without them ever making conditional requests. To do that you just setup your .html file that links to the .js or .css file to only be cached for a day, or an hour, whatever, and so when you update your js or css file you just rename it in the .html which will force the clients browser to re-download the new file.

It’s hard to explain, you really need to get WireShark or check out HTTP Headers in Firefox to see.

As to the use of the query string, I’m aware of the HTTP specifications, and I’m aware of the use of them. However, using a “fake” query string is no more of a hack than using non-existent (as in, not mapping directly to the filesystem) URLs.

Agreed. I used to use the query string hack years ago when I first started, and it is a cool hack.. but it penalizes any file that uses it (especially intermediary caches). Mapping non-existent files to the filesystem does result in some performance loss (like bits compared to gbits of net traffic and cpu from 304 and checksum validations), and it is certainly a hacky hack for those not familiar with mod_rewrite. WordPress will no longer use the query string hack in the future… these hacks are meant for development only, not for production. It’s just nuanced enough that developers aren’t aware. Then you see sites like digg that really depend on their bandwidth and system moving away from the query hack to the best-practice I outline.

Digg uses CDN of course and all that but the way they implemented it can be done without mod_rewrite. Put your static files in a subdirectory, and when you make a change to the files just rename the directory.

http://media.digg.com/css/282/global.css

1.1 Other limitations of HTTP/1.1

Checksums are not free (Used by IP/TCP/HTTP/and of course, conditional requests and validation header generation).

Computing a digest (validator) takes CPU resources (disk resources) and might add latency to the generation of a message. (Some of these costs can be avoided by careful caching at the sender’s end, but in many cases such a cache would not have a useful hit ratio.)

Transmitting a digest consumes HTTP header space (and therefore increases latency and network bandwidth requirements.)

If the message recipient does not intend to use the digest, why should the message sender waste resources computing and sending it?

That pretty much says it all. RFC’s can help you figure it out, especially the caching information in RFC 2068

Download WireShark and you will quickly and easily be able to see what I’m talking about. I’ll probably post an article on this eventually.. as this is one of my favorite topics. Here is what RFC3230 said about validation and checksums, which is a pretty good summary of the optimizations I am using on this site if you care to YSlow it.

On the other hand, the rewrite engine can create higher server load than responding to the 304 headers.

304 If-Modified-Since was designed to reduce bandwidth for the server and for the client by not having to send the file if it has not been modified. The bandwidth and load used by a 304 is less than the load caused by using the rewrite engine in most cases (from my own studies, YMMV).

As to the use of the query string, I’m aware of the HTTP specifications, and I’m aware of the use of them. However, using a “fake” query string is no more of a hack than using non-existent (as in, not mapping directly to the filesystem) URLs.

It’s actually not a good idea, it is a great idea. The problem with the query string method is pretty large, and it always frustrates me when I see a site using it, as that tells me the developers don’t know much about the HTTP protocol.

The query string is made for one reason, to pass session specific information from the client browser back to the server. Many urls use this correctly, like:

• /wp-login.php?action=register
• /wp-login.php?action=register&redirect_to=/

And depending on the action and GET variables in the query string of the request, the server responds with a different page.

This simple method of responding differently based on the query string is exactly the way it is supposed to be used, and that is also the reason that cache’s and cache software that adheres to the HTTP protocol correctly DO NOT CACHE URLS WITH QUERY STRINGS.

Just like you wouldn’t expect or want a proxy or cache to cache a POST to a form you shouldn’t expect a proxy or cache to cache a GET request that contains a query string.

It’s not some kind of bias or anything, thats just the way the protocol was designed and specified.

The sad thing is that some developer a long time ago noticed that when a query string was added to a url, the file wasn’t loaded from cache but was re-requested. Without looking up the protocol specs or looking at any caching source code they decided that this was an effective way to implement a poor-mans-version control. Not because it was effective but because it was easy and faster than fast.

Without examining the HTTP protocol level of the communication that takes place between a browsing client and a query string hacked resource, a developer wouldn’t ever realize that in fact all the resources that have a query string on them do not get cached for real. They do get cached when the caching headers are present, the problem is that having a query string in the url triggers rules in the browsers, caches, and proxies that are based on HTTP specifications that require any cached resource containing a query string to be re-validated on each and every request. This is done by sending a 304 If-Modified-Since, or other validation request to the origin server to check if the query string resource has been modified or changed since the last request.

That is the purpose of the E-Tag, Last-Modified, Content-MD5, and other validation response headers and they need to be checked by the origin server and compared to the 304 request for each request.

When I first discovered this new method back in 2005, and started telling everyone about it, I got into this exact discussion over and over, a lot of the pros checked it out and found out I was right, most developers just didn’t want to change what they were doing, I call it lazy.

digg.com was a notorious user of the query string hack, and after discussions with them they discovered how much bandwidth and CPU and Disk IO they were wasting on these requests due to doing things against RFC/IETF Specs, and they changed. Now you can go look at digg.com’s source code and you can see a variation on this same technique.

<link rel="stylesheet" type="text/css" href="http://media.digg.com/css/277/global.css" />
<link rel="stylesheet" type="text/css" href="http://media.digg.com/css/277/lightbox.css" />