Drupal private files - 4.8

The private files system in Drupal is OK for minor usage but does not scale well. The performance problem (yeah, there are non performance problems too) is that every private file request requires a full bootstrap of Drupal which means that all the modules must be loaded and parsed. Now imagine that your site becomes very popular and you have 100 images on your home page that all require this bootstrap. This is not theory - it happenned recently on one of our (Bryght) sites. Server was quite stressed

I'm thinking that temporary URLs are our only hope here. Basically, I propose that all links to private files have tokens on them. When one of these image requests is found, the token is checked for correctness and the image is served. The token is tied to the user's session ID. Drupal need not even make a database connection.

The annoyance with temporary URLs is that they are temporary. You can't just stick one in your HTML and expect it to work forever. So, I propose that we have a placeholder syntax like [[file_path|images/fire.jpg]] wherever we want an image inside of a node or other textarea. Then the filtering system translates this at render time to a valid temporary URL. This scheme will properly show images during preview, but not in a WYSIWYG editor like TinyMCE.

One big downside to this is that any node with a private file reference can't be cached in the filter cache. This may or may not be a big problem, and may or may not solvable. We need some more minds on this issue.

Update

UnConeD came up with a couple improvements which remove the need for the filter tokens and solve the WYSIWYG problem. We use current private file urls during compose. Thos are working urls, and when WYSIWYG sees them, they work as expected. The filter system then transforms those urls into temporary urls at rendering time. This is a brilliant improvement, since we have effectively made this change transparent to all current Drupal modules and such. Old urls will still work. But Drupal will never vend them except during a compose situation.

The filter cache problem remains unsolved. Actually, I've learned that it is worse - all text written in this input format can't be cached - regardless of whether it uses temporary urls. I'm thinking that we have to direct posts with these urls into their own 'nocache' input format(s). Maybe each input format on a site has a mirror one that has temporary urls enabled? Sounds dirty.

Technorati Tags: Drupal