Putting /files on a file server over NFS is really the way to go. Also, putting your temp dir into the shared files directory is necessary if your load balancer isn't session-aware, or you'll hit weirdness with file uploads.

You can then do 2 things:
1) Let your web servers serve the file over nfs
2) Run a web server on the file server, and rewrite requests for /files to that static web server. While I haven't actually tried 2, it seems like it may remove some of the latency and overhead of serving a file from an nfs share.