UESPWiki:File Server

This page discusses some design for a file/image web server used to server images, files, and other static content for the site.

Purpose

When a client (user) requests a page on the Wiki it results in multiple connection requests to the web server. For example, the main page results in 8 seperate requests including content, stylesheets, JavaScript, and images. On pages containing many images there could be dozens of requests. Some requests will use the client's cache which reduces the amount connections per page.

Each connection request takes up a single Apache client. The maximum number of clients, currently set to 50, sets the number of concurrent connections and is mainly limited by the server's RAM (more than 50 and we begin to hit the swap which causes performance to plummet). Each Apache client takes up a large amount of RAM due to the number of mods required to run the site (mod_php, mod_rewrite, etc...).

When serving simple static content like files and images the web server doesn't really need most of the set options. Unfortunately you cannot set which options to use during run time so a simple 1kb image takes the same amount of memory as a full PHP page.

A better system would be to have two (or more) seperate web servers running:

1. A full Apache web server with all the required options for dynamic content.
2. A very simple and stripped down web server for files and images.

The end result would be to make the server more efficient and allow it to handle more concurrent connections on the same hardware.

Lightweight Web Servers

While there are a few choices available for lightweight web servers we'll be using Lighttpd. Its popularily, ease of setup, and similarily to Apache make it an obvious choice for the time. For the most part we can change to alternative web servers in the future with not too much additional effort.

Apache Proxy

A popular option is to setup the main Apache web server to pass requests for static content onto the lightweight web server by proxy:

1. Setup the static content web server to server the necessary content on a different port than the default 80 (i.e., 81).
2. Set the Apache server to pass specific content (ex: /w/images/*) to the lightweight web server.

Advantages

• Very simple to setup
• Transparent from the client side (no apparent change in content location)
• No need for a second IP address or subdomains

Disadvantages

• Static content requests still seem to take up a slot in the Apache server (may not receive much performance benefit)
• Must setup each directory of static content to be served manually
• Slightly harder to scale to using a seperate server for static content (all changes must be done manually)

Seperate Subdomain

Another option is to serve static content from specific subdomains, i.e., images.uesp.net or files.uesp.net.

Advantages

• Scales very easily to seperate static content servers in the future
• Slightly easier to setup what content is server from the lightweight server (all content from the specific subdomains)
• Apache server is completely freed from serving any static content (no worries about proxy requests still taking up a connection slot)

Disadvantages

• Must setup subdomains and possible requires a second IP address
• All existing image/file links need to be changed (not sure how to change how the Wiki serves images)