Referrer module
Referrer header validation and denial
The problem: bandwidth 'stealing'
Documents on the World Wide Web are marked up using HTML, which was designed to easily reference files on any server worldwide, using the URI scheme. In recent times, as information has become more valuable and sites more popular, some sites have been known to link directly to a remote site's valuable documents (typically images) without linking through an entrance page.
Typically, this action is seen in a negative light, since this often deprives the target site of the credit of creation, whilst the user viewing the pages are not aware this process is happening. In addition, the browser is consuming bandwidth from target site being linked to, in effect, bandwidth 'stealing'!
This practise is often known as "embedding", where an HTML or XML document on another server uses a tag such as img. Typically images are the most targetted candidates, but it could be any file. For example, if an HTML page at http://otherserver/page.html might include the line:
- <img src="http://yourserver/yourimage.gif">
The solution: denying directly linked requests from remote sites
The Zeus Web Server referrer module can prevent external sites linking directly to certain filetypes on your site. It does this by checking the 'Referer:' header sent by the user's browser. When a user's web browser accessing this HTML page parses this declaration, it will start an HTTP request to yourserver asking for /yourimage.gif and will set the Referer: header to be http://otherserver/page.html. (The header is misspelled in the HTTP/1.0 specification; referrer is spelled with four Rs)
The Zeus Web Server can look at this header and deny access to certain MIME types if it doesn't match the list that you specify. It would not be sensible to deny access to file of type text/html, since no hyperlinks would then work to your site, but image and other content files might be listed so they can only be referenced from HTML files on your servers, or friends you wish to allow.
Configuring the referrer module is done in three parts
- a list of MIME types to check the referrer header for
- a list of valid referrer hostnames or IP address masks
- the URL to which the browser will be re-directed to if the referrer header isn't valid.
Each of these settings is on a per-Virtual Server basis. The name of the Virtual Server and any aliases specified for the server will be automatically included as valid referrers, hence these do not need to be added the list of valid referrers.
The list of MIME types should be constructed from the pull-down list provided. The list of valid referrers can be altered by adding a new one in the input box provided, or by checking the "Delete" box next to an existing hostname or IP address mask, and updating the configuration.
When a browser is denied to a file because the referrer header doesn't match either the servername or one of the valid referrers, a message is logged to the Zeus error log file which includes the name of the Virtual Server, the URL which caused the denial, and the page the browser was redirected to.
Note that the referrer will contain an IP address if the server which referenced a file on your server was originally accessed via an IP address instead of a DNS name.