The Java Apache Project

How to : Scalability - Load-Balancing - Fault tolerance

with Apache JServ 1.0b4 and higher


document version: 1.1 (27 Apr 1999)

Summary

  1. Introduction
  2. Changes from previous versions
  3. Features
    1. Scalability
    2. Openess
    3. Security
    4. Load-balancing
    5. Session handling
    6. Fault tolerance
    7. Manageability
  4. Configuration
  5. Internals
    1. Load-balancing algorithm
    2. Watchdog process
    3. Shared memory
  6. Internal state - Admistration tasks
  7. Large sites
  8. Known problems
  9. FAQ
  10. Tips
  11. Authors


    12.  
     
     
     
     
     

    1. Introduction

    Scalability, load-balancing, fault-tolerance are key issues for internet content providers : As the market becomes more mature, the need for static pages websites decreases, and the dynamic content generation, personnalized, and transactionnal becomes a must.
    We strongly believe that Java(tm) is a key technology on the server side, because of its rapid prototyping and validation phases, which give a time-to-market bonus to those that are using it.

    However, as the Java Virtual Machines become faster over years, people tend to ask for always more personnalized and dynamic contents, and this type of application requires always more and more CPU power. Apache JServ addresses this requirement and lets you distribute your application load over as many hosts as needed.

    At the same time, applications and user transactions are based on HTTP sessions, and Apache JServ ensures that the system will take care of them. Of course the whole system has to be fault-tolerant.

    And of course all for free ... ;-)

    This document describes how to use Apache JServ load-balancing and fault tolerance features.

    Thank you for using Apache JServ.
     
     

    2. Changes from previous versions

    1.0b3 has a full load-balancing but only partial fail-over support (no shared memory, no watchdog process). (U**x and NT)
    1.0.b4 and over (including final 1.0) will have both features (at least on U**x).
    1.0b3 and above use new configuration parameters for load-balancing.
     

    3. Features


    4. Configuration

    Nothing special to do on the Java side. NO new parameter in properties file.
    Administrator installs JServ and then chooses to use load-balancing or not from the Apache side.
     

    1 - defining hosts and routing parameters (in red)

    Requests with cookies containing .JS2 will be routed to PC2 using protocol ajpv11 on port 8888 host 192.168.0.52.
     

    2 - defining hosts weight (in orange)


    ApJServBalance set1  SPARK 4

    Default weight is = 1. This one (SPARK) is a 4*CPU engine.
     

    3 - defining set (here called set1) of equivalent JServs (in green)

    Set "set1" contains PC1 + PC2 + PC3 + SPARK
     

    4 - defining load-balanced servlet mount point (in blue)

    Requests for /servlet will be load-balanced on set "set1".
    SPARK should receive 4 times(average) more requests than others.

    If one of the JServs fails; requests will be redirected to other members in set "set1".
     

    5 - defining the shared memory file (in pink)

    Defines the path of the file that will be used as shared memory between processes (needs read+write+create rights for the JServ's userid).
    This can be an absolute or relative file path.

    ApJServShmFile log/jserv_shm
     

    <IfModule mod_jserv.c>
    ####################################################
    #                   Apache JServ Configuration File                                     #
    ####################################################
    # Note: this file should be appended to httpd.conf
    ApJServManual on

    ApJServMount /oldservlet ajpv11://192.168.0.1:7777/zone2   // old style config

    ApJServMount /servlet balance://set1/zone1

    ApJServBalance set1  PC1
    ApJServBalance set1  PC2
    ApJServBalance set1  PC3
    ApJServBalance set1  SPARK 4

    ApJServHost PC1 ajpv11://192.168.0.51:7777
    ApJServHost PC2 ajpv11://192.168.0.52:8888
    ApJServHost PC3 ajpv11://192.168.0.53:9999
    ApJServHost SPARK ajpv11://192.168.0.54:7777

    ApJServRoute JS1 PC1
    ApJServRoute JS2 PC2
    ApJServRoute JS3 PC3
    ApJServRoute sp1  SPARK

    ApJServShmFile log/jserv_shm

    ApJServSecretKey DISABLED
    </IfModule>
     

    5. Internal

    5.1 Load-balancing algorithm

    All the job is done by mod_jserv.

    When Apache (re)starts, the configuration file (httpd.conf) is parsed. For each set (of servlet engines), a circular list of engines is created. Every engine is inserted exactly n times, where n is the weight as described above.
    In our example, the circular list contains :
    PC1 - PC2 - PC3 - SPARK - SPARK - SPARK - SPARK
      

    for each HTTP request coming for a balanced Servlet mount point:
do

    if is the first HTTP request for a balanced Servlet mount point
    then
        process_mount_default_target = randomly chosen in set
    endif

    if a session cookie is found
    then
        the process finds the JServ which owns the session (using ApJServRoute parameters)
        if the JServ is not dead/stopped (in shared memory)
        then
            the process sends the request to this JServ.
            if the JServ replies
            then
                return
            else
                mark the JServ dead (in shared memory)
            endif
        endif
    endif

    # here we have either got a request without session cookie
    # or a broken session. In this last case, we send te request 
    # to another target in the list without saying it to the client.
    # (the application will have to notice the broken session anyway)

    if process_target != process_mount_default_target AND 
       our process_mount_default_target is alive (in shared memory)
    then
        process_target = process_mount_default_target
    endif
    while (process_target exists)
    do
        if process_target is alive (in shared memory)
        then
            the process sends the request to this JServ.
            if the JServ replies
            then
                return
            else
                mark the JServ dead (in shared memory)
            endif
        endif
        process_target = next in set
    done
done
     

    5.2. Watchdog process

    At Apache startup a process is created. This process is a httpd process, which doesn't listen to HTTP requests. This process wakes up every 10 sec, reads the shared memory, and tries to connect to dead JServs. When a dead JServ replies, it is marked running in shared memory. 
    while true
do
    sleep(10)

    if this process is not the default watchdog (in shared memory )
        break
    endif

    for each JServ present in shared memory list
    do
        if JServ.state = down
            connect JServ
            if failed
                continue
            else
                mark it alive in shared memory
            endif
        endif
    done
done
     

    5.3. Shared memory

    It is used to communicate between httpd processes.
    Every httpd process opens a regular file on the disk (open read+write), and then mmaps it. This makes the file appear as regular memory.
    As many processes can do the same at the same moment, modifications in this memory zone are immediately seen by other processes.
    This is a very convenient way to share information about server state between httpd processes, and avoids a lot of infructuous connexions when a JServ is down.
    This shared memory can be read/written by any process (according to file permissions), and we can just imagine CGI programs, SNMP proxies, inetd started programs than can run and access this shared memory, mainly for admistration/monitoring tasks.
    The information stored in this shared memory is NOT MT-safe, not MP-safe : accesses are NOT synchronized.
    Why ? Well, the fault-tolerance part doesn't need it (really not). This would be just annoying if we wanted to share counters (nb of hits by example : the LB algorithm could use it to distribute the load, but this would work only for 1 Apache/N JServs. Well, random+weight is not a so bad choice, and easier to implement ;-)
    This file is not human readable/writable. use a C/Perl program to access it. (structure is in jserv.h).