Monitoring the server


ou can monitor your server's activity using several different methods. You can view the server's status in real time by using the Hypertext Transfer Protocol (HTTP) or the Simple Network Management Protocol (SNMP). You can also monitor your server by recording and viewing log files.

Working with log files

Server log files record your server's activity. You can use these logs to monitor your server and to help you when troubleshooting. The error log file, located in httpd-servername/logs in the server root directory, lists all the errors the server has encountered. The access log, located in
httpd-servername/logs in the server root directory, records information about requests to the server and the responses from the server. You can use the Server Manager to specify what to include in the access log file. Use the log analyzer to generate server statistics. You can back up server error and access log files by archiving them.

Viewing an access log file

You can view the server's active and archived access log files from the Server Manager.

To view an access log:

  1. From the Server Manager, choose Server Status|View Access Log. The View Access Log form appears.
  2. Choose the access log file you want to see. Active log files for resources and archived log files appear in the list.
  3. To limit how much of the access log you see, type the number of lines you want to see in the "Number of entries" field. The order of the log entries on the screen is the order in which they were recorded in the log.
  4. If you'd like to filter the access log entries for a particular word, type the word in the "Only show entries with" field. Case is important; make sure the case for your entry matches the case of the word you're searching for. (For example, if you want to see only those access log entries that contain "POST," type POST.)
  5. Click OK.

Here is an example of an access log in the Common Logfile Format:

wiley.a.com - - [16/Feb/1996:21:18:26 -0800] "GET / HTTP/1.0" 200 751
wiley.a.com - - [17/Feb/1996:1:04:38 -0800] "GET /docs/grafx/icon.gif HTTP/1.0" 204 342
wiley.a.com - - [20/Feb/1996:4:36:53 -0800] "GET /help HTTP/1.0" 401 571
arrow.a.com - john [29/Mar/1996:4:36:53 -0800] "GET /help HTTP/1.0" 401 571
Table 10.1 describes the last line of the sample access log.
The fields in the last line of the sample access log file

Access Log Field

Example

Hostname or IP address of client

arrow.a.com. (In this case, the hostname is shown because the web server's setting for DNS lookups is enabled; if DNS lookups were disabled, the client's IP address would appear.

RFC 931 information

- (RFC 931 identity not implemented)

Username

john (username entered by the client for authentication)

Date/time of request

29/Mar/1996:4:36:53 -0800

Request

GET /help

Protocol

HTTP/1.0

Status code

401

Bytes transferred

571

Here is an example of an access log using the flexible logging format:

wiley.a.com - - [25/Mar/1996:12:55:26 -0800] "GET /index.htm HTTP/1.0" "GET" "/?-" "HTTP/
   1.0" 304 0 - Mozilla/2.0 (WinNT; I) 
wiley.a.com - - [25/Mar/1996:12:55:26 -0800] "GET / HTTP/1.0" "GET" "/?-" "HTTP/1.0" 304 0 - Mozilla/2.0 (WinNT; I)
wiley.a.com - - [25/Mar/1996:12:55:26 -0800] "GET / HTTP/1.0" "GET" "/?-" "HTTP/1.0" 304 0 - Mozilla/2.0 (X11; I; IRIX 5.3 IP22)
The access log in the flexible logging format looks similar to the access log using the Common Logfile Format.

Viewing the error log file

The error log file contains errors the server has encountered since the log file was created; it also contains informational messages about the server, such as when the server was started. Incorrect user authentication is also recorded in the error log. Use the error log to find broken URL paths or missing files.

To view the error log file from the Server Manager:

  1. From the Server Manager, choose Server Status|View Error Log. The View Error Log form appears.
  2. If you want to see more or less than 25 lines of the error log, use the "Number of errors to view" field to enter the number of lines you'd like to see. The order of the log entries on the screen is the order in which they were recorded in the log.
  3. If you'd like to filter the error messages for a particular word, type the word in the "Only show entries with" field. Case is important; make sure the case for your entry matches the case of the word you're searching for. (For example, if you want to see only those error messages that contain "warning," type warning.)
  4. Click OK.

Here is an example of an error log:

[13/Feb/1996:16:56:51] info: successful server startup
[20/Mar/1996 19:08:52] warning: for host wiley.a.com trying to GET /report.html, append-trailer reports: error opening /usr/ns-home/docs/report.html (No such file or directory)
[30/Mar/1996 15:05:43] security: for host arrow.a.com trying to GET /, basic-ncsa reports: user jane password did not match database /usr/ns-home/authdb/mktgdb
In this example, the first line is an informational message--the server started up successfully. The second log entry shows that the client wiley.a.com requested the file report.html, but the file wasn't in the primary document directory on the server. The third log entry shows that the password entered for the user jane was incorrect.

Setting log preferences

During installation, an access log file named access was created for the server. You can customize access logging for any resource by specifying whether to log accesses, what format to use for logging, and whether the server should spend time looking up the domain names of clients when they access a resource.

Server access logs can be in Common Logfile Format, flexible log format, or your own customizable format. The Common Logfile Format is a commonly supported format that provides a fixed amount of information about the server. The flexible log format allows you to choose (from the Server Manager) what to log. A customizable format uses parameter blocks that you specify to control what gets logged. Once an access log for a resource has been created, you can't change its format unless you archive it or create a new access log file for the resource.

To set access logging preferences:

  1. From the Server Manager, choose Server Status|Log Preferences. The Log Preferences form appears.
  2. Use the Resource Picker to choose the resource you'd like to apply custom logging to.
  3. Select whether to log client accesses.
  4. Type the full path for the log file.
    As a default, the log files are kept in the logs directory in the server root directory. If you specify a partial pathname, the server assumes the path is the logs directory in the server root.
  5. Choose whether to record domain names or IP addresses in the access log.
  6. Choose the format in which the log file should be: Common Logfile Format, or flexible log format (Only log radio button), or custom format. If you click Only log, you can choose any or all of the following flexible log format items:
    If you choose a custom format; type your custom format in the Custom format field. For more information about the parameters you should use, see Netscape's DevEdge online documentation web site at
    http://developer.netscape.com/library/documentation/.
  7. If you don't want to log client access from certain hostnames or IP addresses, type them in the Hostnames and IP Addresses fields. Type a wildcard pattern of hosts the server should ignore when recording accesses. For example, use *.netscape.com if you don't want to log accesses from people whose domain is netscape.com. You can type wildcard patterns for hostnames, IP addresses, or both.
  8. Choose whether to include the format string in the logfile. If you are using the proxy server's log analyzer, you should include a format string. If you are using a third-party analyzer, you may not want to include a format string in your log file.
  9. Click OK.

Archiving log files

You can archive the access and error log files and have the server create new ones.

When you archive log files, the server renames the current log files and then creates new log files with the original names. You can back up or archive (or delete) the old log files, which are saved as the original filename followed by the date and time the file was rotated. For example, access might become access.24Apr-04AM.

You can archive log files immediately or have the server archive log files at a specific time on specific days. The information about when to archive log files is stored in the cron.conf file in the admin-serv directory in the server root directory; the server's cron configuration options are stored in ns-cron.conf in the admin-serv directory.

Note
Before running the log analyzer, you should archive the server logs.
To archive log files:

  1. From the Server Manager, choose Server Status|Archive Log. The Archive Log Files form appears.
  2. Click Archive if you want to rotate the log files immediately.
    If you want archiving to occur at specific times on specific days, click the "Rotate log at" button, choose a time from the pull-down menu, and select the days for archiving to occur.

  3. Click OK.
  4. Shut down the administration server by clicking the "shut down" link on the page that appears.
  5. Restart the administration server.

Monitoring the server using HTTP

You can monitor your server's usage with the interactive server monitor. You can see how many requests your server is handling and how it is handling these requests. If the interactive server monitor reports that the server is handling a great number of requests, you may need to adjust the server configuration or the system's network kernel to accommodate the requests. The interactive server monitor is shown in Figure 10.1.

To monitor your server from the Server Manager:

  1. From the Server Manager, choose Server Status|Monitor Current Activity.
  2. Click "Monitor current activity on port port_number". The interactive server monitor shown in Figure 10.1 appears.

The interactive server monitor

The interactive server monitor reports the totals for the following server values:

Working with the log analyzer

Use the log analyzer to generate statistics about your server, such as a summary of activity, most commonly accessed URLs, times during the day when the server is accessed most frequently, and so on. You can run the log analyzer from the Server Manager or the command line.

Note
Before running the log analyzer, you should archive the server logs. For more information about archiving server logs, see "Monitoring the server using SNMP" on page 171.

Running the log analyzer from the Server Manager

To run the log analyzer from the Server Manager:

  1. From the Server Manager, choose Server Status|Generate Report. The Generate Report form appears.
  2. Type the name of your server; this name appears in the generated report.
  3. Choose whether the report will appear in HTML or plain text format.
  4. Select the log file you want to analyze.
  5. If you want to save the results in a file, type an output filename in the Output file field. If you leave the field blank, the analyzer prints results on the screen. For large log files, you should save the results to a file because printing the output to the screen might take a long time.
  6. Select whether to generate totals for certain server statistics. You can generate the following totals:
  7. Select whether to generate general statistics. You can generate the following general statistics:
  8. Select whether to generate a list of server access statistics. You can generate a list of the following:
  9. Specify the order in which you want to see the results.
  10. Click OK.

Running the log analyzer from the command line

To analyze access log files from the command line, run the tool, flexanlg, which is in extras/flexanlg in your server root directory.

To run flexanlg, type the following command and options at the command prompt:

flexanlg [ -P ] [-n name] [-x] [-r] [-p order] [-i file]* [ -m metafile ]* 
[ o file][ c opts] [-t opts] [-l opts]
The following describes the syntax. (You can get this information online by typing flexanlg -h.)

-P: proxy log format                                  Default: no
-n servername: The name of the server
-x : Output in HTML Default: no
-r : Resolve IP addresses to hostnames Default: no
-p [c,t,l]: Output order (counts, time stats, lists) Default: ctl
-i filename: Input log file(s) Default: none
-o filename: Output log file Default: stdout
-m filename: Meta file(s) Default: none
-c [h,n,r,f,e,u,o,k,c,z]: Count these item(s) - Default: hnreuokc
h: total hits
n: 304 Not Modified status codes (Use Local Copy)
r: 302 Found status codes (Redirects)
f: 404 Not Found status codes (Document Not Found)
e: 500 Server Error status codes (Misconfiguration)
u: total unique URL's
o: total unique hosts
k: total kilobytes transferred
c: total kilobytes saved by caches
z: Do not count any items.
-t [sx,mx,hx, xx,z]: Find general stats - Default:s5m5h24x10
s(number): Find top (number) seconds of log
m(number): Find top (number) minutes of log
h(number): Find top (number) hours of log
u(number): Find top (number) users of log
a(number): Find top (number) user agents of log
r(number): Find top (number) referers of log
x(number): Find top (number) for miscellaneous keywords
z: Do not find any general stats.
-l [cx,hx]: Make a list of - Default: c+3h5
c(x,+x): Most commonly accessed URLs
(x: Only list x entries)
(+x: Only list if accessed more than x times)
h(x,+x): Hosts (or IP addresses) most often accessing your server
(x: Only list x entries)
(+x: Only list if accessed more than x times)
z: Do not make any lists

Monitoring the server using SNMP

You can monitor your server in real-time by using the Simple Network Management Protocol (SNMP). SNMP is a protocol used to exchange data about network activity. With SNMP, data travels between a managed device and a network management station (NMS) where users remotely manage the network.

A managed device is anything that runs SNMP (for example, hosts, or routers). Your Enterprise Server is a managed device. An NMS is usually a powerful workstation with one or more network management applications installed. A network management application graphically shows information about managed devices (which device is up or down, which and how many error messages were received, and so on).

Every managed device contains an SNMP agent that gathers information regarding the network activity of the device. This agent is known as the subagent. Each Netscape server (except the administration server) has a subagent.

Another SNMP agent exchanges information between the subagent and NMS. This agent is called the master agent. A master agent runs on the same host machine as the subagents it talks to. You can have multiple subagents installed on a host machine. All of these subagents can communicate with the master agent.

Values for various variables that can be queried are kept on the managed device and reported to the NMS as necessary. Each variable is known as a managed object, which is anything the agent can access and send to the NMS. All managed objects are defined in a management information base (MIB), which is a database with a tree-like hierarchy. The top level of the hierarchy contains the most general information about the network. Each branch underneath is more specific and deals with separate network areas.

How does SNMP work?

SNMP exchanges network information in the form of protocol data units (PDUs). PDUs contain information about various variables stored on the managed device. These variables, also known as managed objects, have values and titles that are reported to the NMS as necessary. Communication between an NMS and managed device can take place in one of two forms:

NMS-initiated communication: NMS-initiated communication is the most common type of communication between an NMS and a managed device. In this type of communication, the NMS either requests information from the managed device or changes the value of a variable stored on the managed device.

These are the steps that make up an NMS-initiated SNMP session:

  1. The NMS searches the server's MIB to determine which managed devices and objects need to be monitored.
  2. The NMS sends a PDU to the managed device's subagent through the master agent. This PDU either requests information from the managed device or tells the subagent to change the values for variables stored on the managed device.
  3. The subagent for the managed device receives the PDU from the master agent.
  4. If the PDU from the NMS is a request for information about variables, the subagent gives information to the master agent and the master agent sends it back to the NMS in the form of another PDU. The NMS then displays the information textually or graphically.
    If the PDU from the NMS requests that the subagent set variable values, the subagent sets these values.

Managed device-initiated communication: This type of communication occurs when the managed device needs to inform the NMS of an event that has occurred. A managed device such as a terminal would initiate communication with an NMS to inform the NMS of a shut down or start up. Communication initiated by a managed device is also known as a "trap."

These are the steps that make up a managed device-initiated SNMP session:

  1. An event occurs on the managed device.
  2. The subagent informs the master agent of the event.
  3. The master agent sends a PDU to the NMS to inform the NMS of the event.
  4. The NMS displays the information textually or graphically.

For information on setting up and configuring your server to use SNMP, see Managing Netscape Servers.

The Enterprise Server MIB

Each Netscape server has its own MIB (management information base). The Enterprise Server's MIB is a file called netscape-http.mib. This MIB contains the definitions for various variables pertaining to network management for the Enterprise Server. These variables are known as managed objects. Using the Enterprise Server MIB and network management software, such as HP OpenView, you can monitor your web server like all other devices on your network.

The Enterprise Server MIB has an object identifier of netscape 1 (that is, http OBJECT IDENTIFIER : := { netscape 1 }) and is located in the server_root/plugins/snmp directory.

You can see administrative information about your web server and monitor the server in real time using the Enterprise Server MIB. Table 10.2 lists and describes the managed objects stored in the netscape-http.mib.
netscape-http.mib managed objects and descriptions

Managed object

Description

httpEntityDescr

A description of the server (includes operating system information).

httpEntityId

The enterprise subtree for vendors (for example, Netscape's MIB has an object identifier of 1.3.6.1.4.1.1450).

httpEntityProtocol

The HTTP version number.

httpEntityVersion

The server software version number.

httpEntityOrganization

The organization responsible for the server.

httpEntityLocation

The full path for the server.

httpEntityContact

The person(s) responsible for the server and contact information.

httpEntityAddress

The IP address of the machine the server is running on.

httpEntityPort

The port number on which the server is listening.

httpEntityName

The server's identifier name (for example, server2.a.com).

httpEntityType

The type of server.

httpEntityMethods

The methods supported by the server (for example, GET, POST, PUT).

httpEntityMaxProcess

The maximum number of active processes on the server.

httpEntityMinProcess

The minimum number of active processes on the server.

httpEntityMaxThread

The maximum number of active threads on the server.

httpEntityMinThread

The minimum number of active threads on the server.

httpStatisticsPort

The port number on which this server is listening.

httpStatisticsAddress

The IP address to which this server is bound.

httpStatisticsStatus

The status of the server (up or down).

httpStatisticsUptime

The uptime of the server since it was started.

httpStatisticsNumProcessIdle

The number of idle threads.

httpStatisticsNumProcessProc

The number of threads that are processing requests.

httpStatisticsNumProcessDns

The number of threads resolving host names.

httpStatisticsRequests

The total number of requests received and generated.

httpStatisticsRequestError

The total number of request errors detected.

httpStatisticsInUnknowns

The total number of unknown messages received/generated.

httpStatisticsInBytes

The total number of bytes received.

httpStatisticsOutBytes

The total number of bytes sent by the server.

httpStatisticsTimeOut

The total number of times the server has timed out.

httpStatisticsProcessNum

The number of running processes.

httpStatisticsThreadNum

The number of running threads.

httpStatisticsNumBytes

The total number of bytes sent by the server.

httpStatisticsNum2xx

The number of 200-level status requests handled by the server.

httpStatisticsNum3xx

The number of 300-level status requests handled by the server.

httpStatisticsNum4xx

The number of 400-level status requests handled by the server.

httpStatisticsNum5xx

The number of 500-level status requests handled by the server.

httpStatisticsNum200

The number of 200 (Transfer OK) requests.

httpStatisticsNum302

The number of 302 (Moved Temporarily) requests.

httpStatisticsNum304

The number of 304 (Not Modified) requests.

httpStatisticsNum401

The number of 401 (Unauthorized) requests.

httpStatisticsNum403

The number of 403 (Forbidden) requests.

Installing subagents on AIX

If your SNMP daemon is running on AIX, it supports SMUX. For this reason, you don't need to install a master agent. However, you do need to change the AIX SNMP daemon configuration.

AIX uses several configuration files to screen its communications. One of them, snmpd.conf, needs to be changed so that the SNMP daemon accepts the incoming messages from the SMUX subagent. For more information, see the online manual page for snmpd.conf. You need to add a line to define each subagent.

For example, you might add this line to the snmpd.conf:

smux 1.3.6.1.4.1.1.1450.1 "" IP_address net_mask
IP_address is the IP address of the host the subagent is running on, and net_mask is the network mask of that host.

Note
Do not use the loopback address 127.0.0.1; use the real IP address instead.
If you need more information, see your related system documentation for details.

Enabling the subagent

After you've installed the master agent that comes with your administration server, you need to enable the subagent for your web server. For more information on installing the master agent, see Managing Netscape Servers. You can use the Server Manager to enable the subagent.

To enable the SNMP subagent:

  1. From the Server Manager, choose Server Status|SNMP Subagent Configuration. The SNMP Configuration form appears.
  2. Type the name of the system that has the master agent installed on it. (The default is the local system.)
  3. Type a description.
  4. Type your organization name.
  5. Type the web server's location.
  6. Type the contact person for the web server.
  7. Click the On radio button.
  8. Click OK.
  9. Start the subagent from the SNMP Subagent Control form. For more information on starting the subagent, see "Starting, stopping, and restarting the subagent" on page 177.

Starting, stopping, and restarting the subagent

Once you have enabled the subagent, you can start, stop or restart it from the SNMP Subagent Control Form.

To start, stop, or restart the subagent:

  1. From the Server Manager, choose Server Status|SNMP Subagent Control. The SNMP Subagent Control form appears.
  2. Click the Start, Stop, or Restart button.


Copyright 1997 Netscape Communications Corporation. All rights reserved.