Through this page, you can configure the Catalog Agent including with which CSID to register its results, which URLs from where to start the Catalog Agent, and several parameters which tailer the behavior of the Catalog Agent.

The Catalog Agent has two parts:

If you don't want to configure the robot, you can

Enter the CSID of your local Resource Description Server (RDS). Please specify a list of URLs from which the Catalog Agent will start enumerating. The policy determines the order in which the Catalog Agent will process URLs, including: fifo, depth-first, or breadth-first. The default is fifo which consumes the least amount of resources on your server. Using depth-first the Catalog Agent will always process URLs which are the farthest away from the seed URLs. Using breadth-first the Catalog Agent will always process URLs which are the closest to the seed URLs. The speed determines how quickly the Catalog Agent will retrieve URLs, and how long it'll wait between server contacts. Generally, the higher the speed, the faster the Catalog Agent will run:

SpeedConcurrent
Retrievals
Server
Delay
1160
2130
3115
415
511
620
740
880
9120
10160

WARNING: the Catalog Agent can process a large number of URLs very quickly, and may cause a large amount of network and server usages. The strict-order flag determines how strict the Catalog Agent will adhere to the policy decisions above. When set to false, the Catalog Agent only adheres loosely to the policy decisions, always trading off for using less resources. When set to true, the Catalog Agent will always adhere to the policy decisions. Specify a valid Internet email address for the Catalog Agent to use if it is prompted for an identity. For example, this email address will be sent in the From: field of HTTP headers. Specify a user name for the Catalog Agent to use when it is prompted for authorization. Typically, the user is anonymous for FTP URLs. Specify the password associated with the username above. Typically, the password is a valid Internet email address for FTP URLs.