Replied: Mon, 08 Mar 1999 10:50:39 -0500 Replied: Ken Flurchick Received: from osc.edu (atlantis.osc.edu [192.148.249.4]) by postoffice.npac.syr.edu (8.9.1a/8.9.1) with ESMTP id NAA19215; Mon, 1 Mar 1999 13:48:38 -0500 (EST) Received: from bedrock.osc.edu (bedrock.osc.edu [192.148.249.6]) by osc.edu (8.8.6/8.8.6/OSC 1.1) with ESMTP id NAA13876; Mon, 1 Mar 1999 13:48:29 -0500 (EST) From: Ken Flurchick Received: (from kenf@localhost) by bedrock.osc.edu (8.8.6/8.8.6) id NAA15295; Mon, 1 Mar 1999 13:48:32 -0500 (EST) Date: Mon, 1 Mar 1999 13:48:32 -0500 (EST) Message-Id: <199903011848.NAA15295@bedrock.osc.edu> To: William.Asbury@msrc.wpafb.af.mil, gcf@npac.syr.edu, haupt@npac.syr.edu, rhp@osc.edu Subject: PSE specification Content-Type: text Content-Length: 13256 PSE Specification
Memorandum
To: Geoffrey Fox and Bill Asbury
From: Ken Flurchick, Jan Labanowski and Armen Ezekielian
Ohio Supercomputer Center
Subject: Draft PSE Specification
Date: 03/01/99

The PSE System Specification
Draft

System Objectives

The Problem Solving Environment (PSE) is a flexible application environment which allows the DoD researcher to more effectively access and utilize available resources in developing computational solutions for their research problems. The PSE is a component of the Gateway system and interfaces to the 'Middle Tier' component, which processes user requests and interacts with all the available resources. Additionally, the user can obtain assistance from the PSE to describe, in science terms, the problem to be solved, investigate a variety of computational methods, plus the PSE which can suggest application codes to solve the problem.

The PSE design decomposes the interaction between the user and the 'Middle Tier' into four layers which cover all the activities with the available resources. The four layers are:

Entry Layer
What it does. This layer takes care of the registered user entry to the Gateway. This layer interacts with the 'Middle Tier' to obtain authentication and authorization to access the Gateway services and compute resources.

What it looks like. The user enters a URL and receives a prompt to log in. After authentication, the user will see a dynamically created page with user specific information displayed.

Problem Description Layer PDL
What it does. The PDL acts as a pedagogical step between initial entry to the PSE and the code submission step. This layer will translate a science based problem description into a computational based solution accessed via a set of specific codes. (An XML based mechanism to develop and maintain information at this layer is being designed.) It is designed to provide:
  • A selection of useful documentation including but not limited to, computational methodologies for a specific CTA. This will provide users immediate access to information about the possible avenues for solving a problem.
  • A code selection tree to guide users through the method and code selection process.
  • Solution requirements in terms of compute resources, storage requirements, memory requirements etc, can also be provided at this layer.

What it looks like. The user will see a page, with possibly several menus, to select the CTA relevant documentation and discussions. This can include links to external material. This layer is CTA specific and can be extremely useful to users.

Applications Code Layer
What it does. This layer provides a step by step process for generating input files (or reading an input file in). The PSE server maintains:
  • a database with available application codes and resources (the database is described in more detail below);
  • the platforms on which the codes are installed;
  • testing information;
  • version levels and example input files;
  • other tools for input generation, visualization, data conversion, etc.
Note, some information is available to MSRC CCM users in the form of a RIB (Repository in a Box) deployment grid.

What it looks like. This layer consists of a page with a set of links to the input pages for each application. In addition, each application page will contain links to documentation for that application (if it exists). Entry to this layer is from a variety of sources. Some of the possible entry scenarios are:

  • from the "User Entry Layer Page";
  • from the PDL code selection tree;
  • from the documentation links in the PDL;
Results Layer
What it does. This layer is designed to display sufficient information to allow the user to decide that the run completed successfully, in terms of the needs of the user. In addition, this layer may provide links to more detailed analysis tools if available. A mechanism to tokenize the output file for database storage of the results will be considered (past phase 1).

What it looks like. This layer will provide a set of browser based tools to view a subset of the output from a computer run in either a textual or graphical form. Links to the output data files are displayed in the "User Entry Layer Page", and clicking on these links brings the user to this layer.

Design Overview

The four layers provide a mechanism to incorporate all of the CTA user activities. The four layer PSE needs to communicate with the 'Middle Tier' component of the Gateway system as well as among each other. The services needed - such as submitting a job, checking job status, moving files, etc. - are provided by the the 'Middle Tier' component API. That is, by invoking 'Middle Tier' object interfaces and allowing the ORB to resolve the mechanism by which the request will be processed. The four layers of the PSE communicate among each other via what we will refer to as "database". There are currently three "database" being considered:

  1. User Database (UDB)
    This "database" will maintain user information for authentication and authorization plus the information about running jobs, recently completed jobs, data files, resource usage, and other user information. This information will be used to verify the user for access to resources, provide links to data files and maintain user state information so the user can quickly pick up from where they left off. This "database" may be a fully relational database such as Oracle or a set of files using XML to describe the state.
  2. Resource Database (RDB)
    This "database" will contain mappings between the available resources: compute servers, mass storage systems, etc, and applications. A different database will contain authorization information. This will be primarily under the auspices of the 'Middle Tier' and will contain site specific information.
  3. PDL Database (PDLDB)
    This "database" may be incorporated in the UDB or may be a separate component. This will contain user state information, such as job history, pointers to pages in the PDL, etc., within the PDL and also provide a search capability.

The PSE provides different level tracks. Currently there are two tracks: User and Advanced. The User track is the default track and the the Advanced track is accessed via links and/or buttons. The User track will provide the user with default selections for application software, compute platforms and other tools based on the problem description provided in the PDL. The Advanced track allows the user additional flexibility via a set of options to control the parameters of, for example, job submission, hardware platforms and tools, as well as CTA specific issues. Thus, the Advanced track and the User track will have different browser interfaces.

Front End Requirements

The target for the PSE is a client browser. Current development for CCM only, has been tested using Netscape Navigator version 4.5. In general the PSE should target the commonly used browsers.

Middleware Requirements

The PSE utilizes 'Middle Tier' services which are accessed via the API. There are many different 'Middle Tier' services required:

  1. User Entry Layer:
    • A mechanism for user authentication an authorization.
    • A mechanism to enter the Gateway Portal.
    • Support for all the "databases".
  2. PDL Layer:
    • A search engine for the PDL.
    • Delivery mechanism for the required pages.
    • XML parsers and possible co-development of an XML translator.
    • Support for the PDLDB or UDB entries.
    • A mechanism to find other users of a particular methodology and/or applications.
    • A mechanism to share files among users. One possible process involves the shared file system and allowing the user to change permissions.
  3. Code Layer:
    • A mechanism to read and write files on Gateway system.
    • A mechanism to pass files between the user workstation and the Gateway system. This may be accomplished by http or other protocols.
    • A mechanism to submit jobs to the user authorized compute servers.
    • A mechanism to query job status.
    • A mechanism to obtain the status of the back end resources
    • Access to the underlying submit process for the Advanced Users to alter runtime specifications such as memory, time of the execution, etc.
  4. Results layer:
    • A mechanism to process files to display the requested results.
    • Support for access to other analysis tools This is done by creating modules in the 'Middle Tier' API.
    • A search capability for the user to search their own results files.

Note that access for reading and writing to the "databases" is required for all layers in the PSE. In addition, the 'Middle Tier' needs to provide mechanisms for accounting and statistics of application usage.

Back-End requirements

No current requirements.

Other Requirements

No current requirements.

----------------------------------------- PSE Timeline
Memorandum
To: Geoffrey Fox and Bill Asbury
From: Ken Flurchick, Jan Labanowski and Armen Ezekielian
Ohio Supercomputer Center
Subject: Draft PSE Timeline
Date: 03/01/99

Proposed Timeline
(addressing only PSE issues)

March 1st
Initial draft of the PSE specification.
Mid March
Installation of the pre-alpha deployment of the IPSE system at OSC. Learning the Gateway system and integrating with the PSE functions.
April
Final version of the PSE specification based on the integration with the Gateway system. Installation of the alpha deployment of the IPSE system at OSC. Installation of the CCM-PSE on the secure testbed at ASC.
Mid June
Alpha release of the CCM-PSE based on the pre-release on the testbed at ASC.
Mid September
Beta release of the CCM-PSE.
November
Release 1.0 Final deployment of the system, to be demonstrated at Supercomputing '99