The equipment breakdown is listed below. School of Computational Science & Information Technology (CSIT). CSIT hosts
A 30 Pentium Pro cluster with dual processors (400 Mhz), 256 Mbytes memory and 18 Gbytes disk per CPU, and a 100 Mbit/sec ethernet network. The cluster supports the activities in physics in collaboration with Jefferson Labs in Newport News, VA.
A 16-processor IBM SP2: 8 wide nodes, each with 1 gigabyte RAM, 12 gigabyte disk and 1024 Mbytes of memory, 8 thin nodes each with 256 Mbytes RAM and 12 gigabytes disk.
A second 30 Pentium Pro cluster with dual processors (450 Mhz), 256 Mbytes memory and 18 Gbytes disk per CPU is on order for research in Physics.
Two ES40 Compaq machines with 8 Gbytes of physical memory and 4 CPUs each.
An assortment of older heterogeneous clusters comprised of HPs, IBM RS/6000s, and Alpha Stations.
Department of Meteorology. They possess one 8 node IBM SP2 with 10 Gigabytes memory and 60 Gigabytes disk, one SGI power Challenge server, one SGI Origin 200 server, and 5 SGI workstations.
Department of Computer Science. They possess one 8 node IBM SP2 with 10 Gigabytes memory and 60 Gigabytes disk, one SGI power Challenge server, one SGI Origin 200 server, and 5 SGI workstations.
COAPS. The Center for Ocean-Atmosphere Prediction provides access to a 16 processor SGI Origin 2000 with 4.6 Gigabytes of memory and 320 Gigabytes of disk. In addition, the system possesses a 6 Terabyte archival system. This machine is a university-wide resource, and is not amenable to the types of computation envisioned herein.
The above equipment is supplemented by printing and backup facilities in the various groups. The research we propose in each area is very computationally intensive and must be integrated with advanced pre- and post-processing facilities. Research will focus on the development of tools which can semi-automatically generate highly efficient parallel code in several applied disciplines. The requested infrastructure will have a peak performance of at least one order of magnitude faster than the the fastest machine available on campus, with more disk, and more memory. This computer will allow tuning of codes with a view towards executing them on much larger supercomputers (e.g. those used by the Accelerated Strategic Computing Initiative (ASCI)) while retaining optimal scaling properties.
Description of Requested Equipment.
We will purchase equipment in three categories: 1) one cluster architecture
(SCM) constructed from commodity components (those used now or in the near
future by consumers), 2) one experimental parallel system (EPS)
(distributed or single image), and 3) Information and Pervasive
Infrastructure. While we have determined
the categories, it is not realistic to make a decision on a particular
vendor. To better understand the relationship between price and performance
characteristics, we have elicited quotes for both the cluster and the
EPM from Compaq, IBM, and SGI. Only SGI presented us with a cluster
with a Linux operating system, yet based on the yet to be announced
Merced chips. The Merced chip is based on the IA-64, that will be
supported by most major vendors. We expect that within 6 months that
other vendors will provide linux-based cluster solutions based
Merced.
Equipment. We anticipate in year 1 the installation of the workstation cluster, with a Gbit backbone. The cost will be approximately $600,000, including 2 years maintenance. The cluster processors will be upgraded in year 3 and a slightly lesser cost ($550,000). In year 2, we will install the 32 cpu EPM at an estimated cost of $600,000. with an upgrade in year for at a cost of $550,000.
Peak performance of both commodity (IA-64) and proprietary (EV67, Powerx) chips are projected to increase by a factor 2 to 3 yearly to satisfy the requirements of ASCI. Depending on our purchase, we will replace IBM power 3 chips with IBM Power 4 chips, Compaq EV67 chips with EV7 chips, and SGI Merced IA-64 chips with McInckinly chips. A specific upgrade program that permits the replacement of chips will be negotiated with the vendor when the system is purchased.
To support the effort in the core technologies and the integration of research and education, we will purchase a variety of smaller hardware that includes mobile palm tops and mobile laptops with associated PCI cards (they will become available within the next year), file and database servers, video servers, and trackers. The technology is continuosly changing, so we have allocated between $100,000 and $150,000 yearly for the purchase of this equipment.
The majority of the purchases will be coverered at a cost of 2/3 from NSF funds, and 1/3 from FSU funds.
Software. Software requirements includes operating systems, compilers, parallelization tools, and visualization tools.
The hardware dominates the infrastructure cost. Florida State University has agreements with SGI and IBM that substantially reduces the cost of software (together, software components provided by the vendors will cost less than $50,000). This includes compilers for C, C++, Fortran 77 and Fortran 90. An additional cost of $25,000 to $50,000 for the managed system is anticipated for third party software in the areas of performance optimization and debugging. Packages include TotalView and VAMPIR. Parallel software environments for multinode systems are available from IBM and Compaq. SGI will provide a completely Linux solution, even on its single system image, also based on IA-64 components.
Maintenance. Maintenance on the compute hardware is estimated to cost between 7 and 10 percent of total cost on a yearly basis (based on an average of IBM, SGI, and Compaq pricing). Although maintenance is included the first year, we have negotiated a price with the vendors that includes maintenance for two years. After two years of maintenance, the machines will be upgraded at a cost that once again includes two years of maintenance. There will be no upgrades to the computing infrastructure in the 5th year at which point, we expect CSIT students to maintain (at least) the cluster. The exact specifications of the maintenance contracts will be negotiated upon award of this proposal., and will demand onsite service within 24 hours of failure.
Technical Support. CSIT currently has 3 1/2 system administrators who are responsible for the daily upkeep of CSIT resources, software upgrades, troubleshooting, helping users, and performing backups. The EPM requires direct management to insure that it maintains a high degree of availability. The algorithmic development on these machines is not conducive to low level experimentation. On the other hand, the SCM will be maintained by students and postdoctoral researchers active in the improvement of system tools, operating systems, and other low level development. CSIT system administrators will remain available to provide help as needed. In the first year, some effort will be expended to become familiar with the latest cluster tools, and extend, if necessary, existing scripts at automating system administrative chores. We do not anticipate an increased administrative load once the machines are upgraded, except for a brief period when low level parameters will be readjusted to compensate for various architectural changes.
Equipment. We are proposing infrastructure to support an innovative new multidisciplinary school of Computational Science and Information Technology in the areas of research and education, and their integration in particular. The strongly multidisciplinary nature of the school leads to a strong interaction between computer science activities (for which clusters are a better environment), and computational and application science, for which a more integrated system, more akin to the big teraflop machine (to be used as a production machine) to be ordered and one that tracks ASCI, is best.
Core technology research described in this proposal supports efforts in the areas of coupled-ocean atmosphere modeling, materials science, manufacturing, acoustics, and the modeling of aquifers. In all cases, we will develop highly accurate numerical methods based on the newest implicit algorithms. These algorithms demand large memory sizes, and large caches to maximize their efficiency. The availability of a mixed mode programming model (a distributed set of n-way nodes) will lead to algorithms suitable for the next generation of supercomputers. Indeed, IBM will be announcing 32-way Power 4 chips within the next 1-3 years. Already commodity chips (Xeon III) have 4 processors per node. The algorithm development demands a computer with high availability, low down time, easy maintenance, and very high performance characteristics in terms of communication bandwidth, memory bandwidth and latencies. This leads to the requirement of what we refer to as a ``managed system'', a system on which students will not have access to the lowest levels of the operating system for experimentation. This machine will primarily be used for algorithm development, which is a substantial component of the proposed research.
Among the remaining core technology considered in this proposal, problem solving environments parallel Java, and the handling of large databases demand a combination of clustered computing and computing on a more integrated system. Problem solving environments are best built on a three tier system (client, server, cluster), which can accommodate many processes (handled by the cluster) and the interaction of many simultaneous users (clients). In most cases, the system is dominated by computational requirements, not communication bandwidth (the ideal situation for clusters). Throughput is not the primary concern of a problem solving environment. Users can wait 30 percent longer to have the system automate their code generation in a period of hours (as compared to months of labor of hand-coding is necessary). The research on HPJava leads to both a thread and an MPI implementation. The former requires a moderate size SMP or xUMA (UMA,ccNUMA, or NUMA) machine while the latter is best implemented on a cluster. In particular, research on Jini, meant to support the dynamic invocation of parallel codes is best performed on a cluster architecture. Applications of this technology includes roaming agents, computational steering, and geographically distributed computing to name a few. We anticipate installing beta versions of GSN running at 6.4 Gbits/sec under the ST protocol through a partnership with SGI. The SGI linux cluster was showcased at Supercomputing '99. Networks of this speed and beyond will be deployed nationally within the next decade. Experiments with a Gigabit or Gigabyte backbone will serve to predict and correct problems that might arise when many users interact on a wide area network at these speeds.
Both the SCM and the EPM will serve as learning tools for CSIT students. These two machines will complement each other and provide several programming paradigms for students to experiments with (OpenMP, MPI and a combination of the two). Students will have the opportunity to access the SCM to make kernel modifications for system tuning. This will not be possible on the EPM due to the need to low availability (i.e. single point of failure) of this class of computer. Science portals developed in the next several years will support both research and education and will be strongly dependent on general information and pervasive computer infrastructure such as mobile computing (hand held and laptop), database servers, video servers, etc. From previous work, the portals will emphasize web-based interaction with tools such as XML, MML, DHTML, and will require the acquisition of a variety of general purposes servers and storage to accomodate the expected growth in the data made available to the outside community.
We are requesting computer infrastructure in three distinct classes. The first is a cluster of workstations, based on the latest commodity chips, with a switched network of at least one Gigabit/sec, 1/2 to 1 Gigabyte of memory, and at least 1/2 Terabytes of disk space. The second system will be a Numa or SMP system. Except for the interconnect, the machine characteristics per prcessor are expected to be identical, as will the disk. We refer to the latter computer as a managed system (that requires formal system administrator, and cannot be experimented on at a low level by students or researchers). Recognizing that for algorithmic development, it is not important to have the largest possible system, we have allocated equal funds for both machines, to be purchased in two installments staggered at two year intervals. We expect the cluster to have twice the peak performance of the managed system. The funds allocated in years 1 and 2 for these machines are larger than the cost to upgrade. This is justified by the continued expected strong increase in the price/performance ratio. A slightly reduced cost for the system upgrade will degrade only slightly the potential performance characteristics of the machine, while still providing a substantial boost in performance.
Archival storage is not required in this proposal since at least 60 Terabytes will come online as part of the TFM.
Software. Requested software includes the tools necessary for compilation in C, C++, Fortran 77, Fortran 90, and Java, along with performance analysis tools to help optimize parallel software. The SCM will be linux-based and most of the software is free. open source.
Maintenance. Rather than pay maintenance for equipment on a yearly basis for the duration of the proposal, we will instead negotiate with the vendors to include a two year maintenance contract into the quipment at the time it is purchased. Upon upgrade, a similar contract will be negotiated. To reduce the cost, we will request service within 24 hours of failure. Given the high availability of a cluster (does not have a single point of failure), it is not necessary to obtain service quite as fast.
The campus network provides access to a number of regional, national, and world-wide networks including ESnet, NSFnet, HEPnet, BITnet, FIRN and SURAnet. In addition, two T1 connections to ESnet via the University of Texas at Austin and Oak Ridge National Labs exist. The campus backbone is a 1 Gigabit FDDI ring that connects the individual research groups involved in this effort. The TFM is will be located at Innovation park (outside the main campus). Current access to the CSIT machine room (main campus) is at 100 Mbit/sec. However, we anticipate a 1 Gbit/sec upgrade that will allow high bandwidth between the TFM and the smaller EPM/SCM at CSIT. Florida State University is a member of Internet 2.
Off-campus connectivity is available via 28.8k or 56k modems. The Academic Computing & Network Services which provides computer-related services to the university, has new hardware for 576 lines with v.90 capability.
A typical footprint for a single frame (in which nodes are housed) 76" H ×36" W ×51" D with a weight of 2000 lbs fully configured with processors, I/O and peripherals. Each tall frame holds a maximum of 16 nodes (32 to 64 processors). We require a total of 1-2 frames for a total surface area of 57 ft2. In addition, space is required for the disk farm (the archival tape unit will reside at Innvotion Park) . Lastly, a prototypical configuration for the 1 Terabyte disk subsystem is a 4 Model D40's (from IBM) that reside in a rack of dimensions 60" H ×19" W ×28" D ( » 4 ft2). A spacing of 3ft around every cabinet/rack should be left unused for maintenance. No funding from the university is requested.
The Florida State University has agreed to provide matching funds in the amount of $1,000,000 to support the purchase of equipment to support the research and education activities proposed by the School of Computational Science & Information Technology.
Spefically, Florida State University funds will be responsible for the purchase of the Experimental Parallel System (EPS), to be mostly used for research in core information technology and education. This system will also serve as a testing platform prior to the transfer of application simulation codes to the larger Teraflop Machine (TFM).
Florida State University is committed to the allocation of 5 to 8 million dollars to the the purchase of a massively parallel supercomputer that will achieve over a Teraflop of processing power over the next two years. A request for proposals will be sent out at the end of January, and the system is scheduled for installation by the end of October of this year. We estimate that upwards of 512 processors, each with a peak performance of 2 Gflops, 1 Gigabyte of memory, will provide computing cycles for university researchers to conduct production runs. About 10 percent of the machine will be reserved for computational science activities such as benchmarking, scalability testing etc.
********************************************************************** Gordon Erlebacher School of Computational Science
Tel: (850) 645-0308 FAX: (850) 645-0300 **********************************************************************