At IU, what is Big Red?

On this page:

• Introduction
• Hardware configuration
• Networking
• Storage
• More information

---

Introduction

Big Red is one of the most powerful university-owned computers in the US, and one of the 50 fastest supercomputers in the world. Part of a comprehensive strategy to build an advanced cyberinfrastructure to support research at Indiana University, Big Red has a theoretical peak performance of more than 30 teraflops, and has achieved more than 21 teraflops on numerical computations.

Access to Big Red is provided to all Indiana University faculty, graduate students, and staff. Undergraduates must have an IU faculty or staff sponsor. Instructional use is limited to courses that have been approved by the Director for Research Technologies.

The Big Red cluster runs the SuSE Linux Enterprise Server operating system. Batch jobs are managed with IBM's LoadLeveler and the Moab Workload Manager.

Big Red is configured for massively parallel computing. User support, including migrating code to Big Red and parallelizing it, is available from the UITS Research Technologies division's High Performance Applications team. For information, email High Performance Applications.

Hardware configuration

Big Red is a distributed shared-memory cluster, consisting of 768 IBM JS21 Blades, each with two dual-core PowerPC 970 MP processors, 8GB of memory, and a PCI-X Myrinet 2000 adapter for high-bandwidth, low-latency MPI applications. In addition to local scratch disks, the Big Red compute nodes are connected via gigabit Ethernet to a 266TB GPFS file system, hosted on 16 IBM p505 Power5 systems.

Following is information about Big Red's cluster nodes:

• Compute (768 JS21 Bladeserver nodes):
  
  
  • 2 x 2.5GHz dual-core PowerPC 970MP processors
  • 8GB 533MHz DDR2 SDRAM
  • 73GB SAS disk (67GB in /scratch)
  • 1 x Myricom M3S-PCIXD-2-I (Lanai XP)
  
  
• User (4 JS21 Bladeserver nodes):
  
  
  • 2 x 2.5GHz dual-core PowerPC 970MP processors
  • 8GB 533MHz DDR2 SDRAM
  • 73GB SAS disk (67GB in /scratch)
  • 1 x Myricom M3S-PCIXD-2-I (Lanai XP)
  
  
• Storage (16 pSeries 505 nodes):
  
  
  • 2 x 1.65GHz dual-core Power5+ processor
  • 8GB 533MHz DDR2 SDRAM
  • 73GB SAS disk
  • 2 x Emulex LP10000 PCI-X/133MHz FC adapters

See the technical schematic for an illustration of how Big Red is structured.

Networking

Four 256-port Myricom M3-CLOS-ENCL switches provide a 2+2Gb/s low-latency interconnect for the 768 compute nodes.

Storage

The disk for Big Red's 266TB GPFS volume is hosted on 10 DataDirect Networks S2A 9500 storage controllers, each dual-pathed to 5 SAF 4248 Chassis. Physical disks are aggregated in an 8+2 RAID configuration; combined with the dual-pathed controllers and active/active GPFS storage hosts, this provides multiple levels of redundancy for Big Red's storage space.

Home directories are available via NFSv3 over Gigabit Ethernet.

More information

For more information about Big Red, see:

• The Big Red home page
• Software available on Big Red
• Getting started on Big Red

This document was developed with support from the National Science Foundation (NSF) under Grant No. 0503697 to the University of Chicago and subcontracted to Indiana University. Additional support was provided by IU through its participation in the TeraGrid, which is supported by the NSF under Grants No. 0833618, SCI451237, SCI535258, and SCI504075. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.

Also see:

• At IU, what supercomputer systems are available for academic research?
• Where can I get help using IU's supercomputers?
• At IU, what research computing services are available?
• If I plan to use IU's research computers, what wording should I use in my grant applications?
• Policies regarding UITS research systems