Genetic Cluster Computer

Technical Details

The Genetic Cluster Computer is part of the Dutch national compute cluster Lisa, hosted by SURFsara.

The LISA system configuration

624 compute nodes:

  • 144, each equipped with:
    • Processors: 2 Intel® octo-core Xeon” E5-2650L
    • Clock: 1.80 GHz
    • Memory: 32 GB QPI 8.00 GT/s
    • Cache: 20 MB
    • Local scratch space: 750 GB
  • 384, each equipped with:
    • Processors: 2 Intel® quad-core Xeon” L5520
    • Clock: 2.26 GHz
    • Quick Path Interconnect: 5.86 GT/s
    • Memory: 24 GB QPI 5.86 GT/s
    • Cache: 8 MB
    • Infiniband: Qlogic on 256 nodes
    • Local scratch space: 65 Gbyte
  • 96, each equipped with:
    • Processors: 2 Intel® six-core Xeon” L5640
    • Clock: 2.26 GHz
    • Quick Path Interconnect: 5.86 GT/s
    • Memory: 24 GB QPI 5.86 GT/s
    • Cache: 12 MB
    • Infiniband: Qlogic on 64 nodes
    • Local scratch space: 220 Gbyte
  • Qlogic 4x DDR InfiniBand network:
    • Bandwidth: 1600 MB/sec
    • Latency: <6 µsec
  • Total peak performance: 20 TFlop/sec
  • Disk space: 48 TB SUN® Thor systems for the home file systems
  • Operating system: Debian Linux AMD64 OS.

There are five types of compute nodes:

NumberCoresMemoryInfiniBand
1441624 GBQlogic
128824 GB-
256824 GBQlogic
321224 GB-
641224 GBQlogic

Managing the national compute cluster

In order to keep the national compute cluster manageable, use is made of the following software:

Cfengine2
a configuration engine.
SystemImager
SURFsara has developed an add-on for SystemImager to manage the PXE boot process. It is available here. There is an excellent description available at the Technical University of Denmark.
OpenPBS and TORQUE
SARA has developed a Python interface for both batch systems.
So it is now possible to develop batch utilities in Python instead of C. The software package contains an example directory with an ASCII xpbsmon utility.
Ganglia
For Ganglia SURFsara has developed an add-on to monitor OpenPBS and TORQUE queues. This creation is based on software included in the rocks toolkit.
torque_2_deb
SURFsara has developed software to make a debian package from the Torque source
Dell
OpenManage Server Administrator (OMSA): SURFsara has ported the OMSA tools to Debian