Architecture¶

Boréale Vector Cluster¶

The Boréale cluster is composed, at the hardware level, of :

• 1 NEC access frontend server, having:
  • 2x Intel Xeon Ice Lake 6338 32-core CPU @ 2.0 GHz
  • 512 GB (16x 32 GB) of DDR4-3200 ECC RAM
  • 1x NEC SX-Aurora TSUBASA 20B Vector Engine card (8 cores @ 1.6 GHz - 48 GB HBM2)
• 1 NEC access frontend server, having:
  • 2x Intel Xeon Cascade Lake 6226 12-core CPU @ 2.7 GHz
  • 192 GB (12x 16 GB) of DDR4-2933 ECC RAM
  • ConnectX-6 EDR IB/HDR100
  • 2x NEC SX-Aurora TSUBASA 10BE 48 GB Vector Engine cards (8 cores @ 1.4 GHz - 48 GB HBM2)
• 1 visualization server
  • 2x Intel Xeon Ice Lake 6338 32-core CPU @ 2.0 GHz
  • 512 GB (16x 32 GB) DDR4-3200 ECC RAM
  • 1x Nvidia Quadro RTX A5000 24 GB GDDR6 (8192 CUDA cores)
• 9 NEC SX-Aurora TSUBASA compute servers each consisting of :
  • 2x Intel Xeon Ice Lake 6326 16 cores @ 2.9 Ghz
  • 256 GB (16x 16 GB) of DDR4-3200 ECC RAM
  • 8x NEC SX-Aurora TSUBASA 20B vector boards (each with 8 cores @ 1.6 GHz - 48 GB HBM2)
    • direct communications between vector boards are done either in DMA (intra-node) or via Infiniband using NEC MPI libraries with full MPI 3.1 support.
• IBM Spectrum Scale (GPFS) shared storage for approximately 500 TB of useful capacity and 3.5 GB/s bandwidth
• a fast low-latency Nvidia Mellanox Infiniband HDR 200 Gbps network

The cluster uses the Slurm submission software for user job management and the Rocky Linux operating system.

The software environments are also adapted to the type of machines:

• NEC vector development environment
• Intel OneAPI scalar development environment