ScaDS Logo

COMPETENCE CENTER
FOR SCALABLE DATA SERVICES
AND SOLUTIONS

Big Data Cluster in “Shared Nothing” Architecture in Leipzig

Beitragsseiten

The Galaxy Cluster

The state of Saxony funded a notable shared nothing cluster located at the University of Leipzig and the Technical University of Dresden. Here we want to give a short overview on this new “Galaxy” cluster which is a very nice asset for ScaDS.

Shared nothing is probably the most referenced architecture when talking about big data. The idea behind this cluster architecture is to use large amounts of commodity hardware to store and analyze big amounts of data in a highly distributed, scalable and cost effective way. It is optimized for massive parallel data oriented computations using e.g. Apache Hadoop, Apache Spark or Apache Flink.

Cluster Facts Overview:

Architecture

shared nothing

Number of servers

90 (30 located in Dresden, 60 in Leipzig)

CPUs

180 sockets, 1080 computing cores

Harddrives

540 SATA discs, >2 petabyte in total

RAM

>11.5 terabyte

Network

10 gigabit/s Ethernet

A part of the new shared nothing big data cluster galaxy in Leipzig
A part of the new shared nothing big data cluster galaxy in Leipzig. Photograph: Dr. Philipp, URZ Leipzig

The Galaxy Cluster consists of 90 servers of which 30 are located in Dresden and 60 in Leipzig and is managed by the Leipzig University Computing Center. As there is a high diversity of researchers requirements, the cluster is organized with respect to its size in quite a flexible way. Different computing partitions can be organized and managed depending on scientists’ needs. Thus Galaxy offers our big data scientists a very powerful computing and research environment. It is an important building block and good fit in combination with the other computing options available at ScaDS.

In the following sections we describe the hardware and infrastructure in detail before coming to the operations concept and finishing with a short overview on other computing options.