ScaDS Logo


ScaDS.AI Participation at Student Panel of DI2KG Workshop

The 2nd International Workshop on Challenges and Experiences from Data Integration to Knowledge Graphs (DI2KG) aims to foster innovation in the fields of data integration and knowledge graph construction. Research areas that also receive high attention in ScaDS, e.g. with the FAMER framework. This is why ScaDS already participated in the first iteration of the workshop, which also resulted in a short publication that described our winning solution of last year's challenge.

Weiterlesen ...

Ray Reiter Best Paper Award for Dr. habil. Ringo Baumann, Prof. Dr. Gerhard Brewka and Dr. Markus Ulbricht

The ScaDS.AI researchers Ringo Baumann, Gerhard Brewka and Markus Ulbricht recently won the Ray Reiter Best Paper Award at the 17th International Conference on Principles of Knowledge Representation and Reasoning.


Weiterlesen ...

Honorable Mention at the Dissertation Award of the European Association for Artificial Intelligence for Dr. Markus Ulbricht

Dr. Markus Ulbricht reached an honorable mention for his PHD thesis at the Dissertation Award of the European Association for Artificial Intelligence. The thesis contributes to the research area of knowledge representation and reasoning (KR).

Weiterlesen ...

Kick-Off: New joint research project about Cyber Security (ZIM network)

Today we are happy to announce the Kick-Off of a new and exiting joint research project with ScaDS.AI collaboration. Thanks to all the project partners ITPower Solutions GmbH, quapona technologies GmbH, Fraunhofer Institute for open communication systems FOKUS and Leipzig University. Our fellow researcher Martin Grimmer wrote a short sketch about his interesting new project which is attached below. For now, we wish you a successful time. 


Weiterlesen ...

Virtual ScaDS.AI Summer School on AI and Big Data attracts international crowd

Since 2015, the data science center ScaDS.AI (Center for scalable Data Services and Artificial Intelligence) and the preceding Big Data center ScaDS Dresden/Leipzig run a yearly international summer school. Its 6th edition was planned to take place for a full week  in July 2020 at the Univ. of Leipzig. Due to the Covid-19 pandemic it was replaced by a virtual and more compact 2-day event that took place at July 7-8.  This opened the summer school on current AI (artificial intelligence) and Big Data topics not only for a broader and more international crowd of participants but also for internationally renowned  speakers. With more than 250 registrations from North and South America (USA, Ecuador, Brazil), Europe (Germany, Switzerland, Italy, Spain, Norway, France, UK, Romania, Ukraine), Asia (Russia, India, Thailand), Africa (Morocco, Turkey, Iran) and Australia the ScaDS.AI Summer School of 2020 achieved a great international outreach and better participation than in previous years with 70-100 participants.  

Weiterlesen ...

Rückblick Summerschool 2019

From August 17th to 23rd 2019 the two Germany based Big Data Competence Centers ScaDS Dresden/Leipzig and BBDC held the fifth international summer school on Big Data and Machine Learning in Dresden. This time, the summer school bridged the gap between the research fields Big Data and machine learning, with contributions from many internationally well-known experts from various fields. The highly recognized program included key notes from IBM, NVIDIA, Intel, and speakers from academia of both competence centers BBDC and ScaDS Dresden/Leipzig as well as invited speakers. The topics span a wide range of topics around large scale and data intensive computing (Big Data) and exciting new trends in machine learning, such as uncertainty quantification, distributed machine learning and architectural optimization for deep learning. Almost sixty participants could not just take part and connect to the expert, but could also contribute a poster about own research activity in a poster session and during the whole week to trigger discussions between participants.  As social activity an archery tournament brought fun and a contrast into the program as well as triggered some competition among the participants. Stay in touch with us about future activities, e.g.the Big Data and AI in Business Workshop @September 19.-20. in Leipzig!

Weiterlesen ...

Rückblick Summerschool 2018


Last year the BBDC Berlin and the Big Data Competence Center ScaDS Dresden/Leipzig invited to the 4th international Summer School for Big Data and Machine Learning with Hackathon ( From 30.06. to 06.07.2018, the University of Leipzig offered a wide-ranging program that gave the more than 80 participants from industry and research insights into new findings and challenges in dealing with very large amounts of data and machine learning and enabled a lively exchange.

As already in the year 2017 there were again exciting and current lectures and discussions on the individual topics, which the overriding topic Big Data and machine learning raises. We would like to take this opportunity to thank all speakers and participants once again for their participation in a successful event.

Speakers from well-known companies (e.g. Microsoft, neo4j, Zalando) as well as speakers from various universities (University of Munich, Politecnico di Milano, FZ Jülich and many more) reported on problems, current research points and solutions.
At the same time there was a colorful accompanying program, which invited to explore Leipzig with dragon boat trips and city tours and promoted the common exchange.

Weiterlesen ...

Big Data in Business 2017

Auch dieses Jahr lud der Leipziger Standort des Big Data Kompetenzzentrums ScaDS Dresden/Leipzig wieder zu einem Workshop zum Thema „Big Data in Business“ ein ( Am 15. und 16.06.2017 wurde im Felix-Klein-Hörsaal der Universität Leipzig ein breit gefächertes Programm geboten, das den über 50 Teilnehmern aus Wirtschaft und Forschung Einblicke über neue Erkenntnisse und Herausforderungen im Umgang mit sehr großen Datenmengen gab und einen regen Austausch ermöglichte.
Wie bereits im Jahr 2015 gab es wieder spannende und aktuelle Vorträge und Diskussionen zu den einzelnen Themen, die die übergeordnete Thematik Big Data mit sich bringt. An dieser Stelle möchten wir uns noch einmal ganz herzlich bei allen Referenten und Teilnehmern für ihre Beteiligung an einer gelungenen Veranstaltung bedanken.

Referenten bekannter Unternehmen (u.a. BMW Group, Immowelt AG, Huawei Technology), sowie lokale Startups, berichteten von Problemstellungen aus der Praxis und ihren Lösungen. Gleichzeitig wurde das bewährte Begleitprogramm aus dem Jahr 2015 weitergeführt, indem Wissenschaftler der Universität Leipzig Forschungsprojekte und -prototypen (z.B. Gradoop und Exploids) des Big-Data-Kompetenzzentrums vorstellten.

Weiterlesen ...

Virtual Cloud Infrastructure for Data Analysis


Nowadays, data analysis is one of the crucial parts in the field of science and research and in business as well. The data analysis process includes different steps and areas. These are mainly data collection, data pre-processing (checking, cleaning etc.), data analysis itself and visualization/interpretation of the results. Thereby, every single step can be realized by using a big variety of tools. Developing an efficient and powerful analysis process, especially in connection with big data, can be a technical challenge. Therefore it is of advantage, to have an infrastructure that allows testing, modifying and evaluating every single part of the analysis as well as the whole process.

The cloud structure as described in this article provides a cost-efficient and flexible platform in order to develop and evaluate complex data analysis processes. In the following article, an example of the cloud infrastructure itself is presented at first. In the second part, we demonstrate an application of the infrastructure in order to realize a data analysis task.

Weiterlesen ...

Static Publications Site-Tutorial (ORC-Schlange)

Probably every research group in the world faces the problem of collecting all publications of all group members. Usually, the list publications is displayed in structured way on the group's homepage to provide an overview of the research topics and impact of the group.

The larger and older the group, the more publications are in this list and the more painful is the manual collection of the publication list. Additional features such as searching for authors, keywords, and titles, linking additional author data to the publication (such as membership periode in the group), and handling name changes turn a simple publication list in a interesting use case for big data.

A effective solution to this problem is given in this tutorial. The tutorial is written for python starters and gives an introduction in many techniques:

  • Advanced features of python 3.6
  • Interacting with SQLite in python
  • Interacting with a REST-API in python.
  • Interacting with the ORCID public API
  • Reading and writing bibtex files in python
  • Creating HTML of a bibtex file in python
  • Filtering HTML content with javascript

Understanding basic python syntax is required for this tutorial but all advance features are explained.

The tutorial is subdivded into 8 parts. Each part introduces a technique and demonstrate the its usage for the use case. You can, thus, jump to the part of interest or follow the tutorial step by step. A full understanding of the use case can only be achieved by reading the complete tutorial.

Weiterlesen ...

Large-scale map analysis for land-use change monitoring using machine learning methods within an HPC environment

Historical topographic maps are a valuable and often the only data source for tracking long-term land-use changes. Their availability and vast spatio-temporal coverage make these maps an important source of information for climate and earth system modeling (ESM). However, the automated retrieval of complex and compound geographical objects from these historical maps is a challenging task. To facilitate the laborious information extraction from these maps, we present a two-stage machine learning-based approach for segmenting urban land-use from gray-scale scans using only a small set of training samples. We employ a Conditional Random Field (CRF) which obtains its unary potentials from a Random Forest (RF). The method is tested using two inference algorithms. To evaluate the performance and the scalability of the approach over large amounts of data sets, we conduct parallel computing experiments within a High Performance Computing (HPC) environment at the Center for Information Services and High Performance Computing at TU Dresden. We evaluated the methodology on the first Central-European set of trigonometry-based maps (1:25000) from 1850-1940 with large spatial and temporal coverage, which makes them particularly valuable for land-use change research and historical geo-information systems (HGIS). Experimental results indicate the suitability of both, the methodological approach and its parallel implementation.

This work has been presented at the GEOBIA 2016 Conference. A conference paper has been published and is available online.

Demonstration service for binary image segmentation

Binary image segmentation is a technique to identify various segments in a digital image. The main goal of segmentation is to enhance the information content of the image and to provide a standardized representation of the reconstructed segments. Image segmentation can be used in various ways, its applications range from low-level vision like 3D-reconstruction and motion estimation to high-level problems like image understanding and scene parsing. This demonstrator shows the applicability of this method for different raw image types to illustrate the possibly large range of application areas.

The main focus of this work was not to reach good performance in terms of e.g. pixel accuracy. Instead, we focus on usability, computational efficiency and generalization.
Usability: unlike many of the existing systems, here, the training data may be incomplete, like e.g. in GrabCut binary segmentation, where a user provides only scribbles or bounding boxes marking some pixel as the background or foreground. In doing so, usually, most of image pixels remain non-marked. Actually, we only have ground truth information for the pictures of one use-case. This ground truth information is however not used for learning but only to check the results afterwards. To summarize, we employ semi-supervised learning using a quite incomplete user information.
Computational efficiency: most of the system is implemented for processing on a GPU. For the use-cases below, the full pipeline (computing features, learning, inference etc.) takes about a couple of minutes depending on the image's size.
Generalization: for each use-case, we learn only relatively few unknown parameters. In particular, we do not learn image features. We use a pre-trained Convolutional Neural Network to do this. The default values work quite stable and sufficient for most cases.

Further Information can be found here.


Random numbers are hard - even harder in virtual machines

Random numbers are an essential element of cryptography and therefore security in general. Like most security aspects, random numbers sound simple but proof to be hard to get right. This text will discuss problems regarding random numbers in computers, especially in virtual machines.

Weiterlesen ...

Big Data Reference architectures: Are they really needed?

Reference architectures are a key research topic in business information systems. They try to simplify software development by reusing architectural and software components. But reusability leads also to a trade-off in making reference architectures on a higher level to reuse it in many domains and applications. Or to concentrate them on a subject and hence easier to reuse.

In this blog post we discuss, whether big data references architectures are really needed. Our hypothesis is that current big data reference architectures are not sufficient to provide real benefit for implementing big data projects.

Weiterlesen ...

Big Data Frameworks on highly efficient computing infrastructures


Big Data is usually used as a synonym for Data Science on huge datasets and dealing with all kinds of obstacles coming with that. Having access to a large amount of data offers a high potential to find more accurate results for many research questions. Moreover, the ability to handle Big Data volumes may facilitate solutions to previously unsolved problems. However, many research groups have not the necessary facilities to run large analysis jobs using computing resources they have access to at their home institution. Furthermore, the installation, administration and maintenance of a complex and agile software stack for data analytics is often a challenging task for domain scientists. One of the key issues of the Big Data competence center ScaDS Dresden/Leipzig is therefore to provide multi-purpose data analytics frameworks for research communities, which can be used directly at the computing resources of the Center for Information Services and High Performance Computing (ZIH). Using the high performance computing (HPC) infrastructure of ZIH, ScaDS Dresden/Leipzig members and collaborating researchers can run their data analytics pipelines massively in parallel on modern hardware. The following general purpose data analytics frameworks are currently available:

Weiterlesen ...

OSTMap - Open Source Tweet Map


It is often necessary to build a proof of concept to show the ease and feasibility of Big Data to customers / project promoters or colleagues. With OSTMap (Open Source Tweet Map) mgm partners with the ScaDS to prove that it is possible to accomplish a lot with the right choice of technologies in a short time frame.

Weiterlesen ...

Big Data Cluster in “Shared Nothing” Architecture in Leipzig

The Galaxy Cluster

The state of Saxony funded a notable shared nothing cluster located at the University of Leipzig and the Technical University of Dresden. Here we want to give a short overview on this new “Galaxy” cluster which is a very nice asset for ScaDS.

Shared nothing is probably the most referenced architecture when talking about big data. The idea behind this cluster architecture is to use large amounts of commodity hardware to store and analyze big amounts of data in a highly distributed, scalable and cost effective way. It is optimized for massive parallel data oriented computations using e.g. Apache Hadoop, Apache Spark or Apache Flink.

Cluster Facts Overview:

Weiterlesen ...

Introduction to Privacy Preserving Record Linkage

 Many companies and organizations collect a huge amount of data about people simple by offering their services in form of online applications. Another “more official” way to gather such data is asking the people (by the mean of printed forms) as is the case in hospitals and administration. In both cases each data owner holds information that cover only one or few aspects of each person. However, analyzing such data and mining interesting patterns or improving decision making processes generally require clean and aggregated data, which are held by several organizations. Record linkage operates as a preprocessing step for these tasks with the main goal to find records, stored in different databases, which refer to the same real world object or person. This process finds application in many areas like healthcare, national security or business. In healthcare for example, linking records from two or more hospitals allows the adaptation of disease’s treatment of patients.

The main impediment when linking person related data across many organizations is the privacy aspect.  In several countries processing such data is subject to strict privacy policies, e.g. how and where to store the data and whether or not such data can be exchanged with a third party. Privacy Preserving Record Linkage (PPRL) presents techniques and methods to efficiently link similar records in different databases without compromising the privacy and confidentiality.

Weiterlesen ...

Webcrawling gebäuderelevanter Informationen

Für die Planung von Städten, Infrastruktur oder Energieversorgung  werden kleinräumige  Informationen  bis auf Gebäudeebene benötigt. Eine besondere Rolle spielen Referenzinformationen zur Gebäudeform, Nutzung, Baualter, Zustand oder der Geschossigkeit des Gebäudes, auf deren Basis mit Hilfe räumlicher Modellierungsansätze Verteilungsmuster von Wohnungen, Einwohnern, Arbeitsstätten  und  Infrastrukturen kartiert oder energetische Bedarfe abgeschätzt werden können. Die Erhebung dieser Referenzdaten ist allerding nur durch Ortsbegehung möglich und damit sehr aufwändig. Eine weitere Möglichkeit zur Sammlung dieser Informationen bietet die automatisierte Auswertung nutzergenerierter Inhalte und Bilder (z.B. OpenStreetMap, Mapillary, WikiMapia). Im Kontext von Gebäuden spielt hier WikiMapia eine besondere Rolle, da diese Platform neben Daten zur Gebäudenutzung auch georeferenzierte Street View Daten hinterlegt werden können. Über eine API lässt sich der Inhalt strukturiert auslesen.
Ziel war darum die Entwicklung eines WebCrawlers zum strukturierten Auslesen von gebäudebezogenen Inhalten. Dabei liegt das Hauptaugenemerk auf Eigenschaften wie Name, Art, Alter, etc. und natürlich auch auf den georeferenzierten Bildern, die für einen Teil der Gebäude vorliegen. Mit Hilfe des Programms wird zunächst selbständig eine Verbindung zum Wikimapia Tool aufgebaut. Über eine Abfrage kann vom Nutzer räumlich über Koordinaten oder auch semantisch durch Auswahl bestimmter Inhalten (z.B. mit Fokus auf bestimmte Gebäudetypen) die Inhalte ausgelesen und in einer Ausgabedatei gespeichert werden. Die Umsetzung des Programms erfolgte in der Programmiersprache Java unter Nutzung der Bibliotheken von und in der Entwicklungsumgebung Eclipse-Luna. Dem Anwender steht eine ausführbare .jar Datei zur Verfügung, die kommandozeilenbasiert mit dem Befehl java -jar wikimapia.jar ausführbar ist. Des Weiteren kann der Nutzer Parameter mitgeben, mit der die Abfrage räumlich und semantisch gesteuert wird. Das Programm läuft sowohl unter Linux als auch unter Windows (getestet unter Windows 7).



Moritz Haferburg: Diese E-Mail-Adresse ist vor Spambots geschützt! Zur Anzeige muss JavaScript eingeschaltet sein!

Robert Hecht: Diese E-Mail-Adresse ist vor Spambots geschützt! Zur Anzeige muss JavaScript eingeschaltet sein!

Versioning system for modeling environmental data based on an automatic meta-data generation strategy

The Helmholtz-Centre for Environmental Research (UFZ) is one of the world's leading research centres in the field of Earth system science. The Department of Environmental Informatics of the UFZ develops software for the simulation of environmental phenomena via coupled thermal, hydrological, mechanical and chemical processes by using innovative, numerical methods. Examples include the prediction of groundwater contamintion, the development of water management schemes or the simulation of innovative means of energy storage. The modeling process is a complete workflow, starting with data acquisition and -integration to process simulation to analysis and visualization of calculated results.

Unfortunately this modeling process is not transparent and traceable and often poorly documented. A typical model is developed over many weeks or months and usually  a large number of revisions are necessary for updating and refining the model such that the simulation is as exact as possible. The first setup of a model is often used to get an overview over existing data and to detect potential problems in both data and numerical requirements. Further revisions try to solve these problems by adding data, refining or adjusting finite element meshes or updating and ajusting processes and their parametrization. Both input- and parameter files range from few/small files up to hundreds of files containing detailed spatial, temporal or numerical information. Likewise, changes from one modeling step to the next may be small (e.g. one parameter value in a single input file) or major (e.g. geometrical input changes and requires a new discretization of the FEM domain as well as a new parameterization).

Weiterlesen ...

Dynamics of Open Quantum Systems

The description of the dynamics of open quantum systems is subject of ongoing research in theoretical quantum physics (solid state physics, quantum optics, quantum chemistry). Real (quantum) systems are never perfectly isolated from environmental fluctuations or forces. In case of weak environmental influence various approaches have been developed. By contrast this project focuses on the description of open quantum systems facing a significant influence of structured surroundings. Examples of experimental implementation can be found in energy transfer processes in molecular aggregates ([1],[2]) or quantum bits in solids ([3], [4]). Here, an exact and complete quantum mechanical description would be desirable. However, due to the exponential growth of Hilbert space dimensions of many-body quantum systems limits of computational resources are reached soon. We attack this challenge by means of a stochastic Schrödinger equation.

Weiterlesen ...

Multiskalen-Visualisierung – Der Schlüssel zu einem besseren Werkstoffverständnis

Bei der rechnergestützten Auslegung von Faserverbunden für Leichtbaustrukturen muss deren hierarchischer Aufbau berücksichtigt werden: ausgehend von Faser und Matrix, über den Roving, das Verstärkungstextil, die Einzelschicht bis hin zum Mehrschichtverbund. Dieser hierarchische Gedanke setzt sich über das Fügen von Komponenten zu einer Struktur und der Interaktion mehrerer Strukturen in einem System fort. Bei der Entwicklung müssen für jede dieser Skalen geeignete Simulationsmodelle bereitgestellt werden. Eine modellübergreifende, durchgängige Visualisierung der einzelnen Berechnungsergebnisse ist bisher jedoch nicht möglich.
Im Rahmen des Vorhabens „ScaDS Dresden/Leipzig - Competence Center for Scalable Data Services and Solutions“ haben das Institut für Leichtbau und Kunststofftechnik und die Professur für Computergraphik und Visualisierung (beide TU Dresden) eine browserbasierte Software entwickelt, die erstmals eine konsistente Visualisierung der Ergebnisse über alle Skalen hinweg erlaubt (Abbildung 1). Damit kann das Potenzial einer Multiskalen-Visualisierung zur Verbesserung des Werkstoffverständnisses aufgezeigt werden.  Grundlage sind die Simulationsdaten, die bei der Entwicklung einer adaptiven Blattfeder im SFB639 entstanden sind.
Im Video werden der Funktionsumfang sowie die Vorteile der browserbasierten Software aufgezeigt. Die Software wird einem breiten Spektrum an potenziellen Anwendern auf der Composite Europe – 11. Europäische Fachmesse & Forum für Verbundwerkstoffe, Technologien und Anwendungen, 29.11. - 1.12.2016, Messe Düsseldorf vorgestellt.  Dann können die Interessenten selbstständig die Software bedienen und sich einen eigenen Eindruck von deren Möglichkeiten machen.

Weiterlesen ...

Graph Mining für fortgeschrittene Datenanalysen

Um komplexe analytische Fragestellungen zu beantworten, werden Data Mining-Verfahren oft mit anderen Schritten der Datenverarbeitung kombiniert, zum Beispiel zur Vorbereitung des Suchraums oder zur Nachbearbeitung der Ergebnisse. Um die Kombination von Data Mining-Algorithmen mit anderen Operatoren zu ermöglichen, bieten produktive Lösungen zur Analyse relationaler oder multidimensionaler Daten meist umfangreiche Toolkits an. Anders ist das in Bezug auf die weniger etablierten Verfahren des Graph Mining. Hier existieren zwar Forschungsprototypen, aber keine Lösung, die komplexe analytische Programme aus mehren Graphoperationen unterstützt. Das an der Universität Leipzig und dem ScaDS Dresden/Leipzig entwickelte Open Source System Gradoop hat sich zum Ziel gesetzt, dies zu ändern. Gradoop ist das erste System, welches es ermöglicht, in einfachen Skripten ein oder mehrere Graph-Algorithmen mit weiteren vor- und nachgelagerten Graph-Operatoren zu kombinieren. Hierdurch werden neuartige Anwendungsfälle möglich, zum Beispiel die nachfolgend gezeigte Analyse von Geschäftsdaten. Durch den Einsatz aktueller Big Data-Technologie bietet Gradoop nicht nur einen einzigartigen Funktionsumfang, sondern ist auch out-of-the-box horizontal skalierbar. Zudem bietet es eine Schnittstelle für Plug in-Algorithmen und ist damit offen für anwendungsspezifische Erweiterungen.

Weiterlesen ...

Halloween Tutorial: How to do Vertex-Centric Iteration (Pregel) with Gelly

At the 2nd International ScaDS Summer school on Big Data we offered a couple of workshops with the aim to provide an introduction into the three Big Data technologies MongoDB, Flink and Gelly. This post is an extension of the Gelly tutorial to demonstrate the new feature of Gelly: the Vertex-Centric Iteration or Pregel Iteration. 

Find out which child is getting the largest amount of candies in our Halloween-Special of Trick-or-treat...

Weiterlesen ...