ScaDS Logo

COMPETENCE CENTER
FOR SCALABLE DATA SERVICES
AND SOLUTIONS

Halloween Tutorial: How to do Vertex-Centric Iteration (Pregel) with Gelly

Beitragsseiten

At the 2nd International ScaDS Summer school on Big Data we offered a couple of workshops with the aim to provide an introduction into the three Big Data technologies MongoDB, Flink and Gelly. This post is an extension of the Gelly tutorial to demonstrate the new feature of Gelly: the Vertex-Centric Iteration or Pregel Iteration. 

Find out which child is getting the largest amount of candies in our Halloween-Special of Trick-or-treat...

Introduction

Three months after the 2nd International ScaDS Summer school on Big Data, a new stable version of Flink is available. The version 1.1 involves many bug fixes and new features. Along with the new Flink version, also a new Gelly version was released. In this blog post, we will highlight and demonstrate a new feature of the new Gelly version: the Pregel iteration or how Gelly it calls the Vertex-Centric Iteration. It is the third iteration that can be used in Gelly and more general model then the two older types of iterations. The idea was already published in the year 2010 as Pregel: a system for large-scale graph processing, many algorithm are published using this iteration model.

We will explain the Pregel iteration using a halloween example. We first introduce the story of our example and continue afterwards with the implementation in Gelly. Note that only the code relevant for the Pregel iteration is shown but not how to use Flink and Gelly in general. However, the runnable code of the tutorial can be found at the end of the blog post as download.