Introduction to BigData tools

by Michael Waumans (ULB)

Tuesday 19 Nov 2019, 09:45 → 12:45 Europe/Brussels

Pasteur (Bibliothèque des sciences et technologies (BST) )

High-performance compute and BigData are two disciplines that used to be higly compartmented, but are not anymore. More and more HPC users get interested in BigData software, and more and more BigData users get interested in HPC hardware. This session will broadly present the tools and concepts associated with BigData and explain how they can be used in the CÉCI context.
Contents: MapReduce and Spark Zeppelin and Jupyter BigData on HPC	Prerequisite: Being able to use SSH with private keys  Being familiar with a text editor  Mastering the Linux command line and the GNU utilities (mkdir, cp, scp, etc.)
	Type: Hands-on Target audience: Beginner Must: This session is a must-have for anyone who would like to go see what exist behind the buzzword.

CECI - 0 - Introduction.ipynb

CECI - 1 - HDFS.ipynb

CECI - 2 - Map Reduce.ipynb

CECI - 3 - Spark.ipynb

CECI - 4 - Kafka.ipynb

CECI - 5 - SparkStreaming.ipynb

Introduction.pdf