Introduction to data storage and access

• Damien François (UCLouvain/CISM)

Room: CYCL E349 <table border="0" cellpadding="10px"> <tbody> <tr> <td colspan="2"> <p>Storing data in an efficient way is very important for many scientific applications. Yet, most of the time, a myriad of small files is used, imposing a large burdun on the file system, spending a lot of time in file access, making transfers very inefficients, etc. Other solutions exist and are presented in this session.</p> </td> </tr> <tr> <td rowspan="2"> <p><strong>Contents:</strong></p> <ul> <li>Storing in files vs in database</li> <li>Using an in-memory database</li> <li>Using HDF5 CLI tools and libraries</li> </ul> </td> <td> <p><strong>Prerequisite:</strong></p> <ul> <li>Being able to use SSH with private keys&nbsp;</li> <li>Being familiar with a text editor&nbsp;</li> <li>Mastering the Linux command line and the GNU utilities (mkdir, cp, scp, etc.)</li> <li>Passive knowledge of either C, Fortran, Octave, Python or R</li> <li>Working knowledge of C or Fortran</li> <li>Familiarity with OpenMP and MPI</li> </ul> </td> </tr> <tr> <td> <p><strong>Type:</strong>&nbsp;Hands-on<br /> <strong>Target audience</strong>: Everyone<br /> <strong>Must:&nbsp;</strong>This session is a must-have for anyone who thinks generating a million small files is an optimal way of storing data.</p> </td> </tr> </tbody> </table>

Efficient data storage on CECI clusters

• Ariel Lozano (ULB)

Room: CYCL E349