This is the documentation of the BIH high-performance compute (HPC), also called HPC 4 Research. The BIH HPC cluster is maintained by CUBI (Core Unit Bioinformatics).
This documentation is maintained by BIH CUBI and the user community. This is a living document that you can update and add to. See How-To: Contribute to this Document for details.
The global table of contents is on the left, it is on the right for the current page .
Connecting to the Cluster
- Web Access: https://hpc-portal.cubi.bihealth.org
# Interactive Login as Charite/MDC user ssh -l user_c hpc-login-1.cubi.bihealth.org # OR login-2... ssh -l user_m hpc-login-1.cubi.bihealth.org # OR login-2... # File Transfer as Charite/MDC user sftp firstname.lastname@example.org # OR hpc-transfer-2... sftp email@example.com # OR hpc-transfer-2... # Of course, you can also log into the transfer nodes (no Slurm) ssh -l user_c hpc-transfer-1.cubi.bihealth.org # OR hpc-transfer-2... ssh -l user_m hpc-transfer-1.cubi.bihealth.org # OR hpc-transfer-2...
To get started, the following is a suggested (in order) set of pages to read after your first successful connection to the cluster.
- Getting Access.
- Getting Help (Writing Good Tickets; if no answer found, contact the HPC Helpdesk).
- For the Impatient.
- The Cluster Tutorial: First Steps.
Then, continue reading through the manual.
Acknowledging BIH HPC Usage
Acknowledge usage of the cluster in your manuscript as "Computation has been performed on the HPC for Research/Clinic cluster of the Berlin Institute of Health". Please add your publications using the cluster to this list.
Current and Future Maintenances
- August 30: Replace defect Nvidia V100 and make hpc-gpu-4 available to slurm users again.
- March 9: Updated TensorFlow How-To to Slurm and Tensorflow 2.0
- March 22-23: Compute and Storage Downtime.
- March: Deprecation of using DRMAA.
- March 1: New scheduler settings to address high job per user count.
- February 14: Limiting allocateable memory per user.
- February 3: Adding Ganglia documentation.
- February 3: Ganglia monitoring of GPFS and NVIDIA GPU metrics.
- January 31: Enforcing usage of
localtmpresource above 100MB of node-local storage.
See Maintenance for a detailed list of current, planned, and previous maintenance and update work.
Connecting to the Cluster¶
You will need to perform some configuration steps after you have been registered with the cluster (via email to firstname.lastname@example.org by a group leader/PI). Here are the most important points:
- Generating SSH Keys in Linux or Windows.
- Submitting the key to Charite or to MDC.
- Configuring your SSH client on Linux and Mac or Windows.
- Bonus: Connecting from external networks .
There are various other topics covered in the "Connecting" section that might be of interest to you.
The documentation is structured as follows:
- Administrative information about administrative processes such as how to get access, register users, work groups, and projects.
- Getting Help explains how you can obtain help in using the BIH HPC.
- Overview detailed information about the cluster setup. This includes the description of the hardware, network, software, and policies.
- Connecting technical help on connecting to the cluster.
- First Steps information for getting you started quickly.
- Slurm Scheduler technical help on using the Slurm scheduler.
- Best Practice guidelines on recommended usage of certain aspects of the system.
- Static Data documentation about the static data (files) collection on the cluster.
- How-To short(ish) solutions for specific technical problems.
- Miscellaneous contains a growing list of pages that don't fit anywhere else.