Incident description

Service Incident status Start Date End Date
Nibi Closed
Created by Mark Hahn on

Title


Scheduler problem - Problème d'ordonnanceur


Summary


The job scheduler is misbehaving, which may impair the ability to start new jobs. We are investigating. / L'ordonnanceur a un problème, ce qui peut empêcher de démarrer de nouvelles tâches. Nous investiguons actuellement ce qui se passe.

The ultimate trigger of this problem seems to have been a power issue early this morning.  It caused a cascade of effects involving VMs being moved between hypervisors, some VMs rebooting, possibly some storage issues, and affected a number of daemons (sssd, munge, slurmd on compute nodes, slurm controller and DB VMs) that would give user-visible errors.


Updated by Mark Hahn on