Half a cluster's energy went on jobs that failed.
On a shared cluster, jobs fail part-way through on a bad input, a memory limit or an unhandled edge case, then get fixed and resubmitted. A study of a Dutch national cluster found about half its energy went on jobs that ended unsuccessfully.
Researchers in Amsterdam and Vienna found that half of one cluster's energy went on jobs that failed. GreenCode is exploring checks like this for jobs on shared clusters.
GreenCode helps tackle this by establishing a repeatable energy baseline for research software and measuring again after each optimisation. A university research computing cluster is one of the project's use cases. More on this.
Half of one cluster's energy went on jobs that failed. On a shared computer cluster, long jobs fail part-way through. A bad input, a memory limit, or a case nobody handled. Then they are fixed and sent again. Researchers in Amsterdam and Vienna studied one Dutch national cluster. About half its energy went on jobs that ended in failure. Here is the fix you can use today. Run on a small input first. Set realistic memory and time limits, and check inputs before the long run. Save checkpoints, so a failure does not restart from zero. A job sent three times has run four times. Estimate a run's carbon first, using tools like the Green Algorithms calculator and CodeCarbon. Afterwards, Slurm's energy accounting shows the energy each job used. GreenCode helps tackle this by measuring research code's energy before and after each fix. A university research cluster is one of its test beds. That was GreenCode Tech Tip number thirty. Follow for the next tip, and find more on greencode.ai/tech-tips.
Share this
