> Datacenters run at roughly 30% to 40% effective utilisation
I wonder what is stopping datacenters from passing this benefit to customers by launching better tuned plans. For example, t series EC2 instances on AWS.
This is a cool idea—I know from snooping on sumbit scripts and node utilization on the HPC that I use at my institution that most submissions leave some compute on the table (and many of them are egregiously bad). I'd probably vote in favor of sending every submitted sbatch script through an LLM (at least for everyone else, I'd would prefer tuning my own usage myself :) ).
Presumably the underlying model here is also an LLM? To what degree is it "fine-tuned", or is it just given a set of tools to build a good picture of cluster usage?
One traditional enterprise goal of 40% utilization was to cover DR/failovers, so one region could take on 100% of traffic from another, with 20% headroom.
I'm curious about the granularity of contracts around granting/selling excess capacity. Are they short term? Can the owner evict those workloads (with a penalty)?
Good point - people do set capacity aside, reserving it for later.
But our utilisation measurements are from waste within a users allocation. It’s waste of what users are actually requesting and running, not from any reserved idle capacity.
For now we sit only on the prediction/intelligence layer; we don’t do any scheduling. We don’t grant or sell capacity, we just tell the scheduler (and user) what a job actually needs.
boringperson | 2 hours ago
I wonder what is stopping datacenters from passing this benefit to customers by launching better tuned plans. For example, t series EC2 instances on AWS.
keremimo | 2 hours ago
aleksiy123 | an hour ago
I feel like it’s probably just complexity.
Different workloads benefit from specific types of optimisations.
rjpruitt16 | an hour ago
https://www.linkedin.com/posts/rahmi-pruitt-a1bb4a127_agentn...
ray__ | an hour ago
Presumably the underlying model here is also an LLM? To what degree is it "fine-tuned", or is it just given a set of tools to build a good picture of cluster usage?
flounder3 | 25 minutes ago
I'm curious about the granularity of contracts around granting/selling excess capacity. Are they short term? Can the owner evict those workloads (with a penalty)?
[OP] ismaeel_bashir | 10 minutes ago
But our utilisation measurements are from waste within a users allocation. It’s waste of what users are actually requesting and running, not from any reserved idle capacity.
For now we sit only on the prediction/intelligence layer; we don’t do any scheduling. We don’t grant or sell capacity, we just tell the scheduler (and user) what a job actually needs.
syngrog66 | 10 minutes ago