> Apple’s attestation servers will
only generate the FreshnessCode for a genuine device that
checks in via APNs. A software-only adversary cannot
forge the MDA certificate chain (Assumption 3). Com-
bined with SIP enforcement (preventing binary replace-
ment) and Secure Boot (preventing bootloader tampering),
this provides strong evidence that the signing key resides
in genuine Apple hardware.
Epyc has that VM encrypted memory thing, which comes pretty close. It does raise an interesting question, though: would a PCIe card passed through to a VM be able to DMA access the memory of neighboring devices?
Away this looks like a great idea and might have a chance at solving the economic issue with running nodes for cheap inference and getting paid for it.
They use the TEE to check that the model and code is untampered with. That's a good, valid approach and should work (I've done similar things on AWS with their TEE)
The key question here is how they avoid the outside computer being able to view the memory of the internal process:
> An in-process inference design that embeds the in-
ference engine directly in a hardened process, elimi-
nating all inter-process communication channels that
could be observed, with optional hypervisor mem-
ory isolation that extends protection from software-
enforced to hardware-enforced via ARM Stage 2 page
tables at zero performance cost.[1]
I was under the impression this wasn't possible if you are using the GPU. I could be misled on this though.
While they do make this argument, realistically anyone sending their prompt/data to an external server should assume there will be some level of retention.
And more so in particular, anyone using Darkbloom with commercial intents should only really send non-sensitive data (no tokens, customer data, ...) I'd say only classification tasks, imagine generation, etc.
> PT_DENY_ATTACH (ptrace constant 31): Invoked
at process startup before any sensitive data is loaded.
Instructs the macOS kernel to permanently deny all
ptracerequests against this process, including from
root. This blocks lldb, dtrace, and Instruments.
> Hardened Runtime: The binary is code-signed with
hardened runtime options and explicitly without the
com.apple.security.get-task-allow
entitlement. The kernel denies task_for_pid()
and mach_vm_read()from any external process.
> System Integrity Protection (SIP): Enforces both of
the above at the kernel level. With SIP enabled, root
cannot circumvent Hardened Runtime protections, load
unsigned kernel extensions, or modify protected sys-
tem binaries. Section 5.1 proves that SIP, once verified,
is immutable for the process lifetime.
Apple is perfectly capable of doing remote attestation properly. iOS has DCAppAttest which does everything needed. Unfortunately, it's never been brought to macOS, as far as I know. Maybe this MDM hack is a back door to get RA capabilities, if so it'd certainly be intriguing, but if not as far as I know there's no way to get a Mac to cough up a cryptographic assertion that it's running a genuine macOS kernel/boot firmware/disk image/kernel args, etc.
It's a pity because there's a lot of unique and interesting apps that'd become possible if Apple did this. Darkbloom is just one example of what's possible. It'd be a huge boon to decentralization efforts if Apple activated this, and all the pipework is laid already so it's really a pity they don't go the extra mile here.
Looking at their paper at [1], there's a gaping hole: there's no actual way to verify the contents of the running binaries. The binary hash they include in their signatures is self-reported, and can be modified. That's simply game over.
A note, as others have posted on this thread: I mention this as a concrete and trivial flaw in their whole strategy, but the issue is fundamental: there's no hardware enclave for third-party code available to do the type of attestation that would be necessary. Any software approach they develop will ultimately fall to that hole.
This entire paper smells of LLM, I'm sure even the most distinguished academic would refrain from using notation to prove that the SIP status cannot change during operation.
I have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?
I guess if it only works at scale capital is maybe the answer. Like enough cash to buy 5 or 10 or even 100 minis seem doable - but if the idea only works well when you have 10,000 running - that makes some sense.
There are many people who do not have ready access to a million dollars to purchase said Mac minis, much less the operating capital to rack & operate them.
Very smart play to build a platform, get scale, and prove out the software. Then either add a small network fee (this could be on money movement on/off platform), add a higher tier of service for money, and/or just use the proof points to go get access to capital and become an operator in your own pool.
If those numbers are true, they could tart with one Mac and can double every few months. But, I guess there are also many people who do not have ready access to whatever a Mac mini costs either...
Solid q. I think the part of it is that it’s really easy to attract some “mass” (capital) of users, as there are definitely quite a few of idle Macs in the world.
Non-VC play (not required until you can raise on your own terms!) and clear differentiation.
If you want to go full-business-evaluation, I would be more worried about someone else implementing same thing with more commission (imo 95% and first to market is good enough).
I think the point they’re making though is that the numbers seem too good to be true.
ie. Does anyone know the payback time for a B100 used just for inference? I assume it’s more than a couple of months? Or is it just training that costs so much?
> These are estimates only. We do not guarantee any specific utilization or earnings. Actual earnings depend on network demand, model popularity, your provider reputation score, and how many other providers are serving the same model.
The numbers are obviously high, because if this takes off then the price for inference will also drop. But I still think it’s a solid economic model that benefits low income countries the most. In Ukraine, for example, I know people who live on $200/month. A couple Mac Minis could feed a family in many places.
As a business owner, I can think of multiple reasons why a decentralized network is better for me as a business than relying on a hyperscaler inference provider. 1. No dependency on a BigTech provider who can cut me off or change prices at any time. I’m willing to pay a premium for that. 2. I get a residential IP proxy network built-in. AI scrapers pay big money for that. 3. No censorship. 4. Lower latency if inference nodes are located close to me.
They already have an iPhone. They could save up or borrow for a Mac Mini if they had to. Some of those people I know who live on $200/month have $30k in the bank.
Not really. There are lots of people who have low income and low spending, but not low savings. Retired pensioners with savings. Young families who inherited from deceased parents/grandparents. Highly paid professionals on sabbatical. I've met people from all of those categories in Ukraine who live on $200/month.
On the latency point - your requests are still going through the coordinator of the system here. So on average strictly worse than a large provider.
You - Darkbloom - Operator - Darkbloom - you, vs
You - Provider - you
---
On the censorship point - this is an interesting risk surface for operators. If people are drawn my decentralized model provisioning for its lax censorship, I'm pretty sure they're using it to generate things that I don't want to be liable for.
If anything, I could imagine dumber and stricter brand-safety style censorship on operator machines.
I'm not talking about Darkbloom specifically, but rather this business model in general. I'm sure a future version of Darkbloom could be P2P for better latency. Or their central operator nodes could be geo-balanced. Liability for censorship doesn't matter if it's truly zero trust. Anyway censorship is not my main concern. Low-latency decentralized inference with no US BigTech dependency is a much bigger selling point in Europe.
Isn't this same premise as "lets buy few GPUs to mine crypto and have passive income"?
It didn't work very well and it probably won't work now either. If there is money to be made, bigger players will get in there, buy out all mac minis they can, drive price up for regular people and inevitably drive inference price down so you'll be lucky to get initial investment back
No it's not the same premise at all. Crypto doesn't do anything useful for legitimate businesses. AI inference is very useful for legitimate businesses, and so are residential IP proxies for scraping. And by definition, residential IPs cannot be centralized. And as building GPUs becomes more expensive, the existing pool of second hand unused hardware becomes more valuable, not less. The problem with crypto mining is that it quickly becomes unprofitable for small scale deployments. I'm not sure if AI inference would be, especially for the decentralized benefits of lower latency.
It is the same premise, because the person you are responding to is not talking about the moral implications at all, only about the financial / hardware implications.
Running AI inference increases the power draw, and requires certain hardware.
Mining bitcoin increases the power draw, and requires certain hardware.
OP's point thus stands: Bad players will find places to get far cheaper power than the intended audience, and will buy dedicated hardware, at which point the money you can earn to do this will soon drop below the costs for power (for folks like you and me).
Maybe that won't happen, but why won't that happen?
The main problem with crypto is there is no universal need for it. The demand for crypto doesn’t keep increasing as compute gets cheaper. But the demand for AI inference is only growing, and making it cheaper would likely only increase demand. So it’s not a race to the bottom. Sure hyper focused players can earn more at higher margins. But average players can probably still earn decently. Take for example electricity. It can still be profitable for a home in Germany to install balcony solar and make a little money selling back to the grid even though it’s obviously not as efficient as an industrial power plant. Mom and pop AI inference don’t have to be super efficient as long as they serve a universal need - it will be like balcony solar in Europe.
Because they don’t have that much initial money in their pocket, while the idle computer is already there, and the biggest friction point is convincing people to install some software. Both producing rhetoric and software are several order of magnitude cheaper than to directly own and maintain a large fleet of hardware with high guarantee of getting the electrical stable input in a safe place to store them.
Assuming that getting large chunk of initial investment is just a formality is out of touch with 99% of people reality out there, when it’s actually the biggest friction point in any socio-economical endeavour.
The numbers are optimistically legit -- it's calculated based purely considering we have demand for all machines at all times. We don't have that right now, but fairly optimistic that people will do it.
That's why we don't recommend purchasing a new machine. Existing machine is no cost for you to run this.
Electricity is one cost, but it will get paid off from every request it receives. Electricity is only deducted when you run an inference. If you have any questions, DM me @gajesh on Twitter.
You're not taking into account the thermal strain on the machine, though. A machine that's 100% utilized (even worse if it's in bursts) will last less than an idle machine.
> A machine that's 100% utilized (even worse if it's in bursts) will last less than an idle machine.
How much though? Say I have three Mac Minis next to each other, one that is completely idle but on, one that bursts 100% CPU every 10 minutes and one that uses 100% CPU all the time, what's the difference on how long the machines survives? Months, years or decades?
That solution actually makes great sense. So Apple won in some strange way again?
Guess there are limitations on size of the models, but if top-tier models will getting democratized I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns.
I think batch-evals for non-sensitive data has great PMF here.
Heh, what did they win exactly? This is just a way for another company to extract value out of the single region of the world where Apple is a relevant vendor, and it happens to be the one where it's the easiest to pull people into schemes.
I'd love a way to do this locally -- pool all the PCs in our own office for in-office pools of compute. Any suggestions from anyone? We currently run ollama but manually manage the pools
Seems like so much more work than "just" paying for https://huggingface.co or whichever other neocloud who already did all the setup for you and just waits for your credit card per minute/seconds/token.
If you set CPUSchedulingPolicy=idle Nice=19 IOSchedulingClass=idle in the ollama server configuration it should run in the background with lowest priority.
OpenAI has only about 5% paying customers, how does it generate revenue?
I don’t think this is a sustainable business model. For example, Cubbit tried to build decentralised storage, but I backed out because better alternatives now exist, and hardware continues to improve and become cheaper over time.
Your electricity and ownership are going to get lower return and does not actually requce CO2.
Yeah, only way to get there is assuming they're not giving prompt caching discounts while my laptop is getting prompt caching benefits, with very many large prompts. So yes I am skeptical of their numbers.
Any idea what makes for such a diff between your and theirs numbers? Batching? Or could they do a crazy prefix caching across all nodes to reduce the actual processing.
Well. Running your machine to do inference will utilize more than 50W sustained load, I'd say more than double that. Plus electricity is more expensive here (but granted, I do have solar panels). Plus don't forget to factor in that your hardware will age faster.
Their estimate is based on significantly lower consumption when under load. E.g. 25W for an M4 Pro mac mini. I have no idea if that’s realistic - but the m4s are supposedly pretty efficient (https://www.jeffgeerling.com/blog/2024/m4-mac-minis-efficien...)
Your hardware will age slower if you have consistent load.
Thermal stress from bursty workloads is much more of a wearing problem than electromigration. If you can consistently keep the SoC at a specific temperature, it'll last much longer.
This is also why it was very ironic that crypto miner GPUs would get sold at massive discounts. Everyone assumed that they had been ran ragged, but a proper miner would have undervolted the card and ran it at consistent utilization, meaning the card would be in better condition than a secondhand gamer GPU that would have constantly been shifting between 1% to 80% utilization, or rather, 30°C to 75°C
Their example big earner models are FLUX.2 Klein 4B and FLUX.2 Klein 9B, which i imagine could generate a lot more tokens/s than a 26B model on your machine.
I have no idea if that is a good estimate of how much an M5 Pro can generate - but that’s what it says on their site.
They do a bit of a sneaky thing with power calculation: they subtract 12Ws of idle power, because they are assuming your machine is idling 24/7, so the only cost is the extra 18W they estimate you’ll use doing inference. Idk about you, but i do turn my machine off when i am not using it.
Unfortunately, verifiable privacy is not physically possible on MacBooks of today. Don't let a nice presentation fool you.
Apple Silicon has a Secure Enclave, but not a public SGX/TDX/SEV-style enclave for arbitrary code, so these claims are about OS hardening, not verifiable confidential execution.
It would be nice if it were possible. There's a lot of cool innovations possible beyond privacy.
As if you get privacy with the inference providers available today? I have more trust in a randomly selected machine on a decentralized network not being compromised than in a centralized provider like OpenAI pinky promising not to read your chats.
I would say the chances of OpenAI itself getting hacked and your secrets in logs getting leaked are about the same or less as the chances of a randomly selected machine on a decentralized network being reverse-engineered by a determined hacker. There's no risk-free option, every provider comes with risks. If you care about infosec you have to do frequent secret rotation anyway.
I wrote a whole SDK for using SGX, it's cool tech. But in theory on Apple platforms you can get a long way without it. iOS already offers this capability and it works OK.
macOS has a strong enough security architecture that something like Darkbloom would have at least some credibility if there was a way to remotely attest a Mac's boot sequence and TCC configuration combined with key-to-DR binding. The OS sandbox can keep apps properly separated if the kernel is correct and unhacked. And Apple's systems are full of mitigations and roadblocks to simple exploitation. Would it be as good as a consumer SGX enclave? Not architecturally, but the usability is higher.
"These are estimates only. We do not guarantee any specific utilization or earnings. Actual earnings depend on network demand, model popularity, your provider reputation score, and how many other providers are serving the same model.
When your Mac is idle (no inference requests), it consumes minimal power — you don't lose significant money waiting for requests. The electricity costs shown only apply during active inference.
Text models typically see the highest and most consistent demand. Image generation and transcription requests are bursty — high volume during peaks, quiet otherwise."
I installed this so you don't have to. It did feel a bit quirky and not super polished. Fails to download the image model. The audio/tts model fails to load.
In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations.
At the moment they don't have enough sustained demand to justify the earning estimates.
and I don't think they ever will unless they're highly competitive (hopefully that price they have stays? at least for users)
I was thinking of building this exact thing a year ago but my main stopper was economics: it would never make sense for someone to use the API, thus nobody can make money off of zero demand.
I guess we just have to look at how Uber and Airbnb bootstrapped themselves. Another issue with my original idea was that it was for compute in general, when the main, best use-case, is long(er)-running software like AI training (but I guess inference is long running enough).
But there already exist software out there that lets you rent out your GPU so...
People underestimate how efficient cost/token is for beefy GPUs if you are able to batch. Unlikely for one off consumer unit to be able to compete long term.
weird to learn that they do not generate inference requests to their network themselves to motivate early adopters at least to host their inference software
If they paid promised > $1k/m for FLUX 2B on a Mac they would go broke in less than a month. On a single 5090 that model would provide an inference througput so high they'd have to pay close to $50k/m for the results.
The numbers are absolute fraud. You shouldn't be installing their software cause fraud could be not just about numbers.
Can you rephrase that? I don't think I've read it correctly. It sounds like you are saying it would normally cost $50k on a 5090 and they can do equivalent work paying $1k. That's sounds like a $49k profit margin, but you say they will go broke.
I kind of see your point, but I also kind of don't.
Sure, it would be great if you'd immediately get hammered with hundreds of requests and start make money quickly. It would also be great if it was a bit more transparent, and you could see more stats (what counts as "idle"? Is my machine currently eligible to serve models?). But it's still very new, I'd say give it some time and let's see how it goes.
If you have it running and you get zero requests, it uses close to zero power above what your computer uses anyway. It doesn't cost you anything to have it running, and if you get requests, you make money. Seems like an easy decision to me.
Bootstrapping will be near-impossible (or incredibly costly) unless they offer inference consumers models with established demand arriving at some least-cost router service where they can undercut the competition (if they actually can). And then dogfood the opportunistic provider side on their own Macs, but with a preference to putting third parties first in the queue. Everything else is just wishful thinking.
They are almost claiming FHE, isn't it just a matter of creating the right tool to get the generated tokens from RAM before it gets encrypted for transfer. How is it fundamentally different than chutes?
I'm not sure how the economics works out. Pricing for AI inference is based on supply/demand/scarcity. If your hardware is scarce, that means low supply; combine with high demand, it's now valuable. But what happens if you enable every spare Mac on the planet to join the game? Now your supply is high, which means now it's less valuable. So if this becomes really popular, you don't make much money. But if it doesn't become somewhat popular, you don't get any requests, and don't make money. The only way they could ensure a good return would be to first make it popular, then artificially lower the number of hosts.
Three binaries and a Python file:
darkbloom (Rust)
eigeninference-enclave (Swift)
ffmpeg (from Homebrew, lol)
stt_server.py (a simple FastAPI speech-to-text server using mlx_audio).
The good parts:
All three binaries are signed with a valid Apple Developer ID and have Hardened runtime enabled.
Bad parts:
Binaries aren't notarized. Enrolls the device for remote MDM using micromdm. Downloads and installs a complete Python runtime from Cloudflare R2 (Supply chain risk). PT_DENY_ATTACH to make debugging harder. Collects device serial numbers.
You have to install their MDM device management software on your computer. Basically that computer is theirs now. So don't plan on just handing over your laptop temporarily unless you don't mind some company completely owning your box. Still might be a validate use for people with slightly old laptops lying around, but beware trying to share this computer with your daily activities if you e.g. use a bank on a browser on this computer regularly. MDM means they can swap out your SSL certs level of computer access, please correct me if I'm wrong.
MDMs on macOS are permissioned via AccessRights, and you can verify that their permission set is fairly minimal and does not allow what you've described here (bits 0, 4, 10).
That said, their privacy posture at the cornerstone of their claims is snake oil and has gaping holes in it, so I still wouldn't trust it, but it's worth being accurate about how exactly they're messing up.
You are right - the "nonce binding" the paper uses doesn't seem convincing. The missing link is that Apple's attestation doesn't bind app generated keys to a designated requirement, which would be required to create a full remote attestation.
> If you can prove a public key is generated by the SEP of a machine running with all Apple's security systems enabled, then you can trivially extend that to confidential computing because the macOS security architecture allows apps to block external inspection even by the root user.
It only effectively allows this for applications that are in the set of things covered by SIP, but not for any third-party application. There's nothing that will allow you to attest that arbitrary third-party code is running some specific version without being tampered with, you can only attest that the base OS/kernel have not been tampered with. In their specific case, they attempt to patch over that by taking the hash of the binary, but you can simply patch it before it starts.
To do this properly requires a TEE to be available to third-party code for attestation. That's not a thing on macOS today.
I wiped my post because you are right. I don't think it needs a full SGX-style TEE. What's missing is a link to designated requirements. Abusing a nonce field doesn't seem to work, or if it does I can't figure out how. The MDM/MDA infrastructure would need to be able to include:
public key from SEP -> designated requirement of owning app binary
The macOS KeyStore infrastructure does track this which is why I thought it'd work. But the paper doesn't mention being able to get this data server side anywhere. Instead there's this nonce hack.
It's odd that the paper considers so many angles including things like RDMA over Thunderbolt, but not the binding between platform key and app key.
Reading the paper again carefully I get the feeling the author knows or believes something that isn't fully elaborated in the text. He recognizes that this linkage problem exists, proposes a solution and offers a security argument for it. I just can't understand the argument. It appears APNS plays a role (apple push notification service) and maybe this is where app binding happens but the author seems to assume a fluency in Apple infrastructure that I currently lack.
I can buy the idea that if you can have the MDM infrastructure attest the code signing identity through the designated requirements, that you can probably come pretty close, but I'm still not quite sure you get there with root on macOS (and I suspect that this is part of why DCAppAttest hasn't made it to macOS yet).
Certainly, it still doesn't get you there with their current implementation, as the attempts at blocking the debugger like PT_DENY_ATTACH are runtime syscalls, so you've got a race window where you can attach still. Maybe it gets you there with hardened runtime? I'd have to think a bit harder on that.
Yeah I didn't quite understand the need for PT_DENY_ATTACH. Hardened runtime apps that don't include get-task-allow are already protected from debugger attach from the start of the process, unless I misunderstood something.
I'm not quite sure why Apple haven't enabled DCAppAttest on macOS. From my understanding of the architecture, they have every piece needed. It's possible that they just don't trust the Mac platform enough to sign off on assertions about it, because it's a lot more open so it's harder to defend. And perhaps they feel the reputational risk isn't worth it, as people would generalize from a break of App Attest on macOS to App Attest on iOS where the money is. Hard to say.
Interesting to see an offering with this heritage [1] proposing flat earnings rates for inference operators here, rather than trying to sell a dynamic marketplace where operators compete on price in real-time.
Right now the dashboards show 78 providers online, but someone in-thread here said that they spun one up and got no requests. Surely someone would be willing to beat the posted rate and swallow up the demand?
I expect this is a migration target, but a tactical omission from V1 comms both for legitimate legibility reasons (I can sell x for y is easier to parse than 'I can participate in a marketplace') and slightly illegitimate legibility reasons (obscuring likely future price collapse).
Still - neat project that I hope does well.
[1] Layer Labs, formerly EigenLayer, is company built around a protocol to abstract and recycle economic security guarantees from Ethereum proof of stake.
Love the concept, with some similarity to folding@home, though more personal gain.
But trying it out it still needs work, I couldn't download a model successfully (and their list of nodes at https://console.darkbloom.dev/providers suggests this is typical).
And as a cursory user, it took me some digging to find out that to cash out you need a Solana address (providers > earnings).
I think it’s important that systems like this exist, but getting them off the ground is non-trivial.
We’ve been building something similar for image/video models for the past few months, and it’s made me think distribution might be the real bottleneck.
It’s proving difficult to get enough early usage to reach the point where the system becomes more interesting on its own.
Curious how others have approached that bootstrap problem. Thanks in advance.
Having strong SETI@Home vibes from 25 years ago, except of course, this is not for the greater good of humanity, but a for-profit project.
Problem is, from a technical point of view, what kind of made sense back then (most people running desktops, fans always on, energy saving minimal) is kind of stupid today (even if your laptop has no fan, would you want it to be always generating heat?)...
I definitely want my laptops to be cool, quiet and idle most of the time.
I some times play about with local models via ollama/comfyui and more recently ace-step to generate music.
This is short bursts of heat 5-10 m during the render I would not be happy with that for multiple hours a day. I am sure that would have a negative effect on battery health.
Hey Gajesh! I sent you an email with some of the teething problems I ran into trying to get started as a provider. Hope it didn't end up in your spam folder!
DeathArrow | 6 hours ago
stryakr | 6 hours ago
btown | 6 hours ago
> Apple’s attestation servers will only generate the FreshnessCode for a genuine device that checks in via APNs. A software-only adversary cannot forge the MDA certificate chain (Assumption 3). Com- bined with SIP enforcement (preventing binary replace- ment) and Secure Boot (preventing bootloader tampering), this provides strong evidence that the signing key resides in genuine Apple hardware.
saagarjha | 4 hours ago
nl | 6 hours ago
NVidia data center GPUs have a similar path, but not their consumer ones. Not sure about the NVidia Spark.
It's possible AMD Strix Halo can do this, but unlikely for any other PC based GPU environments.
MrDrMcCoy | 6 hours ago
rvz | 6 hours ago
Away this looks like a great idea and might have a chance at solving the economic issue with running nodes for cheap inference and getting paid for it.
nl | 6 hours ago
The key question here is how they avoid the outside computer being able to view the memory of the internal process:
> An in-process inference design that embeds the in- ference engine directly in a hardened process, elimi- nating all inter-process communication channels that could be observed, with optional hypervisor mem- ory isolation that extends protection from software- enforced to hardware-enforced via ARM Stage 2 page tables at zero performance cost.[1]
I was under the impression this wasn't possible if you are using the GPU. I could be misled on this though.
[1] https://github.com/Layr-Labs/d-inference/blob/master/papers/...
flockonus | 6 hours ago
And more so in particular, anyone using Darkbloom with commercial intents should only really send non-sensitive data (no tokens, customer data, ...) I'd say only classification tasks, imagine generation, etc.
joelthelion | an hour ago
ramoz | 6 hours ago
Macs have secure enclaves.
nl | 6 hours ago
But they argue that:
> PT_DENY_ATTACH (ptrace constant 31): Invoked at process startup before any sensitive data is loaded. Instructs the macOS kernel to permanently deny all ptracerequests against this process, including from root. This blocks lldb, dtrace, and Instruments.
> Hardened Runtime: The binary is code-signed with hardened runtime options and explicitly without the com.apple.security.get-task-allow entitlement. The kernel denies task_for_pid() and mach_vm_read()from any external process.
> System Integrity Protection (SIP): Enforces both of the above at the kernel level. With SIP enabled, root cannot circumvent Hardened Runtime protections, load unsigned kernel extensions, or modify protected sys- tem binaries. Section 5.1 proves that SIP, once verified, is immutable for the process lifetime.
gives them memory protection.
To me that is surprising.
ramoz | 5 hours ago
Protection here is conditional, best-effort. There are no true guarantees, nor actual verifiability.
dinobones | 5 hours ago
If it's not running fully end to end in some secure enclave, then it's always just a best effort thing. Good marketing though.
saagarjha | 4 hours ago
mike_hearn | 3 hours ago
Apple is perfectly capable of doing remote attestation properly. iOS has DCAppAttest which does everything needed. Unfortunately, it's never been brought to macOS, as far as I know. Maybe this MDM hack is a back door to get RA capabilities, if so it'd certainly be intriguing, but if not as far as I know there's no way to get a Mac to cough up a cryptographic assertion that it's running a genuine macOS kernel/boot firmware/disk image/kernel args, etc.
It's a pity because there's a lot of unique and interesting apps that'd become possible if Apple did this. Darkbloom is just one example of what's possible. It'd be a huge boon to decentralization efforts if Apple activated this, and all the pipework is laid already so it's really a pity they don't go the extra mile here.
saagarjha | 4 hours ago
mirashii | 4 hours ago
[1] https://github.com/Layr-Labs/d-inference/blob/master/papers/...
mirashii | 3 hours ago
nitros | 3 hours ago
mike_hearn | 2 hours ago
kennywinker | 6 hours ago
foota | 6 hours ago
kennywinker | 6 hours ago
gleenn | 6 hours ago
kennywinker | 6 hours ago
runako | 6 hours ago
Very smart play to build a platform, get scale, and prove out the software. Then either add a small network fee (this could be on money movement on/off platform), add a higher tier of service for money, and/or just use the proof points to go get access to capital and become an operator in your own pool.
nxpnsv | 5 hours ago
ffsm8 | 6 hours ago
chaoz_ | 6 hours ago
Non-VC play (not required until you can raise on your own terms!) and clear differentiation.
If you want to go full-business-evaluation, I would be more worried about someone else implementing same thing with more commission (imo 95% and first to market is good enough).
jonplackett | 3 hours ago
ie. Does anyone know the payback time for a B100 used just for inference? I assume it’s more than a couple of months? Or is it just training that costs so much?
Saline9515 | 39 minutes ago
thih9 | 5 hours ago
Others are reporting low demand, eg.: https://news.ycombinator.com/item?id=47789171
znnajdla | 5 hours ago
As a business owner, I can think of multiple reasons why a decentralized network is better for me as a business than relying on a hyperscaler inference provider. 1. No dependency on a BigTech provider who can cut me off or change prices at any time. I’m willing to pay a premium for that. 2. I get a residential IP proxy network built-in. AI scrapers pay big money for that. 3. No censorship. 4. Lower latency if inference nodes are located close to me.
kennywinker | 5 hours ago
znnajdla | 2 hours ago
lxglv | 2 hours ago
znnajdla | 2 hours ago
NiloCK | 3 hours ago
You - Darkbloom - Operator - Darkbloom - you, vs
You - Provider - you
---
On the censorship point - this is an interesting risk surface for operators. If people are drawn my decentralized model provisioning for its lax censorship, I'm pretty sure they're using it to generate things that I don't want to be liable for.
If anything, I could imagine dumber and stricter brand-safety style censorship on operator machines.
znnajdla | an hour ago
yard2010 | 3 hours ago
aacid | an hour ago
znnajdla | an hour ago
rzwitserloot | an hour ago
Running AI inference increases the power draw, and requires certain hardware.
Mining bitcoin increases the power draw, and requires certain hardware.
OP's point thus stands: Bad players will find places to get far cheaper power than the intended audience, and will buy dedicated hardware, at which point the money you can earn to do this will soon drop below the costs for power (for folks like you and me).
Maybe that won't happen, but why won't that happen?
znnajdla | 51 minutes ago
dnnddidiej | 5 hours ago
Prolly gonna make $50 a year tops.
agnosticmantis | 4 hours ago
- Elon Musk during Tesla's Autonomy Day in April 2019.
psychoslave | 3 hours ago
Assuming that getting large chunk of initial investment is just a formality is out of touch with 99% of people reality out there, when it’s actually the biggest friction point in any socio-economical endeavour.
eigengajesh | 46 minutes ago
That's why we don't recommend purchasing a new machine. Existing machine is no cost for you to run this.
Electricity is one cost, but it will get paid off from every request it receives. Electricity is only deducted when you run an inference. If you have any questions, DM me @gajesh on Twitter.
stavros | 24 minutes ago
embedding-shape | 8 minutes ago
How much though? Say I have three Mac Minis next to each other, one that is completely idle but on, one that bursts 100% CPU every 10 minutes and one that uses 100% CPU all the time, what's the difference on how long the machines survives? Months, years or decades?
chaoz_ | 6 hours ago
Guess there are limitations on size of the models, but if top-tier models will getting democratized I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns.
I think batch-evals for non-sensitive data has great PMF here.
rvz | 6 hours ago
Because they were already at the finish line with Apple Silicon.
> I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns.
The whole inference is end-to-end encrypted so none of the nodes can see the prompts or the messages.
chaoz_ | 6 hours ago
That would finally be a crypto thing which is backed by value I believe in.
59nadir | 4 hours ago
Heh, what did they win exactly? This is just a way for another company to extract value out of the single region of the world where Apple is a relevant vendor, and it happens to be the one where it's the easiest to pull people into schemes.
bentt | 6 hours ago
TuringNYC | 6 hours ago
damezumari | 6 hours ago
utopiah | 3 hours ago
zozbot234 | 3 hours ago
pants2 | 6 hours ago
My M5 Pro can generate 130 tok/s (4 streams) on Gemma 4 26B. Darkbloom's pricing is $0.20 per Mtok output.
That's about $2.24/day or $67/mo revenue if it's fully utilized 24/7.
Now assuming 50W sustained load, that's about 36 kWh/mo, at ~$.25/kWh approx. $9/mo in costs.
Could be good for lunch money every once in a while! Around $700/yr.
MrDrMcCoy | 6 hours ago
pants2 | 6 hours ago
todotask2 | 6 hours ago
I don’t think this is a sustainable business model. For example, Cubbit tried to build decentralised storage, but I backed out because better alternatives now exist, and hardware continues to improve and become cheaper over time.
Your electricity and ownership are going to get lower return and does not actually requce CO2.
chaoz_ | 6 hours ago
I’d imagine 1 year of heavy usage would somehow affect its quality.
pants2 | 6 hours ago
xendo | 6 hours ago
mavamaarten | 6 hours ago
I'd say it's not worth it. But the idea is cool.
kennywinker | 5 hours ago
jorvi | 3 hours ago
Thermal stress from bursty workloads is much more of a wearing problem than electromigration. If you can consistently keep the SoC at a specific temperature, it'll last much longer.
This is also why it was very ironic that crypto miner GPUs would get sold at massive discounts. Everyone assumed that they had been ran ragged, but a proper miner would have undervolted the card and ran it at consistent utilization, meaning the card would be in better condition than a secondhand gamer GPU that would have constantly been shifting between 1% to 80% utilization, or rather, 30°C to 75°C
kennywinker | 5 hours ago
For Gemma 4 26B their math is:
single_tok/s = (307 GB/s / 4 GB) * 0.60 = 46.0 tok/s
batched_tok/s = 46.0 * 10 * 0.9 = 414.4 tok/s
tok/hr = 414.4 * 3600 = 1,492,020
revenue/hr = (1,492,020 / 1M) * $0.200000 = $0.2984
I have no idea if that is a good estimate of how much an M5 Pro can generate - but that’s what it says on their site.
They do a bit of a sneaky thing with power calculation: they subtract 12Ws of idle power, because they are assuming your machine is idling 24/7, so the only cost is the extra 18W they estimate you’ll use doing inference. Idk about you, but i do turn my machine off when i am not using it.
pants2 | 52 minutes ago
znnajdla | 5 hours ago
sethherr | 5 hours ago
tonyedgecombe | 4 hours ago
znnajdla | an hour ago
nnx | 5 hours ago
This seems high. At which quantization? Using LM Studio or something else?
Note: Darkbloom seems to run everything on Q8 MLX.
pants2 | 51 minutes ago
torginus | an hour ago
I figured since I already used it a lot, and I've never had a GPU fail on me, it would be fine.
The fans on it died in a month of constant use, replacing them was more money than what I made on mining.
BingBingBap | 6 hours ago
What could possibly go wrong?
pants2 | 6 hours ago
rvz | 6 hours ago
kennywinker | 5 hours ago
;P
ramoz | 6 hours ago
Apple Silicon has a Secure Enclave, but not a public SGX/TDX/SEV-style enclave for arbitrary code, so these claims are about OS hardening, not verifiable confidential execution.
It would be nice if it were possible. There's a lot of cool innovations possible beyond privacy.
geon | 6 hours ago
znnajdla | 5 hours ago
znnajdla | 5 hours ago
ramoz | 5 hours ago
You have no guarantees over any random connected laptop connected across the world.
znnajdla | 2 hours ago
mike_hearn | 3 hours ago
macOS has a strong enough security architecture that something like Darkbloom would have at least some credibility if there was a way to remotely attest a Mac's boot sequence and TCC configuration combined with key-to-DR binding. The OS sandbox can keep apps properly separated if the kernel is correct and unhacked. And Apple's systems are full of mitigations and roadblocks to simple exploitation. Would it be as good as a consumer SGX enclave? Not architecturally, but the usability is higher.
dr_kiszonka | 6 hours ago
When your Mac is idle (no inference requests), it consumes minimal power — you don't lose significant money waiting for requests. The electricity costs shown only apply during active inference.
Text models typically see the highest and most consistent demand. Image generation and transcription requests are bursty — high volume during peaks, quiet otherwise."
dcreater | 6 hours ago
stuxnet79 | 6 hours ago
JaggerJo | 5 hours ago
tgma | 5 hours ago
In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations.
At the moment they don't have enough sustained demand to justify the earning estimates.
thatxliner | 4 hours ago
I was thinking of building this exact thing a year ago but my main stopper was economics: it would never make sense for someone to use the API, thus nobody can make money off of zero demand.
I guess we just have to look at how Uber and Airbnb bootstrapped themselves. Another issue with my original idea was that it was for compute in general, when the main, best use-case, is long(er)-running software like AI training (but I guess inference is long running enough).
But there already exist software out there that lets you rent out your GPU so...
tgma | 4 hours ago
starkeeper | 4 hours ago
lostmsu | 58 minutes ago
lxglv | 4 hours ago
lostmsu | an hour ago
The numbers are absolute fraud. You shouldn't be installing their software cause fraud could be not just about numbers.
rjmunro | 23 minutes ago
splittydev | 4 hours ago
tgma | 4 hours ago
Still, absolute zero is an unacceptable number. Had this running for more than an hour.
splittydev | 4 hours ago
Sure, it would be great if you'd immediately get hammered with hundreds of requests and start make money quickly. It would also be great if it was a bit more transparent, and you could see more stats (what counts as "idle"? Is my machine currently eligible to serve models?). But it's still very new, I'd say give it some time and let's see how it goes.
If you have it running and you get zero requests, it uses close to zero power above what your computer uses anyway. It doesn't cost you anything to have it running, and if you get requests, you make money. Seems like an easy decision to me.
tgma | 3 hours ago
subroutine | 3 hours ago
oneeyedpigeon | 3 hours ago
yard2010 | 3 hours ago
usrusr | an hour ago
subroutine | 3 hours ago
koliber | 5 hours ago
jboggan | 5 hours ago
gndp | 5 hours ago
resonanormal | 5 hours ago
Havoc | 16 minutes ago
0xbadcafebee | 5 hours ago
utkarsh_apoorva | 5 hours ago
They lost me with just one microcopy - “start earning”. Huge red signal.
Hamuko | 4 hours ago
jaylane | 5 hours ago
amdivia | 4 hours ago
woadwarrior01 | 4 hours ago
Got the latest v0.3.8 version from the list here: https://api.darkbloom.dev/v1/releases/latest
Three binaries and a Python file: darkbloom (Rust)
eigeninference-enclave (Swift)
ffmpeg (from Homebrew, lol)
stt_server.py (a simple FastAPI speech-to-text server using mlx_audio).
The good parts: All three binaries are signed with a valid Apple Developer ID and have Hardened runtime enabled.
Bad parts: Binaries aren't notarized. Enrolls the device for remote MDM using micromdm. Downloads and installs a complete Python runtime from Cloudflare R2 (Supply chain risk). PT_DENY_ATTACH to make debugging harder. Collects device serial numbers.
TL;DR: No, not touching that.
WatchDog | 4 hours ago
Interesting idea, but needs some work.
gleenn | 3 hours ago
mirashii | 3 hours ago
That said, their privacy posture at the cornerstone of their claims is snake oil and has gaping holes in it, so I still wouldn't trust it, but it's worth being accurate about how exactly they're messing up.
mike_hearn | 2 hours ago
You are right - the "nonce binding" the paper uses doesn't seem convincing. The missing link is that Apple's attestation doesn't bind app generated keys to a designated requirement, which would be required to create a full remote attestation.
mirashii | 2 hours ago
It only effectively allows this for applications that are in the set of things covered by SIP, but not for any third-party application. There's nothing that will allow you to attest that arbitrary third-party code is running some specific version without being tampered with, you can only attest that the base OS/kernel have not been tampered with. In their specific case, they attempt to patch over that by taking the hash of the binary, but you can simply patch it before it starts.
To do this properly requires a TEE to be available to third-party code for attestation. That's not a thing on macOS today.
mike_hearn | 2 hours ago
It's odd that the paper considers so many angles including things like RDMA over Thunderbolt, but not the binding between platform key and app key.
Reading the paper again carefully I get the feeling the author knows or believes something that isn't fully elaborated in the text. He recognizes that this linkage problem exists, proposes a solution and offers a security argument for it. I just can't understand the argument. It appears APNS plays a role (apple push notification service) and maybe this is where app binding happens but the author seems to assume a fluency in Apple infrastructure that I currently lack.
mirashii | 2 hours ago
Certainly, it still doesn't get you there with their current implementation, as the attempts at blocking the debugger like PT_DENY_ATTACH are runtime syscalls, so you've got a race window where you can attach still. Maybe it gets you there with hardened runtime? I'd have to think a bit harder on that.
mike_hearn | 2 hours ago
I'm not quite sure why Apple haven't enabled DCAppAttest on macOS. From my understanding of the architecture, they have every piece needed. It's possible that they just don't trust the Mac platform enough to sign off on assertions about it, because it's a lot more open so it's harder to defend. And perhaps they feel the reputational risk isn't worth it, as people would generalize from a break of App Attest on macOS to App Attest on iOS where the money is. Hard to say.
keremimo | 2 hours ago
Even moreso, not for pennies/month
egorfine | 3 hours ago
NiloCK | 3 hours ago
Right now the dashboards show 78 providers online, but someone in-thread here said that they spun one up and got no requests. Surely someone would be willing to beat the posted rate and swallow up the demand?
I expect this is a migration target, but a tactical omission from V1 comms both for legitimate legibility reasons (I can sell x for y is easier to parse than 'I can participate in a marketplace') and slightly illegitimate legibility reasons (obscuring likely future price collapse).
Still - neat project that I hope does well.
[1] Layer Labs, formerly EigenLayer, is company built around a protocol to abstract and recycle economic security guarantees from Ethereum proof of stake.
v9v | 2 hours ago
Jn2G3Np8 | 2 hours ago
But trying it out it still needs work, I couldn't download a model successfully (and their list of nodes at https://console.darkbloom.dev/providers suggests this is typical).
And as a cursory user, it took me some digging to find out that to cash out you need a Solana address (providers > earnings).
miki123211 | 2 hours ago
Is there some actual cryptography behind this, or just fundamentally-breakable DRM and vibes?
grvbck | an hour ago
puttycat | an hour ago
Afaik you will need to decrypt the data the moment it needs to be fed into the model.
How do they do this then?
mr_mitm | 53 minutes ago
Remember, all encryption is E2EE if you're not picky about the ends.
subpixel | an hour ago
chakintosh | an hour ago
ianpurton | an hour ago
heddycrow | an hour ago
We’ve been building something similar for image/video models for the past few months, and it’s made me think distribution might be the real bottleneck.
It’s proving difficult to get enough early usage to reach the point where the system becomes more interesting on its own.
Curious how others have approached that bootstrap problem. Thanks in advance.
haspok | an hour ago
Problem is, from a technical point of view, what kind of made sense back then (most people running desktops, fans always on, energy saving minimal) is kind of stupid today (even if your laptop has no fan, would you want it to be always generating heat?)...
I definitely want my laptops to be cool, quiet and idle most of the time.
vorticalbox | an hour ago
This is short bursts of heat 5-10 m during the render I would not be happy with that for multiple hours a day. I am sure that would have a negative effect on battery health.
Fokamul | 51 minutes ago
eigengajesh | 49 minutes ago
0xc133 | 46 minutes ago
drob518 | 42 minutes ago
bprasanna | 16 minutes ago