I dunno. Regardless of my feelings about LLMs, if people are going to use them (which they do), I'd rather they have a way to do it that doesn't involve shoveling data into the surveillance machine.
If you sincerely object to the surveillance machine, don't consume its fruits. If you use locally-hostable LLMs you're still supporting the narrative that enables the non-locally-hostable LLMs to be created. The "privacy-preserving" version of this misbegotten technology is still built on the mass surveillance and exploitation of the commons, disrespecting the wishes and efforts of countless millions of unconsenting human contributors. Absolutely nobody needs LLMs, under any circumstances, even "as a treat".
I think that this is a false dilemma given that llama.cpp exists; people can run LLMs locally on commodity hardware without any tokens leaving their LAN.
I think Corbin's argument isn't that end users can run the literal llama.cpp, but that it's possible for end users to run this sort of software on their devices in general. You could absolutely make a local LLM chatbot thing that doesn't target tech people, and that is instead supposed to be as simple as possible.
I agree 100%. My personal LLM usage is negligible but I'm also happy something is being done in this space. LLM conversations encrypted via TPM TEE is quite an interesting way to do it, and I kind of see the vision here.
That said, trust in Moxie is dwindling, but I know he can deliver on the tech, as evidenced with Signal (excluding mobile coin drama, and excluding ridiculous delay on usernames).
Introducing Confer, an end-to-end AI assistant that just works.
I assumed end-to-end [encryption] was a buzzword accidentally added into the copy by a redactor that didn't know better, because you're just connecting to a central server - but no,
Data and conversations originating from users and the resulting responses from the LLMs are encrypted in a trusted execution environment (TEE) that prevents even server administrators from peeking at or tampering with them.
Conversations are stored by Confer in the same encrypted form, which uses a key that remains securely on users’ devices.
I assume the point is that they're stored so the LLMs can use them as context in future conversations? Storing that much user data feels weird, considering e.g. Signal didn't even support cloud backups until recently.
The article focuses on passkeys for some reason, but I'm curious how they do inference. I assume that just runs under TEE? I heard homomorphic encryption is still way too slow to be used for this sort of thing, but I don't really follow that space. Nor know much about this entire area.
I can't visit Confer's page because I'm not using "a device with platform authentication (Face ID, Touch ID, Windows Hello, etc.)", so I guess I won't get my answers there. Does anyone know how to install Face ID on Debian?
The article focuses on passkeys because that’s how Confer is able to generate and persist a secure private key on the client, and as a bonus sync it securely to the user's other devices. There’s a blog post about it.
That is a much better user experience than being shown 12 random words to write down and “store securely.”
I wish he explained how passkeys manage to avoid this. Sure, you can use some fancy hardware security features to lock the secrets behind a relatively weak secret such as a fingerprint or a face scan, but how do you back that up? How are the backups protected? Unless you have a second phone, you can't rely on these features anymore, right?
Most LLM apis require the caller to provide the complete context in each request. Obviously in this case the context would still be plain text when it's processed by the model, but I can see a workflow where there's nothing persisted on the server. It would just stored in memory when it's actively being processed.
gcupc | 8 hours ago
This doesn't make me trust AI more, it makes me trust Moxie less.
David_Gerard | an hour ago
remembering how Moxie got into cryptocoins before this
jdpage | 5 hours ago
I dunno. Regardless of my feelings about LLMs, if people are going to use them (which they do), I'd rather they have a way to do it that doesn't involve shoveling data into the surveillance machine.
Internet_Janitor | 4 hours ago
If you sincerely object to the surveillance machine, don't consume its fruits. If you use locally-hostable LLMs you're still supporting the narrative that enables the non-locally-hostable LLMs to be created. The "privacy-preserving" version of this misbegotten technology is still built on the mass surveillance and exploitation of the commons, disrespecting the wishes and efforts of countless millions of unconsenting human contributors. Absolutely nobody needs LLMs, under any circumstances, even "as a treat".
Corbin | 5 hours ago
I think that this is a false dilemma given that llama.cpp exists; people can run LLMs locally on commodity hardware without any tokens leaving their LAN.
apg | 4 hours ago
The general public is not going to run their own llama.cpp, but they might still want to interact with a Lemon.
the-pete | 2 hours ago
I suppose the same argument could have been made for pgp or matrix servers.
Moxie isnt making this for tech people.
dzwdz | an hour ago
I think Corbin's argument isn't that end users can run the literal llama.cpp, but that it's possible for end users to run this sort of software on their devices in general. You could absolutely make a local LLM chatbot thing that doesn't target tech people, and that is instead supposed to be as simple as possible.
the-pete | an hour ago
I get that, and I also don’t love all the decisions signal made, especially being tied to a phone number and defaulting to auto discovery.
Moxie is going to make moxie choices, and I probably won’t love them but generally his goal is ease of use.
dataangel | 3 hours ago
LLMs are here to stay, high quality open source models will be important if we don't want to be forever beholden to large cloud providers
zk | 2 hours ago
I agree 100%. My personal LLM usage is negligible but I'm also happy something is being done in this space. LLM conversations encrypted via TPM TEE is quite an interesting way to do it, and I kind of see the vision here.
That said, trust in Moxie is dwindling, but I know he can deliver on the tech, as evidenced with Signal (excluding mobile coin drama, and excluding ridiculous delay on usernames).
dzwdz | 11 hours ago
I assumed end-to-end [encryption] was a buzzword accidentally added into the copy by a redactor that didn't know better, because you're just connecting to a central server - but no,
That makes sense actually. Trusting hardware owned by others is definitely a reasonable thing to do, after all.
I assume the point is that they're stored so the LLMs can use them as context in future conversations? Storing that much user data feels weird, considering e.g. Signal didn't even support cloud backups until recently.
The article focuses on passkeys for some reason, but I'm curious how they do inference. I assume that just runs under TEE? I heard homomorphic encryption is still way too slow to be used for this sort of thing, but I don't really follow that space. Nor know much about this entire area.
I can't visit Confer's page because I'm not using "a device with platform authentication (Face ID, Touch ID, Windows Hello, etc.)", so I guess I won't get my answers there. Does anyone know how to install Face ID on Debian?
edit: My phone had all "these advanced features" required to be able to see the link to their blog, https://confer.to/blog/, which does actually work on my laptop. It does seem they run inference under TEE.
[OP] snej | 7 hours ago
The article focuses on passkeys because that’s how Confer is able to generate and persist a secure private key on the client, and as a bonus sync it securely to the user's other devices. There’s a blog post about it.
dzwdz | 6 hours ago
I wish he explained how passkeys manage to avoid this. Sure, you can use some fancy hardware security features to lock the secrets behind a relatively weak secret such as a fingerprint or a face scan, but how do you back that up? How are the backups protected? Unless you have a second phone, you can't rely on these features anymore, right?
xjix | 9 hours ago
Most LLM apis require the caller to provide the complete context in each request. Obviously in this case the context would still be plain text when it's processed by the model, but I can see a workflow where there's nothing persisted on the server. It would just stored in memory when it's actively being processed.
judson | 11 hours ago
Wonder if he will put Mobilecoin into this as well
ag | 11 hours ago
I wonder how this server infrastructure compares to Private Cloud Compute
freddyb | 9 hours ago
PCC is just the Apple name for TEE. Whereas TEE is just a slightly more generic term for Intel SGX.
None of then have been proven suffixes trustworthy imho.
dvogel | 5 hours ago
What do you mean by suffixes trustworthy?
caius | 5 hours ago
I suspect the OP meant "sufficiently trustworthy"