This company's criminal levels of incompetence single-handedly caused an instant international disaster, and continues to fail us all.
Speaking as a citizen, ignoring "all those bloody foreigners amplifying this on social media" for a (very very brief) moment to dwell on (suppresses gag reflex) ideological "national security" rhetoric which these people kowtow to... How have they not proactively taken out all stops to fix this already? On a national-disaster-scale emergency footing, which this is!
So... is there anything actually indicating "criminal levels of incompetence" on eMudhra's side yet?
As I understand the story so far, this is the main bit of information we've got:
As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)
My interpretation of this is that they created an additional keypair on the HSM ("second pair of keys being created onboard"), and then mixed up the two pairs due to human error. They shouldn't have mixed up these keys, but this, in of itself, isn't something that should have had such catastrophic consequences. The second keypair should've been just as secure; a HSM should never let you generate such weak keys. That's the entire point of a HSM.
Something else went wrong, and we don't yet know the whole story. eMudhra's response so far hasn't been... entirely satisfactory either, but I feel like a lot of people are interpreting everything they say in absolutely the worst way possible.
FWIW, I'm not a cryptographer, I've never worked with an HSM, etc. Maybe I'm missing some piece of damning evidence? I'm hoping someone with more relevant experience can speak out here.
I don't know about 'criminal', but the event itself and the fact it was due to a 'deviation' in their process and a flaw in their code is proof that eMudhra is incompetent to hold their extremely privileged position. If some change to the process was needed, it should have been rigorously reviewed and tested by experts. Like you say, why was the creation of a weak key ever possible?
In my opinion Sectigo should also have reviewed eMudhra's code and processes (perhaps they attempted to but were misled?) and probed the CSRs for obvious weaknesses given they cross-signed them.
It sounds like everyone involved was treating this process without the seriousness it deserves. This kind of thing keeps happening and the consequences are enormous which points to the system being insufficient.
I do not believe that an HSM bearing the various certifications the Utimaco units bear should be able to internally generate a series of private keys that have the unfortunate property these have.
One single bad-actor entity foisted weakpractically no encryption onto everybody in the world.
Only by sheer luck did it not poison everything or get visibly exploited---yet. We don't know the risk exposure from activity that is dark / unreported (certs not being publicly logged). It could be nothing, but that's not how security is done. If you leak a secret, you drop everything and immediately rotate the secret and everything derived from that secret, no matter what. You do not hide behind a fig leaf of weak sauce lying PR to the tenor of "well, nothing to see here move along... 'coz, like, it's only in our house and there has been no impact here (according to us), and like, besides, it wasn't for thaaat long anyhow".
Emergency fixes arrived post-disclosure only due to the "all-hands-on-deck" hard work of a very small number of competent security people.
None of the work was initiated or driven by the entity. The concerned security community had to hand-hold them and egg them on at every step.
And then, tadaaa! The entity proceeds to literally leave the chat... Ghosts the bugzilla thread, announces shuttering of the PKI business, releases incomplete, misleading, and responsibility-abdicating PR.
While we understand that many things are happening in parallel right now, "social media scrutiny" and "analysis by investors/customers" are not acceptable explanations for delayed responses here.
That scrutiny is the predictable result of eMudhra's own failure (i.e., intermediate CA keys that anyone could recover -- resulting in anyone being able to issue certificates for any domain), not an external circumstance that befell it. Offering it as an excuse conveys that reputation management is being prioritized over the obligations that eMudhra still owes to the billions of relying parties its mistake exposed. Those obligations do not relax because the consequences are uncomfortable.
We expect eMudhra to prioritize its review of this issue and its security response.
The impact of this incident on Indian websites and applications have been very limited and for a very short interval.
That last claim is wrong on both counts. The interval wasn’t short: the flawed keys were generated in April 2025, so anyone could have recovered them for 18 months before the report.
I’m actually okay with describing the impact as a very short interval. Sure, the window was wide, but the impact was narrow. It seems likely that no one detected it during that time. Certainly from Certificate Transparency logs we know that no one targeted it for mainstream browsers, which is the most obvious and juiciest target. Maybe someone did detect and exploit it in other ways, but there’s no evidence to suggest it.
Separately, on 19th October eMudhra revoked 9,356 certificates on its unaffected RSA hierarchy (emSign Root CA - G1) with the reason “Superseded”.
Are these are the certificates listed later on the page? I’ve looked at the one that’s affecting me now (ociservices.gov.in) and it’s G1 rather than G3 which https://emudhra.we-have-a-problem.com/ indicates was the stupid one. Or are some in the list G1 and some G3?
Why did eMudhra do this G1 thing, abruptly revoking ones that were fine and making the impact of the incident worse?
However it is, it’s pretty wild that so many government sites, some major, have just been broken for most of the week. Surely eMudhra would have contacted every affected customer and they could sort it out in under a day?
Unrelated to the content: this page is a good demonstration of why not to put html, body { overflow-x: hidden } in your styles. (It’s more common than it should be because it fixes an unimportant overflow issue in one of the traditional full-bleed decoration techniques.) Some time later you’ll put something like a wide table in, and whoops, its right columns are inaccessible.
Not an expert, but if I'm reading correctly, when the cert is submitted to the CT log operator, they get back a "signed certificate timestamp (SCT)" which is basically "we promise to publish this in our log within N hours". The browser has a list of trusted CT log operator keys and checks it via whatever public-key mechanism.
It doesn't check the log itself so in theory a log operator could collude with whoever's issuing the malicious cert to return an SCT but not publish it, but then they'd also get in trouble when the cert is discovered to have an SCT but no log entry.
If the claim is intended to mean that Indian websites weren’t impersonated, the claim is overly certain. If the claim is intended to mean that Indian websites could only be impersonated for a very short time, the claim is wrong.
[OP] adityaathalye | a day ago
This company's criminal levels of incompetence single-handedly caused an instant international disaster, and continues to fail us all.
Speaking as a citizen, ignoring "all those bloody foreigners amplifying this on social media" for a (very very brief) moment to dwell on (suppresses gag reflex) ideological "national security" rhetoric which these people kowtow to... How have they not proactively taken out all stops to fix this already? On a national-disaster-scale emergency footing, which this is!
dzwdz | 19 hours ago
So... is there anything actually indicating "criminal levels of incompetence" on eMudhra's side yet?
As I understand the story so far, this is the main bit of information we've got:
My interpretation of this is that they created an additional keypair on the HSM ("second pair of keys being created onboard"), and then mixed up the two pairs due to human error. They shouldn't have mixed up these keys, but this, in of itself, isn't something that should have had such catastrophic consequences. The second keypair should've been just as secure; a HSM should never let you generate such weak keys. That's the entire point of a HSM.
Something else went wrong, and we don't yet know the whole story. eMudhra's response so far hasn't been... entirely satisfactory either, but I feel like a lot of people are interpreting everything they say in absolutely the worst way possible.
FWIW, I'm not a cryptographer, I've never worked with an HSM, etc. Maybe I'm missing some piece of damning evidence? I'm hoping someone with more relevant experience can speak out here.
berserk | 18 hours ago
I don't know about 'criminal', but the event itself and the fact it was due to a 'deviation' in their process and a flaw in their code is proof that eMudhra is incompetent to hold their extremely privileged position. If some change to the process was needed, it should have been rigorously reviewed and tested by experts. Like you say, why was the creation of a weak key ever possible?
In my opinion Sectigo should also have reviewed eMudhra's code and processes (perhaps they attempted to but were misled?) and probed the CSRs for obvious weaknesses given they cross-signed them.
It sounds like everyone involved was treating this process without the seriousness it deserves. This kind of thing keeps happening and the consequences are enormous which points to the system being insufficient.
krn | 16 hours ago
From the bug:
gerikson | 18 hours ago
1990s: you're a criminal if you use strong encryption.
2020s: you're a criminal if you use weak encryption.
[OP] adityaathalye | 16 hours ago
Flippant at best. Misdirection as read literally.
One single bad-actor entity foisted
weakpractically no encryption onto everybody in the world.For everyone splitting hairs in this thread, I will simply defer to Chrome team's latest remark on the bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c41
Emphasis theirs.
chrismorgan | 22 hours ago
I’m actually okay with describing the impact as a very short interval. Sure, the window was wide, but the impact was narrow. It seems likely that no one detected it during that time. Certainly from Certificate Transparency logs we know that no one targeted it for mainstream browsers, which is the most obvious and juiciest target. Maybe someone did detect and exploit it in other ways, but there’s no evidence to suggest it.
Are these are the certificates listed later on the page? I’ve looked at the one that’s affecting me now (ociservices.gov.in) and it’s G1 rather than G3 which https://emudhra.we-have-a-problem.com/ indicates was the stupid one. Or are some in the list G1 and some G3?
Why did eMudhra do this G1 thing, abruptly revoking ones that were fine and making the impact of the incident worse?
However it is, it’s pretty wild that so many government sites, some major, have just been broken for most of the week. Surely eMudhra would have contacted every affected customer and they could sort it out in under a day?
Unrelated to the content: this page is a good demonstration of why not to put
html, body { overflow-x: hidden }in your styles. (It’s more common than it should be because it fixes an unimportant overflow issue in one of the traditional full-bleed decoration techniques.) Some time later you’ll put something like a wide table in, and whoops, its right columns are inaccessible.dulaku | 18 hours ago
I might be missing something, but why would CT logs be relevant if people can forge their own cert chains offline?
chrismorgan | 17 hours ago
All mainstream browsers require proof that the certificate has been logged. It doesn’t avoid illegitimate issuance, but it does mean you can see it’s happened (and it didn’t).
altano | 11 hours ago
Wow how does that work? How do browsers check the logs? (Most curious about Firefox)
quasi_qua_quasi | 6 hours ago
Not an expert, but if I'm reading correctly, when the cert is submitted to the CT log operator, they get back a "signed certificate timestamp (SCT)" which is basically "we promise to publish this in our log within N hours". The browser has a list of trusted CT log operator keys and checks it via whatever public-key mechanism.
It doesn't check the log itself so in theory a log operator could collude with whoever's issuing the malicious cert to return an SCT but not publish it, but then they'd also get in trouble when the cert is discovered to have an SCT but no log entry.
schuelermine | 14 hours ago
If the claim is intended to mean that Indian websites weren’t impersonated, the claim is overly certain. If the claim is intended to mean that Indian websites could only be impersonated for a very short time, the claim is wrong.
riking | 12 hours ago
If websites were impersonated, it was only to non-browser software.