You misunderstand - the government issued a directive to Anthropic that effectively forced them to pull access from everyone, even their own employees.
The directive was to remove access to non-Americans, not to pull access from everyone. It’s because Anthropic cannot verify the identity of its users that it pulled access from everyone, not because the government explicitely requested that.
If their operation team is not US based that's going to be difficult to operate. They would have to reorganize the whole company as I'm pretty sure that they are not employing only US citizen.
Well, kinda in effect. He lacked the authority to make the call, and it is quite obviously being ignored by most suppliers. If it were actually enforced, I believe no companies that are contractors for the DoD could even host Anthropic- like GCP, Azure, and AWS. Perhaps they are currently figuring out how to get off all cloud provider govclouds, but I doubt it.
Which is a fundamental mistake to make with the U.S. government, even if we’re talking only about the executive branch, even if we’re only talking about DoD, even if we’re only talking about the IC.
Yeah... NSA literally has MITM proxies/interception of any traffic they want inside every major US tech company (based on my reading/following of Snowden leaks and others). Anthropic wouldn't be able to exist without implicit NSA approval. This article reads more like a marketing piece for Anthropic/Mythos... and ends by talking about how much NSA wants Anthropic models.
It's just not technically feasible, so there's nothing to lie about. They're not MITMing petabytes/sec across dozens (hundreds?) of companies and they haven't broken TLS1.3.
If I have a box at Digital Ocean and I'm communicating with it with TLS1.3 using a Let's Encrypt cert that I generated, where, exactly, does this magical MITM box come into play?
Of course it's feasible, you just intercept the traffic post-decryption on the cloud/server side. You don't control how/where your traffic to 3p cloud services is decrypted.
You keep saying this, but it's nonsensical. If I terminate TLS on the box that does processing, there's nothing to intercept.
And these days (especially post-Snowden), many (most?) companies encrypt data when sending between servers within their own (private network) infrastructure.
You have no control about where TLS is terminated when you're talking to a 3p cloud service (with services you don't control/run like cloud LLM APIs). You also have no control about what spyware is installed on/around VMs you rent (and there's a lot). Also when talking about encryption between servers within datacenters you seem to be missing that in order for such multi -stage/path encryption (separate certs/keys) to be possible the data first has to be decrypted at each point, not to mention every major US tech company generally cooperates with the NSA and gives them access to anything they request (including allowing the installation of dedicated hardware to intercept decrypted traffic as has been publicly exposed documented many times already).
Yours and others' claims that it's impossible and nonsensical is based on lack of understanding.
Yours and others' claims that things somehow got better after Snowden is just a completely baseless statement - if you actually looked into what happened post-Snowden - absolutely nothing was done to prevent NSA spying on any communications they want, in fact it got significantly worse.
> Yours and others' claims that it's impossible and nonsensical is based on lack of understanding.
lol, no, it's really not.
> Also when talking about encryption between servers within datacenters you seem to be missing that in order for such multi -stage/path encryption (separate certs/keys) to be possible the data first has to be decrypted at each point
Why would I want the data to be decrypted at each point and why would datacenters do that? Encrypting and decrypting data is expensive computationally, so that's not how things work at all. There's no need to decrypt data to know where it needs to go. That's why we have TCP/IP and other similar stadards.
The datacenters can maybe add another layer of encryption on top of my data as its moving around their networks, but there's absolutely no way for them to strip off my encryption.
> Yours and others' claims that things somehow got better after Snowden is just a completely baseless statement
Things didn't magically get better. A lot of people worked hard to improve the overall security posture of the industry.
> Why would I want the data to be decrypted at each point and why would datacenters do that?
I think they mean the data must have existed in plain text before it was encrypted, and will exist in plain text after it is decrypted.
At some point “your” server in a datacenter somewhere needs to decrypt the data to do something useful with it, after all you’re paying for compute, and homeomorphic encryption is too slow, so the work is done in unencrypted data.
There it is. Your data in plain text in RAM.
TLS will protect your data in transit, but it can’t protect you against a compromised recipient.
So the NSA streams the ram of every virtual machine and bare metal server on the internet to themselves so they can analyze the plain text that's being processed in ram and no one has noticed this network traffic? How could that even be possible? If I buy a 100Mbps network connection from someone, they just provision a bit more so that the NSA streaming doesn't impact or show up?
> Why would I want the data to be decrypted at each point and why would datacenters do that?
When we talk about data that is sent for processing to a 3p server (like anthropic in this case) the data obviously needs to be decrypted to be processed.
As to why data is decrypted at each point in a typical large backend system - because other than network routing there are presumably multiple services that need to receive and act on this data somehow - you're not just sending encrypted data around to random servers.
> there's absolutely no way for them to strip off my encryption.
You don't seem to understand that you have no control over the encryption or decryption done on the backends of cloud services you use. I don't know how to make it more simple and obvious at this point.
Again, the context here is Anthropic and sending your data to their (or any other big tech API). But even if we move away from this model and suppose you are running your own services on rented cloud VM - then it should be obvious that you don't have full control or even access to this VM... any actor with access can install or modify any software, install/modify EBPF, modified crypto libraries, etc. - you have absolutely no control or say over this.
> Things didn't magically get better.
Things didn't get better at all, they got much worse.
You just intercept the traffic after its decrypted on the server side, or are you suggesting you somehow send encrypted traffic that never gets decrypted?
So the NSA streams the memory contents of every virtual machine and bare metal server on the internet to get the decrypted traffic? How would that even work at the scale of the internet?
How it works is they build a huge virtual strawman which decrypts and reads all of the data for them then posts online about how NSA spying on people is literally impossible.
I second this: HTTPS (as most consumers use it) is probably a front (who are these CA's really anyway?)
Plot twist: _Perhaps_ Mythos / Fable keeps explaining ways (that we can't comprehend or don't always work) to break HTTPS due to the three letter agencies making sure they had input on their creation (and thus backdoors, I mean "bugs"), so the real catastrophe they are hiding is that HTTPS is broken (for most people, most of the time.)
Remember when Quantum computing was the threat to HTTPS? Turns out it was the humans own inability to think outside of the box!
Which would be easily detectable if the cert I'm using on my server didn't match the one that was being served publicly.
There's really no way this conspiracy theory works if "they" have a copy of every single private cert generated. Which would be impressive because I can generate one myself and get it trusted without ever sending it and would be easily able to detect a MITM attack.
Not to mention most sites are going to use pinned certs so any repeat visitors to a site will notice a cert change associated with a MITM.
This whole idea relies on the assumption that everyone is trusting third parties with their private certs. That is not at all required.
> How are they going to MITM communications with certs that never left my machine?
The long game. They:
- make sure you wouldn't be in a position to need to transmit data anywhere that would receive it without CA's in their hypothetical pocket
- manage the evolution of the cloud industry to make sure portable VM's and Containers can have their data archived (both in-RAM, disk, hey just send us the running VM!)
- backdoor'd encryption algorithms from the design and implementation phase to ensure a global unlocking mechanism for any data encrypted by anybody who used a large class of extremely commonly available software
So, you run your own private bank in a cloud VM with tenant managed keys? They backdoor'd the encryption algorithm your cloud VM disk relies on, because they blackmailed one of the developers at the company who developed the hypervisor system used by your provider. Open source project? Perfect. (If you think this is nonsense, then remember the rapid discovery of ancient "bugs" causing all this drama to begin with.)
Your TLS privately generated certs that are 100% foolproof aren't actually used anywhere encrypting the data they want, because it's either worthless, or, available elsewhere perhaps at a different (or same) time.
If you're a specific target, they have to spend an incredibly number of man-hours and money to get into your private data. This proves my point. This shows the effort required to infiltrate _one_ target and you're suggesting they've infiltrated everything by default.
How would you know about the successes? Thinking this is the one and only time they tried it is... interesting.
(Plus: "it was, for seven years, one of four CSPRNGs standardized in NIST SP 800-90A")
> If you're a specific target, they have to spend an incredibly number of man-hours and money to get into your private data.
No, this demonstrates an actor of that power level doesn't even need to compromise encryption, and can get deeper access to everything, if it's worth it to them.
I recall having a nuclear meltdown personally when I heard about all of this in the mid aughts. Nobody cared. Nobody understands this today. Everyone just complains about the Donald, but I point to this, and they don't realize the connection.
It's generally accepted fact that the NSA broke HTTPS, for some of the time, for some of the services. It's unclear what they do have, but you'd be naive to assume consumer HTTPS is keeping them out.
It's too complicated. Do you know everything about CA, SSL, HTTPS, and so on? You make $250k a year working on it? Do you _really_, _really_, know everything? Then you're fired because you're lying to yourself, so you're probably unbearable to work with.
We were all freaking out about this with AT&T Thing nearly twenty years ago: and when nobody cared (Bush ran two terms! it helped to pretend AT&T was the only one affected), it gave "them" implicit permission to do it again with Google / Yahoo thing (it helped to pretend those were the only two cloud providers affected) ten years ago.
Now, we're all pretending that capitalism is real, and that the three letter agencies are just sittin' on the sidelines, while the world's largest data archiving opportunity is happening voluntarily (some are even PAYING for it!), at some wild-growth companies (with leaders who have too much to lose), who also have existed for just a few years? A 5 year old could probably blackmail Sam Altman, what about all the other middle management? The individual contributors (if they still exist) are of no concern: work is a commodity, it's easy to silo a worker's knowledge.
Surveillance opportunity is 10x social media from last decade, because they still have social media, and now, they've began thinking for people. How easy when it is an app on your smartphone. Those mind control experiments back in the 60's with Acid are looking silly by now. Besides, how do you know that the response you're getting wasn't manipulated (and define 'manipulated' across a spectrum of training to nefarious actors impersonating models, by power of court order.)
If you think all of that is unfounded ridiculous blasphemy, let me distract you with this instead: if the AI bubble bursts, the compute will be repurposed for mass AI / ML driven CCTV surveillance. Hell, maybe they'll find a way to give you a tax break if you sell your CCTV footage.
"NSA literally has MITM proxies/interception of any traffic they want inside every major US tech company" even if this statement is an exaggeration, by playing the long game, they get themselves setup to access what they want in the future.
I'm not for or against, but I do live in a safe place thanks to such surveillance (generally in the USA), and I want you to know that this AI Thing is only the latest chapter in the intelligence story.
>they're cops with too much access, it doesn't take a genius to outsmart a cop
the nsa has an unlimited budget and spend a good portion of that budget recruiting some of the smartest people in the country. while they dont have super powers, they also arent the town cop who took a 6 month course after high school then joined the force.
it does no good to hold them up as mythical figures. it also does no good to pretend they are bumbling idiots.
(every math phd i am acquainted with has been approached by nsa recruiters. none of them have been approached by police agencies.)
Some of the smartest people I know have worked on fighting NSA, but they had a drastically smaller budget than NSA itself, and the mental availability bias is skewed by the fact that the "fighting NSA" people talked about their work all the time, while the "being NSA" people generally didn't.
I do know one extremely smart person who went to work there, and I witnessed a failed recruitment of another extremely smart person.
> every math phd i am acquainted with has been approached by nsa recruiters.
how many of them took them up on the offer, and how many are in leadership roles?
it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I'm not saying there aren't smart people working there but it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments... they just don't
>how many of them took them up on the offer, and how many are in leadership roles?
this is not really relevant to the point, but to satisfy your curiosity: more than one, and one.
>it takes a very narrow range of personality to want to be a cop
the nsa's brightest aren't doing "cop" things. certainly none of the people i know of working there are "cop-minded" in any sense.
they are doing cool research and application things. otherwise they wouldn't be able to entice the phds to stick around. these are people that want to work at the forefront of their field, doing interesting work, and the nsa is one avenue of doing that (with good job security, benefits, etc.).
>it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments
we agree here. they are certainly doing "HNDL" (harvest now, decrypt later) at a very large scale. but obviously they are not able to collect and store every piece of communication at every tech company over years and years. (the intelligence community comprehensive national cybersecurity initiative data center is large, but not that large)
> this is not really relevant to the point, but to satisfy your curiosity: more than one, and one.
What? That's not only relevant to the point, it's incredibly relevant. If the NSA is only able to recruit 2% of the math PhDs they approach, then that's important information.
"More than one" is not particularly useful; you seem to be dodging the question because it undermines your argument.
telling you exactly how many people i know in the NSA is also not particularly useful. i'm one guy. there is no statistically significant information from my answer.
>you seem to be dodging the question because it undermines your argument.
my "argument" is that there are plenty of smart people in the NSA. that's it. i am confused why that is seemingly so offensive to you that you had to reply twice.
> how many of them took them up on the offer, and how many are in leadership roles?
In my cohort? Several, and who knows? The recruitment effort is very visible and intense.
The US math phd market has been a slow-rolling disaster for over a decade. Everyone who can hack it outside the ivory tower is actively looking for the exits.
So why is it surprising that some of them go to work at the NSA?
> it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I don’t think you have context on what math phds are making in entry level positions, post-docs, or adjuncting. I just picked a random entry level NSA role on LinkedIn (doctorate + 0 yrs) and they’re offering solid six digits. There are tenured faculty (post-doc(s) + 5ish yrs) who don’t make that.
No they don't, and if you're going to try to argue something with that as your opener, it very easily casts large amounts of skepticism on whatever you are about to say.
Perhaps you're exaggerating for effect, but that also undermines your point.
>No they don't, and if you're going to try to argue something with that as your opener, it very easily casts large amounts of skepticism on whatever you are about to say.
if you read my comment like we're having a normal conversation instead of a thesis defense, you'll get my point just fine.
I worked on these cases at EFF and I'm skeptical of the automatic "NSA has access to everything" intuition.
What we learned from that era includes things like
(1) spy agencies are incredibly aggressive and pursue tons of different angles to get access to things
(2) spy agencies have a lot of money
(3) spy agencies often have interpretations of law that would surprise the public or legal experts (and sometimes courts have issued sealed rulings permitting them to do things that surprise the public or legal experts later when they're unsealed)
(4) some people throughout different parts of society assume culturally that companies in a country "should" generally help the spy agencies of that country's government because they are the "good guys" or "on the same team" or whatever
These things are all pretty bad and scary, but they still don't imply absolutely infinite power or access, because all of them come with different kinds of pushback. People also just tell them no!
I want to write an article with a colleague about the continuing role of culture here, because I think there are companies or industries where the default reaction is to want to cooperate with the government, and others where the default reaction is not that.
There are certainly secret things that have never come out, e.g. whatever Senator Wyden keeps alluding to, and what kind of program or authority was behind the interception of hardware shipments to covertly tamper with them, and whether there is a bulk financial data interception program, and presumably lots of other stuff. I don't agree with these things, and I want them to be exposed and stopped, and I also don't think they constitute infinite power over all parts of the tech industry.
Propaganda indeed: my instinct says we are being lied to about how three letter agencies and military are paying for services. They give us a PR front that Uncle Sam is a regular paying customer just like you and me, but they're probably running the show: this is the largest data gathering operation since 9/11.
Sorry everyone: but the conspiracy is so obviously not, it's nauseating to admit, because you see all your friends, family and co workers dumping so much everyday data into these services.
The current position seems to be no-one has access, not even Anthropic employees. What powers does the US government have to force them to provide access? If they have that power why did they not use it to force them to provide their products for military use?
Probably not. The US constitution limits what government can force on the people. If the NSA tries to force something that will spend years in court (if anyone wants to fight)
I hear you but, the Patriot act was the gateway. View it as a spectrum from then and how the Administration is now, and suddenly what the Donald is doing doesn't even seem bad: it seems on par for the dystopian road-map laid out long ago (I can't speak for before 9/11)
> The US constitution limits what government can force on the people.
The US constitution also prohibits:
- refusing to spend money that congress has appropriated
- dismantling congressionally-created federal agencies without congressional authorization
- directing federal agencies to selectively apply the law according to the preference of the executive
- giving control of federal agencies to individuals who have not been appointed by the legislative branch
- terminating, detaining, or deporting people without due process
- retaliation against private citizens or corporations for speech protected under the first amendment
- discriminating on protected grounds under the equal protections clause
... and yet the administration has done all these things with impunity while effete judges wring their hands and write sternly-worded letters. The US constitution demonstrably no longer has any force or effect.
If they wanted to officially take the weights the DPA would work and Dario could do nothing. If they wanted to do it in clandestine manner no one could stop them and no one would know. It's very likely they already have all the weights from all the frontier models. I mean all the frontier models are capable of being served from AWS Bedrock so the weights aren't exactly locked in some air-gapped vault.
It would be easy to make a national security justification to take the weights in a clandestine manner especially because Anthropic supposedly got caught giving China access to the model through a cutout.
Pretty sure even under DPA, taking without fair compensation would be a violation of the takings clause of 5th Amendment and wouldn't withstand legal scrutiny. If they wanted to get them clandestinely, yeah, they'd likely get away with it, but it is stealing.
Serious question: do you think the NSA aren't training their own LLMs? (With or without Anthropic and OpenAI's help)
It's a perfect technology for their uses, they get a big chunk of a $100 billion black budget, and they've had access to the research for at least as long as we have.
I don't think there is much overlap between people capable of building cutting edge LLM's and the people who want to build a cutting edge LLM for the government.
The NSA managed to deliberately insert a backdoor into elliptic-curve cryptography right under the noses of everyone capable of making elliptic-curve cryptography.
Mathematicians in academia are paid a little less than AI researchers. Companies are willing to pay billions to steal the few people capable of driving development of frontier LLMs from each other. Cryptographers don't quite enjoy the same popularity.
> The NSA managed to deliberately insert a backdoor into elliptic-curve cryptography right under the noses of everyone capable of making elliptic-curve cryptography.
That sort of proves the opposite point, assuming you're referring to Dual EC DRBG, because the flaw was noticed very early on, by people who weren't even involved in its development.
> Serious question: do you think the NSA aren't training their own LLMs?
Given the evergreen discussion of "are these companies making a profit"*, I think any LLMs that the NSA (or any other government agency worldwide) may be making are quite far from the leading edge.
* Person A: "they are making a loss!" Person B: "Only if you count training, they make a profit on inference, look at what it costs to run comparable open models on generic cloud servers" A: "Sure, but if they don't train new models they'll be left behind, so they're still making a loss"
That and the way compute is now measured in GW, I think even random low budget vloggers just getting started would be able to spot if the NSA was doing anything significant just from the extra heat emissions or power plants getting built.
The rate of inference compute to training compute is ~10:1, for popular frontier models. Models are routinely overtrained past the Chinchilla optimum now because it makes an immense amount of economic sense to do so.
Worse the more niche and unused your models get, but when this "making a loss" fuckery pops up, it's usually about the big guys like Anthropic, OpenAI, GDM and maybe xAI and Meta. Of which only the latter can be accused of not selling enough inference to offset the training runs.
The real money sinks are: R&D and infrastructure buildouts.
Serious question, do you realize that the NSA are mere mortals? Do you realize how much it takes to train a model? Does the NSA make their own chips or planes? The NSA buys a lot of technology because they can't make their own.
NSA has had their own supercomputing program for decades. they design and produce their own large scale machines. chips, fabrics, arithmetic units, all of it. they also employ quite a number of hardcore mathematicians, computer scientists, and systems wranglers. if they decided it was of strategic importance there is absolutely no reason they couldn't train their own models.
I guess we're just conspiracy theorists for landing at the objective conclusion that three letter government agencies:
- find "modern AI" to have strategic importance
- have ways to spend loads of money while having a front-facing budget on the record
- could be running a PR program to have Americans think they "buy" access to models like they do, but the AI companies were taken over by these agencies long ago
Look at Google, Microsoft...Apple got away with it by having as much on-device operation as possible so they could wash their hands, honestly saying "We don't have it."
This is the world's largest data gathering operation. Remember after 9/11 when the NSA copied as much Internet back bone traffic as they could?
I'm not for or against, even as a resident, but we certainly shouldn't be naive.
as someone who actually worked at the NSA pointed out earlier in this thread, they have plenty of resources, but also plenty of politics and some execution problems. so I wouldn't put money on them making a great model, but to say that they are completely incapable of doing anything is probably quite wrong.
the issue here that is a forgone conclusion, regardless of where the model comes from and which chips it runs on, is that now they can reasonably comb through all the stuff that they've been collecting. that's a pretty huge operational change.
You mean "Rhetorical question," and I didn't need patronising.
They have at least one pretty vast, largely classified data centre in Utah, with a sizeable chunk of the black budget and they also have pretty large data sets.
I can't say what they're doing now because I worked for the NSA 15 years ago but the view of them as an omnipotent power is a product of Hollywood. The government is good at throwing an ungodly amount of resources at something to get a result through sheer attrition, and so they are often the source of original development of technologies. The private sector has always been much better at building a technology to greater sophistication and efficiency. There may be blue badgers in Fort Meade trying to train models but there is no chance they are competitive with the frontier AI companies. It's like saying the government has an amazing home-grown fighter aircraft that is beyond what Lockheed has ever made...they delegate that stuff to private companies for a reason.
Blue badges were for government employees (like I was), and green badges were for private contractors. And yes they have a lot of math and physics guys; my own physics lecturer was in my orientation class, actually. He was there for quantum computing, which reinforces my point. The government can be good at pioneering unproven / uncommercialized technologies, but in general they are like a blunt weapon; the profit motive and lack of bureaucracy eventually makes the private sector far better for improving the technology later. In the case of LLMs, they didn't even originate in government, and I don't think there's any chance they are being developed there at a more advanced level.
Crypto and AI are deeply connected, and you see similar structures/problems in both. Shannon, the “Father (or whatever) of AI”, worked for the NSA and published many papers there that were later declassified.
Here is a banger quote on this by Shannon’s boy Warren Weaver, keeping in mind LLMs came from translation problems:
“One naturally wonders if the problem of translation could conceivably be treated as a problem in cryptography. When I look at an article in Russian, I say: 'This is really written in English, but it has been coded in some strange symbols. I will now proceed to decode.”
The NSA is government agency. They are certainly not training any world class LLMs. They probably have some specialized fine tunings of existing models, but that's it. They don't have the capacity.
Understatement. They have 14 offices, only 4 of them are in the US (6 are in EMEA, 4 in APAC).
Did Hegseth pull his supply-chain risk BS?
The DPA only gives that power to the President [1].
[1] https://en.wikipedia.org/wiki/Defense_Production_Act_of_1950
Propaganda.
IPO incoming.
No, they don't.
If I have a box at Digital Ocean and I'm communicating with it with TLS1.3 using a Let's Encrypt cert that I generated, where, exactly, does this magical MITM box come into play?
And these days (especially post-Snowden), many (most?) companies encrypt data when sending between servers within their own (private network) infrastructure.
Yours and others' claims that it's impossible and nonsensical is based on lack of understanding.
Yours and others' claims that things somehow got better after Snowden is just a completely baseless statement - if you actually looked into what happened post-Snowden - absolutely nothing was done to prevent NSA spying on any communications they want, in fact it got significantly worse.
lol, no, it's really not.
> Also when talking about encryption between servers within datacenters you seem to be missing that in order for such multi -stage/path encryption (separate certs/keys) to be possible the data first has to be decrypted at each point
Why would I want the data to be decrypted at each point and why would datacenters do that? Encrypting and decrypting data is expensive computationally, so that's not how things work at all. There's no need to decrypt data to know where it needs to go. That's why we have TCP/IP and other similar stadards.
The datacenters can maybe add another layer of encryption on top of my data as its moving around their networks, but there's absolutely no way for them to strip off my encryption.
> Yours and others' claims that things somehow got better after Snowden is just a completely baseless statement
Things didn't magically get better. A lot of people worked hard to improve the overall security posture of the industry.
I think they mean the data must have existed in plain text before it was encrypted, and will exist in plain text after it is decrypted.
At some point “your” server in a datacenter somewhere needs to decrypt the data to do something useful with it, after all you’re paying for compute, and homeomorphic encryption is too slow, so the work is done in unencrypted data.
There it is. Your data in plain text in RAM.
TLS will protect your data in transit, but it can’t protect you against a compromised recipient.
They could just do this to the specific servers they want, at specific times.
Just like wiretapping didn’t mean listening to every phone, and every conversation.
Yeah it definitely is lol.
> Why would I want the data to be decrypted at each point and why would datacenters do that?
When we talk about data that is sent for processing to a 3p server (like anthropic in this case) the data obviously needs to be decrypted to be processed.
As to why data is decrypted at each point in a typical large backend system - because other than network routing there are presumably multiple services that need to receive and act on this data somehow - you're not just sending encrypted data around to random servers.
> there's absolutely no way for them to strip off my encryption.
You don't seem to understand that you have no control over the encryption or decryption done on the backends of cloud services you use. I don't know how to make it more simple and obvious at this point.
Again, the context here is Anthropic and sending your data to their (or any other big tech API). But even if we move away from this model and suppose you are running your own services on rented cloud VM - then it should be obvious that you don't have full control or even access to this VM... any actor with access can install or modify any software, install/modify EBPF, modified crypto libraries, etc. - you have absolutely no control or say over this.
> Things didn't magically get better.
Things didn't get better at all, they got much worse.
Do you know what hypervisor is managing it? :)
https://en.wikipedia.org/wiki/Room_641A
Yeah, they did (and probably do).
Are you suggesting they broke TLS or that they've somehow acquired every private cert generated?
Plot twist: _Perhaps_ Mythos / Fable keeps explaining ways (that we can't comprehend or don't always work) to break HTTPS due to the three letter agencies making sure they had input on their creation (and thus backdoors, I mean "bugs"), so the real catastrophe they are hiding is that HTTPS is broken (for most people, most of the time.)
Remember when Quantum computing was the threat to HTTPS? Turns out it was the humans own inability to think outside of the box!
It just doesn't protect you all that well from nation-scale adversaries.
There's really no way this conspiracy theory works if "they" have a copy of every single private cert generated. Which would be impressive because I can generate one myself and get it trusted without ever sending it and would be easily able to detect a MITM attack.
Not to mention most sites are going to use pinned certs so any repeat visitors to a site will notice a cert change associated with a MITM.
This whole idea relies on the assumption that everyone is trusting third parties with their private certs. That is not at all required.
I'm not sure why your focus is so heavily on your server. Is that the only thing on the internet you care about?
> Not to mention most sites are going to use pinned certs so any repeat visitors to a site will notice a cert change associated with a MITM.
Most haven't even heard of pinned certs.
https://dl.acm.org/doi/10.1145/3517745.3561439
"we find that 0.9% to 8% of Android apps and 2.5% to 11% of iOS apps use certificate pinning at run time"
The long game. They:
- make sure you wouldn't be in a position to need to transmit data anywhere that would receive it without CA's in their hypothetical pocket
- manage the evolution of the cloud industry to make sure portable VM's and Containers can have their data archived (both in-RAM, disk, hey just send us the running VM!)
- backdoor'd encryption algorithms from the design and implementation phase to ensure a global unlocking mechanism for any data encrypted by anybody who used a large class of extremely commonly available software
So, you run your own private bank in a cloud VM with tenant managed keys? They backdoor'd the encryption algorithm your cloud VM disk relies on, because they blackmailed one of the developers at the company who developed the hypervisor system used by your provider. Open source project? Perfect. (If you think this is nonsense, then remember the rapid discovery of ancient "bugs" causing all this drama to begin with.)
Your TLS privately generated certs that are 100% foolproof aren't actually used anywhere encrypting the data they want, because it's either worthless, or, available elsewhere perhaps at a different (or same) time.
https://en.wikipedia.org/wiki/Dual_EC_DRBG
https://en.wikipedia.org/wiki/Bullrun_(decryption_program)
If you're a specific target of a nation-state level actor, things get worse; they just grab your hardware mid-shipment on its way to you.
https://www.nbcnews.com/tech/tech-news/report-nsa-intercepts...
And failed.
> If you're a specific target...
If you're a specific target, they have to spend an incredibly number of man-hours and money to get into your private data. This proves my point. This shows the effort required to infiltrate _one_ target and you're suggesting they've infiltrated everything by default.
How would you know about the successes? Thinking this is the one and only time they tried it is... interesting.
(Plus: "it was, for seven years, one of four CSPRNGs standardized in NIST SP 800-90A")
> If you're a specific target, they have to spend an incredibly number of man-hours and money to get into your private data.
No, this demonstrates an actor of that power level doesn't even need to compromise encryption, and can get deeper access to everything, if it's worth it to them.
It's too complicated. Do you know everything about CA, SSL, HTTPS, and so on? You make $250k a year working on it? Do you _really_, _really_, know everything? Then you're fired because you're lying to yourself, so you're probably unbearable to work with.
We were all freaking out about this with AT&T Thing nearly twenty years ago: and when nobody cared (Bush ran two terms! it helped to pretend AT&T was the only one affected), it gave "them" implicit permission to do it again with Google / Yahoo thing (it helped to pretend those were the only two cloud providers affected) ten years ago.
Now, we're all pretending that capitalism is real, and that the three letter agencies are just sittin' on the sidelines, while the world's largest data archiving opportunity is happening voluntarily (some are even PAYING for it!), at some wild-growth companies (with leaders who have too much to lose), who also have existed for just a few years? A 5 year old could probably blackmail Sam Altman, what about all the other middle management? The individual contributors (if they still exist) are of no concern: work is a commodity, it's easy to silo a worker's knowledge.
Surveillance opportunity is 10x social media from last decade, because they still have social media, and now, they've began thinking for people. How easy when it is an app on your smartphone. Those mind control experiments back in the 60's with Acid are looking silly by now. Besides, how do you know that the response you're getting wasn't manipulated (and define 'manipulated' across a spectrum of training to nefarious actors impersonating models, by power of court order.)
If you think all of that is unfounded ridiculous blasphemy, let me distract you with this instead: if the AI bubble bursts, the compute will be repurposed for mass AI / ML driven CCTV surveillance. Hell, maybe they'll find a way to give you a tax break if you sell your CCTV footage.
"NSA literally has MITM proxies/interception of any traffic they want inside every major US tech company" even if this statement is an exaggeration, by playing the long game, they get themselves setup to access what they want in the future.
I'm not for or against, but I do live in a safe place thanks to such surveillance (generally in the USA), and I want you to know that this AI Thing is only the latest chapter in the intelligence story.
As for the rest of this... how many conspiracy theories are you trying to pack into a statement?
> "even if this statement is an exaggeration"
It's not an exaggeration, it is simply false.
the nsa has an unlimited budget and spend a good portion of that budget recruiting some of the smartest people in the country. while they dont have super powers, they also arent the town cop who took a 6 month course after high school then joined the force.
it does no good to hold them up as mythical figures. it also does no good to pretend they are bumbling idiots.
(every math phd i am acquainted with has been approached by nsa recruiters. none of them have been approached by police agencies.)
Some of the smartest people I know have worked on fighting NSA, but they had a drastically smaller budget than NSA itself, and the mental availability bias is skewed by the fact that the "fighting NSA" people talked about their work all the time, while the "being NSA" people generally didn't.
I do know one extremely smart person who went to work there, and I witnessed a failed recruitment of another extremely smart person.
how many of them took them up on the offer, and how many are in leadership roles?
it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I'm not saying there aren't smart people working there but it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments... they just don't
this is not really relevant to the point, but to satisfy your curiosity: more than one, and one.
>it takes a very narrow range of personality to want to be a cop
the nsa's brightest aren't doing "cop" things. certainly none of the people i know of working there are "cop-minded" in any sense.
they are doing cool research and application things. otherwise they wouldn't be able to entice the phds to stick around. these are people that want to work at the forefront of their field, doing interesting work, and the nsa is one avenue of doing that (with good job security, benefits, etc.).
>it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments
we agree here. they are certainly doing "HNDL" (harvest now, decrypt later) at a very large scale. but obviously they are not able to collect and store every piece of communication at every tech company over years and years. (the intelligence community comprehensive national cybersecurity initiative data center is large, but not that large)
What? That's not only relevant to the point, it's incredibly relevant. If the NSA is only able to recruit 2% of the math PhDs they approach, then that's important information.
"More than one" is not particularly useful; you seem to be dodging the question because it undermines your argument.
telling you exactly how many people i know in the NSA is also not particularly useful. i'm one guy. there is no statistically significant information from my answer.
>you seem to be dodging the question because it undermines your argument.
my "argument" is that there are plenty of smart people in the NSA. that's it. i am confused why that is seemingly so offensive to you that you had to reply twice.
In my cohort? Several, and who knows? The recruitment effort is very visible and intense.
The US math phd market has been a slow-rolling disaster for over a decade. Everyone who can hack it outside the ivory tower is actively looking for the exits.
So why is it surprising that some of them go to work at the NSA?
> it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I don’t think you have context on what math phds are making in entry level positions, post-docs, or adjuncting. I just picked a random entry level NSA role on LinkedIn (doctorate + 0 yrs) and they’re offering solid six digits. There are tenured faculty (post-doc(s) + 5ish yrs) who don’t make that.
No they don't, and if you're going to try to argue something with that as your opener, it very easily casts large amounts of skepticism on whatever you are about to say.
Perhaps you're exaggerating for effect, but that also undermines your point.
if you read my comment like we're having a normal conversation instead of a thesis defense, you'll get my point just fine.
https://www.wired.com/2013/10/nsa-hacked-yahoo-google-cables...
https://www.eff.org/nsa-spying
What we learned from that era includes things like
(1) spy agencies are incredibly aggressive and pursue tons of different angles to get access to things
(2) spy agencies have a lot of money
(3) spy agencies often have interpretations of law that would surprise the public or legal experts (and sometimes courts have issued sealed rulings permitting them to do things that surprise the public or legal experts later when they're unsealed)
(4) some people throughout different parts of society assume culturally that companies in a country "should" generally help the spy agencies of that country's government because they are the "good guys" or "on the same team" or whatever
These things are all pretty bad and scary, but they still don't imply absolutely infinite power or access, because all of them come with different kinds of pushback. People also just tell them no!
I want to write an article with a colleague about the continuing role of culture here, because I think there are companies or industries where the default reaction is to want to cooperate with the government, and others where the default reaction is not that.
There are certainly secret things that have never come out, e.g. whatever Senator Wyden keeps alluding to, and what kind of program or authority was behind the interception of hardware shipments to covertly tamper with them, and whether there is a bulk financial data interception program, and presumably lots of other stuff. I don't agree with these things, and I want them to be exposed and stopped, and I also don't think they constitute infinite power over all parts of the tech industry.
Sorry everyone: but the conspiracy is so obviously not, it's nauseating to admit, because you see all your friends, family and co workers dumping so much everyday data into these services.
https://en.wikipedia.org/wiki/Defense_Production_Act_of_1950
This would not be a particularly big stretch here, either.
The US constitution also prohibits:
- refusing to spend money that congress has appropriated
- dismantling congressionally-created federal agencies without congressional authorization
- directing federal agencies to selectively apply the law according to the preference of the executive
- giving control of federal agencies to individuals who have not been appointed by the legislative branch
- terminating, detaining, or deporting people without due process
- retaliation against private citizens or corporations for speech protected under the first amendment
- discriminating on protected grounds under the equal protections clause
... and yet the administration has done all these things with impunity while effete judges wring their hands and write sternly-worded letters. The US constitution demonstrably no longer has any force or effect.
It would be easy to make a national security justification to take the weights in a clandestine manner especially because Anthropic supposedly got caught giving China access to the model through a cutout.
John Cook?
If a government can just seize the product of someone else's labour, either they will end up as slave owners or without willing workers.
It's a perfect technology for their uses, they get a big chunk of a $100 billion black budget, and they've had access to the research for at least as long as we have.
I wouldn't count them out.
Especially academia tend to do their work out of interest, their monetary gain isn't their primary goal
Of course, that doesn’t mean nobody will do job B for other, non-financial reasons.
That sort of proves the opposite point, assuming you're referring to Dual EC DRBG, because the flaw was noticed very early on, by people who weren't even involved in its development.
Given the evergreen discussion of "are these companies making a profit"*, I think any LLMs that the NSA (or any other government agency worldwide) may be making are quite far from the leading edge.
* Person A: "they are making a loss!" Person B: "Only if you count training, they make a profit on inference, look at what it costs to run comparable open models on generic cloud servers" A: "Sure, but if they don't train new models they'll be left behind, so they're still making a loss"
That and the way compute is now measured in GW, I think even random low budget vloggers just getting started would be able to spot if the NSA was doing anything significant just from the extra heat emissions or power plants getting built.
The rate of inference compute to training compute is ~10:1, for popular frontier models. Models are routinely overtrained past the Chinchilla optimum now because it makes an immense amount of economic sense to do so.
Worse the more niche and unused your models get, but when this "making a loss" fuckery pops up, it's usually about the big guys like Anthropic, OpenAI, GDM and maybe xAI and Meta. Of which only the latter can be accused of not selling enough inference to offset the training runs.
The real money sinks are: R&D and infrastructure buildouts.
They probably already have access to Sentinel, so they wouldn't need to train their own.
- find "modern AI" to have strategic importance
- have ways to spend loads of money while having a front-facing budget on the record
- could be running a PR program to have Americans think they "buy" access to models like they do, but the AI companies were taken over by these agencies long ago
Look at Google, Microsoft...Apple got away with it by having as much on-device operation as possible so they could wash their hands, honestly saying "We don't have it."
This is the world's largest data gathering operation. Remember after 9/11 when the NSA copied as much Internet back bone traffic as they could?
I'm not for or against, even as a resident, but we certainly shouldn't be naive.
the issue here that is a forgone conclusion, regardless of where the model comes from and which chips it runs on, is that now they can reasonably comb through all the stuff that they've been collecting. that's a pretty huge operational change.
They have at least one pretty vast, largely classified data centre in Utah, with a sizeable chunk of the black budget and they also have pretty large data sets.
Anyways, isn’t NSA one of the largest employers of mathematicians in the world? Surely they’re doing something useful.
Here is a banger quote on this by Shannon’s boy Warren Weaver, keeping in mind LLMs came from translation problems:
“One naturally wonders if the problem of translation could conceivably be treated as a problem in cryptography. When I look at an article in Russian, I say: 'This is really written in English, but it has been coded in some strange symbols. I will now proceed to decode.”
I mean yes, in both deal with information theory.
That's a long way from any practical insight.