AI-generated text warning (I submitted - but did not author - the piece), but it seems MS Paint and MS Photos add both a visible (can be turned off) and invisible (cannot be disabled and happens silently in the background with no user notice) watermarks to photos that have been AI-manipulated, even when using a local model to perform the action. It's not clear if this applies to even things like using AI-enhanced background delete/remove, but the invisible watermark is embedded in both the image pixels and the image metadata, both containing a GUID that can be linked to the exact prompt that was used and the originating device/user (on Microsoft's end).
Obvious next step is to explore if you can replace watermarker.dll with a (signed) no-op shim or MITM the API call to at least use your own (nil?) GUID that isn't linked to your device/account.
In case it's not obvious, my bigger concern isn't "this image can be identified to have been generated with/by AI" so much as it is "digital yellow printer dots have been forced upon us, except they can identify and retrieve the exact user/device/time/place/document/etc", completely destroying any and all illusions of privacy left.
I'd like to know more about the GUID part and how easy is it so deanonymize yeah.
But if it's only on ai generation and not on all images it seems easy enough to work around that part? Still better than printers doing it no matter what you're printing.
Sounds better to do postprocess filtering explicitly? For example, adding random noise with amplitude 1/255 (which should be as invisible as the watermark), followed by a smart blur that blurs more in directions where the colors are more similar (making the blur less obvious to humans).
But this is all moot really, if MS Paint is watermarking shit, it’s better to just use something else. Nothing from Microsoft is trustworthy.
There are several paragraphs where output looks very AI-like (and Claude flavored one at that), e.g.
> In other words, “generated locally” does not mean that the complete operation is local. Microsoft receives and moderates the prompt, then issues the unique GUID that Paint embeds into the locally generated image. Paint also sends the previous promptGenerationId as lastPromptGenerationId with its next moderation request, allowing successive requests to be linked explicitly.
> That relationship is important. C2PA calls this a soft binding: a value derived from, or embedded into, the content so that the content can still be matched with its provenance record after the file-level manifest has been removed. For a watermark soft binding, the value is the watermark’s content identifier. Microsoft cryptographically signed this assertion.
> After an AI result is applied to the Paint canvas, the available formats are still restricted to PNG, JPEG, GIF, and Paint’s own .paint format. BMP—the classic Paint format—is conspicuously absent.
Interesting. I really didn't think watermarks would end up going anywhere, but maybe with enough adoption we can have easy ai generated content flagging after all?
It actually might make the new horrible world even worse. Imagine the populace getting used to a AI image detector flagging things as fake using this fairly easily defeated GUID marker system. Most people are just making memes or cat videos and don't even try to remove this so eventually the populace starts to believe these things actually work.
Now some one slightly more sophisticated starts creating deepfakes of a woman and uploading them or fabricating video of an political event without this marker. The subject protests it's fake and AI generated but a loud majority of ignorants feed it into Microsoft AI detector and call you a liar and say it's confirmed real. Most people don't know any better and eat it up because a computer said so.
US Social Security Numbers are a useful analogy: The designers knew they weren't something anyone could securely depend on, and told people not to do it... but companies did it anyway, for their own convenience and cheap security-theater. Ultimately a lot of individual victims suffered for it.
It would have almost been better if one big incident blew the "knowing an SSN means something" myth apart early on.
Presumably the OP is proposing something like a TPM attached to the image sensor that signs the sensor output or something like that. You can’t sign it because you can’t get the key out. The key could be per-camera and be a published list.
I suppose a dedicated fraudster could still stage an appropriate scene. An appropriately lit matte image might even suffice.
That's assuming they don't just have a backdoor inserted expressly for this purpose. Now only the rich or powerful can produce an "authentic" recording of an event and the same system can be used to hunt down whistleblowers and political enemies by looking up who bought the camera.
At the very least there is always the analog hole where you point the camera at a sufficiently high quality screen. The sensor can't tell whether the light it receives was emitted by sunlight reflecting (and refracting) from someone's face or by a bunch of LEDs pointed at the sensor.
Keys could be stored in something like TPM on Camera, and could sign the image. The key could then be verified from the camera itself to prove the authenticity of the image.
If we as a society deemed it necessary, the camera manufacturer could also provide a list of keys for devices they have manufactured. And an image/key could be provided, and the manufacturer could verify the authenticity that way.
The TPM signing could be tied into the sensor hardware itself, making it difficult, but not impossible, to sign arbitrary images with the TPM.
The point of the key (as the for some reason dead comment points out), is not to prove who took the photo, but what device took the photo. Just as if someone stole a hardware token with a PGP key on could impersonate the owner. The key itself doesn't prove a person, just a device.
If a key was reset, a revocation of the original key could be issued, showing that the key was associated with the device for this particular time span. And then the new key registered.
This is ripe for abuse though, so resetting a TPM might not be accepted for this use case. I'm not certain in which case you'd want to reset a TPM for this use case though. Unless you took enough photos with the device to risk a birthday attack if you were using something like ECDSA.
If the TPM signs the original image taken by the camera, then even the slightest image processing on another device would invalidate the signature. Routine changes like cropping, scaling, converting between image formats / quality levels, or applying image filters would invalidate the signature.
An adversary can manipulate the date/time settings on the camera and forge evidence to frame someone. "This cryptographically signed photo, with timestamp, proves that you were here at this time!"
And camera sensors can get damaged and need replacement. But if replacement of the TPM-and-sensor chip is allowed, then you can just as well replace it with a sensor from another camera. In which case a signature from a specific TPM+sensor doesn't prove that a specific camera took the photo, at best it might be evidence that a specific camera model took the photo.
If the manufacturer will happily ship a replacement TPM+sensor for a specific camera, someone can fraudulently claim that their sensor is broken and be given a new TPM+sensor for the same camera. And there will now be multiple TPM+sensors in existence that have the same key. Since this module can be switched between cameras, there could be multiple cameras that signed a given picture.
To ship (identical) replacement TPMs, the manufacturer would also need to know the private keys of all the cameras, so the manufacturer could forge arbitrary signatures at will.
Finally maybe the manufacturer doesn't want to deal with the above problems and decides that repairing the camera's TPM is not allowed after all, or that only the manufacturer is allowed to repair a camera, but then they may be in violation of right-to-repair laws in several jurisdictions.
If C2PA and similar signature systems ever become a meaningful authenticity signal, they will create huge incentives for someone (potentially a state actor) to hack at least one camera in order to sign images of arbitrary provenance with its private keys. This will in turn inevitably lead to the same game of cat-and-mouse we have seen play out with video DRM schemes, where keys are regularly extracted from exploitable devices and used to decrypt as much content as possible before the device gets blacklisted entirely (harming all legitimate owners in the process).
I've done this btw. I went for the Pixel Camera app since they were the ones bragging the hardest about their "security". Writeup + PoC should be dropping some time tomorrow. Despite 90+ days from initial report, it remains unpatched.
I don't think that that's a good idea, because it implies trust when there actually isn't any.
Being signed with something just means that whoever has that key could've done that. That might be the owner of a specific camera, but it might also be the camera manufacturer, anyone else in the supply chain, or anyone who dumped the key.
Imagine fake evidence signed with the same key as your camera uses being used in court against you. And the court believes it because it has this signature attached and those computers are very secure and all.
Exactly that will happen. Not widespread, of course, but it will.
When I was in photography class in college, I created backplates in photoshop for still life portraits of small trinkets I was photographing. The photos were taken on black and white film and developed in the campus dark room. Led to some impressive photos. In our class's critiques, I explained how it was done. A lot of peers went from impressed to meh'd. The point: the black and white film laundered the new-age manipulation, and a digitally signed photo from a modern camera remains vulnerable to the same premise.
"I think it would be nice if all pens added a unique isotopic tracer signature to their ink. You could tell exactly who wrote everything."
"I think it would be nice if all typewriters had their unique fine-detail type artifacts registered with the government. You could tell exactly who authored a given document."
I think it would be nice if you took these ideas back to Stalinist Russia where they belong.
No. You'd only ever be able to show that key material belonging to $specific_camera was used to sign/mark the image.
Was the camera manufacturer breached? Did somebody on the factory floor steal some keys during the provisioning step? Or did somebody build their own photo-sensor simulator and plug _that_ in to the camera's motherboard to feed it a "real" image? Before going _that_ far, just point the unmodified camera at a sufficiently high resolution display...
In the imaginary dream world that Adobe, Google et al live in, the final file does indeed have a signature.
Each piece of software in the chain must use TPM-like technologies (yes, even GIMP) to make sure it's running a "legitimate" build of the software, on "legitimate" hardware, and re-sign the file at each step along the way (using keys provisioned during some flavour of remote attestation flow, or using a RA-authenticated remote-signing oracle).
The final file embeds every preceding manifest, so you can "verify" all the way back to the original.
If this all sounds patently unworkable, that's because it is.
OK, but given that GIMP is a general purpose tool, what use is the signature if all of them verify it, when I can drastically change the image to whatever I want it to be?
The manifests at each step can embed a thumbnail (although this is optional, iiuc!), so looking at the thumbnail history it should be obvious that the edit was significant.
I had this trigger the other day incorrectly and went and installed Paint.net. I pasted in a screenshot I took and just wanted to resize it. I got a banner saying it was made with AI and would be updated to reflect that.
I guess it shouldn't be surprising if an application called "paint.net" can determined if AI was used when connected to the internet. (I have used Paint.net more than a decade ago).
I was stating that MS Paint mis categorized a Print Screen screenshot as AI generated when pasted. Which pushed me to install PaintdotNet onto my laptop instead.
> mis categorized a Print Screen screenshot as AI generated
Maybe. Question is, was that a false positive, or merely a bad user-facing message?
Assuming you didn't have any AI-generated images caught in the screenshot (e.g. some advert - plenty of those even in Microsoft apps like Weather, nowadays), what if - and I'm speculating here; I don't know if anyone actually does it yet, but it's so obvious they eventually will -
- what if parts of your screenshot already had an invisible watermark on it, like the ones for tagging AI images? You can imagine an app rendering its window to texture, and embedding a watermark on those pixels before sending it over to the system compositor, and the reasons have nothing to do with AI.
Watermarks are DRM tech. "AI generated" is just a specific kind of metadata that can be put in one, and a huge red herring for discussions.
It's only a matter of time before we'll need to have software for detecting and removing real-time watermarks from display output at OS level. (Unless, of course, platforms decide to add app window live watermarking as a "feature", and given the story with remote attestation, I think it's more than likely.)
I get the privacy concerns, and we are right to expect Microsoft to say that this is what their tool may be doing. However, I fear that one day we will look back and wonder why we didn't do more to sign and preserve human authenticity. Having a stamp saying "AI manipulated" should be a part of digital lineage tooling.
Thanks Microsoft, for adding my signature so I won't have to claim authorship when it ends up in a museum in 200 years, and the NSA archives are declassified for art historians filing a FOIA in 2226, who find out, "yep, it was from his PC."
No, I wasn't suggesting that. I was saying that if there was digital art (human made) aesthetically significant that a curator would want to display it in a museum, Microsoft's GUID supplied to a data collection agency would make it possible to retrieve if ever/whenever that data were declassified (assuming it isn't purged)
It's possible a very bad curator with a terrible taste in art might select slop to display, but I was refering to "fine art" or at least finer art that is digital.
Microsoft quietly embeds a hidden tracking identifier in every AI-generated image you create using Paint or Photos on Windows.
A researcher discovered that these apps embed a server-issued GUID (a globally unique identifier) as an invisible watermark in locally generated AI images.
The watermark is tied to the prompts you type. And since those prompts are associated with your Microsoft account, Microsoft could "theoretically" trace any watermarked image back to the user who created it.
This is recycling an idea from the 80s. Back then, laser printer manufacturers added tiny yellow dot patterns to every printed page. With this, they could identify the printer.
Now Microsoft has brought the same idea to AI image generation, and added it to two of the most widely used default Windows apps.
Microsoft had disclosed its AI safety measures in official documentation, but the practical implication, that your output image file carries an invisible fingerprint linked to your identity, was never clearly mentioned, of course.
The researcher found this by reading Microsoft's own published documentation and analyzing the watermarking mechanics.
AI watermarking is a requirement by law in the EU. But "this image was generated by AI" is different than "this image was generated by AI by Mr. Winston Smith".
For anyone who values privacy, this is something to worry about. If you generate an image locally, on your own device, why should it carry a tag that can identify you to the company whose software you used?
But then, anyone who values their privacy won't be using Microsoft Windows anyway."
Since the cat is out of the bag, I wouldn't be surprised if Microsoft generates an invisible watermark for ALL files and not just AI generated ones. The real story is that since AI watermarks are possible, there is no technical barrier to them adding personal EXIF metadata to a file where it can't be seen, removed, or decrypted, whether it is media, a document or other file.
The AI aspect of this is a red herring. The real problem is that they're secretly adding in a unique identifier into every image you create. If somebody does not like your meme, they can just send a copyright subpoena to Microsoft to instantly get your full name, address, email, phone number, and any other data associated with your Microsoft account. Just like age verification, this is another weapon in the war against internet anonymity.
It does kind of say: The GUID is coming from the moderation endpoint, which is hit when you generate a local or cloud AI image based on your prompt. If there is no prompt, there is no endpoint, and likely no GUID.
Obviously Paint could have been watermarking prior to AI though, but this specific AI watermarking appears to be only that.
Since the watermarker runs locally, you could also do the reverse: generate some porn with a different open model, then run that code to tag it as "Content watermarked by Microsoft Responsible AI"
Probably both so they can change what they censor without waiting for you to run windows update and also so they can collect your information (IP, timestamp, etc) to associate with whatever you did
Do not tell Microsoft where you sleep. IANAL but they are not a government or financial institution and do not have a right to that information.
Make sure you register mailing addresses with your credit card institutions in addition to your residential addresses, and make your mailing address your billing address so that you aren't forced to tell a thousand businesses like Microsoft where you sleep.
Windows 11 effectively forces users to register with a Microsoft account. Once that's established, all it takes is for an unaware user to fill out an e-commerce form and save an address for auto-fill.
My comment was in regard to the average PC user. The kinds of folks who would never install an operating system and would likely agree to use Edge while signed in to their Microsoft account.
Microsoft has also been pushing hard for Recall and recording all local activity forever. Not impossible to imagine that anything looking like a home address “somehow” gets ingested in the telemetry.
A bypass existing is actually not good because that's how the frog is boiled. Ignorant users get an account while power user are placated for now. Then at some point the bypass is removed and when power users speak up they will be told shut down by all the drones with "you have needed an account for years, why are you throwing a fuss now".
> Windows 11 effectively forces users to register with a Microsoft account.
While it's definitely a dark pattern that I 100% do not agree with, Pro editions still allow you to do a local account. No need for the oobe /bypass command, still can be done through the OOBE GUI setup by selecting a Work/School account option then selecting Sign in options to then specify a local account to create.
> This is broadly impractical for the average citizen.
No it's not. Sign up for a virtual mailbox for $15-$25/month.
> A scalable solution would be to make this sort of thing illegal.
I'm posting this in the genuine interest of people being able to maintain anonymity from data leaks, privacy leaks, and in general not needing to tell businesses more personal information than is necessary to render services. This is in a country that has no protection of personal safety whatsoever, and any business data leak could mean life or death to average citizens who are being threatened by criminals, stalkers, and more.
It seems every time I post something of this flavor the same handful of you come out of the woods and want to make privacy illegal, and I'm not sure who you are trying to support.
Ahhh okay if that's what they meant, then yes, I 100% agree and apologize in advance.
I fully agree that businesses should not be asking for addresses. Non-financial businesses don't need to KYC in the first place, and financial institutions can KYC without needing to know where you sleep.
I got triggered because people seem to always want to come out of the woods and say "addresses should be public record" or things of that sort and I vehemently disagree in the interest of privacy, in a country where a stalker can just look you up, terrorize you, and the police will do nothing about it.
They (the ones wanting to make privacy illegal) are just the ones that want to develop the tools and consume the data for it because it'll be good for their resume or some bullshit, or they're just intrinsically broken humans.
> No it's not. Sign up for a virtual mailbox for $15-$25/month.
Did you notice that it's an affordability crisis out there? An absolutely enormous number of people are skipping bills, taking on credit, and using predatory lenders to make end meets, and your recommendation is to add another fee on top of things.
It's not practical or useful guidance for the vast majority of people. I strongly agree with supporting privacy, but this sort of behaviour (adding trackers, etc) to normal functions without a full disclosure and opt-out mechanism must be made illegal, otherwise we are just creating markets for "privacy preserving" technologies that are increasingly less likely to actually be effective for that purpose but sure do put on a good theatre of seeming that way.
At least in the US it's not even a good method either. Own a house? Boom, your information is publically available and you won't be able to remove it without a court order, which in most states is impossible to get. In most states, you don't even need to own a house, decide to vote? You information again is publically available. Tie your real name to a single account to purchase something, and you are trackable. Want privacy? You'd need to remove yourself from the internet, dump your phone, destroy your ID and work underground for cash. We can push against iot, and people have been forever, but at the end of the day the government will always they have a vested interest in being able to identify it's citizens.
My sister was a member of Scientology for 2 decades and she was encouraged to do this. As a result, she has almost no Social Security earnings and her Social Security check is about 1/4 of what she would be getting if she reported (and paid taxes to the government instead of to her "church") her income.
> decide to vote? Your information again is publicly available
I ran for elected office in the past. When I asked the voter registration office for a "walking list", they only checked that I did get on the ballot and I was provided with a list of every registered voter in the district I was running for. It had names, addresses, phone numbers, political party and a list of what elections you voted in (I think it went back 8 years). Some people were rather upset that I had access to that information when I knocked on their door and asked them to vote for me.
In many states voter lists are publically available online. And those same databases will tell you what party the person voted in for primary elections. The amount of information publicly available about everyone in the US is astounding.
> Sign up for a virtual mailbox for $15-$25/month.
That isn't going to stop Microsoft from collecting your address by collecting your wifi info, or from the data you enter into websites or documents. When the maker of your OS is the enemy you will always lose.
> IANAL but they are not a government or financial institution and do not have a right to that information.
As soon as you purchase something from Microsoft - e.g. your Office 365 subscription - they have at the very least your billing address on file for the credit card.
Seems to me that it's a challenge to make an apple account without adding a credit card to your account as well. A carfully chosen Linux distro (so many options!) is the way to avoid the brunt of these privacy issues, lots of open software calls home in some way or another so user beware!
We gotta stop normalizing that. Why do you need my address to receive my money? My birth date?
Even in places with strong privacy regulations requiring businesses not to collect data they don’t need, businesses apparently get away with asking this.
The problem is CC processors. You can get away as a merchant with just a number and expiration date, that's the minimum you need to process a CC payment, but you pay exorbitantly higher fees. The more details you collect and pass on, the lower your processor's risk fee will be because the more info, the more likely it is that the customer is actually the cardholder and not using fraudulently obtained data.
And as soon as you connect with telephony systems (e.g. VoIP numbers) or rent out servers, some countries' telecom KYC laws apply that also force MS to collect validated address data.
Even without the CC processor requirements they would want to be able to identify you in case of a payment dispute. Digital payments are more like an IOU than like cash.
In the rest of the world CC processing uses a PIN code + online revocation check (when the card is present) or a redirect through your bank (which hopefully uses some type of 2FA). Works a lot better & safer, and you should wonder why they pretend to need to violate your privacy in the USA…
Because of the volume of stolen credit card numbers. If you have the address, you are much less likely to be someone using stolen CCs. Seriously, most of the crooks are that lazy. So the real reason is the credit card processing companies.
Also, post 911, there is this thing called "know your customer". It was set up to fight money laundering and financing terrorists.
As the article explained, the EU does not mandate a prompt-specific GUID, only the ability to identify the content as AI-generated. The highly privacy-invasive level of provenance tracking which Microsoft has added goes beyond the EU’s new mandates.
So you will defeat chat control by sitting there going “but you don’t have muh gdpr consent for that!”?
The whole EU vision is they know what the little people do all the time, think all the time, and spend their money on all the time. For the children, obviously.
If you think "chat control" is going to pass, I've a bridge to sell you.
In the meantime, the GDPR is your friend. The amount of FUD spread about it on here by those working in Adtech (and whose very salaries are dependent on invading peoples privacy) is insane.
> If you think "chat control" is going to pass, I've a bridge to sell you.
I genuinely wish you were right, but you are so naive it is frightening.
> The amount of FUD spread about it on here by those working in Adtech
You are confused. You are arguing with people that want to protect privacy. The EU demonstrably is not doing that, as WhatsApp gets ever more de facto mandated by the day.
Obvious next step is to explore if you can replace watermarker.dll with a (signed) no-op shim or MITM the API call to at least use your own (nil?) GUID that isn't linked to your device/account.
In case it's not obvious, my bigger concern isn't "this image can be identified to have been generated with/by AI" so much as it is "digital yellow printer dots have been forced upon us, except they can identify and retrieve the exact user/device/time/place/document/etc", completely destroying any and all illusions of privacy left.
But if it's only on ai generation and not on all images it seems easy enough to work around that part? Still better than printers doing it no matter what you're printing.
convert file.jpg file.bmp; convert file.bmp file.jpg
But this is all moot really, if MS Paint is watermarking shit, it’s better to just use something else. Nothing from Microsoft is trustworthy.
This seems incorrect to me. Are you basing that on the use of bullet points?
> In other words, “generated locally” does not mean that the complete operation is local. Microsoft receives and moderates the prompt, then issues the unique GUID that Paint embeds into the locally generated image. Paint also sends the previous promptGenerationId as lastPromptGenerationId with its next moderation request, allowing successive requests to be linked explicitly.
> That relationship is important. C2PA calls this a soft binding: a value derived from, or embedded into, the content so that the content can still be matched with its provenance record after the file-level manifest has been removed. For a watermark soft binding, the value is the watermark’s content identifier. Microsoft cryptographically signed this assertion.
> After an AI result is applied to the Paint canvas, the available formats are still restricted to PNG, JPEG, GIF, and Paint’s own .paint format. BMP—the classic Paint format—is conspicuously absent.
Personally it didn't bother me too much.
Everything is spying on us now. Literally everything. I recently downgraded my MacBook M1 to Sonoma to avoid all this AI privacy invading BS.
Took me a moment to realize you're saying someone else generated it, rather than you did.
Now some one slightly more sophisticated starts creating deepfakes of a woman and uploading them or fabricating video of an political event without this marker. The subject protests it's fake and AI generated but a loud majority of ignorants feed it into Microsoft AI detector and call you a liar and say it's confirmed real. Most people don't know any better and eat it up because a computer said so.
US Social Security Numbers are a useful analogy: The designers knew they weren't something anyone could securely depend on, and told people not to do it... but companies did it anyway, for their own convenience and cheap security-theater. Ultimately a lot of individual victims suffered for it.
It would have almost been better if one big incident blew the "knowing an SSN means something" myth apart early on.
I suppose a dedicated fraudster could still stage an appropriate scene. An appropriately lit matte image might even suffice.
If we as a society deemed it necessary, the camera manufacturer could also provide a list of keys for devices they have manufactured. And an image/key could be provided, and the manufacturer could verify the authenticity that way.
The TPM signing could be tied into the sensor hardware itself, making it difficult, but not impossible, to sign arbitrary images with the TPM.
If I need to reset TPM, how do I reclaim photos I took previously?
If a key was reset, a revocation of the original key could be issued, showing that the key was associated with the device for this particular time span. And then the new key registered.
This is ripe for abuse though, so resetting a TPM might not be accepted for this use case. I'm not certain in which case you'd want to reset a TPM for this use case though. Unless you took enough photos with the device to risk a birthday attack if you were using something like ECDSA.
If the TPM signs the original image taken by the camera, then even the slightest image processing on another device would invalidate the signature. Routine changes like cropping, scaling, converting between image formats / quality levels, or applying image filters would invalidate the signature.
An adversary can manipulate the date/time settings on the camera and forge evidence to frame someone. "This cryptographically signed photo, with timestamp, proves that you were here at this time!"
And camera sensors can get damaged and need replacement. But if replacement of the TPM-and-sensor chip is allowed, then you can just as well replace it with a sensor from another camera. In which case a signature from a specific TPM+sensor doesn't prove that a specific camera took the photo, at best it might be evidence that a specific camera model took the photo.
If the manufacturer will happily ship a replacement TPM+sensor for a specific camera, someone can fraudulently claim that their sensor is broken and be given a new TPM+sensor for the same camera. And there will now be multiple TPM+sensors in existence that have the same key. Since this module can be switched between cameras, there could be multiple cameras that signed a given picture.
To ship (identical) replacement TPMs, the manufacturer would also need to know the private keys of all the cameras, so the manufacturer could forge arbitrary signatures at will.
Finally maybe the manufacturer doesn't want to deal with the above problems and decides that repairing the camera's TPM is not allowed after all, or that only the manufacturer is allowed to repair a camera, but then they may be in violation of right-to-repair laws in several jurisdictions.
Some proof: https://verify.contentauthenticity.org/?source=https://retr0...
I could also paste a privkey + cert chain in here but el goog's lawyers might not like that.
Being signed with something just means that whoever has that key could've done that. That might be the owner of a specific camera, but it might also be the camera manufacturer, anyone else in the supply chain, or anyone who dumped the key.
Imagine fake evidence signed with the same key as your camera uses being used in court against you. And the court believes it because it has this signature attached and those computers are very secure and all.
Exactly that will happen. Not widespread, of course, but it will.
"I think it would be nice if all typewriters had their unique fine-detail type artifacts registered with the government. You could tell exactly who authored a given document."
I think it would be nice if you took these ideas back to Stalinist Russia where they belong.
No. You'd only ever be able to show that key material belonging to $specific_camera was used to sign/mark the image.
Was the camera manufacturer breached? Did somebody on the factory floor steal some keys during the provisioning step? Or did somebody build their own photo-sensor simulator and plug _that_ in to the camera's motherboard to feed it a "real" image? Before going _that_ far, just point the unmodified camera at a sufficiently high resolution display...
> build their own photo-sensor simulator and plug _that_ in to the camera's motherboard to feed it a "real" image?
Already been done!
https://hackerfactor.com/blog/index.php?/archives/1102-C2PA-...
TL;DR: Strong cryptography over untrusted data does not make the data more trustworthy.
People take RAW photos. Load it up in a RAW editing tool. Manipulate it. Then load it in Gimp. Manipulate some more.
Will the final result have the signature?
And if it does, what use would it be?
Each piece of software in the chain must use TPM-like technologies (yes, even GIMP) to make sure it's running a "legitimate" build of the software, on "legitimate" hardware, and re-sign the file at each step along the way (using keys provisioned during some flavour of remote attestation flow, or using a RA-authenticated remote-signing oracle).
The final file embeds every preceding manifest, so you can "verify" all the way back to the original.
If this all sounds patently unworkable, that's because it is.
Maybe. Question is, was that a false positive, or merely a bad user-facing message?
Assuming you didn't have any AI-generated images caught in the screenshot (e.g. some advert - plenty of those even in Microsoft apps like Weather, nowadays), what if - and I'm speculating here; I don't know if anyone actually does it yet, but it's so obvious they eventually will -
- what if parts of your screenshot already had an invisible watermark on it, like the ones for tagging AI images? You can imagine an app rendering its window to texture, and embedding a watermark on those pixels before sending it over to the system compositor, and the reasons have nothing to do with AI.
Watermarks are DRM tech. "AI generated" is just a specific kind of metadata that can be put in one, and a huge red herring for discussions.
It's only a matter of time before we'll need to have software for detecting and removing real-time watermarks from display output at OS level. (Unless, of course, platforms decide to add app window live watermarking as a "feature", and given the story with remote attestation, I think it's more than likely.)
It's possible a very bad curator with a terrible taste in art might select slop to display, but I was refering to "fine art" or at least finer art that is digital.
:It's FOSS
3h •
Microsoft quietly embeds a hidden tracking identifier in every AI-generated image you create using Paint or Photos on Windows.
A researcher discovered that these apps embed a server-issued GUID (a globally unique identifier) as an invisible watermark in locally generated AI images.
The watermark is tied to the prompts you type. And since those prompts are associated with your Microsoft account, Microsoft could "theoretically" trace any watermarked image back to the user who created it.
This is recycling an idea from the 80s. Back then, laser printer manufacturers added tiny yellow dot patterns to every printed page. With this, they could identify the printer.
Now Microsoft has brought the same idea to AI image generation, and added it to two of the most widely used default Windows apps.
Microsoft had disclosed its AI safety measures in official documentation, but the practical implication, that your output image file carries an invisible fingerprint linked to your identity, was never clearly mentioned, of course.
The researcher found this by reading Microsoft's own published documentation and analyzing the watermarking mechanics.
AI watermarking is a requirement by law in the EU. But "this image was generated by AI" is different than "this image was generated by AI by Mr. Winston Smith".
For anyone who values privacy, this is something to worry about. If you generate an image locally, on your own device, why should it carry a tag that can identify you to the company whose software you used?
But then, anyone who values their privacy won't be using Microsoft Windows anyway."
Since the cat is out of the bag, I wouldn't be surprised if Microsoft generates an invisible watermark for ALL files and not just AI generated ones. The real story is that since AI watermarks are possible, there is no technical barrier to them adding personal EXIF metadata to a file where it can't be seen, removed, or decrypted, whether it is media, a document or other file.
Obviously Paint could have been watermarking prior to AI though, but this specific AI watermarking appears to be only that.
Why not just mod the app to not call this API?
Besides, you need to sign-in and pay to use that feature. It's very obvious that's not local.
I don't have a Copilot PC, but if you still need to pay MS to run the model on your own hardware it's laughable.
Do not tell Microsoft where you sleep. IANAL but they are not a government or financial institution and do not have a right to that information.
Make sure you register mailing addresses with your credit card institutions in addition to your residential addresses, and make your mailing address your billing address so that you aren't forced to tell a thousand businesses like Microsoft where you sleep.
It takes zero effort to bypass that with Rufus, if you set up your own pc.
While it's definitely a dark pattern that I 100% do not agree with, Pro editions still allow you to do a local account. No need for the oobe /bypass command, still can be done through the OOBE GUI setup by selecting a Work/School account option then selecting Sign in options to then specify a local account to create.
No it's not. Sign up for a virtual mailbox for $15-$25/month.
> A scalable solution would be to make this sort of thing illegal.
I'm posting this in the genuine interest of people being able to maintain anonymity from data leaks, privacy leaks, and in general not needing to tell businesses more personal information than is necessary to render services. This is in a country that has no protection of personal safety whatsoever, and any business data leak could mean life or death to average citizens who are being threatened by criminals, stalkers, and more.
It seems every time I post something of this flavor the same handful of you come out of the woods and want to make privacy illegal, and I'm not sure who you are trying to support.
or make it illegal to ask for address, etc. definitely a little more effective than mailboxes
I fully agree that businesses should not be asking for addresses. Non-financial businesses don't need to KYC in the first place, and financial institutions can KYC without needing to know where you sleep.
I got triggered because people seem to always want to come out of the woods and say "addresses should be public record" or things of that sort and I vehemently disagree in the interest of privacy, in a country where a stalker can just look you up, terrorize you, and the police will do nothing about it.
Did you notice that it's an affordability crisis out there? An absolutely enormous number of people are skipping bills, taking on credit, and using predatory lenders to make end meets, and your recommendation is to add another fee on top of things.
It's not practical or useful guidance for the vast majority of people. I strongly agree with supporting privacy, but this sort of behaviour (adding trackers, etc) to normal functions without a full disclosure and opt-out mechanism must be made illegal, otherwise we are just creating markets for "privacy preserving" technologies that are increasingly less likely to actually be effective for that purpose but sure do put on a good theatre of seeming that way.
https://transparency.ky.gov/search/Pages/SalarySearch.aspx#/...
> and work underground for cash
My sister was a member of Scientology for 2 decades and she was encouraged to do this. As a result, she has almost no Social Security earnings and her Social Security check is about 1/4 of what she would be getting if she reported (and paid taxes to the government instead of to her "church") her income.
> decide to vote? Your information again is publicly available
I ran for elected office in the past. When I asked the voter registration office for a "walking list", they only checked that I did get on the ballot and I was provided with a list of every registered voter in the district I was running for. It had names, addresses, phone numbers, political party and a list of what elections you voted in (I think it went back 8 years). Some people were rather upset that I had access to that information when I knocked on their door and asked them to vote for me.
You are out of touch.
That isn't going to stop Microsoft from collecting your address by collecting your wifi info, or from the data you enter into websites or documents. When the maker of your OS is the enemy you will always lose.
As soon as you purchase something from Microsoft - e.g. your Office 365 subscription - they have at the very least your billing address on file for the credit card.
Even in places with strong privacy regulations requiring businesses not to collect data they don’t need, businesses apparently get away with asking this.
And as soon as you connect with telephony systems (e.g. VoIP numbers) or rent out servers, some countries' telecom KYC laws apply that also force MS to collect validated address data.
Because handling that PII is not only cheap, it’s profitable!
Because of the volume of stolen credit card numbers. If you have the address, you are much less likely to be someone using stolen CCs. Seriously, most of the crooks are that lazy. So the real reason is the credit card processing companies.
Also, post 911, there is this thing called "know your customer". It was set up to fight money laundering and financing terrorists.
https://en.wikipedia.org/wiki/Know_your_customer
At best they'd just disable it for EU... assuming they didn't successfully argue "it was in the ToS ..."
Inserting a tracker that can personally identify me without my explicit opt-in consent is a GDPR breach. I'm surprised you do not not this.
The whole EU vision is they know what the little people do all the time, think all the time, and spend their money on all the time. For the children, obviously.
In the meantime, the GDPR is your friend. The amount of FUD spread about it on here by those working in Adtech (and whose very salaries are dependent on invading peoples privacy) is insane.
I genuinely wish you were right, but you are so naive it is frightening.
> The amount of FUD spread about it on here by those working in Adtech
You are confused. You are arguing with people that want to protect privacy. The EU demonstrably is not doing that, as WhatsApp gets ever more de facto mandated by the day.