Meta announced it is testing new features on Instagram aimed at protecting young people from unwanted nudity and sextortion scams. This includes a feature called Nudity Protection in DM, which automatically blurs images detected to contain nudity.
The tech giant will also issue a warning urging teens to think twice about sharing intimate images and urge teens to protect themselves. Meta says it hopes this will provide better protection against scammers who trick people into sending nude images and sending images of themselves in return.
It's also making changes that it suggests will make it more difficult for potential scammers and criminals to find and interact with teens. Meta says it has developed new technology to identify accounts “potentially” involved in sextortion fraud and is applying certain restrictions on how these suspicious accounts interact with other users. Stated.
In another step Meta announced Thursday, it will increase the data it shares with Lantern, a cross-platform online child safety program, to include more “sextortion-specific signals.”
The social networking giant has a longstanding policy against sending unwanted nudes or coercing other users to send intimate images. But that doesn't stop these problems from rampant online, causing dire situations for many teens and young adults, and sometimes with very tragic consequences.
The latest changes are detailed below.
nude screen
DM Nude Protection aims to protect teenage Instagram users from cyberflash by putting nude images behind a safety screen. Users will be able to choose whether to display it or not.
“We'll also show you a message so you don't feel pressured to respond, with options to block the sender and report the chat,” Mehta said.
The nudity safety screen is turned on by default for anyone under 18 worldwide. Older users will see a notification prompting them to turn this on.
“When nudity protection is turned on, we remind people who send images containing nudity to be careful when sending sensitive photos and to refrain from sending these photos if you change your mind. A message will appear warning you that it can be canceled.”
Anyone attempting to forward a nude image will receive the same warning asking them to reconsider.
Mehta said the feature is powered by on-device machine learning and works within an end-to-end encrypted chat because the image analysis is performed on the user's own device.
Safety tips
In another safeguard, Instagram users will be provided with safety tips when sending or receiving nudes, including information about the potential risks involved, which Mehta said should be guided by experts. It is said that it was developed in response to
“These tips include that people may screenshot or forward images to you without your knowledge, that your relationship with that person may change in the future, and that the person may have said “Contains a reminder to check your profile carefully in case you are not who you say you are.” . “We also link to a variety of resources, including Meta's Center for Safety, Support His Helpline, StopNCII.org for those 18 and older, and Take It Down for those under 18.
We're also testing pop-up messages directing people who may have interacted with accounts Meta has removed for sextortion to relevant specialized resources.
“We are also adding new child safety helplines from around the world to our in-app reporting flow, which will help teens report nudes, threats to share private images, sexual exploitation and “This means that if you report a related issue, such as solicitation, we will direct you to the local child safety helpline where available.”
Techniques to spot a sextortionist
Meta says it will remove sextortionist accounts if it comes across them, but it must first identify the bad actors before it can shut down the accounts. So Meta is trying to go even further. “We are developing technology to help identify accounts that may be involved in sextortion fraud based on a variety of signals that may indicate sextortion activity,” the company said in a statement.
“While these signals are not necessarily evidence that an account is violating our rules, we are taking precautions to prevent these accounts from finding and interacting with teen accounts. ” and added: We're already doing it to prevent other potentially suspicious accounts from finding and interacting with teens. ”
It's not clear what technology Meta uses for this or what signals indicate potential sextortionists (we've contacted them for more information). But perhaps they are trying to detect malicious actors by analyzing patterns of communication.
Accounts flagged by Meta as potential sextortionists will be restricted in how they can send messages and interact with other users.
“[A]Any message requests that a potential sextortion account attempts to send are sent directly to the recipient's hidden request folder, so the recipient is never notified of the message and does not need to see it.” There is.
Users who are already chatting with potentially fraudulent or sextortion accounts won't have their chats shut down, but will see a safety notice that says, “We encourage you to report any threats to share private images.” , reminding them that they can say no to nuisances that make them feel uncomfortable,'' Mehta said.
Teen users are already protected from receiving DMs on Instagram from adults with whom they don't have a connection (and in some cases from other teens as well). But Meta is taking additional steps against potential sextortion accounts by not displaying the “Message” button on teens' profiles, even if they're connected.
“We're also testing to hide teen users from these accounts in a user's list of followers, follows, and likes, making it harder to find teen accounts in search results,” it added.
Notably, the company has come under increasing scrutiny in Europe over child safety risks on Instagram, with enforcement authorities questioning the company's approach since the Digital Services Act (DSA) came into force last summer. It is showing.
take long, slow steps towards safety
Meta has previously announced sextortion protections, most recently expanding access to Take It Down in February.
Third-party tools allow people to generate hashes of intimate images locally on their devices and share them with the National Center for Missing and Exploited Children, where companies can use them for revenge searches and deletions. You can create a repository of non-consensus image hashes. Porn.
Meta's previous approach had been criticized for asking young people to upload nudes. In the absence of strict laws regulating how social networks must protect children, Meta has been forced to self-regulate for years, with mixed results.
However, with several requirements applying to platforms in recent years, including the UK's Children's Act, which came into force in 2021, and more recently the EU's DSA, tech giants like Meta are finally doing more to protect minors. I have to pay attention.
For example, in July 2021, Meta switched to making young people's Instagram accounts private by default, just before the UK compliance deadline. In November 2022, privacy settings for teens on Instagram and Facebook will become even stricter.
In January of this year, Meta also imposed restrictions on teens on Facebook and Instagram from sending messages to teens they haven't yet connected with, just before the DSA's full compliance deadline begins in February. However, they announced that they will be making stricter message settings the default.
Regarding protections for young users, the slow and iterative movement of Meta's features raises questions about why it took so long to apply stronger protections. This suggests that we have chosen a cynical minimum in our safeguards in order to manage usage impact and prioritize user engagement. safety. (This is exactly what Mehta whistleblower Francis Haugen has repeatedly accused his former employer of.)
When asked why the latest protections announced for Instagram users aren't also being rolled out to Facebook, a Meta spokesperson told TechCrunch: We're reminding teens about the risks of sharing sensitive images. We think it's in Instagram DMs, so that's where we focus first. ”