INTRODUCTION
Artificial Intelligence has been the greatest invention of mankind that made our lives feasible in one or the other way. Whether you want a quick fix for your language or want a big work task to be completed on time. From knowing the uses of specific medicines to learning difficult terminologies, AI proved to be the greatest go-to aid. But this aid too has a darker side. An innovation that was meant to make tasks simpler, is now making lives of women difficult. In a world where a woman anyways have to fight a battle every day in order to protect her dignity in the physical world, loses it unknowingly in the cyber world. And this is because AI has introduced a new and deeply troubling category of harm against women. One that did not exist in any meaningful form even a decade ago.
Deepfakes are emerging as a rapidly growing threat for the human society, specifically women. The number of deepfake content shared online is exploding. In 2023, users shared approximately 500,000 deepfakes across social media platforms. This number escalated to 8 Million by 2025. So by the time you read this article, someone has likely attempted another deepfake attack.
A deepfake is synthetic media, an image, audio clip, or video, generated or altered using artificial intelligence. Creators construct it in a manner that makes it appear authentic, even though the depicted event never actually occurred. This is not merely a technological problem, it is fundamentally a legal one. It is because the pre-existing civil and criminal laws in India were framed long ago. An era when neither the technology to easily create realistic, explicit sexual images or videos of a specific individual, nor the capability to achieve such a high level of realism, existed.
UNDERSTANDING THE PROBLEM
How does the technology creates deepfakes?
A category of machine learning models known as Generative Adversarial Networks (GANs) typically generates deepfakes. In this two neural networks work against each other. One generates an increasingly realistic fake content and the other attempts to detect the fakes. This competitive process produces progressively more convincing synthetic media. In recent years, the same outcome has become achievable through far simpler, publicly accessible AI image and video generation tools. This means that creating a realistic deepfake no longer requires advanced technical expertise.
The Gendered and Age-Based Patter of Misuse
The deepfake technology has legitimate application in films, education, and accessibility but its misuse follows a clear and consistent pattern. The overwhelming majority of malicious deepfakes involve non-consensual sexually explicit content, and the majority of victims are women and girls. This indicates that individuals are extensively using technology that creates hyper-realistic images or videos to sexually humiliate women and to create fabricated material involving child sexual abuse. Thus, the accessibility and accuracy of this technology have made the difficult task of forgery possible within minutes, significantly lowering the barriers to engaging in this form of digital sexual violence.
DOCUMENTED CASES
The Rashmika Mandanna Case (2023)
On 9th November, 2023, a deepfake video went viral on social media. It depicted actress Rashmika Mandanna’s face superimposed onto the body of a different woman entering an elevator. Complaints were raised by the Delhi Commission for Women, the Delhi Police’s Intelligence Fusion and Strategic Operations unit. An FIR was registered against the unidentified persons under Sections 465 and 469 of the IPC, along with Sections 66C and 66E of the IT Act, 2000. It was revealed after investigation that the video was actually created on 13th October 2023. It was done by an individual seeking to boost the follower count of a fan page. The page’s following boosted from 90,000 to 1.08 lakh within two weeks of the video going viral. This incident is widely credited as the trigger for India’s first government advisory on deepfakes.
Subsequent Celebrity Deepfakes (2023-2024)
The Rashmika Mandanna incident was followed, within weeks, by similar deepfake videos targeting other Indian celebrities including actress Katrina Kaif. These prompted the Prime Minister of India to publicly call deepfakes a serious and dangerous problem. Also, reinforcing the urgency behind the regulatory advisory that followed.
The Taylor Swift Deepfake Incident
In late January 2024, sexually explicit AI-generated deepfake images of musician Taylor Swift multiplied rapidly on the platform X. One post was viewed over 47 million times before removal. Investigation by the analytics firm Graphika traced the images to an online community. It was engaged in a deliberate challenge to bypass the safety filters of mainstream AI image generation tools. The incident directly prompted United States senators to introduce the Disrupt Explicit Forged Images and Non-Consensual Edits Act of 2024. This Act allowed victims to sue creators and distributors of non-consensual AI-generated deepfakes.
South Korea Telegram Deepfake Crisis (2024)
In August 2024, authorities uncovered a deepfake crisis in South Korea involving university students and teenagers. Perpetrators created the sexually explicit images by superimposing victims’ faces onto explicit content and shared them in Telegram chat rooms with thousands of members. Police recorded 297 deepfake cases in the first seven months of 2024 alone, compared to 180 for the whole of 2023, with 74% of suspects aged between ten and nineteen years. By early 2025, police had arrested 682 suspects. There was a particularly disturbing instance. The police charged four middle school students, around 12-15 years of age. They were creating, possessing and distributing deepfake pornography of their female classmates, identifying at least nine victims. This case demonstrates that both perpetrators and victims of AI-enabled sexual crimes are increasingly minors.
United States’ Steven Anderegg – The First AI-CSAM Prosecution (2024)
In May 2024, a federal grand jury indicted Steven Anderegg, 42, for producing, distributing and possessing AI-generated images of minors. Those images showed minors being engaged in sexually explicit conduct, and for transferring obscene material to a minor under sixteen. Anderegg allegedly used Stable Diffusion, a publicly available text-to-image generative AI model, to create hundreds of hyper-realistic images of juveniles using highly specific prompts engineered to produce this material.
Observers widely regard this case as the first federal prosecution in the United States to charge the creation of child sexual abuse material produced entirely through AI, without any underlying real photograph of abuse. Federal prosecutors firmly rejected Anderegg’s defence that no real child was depicted.
United States: A Teacher’s Abuse of Real Students Using AI (2025)
In a deeply disturbing 2025 case, prosecutors indicted a teacher named Haslach after he used AI tools to digitally undress photographs he had taken of his own elementary school students, depicting them in sexual acts and erotic poses. Federal agents identified more than ninety victims and recovered close to eight hundred AI-generated abusive images from his devices.
United States: Cody Prater AI-CSAM and the First Amendment (2026)
In a verdict reported in early 2026, a federal jury convicted Cody Prater, 28, for receiving and possessing both real child sexual abuse material and AI-generated obscene depictions of child sexual abuse. The court rejected Prater’s First Amendment challenge before trial, ruling that AI-generated obscene material depicting child sexual abuse is not constitutionally protected speech. This case establishes that the legal system draws no meaningful distinction between AI-generated and authentic child sexual abuse material for purposes of criminal liability.
Taken together, these cases reveal a consistent pattern. AI-enabled sexual crimes overwhelmingly target women and girls, are increasingly perpetrated by and against minors. Such crimes exploit both purpose-built and mainstream commercial AI tools, and spread through platforms that are slow or unwilling to intervene. It is against this real and rapidly worsening backdrop, affecting women and children alike, that the adequacy of India’s legal response must be assessed.
THE INDIAN LEGAL FRAMEWORK BEFORE 2023
Information Technology Act, 2000
Section 67 of the Act punishes the publishing or transmission of obscene material in electronic form. It includes imprisonment of up to three years and a fine of up to five lakh rupees for a first conviction. Section 67A specifically punishes the publishing or transmission of material containing sexually explicit acts, with imprisonment of up to five years and a fine of up to ten lakh rupees. The Section 66E punishes the violation of privacy through the intentional capturing, publishing or transmission of the image of a person’s private area without consent. Further, Section 67B specifically punishes the publishing or transmitting of material depicting children in sexually explicit acts in electronic form.
Though lawmakers never drafted these provisions with deepfakes in mind, police have actually used them as the primary statutory basis for registering deepfake-related FIRs in India, including in the Rashmika Mandanna case. Since Indian law had no provision specifically naming or defining synthetic or AI-generated content prior to 2023.
Indian Penal Code, 1860 (and the BNS, 2023)
Alongside the IT Act, police have historically invoked several IPC provisions in cases involving manipulated or non-consensual images. It includes Section 354C (voyeurism), Section 354D (stalking, including through electronic communication), and Section 509 (insulting the modesty of a woman). Following the enactment of the Bharatiya Nyaya Sanhita, 2023, equivalent provisions continue to operate under the new code.
POCSO Act, 2012, as Amended in 2019
Section 15 of the POCSO Act makes it illegal to store or possess child sexual abuse material for the purpose of transmitting, propagating, displaying or distributing it. The law exempts storage solely for reporting to authorities or use as evidence, and imposes enhanced punishment of three to five years’ imprisonment where a person holds such material for commercial purposes. The 2019 amendment broadened the definition of child pornography to expressly cover any visual depiction of sexually explicit conduct involving a child, a definition broad enough in principle to extend to AI-generated and synthetic depictions, even where no actual photograph of abuse exists.
The Gap That Remained
Despite this patchwork of provisions, none of them used the words “deepfake,” “synthetic media,” or “AI-generated content” anywhere in their text prior to 2023. Prosecutors, therefore, had to build deepfake cases against both women and children entirely on provisions that lawmakers never designed for this specific harm.
RECENT DEVELOPMENTS (2023–2026)
The MeitY Advisory of November 2023
Following the Rashmika Mandanna incident, the Central Government, on 8th November 2023, issued an advisory to significant social media intermediaries. The government instructed intermediaries to exercise due diligence, identify misinformation and deepfakes, and remove reported content within thirty-six hours. This was India’s first official regulatory acknowledgment of deepfakes as a distinct category of online harm.
The IT Amendment Rules, 2026
On 10th February 2026, the Ministry of Electronics and Information Technology notified the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Amendment Rules, 2026, effective from 20th February 2026. It marked to be India’s first comprehensive statutory framework specifically addressing AI-generated synthetic content. This amendment introduces several significant changes.
Indian law now formally defines “synthetically generated information” as audio, visual or audio-visual content artificially created or altered to appear real. The same excludes routine edits, accessibility features, and academic or training material. Platforms must remove unlawful content, including AI-generated material, within three hours of receiving a takedown notice, reduced from the earlier thirty-six-hour window. Synthetic content must carry mandatory labelling through visible disclosures, audio prefixes, and embedded metadata or provenance markers, and platforms cannot remove such labels.
Intermediaries must now deploy reasonable technical measures. It must include automated detection, to stop unlawful synthetic content from generating in the first place. Grievance officers must acknowledge complaints within twenty-four hours and resolve them within seven days. Where a platform fails to comply, it loses safe harbour protection under the IT Act and the law can hold it liable as though it had created the unlawful content itself.
Significance of the 2026 Amendment
This amendment represents the most significant regulatory development in this area since the 2008 amendment, which introduced Sections 66E, 67A and 67B. For the first time, Indian law moves beyond merely punishing publication after the fact, towards actively requiring platforms to detect and prevent the generation of such material. It has a specific and explicit focus on child sexual abuse material and non-consensual intimate imagery.
GAPS IN THE LEGAL FRAMEWORK
Absence of a Standalone Deepfake or NCII Law
India still does not have a dedicated, standalone statute. A statute which specifically criminalizes the creation and distribution of non-consensual intimate imagery, whether real or AI-generated. The 2026 Amendment Rules regulate platform conduct under the IT Act rather than directly defining and punishing the act of creating a deepfake. This stands in contrast to jurisdictions such as the United Kingdom, where the Online Safety Act, 2023 directly criminalizes the sharing of deepfake pornography as a specific offence.
The Three-Hour Takedown Window: Feasibility Concerns
Legal commentators have noted that the compressed three-hour takedown deadline may prove operationally unfeasible for platforms without dedicated round-the-clock moderation teams. The same creates a risk of both erroneous takedowns and a chilling effect on legitimate speech under Article 19(1)(a) of the Constitution, as platforms may resort to pre-emptive over-removal of ambiguous content.
Jurisdictional Challenges and Victim Identification
Many platforms and AI tools used to generate deepfakes operate from servers outside India, making enforcement against original creators exceptionally difficult. A further, distinctly child-specific challenge has emerged internationally. The law enforcement agencies investigating AI-generated child sexual abuse material increasingly struggle to determine whether the children depicted are real and in active danger. This distinction is critical, since it determines which cases require urgent rescue operations and which do not. And reports indicate that the massive volume of AI-generated content is placing an immense burden on specialized units dedicated to child safety, even in areas where resources are plentiful.
Continuing Relevance of Shreya Singhal
Any expansion of platform takedown obligations must operate within the constitutional framework established in Shreya Singhal v. Union of India. The Supreme Court struck down Section 66A of the IT Act for vagueness and disproportionate restriction of free speech. Also, while upholding intermediary takedown obligations under judicial or governmental order. The case remains the guiding constitutional precedent against which courts will likely test any new takedown regime, including the 2026 three-hour rule, if challenged.
CONCLUSION
The shown trajectory of Indian law’s shows response to a harm that did not meaningfully exist a decade ago. Lawmakers originally drafted separate provisions under the IT Act 2000, the IPC, and the POCSO Act without synthetic media in mind. Together, these evolved into the ‘IT Amendment Rules, 2026.’ It marked India’s first comprehensive legal framework to directly address issues related to AI-generated content.
The documented cases demonstrate that this is not a speculative future risk. It is an active, gendered, and rapidly worsening pattern of harm. It is affecting both women and children, often through the very same mainstream commercial AI tools available to the public.
The 2026 Amendment Rules represent genuine progress. It introduced a statutory definition of synthetic content. A compressed three-hour takedown window was introduced. Mandatory labelling, and proactive detection obligations specifically targeting CSAM and non-consensual intimate imagery. Yet significant gaps remain. India still lacks a standalone law directly criminalizing the creation of deepfakes. The individual creator liability remains legally ambiguous. And the cross-border enforcement against foreign AI tools remains largely beyond the reach of domestic regulation.
But the legal framework is still in active motion, reacting to harm as it escalates rather than anticipating it. The AI technology continues to advance at a pace that consistently outstrips legislative response. In such situation, just regulating this harm is not the central challenge for Indian law. But whether it can do so quickly enough to keep pace with the technology itself, for women and children alike.
REFERENCES
- Information Technology Act, 2000.
- Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021.
- Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Amendment Rules, 2026.
- Protection of Children from Sexual Offences (POCSO) Act, 2012.
- Indian Penal Code, 1860.
- Constitution of India, 1950.
-
Shreya Singhal vs U.O.I AIR 2015 SUPREME COURT 1523.
