Creative writing can technically be considered any writing of originalcomposition. In this sense, creative writing is a more contemporary and process-oriented name for what has been traditionally called literature, including the variety of its genres. In her work, Foundations of Creativity, Mary Lee Marksberry references Paul Witty and Lou LaBrant's Teaching the People's Language to define creative writing. Marksberry notes:
Witty and
LaBrant...[say creative writing] is a composition of any type of
writing at any time primarily in the service of such needs as
the need for keeping records of significant experience,
the need for sharing experience with an interested group, and
the need for free individual expression which contributes to mental and physical health.
In academia
Unlike academic writing classes that teach students to write based on the rules of language, creative writing focuses on students' self-expression. While creative writing as an educational subject is often available at various stages of primary and secondary school (K–12), the most refined teaching of creative writing is in universities. Following a reworking of university education in the post-war era, creative writing courses have gained increasing prominence in universities. In the UK, the first formal creative writing program was established as a Master of Arts degree at the University of East Anglia in 1970 by the novelists Malcolm Bradbury and Angus Wilson. With the beginning of formal creative writing programs:
For the
first time in the sad and enchanting history of literature, for the
first time in the glorious and dreadful history of the world, the writer
was welcome in the academic place. If the mind could be honored there,
why not the imagination?
Programs of study
Creative writing programs are typically available to
writers from the high school level all the way through graduate
school/university and adult education. These programs are traditionally
housed in English departments, but creative writing programs have
increasingly spun off into their own departments. Creative writing
undergraduate degrees tend to be Bachelor of Arts (BA) or Bachelor of Fine Arts (BFA) degrees, but Bachelor of Science (BSc) degrees also exist. Postgraduate courses include Master of Arts, Master of Fine Arts, or Master of Studies. Ph.D.
programs are also becoming more prevalent in the field, as more writers
attempt to bridge the gap between academic study and artistic pursuit.
Creative writers often place an emphasis on either fiction
or poetry, and often starting with short stories or poems. Students then
make a schedule based on this emphasis, including reading assignments,
regular writing tasks, and workshops to strengthen their skills,
knowledge and techniques. Screenwriting and playwriting courses may be
housed in film and theatre
departments as well as creative writing departments. Creative writing
students are encouraged to get involved in extracurricular writing-based
activities, such as publishing clubs, school literary magazines or
newspapers, writing contests, writing colonies or conventions, and
extended education classes.
Many educators find that using creative writing can increase students' academic performance and psychological resilience. The activity of completing small goals consistently—rather than having unfinished big goals—engenders pride, which releases dopamine
and increases motivation. Students build resilience by documenting and
analyzing their experiences, providing new perspectives on old
situations and providing a means to sort emotions. It also increases a
student's level of compassion and creates a sense of community among students in what could otherwise be deemed an isolating classroom.
International students
Creative writing may have an influence not only on native-speaking students but also on international students. Educators who advocate for creative writing say incorporating creative
writing classes or exercises has the potential to develop students into
better readers, analysts, and writers. These same people say creative writing can have similar effects on
international students by acting as a platform for them to share their
own heritage, experiences, and values. Scholar Youngjoo Yi conducted a case study that tested this idea over
two years. Yi focused on an international student from Korea and
examined how her creative writing class influenced her in-school and
out-of-school writing. He concluded that taking the creative writing
class made her a more confident writer—not only in English but also in
other languages—and the projects done in her creative writing class
encouraged her to express and connect her Korean heritage with her
English writing.
Composition studies
Argument and research writing is a major focus in the field
of composition studies. The focus on academic writing tends to leave
little room for creative writing in writing studies. Gregory Stephens suggests that focusing heavily on academic writing
prevents students from developing their own unique writing style and
voice. When he applied creative writing pedagogy techniques to STEM students
at the University of Puerto Rico-Mayaguez, he found exercises such as
"self-characterization" and storytelling assignments helped his STEM
students develop empathy, self-awareness, and a narrative voice. He
suggests these skills are transferable to real-world situations such as
professional settings. By engaging in creative writing exercises, students are able to break
free from the "constraints of formal thinking and writing" of academic
writing, potentially boosting students’ confidence, creativity, and
overall writing skills.
In academia
Creative writing is considered by some academics (mostly in the US) to be an extension of English studies,
although it is taught around the world in many languages. Some
academics see creative writing as a challenge to the tradition in
English studies of dealing with the critical study of literary forms,
not the creation of literary forms. In the United Kingdom and Australia,
as well as increasingly in the US and the rest of the world, creative
writing is considered a discipline in its own right, not an offshoot of
any other discipline.
To say that the creative has no part in education is to argue that a university is not universal.
Those who support creative writing programs—either as part
of or separate from the study of English—argue for the academic worth of
the experience. They suggest creative writing hones the students'
abilities to clearly express their thoughts and entails an in-depth
study of literary terms and mechanisms that can improve the writers'
work. The planning, development, critical analysis and creative problem-solving skills are further used in other areas beyond creative writing.
Some people suggest that creative writing cannot be taught. In an article for the New Yorker, essayist Louis Menand quotes Kay Boyle,
the director of the creative writing program at San Francisco State
University for sixteen years, who said, "all creative-writing programs
ought to be abolished by law". The pedagogy
of creative writing is also a source of debate. Critics of MFA and
English graduate programs say the methods of instruction discriminate
against people with disabilities, emphasizing writing practices such as
daily writing requirements or location-based writing that students with
chronic illnesses, physical or mental health barriers, and
neurodivergency are unable to access. The selection of texts used in traditional creative writing programs
has also been criticized, with scholars such as Caleb González saying
that the Western literary canon and writing pedagogy are "historically rooted and linked to exclusion and structural racism in creative writing programs".
In prisons
In the late 1960s, American prisons began implementing
creative writing programs due to the prisoner rights movement that
stemmed from events such as the Attica Prison riot. These creative writing programs, like other art programs, aim to
provide education, structure, and a creative outlet to encourage
rehabilitation of prisoners. These programs' continuation relies heavily
on volunteers and outside financial support from sources such as
authors and activist groups.
The Poets Playwrights Essayists Editors and Novelists, known as PEN,
were among the most significant contributors to creative writing
programs in America. In 1971, PEN established the Prison Writing
Committee to implement and advocate for creative writing programs in
prisons throughout the U.S. The PEN Writing Committee improved prison libraries,
inspired volunteer writers to teach prisoners, persuaded authors to
host workshops, and founded an annual literary competition for
prisoners. Workshops and classes help prisoners build self-esteem, make
healthy social connections, and learn new skills, which can ease prisoner reentry (reoffending).
Creative writing programs offered in juvenile correction
facilities have also proved beneficial. In Alabama, Writing Our Stories
began in 1997 as an anti-violence initiative to encourage positive
self-expression among incarcerated youths. The program found that
participants gained confidence, the ability to empathize and see their
peers in a more positive light, and the motivation to want to return to
society and live a more productive life.
One California study of prison fine arts programs found
that art education increased emotional control and decreased
disciplinary reports. Participation in creative writing and other art
programs results in significant positive outcomes for the inmates'
mental health, their relationships with their families, and the
facility's environment. The study found that improved writing skills
enhanced participants' abilities in other academic areas of study. Teaching prisoners creative writing can encourage literacy, teach
necessary life skills, and provide prisoners with an outlet to express
regret, accountability, responsibility, and a kind of restorative
justice.
Companies in a variety of sectors have used generative AI, including those in software development, healthcare, finance, entertainment, customer service, sales and marketing, art, writing, and product design.
Generative AI has been used for cybercrime, and to deceive and manipulate people through fake news and deepfakes. Generative AI models have been trained on copyrighted works without the rightholders' permission. Many generative AI systems use large-scale data centers, whose environmental impacts include electronic waste, consumption of fresh water for cooling, and high energy consumption that is estimated to be growing steadily.
The origins of algorithmically generated media can be traced to the development of the Markov chain, which has been used to model natural language since the early 20th century. Russian mathematician Andrey Markov introduced the concept in 1906, including an analysis of vowel and consonant patterns in Eugene Onegin. Once trained on a text corpus, a Markov chain can generate probabilistic text.
By the early 1970s, artists began using computers to extend generative techniques beyond Markov models. Harold Cohen developed and exhibited works produced by AARON, a pioneering computer program designed to autonomously create paintings. The terms generative AI planning or generative planning were used in the 1980s and 1990s to refer to AI planning systems, especially computer-aided process planning, used to generate sequences of actions to reach a specified goal. Generative AI planning systems used symbolic AI methods such as state space search and constraint satisfaction and were a "relatively mature" technology by the early 1990s. They were used to generate crisis action plans for military use, process plans for manufacturing and decision plans such as in prototype autonomous spacecraft.
AI generated images have become much more advanced.
In March 2020, the release of 15.ai, a free web application created by an anonymous MIT
researcher that could generate convincing character voices using
minimal training data, was one of the earliest publicly available uses
for generative AI. The platform is credited as the first mainstream service for audio deepfakes.
Other projects, including open-source approaches such as
VQGAN+CLIP and DALL·E Mini (later renamed Craiyon), made similar systems
more accessible to the public.
In November 2022, ChatGPT was released to the public. By 2023, it popularized generative AI for general-purpose text-based tasks.
Private investment in AI (pink) and generative AI (green)
In a 2024 survey by marketing research firm Ipsos, Asia–Pacific
countries were significantly more optimistic than Western societies
about generative AI and show higher adoption rates. Despite expressing
concerns about privacy and the pace of change, 68% of Asia-Pacific
respondents believed that AI was having a positive impact on the world,
compared to 57% globally. According to a survey by SAS
and Coleman Parkes Research, as of 2023, 83% of Chinese respondents
were using the technology, exceeding both the global average of 54% and
the U.S. rate of 65%. A UN report indicated that Chinese entities filed over 38,000 generative AI patents from 2014 to 2023, more than any other country. A 2024 survey by the Just So Soul
social media app reported that 18% of respondents born after 2000 used
generative AI "almost every day", and that over 60% of respondents like
or love AI-generated content (AIGC), while less than 3% dislike or hate
it.
By mid-2025, companies were increasingly abandoning
generative AI pilot projects as they had difficulties with integration,
data quality and unmet returns, leading analysts at Gartner and The Economist to characterize the period as entering the Gartner hype cycle's "trough of disillusionment" phase.
Generative artificial intelligence has been applied across
multiple industries for content creation and automation. In healthcare,
generative models are used for drug discovery and the generation of
synthetic medical data to train diagnostic systems. In finance, they are used for report drafting, data generation, and customer service automation. Media and entertainment industries use generative systems for tasks
such as music composition, script development, and image or video
generation. Researchers and policymakers have raised concerns regarding accuracy, misuse, and impacts on academic and professional work.
In 2016, DeepMind's WaveNet demonstrated that deep neural networks can generate raw audio waveforms. This enabled more realistic speech synthesis compared to earlier
approaches. Subsequent systems such as Tacotron 2 demonstrated
end-to-end neural text-to-speech generation.
Generative AI can be used to produce photorealistic
videos. Systems such as Runway have demonstrated text-to-video
generation capabilities.
Robotics
Generative models can be used for motion planning and robot control by learning from prior data.
3D modeling
Generative models can assist in automating 3D modeling tasks, including generating 3D assets from text or images.
World models
World models are neural networks designed to learn
representations of physical environments, including spatial and dynamic
properties. Recent multimodal systems have expanded these capabilities by integrating vision, language, and action into unified models.
In 2023, Google DeepMind introduced FunSearch,
a method for creating computer programs that solve mathematical and
algorithmic problems. FunSearch was used to discover new mathematical
constructions in the cap set problem and the bin packing problem.
In 2023, Google DeepMind introduced AlphaDev, which was used to discover small sorting algorithms that outperformed previously known human benchmarks and have been integrated into the LLVM standard C++ sorting library. In 2025, Google DeepMind introduced AlphaEvolve,
an AI system for general-purpose algorithm discovery and optimization.
AlphaEvolve uses LLMs to propose code changes, automated evaluators to
assess each candidate, and an evolutionary process to iteratively improve algorithms.
In 2026, in response to the increasing use of generative AI in mathematical discovery, a group of mathematicians issued the Leiden Declaration on Artificial Intelligence and Mathematics,
which recommends disclosing the use of AI in research papers, ensuring
that AI-assisted papers are peer-reviewed, and providing legal resources
and public funding so that academia and for-profit companies can
compete on equal terms.
In 2023, Google DeepMind introduced GNoME, a method to propose candidate inorganic crystal structures for computational screening and experimental synthesis in material science. Other material science methods include MatterGen, CDVAE, and CrystalFlow.
Generative engine optimization
(GEO) is the practice of structuring digital content and managing
online presence to improve visibility in responses generated by
generative AI systems. The practice influences the way large language models (LLMs) retrieve, summarize, and present information in response to user queries. Related terms include answer engine optimization (AEO) and artificial intelligence optimization (AIO).
Smaller generative AI models with up to a few billion parameters can run on smartphones, embedded devices, and personal computers. For example, LLaMA-7B (a version with 7 billion parameters) can run on a Raspberry Pi 4 and one version of Stable Diffusion can run on an iPhone 11.
Larger models with tens of billions of parameters can run on laptop or desktop computers. To achieve an acceptable speed, models of this size may require accelerators such as the GPU chips produced by NVIDIA and AMD or the Neural Engine included in Apple silicon products. For example, the 65 billion parameter version of LLaMA can be configured to run on a desktop PC.
Language models with hundreds of billions of parameters, such as GPT-4 or PaLM, typically run on datacenter computers equipped with arrays of GPUs (such as NVIDIA's H100) or AI accelerator chips (such as Google's TPU). These very large models are typically accessed as cloud services over the Internet.
Workflow for the training of a generative adversarial network
Generative adversarial networks
(GANs) are a generative modeling technique which consist of two neural
networks—the generator and the discriminator—trained simultaneously in a
competitive setting. The generator creates synthetic data
by transforming random noise into samples that resemble the training
dataset. The discriminator is trained to distinguish the authentic data
from synthetic data produced by the generator. The two models engage in a
minimax
game: the generator aims to create increasingly realistic data to
"fool" the discriminator, while the discriminator improves its ability
to distinguish real from fake data. This continuous training setup
enables the generator to produce high-quality and realistic outputs.
Variational autoencoders
Comparison
between images generated by a VAE (left) and a GAN (right). VAEs tend
to produce smoother but blurrier images due to their probabilistic
decoding.
Variational autoencoders (VAEs) are deep learning models that probabilistically encode data. They are typically used for tasks such as noise reduction from images, data compression, identifying unusual patterns, and facial recognition. Unlike standard autoencoders, which compress input data into a fixed latent representation, VAEs model the latent space
as a probability distribution, allowing for smooth sampling and
interpolation between data points. The encoder ("recognition model")
maps input data to a latent space, producing means and variances that
define a probability distribution. The decoder ("generative model")
samples from this latent distribution and attempts to reconstruct the
original input.
The full architecture of a GPT model
Transformers
Transformers became the foundation for the generative pre-trained transformer (GPT) series developed by OpenAI, replacing traditional recurrent and convolutional models. The self-attention mechanism
enables the model to determine the relative importance of each token in
a sequence when predicting the next token, thereby improving contextual
understanding. Unlike recurrent neural networks, transformers process
tokens in parallel, which improves training efficiency and scalability.
In the United States, a group of companies including OpenAI, Alphabet, and Meta signed a voluntary agreement with the Biden administration in July 2023 to watermark AI-generated content. In October 2023, Executive Order 14110 applied the Defense Production Act to require all US companies to report information to the federal government when training certain high-impact AI models.
In the European Union (EU), the Artificial Intelligence Act
includes requirements to disclose copyrighted material used to train
generative AI systems, and to label any AI-generated output as such.
In China, the Interim Measures for the Management of Generative AI Services introduced by the Cyberspace Administration of China
regulates any public-facing generative AI. It includes requirements to
watermark generated images or videos, regulations on training data and
label quality, restrictions on personal data collection, and a guideline
that generative AI services must "adhere to socialist core values".
Generative AI systems such as ChatGPT and Midjourney
are trained on large, publicly available datasets that include
copyrighted works. AI developers have argued that such training is
protected under fair use, while copyright holders have argued that it infringes their rights.
Proponents of fair use training have argued that it is a transformative use and does not involve making copies of copyrighted works available to the public. Critics have argued that image generators such as Midjourney can create nearly-identical copies of some copyrighted images, and that generative AI programs compete with the content they are trained on.
A separate question is whether AI-generated works can qualify for copyright protection. The United States Copyright Office
has ruled that works created by artificial intelligence without any
human input cannot be copyrighted, because they lack human authorship. Some legal professionals have suggested that Naruto v. Slater (2018), in which the U.S. 9th Circuit Court of Appeals held that non-humans cannot be copyright holders of artistic works, could be a potential precedent in copyright litigation over works created by generative AI. However, the office has also begun taking public input to determine if these rules need to be refined for generative AI.
In January 2025, the United States Copyright Office
(USCO) released extensive guidance regarding the use of AI tools in the
creative process, and established that "...generative AI systems also
offer tools that similarly allow users to exert control. [These] can
enable the user to control the selection and placement of individual
creative elements. Whether such modifications rise to the minimum
standard of originality required under Feist will depend on a case-by-case determination. In those cases where they do, the output should be copyrightable" Subsequently, the USCO registered the first visual artwork to be
composed of entirely AI-generated materials, titled "A Single Piece of
American Cheese".
The development of generative AI has raised concerns from governments, businesses, and individuals, resulting in protests, legal actions, calls to pause AI experiments, and actions by multiple governments. In a July 2023 briefing of the United Nations Security Council, Secretary-GeneralAntónio Guterres
stated "Generative AI has enormous potential for good and evil at
scale", that AI may "turbocharge global development" and contribute
between $10 and $15 trillion to the global economy by 2030, but that its
malicious use "could cause horrific levels of death and destruction,
widespread trauma, and deep psychological damage on an unimaginable
scale". In addition, generative AI has a significant carbon footprint.
Societal impacts
Effects on mental health
A study presented in the 2026 Conference on Human Factors in Computing Systems found that overreliance on generative AI can decrease one's ability to discern misinformation, the study tracked participants, mostly from the UK and US, for a period of 4 weeks.
Academic honesty
Generative AI can be used to generate and modify academic
prose, paraphrase sources, and translate languages. The use of
generative AI in a classroom setting has challenged traditional
definitions of academic plagiarism, leading to a "cat-and-mouse" dynamic between students using AI and institutions attempting to detect it. In the immediate wake of ChatGPT's release, many school districts and
universities issued temporary bans on the technology, though many
institutions have since moved toward policies of managed integration. However, the implementation of these policies often lacks clarity.
Research suggests that the burden of interpreting "acceptable use"
frequently falls on individual students and teachers, creating an
environment where academic honesty becomes difficult to define and
enforce.
A commonly proposed use for teachers is grading and giving
feedback. Companies like Pearson and ETS use AI to score grammar,
mechanics, usage, and style, but not for main ideas or overall
structure. The National Council of Teachers of English stated that machine scoring makes students feel their writing is not worth reading. AI scoring has also given unfair results for students from different ethnic backgrounds.
A picketer at the 2023 Writers Guild of America strike. While not a top priority, one of the WGA's 2023 requests was "regulations around the use of (generative) AI".
From the early days of the development of AI, there have been arguments put forward by ELIZA creator Joseph Weizenbaum
and others about whether tasks that can be done by computers actually
should be done by them, given the difference between computers and
humans, and between quantitative calculations and qualitative,
value-based judgements. In April 2023, it was reported that image generation AI has resulted in
70% of the jobs for video game illustrators in China being lost. In July 2023, developments in generative AI contributed to the 2023 Hollywood labor disputes. Fran Drescher, president of the Screen Actors Guild, declared that "artificial intelligence poses an existential threat to creative professions" during the 2023 SAG-AFTRA strike. Voice generation AI has been seen as a potential challenge to the voice acting sector.
However, a 2025 study concluded that the US labor market had so far not experienced a discernible disruption from generative AI. Another study reported that Danish workers who used chatbots saved 2.8%
of their time on average, and found no significant change in earnings
or hours worked.
In January 2023, Futurism broke the story that CNET
had been using an undisclosed internal AI tool to write at least 77 of
its stories; after the news broke, CNET posted corrections to 41 of the
stories. In April 2023, Die Aktuelle published an AI-generated fake interview of Michael Schumacher. In May 2024, Futurism
noted that a content management system video by AdVon Commerce, which
had used generative AI to produce articles for many of the
aforementioned outlets, appeared to show that they "had produced tens of
thousands of articles for more than 150 publishers". In 2025, a report from the American Sunlight Project stated that Pravda network was publishing as many as 10,000 articles a day, and concluded that much of this content aimed to push Russian narratives into large language models through their training data.
In June 2024, Reuters Institute published its Digital News Report for 2024.
In a survey of people in America and Europe, Reuters Institute reports
that 52% and 47% respectively are uncomfortable with news produced by
"mostly AI with some human oversight", and 23% and 15% respectively
report being comfortable. 42% of Americans and 33% of Europeans reported
that they were comfortable with news produced by "mainly human with
some help from AI". The results of global surveys reported that people
were more uncomfortable with news topics including politics (46%), crime
(43%), and local news (37%) produced by AI than other news topics. A 2025 Pew Research Survey found roughly half of all U.S. adults say
that AI will have a very (24%) or somewhat (26%) negative impact on the
news people get in the U.S. over the next 20 years.
Bias
A language model may associate certain professions with specific genders if such patterns are prevalent in the data. Similarly, image generation systems prompted with terms such as "a
photo of a CEO" have been observed to disproportionately generate images
of white male individuals when trained on biased datasets.
AI software, when using voice recognition software in
particular, struggles to recognize and understand speech impediments.
For example, people with a stutter struggle to activate voice-activated
assistants such as Gemini and Siri due to how the software was trained.
Companies that use AI systems to hire for new positions
also filter out people with accents and speech due to voice recognition
software incorrectly transcribing how candidates speak during the
interview process. Because of this, people with disabilities and
uncommon accents don't often make it to a human interviewer when these
generative AI systems are used. This is due to many AI models being trained and produced in the United States, and therefore, off of American accents.
In July 2023, the fact-checking company Logically found that the popular generative AI models Midjourney, DALL-E 2 and Stable Diffusion would produce plausible disinformation images when prompted to do so, such as images of electoral fraud in the United States and Muslim women supporting India's Bharatiya Janata Party.
Instances of users abusing software to generate
controversial statements in the vocal style of celebrities, public
officials, and other famous individuals have raised ethical concerns
over voice generation AI. In response, companies such as ElevenLabs have stated that they would
work on mitigating potential abuse through safeguards and identity verification.
Concerns and fandoms have spawned from AI-generated music.
The same software used to clone voices has been used on famous
musicians' voices to create songs that mimic their voices, gaining both
tremendous popularity and criticism. Similar techniques have also been used to create improved quality or
full-length versions of songs that have been leaked or have yet to be
released.
The New York Times defines slop as analogous to spam: "shoddy or unwanted A.I. content in social media, art, books, and ... in search results." Journalists have expressed concerns about the scale of low-quality
generated content with respect to social media content moderation, the monetary incentives from social media companies to spread such content, false political messaging, spamming of scientific research paper submissions, increased time and effort to find higher quality or desired content on the Internet, the indexing of generated content by search engines, and on journalism itself. Studies have found that AI can create inaccurate claims, citations or
summaries that sound confidently correct, a phenomenon called hallucination.
A paper published by researchers at Amazon Web Services AI
Labs found that over 57% of sentences from a sample of over 6 billion
sentences from Common Crawl, a snapshot of web pages, were machine translated.
Many of these automated translations were seen as lower quality,
especially for sentences that were translated into at least three
languages. Many lower-resource languages (ex. Wolof, Xhosa) were translated across more languages than higher-resource languages (ex. English, French).
In September 2024, Robyn Speer,
the author of wordfreq, an open source database that calculated word
frequencies based on text from the Internet, announced that she had
stopped updating the data for several reasons: high costs for obtaining
data from Reddit and Twitter, excessive focus on generative AI compared to other methods in the natural language processing community, and that "generative AI has polluted the data".
The adoption of generative AI tools led to an explosion of AI-generated content across multiple domains. A study from University College London
estimated that in 2023, more than 60,000 scholarly articles—over 1% of
all publications—were likely written with LLM assistance. According to Stanford University's
Institute for Human-Centered AI, approximately 17.5% of newly published
computer science papers and 16.9% of peer review text now incorporate
content generated by LLMs.
If AI-generated content is included in new data crawls
from the Internet for additional training of AI models, defects in the
resulting models may occur.[180]
Training an AI model exclusively on the output of another AI model
produces a lower-quality model. Repeating this process, where each new
model is trained on the previous model's output, leads to progressive
degradation and eventually results in a "model collapse" after multiple iterations.
On the other side, synthetic data can be deployed to train machine learning models while preserving user privacy. The approach is not limited to text generation; image generation has been employed to train computer vision models.
Generative AI's ability to create realistic fake content has been exploited in numerous types of cybercrime, including phishing scams. Deepfake video and audio have been used to create disinformation and fraud. In 2020, former Google click fraud czar Shuman Ghosemajumder
argued that once deepfake videos become perfectly realistic, they would
stop appearing remarkable to viewers, potentially leading to uncritical
acceptance of false information. Additionally, large language models and other forms of text-generation AI have been used to create fake reviews of e-commerce websites to boost ratings. Cybercriminals have created large language models focused on fraud, including WormGPT and FraudGPT.
A 2023 study showed that generative AI can be vulnerable to jailbreaks, reverse psychology and prompt injection attacks, enabling attackers to obtain help with harmful requests, such as for crafting social engineering and phishing attacks. Additionally, other researchers have demonstrated that open-source models can be fine-tuned to remove their safety restrictions at low cost.
RAG poisoning
In 2025, Israel signed a $6 million contract with the US-based firm Clock Tower X that aimed to influence ChatGPT, Gemini and Grok by spreading pro-Israel information onto social media and websites. This was in an attempt to take advantage of the retrieval-augmented generation (RAG) technique which is used by LLMs to provide more up-to-date information.
Privacy and data governance
Extraterritorial data access
The CLOUD Act
allows United States authorities to request data from covered service
providers, including some AI service providers, regardless of where the
data is physically stored. Courts can require parent companies to provide data held by their
subsidiaries, and such orders may be accompanied by nondisclosure
requirements preventing the provider from notifying affected users. This framework has been described in legal commentary as creating legal tension with Article 48 of the General Data Protection Regulation
(GDPR), which restricts the transfer of personal data in response to
foreign court or administrative orders unless based on an international
agreement. As a result, service providers operating in both jurisdictions may face competing legal obligations under U.S. and EU law.
According
to research institute Epoch AI, energy consumption per typical ChatGPT
query (0.3 watt-hours) is small compared to the average U.S. household
consumption per minute (almost 20 watt-hours). Queries containing long
entries can consume significantly more energy (2.5 watt-hours for a
query of around 7,500 words).
AI has a significant carbon footprint due to growing energy consumption from both training and usage. Scientists and journalists have expressed concerns about the
environmental impact that the development and deployment of generative
models are having: high CO2 emissions, large amounts of freshwater used for data centers, high amounts of electricity usage, electronic waste, and pollution due to backup diesel generator exhaust. There is also concern that these impacts may increase as these models
are incorporated into widely used search engines such as Google Search
and Bing, as chatbots and other applications become more popular, and as models need to be retrained.
The carbon footprint of generative AI globally is
estimated to be growing steadily, with potential annual emissions
ranging from 18.21 to 245.94 million tons of CO2 by 2035, with the highest estimates for 2035 nearing the impact of the United States beef industry on emissions (currently estimated to emit 257.5 million tons annually as of 2024).
Proposed mitigation strategies include factoring potential environmental costs prior to model development or data collection, increasing efficiency of data centers to reduce electricity/energy usage, building more efficient machine learning models, minimizing the number of times that models need to be retrained, developing a government-directed framework for auditing the environmental impact of these models, regulating for transparency of these models, regulating their energy and water usage, encouraging researchers to publish data on their models' carbon footprint, and increasing the number of subject matter experts who understand both machine learning and climate science.
Reliance on industry giants
Training frontier AI models requires an enormous amount of computing power. Usually only Big Tech companies have the financial resources to make such investments. Smaller start-ups such as Cohere and OpenAI end up buying access to data centers from Google and Microsoft respectively.
Tools such as GPTZero can detect content generated by AI. However, they can also make false accusations (false positives). Digital watermarking
is a technique that improves detection accuracy. It works by altering
the generated content at the source, in subtle ways which can be
detected by corresponding software.
In 2023, OpenAI developed a watermarking tool for ChatGPT.
They didn't release it, because they worried that users would switch
to competitors. They also argued that it would be easy to circumvent,
for example by asking another AI to rephrase.
In May 2025, Google deployed its watermarking tool,
SynthID. It marks output from Gemini (text), Imagen (images), and Veo
(video). To detect output from these products, one uses Google's
"SynthID detector" portal.
In June 2025, users mistakenly accused gaming companies of using generative AI for the video games Little Droid and Catly.
.png)
