Read The AI Revolution in Book Publishing (2024)

Table of Contents

• Introduction
  • Is it a book?
  • Why AI now?
  • Will it be just what you need to know?
  • Who is this book for?
  • How much do you need to know about AI to read this book?
  • Book publishers and AI
  • Some housekeeping
• AI: A Very Brief History
• Understanding Key Terms
• AI Software: The System Heavyweights
  • Software Paradigms
  • Comparing the top AI software
• Software for Book Publishers
  • AI Software for Book Publishers: The Startups
• Working with AI Software
• Training for ChatGPT
  • How to Prompt
  • How to GPT
• Hallucinations: A Fly in the Ointment
  • How to Safeguard Against Hallucinations
• What About Images and Video?
• AI and Book Publishing: What Is the Industry Doing?
• AI and Book Publishing: The Uses Cases
  • AI and Metadata
  • Business Systems for Book Publishing
• Strategies For Integrating AI into Publishing Operations
  • Job considerations
  • Trying and Testing
• AI for Audiobooks
• AI for Book Translation
  • Defining “Good Enough” in Book Translation
  • AI for Scholarly Publishing
• AI for Authors
  • AI Software for Authoring
• Concerns and Risks Surrounding AI
  • Copyright infringed?
  • Copyright and AI for authors
  • What are the long-term implications?
  • It’s too late to avoid AI
  • When authors use AI
  • Can AI be detected in writing?
  • Job Loss
  • Education
  • The Future of Search
  • Junk Books on Amazon
  • Bias
• Getting a Handle on the AI Transformation
• Creativity is Publishing’s Shibboleth
• An Afterthought
• Good Stuff Outside of Publishing
• The Real-World Consequences of AI on the Book Publishing Industry
  • Publishing has been in economic decline for decades
  • The three remaining advantages for traditional book publishers
  • What about bookstores?
  • Self-publishing
  • Hybrid publishers
  • Publishing beyond publishers
  • Innovation, technology and book publishing
  • The innovator’s dilemma
  • Can traditional publishers hold onto their remaining advantages?
  • The containerization of content
  • Discovery and Conversion
  • AI can Communicate
  • Futures
• Is there an existential threat to authors?
• Selected Resources
  • Disclosures
  • Acknowledgements
• Sponsors

It’s been a year and a half since ChatGPT was introduced to a public mostly unfamiliar with artificial intelligence. Initially it appeared to have no obvious relevance to book publishing. Since then everything has changed. And nothing has changed. Publishers are beginning to dive into the new AI tools, exploring the edges, engaging in tentative chats with ChatGPT. But there’s no evidence of a revolution in the practice of publishing. It’s just too soon.

What there is, in abundance, is uncertainty around AI in writing and publishing. I’ll explore some of the reasons in this book. Major controversies surround AI’s use in writing and publishing. Still, for many, there’s interest and excitement about the possibilities. I’m going to focus here more on promise than on peril, but I’ll include an overview and some links for those who want to get a handle on some of the potential downsides. I’m not seeking to belittle the concerns. They’re just not what this book is about.

Is it a book?

I’m thinking of this publication more as a “booklet” than a book. The UNESCO definition of a book is “at least 49 pages” (why not 50?), and at 300 words per page, this one could fall short. More importantly, it’s not intended to be the definitive word on anything —- it’s more of a progress report. Artificial intelligence, and its role in book publishing is changing at a breakneck pace, and shows no signs of slowing down. So this book(let), this report, is just a snapshot.

Using the Leanpub platform, I’m releasing my research and analysis as a work in progress —- it will be revised as a living document. It’s something I’ve never attempted before. The reasons are two-fold —- to offer something useful in the short-term, and to have the facility to revise it as required. The AI story is far from over.

I hope my approach will prove useful for readers, while I’m aware that too many updates could soon become annoying.

Why AI now?

The tech industry offers a useful blueprint in how it often talks about the whys of a new software tool. There should be a “pain point” that the software is trying to reduce or alleviate, and a “use case” for the new product.

I’m hearing daily of the pain that people across the publishing industry are facing trying to come to terms with AI. Why is it so painful? There are three reasons that I’m seeing:

All right: It makes sense that publishing people feel AI pain. So then what’s the use case for this book?

The use case is that you work in book publishing, or you’re an author trying to better understand book publishing, and you’re sick of feeling stupid and confused around AI. Too many of your colleagues appear to understand it better than you do, and you’re tired of not being able to engage intelligently on the topic.

My goal for readers is that, by the end of this book, they’ll feel enabled to join the conversation, to express an informed opinion, even while recognizing that there’s lots more that they can learn, and that things are changing so quickly that they’re going to have to keep paying attention till things slow down. I want them to feel equipped to make personal choices around the technology, and have a path for learning how to use AI, if they choose to do so.

I do not want to inundate readers with lots of blah-blah about the underpinnings of AI technology —- I’m not going to talk much about AI in the abstract. I want instead to give you a grounding in AI specifically for book authoring and publishing. If someone asks you, but what does natural language processing really mean, you can answer, as I do: I’m not an AI expert. But I do know how AI is being used in my industry.

Will it be just what you need to know?

I dislike the phrase “what you need to know” because it so rarely lives up to its billing, often missing wide of the mark, either far more than you need to know, or far less. How can I get it right? First, because I’m not using the phrase as click-bait —- I’m serious about trying to convey the minimum. Second, because I think I know what publishers need to know about AI, as I’ve been talking to lots of them, and participating in seminars and webinars designed very specifically to provide the minimum. And third, because this book is a “living document” and I hope readers will tell me if there’s too little or too much, and I’ll revise accordingly.

Rather than endlessly repeating the phrase “what you need to know” I’ll use the logo above, which ChatGPT kindly designed for me. (Interim note: my nifty logo doesn’t display well with the Leanpub engine. I’ll figure out how to fix that in subsequent editions.)

After the very brief “what you need to know” summaries in each section is longer text expanding on those short descriptions. I’ve kept these as short as possible, though, truth be told, sometimes I do go on.

Who is this book for?

The primary audience for this book is trade (consumer) book publishers. The secondary audience is all other book publishers (scholarly, educational and…) to the extent that there are clear overlaps in the industry sectors. My third audience is authors and the many others who are fascinated by book publishing.

The title states that it’s a guide to AI in both writing and publishing —- not just publishing, but also writing. Yes, that’s because I want publishers to understand what writers are doing with this technology. I think that’s important. At its recent annual meeting, the BISG, the Book Industry Study Group, which mostly represents trade publishers, indicated that “it is looking to draw from a broader pool including, literary agents and authors.” That’s a welcome move.

I want writers to read this book, because it can help them to understand what publishers are doing with their manuscripts. Self-published authors always keep one eye on traditional publishing, in part because they are, de facto, publishers themselves, and so the AI technology of interest to publishers can be of interest to them as well.

But, make no mistake, this is not a hands-on how-to for authors about how to use AI to be better and more productive writers. There are numerous books on that topic (sadly, many of them just scammy Amazon rip-offs). And lots of YouTube videos (of varying quality).

Within trade book publishing my ideal reader is not necessarily a decision-maker, but rather one of the many people who work across the industry every day, creating books and finding readers. I’ll be talking about all of the key publishing functions in this book, from editorial through to sales and distribution. And about writers and readers.

I’m based San Francisco, in the U.S., and so I default to thinking about the U.S. publishing industry. But just about everything in this booklet should be as relevant to a publisher in Madagascar as it is to a publisher in Manhattan. While publishers in smaller countries face challenges of scale, so do small publishers in America, and this book is intended for companies of all sizes.

When I talk to small publishers about AI, and when I talk to very large publishers about AI, the big difference that I notice is the notion of the publishing “enterprise.” The largest publishers are often part of (or adjacent to) public companies, and once you’re a public company, issues around reputation and security loom so much larger than they do for an everyday publisher, not accountable in the same way to shareholders and the news media. While an editor at a Big 5 publisher has the same use for AI tools as a freelance editor working for smaller outfits, the context in which they use the tools is quite different.

How much do you need to know about AI to read this book?

I’m assuming that my audience has roughly zero knowledge about AI’s innards, but that they’re smart readers. So I’ll be treading that fine line between zero knowledge, and not talking down to the reader.

I do imagine that just about everyone has heard about the new generation of AI: the latest YouGov poll (pdf), in March 2024, shows that only 7% of Americans know “nothing at all” about AI. Writers and publishers have almost certainly heard about some of the copyright complaints, but don’t have a grasp of the issues. So I don’t have to explain that AI stands for “Artificial Intelligence.” But I won’t assume readers can define what AI is, what it means, its potential impact, and so on. That’s why you’re here.

Book publishers and AI

We still don’t know all that much about what publishers are actually doing with AI tools —- there’s very little available information. There are several reasons for this. The truth is that publishers are not in fact doing very much. Because of author concerns, publishers don’t want to reveal their AI-related activities. And, as a smart observer mentioned to me the other day, publishers are traditionally very secretive about their publishing techniques, fearing that competitors might take advantage of their insights. I said to him, “but everyone is doing the same things with AI. There are no secrets!” Perhaps, perhaps not.

Some housekeeping

As indicated, this book is short. When you call a short-ish document “a book” you run the risk of buyers protesting, “I thought I was buying a book, but it’s only 50 pages long!” I’ve done everything I can to make it clear on the sales page that this is not a full-length book, but if you’re disappointed, keep in mind that Leanpub offers a 60-day money-back guarantee, and I’ll happily extend that refund offer to “forever.” I want readers to be delighted, not disappointed.

The book is very heavily hyperlinked. I’ve always been a believer in linking to original sources in my work, so that readers can verify my sometimes grand claims. But I’m aware also that hyperlinks and footnotes can be distracting. I apologize for that. The best bet is to ignore the links as you read, and return to them if necessary.

This book has sponsors, something that I know is unusual for “a book.” I explain how the sponsorship works in my “disclosures” section at the end of the text.

Until a couple of years ago, the average person heard about AI as an abstraction, either as science fiction or as something that could play chess well. Suddenly AI is everywhere, which creates a false impression that it’s new. Even the new stuff isn’t exactly new. But that’s a moot point. AI meant very little to book publishing before ChatGPT. Now it means a lot.

Understanding the roughly 70-year development of AI can be fascinating, but it’s in no way required to appreciate what’s going on today.

The current generation of AI was developed mostly over the last decade. Then, suddenly, ChatGPT appeared on November 30, 2022. Two months later it had 100 million monthly users, the fastest that any technology has ever moved into the consumer space (by comparison, Facebook took over two years to reach 100 million users).

Why the rapid adoption? First, it’s free. Second, you don’t need to buy a new device to use it. And third, you don’t need any training to access ChatGPT. Those same factors applied also to Facebook, so why ChatGPT?

As Arthur C. Clarke famously noted, “Any sufficiently advanced technology is indistinguishable from magic.” ChatGPT is magic. The experience of “talking” in everyday language to a machine… it’s magical. The experience of saying “I want an image of a book in a balloon in a cloud near the sun,” and, seconds later,

… also magical. GPT-generated images are starting to look similar in style, colorful and fanciful. So I sent a second prompt “now in a style that looks like a 15th century illustration.” And so:

If I want a video of a book in a balloon in a cloud near the sun, there are over a dozen tools to choose from, and presto. And a musical soundtrack to go with the video. Well, how does this sound? It’s just like magic.

AI arrives with a plethora of terminology, much of it inscrutable to all but data scientists. And much of it is used carelessly, or interchangeably, without precise intention.

Users of ChatGPT don’t require an in-depth knowledge of the AI terminology nor technical concepts involved. The system’s conversational nature allows intuitive interactions without needing specialized background knowledge. Focusing on what ChatGPT can do is more important.

Perhaps the only term you need to be aware of is “chatbot.” That’s the category that ChatGPT falls into —- software designed to simulate human conversation. ChatGPT is a chatbot, but it’s not the only one: it’s important to recognize that ChatGPT is only the most famous chatbot, but far from the only one, and far from the only important one. I’ll look at that below.

That being said, there are a few other terms that it’s useful to be aware of, if only because when you hear them you’ll recognize that they’re part of a conversation about AI. Don’t concern yourself about their precise meaning.

Artificial Intelligence (AI)

AI is an overarching label for what’s going on now with chatbots, encompassing a broad field of which chat is but a part.

Machine Learning (ML)

Machine learning is a subset of AI. It enables software to predict results without being explicitly programmed to do so.

Natural Language Processing (NLP)

NLP enables computers to respond to (natural) human language in natural ways. ChatGPT’s ability to understand queries and provide coherent replies is rooted in advanced NLP techniques.

The next three terms related very specifically to the software we’ll be talking about in this book, so I’ll go into a bit more depth here. In order to make decisions about the value of the software for your use, it’s helpful to understand just a bit about what lies underneath.

Large Language Model (LLM)

Large Language Models work by analyzing huge amounts of (mostly) written material, allowing them to predict what words or sentences should come next in a conversation or a piece of writing. They don’t “understand” language in the human sense, instead processing text by breaking it down into smaller pieces (called tokens), and then converting the tokens into numbers. They process the text as numbers, regurgitating more numbers, which are then converted back into text on output. That’s an overly simplified explanation of why ChatGPT does not “contain” copyrighted work: it’s built with numbers that represent a vast abstraction from the underlying texts.

LLMs are trained on how language is typically used and then generate responses based on this understanding. We tend to underestimate just how predictable most language is. ChatGPT can generate text that is (sometimes shockingly) similar to existing literature, but, by design, it doesn’t have the capability to retrieve specific excerpts or copies of copyrighted texts. (I know, many of you have heard about the New York Times lawsuit against OpenAI —- the Times was able to get ChatGPT to regurgitate some portions of NYT articles verbatim. More on this below.)

Generative AI

The most important thing to understand this term is the “generative” part. Generative AI generates new text.

Generative Pre-trained Transformer (GPT)

This, the nerdiest of the terminology here, describes a specific type of LLM developed by OpenAI. “Generative” indicates its ability to create text, “pre-trained” signifies that it has been trained on a large body of text data, and “transformer” references to software that it uses. Knowing what GPT stands for is helpful only so that you understand what the GPT in ChatGPT represents.

ChatGPT

ChatGPT is the software you see; GPT is what’s behind the software. Users experience ChatGPT, not the GPT behind it.

One more term that you’ll encounter frequently that is unfamiliar to many is

Corpus

The dictionary definition of corpus is “a collection of written texts” (though, in fact, it’s not always text). The term is used in reference to what GPTs are trained on: vast corpuses of (mostly) text. We’re told that most largest corpuses contain hundreds of billions of words. For mere mortals that’s impossible to comprehend. Don’t you think of Wikipedia as enormous, containing a vast number of words? Well, there are a mere 4.5 billion words in Wikipedia —- GPT-4 was trained on well over a trillion.

I think that it’s important to consider this scale. Authors, understandably, are worried that the 75,000 words, plus or minus, in their book might have been sucked into a large language model. Perhaps they have (more below). But assuming this is the case, consider just how little value any one book has to the total power of today’s large language models. It’s truly insignificant. Beyond insignificant. Even 10,000 books is chump change.

“Which software tools do I use?” was the number one question that emerged from the September Publisher Weekly AI conference, understandably so. Everyone has heard about ChatGPT —- is that all there is? Do you just sign up for ChatGPT, and that’s it?

If only it were so simple. ChatGPT has competitors. The leaders and the losers shift, but as of late April 2024, the competition includes software from well-established heavyweights, like Google, Meta (Facebook) and Microsoft, and newcomers, like Anthropic.

There are two levels of AI software. The first level is what could be called the foundational software, software like ChatGPT and its half-dozen competitors. That’s how most people experience AI today.

The next level is all of the other software that provides what is essentially a window into the foundational software. Newcomers to AI chat will mostly not have encountered this software, though it is not rare or obscure. Some are familiar with Jasper (marketing software) or Perplexity (search). This software would simply not exist without ChatGPT and its brethren.

Keep in mind that mainstream software from Adobe, Microsoft, Grammarly, Zoom, etc., now incorporate AI tools, but this is different from software that’s built from the ground up based on chatbot interfaces. (Others like Otter (transcriptions) predate ChatGPT but are now imbued with large language model technology.)

Below we’ll look at publishing-specific software that capitalizes on generative AI’s abilities.

Software Paradigms

Way back when software was code and only engineers understood how it worked. What made personal computers succeed, beyond their relatively low cost, was software that was (relatively) easy to use. We don’t give much thought to all of this; it just surrounds us.

But consider, for a moment, Microsoft Word, a program that I can safely assume is used by 99% of the readers of this book. It takes about 10 minutes to show somehow how to write a paragraph of text in Word. Then you can show them how to create bold text, and italics, and how to change the base font, perhaps from Calibri to Arial. Then, how to save the file and print the file. How to search and replace.

And then you leave them on their own.

Years later you find out that they’re still formatting their Word paragraphs with spaces and tabs. And still forgetting to save their files, or getting mixed up about which version they saved, sometimes having to recreate text from scratch. But they “know how to use Word.”

Word has hundreds of features, but a third of the time users employ only 5 commands: Paste, Save, Copy, Undo and Bold. It takes several weeks of training to properly learn Word’s advanced features; features that could help a publisher save many hundreds of hours of time. But they never do —- we never do. I’ve been using Word since 1985, nearly forty years, and I’ve never taken a course. I don’t even know what I don’t know about how to extract the full power from the software.

I go on about this because it’s a helpful way to get your head around why AI software is so very different than any software you’ve ever worked with. It doesn’t come with a manual, not even a Quick Start guide. It comes with a blank screen and a question: “What can I help you with.”

There’s no menu bar, no button to push, no keyboard shortcuts. You ask a question, start a chat, and you get some answers.

Claude:

“There are several key differences between working with modern conversational AI like myself compared to traditional tools like Microsoft Word or Excel:

“1. Natural language interaction: With conversational AI, you can communicate your intent using everyday language rather than having to learn and navigate complex menus and functions. You simply explain what you want to accomplish conversationally.

“2. Contextual understanding: AI assistants maintain context and understand the flow of the conversation, allowing for more natural back-and-forth dialog. Traditional software has no contextual awareness from one command to the next.

“3. Adaptability: AI can dynamically adapt and respond to a wide variety of requests, while traditional tools are limited to a predefined set of functions. An AI assistant can help with writing, analysis, math, coding, ideation, and more within a single interface.”

That was useful, but I wanted something easier to grasp. So I asked:

Claude:

“Here’s a simple way to explain the difference to someone without a technical background:

“Think of traditional software like Microsoft Word or Excel as a tool, like a hammer or a calculator. It has a fixed set of functions that it can perform, and you need to learn how to use it properly to get the result you want.

“On the other hand, think of modern conversational AI as a smart assistant, like a knowledgeable colleague or friend. You can just talk to it using plain English, and it understands what you’re trying to do. It can help you with all sorts of tasks, from writing to math to coming up with ideas.

“You don’t need to learn any special commands or techniques —- just chat with it naturally. It adapts to your needs and remembers the context of your conversation. Over time, it gets to know you and your preferences.

“So while traditional software is like a static tool that you use, conversational AI is more like an intelligent partner that you interact with naturally to get things done. It’s a more flexible, personalized and efficient way of working.”

Hmm. Not quite what I wanted. So I continued the conversation:

Claude:

“Sure, let me try again without using technical terms:

“Imagine you’re talking to a friend who is really knowledgeable and eager to help you out. You can just talk to them in plain English about what you’re trying to do, whether it’s writing an essay, solving a math problem, or brainstorming ideas for a project.

“Instead of having to learn and remember a bunch of specific steps or commands, you can just explain what you need in your own words. Your friend understands the context and adapts their help based on what you’ve already discussed.

“It’s a back-and-forth conversation, where you can ask questions, clarify things, and get advice. And the more you talk to this friend, the better they understand your unique needs and communication style.

“Now, imagine if you could have that same type of helpful, easy interaction, but with a computer program. That’s the basic idea behind new types of software you might be hearing about.

“Instead of the old style of software, where you have a rigid set of functions and have to learn the right way to use it, this new software is more like a knowledgeable friend. You can just talk to it conversationally and it figures out how to help you based on the natural flow of your discussion.

“So it’s a smarter, more personalized, and easier way to get help with all sorts of tasks on your computer, just by talking to it like you would a person. That’s the key difference from the older, more rigid style of software.”

Summary: I was trying to do two things there. One was to try to offer a dead simple example of why chatbot software is so very different than any software we’ve ever used. And the other was to illustrate how a chat can evolve. It’s not just a single question and a single answer. It’s a conversation. That’s a big part of the breakthrough.

Comparing the top AI software

OpenAI

ChatGPT, from OpenAI, is, unquestionably, number one. It’s built on OpenAI’s underlying GPT-3.5, and the more recent GPT-4 (“10 times more advanced” than GPT-3.5). GPT-5 is in the works, with no release date set.

When people talk about using AI, they usually mean that they’re playing with ChatGPT. If they’re not paying for it, they’re using version 3.5. If they’re paying ($20/month), they’ve got GPT-4.

Microsoft

Microsoft occupies an ambiguous space in this hemisphere. It has the exclusive third-party license to OpenAI’s underlying technology. But of course it has lots of its own technology to layer onto OpenAI, and so Microsoft’s offering is a bit of a hodge-podge, and vastly confusing. On top of this, the Microsoft Office Suite (now called Microsoft 365) has included a variety of AI-enabled tools for a while now.

None of it is called ChatGPT per se, though much of it is built on OpenAI’s product. Consumers mostly encounter Microsoft Copilot (which until a month ago was called Bing Chat). Copilot can be accessed directly in Windows computers, or through browsers, or, via the Microsoft 365 suite of software, most notably Word, Excel, Outlook, and PowerPoint. Copilot is the chatty interface, although AI tools are starting to emerge from deep within the everyday Microsoft toolset. Have you tried Word’s transcribe feature? It’s powerful and accurate. How about the “Rehearse with Coach” tool in PowerPoint? Unprecedented. AI is seeping into much of our day-to-day software. (Adobe has, thus far, focused on AI for images.)

Anthropic

OpenAI competitor Anthropic calls its chat interface “Claude.” Claude has been appealing to publishers because it can work with large documents, including, of course, manuscripts. It’s most recent version can ingest files up to about 150,000 words. Editors can ask Claude to analyze files for structure and content, to generate summaries, to build Q&As. It’s powerful stuff.

Google

Google now offers a ChatGPT competitor called Gemini (formerly known as Bard). Of course it has variants, in this case Gemini Ultra, Gemini 1.5 Pro and Gemini Nano. Their $19.99/month subscription model is called the Google One AI Premium plan. Is that sufficiently confusing? The bottom line: if you want to test Google’s AI at full power against Anthropic (Claude) and OpenAI (ChatGPT), the $19.99 package is the way to go.

Is it any better than the others? Opinions change daily.

Meta

More…

Everyone Else

There are a ton of other companies offering large language models and generative software. All you need to know is that these won’t matter to you in the short term: I’ll let you know if that changes.

There is very little AI software available specifically for book publishers (though there’s lots available for authors).

My Publishers Weekly Startup Database now lists over 175 AI-focused publishing startups. The bulk, by far, are directed to authors, rather than publishers. Some look to generate whole books, others seek to guide authors on their journey. Sudowrite remains the most prominent in this class of vendors.

For publishers the choices are beginning to coalesce around editing and marketing tools.

Shimmr, a sponsor of this book (see below), is a high-profile player in the latter category. More…

Veristage, a sponsor of this book (see below), is building Insight, its “AI Publishing Assistant,” a task-specific front end to multiple publishing functions. More…

More software…

AI Software for Book Publishers: The Startups

As you’ll see in the report, I look pretty broadly at startups across the book publishing spectrum. I don’t include new publishing companies, unless they are doing some unusual things with technology. The basic criteria is: do you use technology to try to invigorate some aspect of the book(ish) publishing process? I’ve got nearly 1,600 companies in the database, mostly launched after Amazon released the first Kindle (2007).

After you get an initial sense of the database, you can start to dive in more deeply. On the far right tab you’ll see a way to sort only by the AI-related publishing startups:

As you see here, there are 175 of them (as of 4/17/24). The majority, about 130, were launched after ChatGPT first appeared in November of 2022. That volume of new businesses is unprecedented within book publishing. It’s astounding.

As is characteristic of the full database, these AI startups mostly target authors (70%). Some 15% are looking to serve publishing companies, per se. The balance want to help readers discover books, or some combination of authors and readers to create and enjoy books.

Also, as is characteristic of the larger startup cohort, many of these startups are between tiny and miniscule, just one person with a website and a half-baked idea. I include them all —- who knows where they’re headed.

I’ll highlight a few of the startups here, ones that fascinate me. I strongly encourage you to play with the database. Click a few links. I think you’ll be tickled by the innovation and audacity of many of these organizations.

I’m a Windows guy, sadly (it’s a long story as to why), and so I’m talking about my experiences accessing AI software via a Windows computer, not a Mac. Because all of the software here is browser-based, there shouldn’t be much of a difference using these tools on a Mac. But I can offer no guarantees.

You will need some money to play the AI game, but not a helluva lot. As described above, much of the foundational software can be accessed via free versions. But you’ll usually want a paid description while you test it, and the subscription should be about $20/month. Cancel after you’ve tried it (don’t forget!). If you see something that’s $79/month or, god forbid, $159/month, you can ignore it. They’re not thinking of you as a potential user; they’re thinking of large mainstream corporate users.

Today’s AI is deceptively simple. Anyone who can type a question can use it. But using it well is complicated, which makes no apparent sense, until you dive into the complexities around “prompts” and similar ideas.

In order to access its intelligence sometimes you have to talk to it as if it were an idiot. Or perhaps an idiot savant. It’s a bit like Dustin Hoffman in Rain Man.

Anyone can use ChatGPT for free. Just go to chat.openai.com. Accessing the most recent version, GPT-4, costs $20/month, or is free at copilot.microsoft.com (best accessed via Microsoft’s Edge browser). And now (in early April) there’s a GPT-4 Turbo version for paying customers. This will change again soon: the updates are relentless.

The opinions vary as to whether GPT-4 is worth it. When it comes to software I always advise using the latest version.

Keep in mind that training the underlying language models is date-sensitive. OpenAI’s ChatGPT Turbo was recently updated to information from December 2023. It will tell you, as it just told me that it doesn’t “have direct access to external websites like Bing or other search engines,” and when asked about recent events surfaces recent news.

When you go to ChatGPT you find essentially a blank screen and the question “How can I help you today?”

You can ask questions. Better still, you can upload a 512mb PDF though it might only process, analyze, or respond to, a portion of the text at one time. You can also upload images, which it can describe, or a scanned page: it can recognize the text.

What has emerged as a gating issue for the successful use of ChatGPT (and its competitors) is learning how to “speak” with it (which you can literally do in the latest mobile app version). In ChatGPT-ese is called creating “prompts” or “prompting.” Users have discovered that the more precise and detailed their prompts are, the better the responses they receive from ChatGPT. Further, prompts are not just one-off. ChatGPT can continue a conversation for quite a while (though not indefinitely), and if you don’t get the answer you’re looking for you can revise and refine your prompts. This takes a lot of getting used to, and has spawned a series of how-tos, written and online, to train users on how to get the most out of prompting.

Try asking ChatGPT to explain a concept like developmental editing. Then ask it to craft an explanation that a 12-year-old could understand. The results are dramatically different. Amusingly, ChatGPT seems also to respond to emotional pleas. Adding “this is very important to my career” to a prompt will solicit more useful responses.

(This is as good a time as any to add a parenthetical: It’s both reassuring and deeply troubling that the top scientists working on language-based AI are unable to explain why things like this occur.)

Prompting is now at least partially supplanted by a new technology developed by OpenAI, called, confusingly, GPTs. GPTs are packaged prompts, specific tasks encoded as plug-ins to ChatGPT (easily confused with actual ChatGPT plug-ins, which are now being phased out). They’re easy to create —- you can ask ChatGPT to help you create a custom GPT. How they differ from standalone prompts is less clear.

At the same time, numerous developers are building applications that offer a front end to ChatGPT and other chat interfaces, without exposing the underlying chat layer. They then move beyond mere chat functionality. Copy.ai and Jasper.ai, for example, provide front-ends to multiple LLMs, focused on marketing tasks.

How to Prompt

How to GPT

Apart from any of the broader “systemic” issues that AI has brought upon us, there’s a single day-to-day functional issue that every user faces all the time when using LLM-based AI: It makes stuff up. It just does. It’s mostly called “hallucinations.” It’s a real problem, a serious problem. You need to understand hallucinations if you’re going to work with AI.

Cambridge Dictionary’s Word of the Year for 2023 was “Hallucinate,” whose definition has been expanded to include “When an artificial intelligence… hallucinates, it produces false information.” (Other additions to the 2023 dictionary include “prompt engineering,” “large language model,” and “GenAI”.)

AI hallucinations, Cambridge notes, “sometimes appear nonsensical. But they can also seem entirely plausible–even while being factually inaccurate or ultimately illogical.” This, sadly, is quite true, and as of April of 2024 remains a dramatic limitation for using generative AI for mission-critical tasks. It’s one of the several great oddities of AI, and it takes people awhile to get their heads around it. Remember, generative AI is mostly a next word prediction engine, not a database of facts. (Given this, that you can extract factual information at all from ChatGPT is truly a wonder.) Hence the need for HITLs, Humans-In-The-Loop, as we’re now known, double-checking AI output. And again, it’s remarkable that we can get such extraordinary value from a technology that can produce provably inaccurate output. So it goes.

Garey Marcus, an experienced and well-informed AI-critic, compares AI hallucinations to broken watches, which are right twice a day. “It’s right some of the time,” he says, “but you don’t know which part of the time, and that greatly diminishes its value.”

Ethan Mollick, a keynote speaker at the Publishers Weekly September 2023 conference, notes that people using AI expect 100% accuracy. Hallucinations, he says, are similar to “human rates of error” which we tolerate daily.

Andrej Karpathy, a noted scientist specializing in AI, who currently works at OpenAI, writes about hallucinations:

How to Safeguard Against Hallucinations

Include here Retrieval-Augmented Generation (RAG)…

Because trade book publishing is dependent more on text than on images, publishers tend to overlook the amazing AI-based tools for images and video.

More to follow.

The main AI-related activities of the most prominent publishing trade groups appear to be adding their voices to the chorus of the copyright concerned. The Association of American Publishers “filed reply comments in the U.S. Copyright Office inquiry into the intersection of copyright law and artificial intelligence (AI) in response to flawed and inaccurate assertions submitted by some tech companies and/or their investors in the first comment round including tired assertions that the rights of authors and publishers are an obstacle to innovation.”

The U.K. Publishers Association notes that “it is of the utmost importance that the Government puts in place tangible solutions as soon as possible to protect the human creativity and knowledge that underpins safe and reliable AI. Human creativity is the bedrock of the publishing and wider creative industries.”

On the other hand, the Independent Publishers Guild (IPG), with more than 600 members, offers AI guidance and training to its members, including its “Practical Guide to AI in Publishing,” conducted by a noted AI and publishing expert, George Walkley.

Here is the U.S., I’m delighted to be part of the Book Industry Study Group’s (BISG) AI Working Group, which reports to the Workflow Committee. The Working Group has outlined several potential strategic initiatives:

Similarly, the Independent Book Publishers Association (IBPA) is getting in front of the topic, for example with a session at its April 2024 IBPA Publishing University, “AI (Artificial Intelligence): How to Use It, and What IBPA Should Be Doing About It.” I’ll be a participant in that session.

The specific use cases for AI and book publishing, across different functions, are easy to describe conceptually. But there’s very little available information about what publishers are actually doing.

More to follow.

AI & Editorial: What Happens When AI Reads a Book?

Among the things that qualify Mollick as a commentator is that he has no skin in the game. He doesn’t need to sell AI, nor to trash it. He has merely committed to exploring AI in its many impacts, mostly upon education, culture, writing, and publishing. And he’s a fine, clear writer.

If you browse through Mollick’s newsletter archives you’ll see that he didn’t begin to focus on AI until December 2022. It wasn’t his beat —- like most of us, AI dropped in on his day job, and he couldn’t take his eyes off it.

In this post his insights falls closest to our interest as publishing professionals. “Might AI,” he asks, “change the way we interact with books?”

To answer the question, Mollick notes, “we need both an AI with a memory large enough to hold a book, and an author who knows their own book well enough to judge the AI’s results.” Mollick tests one of his several titles (he doesn’t specify which one, but from the chats it’s clearly The Unicorn’s Shadow: Combating the Dangerous Myths that Hold Back Startups, Founders, and Investors, a book very favorably reviewed on Amazon, though not a current bestseller).

Mollick looks at different aspects of AI’s potential value to an author, publisher or reader, including “AI as reader and editor,” and “a practical use: help for instructors.” He asks an LLM —- large language model —- not ChatGPT —- to summarize the book. It succeeds to Mollick’s satisfaction.

Then a tougher challenge: “Give me examples of metaphors in the book.” Metaphor, he points out, “is challenging for even human readers, as it involves finding a use of figurative language without any clear markers (unlike a simile, there are no “likes” or “as”).” The results, he records, “are impressive, though there are minor errors.”

The LLM is less successful as an editor: it’s failings in this department, Mollick notes, illustrate “something that has become clear about the current state of AI: if you are a very good writer or editor, you are better than current AI…”

Nonetheless, “AIs have, or at least have the appearance of having, an understanding of the context and meaning of a piece of text.” As a result, Mollick believes that “how we relate to books is likely to change as a result of AI.”

I think so too.

AI & Book Design & Production

More to follow.

AI & Book Marketing

More to follow.

You might think that the stately world of metadata would be slow to wrap its fuzzy head around AI. Not so! Last November, EDItEUR, the keeper of the ONIX standard, released a short Application Note called “Aspects of AI in ONIX” (pdf).

With his typical wisdom, Graham Bell, the organization’s director, notes that “one reaction to (the controversies surrounding the technology) is to forswear use of AI or to avoid trading in AI-created products. A more realistic option is simply to be transparent with trading partners and readers when AI has been used. And as some resellers limit or ban AI-based content from their platforms, it is important for reputable publishers to highlight those products that do use generative AI techniques to create content.”

Bell goes on to outline ways that publishers can specify in metadata:

… as well as a method to indicate in the metadata for digital products that the publisher explicitly opt outs of text and data mining (TDM) for uses other than research. There’s also a way to specify a separate license covering commercial or non-research TDM.

As is often that case, what is specified in ONIX may not be uncovered down the food chain, but at least a best effort has been made.

Business Systems for Book Publishing

More to follow.

There are few things that publishing companies are less comfortable doing than integrating complex digital technologies into their day-to-day operations. That’s understandable. AI, in particular, is causing anxiety for everyone, and not just in publishing. It’s new, it’s mysterious, it’s personalized, it’s powerful. People are threatened by AI for numerous reasons. Changing attitudes takes time.

But this is not a great time to be timid with technology.

There are calm and rational approaches available to integrate AI into your publishing operation, regardless of the size of your organization.

The impetus must come from the top. The very top. Senior executives need to embrace a vision of AI’s potentially transformative impact and communicate a program to staff across the organization. The program may be little more than “experiment, document your experiments, and share.” That’s a good start.

Publishing companies are handicapped by the hubbub surrounding copyright. Authors are up in arms. A May 2023 Authors Guild survey found that “90 percent of writers believe that authors should be compensated if their work is used to train generative AI technologies,” and 67 percent said they “were not sure whether their publishing contracts or platform terms of service include permissions or grant of rights to use their work for any AI-related purposes.” Those uncertain authors are now asking their publishers if AI is being used in the editing or production of their work, and some powerful authors are insisting that it not be. They’re looking for the AI equivalent of a peanut-free bakery.

This is a thorny problem for publishers —- if you can’t use AI on the books you’re planning to publish, what can you use it for?

Despite widespread AI use, few publishers have publicly defined their AI policies, and communicated their approaches to AI to the public. The term “the public” has a slippery significance here, when you consider the different publics addressed by trade, scholarly and educational publishers.

For trade publishers the most important audience is authors and their agents. AI is a sensitive topic for that public, to say the least. Scholarly publishers face different obstacles, when they consider AI’s promising impact on research, and then AI’s more problematic impact upon converting research into narrative (Avi Staiman wrote a thoughtful post on this topic). For educational publishers, establishing policies is tricky, as AI’s encroachment on the practice of learning is multifaceted and ongoing.

I think that all publishers face two big challenges as they move forward with AI technologies. The first is to develop a company position about how to approach AI generally, and on how to incorporate AI into their workflows. The second challenge is communicating that position, clearly and unambiguously, to their constituents.

Developing a public position on AI is something that a few of the larger publishers have addressed. But not many, and particularly not trade publishers.

The publisher policies I have seen are mostly flawed. Some of them are in fact policies directed externally, AT authors, with a range of admonitions about what is acceptable practice (not much) and what is not acceptable (lots). O’Reilly’s “AI Use Policy for Talent Developing Content for O’Reilly” goes on for pages and pages, with esoteric guidance, such as “DO NOT use any OSS GenAI Models that produce software Output that is subject to the terms of a copyleft or network viral open source license.”

On the other hand scholarly publisher Elsevier, in the “Elsevier Policies” section of its website, includes statements on “Responsible AI Principles,” “Text and Data Mining,” and “The use of generative AI and AI-assisted technologies in writing for Elsevier.”

The very few internal publisher policies that I’ve seen are conservative, excessively so. These publishers reacted too quickly to the range of perceived threats, and to their authors’ anxieties, and have hamstrung their own ability to engage robustly with this fast-developing, fast-changing technology.

It’s a given that they will use AI “responsibly,” whatever that means. It’s a given that they have the utmost concern for authors’ intellectual property and for aggressively protecting author’s copyrighted work. (Though, of course, these broad principles must be publicly stated, and often re-iterated.)

But what else?

If so they must make this clear, and clearly explain, the thinking behind these policies. Publishers must be brave in countering the many objections of most authors at this time of fear and doubt.

For more

Job considerations

More to follow.

Trying and Testing

The Economist recently referenced an organizational tactic for new technology adoption called “the lighthouse approach.” You create a beacon by selecting one high-profile proof of concept that can be implemented quickly, that everyone can relate to.

… the importance of what’s sometimes called HITL, for “human-in-the-loop,” the reality that AI is not autonomous, but rather an interaction between humans and machines; the machine is merely a tool.

More to follow.

AI for audiobooks works really well. It’s not perfect, but it works. Authors and publishers are now using it frequently.

Using AI for audiobooks is not new; I first reported on the trend in Publishers Weekly in 2021. Large language models have reinvented the technology for automated audiobook narration.

Back in 2021 we noted “Is it perfect? Certainly not. Can it be good enough? Probably, if a publisher is willing to spend the necessary time in the voice editing phase of the project.” Two-and-a-half years later, by many accounts, AI-voices are often undetectable from human voices. You can clone any voice with a minimum of training.

Last November Meta (Facebook) introduced “Seamless,” which is able to “transfer tones, emotional expression, and vocal style qualities” into the translation of 200 languages. An expertly-narrated audiobook can be immediately translated into multiple languages with extraordinary quality (assuming the publisher has obtained the necessary rights from the English-language narrator(s).

Also in 2021 I reported that “Audible’s block on the distribution of audiobooks with non-human narrators is a real problem that may take some time to resolve.” In the meantime, both Google and Apple announced programs to allow authors to create audiobooks with AI-generated voices. On December 5, Findaway Voices by Spotify began accepting “digital voice narrated audiobooks from Google Play Books for distribution to select retail partners.”

In early November Amazon announced that Kindle Direct Publishing (KDP) authors would soon have access to a service that would allow them to “quickly and easily produce an audiobook version of their ebook using virtual voice narration, a synthetic speech technology.” In January, 2024, Jane Friedman reported (paywall) “Audible quietly started allowing AI-narrated audiobooks to enter its storefront late last fall, long after other retailers had done the same.”

On May 2, Bloomberg reported that “Over 40,000 books in Audible are marked as having been created with (AI)…”

More to follow.

I’ve got an upcoming webinar, on June 25, 2024:https://www.bisg.org/events/can-ai-be-useful-in-book-translation

Defining “Good Enough” in Book Translation

Much more to follow.

AI for Scholarly Publishing

More to follow.

AI Software for Authoring

Much more to follow.

The concerns around AI are serious. The risks are real. Sometimes they are expressed in hysterical ways, but, when you drill down, the impact of AI has the potential to be enormously destructive.

There are so many issues and concerns surrounding AI, that they fill volumes on their own. Here are the topics I monitor. I’m sure I’m missing a few.

Amazon overwhelmedBias, Bubble,Copyright, Corporate concentration,Data (running out of), Design concerns, Destroy or save humanity,Education, Environment and energy, Ethics,Fake content, Hallucinations,Hate, disinformation, p*rn, spamJob lossLitigation, endlessMismanagement of AI companiesRegulationScammers generallySearch, the future of

There is a lot of information available on these topics, and I encourage you to read as deeply as you can. It’s possible you’ll conclude that the risks outweigh the benefits, and that you don’t want to pursue using AI personally or within your organization. That decision brings its own risks: the usual —- of being left behind —- but that’s a personal choice. (That I would even write that sentence in a book extolling AI’s benefits is an indication of the severity of the risks.)

If you google “books regarding the risks of AI” you’ll find a selection of worthwhile volumes. A recent podcast that I found particularly chilling is Ezra Klein’s chat with Dario Amodei, Anthropic’s co-founder and CEO (the company that develops Claude.ai). You realize that these companies are aware of the risks. Amodei refers to an internal risk classification system called A.S.L., for “AI Safety Levels.” We’re currently at ASL 2, “systems that show early signs of dangerous capabilities —- for example ability to give instructions on how to build bioweapons —- but where the information is not yet useful due to insufficient reliability or not providing information that e.g. a search engine couldn’t. Current LLMs, including Claude, appear to be ASL-2.” ASL 4 he describes in the interview as “enabling state-level actors to greatly increase their capability, which is much harder than enabling random people. So where we would worry that North Korea or China or Russia could greatly enhance their offensive capabilities in various military areas with AI in a way that would give them a substantial advantage at the geopolitical level. And on the autonomy side, it’s various measures of these models are pretty close to being able to replicate and survive in the wild.” Chilling stuff.

With this as a context, I’ll highlight the most pertinent issues for writers and publishers.

Copyright infringed?

The copyright issues are a miasma of complexity and ambiguity. It appears certain that some books still in copyright (not in the public domain) were included in the training of some LLMs. But it’s certainly not the case, as some authors fear, that all of their work was hoovered up into all of the large language models.

The copyright issues are both specific and broad. On the one hand, it appears provable that at least one of the LLMs ingested the actual text of thousands of books not in the public domain. LLMs are built unreservedly on the writing of others, whether it’s newspaper articles, social media posts, web blogs and, apparently, transcripts of YouTube videos. It’s well-known that all of the LLMs are trained on the open web —- blogs and periodicals and so much more that can be scraped from the 1.5 billion sites on the web today. Was it legal to hoover up all of this text to build billion-dollar AI companies, without any compensation to the authors? The AI companies have a strong argument around fair use; the courts will eventually decide. Perhaps it was legal, but was it ethical or moral? The ethics appear less complex than the legal considerations. You decide.

The laws surrounding copyright obviously did not anticipate the unique challenges that AI brings to the issue, and searching for legal solutions will take time, perhaps years. (If you wish to delve further into why the laws are unsuited to the particular problem at hand, read the excellent paper by A. Feder Cooper and James Grimmelmann called “The Files are in the Computer: Copyright, Memorization, and Generative AI.”

Copyright and AI for authors

There are some significant issues that authors face around the copyright-ability of AI-generated content.

The U.S. Copyright Office’s position on the copyright-ability of AI-generated content states that AI alone cannot hold copyright because it lacks the legal status of an author. That makes sense. But this assumes 100% of the work is AI-generated. As discussed elsewhere, few authors are going to let AI generate an entire book. More likely it will be 5%, or 10% or… And here the Copyright Office stumbles (as would I).

In a more recent ruling “the Office concluded that a graphic novel comprised of human-authored text combined with images generated by the AI service Midjourney constituted a copyrightable work, but that the individual images themselves could not be protected by copyright.” Jeez!

Suffice it to say that authors and publishers need to be alert to evolving copyright issues, on multiple fronts.

What are the long-term implications?

Some people compare the current litigation to the Google books lawsuit, which took 10 years to legally resolve. Who knows how long the appeals process will drag out for these numerous suits. In the meantime, publishers are prudent to act as if the AI companies will lose, which, at least in theory, exposes anyone who licenses, or perhaps even uses ChatGPT, to some sort of contingent liability.

But it’s not the (unlikely) liability that’s a publisher’s most serious issue. It’s perception. For many authors, some prominent, some obscure, the AI well has been poisoned. AI is now radioactive within the writing and publishing community. Anything that even smacks of “AI” draws intense criticism.

There are numerous examples. In a recent incident Angry Robot, a UK publisher “dedicated to the best in modern adult science fiction, fantasy and WTF,” announced that it would be using AI software, called Storywise, to sort through an anticipated large of manuscript submissions. It took just five hours for the company to drop the plan and return to the “old inbox.“

This case is particularly instructive of the larger implications for publishing of not using AI tools. As I discuss in the section on what’s happening over at Amazon, even before the increased content deluge enabled by AI tools, the quantity of books currently being written and published is enormous —- some 2.5 million new books published each year, on top of the 50 million plus books already available. And think how many manuscripts are written each year that aren’t published, but which flood publisher email inboxes. The situation was already untenable before AI.

In an in-depth interview on Aidan Moher’s blog, Angry Robot’s publisher Eleanor Teasdale described the reason for adopting an AI-enabled submission tool. “In recent years our response time on open submissions has gotten out of hand,” she said. “We were hoping this would allow us to respond far faster to authors who submitted.” The intention was increased efficiency. “Every submission was still going to be assessed and read by the team,” Teasdale said.

But now, according to Teasdale, “Angry Robot has no plans for using AI-assisted tools in the future.” Is there a winner in this scenario? Not the authors. I’ll look further at job loss below, but there’s no indication that Angry Robot was looking to decrease staffing levels by using Storywise, only that it was trying to speed things up.

The unbearable dilemma for trade publishers in exploring using AI tools internally: if the authors find out, you’ll have a hard time weathering the resulting storm. I believe that publishers have no choice but to be brave, to adopt (some of) the tools, clearly explain how those tools are trained and how they’re utilized, and push on.

In the UK, The Society of Authors takes a hardline approach: “Ask your publisher to confirm that it will not make substantial use of AI for any purpose in connection with your work —- such as proof-reading, editing (including authenticity reads and fact-checking), indexing, legal vetting, design and layout, or anything else without your consent. You may wish to forbid audiobook narration, translation, and cover design rendered by AI.”

The Authors Guild appears to accept that “publishers are starting to explore using AI as a tool in the usual course of their operations, including editorial and marketing uses.” I don’t think that many members of the Guild are as understanding.

It’s too late to avoid AI

For authors and publishers who prefer not to be sullied with AI, the news is bad: you’re using AI today, and have been using it for years.

Artificial intelligence, in different forms, has already been integrated into most of the software tools and services we use every day. People rely on AI-powered spell- and grammar-checking in programs like Microsoft Word or Gmail. Microsoft Word and PowerPoint apply AI to provide writing suggestions, to offer design and layout recommendations, and more. Virtual assistants like Siri and Alexa use natural language processing to understand voice commands and respond to questions. Email services leverage AI to filter messages, detect spam, and send alerts. AI powers customer service chatbots and generates product recommendations based on your purchase history.

And much of this is based on Large Language Models, such as the one used behind the scenes at ChatGPT.

For an author or editor to say, “I don’t want AI used on my manuscript,” is, broadly speaking, all but impossible, unless both they and their editors work with typewriters and pencils.

They could try saying, “I don’t want generative AI” used on their book. But that’s a tough one to slice and dice. Grammar-checking software was not originally built on generative AI, although Grammarly has added it as an ingredient to its product, as, inevitably, all other grammar checkers will as well. Generative AI is core to all of the marketing software on offer.

When authors use AI

Another issue that touches on copyright, but is not primarily about copyright, surrounds authors and the use of AI. In the extreme, we’re seeing 100% AI-generated content being published on Amazon. Most of it (all of it?) is of terrible quality, but that doesn’t prevent it from being published. (See also the Amazon section.) More concerning for publishers is AI-generated submissions. Yes, AI ups the quantity, but publishers already have a filter for quantity. The filters are called agents. They are the ones who are going to have to figure out how to handle the quantity problem, and apparently they’re going to have to find a solution that doesn’t use AI.

And it’s not the quality problem —- agents are the first filter there as well.

It’s more of an existential problem —- do I want to publish a book written by “a machine”? For most publishers that’s an unequivocal “no.” Easy peasy. Well, what about a book where 50% of the content was generated by an LLM, under a capable author’s supervision? Hmm, let’s try a “no” on that as well. OK: then what about 25%, or 10%, or 5%? Where do you draw the line?

I can’t find any trade publisher who has declared that they will not publish a work with a specific quantity of AI-generated text. Here’s the Authors Guild on the topic:

“If an appreciable amount of AI-generated text, characters, or plot are incorporated in your manuscript, you must disclose it to your publisher and should also disclose it to the reader. We don’t think it is necessary for authors to disclose generative AI use when it is employed merely as a tool for brainstorming, idea generation, or for copyediting.”

Needless to say, “appreciable” is not defined (Oxford defines it as “large enough to be noticed or thought important”), but the post goes on to explain that the inclusion of more than “de minimis AI-generated text” would violate most publishing contracts. De minimis, in legal terms, is not precisely specified, but, generally speaking means more or less the same thing.

Can AI be detected in writing?

For authors and publishers, AI can be radioactive. Fearing the threat of unknown outcomes, the true conservative won’t let AI anywhere near their words. Publishers bear a special burden —- they don’t create the text, but, once published, they shoulder a substantial obligation to the text. We’ve seen lots of dynamite blow near explosive books, whether it be around the social implications of the content, or the plagiaristic purloining of other writer’s words and ideas. Now with AI we face a whole new set of ethical and legal issues, none of which were outlined in publishing school.

Wouldn’t it be great if you could just feed each manuscript into some software that would tell you if AI had be used in creating the text?

Leaving aside the issue that the only way to do this would be by employing AI tools, the more important question is, would the software be (sufficiently) accurate? Could I rely on it to tell me if AI had been used in creating a manuscript? And could I depend on it not to produce “false positives” —- to indicate that AI had been used, when in fact it had not?

There’s now lots of software on the market that tackles these challenges. Many of the academic studies evaluating this software point to its unreliability. AI-generated text slips through. Worse, text that was not generated by an AI is falsely-labeled as having been contaminated.

But book publishers are going to want some kind of safeguards in place. It appears that, at best, these tools could alert you to possible concerns, but you would always need to double check. So perhaps it might alert you to texts that need to be more carefully examined than others? Is this an efficiency?

True efficiency will be found in moving beyond concerns about the genesis of a text, instead maintaining our existing criteria as to its quality.

Job Loss

Job loss from AI adoption could be catastrophic. The estimates vary, but the numbers can be grim. There are obvious examples: San Francisco’s driverless taxis eliminate… drivers. AI-supported diagnostics appear likely to reduce the need for medical technicians.

Ethan Mollick’s study with the Boston Consulting Group (BCG) was an experiment that aimed to better understand AI’s impact on work, especially on complex and knowledge-intensive tasks. The study involved 758 BCG consultants, randomly assigned to use or not use OpenAI’s GPT-4 for two tasks: creative product innovation and business problem solving. The study measured the performance, behavior, and attitudes of the participants, as well as the quality and characteristics of the AI output. Among the findings was that “AI works as a skill leveler. The consultants who scored the worst when we assessed them at the start of the experiment had the biggest jump in their performance, 43%, when they got to use AI. The top consultants still got a boost, but less of one.” The full article is revealing, and as with all of Mollick’s work, provocative yet accessible.

What about publishing? Which jobs are vulnerable? Neither Microsoft Word’s spell- and grammar-checking, nor Grammarly, have obliterated the position of copy editors and proof readers. Will marketers’ careers be curtailed by GPT-enabled marketing texts, or will their effectiveness be much enhanced?

At the recent PubWest conference in Arizona a speaker from outside the publishing industry suggested that one of the uses for AI will be replacing interns. The room burst into flames. She meant well —- indeed an April 10, 2024 report in the New York Times describes how Wall Street investment banks are looking to replace many of their interns with AI. As it is with publishing, an obvious concern is how do you find senior analysts if they can’t start off as junior analysts?

The publishing industry has always relied on internships. In part it’s a way to get the grunt work dispatched at a reasonable cost. But that factor pales against the reality that no publishing school can equip someone to join a publishing company at the level of middle-manager. The only way to develop the skilled staff of tomorrow is to train interns, apprentices (call them what you will), today.

So what impact does AI have on this equation? If nothing else it will reduce the number of interns required, as the grunt work becomes more efficient. Beyond that, I look broadly at the future of AI in publishing at the conclusion of this report.

Education

Education has been front-and-center in the pro and con debates about AI. The introduction of AI into classrooms is largely seen as a curse, or at least a challenge. Other educators, like PW’s keynoter Ethan Mollick, embrace AI as a remarkable new tool for educators; Mollick insists that his students work with ChatGPT.

I’m not going to delve into educational publishing in this book —- it’s a vast topic, and demands a separate report. A superficial account would be a disservice.

The Future of Search

Search is a fraught topic in AI. I encourage you to visit perplexity.ai to get a glimpse into where things are headed (or use Microsoft Copilot, a similar but less elegant tool). The next couple of times you’re thinking of starting a Google search head over to Perplexity instead. It won’t seem dramatically different —- it’s similar to the knowledge graphs that Google often pops onto the right hand side of a search screen, or sometimes on top of the search result listings. Instead of having to click a link, the information is right there for you.

Perplexity goes a step further, rephrasing the information it gathers from multiple sources so that you really don’t have to click a link. It provides links to its sources, but clicking them is usually unnecessary —- you’ve already got the answer to your question.

This seemingly modest shift has huge implications for every company and every product that relies, at least in part, on being discovered through search engines. If searchers are no longer being sent to your site, how can you engage them and convert them to customers? Simple answer, you can’t.

Joanna Penn is generally at the forefront of thinking about the impact of new technologies on writing and publishing. She tackled this complex topic on her podcast and blog last December.

It’s very early days for AI and the transformation of search.

Junk Books on Amazon

AI-generated junk books on Amazon are a problem, though their severity may be more visceral than literal. On the one hand these books are spamming the online bookstore with low-quality and plagiarized content, sometimes using the names of real authors to deceive customers and take advantage of their reputations. The books are not only a nuisance for readers but also a threat for authors, depriving them of hard-earned royalties. AI-generated books also affect the ranking and visibility of real books and authors on Amazon’s site, as they compete for the same keywords, categories, and reviews.

Just now I searched Amazon for “AI-generated books.” Try it. There are lots. Many of the results are how-to books about the use of AI for creating books. But others are, unabashedly, AI-generated. “Sauna Girls Relaxing Beauties AI-generated photo collection” is by “jhk AI,” who also authored “Ninja Girls Kunoichi beauties running around the world AI-generated photo collection.”

“Comrades of Mars: AI Generated Graphic Novel” by Berkay Oğuz Aykan, who is perhaps a real person, was published in June last year, and languishes at #1,486,395 in the Kindle Store (despite a single 5 star rating).

Bias

As LLMs are training on that which has already been published, and given that, taken as a whole, what has already been published is rife with bias, LLMs reflect that bias. And of course not just bias, but hate, hate reflected in its learnings, and hate as a potential output in words and images. And of course p*rn is a natural beneficiary of the AI’s remarkable facility with images, and there are recent troubling stories of young women finding fabricated nude images, their males classmates as likely suspects. The New York Times reported separately about an increase in online images of child sexual abuse.

Authors and publishers need to be aware of this built-in bias when using AI tools, particularly for text generation.

How Big of a Deal is AI?

Where Are We Headed?

Needless to say, a work in progress

▪ Augmentative or transformative?

▪ Augmentative is just a tool

▪ Transformative indicates a new way of doing business

A technology becomes transformative when it moves beyond merely augmenting existing processes to fundamentally altering them, leading to significant changes in behaviors, practices and societal norms.

The polarizing question of whether machines can be truly creative remains a topic of ongoing debate. It depends largely on how we define creativity.

We all know what creativity is, don’t we?Cambridge: “the ability to produce or use original and unusual ideas”

Brittanica “creativity, the ability to make or otherwise bring into existence something new, whether a new solution to a problem, a new method or device, or a new artistic object or form.”

I don’t in any way dispute that true creativity is well beyond the reach of LLMs et al.

My argument is that most of what’s passed off as creative, and often appears quite creative, is, in fact, iterative. Which works perfectly well for many things, including most advertising.

But to raise the “creativity” stanchion, and then diminish LLMs (et al.), is to create an impossibly high barrier, at which AI invariably fails. Then people dismiss it as “well I told you it’s not creative.”

And thereby miss the iterative stuff that it’s very good at.

Jeremiah Owyang is an industry analyst based in Silicon Valley. He’s an AI booster, but he’s not new to hype. He notes that he’s “never seen such rapid evolution in such a short time.”

As Owyang puts it, small teams of programmers now can use LLMs to assemble “sentient creatures” —- like a 4-year-old —- in two days. They are capable of:

This is extraordinary and unprecedented. Will it matter to publishing? You decide.

AI is behind some groundbreaking achievements outside of publishing. Medicine is AI’s poster child, but there are some powerful stories in other industries. The relevance to publishing stems from a rhetorical question: AI can save lives. You don’t think it can help publishers?

The idea of this section is merely to counter the argument that, unconvinced that AI is going to help publishing, can AI possibly help anyone at all?

AI as Individual Performance Booster

“You’re not going to be replaced by AI. You’re going to be replaced by someone who knows how to use AI.”

AI and Medicine

AI and Software Development

I was talking recently with Peter Brantley, my partner in the AI webinar weeds, about how to shape upcoming AI events. Peter works much of the time within the library community, and had just returned from a day-long event in Los Angeles with some 150 of his library colleagues, where they explored, in person, AI’s potential impact on their sector. The conversation was lively and in-depth. Why, he wondered, don’t trade book publishers have gatherings like this?

The answer he came up with is that trade book publishers haven’t yet concluded that AI is going to have any real-world consequences on their business. On the core of their business. On their ability to conduct business. Sure, it might help write a press release or pen a manuscript rejection letter, but they assume that publishing will still putter along as it’s been doing for decades, with the average time from contract to print book dragging on for up to two years.

At the same time, of course publishers face near-term challenges more pressing than AI. Paper prices. Shipping costs. Shrinking margins. Sales flat overall. Who cares about AI?

Other book publishing sectors are working to get a handle on AI’s impact. Scholarly publishing is deep into AI. Educational publishing sees the truck coming down the road. But trade publishers are acting like there will be few real-world consequences from AI. Why? Is it supply? Demand?

Supply: Yes, AI is the culprit behind a bunch of new garbage books on Amazon. Hundreds? Certainly. Thousands? Maybe. Millions? No. How many books are on Amazon? More than 50 million. It’s very crowded there already.

That aside, it’s clear that in the near term, AI isn’t going to be writing whole books that people will actually want to read. Its value thus far is as a writing buddy —- critiquing, suggesting, occasionally spinning out a few paragraphs of usable text. No, in the near term, AI is not going to be radically changing the inputs of trade publishing, the supply.

Demand: Nope, no one is demanding AI-generated books. That’s not a factor.

What about process? Yes, that will change. At least a little. As we discovered at our Publishers Weekly AI event last fall, publishers are looking to AI to help here and there: marketing, editorial, a bit of this and a bit of that.

So, it’s possible that there are few real-world consequences of AI on trade book publishing? Perhaps it’s just a toy.

Part of the problem is that ChatGPT and its competitors are deeply flawed software. The hallucinations are well-known, but it’s not just the problem of making stuff up.

For many queries, particularly from novices, the responses are mundane, off-target or simply unhelpful. ChatGPT cannot count: Ask it for a 300-word blog post and you’ll be lucky to get 30.

And each of the AI companies, in order to reduce bias and to avoid answering “how-to-build-a-bomb” queries, has had to erect tight response guardrails around their products: all too often, the response to a question is, essentially, “No, I won’t answer that.” I asked Google Gemini to review a draft of this text and was cautioned that “it’s essential to get the author’s approval before publishing.”

Publishing has been in economic decline for decades

I think that the impact of AI on book publishing should be evaluated within the wider trends in publishing and the media industries. The trade book publishing has been in a gentle economic decline for decades, some years up a few percent, some years down. In 2023, according to StatShot “trade revenues were down 0.3%, at $8.9 billion for the calendar year.” As far as I can tell, AAP’s StatShot does not account for inflation, which at 3.4% would indicate that the sales decline was in fact 3.6%, which is closer to the 2.6% decline in unit sales reported by Circana.

Without the growth of alternative formats, ebooks and audiobooks, the industry might be in very bad shape. Ebooks were a growth engine for years. Audiobook sales continue to climb, up nearly 15% in 2023. Yes, some of that is substitution sales, but a lot of the customers for ebooks and audiobooks are new customers, not necessarily regular book readers. Ebooks and audiobooks accounted for 21% of trade sales last year. Would all of those customers buy print if that’s all that was available?

Retail pricing is becoming an increasing concern as costs and retailer discounts squeeze publisher margins. Academic studies suggest that there is a degree of price elasticity for books, but surely we’re reaching the resistance point: hardcover bestsellers are running up against what appears possibly to be a $35 price ceiling. “Subscription fatigue” is causing video-on-demand customers to drop services, focusing on the increasing cost of each service. Netflix’s Premium plan now runs to $22.99/month; the Disney Plus Duo Premium is $19.99/month. As one colleague remarked, how many cash-strapped subscribers will say, oh, let’s cancel Netflix so I can buy another hardcover next month?

The three remaining advantages for traditional book publishers

The three remaining advantages for book publishers are (i) cachet, (ii) access to major media, and (iii) access to bookstore distribution. All of the other services that publishers currently provide can be purchased on the open market, with similar quality, at affordable prices.

The cachet of traditional publishing is the reason that most of us joined the industry. We grew up loving books, went on to study English or something similar in college, kicked about for a while, perhaps got a job in a bookstore, perhaps got a Masters of Publishing certificate, then worked as an intern at a publishing company, and here we are today. You love your job, though the pay sucks and the working conditions are not always pleasant.

That same cachet feeds the publishing supply chain. On the input side of the supply chain: it means a lot to a lot of authors. We see it all the time. The economics of traditional publishing makes no sense, and it takes four times as long to get into print, but, maybe, just maybe, I’ll be the next Delia Owens (Where the Crawdads Sing) or Amor Towles (A Gentleman in Moscow).

It’s like Vegas: except the odds are better in Nevada. Play one number in roulette and the odds against you are 36 to 1. Play traditional publishing and the odds against you are… well, you do the math.

The odds of getting a contract with PRH (Penguin Random House) are deplorable. The company publishes just 15,000 new titles each year. If you can get an agent and get through that gate, the odds start to improve.

In 2021 PRH managed to get 196 books onto the Publishers Weekly bestseller lists. “PW publishes four adult bestseller lists, each with 20 titles: hardcover fiction, hardcover nonfiction, trade paperback, and mass market paperback. That means that over the course of a year, there are 2,080 hardcover positions on our lists and 2,080 paperback positions.”

We’ll use that as a proxy for the other lists. Not all of the bestsellers were newly-published that year —- let’s guess generously that 90% were, 176 books…. (not sure where I’m going with this).

The New York Times has 15 bestseller slots per week for each category of book (fiction, nonfiction, children’s; separate lists for YA and how-to; hardcover separate from paperback, plus combined lists). For an individual author, that means 780 available slots per year, although most of those slots are filled with repeat sellers. (The Body Keeps the Score by Bessel van der Kolk has been on the paperback list for five-and-a-half years!) So let’s divide that in half, 390 available slots. You have to sell a lot of books to grab one of those.

As author Alexis Hall pointed out in a recent interview with The Hot Sheet (paywall), “the actual effects of getting on a (bestseller) list are often pretty marginal. It’s basically another tool in your publicity toolkit.”

What about a slot in the New York Times Book Review? In 2016, Pamela Paul, then the editor of the Review, claimed that “The Book Review at The Times reviews about 1% of the books that come out in any given year.” Clearly she means 1% of the books from the top publishers —- she can’t be thinking about small presses or self-published authors. And so perhaps 150 of PRH’s titles are reviewed each year.

Most observers agree that while getting attention from top media is still important, and can be powerful, the impact of reviews is not what it once was, if for no other reason that fewer book readers interact with the major review outlets.

Let’s now turn our attention to the supply chain downstream, to distribution. While publisher brands carry weight with authors, it’s always been a truism that, unlike many consumer goods, publisher brands carry next to no weight with book buyers and readers. That’s true. But they do carry a lot of weight with distributors and booksellers. This reflects the historic business-to-business nature of publishing (rather than B2C —- to consumers). Publisher cachet is disseminated via distribution and retail channels: Barnes & Noble carries a book because it’s from PRH, and shoves it in front of buyers’ faces; thus is the cachet disseminated to readers.

The Big 5 each run successful and aggressive distribution divisions: many publishers aspire to join (perhaps only 4 of the Big 5). Some smaller publishers have become masters of distribution: Microcosm Publishing is winning awards for its distribution strategy, which, among other achievements, keeps Amazon out of the loop.

What about bookstores?

It’s easy to get sales data about bricks & mortar bookstore sales —- the U.S. Census Bureau monitors these very closely. Although not necessarily reliably. In it’s 2021 report, Don’t Turn the Page on Bookstores, it revealed a statistic that I was unaware of: Trade book sales are just 46% of the Census reported bookstore sales. In 2017 these totaled $3.25 billion (which is impossible if the AAP report is accurate: overall trade sales of $8.9 billion). Statista has a chart showing bookstores sales from 1992 to 2023. The trend is “down slightly,” like the industry as a whole.

Census data appears even less reliable when it comes to online sales —- they show Amazon with about 22% of book sales. Many observers (here and here) put the number closer to 50% of print book sales and 80% of ebooks sales. Some reports show Amazon’s marketshare even higher.

Notably, bookstores have been left out of the only growth sectors in publishing in the last 20 years: ebooks, audiobooks (largely) and self-publishing.

Still, the story isn’t over. According to a recent New York Times profile, James McBride’s The Heaven & Earth Grocery Store “found its success in large part through support from bookstores.”

Self-publishing

The real growth in trade publishing has been in self-publishing. So much of the data around self-publishing remains hidden and so it can be easy to ignore its significance. But all of the available data shows that self-publishing has claimed a significant portion of the publishing market. (For example, here, and here.)

As Kobo CEO Michael Tamblyn (somewhat) famously noted, “One in 4 books we sell in English is a self-published title, which means that effectively, for us, self-publishing is like having a whole other Penguin Random House sitting out in the market that no one sees. It’s like the dark matter of publishing.”

The trends in the self-publishing market are arguably the only publishing trends that really matter. Self-published authors are showing the way. They are adventurous and uninhibited by the customs of the trade industry. Many are fluent with social media. They are close to their customers, their readers. If you want to learn the latest marketing techniques, follow author forums, blogs, and newsletters.

It’s significant to dig deep on author income, the economics of writing, and compare the income decline of traditionally-published authors versus the significant income gains of self-published authors. In an international survey and report that I conducted with Steve Sieck for ALLi, in 2023 (pdf), we discovered that “the median writing and self-publishing-related income in 2022 of all self-publishers responding was $12,749, a 53% increase over the previous year. Average (mean) incomes skewed much higher: $82,600 in 2022, a 34% increase.”

In a separate survey, conducted by Peter Hildick-Smith, the Authors Guild reported that “full-time self-published authors, who had been publishing since at least 2018, reported a mean income of $24,000 compared to $13,700 in 2018, a 76 percent increase.”

Hybrid publishers

For the sake of thoroughness, I should mention hybrid publishers as well. I’ve become increasingly enthusiastic about the hybrid model, and the hybrid segment is significant and growing. But it doesn’t change the calculus for this equation.

Publishing beyond publishers

I am co-authoring a report with Rüdiger Wischenbart, based on his original study “Publishing Beyond Publishers.” We’re trying to understand, and to quantify, as best we can, all of the book-ish publishing activity, worldwide, that could (and should) be included in a more comprehensive view of the modern publishing ecosphere.

The poster child for our project is Wattpad, where “97 million people spend over 23 billion minutes a month engaged in original stories.” The stories are experienced online, mostly on smartphones, in short chunks. Few become published books. They are just as likely to find their way into “Wattpad WEBTOON Studios, the company’s TV, film, and publishing counterparts.”

As we’ve been hearing through the grapevine, younger readers are more attuned to online digital reading than their more senior counterparts. According to a January 2024 Wattpad survey, “digital formats are increasingly popular among younger generations, with 65% of Gen Z and 71% of Millennials embracing webnovels, ebooks and webcomics, while less than half of the Gen X and Boomer generations say the same.”

And, of course, as the traditional publishing industry is now learning: “Diversity is a key driver for embracing digital formats: 61% of Gen Z and 70% of Millennials agree that ebooks, webnovels, and webcomics give them access to content that’s harder to find in bookstores and libraries, including LGBTQ+ and minority-focused content.”

Wattpad has competitors, including Inkitt, which recently raised $37 million from investors “including Stefan von Holtzbrinck, the owner of Holtzbrinck Publishing Group, which owns Macmillan, and former Penguin CEO Michael Lynton; it has now pulled in a total of $117 million in investments. In February 2023, the Financial Times said that Inkitt was the eighth fastest growing company in Europe, and #1 in Germany.” How many of you have heard of Inkitt, visited its publishing platform, or downloaded its Galatea reading app?

But our “Publishing Beyond Publishers” report will go beyond online story platforms to consider how “content can be created and disseminated in a diversity of formats (print, digital), media (books, audio, movies, games), distribution channels (communities, platforms, streams) and business models (sales of products, subscriptions, streaming, freemium, paid models) in mostly digitally-defined supply and marketing chains.” How many other opportunities are book publishers missing because they’re not in the current business model?

Innovation, technology and book publishing

Except for ebooks, modern publishing has never faced a threat from technology. (Audiobooks were greeted more with joy than apprehension.) The dawn of the Internet era offered publishers opportunity more than threat; it’s just a single retailer, enabled by the Internet, who has upset the apple cart. I’ve never seen a definitive study on whether Amazon has been, when all aspects of the vast changes are fully factored, a net positive for the book publishing industry. It reaches many buyers who aren’t served by bricks and mortar. But at what cost?

Still, don’t criticize Amazon in front of a self-published author. They would not be in business without The Everything Store.

The innovator’s dilemma

Christensen looks at how existing (incumbent) firms succumb to the forces of innovation.

Successful, well-managed companies can fail when disruptive changes come to their industry —- the management practices that allowed them to become industry leaders make it difficult for them to embrace the disruptive technologies that ultimately steal away their markets.

Part of the blame is the customers. Disruptive innovations are initially considered inferior by most of an incumbent’s customers. Incumbents ignore disruptive products because they compare poorly to current products and because their most profitable customers generally can’t use and don’t want them.

Established firms try to fight off disruptive threats by investing even more in their current products and listening even more carefully to their best customers. But this just makes them less likely to embrace the disruptive technology.

The book’s key insight is that by doing the “right” thing and listening to customers, successful companies leave themselves vulnerable to disruptive innovations. They focus on their current market and ignore new technologies that initially target small, less profitable markets. This leaves an opening for agile startups to eventually disrupt the leaders.

Chris Dixon, in discussing the book, points out that “the reason big new things sneak by incumbents is that the next big thing always starts out being dismissed as a ‘toy.’” Hmm, yes, a lot of people see ChatGPT as a toy.

Can traditional publishers hold onto their remaining advantages?

The cachet of traditional publishing has at least a few years left, and may linger. The recent launch of Authors Equity suggests a morphing of some aspects of the traditional business model, while also reinforcing the cachet of New York publishing heavyweights.

Access to major media has some residual value, though ever-diminishing. On-the-ground distribution has value, though, for many authors, their books never touch the ground.

The containerization of content

The reading public, particularly it’s younger members, is increasingly “media agnostic.” They don’t much care if they’re watching a YouTube or TikTok video, a Netflix series, listening on Spotify, scrolling through Instagram (“Insta”), catching up on the news, or, occasionally, reading a book.

It’s hard to shake a business model built upon a particular container. Movies are not books. YouTube videos are not movies. Podcasts are not audiobooks. And so on.

We train content creators to move into container silos. We have writing programs and film programs and music programs and those seem logical and necessary to us. Whatever happened to transmedia?

Good content will always win. But, over time, the content containers matter less and less. The digital generation may not be container-agnostic. But they are container-flexible. They encounter most of their content digitally, via smartphones. Print will continue to find an audience, but it will never again command the center.

The “publisher of the future” needs to be container-flexible. Most of the growth in content distribution is not via ink on paper, nor words in EPUB3 files.

The challenge lies in the need to make new and existing content discoverable, and then upon discovery, to turn awareness into purchase decisions.

Discovery and Conversion

The discovery problem has been with us for some time. AI did not create it, but will exacerbate it. The discovery problem is one that industry veteran Mike Shatzkin has pointed to more than once. We have so many books already in print. And digital formats mean that they are continually in print. On top of that two million or more new books are published each year. The quantity problem will get worse, but it’s already out of hand.

Amazon’s algorithms are not selfless —- they do a very good job of raising to the surface what’s saleable, the books that you’re most likely to purchase next. The book you’re most likely to purchase next is not characterized merely by its sales figures. It sold best not because some scammer optimized the online listing. It sold best because it was the best, and delighted readers told others to buy it, both online and in person.

AI can Communicate

“What’s the perfect book for me to read next?” Generative AI can answer that question with an eloquence and a precision that has never been possible. Once AI becomes a reliable recommender engine, authors will need to communicate via that engine. (Amazon, of course, is working on this, and has the benefit of knowing what you’ve previously purchased.)

Writers & Readers

The future of publishing is the intimate relationship between writers & readers. It’s far stronger than the relationship between writers & publishers and readers & publishers. Publishers are a blockade in the relationship between writers & readers. They do not enable it; they impede it.

For a long time, the only way to access high-quality written content was through books or via a modest selection of periodicals. That’s certainly no longer the case. Quite apart from the distractions of other media, there are now so many different ways to access (non-containerized) high-quality written content. Books no longer have the primacy that they enjoyed.

One colleague pointed out that, “and most of that online content is free.” Yes, a few well-known paywalls notwithstanding, admittedly so. But I do not believe that price has ever been a major issue inhibiting the buying of books. The book industry has proven to be largely recession-proof —- even when people are hurting in their pocketbooks, they buy books.

Futures

Concluding paragraph:

- What other roles might publishers play?

- The old thing doesn’t have to die for the new thing to succeed: this is not about the end of the writing and reading of books, only about the declining role of traditional trade publishers.

- Commit to doing fewer books, and doing more with the books you publish.

I had lunch with a colleague the other day and said, out of the blue, “I’m an optimist.”

“But,” he said, “you’re really down on the future of publishing.”

“Perhaps,” I replied. “But I’ve very optimistic about writers and about readers, or, to follow my arguments here, about the power of human creativity, and about the enthusiastic audiences that creativity always finds.”

After all, isn’t that what matters?

The publishing industry’s putative failure to innovate can be a bad rap. Being first to the technology party doesn’t necessarily matter, as long as you arrive before the guests move on. The publishing industry adjusted to the challenges of ebook and audiobook production and distribution. This represents a real triumph that’s sometimes not fully appreciated. The industry remains tech-averse, but it can move when required to do so.

More…

I spoke to a very smart author’s agent, who has been thinking a lot about what AI means to her clients, and to all professional authors. The conversation turned to the idea of the book as a “container,” and I asked her to expand on that idea. Her response:

“My thought on the book as a container is not fully formed. But the basic idea is that when authors/agents are placing a book with a publisher, it is just that: a book. There is an understanding that it might later become one of the myriad forms of derivative works we have come to know (translation, dramatic adaptation, graphic novel, etc.) but all of that is secondary to the “the Work” itself. The Work is not a bag of words or phrases or facts, it is something structured by the author in a way unique to that author. Breaking the container, shaking the Scrabble bag of letters and drawing out a random selection (that is not entirely random because it is a Scrabble bag made up of the syntax and semantics and style of the specific author), is not what is being considered when we make a grant of rights to a publisher.

“The book is the integral whole that is greater than the sum of its parts. It’s the product of a creative epiphany (and a ton of creative intellectual work) that brought it all together as a book, not just “a” book, but the book by that particular author.”

I responded:

When thinking about nonfiction books, to me, breaking the container is one of the upsides of the AI era. You break the container, and like Fabergé eggs, there are treasures contained within. The constraints of the container are both a feature and a drawback. On the one hand “the Work is not just a bag of words or phrases or facts, it is something structured by the author in a way unique to that author.” On the other hand, in creating the container, the author, by dint of the inherent limitations of the bookish container, was forced to reduce, reject, rewrite, and reconfigure. The work is polished, but it is (and I think most nonfiction authors admit this) distilled, and some things were lost along the way.

It’s not going to be easy for established authors to throw off their chains. But they probably have to learn to, or to risk being overshadowed by a new generation of skilled researchers and writers who can express themselves both with and without containers, as the occasion calls for.

I’m thinking (a lot right now) about my AI book. It’s a fun adventure. I told you I’m going to release it on Leanpub, which allows me to release early, and release often. The container will never be well-polished. I’ll miss that, but it’s not part of this process. Although there still is a container. In this process, I’ll achieve 75% complete, and then hit the “publish” button. The reader knows well that this is fast-changing technology. If it was a book about the war in the Middle East, it would be “fast-changing news,” or “a rapidly developing situation” etc.

In the current container paradigm, publishers offer only either (a) a polished container (well) after the fact or an (b) “instant book” (as with government reports, like the Mueller Report). These two container forms impose far too many limits, and limitations.

My “AI Book” could just as easily be a Substack newsletter, and I could try to monetize it with a subscription model. But I don’t think that people want just an episode per week or per month. They’d like a substantial “bookish” body of work to consult, even while knowing that it’s fluid, and inherently incomplete. Leanpub affords that.

Just imagine if you were trying to agent my book. Publishers would say, are you crazy -- this stuff is changing so fast, and by the time we get a book out it will be obsolete. Correct. so the container market doesn’t work here, but I’ve got some great “content” I want to share. I can offer podcasts. YouTube videos. Or I can offer a bookish container, because that’s my métier (though I know that a course might well be a better way to disseminate my knowledge —- I just haven’t a clue about building online courses —- I would need a newfangled “publisher” to help me, and I don’t know any, and nor do authors agents.)

What it comes down to is that the business of writing has irredeemably changed. It’s like the end of the scribes. In this case, our monkish authors must get out from behind their desks and their containerized view of the book, and engage with form and with their audiences.

Oh yes, I know well the author rejoinders: But I’m a writer, I’m not a promoter. I’m not on social media. I have no interest in actually communicating with my readers other than by my work and with occasional emails and ever-more-occasional appearances. I have a website (but, truth be told, I never update it). My work is my gift to readers. They can accept my work, or reject my work, but I will not be joining them in their living rooms.

Well, you can put away the scrolls and the ink made from the blood of ermines. The occupation of “author” now demands that you get out from behind your desk and meet your readers where they live, in their living rooms, and on their Facebook pages. You can lament it all you want, as we shut down your grants and your tenure and your publishing opportunities. This is the brave new world of writers and readers, sans publishers, those dreadful intermediaries who do so little to enhance your work, yet constantly interrupt your direct connection to readers, the people who value it most.

None of this precludes the ongoing commerce of great nonfiction books reaching an attuned audience. Au contraire. Those books can continue to shine, though the economics will be found to be impossible for all but the top selling writers. There will still be bestselling nonfiction. The current model doesn’t need to disappear for the new model to thrive.

I think of these great nonfiction books as “beautiful.” “Beautiful” as in beautifully-written, carefully edited, designed with care and deliberation, and printed, with deckled edges, on FSC-certified paper. They are sold in boutiques.

The boutiques are called “bookstores,” but they have all the characteristics of boutique retail —- lovely objects, chosen with care by the owner and the managers, a bit pricey, but for those who treasure such objects, worth the cost.

But the day-to-day work of most nonfiction authors is now to abandon the container, as well as the publisher as gatekeeper.

I think we’re on the verge of a significant transformation in publishing, and that’s exciting. Having traversed these dusty halls for decades, I know that change will not be rapid. But I’m still excited.

There is now a near-endless supply of material available to help inform you on AI, far too much, of course. Choose your preferred medium: books, blogs, newsletters, videos, podcasts, courses and mental divination. I’ve linked to some key sources through this book, it’s worth clicking a few random links to see where they take you.

There’s only one excellent AI book for beginners, Ethan Mollick’s Co-Intelligence, which was released at the beginning of April, 2024. It’s drawn from his equally excellent newsletter, “One Useful Thing.”

More resources to follow …

Disclosures

This book has several sponsors. I knew when writing this book that there would be few riches to be made, and I decided to underwrite my efforts by inviting sponsors to participate.

Sponsors imply a conflict of interest and confusion around advertising. If I was working for the New York Times it would be simple: “No.” Working for myself, disclosure is my weapon to satisfy these obligations: I will tell you what consideration I have received and you can judge, in my work, if I have been compromised.

I chose the sponsors that I invited to this project because I was familiar not only with their work but with the individuals involved in their organizations. They are colleagues and friends. I told them that their products might be discussed in the text of this book and they would have no control over those words. What they could control was their advertisem*nts at the end of the book —- I would post those as supplied. That’s what I’ve done.

I have received payments from others that may have influenced my work on this book —- I can catalog the following:

Please let me know if you detect any favoritism that you think may have resulted from these engagements. My bias in favor of AI was formed before the vast riches flowed.

Acknowledgements

I wouldn’t be able to talk with any credibility about AI if it wasn’t for the support of Publishers Weekly, including the editor emeritus, Jim Milliot, and the CEO, Cevin Bryerman. Jim supported my early writing about AI, and Cevin was instrumental to the success of last fall’s conference, AI and the Revolution in Book Publishing.

Several colleagues…

David Lamb and Susan Reich formed Book Advisors LLC in 2016 to consult on mergers and acquisitions in publishing. We have completed more than a dozen transactions, as well as a variety of financial, distribution, and operational consulting projects, including fair market valuations. We are pleased to have recently worked in association with Clarke & Esposito; Thad McIlroy of The Future of Publishing; and Kuo-Yu Liang of Ku Worldwide.

We pride ourselves on our relationships, diligence, and discretion. Whether you are interested in buying, selling, financing, or consulting, please get in touch for a confidential discussion.

bookadvisorsllc.com | bookadvisors@gmail.com

“Adopting Insight for our publishing house was like swapping a bicycle for a Ferrari. Suddenly, everything moves faster.”

Work Smarter, Achieve More

Developed by Veristage, Insight empowers publishers to thrive in the evolving AI landscape. Using AI and a robust suite of digital tools, Insight analyzes and understands your documents and books, in order to help you:

Unlike generic AI platforms, Insight is engineered to function within publishing workflows and to help you create publishing-specific information and assets.

How It Works

Security and Copyright

The Veristage team is committed to providing AI services that are secure and protect publishers’ intellectual property. Our platform has advanced information security systems in place, and we only work with AI models that do not store or use your data to train future models.

Solutions for Publishers of All Sizes

Insight is designed to work for publishers of all shapes and sizes, from consumer to academic, from three-person teams to multinational organizations. We offer competitive pricing and scalable solutions.

Talk to us to find out how Insight can benefit your business: