I am playing around with LLMs as beta readers for my novella.

It’s really fun. But they are way too nice.

I upload the entire text, give them a detailed persona and books they like that are like my story and then ask for good/bad/etc feedback on each part/act/chapter/etc.

The personas are coarse-grained. More “people who like these books and movies…” rather than a specific personal history.

I used thinking models, specifically I tried grok3+thinking and gpto1.

They say so many nice things (“oh wow, you really like it?”), it’s hard to focus on what they think needs work. I need to turn up the critical, and make the feedback more specific.

Thankfully, I can ask another chat instance about how to do that :)

There are many things to try.

The first thing is to get more feedback on things I know have a problem, based on a gut reaction from a fresh read-through.

I am beta reader #1, and I know exactly when something is not right or boring or confusing. I start skimming. Cut (or change) the boring.

Next, a good piece of advice is to ask about specific structural elements:

  • Plot Structure: “Does the plot have a clear beginning, middle, and end? Are there any plot holes or inconsistencies?”
  • Pacing: “Is the pacing too fast or too slow in certain sections? Where does the story drag or feel rushed?”
  • Character Development: “Are the characters well-developed and relatable? Do their motivations and actions make sense?”
  • Dialogue: “Is the dialogue natural and engaging? Does it reveal character traits and advance the plot?”
  • World-Building: “Is the setting vivid and immersive? Are there enough details to ground the reader in the world?”
  • Themes and Messages: “What themes does the story explore? Are they conveyed effectively?

Then to pull out specific weaknesses:

  • “What are the weakest parts of the story, and how can I make them stronger?”
  • “Are there any scenes that feel unnecessary or redundant?”
  • “Do any characters feel flat or underdeveloped? How can I give them more depth?”
  • “Are there any moments where the tension or conflict feels weak?”

Anything that is confusing must be straightened out:

  • “Are there any sentences or paragraphs that are confusing or hard to follow?”
  • “Does the story flow smoothly from one scene to the next?”
  • “Are there any repetitive phrases or overused words?”
  • “Can you suggest ways to make the prose more vivid or concise?”

Sniff out the boring:

  • “Are there any parts of the story that might confuse or frustrate readers?”

Genre tropes and such that I may have missed or messed-up:

  • “Does this story meet reader expectations for [insert genre, e.g., sci-fi, romance, mystery]?”
  • “Are there any tropes or clichés I should avoid or lean into?”
  • “How can I make the story more unique or fresh within its genre?”

Of course, the typical copy-edits and proofreading aspects:

  • “Are there any grammar, spelling, or punctuation errors in this passage?”
  • “Can you suggest more descriptive or evocative word choices for this scene?”
  • “Does the tone remain consistent throughout the story?”

And on.

There are a seemingly infinite number of things to explore and fix.

We’re only limited on knowing what to ask for, that is the key bottleneck.

Gut checks are a good start, but we have to go beyond and ask more sophisticated questions, questions that a good editor would ask.

I could do this all year long. I won’t, I have too many other cool things to explore, but one could. A perfectionists worst nightmare.

This is no substitute for real idiosyncratic human feedback. But good as a step before that.