Newest stuff

You wanted tips, so I have a tip!

Author

Mignon Fogarty
June 05, 2025

Issue #58

On today’s quest:
— ProWritingAid review
— Mary Meeker says AI is “unprecedented”
— People report massive time savings with AI
— Different AI models are really different
— A chat about AI for reditors
— AI is not just fancy autocomplete
— Court orders OpenAI to retain all chat records

Review: AI Editing Tools

Ariane Peveto, writing for Jane Friedman’s blog, tested four different editing tools on a piece of short fiction and has the pros and cons of each — Hemmingway Editor, ProWritingAid, Grammarly, and AutoCrit. It’s definitely worth a read.

I was intrigued enough to run a small section of my likely-never-to-be-finished novel through ProWritingAid just to see what it was like, and it made a couple of good suggestions to remove adverbs. I had someone’s eyes “immediately darting” to something, and yeah, “immediately” is unnecessary.

The more overarching critique was largely positive, but I suspect it’s just doing the sycophantic AI thing. I mean, maybe my “pacing is perfect,” but with the free version it’s only looking at the first 500 words, so I’m not sure how it would tell.

The negative feedback … excuse me, the “potential improvements” … were mostly OK. It suggested adding some details to “enhance the mystery,” which could be a good idea but also sounds very generic, and it also seemed to not quite get what a prologue is.

Overall, ProWritingAid was fun to play with, and maybe if I let it look at more than 500 words, it would be able to do more. I can see how getting instant feedback without the cringey feeling of showing your writing to another person or feeling like you’re imposing on someone’s time could be exciting, especially for authors without a strong support network.

I suspect the company will do well. But like Ariane, I didn’t come away thinking AI tools can replace human feedback and editing.

Mary Meeker says AI is ‘unprecedented’

Famed tech analyst Mary Meeker put out her first report in six years and used the term “unprecedented” 51 times, noting that AI adoption has happened faster than any previous technology. If you’re too young to remember Mary Meeker from the dotcom days, all I can say is her reports were a Very Big Deal.

The whole report is available, but these are the most interesting takeaways to me:

— AI is an international phenomenon. The U.S. makes up only ~9% of global ChatGPT use. And although ChatGPT is still the leader, China’s DeepSeek model is in third place globally, after Gemini, and coming on strong.

— Opinions differ significantly in different countries: 83% of respondents in China said AI products have more benefits than drawbacks, whereas only 39% of U.S. respondents felt the same way.

If you’re in the U.S., it’s good to remember that what you hear and see going on around you is only the tip of the iceberg. — AI Breakdown podcast, ZDNet

People report massive time savings with AI

In a recent study titled “The Labor Market Effects of Generative AI” people in the U.S. reported stunning gains in productivity by using AI. The entire report is interesting, but what also jumped out at me is that people making less than $35K per year reported the most gains, but generative AI use rose by income.

Different AI models are REALLY different

Now that I have a better computer (a Mac Mini with 24 GB of RAM), I can install and use open source AI models. Getting it working was a real pain that I can’t recommend, probably because the video I followed was out of date,1 but with extensive troubleshooting from ChatGPT, it’s now working fine, and I’m marveling at 1) how easy it is to try new models, 2) how different they are, 3) how much worse they are than commercial models.

Popping a new model into the system takes about a minute.

Llama 3 lied to me constantly. For example, it claimed to be evaluating images I uploaded when it doesn’t have that capability, and it repeatedly insisted it was working on a big prompt when it wasn’t. I deleted it.

DeepSeek-r1 (8 GB) is supposed to be one of the best open source models available, and it’s much more impressive — for a while. I haven’t caught it lying, but it has a limited context window, and if a prompt is too long or you talk to it for too many turns, it goes absolutely mad. One minute I’m getting an impressive critique of a document, and the next minute it’s spitting out mathematical formulas and Asian characters mixed with strangely poetic phrases like “the pears garden left.”  I’ve never seen anything like it.

At that point, I have to uninstall and reinstall it to return it to sanity. It only takes a few seconds, but all the context built up from the old chat is gone.

As long-time readers may remember, running an LLM privately and on my home computer so it’s using my solar power is a dream that didn’t pan out when I tried it eight months ago. I feel like I’m closer, but still not there. I can imagine using DeepSeek for small things, and I’m still hopeful I’ll be able to do more in the future.2

Using these crummy models reminded me just how much progress LLMs have made in the last year, and how people who haven’t tried them recently don’t have an accurate idea of what they can do.

Every day, I see writers on social media saying AI can’t do anything, is worthless, is a scam, and so on. And I get why they’d think so if they only tried a free model a year ago. And I get why most people don’t want to constantly test something — they tried it once and it sucked. Why waste time trying it again? But I worry that their resistance to the idea that AI can be useful is keeping them from understanding what’s coming.

A chat about AI for editors

I met Daniel Heuman in the early days of IntelligentEditing, his company that makes the PerfectIt editing software, and we’ve kept in touch over the years — and I’ve been following his AI editing product, Draftsmith, with interest.

In today’s Grammar Girl podcast, we talked about how Draftsmith actually works — on the front end and back end — plus:

— Why Daniel still finds AI writing unsatisfying
— What’s changed in the models since GPT-3.5
— How AI could help level the playing field for multilingual academics
— What all this might mean for energy use, editing jobs, and the future of the profession

Listen to the podcast on the go: Apple Podcasts, Spotify

AI both is and is not just fancy autocomplete

I enjoyed this 10-minute YouTube video explaining how it is that nobody understands how AI actually works.

It explains why AI isn’t just “fancy autocomplete,” although I understand why people use the term. When Llama 3 lied to me by replying to “Are you really still working on this project?” with “I’ll have it to you soon,” it was pretty hilarious because that’s exactly how a person would respond, isn’t it? So there’s probably a lot more of that in the training data than something like “You’ve exceeded my context window, and I can’t complete the task).

This error is actually a less dramatic example of a disturbing exchange in which ChatGPT lies, admits it lies, and keeps lying over and over again that was getting a lot of play on social media this week.

I wish I knew which version of ChatGPT this person was using: Free or paid? Reasoning or not? There is no single “ChatGPT” anymore, and I saw a post elsewhere (not related to this story) from an AI person saying it seems like ChatGPT o3 is getting worse and not accessing the web when it should.

Court orders OpenAI to retain all chat records

A court has ordered OpenAI to keep copies of all user chats, including those that users have deleted, temporary chats, and chats from paying customers — all of which the terms of service say the company will not save. The order came from the New York Times copyediting suit against OpenAI, and this is a developing story. For now, you might want to be wary about entering anything into ChatGPT that you want to remain private. — Ars Technica

Quick Hits

AI being better than humans

Societal problems

Pro-AI subreddit bans an ‘uptick’ in users who suffer AI delusions. A moderator said, “I would guess there are at least tens of thousands of users who are at this present time being convinced [they are demigods or AI is a demigod] by LLMs” — 404 Media

Teachers are not OK — 404 Media

Video

Everyone [in Hollywood] is using AI (and hiding it). “We can say, ‘Do it in anime, make it PG-13.’ Three hours later, I’ll have the movie.” — Vulture

Other

Which AIs do which tasks best, explained (with an interesting section on fiction writing) — Vox

BBC harnesses AI to create writing classes given by Agatha Christie — The Guardian

Future Knowledge — A new podcast from the Internet Archive and Authors Alliance. I listened to Episode 2 and heard about copyright, fair use, and national security in ways I haven’t heard about deeply before. It made me think courts are going to largely rule in favor of AI companies in copyright cases.

Addendum to the last newsletter: I referred to “prompt theory” as a fringe idea, but I’m pretty sure now that it’s just a joke. (Who can tell anymore?)

Start learning AI in 2025

Keeping up with AI is hard – we get it!

That’s why over 1M professionals read Superhuman AI to stay ahead.

  • Get daily AI news, tools, and tutorials

  • Learn new AI skills you can use at work in 3 mins a day

  • Become 10X more productive

  1. There has to be a better way, but I hate articles that don’t tell me how someone did something, so here we go: This is the video I followed. I then spent about an hour with ChatGPT 4o telling it all the error messages I got and getting advice about how to fix it.

  2. I’m starting to suspect the problem is my web wrapper — the interface that makes it look similar to ChatGPT — because it’s working much better when I just use in the terminal.

Written by a human (except the email subject line)