Issue #46

On today’s quest:

— Update: Running AI on your own computer
— NotebookLM podcasts can’t make you laugh (yet)
— ChatGPT as a replacement for Google Translate?
— Just interesting: More Nazca glyphs

Update: Running AI on your own computer

I tried running Llama 3.1 on my computer as a more climate-friendly way of using AI, as I described a couple of newsletters ago, and the 8B version brought my 2019 iMac to its knees. I easily got it installed, but when I submitted the first prompt, the response came out one word at a time — with a 20-second pause between each word and no way to abort.

I had skipped the step in the how-to video where you assess whether your computer can run the model. You shouldn’t.

Next, I installed a much smaller Llama3.2 model on my newer 2022 Macbook, and it’s working smoothly. I’m now running a private chatbot 100% on solar, baby! ☀️

Llama is slower than ChatGPT and the like, but not painfully so. All in all, it’s much, much faster than I expected based on how it nearly bricked my iMac, and it should be fine for low-level tasks that make up the majority of my use, like writing alt text for images and brainstorming about headlines and social media teasers.

On the other hand, it seemed quite bad at answering factual questions, not even seeming to know things that are in Wikipedia.

There’s a lot of customization you can do, including uploading your own documents to give it more context, so I’ll probably be playing with it for a while.

How To: If you’re interested in trying this yourself, use this how-to video, and I highly, highly recommend reading Christopher Penn’s most recent post about the pros and cons of these smaller models. I would have given up and also would have been disappointed in the results if I hadn’t read his post and understood what to expect.

Big Picture: Watching the bigger model heat up my iMac and grind it to a halt drove home for me how much energy these things actually use. And this was a relatively small “efficient” model. I never used LLMs much in a frivolous way before anyway, but now I’m even more resolved to be sure it’s worth it when I do use the big models like ChatGPT, Gemini, and Claude.

ChatGPT’s new audio-chat interface, which makes it feel more like you’re talking to a real person, has been on my mind a lot in this context too. For example, I’m inclined to say “thank you” to close out the conversation, and now I’m thinking about how much energy I’m using to thank a computer — and then how much energy it’s using to give me a “No problem” reply. Ugh.

But I also continue to think this is primarily a systemic problem given that companies are building AI into almost everything we use online. Yes, we can all individually turn off Google’s AI Overviews, but it would be vastly better if Google just stopped automatically including them.

• Technology Review on whether AI can solve climate change

• The Atlantic on AI water and energy use

• The New York Times on AI and energy

NotebookLM podcasts can’t make you laugh (yet)

Kevin Roose of The New York Times was as impressed as I was about NotebookLM and it’s podcast feature.* He and Casey Newton talked with Steven Johnson — one of the lead guys on the project from Google for the Hard Fork podcast. It turns out Johnson is a writer and has been on the project from the beginning, which is why it’s such an impressive tool for organizing research too. The NotebookLM section starts at 24:30.

One key takeaway to give podcasters hope is that Johnson said he’s listened to hundreds of examples of the podcasts and they are never funny; they never make him laugh.

ChatGPT as a replacement for Google Translate?

I haven’t tried voice ChatGPT for translation, but I thought some of you might find it useful.

I visited South America a few years apart and noticed a huge improvement in my ability to communicate the second time after Google Translate became available. It sounds like ChatGPT could be another leap.

Just interesting: More Nazca glyphs

A few years ago, on one of those trips to South American, I was lucky enough to see the mysterious Nazca lines in Peru, and now researchers have used AI to analyze arial photographs to identify new glyphs. The technique has allowed them to nearly double the number of known glyphs in the area.

Quick Hits

AI can solve 100% of CAPTCHAs. (Will our errors soon prove that we’re human?) — arXiv

Chatbots can be very persuasive. One chatbot was able to disabuse people of conspiracy theories. Presumably, it could be effective in the other direction too. — Science

Meta using celebrity voices for chatbots — The Verge

Meet the editor who turned himself into a news anchor — Big Technology newsletter (via Nancy Friedman)

Project analyzing human language by scraping the web has shut down because AI is polluting the data — 404 Media

An interesting profile of Duolingo’s CEO and the company’s successful moves into integrating AI into the product — Forbes

Zuckerberg suggests your Facebook feed will have AI-generated content — The Verge

What is AI sidequest?

Using AI isn’t my main job, and it probably isn’t yours either. I’m Mignon Fogarty, and Grammar Girl is my main gig, but I haven’t seen a technology this transformative since the development of the internet, and I want to learn about it. I bet you do too.

So here we are! Sidequesting together.

If you like the newsletter, please share it with a friend.

  ☀️ My house runs on solar.

^{* I feel like I’ve actually seen at least 20 articles and countless social media posts about this since I included it in the last newsletter. EVERYONE is freaking out/impressed with the NotebookLM podcasters.}

^{Written by a human.}