So I Did a Thing… Part 2
In part 1 I talked a bit about the day job and my day to day interaction with the big-corp LLMs on the market. Let me once again frame this with my stance on this whole subject. If I haven’t been clear so far, here’s it is:
All the evil being done in the name of AI is traceable back to these few companies.
The tech, the research, and even the training, was and is completely benign at worst. No one had a problem with training LLMs when they were for research, or translation, or as a technique to train other ML models. Everyone accepted scraping the web when it was just for search and archival. That is until these jerkwads framed LLMs as a panacea to replace the workers of the world and started this rat race to the bottom of the slop heap.
It is firmly the humans who are being unethical. Intent matters.
Even the “do no evil until it was inconvenient” Google didn’t want to go down this sloppery slope. They said as much, until OpenAI and Microsoft started threatening them. Now they’re stuck on the sloppy slip-and-slide to the bottom of the world’s environment and economy with the rest of us.
It’s with that stance, that everything I’m about to say should be framed. I see real uses for LLMs and their descendant model architectures. I see them as technology and as tools.
I also see a whole lot of friends and vital creative people being actively hurt by these oligarchic corporations and the narrative they push. They falsely and fraudulently assert that LLMs can and should be used to create work that belongs in human hands. The horror stories from the creative industry started within months of Nov 2022 and haven’t let up.
And I’d be instantly fired if I didn’t do business with them in the day job.
We live in a late-stage capitalist society. There is no ethical consumption here. All we can do is some utilitarian-esque harm reduction. Just like so many authors and creatives playing the game with Amazon and Youtube and Tiktock and so forth, feeding these algorithms, those datacenters, those oligarchies, while degrading our art into content. I gotta keep the lights on, the family fed, and the health insurance active. So, yeah, I use the big bad guys’ weaponized slop machines at work.
BUT ALSO, when I see things like this:
all I can feel is schadenfreude and a little bit of urgency to get a few more months of those Max plans while the loss leader is still leading, and that piper is still playing. Because whoo boy, the payment time is coming…
That brings me back to the titular “thing”.
After my experiments on the big 3 using my novel drafts as guinea pigs to demo just how far the so called “frontier” LLMs had to go before they could be useful, I started breaking down all the problems with the tech and the companies pushing them, and I formed a framework of ethical practice for myself and my creative work.
Once again:
- No writing text. The author is the only source of manuscript.
- No LLMs trained on stolen art. All the above testers were out.
- No running in hyperscaler datacenters.
- No grifting. i.e. no pitching to anyone who would pay a real (good) human editor
- No overselling. State plainly what a tool can be used for and what it can’t.
Where I did compromise was on the coding front. I needed to bootstrap something, and the proof of concept from work was locked in the form of a Claude Skill. I had to get it out of Claude and to a state where it could run on low wattage hardware and use an ethically sourced model. So 2.0 needed Claude Code to build a frame around the process. I needed Big AI to code itself out of a job for once.
So I had it do the coding busy work of building the UI and multi tier MVC app with solid safeties for any author using it. I’m no slouch at coding, but my strength in that world is in architecture and design. Turns out, that makes me a freakin good vibe coder. (And not just because the LLM says I am. Folks, the sycophancy problem in that space is REAL) You actually do need to know what you’re looking at to validate generated code. You do need to understand the full stack and security implications across all of it. Anyone who tells you otherwise is… once again… <say it with me> trying to sell you something.
So it took 6 months to build 2.0, validate the whole thing, test the open models, and get it the point I was looking for. I’m still not completely happy with it, but I learned a lot.
I built upon the above principles with a few more:
- Fully encrypted author-controlled data that even I can’t read.
- Runs on low wattage hardware, deployable anywhere.
- Flexible beyond my writing, beyond fiction novels, and beyond western plot bias.
- Never edit the author’s work
- Never collect data about the text or the author
Those last ones were very important to me. I’ve really been trying to hit all the points people object to when it comes to LLMs and show a product that contradicts all of them.
But Why? Why do you need to do this Harry? Are you another AI fanboy shilling with the rest of them? Why not just go down that road? There’s a nice handbasket for you to sit in…
Well, that’s a story:
A month or so after the initial revelation that this thing was possible, I started asking a few of my close author friends about it and its possibilities. The opinions out there about LLMs are bordering on pitchforks and torches. This was purely a curiosity―a bit of adversarial research to fight oligarchic AI with ethical AI. I asked a few if they had any trunked novels I could test it against to see if there was any there there.
One friend in particular was very interested. He was a little more friendly with ChatGPT, and he’d already been “collaborating”. What that amounted to was assisted brainstorming, pointed questions about craft, thinking of options for rewriting scenes, and letting OpenAI remember his work. He still insisted that every word in the book passed through his fingertips, but he definitely wasn’t throwing any Sabot at the datacenter.
This author offered up a book he was working on revising. It was one he was actively pursuing for publication, so I took it very seriously. His example helped me work some bugs out of an intake engine that parses the novel into its parts. That process is 99% LLM-free. You don’t need language inference to parse documents.
But when it read through and sent back a developmental edit report and story bibles, the dude was gobsmacked. The words, “Holy shit.” were uttered a few times.
Then something amazing happened. My report pointed out a little nugget in the plotline of his mystery novel. The answer to one of the main questions at the heart of his story was left unsaid. The plot had wrapped up, all the facts were known, and the crooks were caught, but no one in the book ever came out and just Said The Thing. At first he saw it, and thought, “Hmm, not a bad thought, I could write that in the epilogue. It would be just a sentence or three. Why not?”
So he did.
Let me take a step back and point this out: This is the type of thing I’m talking about. Not an LLM doing the writing, not an LLM telling someone what to write, just a mix of deterministic and LLM-based inferences pointing through a craft lens. This was the type of thing no one would have batted an eyelash at 4 years ago.
The next day, I had a grown man in my office, who sought me out at work, breaking down crying at the lines he had just written. My editor didn’t tell him what to write. It just pointed to a craft hole, a missing piece. The human did all the work. He just needed a nudge to find the beating heart of his whole book.
A book he thought was finished.
That was reason 3. If you missed Part 1, reason 2 was all the poor editorial advice I’d seen my friends get from some (human) grifty, borderline-predatory characters, some of which really hurt their will to write and carry on as authors. And reason 1 was the self interest. I’m one of those authors too.
So I built the thing…or tried to. That’ll be in Part 3.
Featured Image Datified Relations by Marcin Wilkowski / Licenced by CC-BY 4.0
