Said it first: pulling my own ideas out of four years of ChatGPT

For almost four years I've been thinking out loud in ChatGPT about pretty much everything, career moves, relationships, training, what I actually want out of my life, and at some point I realized that chat history was probably the most honest journal I had. The problem was that I never wrote it like a journal, so all of that thinking was sitting in there buried under homework and code and pasted articles where I couldn't actually use any of it. So I exported it, all 4,164 conversations, and the obvious move from there would have been to hand the whole thing to a model and ask it to tell me who I am. I didn't do that, and the reason why ended up being most of this project.

Most of it isn't me

I knew going in that the assistant wrote most of the archive (about 83% of the text), but what I didn't expect was how much of my own side wasn't mine either. Of the 701,883 words in my messages, 234,739 were pasted articles, class material, quiz answers, or those suggested follow-ups ChatGPT puts under a reply that you tap instead of typing, which means about a third of the words with my name on them came from somewhere else.

The echo was the part that really got me, because it's so much harder to see. ChatGPT would say something that sounded good, I'd reply "yes exactly" and say its sentence back to it, and now its idea is sitting inside my message looking like mine. It's kind of like being in class when the teacher explains something, and you raise your hand and repeat it back in your own words, and after a while everyone remembers it as your point, including you. If you ask a model to summarize me from an archive full of that, what you get back is a portrait that's half the model.

So I set one rule for the whole thing, which is that the idea has to be mine even if the sentence isn't. A model can draft a note for me as long as it's putting words to thinking I actually did, and every quote in that note has to trace back to something I really said.

What I built

The first layer is a set of scripts that go through every message I sent and tag it as typed by me, dictated by me, pasted, a quiz answer, a tapped suggestion, or an echo of the assistant, and then pull my side out on its own with the assistant deleted. That took the archive from about 6.5 million tokens down to about 1 million, which is small enough for a model to actually read and has a lot less ChatGPT in it for the model to confuse with me.

The second layer is Claude, where agents read my side in parallel chunks and draft notes in one shape: Was / Now / Because. I went with that shape because a note that just says "in 2023 I thought X" doesn't really tell me anything, and what I wanted was a record of what I used to think, what I think now, and what changed my mind in between.

The last layer is me. A script checks every quote before I ever see a draft, and nothing makes it into my notes until I've gone through it and approved it, one section at a time.

What broke

The failures are honestly the part I care about most, because they're where I actually learned something about working with these models.

On the first pass the agent reported that everything was fine, when it had actually put paraphrases in quotation marks twice and quoted a line ChatGPT came up with as something I said, and none of that showed up in its own summary. The only reason I know is that I had a script checking every quote against the original conversation, and the script didn't care what the agent said about its own work.

Then it turned out the checker had a hole in it too. It only compared the beginning of each quote, so something like "yes exactly, [ChatGPT's line]" would pass because the start of it was mine. When I fixed that and ran it again over notes I had already finished and signed off on, it found two quotes that were ChatGPT's words sitting in my notes as mine, and that bothered me more than anything else in the project. That's also where the name comes from, because the fixed checker basically asks who said it first: if I said it and ChatGPT repeated it back, it's mine, and if ChatGPT said it and I repeated it, it isn't.

Long messages fooled my first approach too. My instinct was to penalize them, since a long message is usually a paste, but that would have thrown out my voice memos, which are dictated transcripts of me thinking out loud and ended up being the best material in the whole archive. A pasted article and a voice memo look identical if all you're counting is words, the same way a textbook page and a page from your journal weigh the same on a scale, and what actually tells them apart is how I talk, how often I say "I" or "like" or "honestly," so that's what the scripts look at now.

There was also a bug that quietly ate 24% of my words, where a formatting line in the middle of a conversation cut off everything above it, and since nothing errored, two full passes had already run on the broken version before I caught it. Now every stage counts what goes in and what comes out and complains if the numbers don't match.

The last thing I ran into is that the archive goes stale the day you download it. At one point it showed a direction as something I'd dropped, when I had actually picked it back up weeks after the export, so instead of letting the agents guess from silence, they ask me what's true now.

What I took from it

The notes that made it through with ChatGPT's lines in them had gone through a careful, supervised process where the agent was explicitly told to check its quotes, and it still missed them. That's what made me realize that telling a model to do something and making sure it happens are two different things, so now the only way into my notes is a script that refuses anything that fails verification or can't show where it came from.

That's pretty much how I think about working with AI in general now. An instruction is closer to a suggestion the model will usually follow, while a check written in code runs every single time whether the model agrees or not, kind of like the difference between asking someone to lock the door on their way out and having a door that locks itself. So I use the model for what it's genuinely good at, which is reading and drafting, and I don't let it grade its own work.

The numbers

• 4,164 conversations over nearly four years; 1,628 with enough of me in them to matter
• 126 conversations read and decided on, over about two weeks
• My idea notes went from 73 to 120
• ~330 quotes checked word for word against the source

It's a repo now

I turned the process (not my data) into said-it-first so anyone can run it on their own export. It runs locally in plain Python with no dependencies, Claude Code does the reading, and the next time I export, it only shows me what's new since the last run instead of making me go through everything again.

I built it with Claude Code, where I made the calls on what mattered, what was broken, and what counted as mine, and Claude wrote most of the code and ran most of the tests. That's the same split the tool itself runs on, where the model does the work and the checks decide what I trust.