Training My Chatbot: Notes from a Frustrated Human

What it’s like to work with a chatbot that hallucinates 79% of the time

I recently started using ChatGPT-4o daily, and I’m overall disappointed. Since 2013, over $750 billion has been raised globally for AI. CEOs of Fortune 1000 companies are ranking AI as their top strategic priority. And yet, my ChatGPT believes 68–8 = 62 (true story).

When I ask it to edit my writing, it alters quotes and misattributes sources. Even when I insist on citation fidelity, errors persist. I repeatedly ask it: Why are you getting this repeatedly wrong? Am I vague? How can I better direct you? Should I be nicer? Should I say please? (My husband teases me for saying please and thank you to it.) Given the public hype, I assumed the issue was me.

Turns out I’m not the problem (or at least not entirely); it’s AI. Counterintuitively, newer chatbot models, called ‘reasoning systems’, hallucinate more, not less. ICYMI, hallucinations occur when the AI model produces inaccurate or misleading results, also known as the bot is wrong.

According to OpenAI benchmarks (PersonQA and SimpleQA), the model recently released as “o4‑mini” hallucinates approximately 48% of the time on public-figure questions (PersonQA test) and 79% of the time on general-knowledge queries (SimpleQA test). These rates are substantially higher than for earlier versions. Chatbot technicalities aside, I expected a finished tool, but instead, I got one that requires training.

In my career, I’ve trained dozens of paralegals, interns, and managers. That human connection was a key factor in their success. They saw my commitment to them and their growth, and in return, they were determined to learn and flourish.

How does one train a machine? Does my ChatGPT care about intellectual rigour? Aside from being a people pleaser, is the bot invested in our mutual success?

As a sustainability lawyer likely to remain in the workplace for another couple of decades, mastering AI isn’t optional. However, I now understand that I must not only focus on training my chatbot but also invest in AI governance, as ethics and intellectual rigour are works in progress.

I just asked it how I could train it better. It produced an eight-step “Guide to Training ChatGPT for Strategic Brilliance.”

… To be continued …