The Problem With Using ChatGPT for Accounting
I like to play around with AI. I have paid subscriptions for a few different LLMs and I’ve done every single training CPA Canada offers on the topic because I find it pretty interesting.
So, when I was doing my bookkeeping recently, I decided to see if ChatGPT could produce a correct journal entry for a bank account with about 30 transactions. FYI - I already knew what the correct journal entry was from my old fashioned Excel synoptic. Don’t judge me.. I love a good synoptic.
I thought this was a task that ChatGPT could probably carry out if (1) I told it the role to play (2) I was specific in my instructions in terms of what I needed it to do and deliver, and (3) I provided a .csv file (anonymized)..
Boy, was I wrong.
I ended up going back and forth with it for the better part of 10 mins because it kept producing an unbalanced journal entry. It should’ve debited the shareholder loan account, but because this account is “normally” credited (direct quote from ChatGPT), it didn’t seem to be able to put it on the debit side of things.
Even when I pointed out its error, it would apologize, agree with me, and then re-produce the same erroneous AJE 🫠.
Super frustrating, but it was clear to me that I couldn’t use this AI for simple bookkeeping.
Unfortunately, this seems to be the reality for a lot of people who try to use AI for financial stuff like ratios, bookkeeping questions, tax research, pricing analysis, strategy, etc.
As in, AI kinda sucks for accounting and financial analysis stuff.
A recent survey done by Dext found that 93% of CPAs have noticed financial and tax mistakes in the answers generated by AI tools. Gulp.
That’s consistent with my overall AI experience too. And honestly - usually it doesn't matter much because I usually already know the right answer. I just want another opinion.
But what happens when you don’t already know the right answer to your financial or tax q?
As in, let’s say you’re a busy business owner relying on ChatGPT to help you figure out the direct costs for a new service you’re offering.
It makes a calculation mistake and you end up underpricing the service by $600.
And let’s say 100 clients that buy this service in the first year - that’s $60,000 in lost revenue… hmm.
This is the kinda stuff that makes me cringe because it happens so very often.
I know AI will continue improving to the point that you will be able to trust it with stuff like pricing analysis calcs and recommendations, but it’s just not there yet so please, please don’t rely on it for important financial decisions.
This brings me to my main point (which, if AI wrote this, would’ve been much further up in this newsletter hahah… but here we go):
I don’t think anyone junior on your team should be using AI for technical work.
As someone with 15+ years of accounting experience, I know what an adjusting journal entry should be for a small bank account with very similar transactions, year over year.
But a baby CPA doesn’t.
And the risk of relying on an inaccurate AI output for financial info that influences decision making or how much tax I owe is just too high for me.
So, if you are using AI in your business - and I sure hope you are - please keep some things in mind when asking it financial questions:
It is super fast (and probably way more responsive that your normal CPA),
It always appears confident in its responses, and
It is often wrong in very a non-obvious way.
These three things make a dangerous combo. And if you “don't know what you don't know, ” how are you supposed to catch an error?
So, my strong opinion is:
AI only really works in supporting the technical side of your business when you already know enough to question its outputs aka you or your team members have at least 8-10 years of experience in their chosen field.
As you can probably tell, I’ve been thinking a lot about where AI actually fits (and doesn’t) in financial decision-making for small businesses - especially around pricing and profitability.
I’m putting together something more structured around this, but in the meantime, if you’re unsure about any numbers you’re relying on right now, feel free to send them my way. Happy to take a look.