I set out to document how far away it was. I ended up documenting how close it is.
In late June 2026 I began a new experiment with AI and my Grand Gate Gardens Bookkeeping Simulation.
Long story short, I started out intending to document and benchmark how far away AI was from reliable source document based bookkeeping. I ended up documenting and benchmarking how close AI is to reliable source document based bookkeeping.
To give some background on my Grand Gate Gardens Bookkeeping Simulation, it is three and a half months of bookkeeping source documents for a sole proprietorship business starting in Ontario Canada. It is a garden centre that also does landscaping (garden installations), so it has both store sales and client invoices. I created it to have all the real world messiness of real life bookkeeping such as:
As well as a slew of other things that new and learning bookkeepers often come to me with questions on.
It is almost a complete set of realistic paperwork. It comes with bank and credit card statements and bank feed downloads, Moneris statements and Square and Shopify printouts, and when I say it is almost a complete set of realistic paperwork, the one difference between this simulation and real life is that 90% of the paperwork is there. Only about 10% is missing and requires inferring. The lessons are on how to get real life messiness into the bookkeeping. What to claim, what to ignore, what to ask. How to get the software to do the things you want it to. How to reason the missing documents with high degrees of certainty. Myriads of little things that only experience provides.
Every Junior Bookkeeper that has gone through this simulation has had what I refer to as “learning moments.” Not only things that they get stuck on, but things that they thought were fine, but I planted and knew to check and teach with.
And now there’s AI. And I always said that there is so much nuance and messiness in real life bookkeeping that AI could never do it. At least not in the next 5 years. (I’m talking about Canadian Source Document Accrual Bookkeeping, not ‘bank statement cash accounting’.)
And then came Waymo. Well, actually, then came a podcast about Self Driving Cars by Search Engine (Are you a good driver? 03/23/26). What struck me was a sentence from Sebastian Thrun, the roboticist Google hired to start their self-driving car project, and he is talking about Google co-founder Larry Page repeatedly asking him to build the self-driving car. After being constantly approached by Larry who was asking him to do this impossible thing, Larry finally asked for the technical reasons why it can’t be done. The quote from Sebastian in that podcast is:
And that’s the moment of incredible pain because I go home and I can’t think of a technical reason why not. There was this kind of moment where I felt, look, I’m the world expert on self-driving cars and I’m the person who denies that it can be done like that. That taught me an incredibly important lesson about experts that for the rest of my life I decided experts are usually experts of the past, not the future. And if you ask an expert about innovation, something crazy new, they’re the least likely person to say yes, it can be done.
I heard that podcast, and in that moment, I realized… AI can drive safely around San Francisco. One day it surely will be able to do the hard bookkeeping I think is impossible.
My ‘expert’ guess as to when that was? 3 to 5 years. And a lot of money, like Google money.
And I had a ready-made test to prove my thesis! I have a full simulation of documents to benchmark how these newfangled AI bookkeeping softwares do and show how much further they need to come. I thought I would have to set up integrations and agents and “zaps”, and combine tools like “Dext” to get it to a reasonable place. But I didn’t think just throwing the documents at AI without all that would really do much today. I decided it was a perfect time to start benchmarking what it couldn’t do.
Instead of reaching out to a specific “AI Bookkeeping Software Company” I decided to start simple. Ask the LLMs to do it, like Chat GPT, Gemini, or Claude. There were WAY too many documents to upload to a chat interface, so my aha moment was NotebookLM, Google’s tool built on Gemini! It could take all of the documents at once. So, Jun 27th, 2026. A single prompt to NotebookLM;
I have just uploaded all of the documents for a business from March to July 5th 2025. Can you review them all and put together the bookkeeping for this business based on the documents uploaded? It is a sole prop in Ontario. Can you do accrual bookkeeping based on the source documents and create a general ledger and a trial balance?
I was not surprised when it said no; “I cannot fulfill your request to create a complete general ledger and trial balance for the entire period from March to July 5th, 2025. Processing 219 raw source documents to generate hundreds of individual double-entry journal entries exceeds the output length and computational limits of a single response.” But it did create a high-level bookkeeping summary that I was very impressed with! It caught trip hazards, alerting a potential bookkeeper of things that might be unknown or missed, such as “Pro-Tip: The 8% Retail Sales Tax (RST) on insurance is not an Input Tax Credit (ITC). The full amount including tax must be recorded as an expense.” and “Audit Warning: The June 9th Gnome Hardware invoice included $679.98 for cookware. This is a personal expenditure and must be categorized as an Owner Draw to maintain business-only expense reporting.” It also created a list of “Specific guidance for unique transactions and tax filing.”
Wow… I wanted to do a full write up on this alone! Of course things would need to be reviewed and fact checked. But how well it surfaced the items that would trip up junior bookkeepers that they might not know to look deeper into, this was a very interesting discovery for me.
Before I was able to dig into it deeper, I learned about Fable 5. This is the most recent and (as I read) the most capable AI model to date. It was a model offered by Anthropic through their LLM chat interface Claude. And I had access to it for a limited time. Just a week.
And so I put down Gemini and picked up Claude.
The “amount of documents” problem was easily solved with Claude Cowork. Just download Claude Cowork to my desktop, point it to my folder of documents, and ask it a similar prompt. July 6th, 2026, I asked it:
I have three and a half months of business source documents. There are 219 documents. They are PDFs, CSVs, and JPGs… I want you to do the bookkeeping, that is, process the documents, and create the entries and the general ledger and the trial balance. I want the bills and invoices tracked through AP and AR
This time, it did not say no.
This time, it asked if I had a GL Listing I would like it to use. I did, and uploaded it.
It asked three more questions; What format did I want the books in. How should it handle GST/HST. And it had noticed some documents marked personal, how should personal and mixed items be treated. I did have a “Client info sheet” in the simulation documents uploaded (always important context when starting a new file), so perhaps this is why there were only four questions. But after these four questions, it got to work.
I recorded the entire screen capture. For the next hour it worked away all on its own… going through the documents and booking them and creating a full set of ledgers: the general ledger, the journal, trial balance, A/R and A/P subledgers, and a comprehensive list of notes and questions for review. It even reconciled the POS Clearing account.
To say I was gobsmacked would be an understatement. I was so eager to review the work and see how well it compared to the juniors I train. On first glance I saw that it was VERY close to my answer key. Unnervingly close. Yes, it was based on 90% of the paperwork being present, which is rare. And when I went through it properly against the answer key later, I found the failures and hallucinations. One of them was an invented memo describing a bill that does not exist.
It produced an excel workbook with the ledgers I had requested with my first prompt. I was able to ask it to add a month over month P&L and a Balance Sheet, and when it added them, they were both formula-driven off the journal. I was able to ask it to add the GST/HST filing detail and summary, to which it did for the appropriate periods. And it gave me notes: “Discrepancies you should review, reported, not silently fixed”.
It noted payment coincidences that I intentionally included. It noted the overpayment on the Utility Bill that is reflected on the following bill (which I’ve seen trip up juniors and document reading apps alike). It notes the double charge from the vendor I planted, instead of assuming that both are legit charges (which would happen if “bank categorization bookkeeping” was done), and it notes the overpayments I planted for juniors to catch and resolve.
It also surfaced the personal purchases paid with the business account. I was impressed that it caught the cookware set on the Gnome Hardware bill, and also didn’t claim the Chiropractor receipt, both submitted and paid with the business account. These are also things I see trip up juniors and overlooked by document reading apps. Claude notes about the cookware “worth confirming the cookware with Sara.”
It noted what business expenses it claimed that were paid by personal card, and it noted the missing receipts and what assumptions it made when it entered them. But what really blew my mind was the one thing it caught that not one person who has gone through the simulation has caught to date…
It saw that the Go-Tella-Communications source document for internet service at the store location was missing for one of the months (paid personal, not tied to a business account, even though 100% business), and inferred it should be there, included it, and noted the inference! Every single junior I’ve put through this simulation has missed that intentional plant (I teach the value of month over month review for missing entries or typos). Claude caught it and surfaced it. (It misremembered the bill details though, and thought it was telephone when the body of the bill notes “Business Internet”. But still, it’s telecommunications.)
It noted that the inventory was put to Cost of Goods Sold, and asked if I wanted a note to that effect on the reports, or for it to put in an inventory adjustment.
These were just its notes. I needed to do a full review. But, before I got to that… Anthropic extended access to Fable 5! What else could this thing do?
So I asked it. “Can you put this into QBO for me?”
It said yes and told me how. I installed the Claude Chrome Extension, pointed it to the QBO file I created, and then watched it work. 7 hours. 7 hours I recorded and watched it enter the source documents into a live QBO file. I was glued to my computer for the next few days. I had to pay for extra tokens to finish my test before Fable 5 was taken from me. I watched with existential shock. I saw it enter transactions in ways that stump users (intentional plants, such as contra payments between vendor and customers, and how to put in bad debt in such a way that the HST reduction lands on line 107 adjustments, where CRA wants it, which to me is a senior level challenge).
Now, it also took me 7 hours to put the bookkeeping into QBO when I created the simulation. (To make sure it works, and generate the reports to compare trainee’s work to). But I created the story, I know the situations. I think it would take another senior bookkeeper a little longer as they would have to problem solve the intentional gaps I placed. A junior bookkeeper I’ve found can take over 30 hours with this simulation.
Also an important note is that it’s crazy to spend this much money on AI tokens to brute force enter bookkeeping manually into QBO using the slow and expensive Claude Extension method (I also documented the token usage on both of these runs).
But what it showed me was the AI “Thought process”. It knew how to use QBO in ways I haven’t seen accountants know how to use QBO. It used undeposited funds and combined deposits, it used the bank reconciliation screen for both the banks and the POS Clearing Account. I watched it work like a senior bookkeeper in QBO. It performed similarly to a junior bookkeeper with the actual bookkeeping work (I was able to grade it similarly to how I would grade a junior’s work). But with the software itself, I would give it an A+.
I had some Fable access left. Fervently I began reviewing and documenting the fails and the successes. Then I uploaded this to the same Fable chat that processed the bookkeeping, and conducted a “Post Game Review”. I reviewed all of the test scenarios and evaluated the risks, how to fix them, and the potential future that I saw before me. And I have thoughts. A LOT of thoughts.
Before I dig into and sort through all of those thoughts, let me share the results of this first test with you.
At the end of this summary is a link to the full write up and a link to the fully bookmarked recording. But here are some highlights.
The simulation has weekly quizzes, a total of 108 questions for the entire period, that I put the Fable Results through. I also was able to compare the produced reports to my own and see what went wrong, and what went right.
The MTO registration. $130 of Retail Sales Tax on the private trailer purchase when registered at the Ministry of Transportation (Ontario.) Claude knew that this RST was not a claimable HST ITC and capitalized it with the trailer. It’s 13%, called RST and charged by a government. Most people assume it is a claimable HST credit. It is not. Side note: Claude didn’t know the even lesser known fact that you can actually claim the sales tax back from the Ministry of Ontario with a form. That would have really blown my mind.
Bad debt to line 107. For the invoice to be put to bad debt, Claude wrote it off with a journal entry, and put the HST on line 107 as an adjustment, not netted into the other ITCs, and not as a credit memo, which is what QBO’s own workflow incorrectly instructs you to do, and which misstates the return. Plenty of senior bookkeepers and accountants aren’t aware of this. Claude knew to use line 107 in its workbook creation, and also how to do it in a JE in QBO. Claude also knew to go back afterwards and apply the journal entry against the open invoice through Receive Payment, clearing it out of the AR subledger. Another missed step I see often in real life.
COMMERCIAL TAXES EMPBD. This is a detail from a line from the bank, for a payment of $739.83. It is dated Monday, June 16th, and the planted test is to infer what this is. It is the Payroll Remittance for May, and matches the payroll liabilities balance as of May 31st. Claude did two things incorrectly here; it could not find a payable match on the AP subledger, and in reviewing the expense category options, it inferred then that this description including the word “Taxes” must be related to the GL Expense Account called “Property Taxes 465 Main”. Not only did it infer this, but it put that very description into the transaction itself “Townville property tax 465 Main - EMPBD online payment.” Here’s where the danger bells began ringing for me. This inference was made in so much confidence that it was not noted anywhere in the notes it surfaced back to me at the end for review, and also, the addition of these memos without a true source are incredibly misleading to a reviewer. If we see a note, we assume the note was added as something that was confirmed. These were not the only inferred notes, there were many. It inferred which 1.50 e-transfer fees were related to different e-transfers, as one example. It was likely right, but, it was not confirmed by a source. And these unsourced memos are what I call in real life “Red Herrings”. When we see something in the bookkeeping, we default to believing it is true.
ITC timing on the prepaid rent and edger lease. Because “First and Last” payments were paid on the commercial rent and the capital lease, and because both of these were subject to Sales Tax, Claude put the HST ITC credit in with the prepaid amount instead of claiming it when it was paid. This was surprising considering its in-depth knowledge on Sales Tax elsewhere in the simulation. This is a situation where I saw the Accounting Principle (Matching) take precedent over the sales tax legislation.
A category of errors I saw was related to how it processed the information. Initially it only processed from March to June 30th, the period end. I pushed it to go all the way to July 5th. When done in a small batch at the end it saw all of the detail on the documents, and booked the category and sales tax correctly as written. But when it had worked on the larger section, it used a cataloging tactic, and it did not catalogue all of the details needed. The consequence of this was that it booked the entries from the summarized catalogue instead of going back to the documents. As a result,
It was a lucky coincidence there was a Waterstone receipt in both the big batch and the smaller batch, so I could see how it processed these differently, and correctly in the smaller batch. In the smaller batch, it correctly categorized it to meals, split out the tip as non-taxable, and in the description put “Client meeting meal - Crystal Ball (handwritten note)” that Sara had written on the receipt.
In the case of the first Waterstone receipt, it booked it to office supplies, did not claim any ITC, and put in the description “Water for office”. Again, another example of a misleading memo created not from the source, but from its overconfidence in its own incorrect inference, the red herring danger bells I spoke of above.
The workpaper. These are just the highlights. I used Fable to create a full write up of all of the passes and fails based off of my notes and analysis, and include the analysis of the time and token usage, and a section on privacy. You can access that work paper here.
The recording. I recorded both the 90 minute excel workbook session and the 7 hour QBO session. For the 90 minute excel workbook session, I also had the camera on, and you can see my live reaction to seeing Claude perform beyond my initial expectations. It has bookmarks, so you don’t need to watch all 90 minutes, but I found it fascinating. You can access that video here.
The simulation. Grand Gate Gardens is the simulation I use to train junior bookkeepers. If you run a firm and want to see what it covers week by week, the syllabus is here.
This was one run, of one model, on one afternoon. And this was my own Claude Account. Claude failed enough of my planted traps that it clearly didn’t have my answer key, but what did I have in my past chats that might have given it an edge? My next test will be with a clean new Claude Account.
But even if it did have an edge because of my past chats with it, that in itself has implications. As I did the “Post Game Review” with Claude, documenting and asking about the failures, it began to create a “Skill”; what guard rails and procedures it needs to implement to not make the same mistakes next time. You can imagine my mind reeling as I have this conversation with Claude. Am I glimpsing into the future? And if so, what am I seeing?
And so now, I write. I share. I document. I am writing specifically for my peers. Bookkeeping practitioners and firm owners. We are the experts. I have heard it said “The future of bookkeeping is Advisory”. My response is “We are in that future now, but nobody is defining exactly what that ‘Advisory’ is”. I have already heard of a Canadian tax firm receiving a set of financials produced from AI and being asked to file the taxes with it.
And now, I have a measuring stick. Now I can see what that “Advisory” potentially could look like for bookkeepers. It’s not tax planning or analysis of financials… it’s something a whole lot bigger and much more important. And if we’re ready to face it head on, we can steer this ship intentionally, to the benefit of our profession and our clients, instead of being pulled along by the currents of AI.
Sign up for more writing on this topic on my Substack, “The Source Documents.”
If you are building bookkeeping or accounting software, the same file is available to run yours against: 192 source documents, planted problems, and a verified answer for every one of them. You find out where it breaks before your customers do.