AI Financial Modeling Test for Finance Pros to Understand the Limitations Using Real Tool Comparison
In this episode of The Mod Squad, Paul Barnhurst, Ian Schnoor, and Giles Male put today's leading AI tools to the test in a live financial modeling "bake-off." Using the same modeling case and prompts, they compare ChatGPT, Claude, and Copilot to see how well each tool builds a financial model. The discussion highlights the progress AI has made, where it still falls short, and why financial modeling expertise remains essential.
Expect to Learn:
How ChatGPT, Claude, and Copilot perform on the same financial modeling task
The strengths and weaknesses of AI-generated financial models
Why AI-generated models still require careful review and testing
How custom instructions and skills can improve AI outputs
Why financial modeling knowledge is still critical in an AI-driven world
Here are a few quotes from the episode:
"People are trying to do their jobs and keep up with AI at the same time. It's exhausting." – Ian Schnoor
"AI is a magnifier, not a replacement for financial modeling skills." – Paul Barnhurst
AI is powerful, but it is not a shortcut to quality work. It still needs guidance, structure, and strong fundamentals. The people who benefit most are those who understand both the tools and the work behind them. For now, the best approach is simple: test it, guide it, and never trust it blindly.
Follow Ian:
LinkedIn - https://www.linkedin.com/in/ianschnoor/
Follow Giles:
LinkedIn - https://www.linkedin.com/in/giles-male-30643b15/
In today’s episode:
[00:00] – Trailer
[03:30] – AI Fatigue & Keeping Up
[05:48] – The Modeling Challenge
[08:41] – ChatGPT Results
[17:57] – Claude Results
[27:33] – Copilot Results
[30:36] – Human Review Still Matters
[33:18] – How AI Has Improved
[36:10] – Better Results with Custom Skills
[37:08] – Final Verdict & Key Takeaways
Full Show Transcript
Intro (00:05):
The Mod Squad. We're the Mod Squad. The Mod Squad featuring Ian Schnoor, executive director of Financial modeling Institute, Giles Male, humble MVP and co-founder of Full Stack modeler and Paul Barnhurst, the FP&A guy.
Host: Paul Barnhurst (00:30):
We're having a bake off today. Giles Ian and I are all going to run the case and then we're going to come back at the end and score 'em on the structure. How did they look aesthetically? Were there any critical errors and all kinds of things?
Co-Host 1: Giles Male (00:42):
I'll be leaning into Atropic. I've been a big clawed fan for a long time. I'll be using 4.8
Host: Paul Barnhurst (00:48):
And I'll be doing chat GPT and they don't give you the model in Excel. Which
Co-Host 2: Ian Schnoor (00:54):
Model do I use? I ran a webinar and I said, how many of you here are finding yourself running one model and then using a second one to check the first one to kind of validate?
Host: Paul Barnhurst (01:01):
Go ahead and build the model
Co-Host 2: Ian Schnoor (01:02):
In five minutes, get fired and
Host: Paul Barnhurst (01:03):
Hit. Welcome to another episode of Financial Modeler's Corner. This week we have the mod squad with us, so I'm thrilled to be joined once again by Giles, Male, and Giles. How you been?
Co-Host 1: Giles Male (01:19):
Good. I feel like I'm in a beard competition with you now. I've got a little bit of a way to go, but be scared.
Host: Paul Barnhurst (01:25):
Is your goal how long you can get it? Is that the plan?
Co-Host 1: Giles Male (01:28):
I've hit this phase where it doesn't actually feel like it's growing. I just feel like I look like I'm homeless, but its uncontrollable at times.
Host: Paul Barnhurst (01:37):
Yeah, and how have you been?
Co-Host 2: Ian Schnoor (01:38):
Great. Great. Missing you guys. But I did get a chance to see and catch up with Giles recently at the Global Excel summit in London exactly a month ago, and that was a lot of fun and a lot of great learning and nice to catch up with the global Excel and modeling communities. Life is busy. How could it not be? I mean everybody that I know who's working in business accounting and finance right now is still kind of inundated, overwhelmed, tired, with excited with this world of ai. I mean it's all of the above. We were trying to do our jobs, run our businesses, run our organisations, manage our lives and learn and keep up with, right? It'd be one thing if we just had to learn ai, but what's not about learning ai, we have to keep up with it because changing so fast, it's like whatever skill you learn in Giles, you're probably the best talk with whatever you skill you learn last month. Well now there's a whole set of new things and so now you're trying to keep up and figure out what you should invest your time in and learning. It's exhausting. No js.
Co-Host 1: Giles Male (02:37):
Yeah, I agree. Well, we've talked about it before. I'm kind of almost a fully fledged copilot trainer now and I think it'll extend to Claude. I know Paul, you're doing stuff as well and just trying to keep up with what changed in the last seven days. I mean there were huge announcements from Microsoft two days ago that changed a whole bunch of stuff. So yeah, it is a nonstop battle just to stay broadly up to date. It's crazy
Co-Host 2: Ian Schnoor (03:03):
And it's a different level of exhaustion, isn't it? Sort of. It's a,
Co-Host 1: Giles Male (03:07):
Yeah. What was the term you used? Maybe it was exhaustion, like AI fatigue or something. Fatigue.
Co-Host 2: Ian Schnoor (03:13):
Yeah, fatigue. That's right. It's like a fatigue your feel, right?
Co-Host 1: Giles Male (03:16):
Yeah. And again, I know we are repeating old ground. I wasn't over the Excel fatigue. RegX has been on my list for years and now I've just got to park all of that and become an AI guru.
Host: Paul Barnhurst (03:30):
Agreed. So we'll continuous discussion around AI a little bit more in some different areas, but in the meantime what we want to do is we're having a bake off today, so we're really excited about it. What we're going to do, and then I'll turn it over in to explain the case and we'll kick it off is we've selected a case from FMI Giles in and I are all going to run the case will HCR what tool we're going to use in a minute and then we're going to come back at the end and then we're going to look at how they did and we're going to score 'em on some things like how good was the structure, how did they look aesthetically, were there any critical errors where it's like this just fell because the balance sheet doesn't balance and all kinds of things. So that's the basic idea. So what I'm going to do is I'm going to turn it over to N but before we do that, we're going to have each person say what tool they're going to be testing today. So Giles, what are you testing with?
Co-Host 1: Giles Male (04:18):
I'll be leaning into philanthropic. So I've been a big clawed fan for a long time. I'll be using 4.8. I really wanted to be able to join you today and use Fable, but for well-known reasons. Thank you. US government, I can't. You're welcome. Yeah, I can't
Host: Paul Barnhurst (04:35):
Because I'm the only US citizen here, so I'll take the blame. The other thing I'll say is you train people on copilot, but you're doing Claude. Okay, we'll leave that for later. What fool are you testing?
Co-Host 2: Ian Schnoor (04:47):
I think we talked about it. I'm going to use copilot, but I'm going to use the, did we say the, oh no, Giles, you said that you want me to use the cloud four
Host: Paul Barnhurst (04:56):
Eight right?
Co-Host 2: Ian Schnoor (04:56):
Quad four eight Within copilot though, because we expect that it's going to generate a different model. We're going to look at the Henderson model again to keep it consistent with what we've been testing over the last six months, but we're going to try three current modern tools. So Giles, you are going to use which entropic tool are you going to use?
Co-Host 1: Giles Male (05:13):
So I'll be in the Claude app in Excel and I'll use Opus 4.8 as the model. That'll be that one. And if you can do the same through the app, I think that was what we said, wasn't it? So you're going to use copilot in Excel. Choose 4.8. Yeah, perfect.
Host: Paul Barnhurst (05:28):
Yes,
Co-Host 2: Ian Schnoor (05:29):
And I'll do it that way
Host: Paul Barnhurst (05:30):
And I'll be doing chat GPT. And they don't give you the model in Excel, they just allow you to select do you want it to do fast standard or heavy reasoning? So I'll be doing heavy. Well why don't you take us through the case here.
Co-Host 2: Ian Schnoor (05:43):
Do you want to share the screen? Then
Host: Paul Barnhurst (05:44):
We'll all kick 'em up. So I'll bring it up on the screen and you can walk us through this thing, right?
Co-Host 2: Ian Schnoor (05:48):
So again, we're keeping it consistent. Anyone who's watched it before, we're going to use the Henderson manufacturing company model again, and this is what we all have. We all have the same file, we're all have this exact same file and we're all going to give it the exact same prompt. So in our Excel we have a sheet called model, and on this sheet we simply have the last three years of historical financial statements. We have the income statement, we have the cashflow statement, and then we have the balance sheet. That's it. That's all that we have on this sheet. And then the rest is blank. And then what I did is there's a case study or is the case study. Here's the case study. The case study is what an exam candidate would've been given and it's a two page PDF case study that talks about the company.
(06:27):
It talks about their history, it provides information about the company's sales, about their operating costs around their fixed assets and depreciation, their working capital, their income tax, their debt, their equity. And what I did is I took this case study and I copied it into a separate sheet. I just find it works a little easier. Typically I copied the case study historic into one sheet as text. It's not nice formatted, it's not numbers, it's just gobbly, ook strings of text in cells and then there are instructions at the bottom just like this. I just copied the full case and then what we're all going to do, and I'll show you and I think I was told I am going to be testing right here. This is where you want me to go, right Giles into Opus 4.8. So I am going to run opus 4.8 you can see and literally I'm going to put in this prompt and we all have the same prompt.
(07:21):
It says on the case tab you have been provided with information about a company called Henderson. You have also been provided with three years of historicals on the model tab, build a five-year forecast model with all the required schedules, revenues, cost, depreciation, tax, working capital, debt equity, and there are instructions in sales B 60 to B 66 here. And so there are some, so we'll see how well it incorporates these additional instructions over the last month or two. It's done a pretty good job building scenarios, adding an assumption page, et cetera. That's what we're all going to do. So I'll stop my share now, but we're going to do that and then at the end of the episode we will see how each of these tools performed, right Paul?
Host: Paul Barnhurst (08:00):
Yep. So what's going to happen is we're going to pause here for a minute. We're all going to get it kicked off, answer any questions our AI asks us, let it start running. We'll come back and have the conversation. So we'll be back here in a minute, but we're going to go ahead and pause it for a second while we do that. So you can see right now, and this is something I actually did with co-pilot's, new personalization. One of the instructions I put it is please auto fit the width of each column when you're done because it does this all the time. I don't know if you guys have noticed this but right, so tiny thing, let me just expand it out.
Co-Host 2: Ian Schnoor (08:33):
That's a beauty. I encourage you to hand it into your boss that way with all those
Host: Paul Barnhurst (08:38):
You could see it said, but it did mention it. Fix the print ranges
Co-Host 2: Ian Schnoor (08:41):
Nice.
Intro (08:41):
So
Co-Host 2: Ian Schnoor (08:41):
The
Host: Paul Barnhurst (08:42):
Folks will print. You can't read it but it will print.
Co-Host 2: Ian Schnoor (08:44):
So first of all, some decent points for formatting. Not bad. I mean it's made an effort, right? To try and do something presentable.
Host: Paul Barnhurst (08:54):
Let me make it not quite that big. Let's go there. So how they laid it out, we can see 26, 27, 28. It did a summary. It didn't do it on the model sheet. I would've liked to see, well it did bring it in here as well. It looks like very
Co-Host 2: Ian Schnoor (09:11):
Interesting that it did. It kind of collapsed the columns and made them
Host: Paul Barnhurst (09:15):
So it looks like interesting enough it did it here. This is its summary. Okay, so let's go. I don't like the order. It's model assumption summary.
Co-Host 2: Ian Schnoor (09:27):
Not at
Host: Paul Barnhurst (09:27):
All.
Co-Host 2: Ian Schnoor (09:28):
It's
Host: Paul Barnhurst (09:28):
Summary assumptions model.
Co-Host 2: Ian Schnoor (09:29):
Exactly. It went opposite for some reason. Okay, so
Host: Paul Barnhurst (09:32):
I'm just going to switch that around real quick just as we read through this and I'll put the case at the back. Alright, so let's go ahead. It provided a summary clear enough, it did. I would say decent EBITDA margin, ebit, NetComm ending cash, total debt net capacity utilisation. Let's
Co-Host 2: Ian Schnoor (09:52):
See, can you click on one of the best case numbers? Go down and click on some of the best. I wonder what it did click.
Host: Paul Barnhurst (09:57):
It looks like all the numbers are coming from below. So it put a summary up here.
Co-Host 2: Ian Schnoor (10:02):
Well I'm very curious to know how it's linking to.
Host: Paul Barnhurst (10:05):
That's
Co-Host 2: Ian Schnoor (10:05):
What
Host: Paul Barnhurst (10:05):
I'm trying to get to.
Co-Host 2: Ian Schnoor (10:06):
Based the best and a worst at the same. It was in row age, so
Host: Paul Barnhurst (10:09):
Let me take one. Let's just,
Co-Host 2: Ian Schnoor (10:11):
Oh, 41 and then 81 B 41 control
Host: Paul Barnhurst (10:13):
Bracket. Give me a second. Alright, so net revenue here. So did the what? That doesn't make any sense. Am I? Okay, so hold on. B 41, let's go to B 41 and yeah, so it did a calculation here for net revenue B 38. So units sold times gross price minus freight divided by a thousand.
Co-Host 2: Ian Schnoor (10:40):
Yeah, you don't really want to see any calculations like this on a summary sheet. Correct.
Host: Paul Barnhurst (10:44):
Then this one it looks like where's
Co-Host 2: Ian Schnoor (10:49):
That coming from? What
Host: Paul Barnhurst (10:49):
You did here is it did the calculate, this says capacity utilisation but it's where it did the calculations for the base case. Then it did the calculations for the best case here.
Co-Host 2: Ian Schnoor (11:00):
But
Host: Paul Barnhurst (11:00):
It did 'em off the assumption because
Co-Host 2: Ian Schnoor (11:02):
Basically it's basically built three separate models. It didn't know how to run a base best and worst off one model with scenario management or sensitivity tables or switches. It basically reran all the calculations. Three different ways to get a
Host: Paul Barnhurst (11:16):
Base best, correct. That's what it looks like it did here but it linked into the model. Okay,
Co-Host 1: Giles Male (11:21):
So it's given you a switch on the model has it?
Host: Paul Barnhurst (11:24):
No, no, no. Not a switch. What I'm talking, so this beginning revolver links to the model. The model doesn't have switches. It's decided all three of them the bank debt. Well that makes sense. The bank debt revolver from 25. Your beginning is always going to be the same because it's an
Co-Host 1: Giles Male (11:39):
Actual Yeah, but what's it done on the SA terrible
Host: Paul Barnhurst (11:41):
Way to try to trace it out. But
Co-Host 1: Giles Male (11:43):
What's it done on the forecast? How is it picking one of the three models with the three outputs?
Host: Paul Barnhurst (11:48):
It just linked it to the schedules. It's built on this page. So it did the work on this page of one model then it recreated three for each of the cases. Are they consistent? Not sure yet.
Co-Host 1: Giles Male (12:02):
Okay.
Host: Paul Barnhurst (12:03):
Your look
Co-Host 2: Ian Schnoor (12:04):
Done.
Host: Paul Barnhurst (12:05):
A
Co-Host 2: Ian Schnoor (12:05):
Decent job on the formatting I guess presentable,
Host: Paul Barnhurst (12:07):
Right? So if we go through this, let's just run through its main model and kind of flow. Okay. Do
Co-Host 1: Giles Male (12:12):
You mind zooming in a little bit just for my eyes
Host: Paul Barnhurst (12:15):
Movement? Yeah, I'll zoom in a little bit. How's that?
Co-Host 1: Giles Male (12:18):
Yeah, great.
Host: Paul Barnhurst (12:19):
Okay, so what we have is we got our income statement that's all linked below. That all seems to make sense. I'm not going to go through numbers, right? I just want to see it all appears to be linked.
Co-Host 1: Giles Male (12:31):
Oh you've got some interesting current tax numbers going on there.
Host: Paul Barnhurst (12:35):
Yeah, we'll look at the schedules. I think it did this right in the sense of you got your cashflow statement, it's all linked below. You have your balance sheet, it's all linked below. It's
Co-Host 2: Ian Schnoor (12:47):
Using the wrong colour coding but it's okay.
Host: Paul Barnhurst (12:50):
Balance sheet doesn't balance. Oh it does. Nevermind. I thought it didn't for a minute. I was like okay, so it balances. It does, it does just
Co-Host 1: Giles Male (13:01):
Go up
Host: Paul Barnhurst (13:01):
A little bit and then what It did
Co-Host 1: Giles Male (13:02):
Much going up a little bit just to the cache. What was the C line?
Host: Paul Barnhurst (13:05):
Yeah, give me one. Go up to that first you want to see cash here.
Co-Host 1: Giles Male (13:09):
It's always the first thing that I used to note to is sometimes you had numbers on cash and the revolver and god knows what else.
Host: Paul Barnhurst (13:16):
So the cash goes to zero and stays there, which is a problem for all five years
Co-Host 1: Giles Male (13:20):
Leaning on the revolve. That's the check that I wanted to see. So you never go cash positive, you don't have a minimum balance.
Host: Paul Barnhurst (13:27):
Yeah, I mean you should be paying it off but yeah, get it. So what it's done, which I do is this is one of the few times I've seen where it did not do any calculations on these schedules. Now it did create three calculation details for each scenario, which is bad.
Co-Host 2: Ian Schnoor (13:43):
Sorry, it did not build,
Host: Paul Barnhurst (13:46):
I don't see any calculations taking place outside of
Co-Host 1: Giles Male (13:50):
Yeah, the structural. Yeah, on the
Co-Host 2: Ian Schnoor (13:52):
Three
Host: Paul Barnhurst (13:52):
Statements it builds.
Co-Host 2: Ian Schnoor (13:54):
Where are the schedule? That's good, the schedules are down below.
Host: Paul Barnhurst (13:56):
Now we're into the schedule. So what I was going to do is I'm just going to hide these columns so I can bring this over a little bit and kind of freeze the pain maybe here just as we're moving. Alright, so that's done. Alright, so went off the assumptions and did the math there for units sold, which I think there was a capacity thing. So that seems to look roughly right, the revenue, okay, it goes down. I think that looks freight and warehousing. I mean as I look at this first part goes up to a hundred percent capacity utilisation and stops
Co-Host 2: Ian Schnoor (14:35):
Itself. It feels right. That's what I would've expected for this case. So
Host: Paul Barnhurst (14:38):
I think the revenue schedule just a high level without going through every row looks fine, right? I don't see has any real issues. Now let's go to the cost schedule.
Co-Host 2: Ian Schnoor (14:47):
It's doing a
Host: Paul Barnhurst (14:48):
Nice job, they raw materials, what did it do here? Alright, just assumptions doing a nice job. It seems reasonable at a high level. I mean costs are going up as revenue. Yeah,
Co-Host 1: Giles Male (15:02):
I think it's fine. You've got a single cell inflation assumption somewhere. I think it's just linking everything to Yeah, yeah,
Host: Paul Barnhurst (15:08):
We can look at, so if we look at the assumptions, let's just take a minute. So they did the assumptions for base best and worst they did gross price multiplier, volume growth, annual growth thereafter, I'm not sure. Okay, they did 2026 and then every year after inflation they made assumptions for each case. They made assumptions for D-S-O-D-I-O and DPO. That's a reasonable set of assumptions to vary. So that all seems fine. Annual case inputs, these were the inputs that were given and I liked that it gives you the notes. Case B 11 gives me the cells that it took it from so I can validate it. So I like the assumption page there. Same with the operating. It says hey I took 'em all from there. So it tells me what's the note assumption based on run rate and it goes through and gives all its assumptions. This page
Co-Host 1: Giles Male (16:04):
That looks fine or you got, I mean you could antic and say it's got calculations on the inputs but
Co-Host 2: Ian Schnoor (16:10):
Yeah, I mean I don't love that but historical driver
Host: Paul Barnhurst (16:15):
Calculation that wouldn't
Co-Host 1: Giles Male (16:16):
Have okay,
Co-Host 2: Ian Schnoor (16:17):
Pulls that right under the case. The 5 94 was in the case study, it just hard kit it in. That's not very good. Yeah,
Host: Paul Barnhurst (16:23):
So that's what we have. Anything else? You guys there you can see that. Do
Co-Host 1: Giles Male (16:28):
You know what? I wonder
Host: Paul Barnhurst (16:29):
Why it's doing an index. Oh it's just pulling it from above
Co-Host 1: Giles Male (16:32):
For these one It looks pretty okay. I mean probably a bit of an improvement on a lot of, well actually a lot of improvement on what we saw last year. Still some of the same minor issues I guess some of the formatting, some of the choices to hardcode numbers pulled in different directions, not perfect, but I mean you've got a balancing balance sheet and the numbers looked sensible
Host: Paul Barnhurst (16:58):
So any other, just as I was scrolling down, how did they do their depreciation?
Co-Host 1: Giles Male (17:04):
Oh so that's interesting. They've got a semi fixx range. I mean that's a reasonably intermediate modeling technique to semi fixx a range to do depreciation over time
Host: Paul Barnhurst (17:16):
Pretty. I would like to see a but it's not bad.
Co-Host 2: Ian Schnoor (17:19):
What's the new asset depreciation?
Host: Paul Barnhurst (17:21):
All they did is they basically did a sum of the two years and divided it by an assumption. Yeah
Co-Host 1: Giles Male (17:27):
But look at the next column over go to the next column over to the, right now it's
Co-Host 2: Ian Schnoor (17:32):
Years of CapEx. Yeah,
Co-Host 1: Giles Male (17:34):
Go over to the right and then hit F two. So that's what I'm saying that's impressive that it can do a modeling technique with a semi
Host: Paul Barnhurst (17:41):
You're saying saying did it as a rolling each year
Co-Host 2: Ian Schnoor (17:45):
It's very subtle technique to lock the first cell reference and not the second one so you can talk. Yeah, it was
Host: Paul Barnhurst (17:49):
Smart enough. So yeah
Co-Host 2: Ian Schnoor (17:51):
That's good. Alright, great start. Do you want to,
Co-Host 1: Giles Male (17:57):
I can go next. I think Paul you need to,
Host: Paul Barnhurst (18:00):
I stop sharing and share you, which I believe is this one here. I
Co-Host 1: Giles Male (18:04):
Need to bear with me. I need to share my
Host: Paul Barnhurst (18:09):
Real quick. Well you're going to share overall if you were handed this, I mean we haven't gone through it in detail, but just initial thoughts, kind of final thoughts. I think for me a couple things I really liked it at least did individual schedules. That's an improvement over a lot of what we saw last year. I hate the way it did scenarios of detailed case calculations and linking them so there's linking issues. I would've done a little more detail on the depreciation. So there's some messy things but overall it's pretty good right in the hand of the right modeler you could take this and finish it up.
Co-Host 2: Ian Schnoor (18:44):
I'm not surprised to see this. It's done an amazing job in 20 minutes or 15 minutes and yet I beg people not to ever hand that in to your client or your boss and people will, it is not going to be a good day when you hand in something like that, it's off to a great start and now you'll probably spend an hour or two hours learning it, understanding what it did so you can answer the questions about what's going on, how is it built? You're going to find issues you want to improve formatting issues like you will spend time and you can either iterate with the tool to have the tool do it or you'll just do it yourself. But you need to know what to look for and you need to know where there are challenges. So yeah, I mean I'm not surprised it looks and
Host: Paul Barnhurst (19:27):
One last thing I do like that it has a check section. There's a few more I could add but it has a cash roll forward check and a debt schedule tie.
Co-Host 2: Ian Schnoor (19:34):
Click on one of the balance sheet checks in the third year let's say. Is it? Yeah. Great. Okay Charles were you
Co-Host 1: Giles Male (19:43):
In all of their defences in terms of formatting, we're not throwing full capabilities at this because you do have custom instructions and skills files and all of this stuff that you can add. So we could, I mean 20 minutes like you say, we
Host: Paul Barnhurst (20:00):
Could improve this quite a bit with skills.
Co-Host 1: Giles Male (20:02):
Do you know what? That would be a really interesting next episode we go away, we do the same thing but we put everything on it. We get our custom kind of skills and instructions in there and better prompting. That'd be interesting.
Host: Paul Barnhurst (20:17):
We should do that at some point. I still got to build out those skills.
Co-Host 2: Ian Schnoor (20:21):
So that takes a lot more learning, right? Correct. It's one thing to know how to just put in a pretty basic prompt like we did. It's a whole other skillset to know how to go in and build skills and have a library and a now you're, anyway, it's great. Giles, how did yours do an
Host: Paul Barnhurst (20:37):
Example for I know someone to build 81 skills for a project they're doing and others built 19. That's a lot of time. Alright, we're going to turn it over to Giles now.
Co-Host 1: Giles Male (20:45):
So I haven't touched it, I haven't changed the order of anything so we will just, I'll tell you why don't we start So I have a
Host: Paul Barnhurst (20:52):
Summary, you got summary last as well.
Co-Host 1: Giles Male (20:54):
I got summary last. I actually think the order is okay because it's basically you enlarge it a
Host: Paul Barnhurst (20:58):
Little bit for my old eyes.
Co-Host 1: Giles Male (21:00):
It's treated the original input tab as the output essentially. So actually you do have assumptions, schedules, output and summary. I think that's fine. It's given me a nice little summary, five year kind of cumulative numbers and then the final year, which is fine just at a high level. So this is my five year forecast here formatted nicely I think. I mean let me just go to, sorry about that. I just wanted to see if you look here on the right all formula driven to the,
Host: Paul Barnhurst (21:39):
Although it used a left formula in the bottle,
Co-Host 2: Ian Schnoor (21:43):
That seems a little
Co-Host 1: Giles Male (21:44):
Bit
Co-Host 2: Ian Schnoor (21:44):
Where did you see a lead formula
Co-Host 1: Giles Male (21:47):
But you know what might, that's for the cache. So if you were going where did do that though? I'm curious. Where's probably here?
Co-Host 2: Ian Schnoor (21:54):
Oh my gosh. Holy. That's what it did on your, wow, that's what it did on your this just, wow.
Host: Paul Barnhurst (22:03):
Honestly that's not all that bad if you understand it.
Co-Host 2: Ian Schnoor (22:06):
No. Well it just completely proves and makes my point every single time. This is not how you want to build the revolver section on. There's so many issues with this. First of all, that should not be right on the financing section of a cashflow statement. And second of all really are you going to use a let function, one of the newer, more complex functions in Excel when your client or boss might just want to see a simple step-by-step add up of what you're doing? This is going to be hard to audit and understand I'm not liking it even though it's probably right. Jas, are you going to say something?
Co-Host 1: Giles Male (22:40):
Well I say it's really interesting, obviously I agree this is not how I model this at all. What we saw almost consistently last year was that a lot of that final cash balance or revolver balance stuff was solved in the financial statements. So you could argue if you're going to go down that route, I mean at least ette is trying to make the formula a bit more readable. But yeah, I mean I wouldn't do this.
Host: Paul Barnhurst (23:04):
Wait, it has a note there. What does the note say? Put a note on that cell. You
Co-Host 1: Giles Male (23:09):
Didn't put that there, right? No, that's a flag.
Host: Paul Barnhurst (23:13):
Oh you add it? Okay. I thought it was like was there an explanation note or something? I was going to be impressed. You
Co-Host 2: Ian Schnoor (23:19):
Got excited there Paul for a second you thought, hey it's
Host: Paul Barnhurst (23:22):
I did. I was like wow, it's now giving me explanations.
Co-Host 1: Giles Male (23:24):
So other than that, obviously that is a big I think area of concern. It's pulling from the schedules. I do wait,
Host: Paul Barnhurst (23:33):
Didn't just hard code a zero up there instead of pulling from a schedule
Co-Host 1: Giles Male (23:37):
Scroll
Host: Paul Barnhurst (23:38):
Down
Co-Host 1: Giles Male (23:38):
Link
Host: Paul Barnhurst (23:39):
Where there's all zeros. I thought that was a hard code
Co-Host 1: Giles Male (23:43):
Balancing.
Host: Paul Barnhurst (23:44):
Go up for a second. There was a keep going right there. One of those looked like it was all zeroes there.
Co-Host 1: Giles Male (23:51):
Well
Host: Paul Barnhurst (23:53):
I think it's the common shares issuance that was all zeroes, which
Co-Host 1: Giles Male (23:56):
Is fine if you look at the output, this broadly aligns with what Paul's model saw. So you don't have a cash balance. You are drawing on the revolver year after year. Little bits look sensible like the common shares don't change. The retained earnings is going up. You can see the amortisation on the senior debt. So that's coming down every period. That looks good. I think that's good. And then schedules wise, yeah, again, I'm a fast but I would much rather see flags and blocks, better blocks than this. It's pulling a lot directly from the assumptions, which again is very similar to what Paul saw, which is not my preference but it looks, I mean on the face of it, let's go back when
Host: Paul Barnhurst (24:44):
You could with skills, give it to use the face, the fat nut face, the fast standards.
Co-Host 1: Giles Male (24:50):
Yeah, you absolutely could. But that all looks pretty consistent and what are my assumptions? Look, so here's big improvement on your one. If this does work, I've got a scenario switch, I'm assuming if I change that to two it goes to best. How's it doing that exactly how I would do it. Index you could use choose index, switch, whatever you like. Probably not switch but choose which index. Yeah,
Host: Paul Barnhurst (25:16):
You could. I wouldn't use switch. You could use an nested if Giles
Co-Host 1: Giles Male (25:22):
You could, but I think that's impressive. So it is got a proper scenario tool. I haven't got time to check everything. It's doing the same sort of thing with inflation here of various things. Linking back to an inflation rate formatted, this is very much in line with how half of the Excel financial modeling world formats. It's always either blue fill or yellow font. I think I'm pretty happy with that. It took a lot longer than the two models you ran, but I think that's looking pretty good.
Host: Paul Barnhurst (25:55):
I think what I like it did a better job with its base best and worst case, no question. I think they were both similar on the other one area I do on the assumptions page that mine did is it told me where it pulled it from the document and I didn't ask it to do that. It came from this cell or we used averages so I can at least trace it all out on the whole, I think this one followed a little more, some best practises, although I don't see, did they give us a check section? The other one did
Co-Host 1: Giles Male (26:23):
Balance. Balance. I haven't seen one. And again, I think that's a huge plus point for yours. The fact, I mean I do have the balance sheet check. Yeah,
Host: Paul Barnhurst (26:31):
You have the one.
Co-Host 1: Giles Male (26:32):
I don't have anything else. I don't think.
Host: Paul Barnhurst (26:33):
I probably maybe lean slightly yours so far, but there's plus and minuses in both of them
Co-Host 1: Giles Male (26:39):
I think because of the scenario. Exactly. No, not that I'm not trying to defend my tool. It's the tool. No at all. I agree with you. Scenario thing is probably a big thing because that's kind of fundamental to how do you actually build scenarios in a model.
Host: Paul Barnhurst (26:54):
Agree if there's a flaw in the one I did, it's the way it did. The scenarios is just a mess. That's a D minus type work, right? Just bad
Co-Host 1: Giles Male (27:05):
Two out of two balancing balance sheets even that's a step up from when we do you remember when we started months ago,
Co-Host 2: Ian Schnoor (27:13):
They couldn't even build anything. I mean it was awful. It was horrible six months ago.
Host: Paul Barnhurst (27:18):
Remember one time in inside, are we really going to show this? I'm embarrassed to show this. That tool has shut down so we could name it now we won't, but to where we're at now. So it's a huge improvement. All right, why don't you show us what you got. We'll spend a few minutes there and then we'll wrap up.
Co-Host 2: Ian Schnoor (27:33):
So mine is definitely formatted the worst out of the three of yours, but there's something similar. Again, I used Opus 4.8 right in copilot and there's something similar feeling about our two trials. There's a switch over here, it's going into, well it's actually going into the switch is actually only going into sales prices and sales volumes. So here they are and you use its choose function. It's okay, I like that. Very nice. I looked at these formulas. They're mostly fine. I mean this is 2026 and they're not using colour differences. It's okay, not super clear. If I go to the schedules then there's a sheet of schedules, not very long. It just has some very simple schedules with pricing, some cost assumptions, long formulas here. Look what I did for depreciation. It's okay. Let's see here. I mean this is all good working capital. You do
Host: Paul Barnhurst (28:27):
A depreciation schedule. Did yours do one They, that's the first one that actually did a schedule by year. Did your Giles
Co-Host 2: Ian Schnoor (28:35):
Mine built a summary sheet? I don't really understand. It doesn't look very good. It built a summary sheet for five years under these five items. So anyway, the best and worst cases are just dead numbers. So we need to realise that, that they're not actually adapting. Their dead numbers are typed in. I don't know. I mean listen, I think it's off to a nice, the financial statements look fine. They're a little, they're not quite client ready yet, but they seem like they're okay. All these cells are links. I looked at it quickly before, it does look like after we ran it, I was looking at it. It does look like it's built some pretty good formulas. I like the
Host: Paul Barnhurst (29:18):
Cash available before
Co-Host 2: Ian Schnoor (29:19):
Revolver,
Host: Paul Barnhurst (29:20):
That kind of helper sell.
Co-Host 2: Ian Schnoor (29:21):
It's rightly,
Host: Paul Barnhurst (29:22):
But it
Co-Host 2: Ian Schnoor (29:22):
Looks, I haven't found any big errors. I've only been looking at it for 10 minutes, but I haven't been found any big errors off to a nice start. Here's got the base best and worst and it's typed in dead numbers. There's no formulas here. So I don't know. I don't like that it is a base case. So I just have to trust that they're working base best. Worst. I just have to trust that they're working properly. I don't know. I would be happy to start to use this as my starting point. Sorry Charles,
Co-Host 1: Giles Male (29:51):
Isn't that fascinating though, which was kind of why we did this deliberately that you and I have both used the same underlying LLM 4.8, but mine is directly through the Claude add in and yours is through the copilot add in and yet it must just be the additional context and whatever else is going on in that processing step for and after.
Host: Paul Barnhurst (30:15):
I think more than that, a lot of it also comes back to one, what skills it's using. So what skills does copilot have inside? What tools are it's using, how it's pulling the data. But the other, it speaks to the fact that this is probabilistic in nature and that's why you have you guys heard the term AI harness
(30:36):
Giving AI a harness. Basically it's everything beyond the prompt, all the context you're giving a files and the Claude skills. They're basically saying because if you give a harness, if you're climbing or whatever you're doing, a harness restrains you and protects you. And it's kind of similar to that to ai. It needs that additional structure and guidance or we end up with these three very different versions. I'm sure if we all build a lot of skills, we could get much closer to each other between these three models. There'd still be ones that would be better and differences where we could close some of that gap. But that's a whole new thing people are expected to learn. Now you got to know how to model really well, but now go ahead and figure out how to do AI really well and then figure out how to be a good auditor now instead of someone else auditing it. You got to audit the whole thing, not that you didn't audit before but in a different way.
Co-Host 2: Ian Schnoor (31:25):
And you're going to, to know your
Host: Paul Barnhurst (31:26):
Own and auditing someone else's is very different.
Co-Host 2: Ian Schnoor (31:28):
It's very different for our brains, for many of us that have been building things or building, it's a very different learning process in your brain to build and to fight the struggle. And the fight is important when you're, that's the creative process and a lot of people's learning is absorbed and it happens through osmosis during the struggle, during the fight of figuring it out. And that's important. So I know for me, that's how I often learn things by fighting and struggling and learning through the wins and the losses to get there. And then I'm very, very smart about a topic at the end of it. For me, if you gave me a fixed model and you just said learn it in the same level of intimacy and just review it, I don't know that my brain can actually get to the same level of confidence with it or that I can have the same level of deep understanding with it by trying to understand something now I'm going to have to because the way the world's going and we will. But we all have to find a way, you're right to look at this and still rip it apart, understand it, know what it's doing right, what it's doing wrong, and be able to defend it as if it's our own. And that's a bit of a big mind shift. I don't know if you agree with that, Giles, but that's a shift for me.
Co-Host 1: Giles Male (32:39):
Yeah, I do agree. And the more we look at this, I think we've all been on a journey since. Is this the wrap up session? Is it worth sharing? I feel like we're in the wrap up session, Paul. Alright, let's share our closing thoughts. I think back to when we started, and it was longer than six months ago and at that time I didn't, it was
Host: Paul Barnhurst (33:01):
A year ago now when we first approached this just, yeah, nine,
Co-Host 2: Ian Schnoor (33:05):
10
Host: Paul Barnhurst (33:05):
Months. First episode we did was October Really
Co-Host 1: Giles Male (33:08):
Okay
Host: Paul Barnhurst (33:09):
Week, which means we recorded it in September, which probably means we first discussed it in August, late August. Yeah. We were planning,
Co-Host 1: Giles Male (33:18):
I think back to when we started and this whole concept of agents that you can draw on within an app and it wasn't there. And that's why I think we started with third party tools. They were the only ones that had kind of put that layer on. So the world has completely changed. I think we can see clear improvement. The fact that I think we just all had balancing balance sheets. Awesome. And the more we get into this, the more I'm convinced that focus, this is not a criticism of what we've just done, but the focus on can it build me a financial model so that I don't have to do any of the work. Just feels so it's not the pot of gold at the end of the rainbow for me. And that's partly now I am looking at that space and going, God, you've got teams up to their eyeballs and all this other stuff. Do you remember Ian Bennett talking about all the value you could get from all of the other areas of a modeling project? And I think I'm probably even more convinced that that is where the value is at the moment at least. And what figure all that stuff out. If you're a big company and then by the time you've done all of that, we'll be on Fable. We'll be on mythos, we version nine. And you won't have to get out of bed,
Co-Host 2: Ian Schnoor (34:29):
Right? You'll think it, you'll just,
Host: Paul Barnhurst (34:31):
We'll have alpha. Just
Co-Host 1: Giles Male (34:33):
Think
Co-Host 2: Ian Schnoor (34:33):
It. You'll think something and it will develop and yeah,
Host: Paul Barnhurst (34:37):
That's a scary thought. If it can read my thought. Any lasts thoughts from you on this end? I think we got Giles' dissertation. Let's move to you. No,
Co-Host 2: Ian Schnoor (34:45):
I agree. I agree with Giles. I think this is a different world than nine months ago when we started, when we didn't even, it was only at the end of the year. Remember it was at the end of last year when the LLMs announced that they were starting to incorporate. It wasn't even considered in the fall. We were looking at all these other tools and I haven't kept up with how they're all performing, but I suspect it's a challenge for many of them where they need to differentiate themselves. Obviously if they're going to survive now, people are desperately learning, going deeper, as you said into ai. They're trying to code, they're trying to learn skills, they're trying to get under and think about it in terms of automating their day and their monthly close and their life and their emails and connecting it all together.
(35:28):
We weren't talking about anything like that six, nine months ago. And from what we're seeing today, obviously these three models that we built today, I don't know if there's a real winner. I mean they're all similar I would say, but they are much better than what we looked at nine months ago. But compared to the models from February, March a little bit, the balance sheets balance, which is nice, but we're still with all of them. None of these is ready to hand in. And that's even scary if people think that they have a models, right? Because you're missing the point of the learning process I think. And
Host: Paul Barnhurst (36:03):
Yeah, I don't think there's a market improvement like you said from Opus four seven to what we did today. But
Co-Host 1: Giles Male (36:10):
I'd be really intrigued if you two are up for it. I think we should do, whether it's the next episode or whatever, I think we should do it like, okay, let's give it our best shot. Let's actually put all of the custom instructions we would think of and skills files and whatever else. I'd be really intrigued to be like with our best efforts, how good could we get it straight away?
Co-Host 2: Ian Schnoor (36:33):
But that takes a lot of skill on our part to know modeling, to know accounting, and to know how to really be a strong musician with this tool to work it, to really work our models. And that's going to take a lot of time and effort to get there. And I'm not saying we shouldn't, we should, but that's not free. That comes with investment, right? That's a real investment.
Co-Host 1: Giles Male (36:58):
We're just invoicing Paul for our time. We Isn't
Co-Host 2: Ian Schnoor (37:00):
That correct? Right.
Host: Paul Barnhurst (37:02):
Hours, hours. I'm pulling it to Anthropic.
Co-Host 1: Giles Male (37:04):
Yeah, let's get Nico to pay for it. Well your closing thoughts Paul?
Host: Paul Barnhurst (37:08):
I think it's similar to the two of you. One thing I want to say is do you remember when we tested copilot online, right? That was an episode. Think how far we've come from that. A lot of people have badmouthed, copilot. I've been guilty of it in general. I think Claude's a better tool right now and I think most people do. That's pretty clear. But how far it's come, that's what I keep telling people is you can get value out of it. I don't care which one you're using. I think one message you can get value out of all of 'em. Two, none of them on their own are production ready. Could you potentially get there with skills for certain tasks, for certain things? No doubt. Full financial model maybe depending on what you're doing. But even if you do, you still have to know how to audit it.
(37:53):
You still know have to check it. I am just like the two of you. I say AI is a magnifier. The better you know what you're doing, the more you can get out of it. The one on thing I'm so sick of is the AI slop. We're seeing all over the place of do this in five minutes and I will repeat what I've been saying all along. Go ahead and build the model in five minutes, get fired in 10. It's just stupid. If that's the way you think these tools work. There's not a magic easy answer to anything. And that's true with modeling, but you can get huge benefit if you're willing to pay the price, I think is the message.
Co-Host 2: Ian Schnoor (38:25):
Yep, I agree with that. Jan. Always great to see you.