How Excel AI Agents Like Rosie Work for FP&A Tasks but Fail at Building Models with Giles and Ian

Note: This episode was recorded and scheduled for release before the recent announcement that Rosie AI is shutting down.

In this episode of the ModSquad series on Financial Modeler’s Corner, host Paul Barnhurst is joined by modeling experts Ian Schnoor and Giles Male to evaluate Rosie AI, a new tool that integrates with both Excel and Google Sheets. Together, they test its capabilities in building financial models, solving complex FP&A tasks, and performing real-world use cases. They push Rosie through a series of tests, from basic formula creation to building full three-statement models, and discuss where it excels, where it needs improvement, and its potential future in the world of financial modeling.

Expect to Learn

• How Rosie AI performs in real-world financial modeling tasks

• The strengths and weaknesses of Rosie in building models and formulas

• Key insights into how AI tools are evolving in financial modeling

• The importance of knowing how to validate AI-generated models

• How Rosie compares with other AI tools and traditional financial modeling techniques

Here are a few quotes from the episode:

• “AI can’t replace the need for modelers to understand the logic, but it can help speed up certain tasks.” - Giles Male

• “If you know what you’re doing, these tools can save you time. If you don’t, you need to double-check everything.” - Ian Schnoor

Follow Ian Schnoor:
LinkedIn -  https://www.linkedin.com/in/ianschnoor/

Follow Giles Male:
LinkedIn -  https://www.linkedin.com/in/giles-male-30643b15/

Rosie AI: https://www.askrosie.ai/

In today’s episode:

[02:23] - Excel Turns 40

[07:29] - Microsoft’s AI Agent vs. Modeling Tools
[16:18] - Exploring Rosie’s Website and Features

[25:35] - Real-World FP&A Test: Trial Balance Summary

[35:53] - Deferred Revenue Schedule Test

[50:54] - Detecting Inconsistent Formulas in Financial Models

[01:00:53] - Testing Rosie AI on a Five-Year Forecast Model Build

[01:08:31] - Final Thoughts: Where AI Helps and Where It Doesn’t

Full Show Transcript

[00:01:34] Host: Paul Barnhurst: Welcome to another episode of the Financial Modeler’s Corner. Again this week we have our special series featuring the Mod Squad that includes Giles, Male Giles. Welcome back.

[00:01:48] Co-host 1: Giles Male: Pleasure to be here again, Paul. Nice to see you. Who, me or Paul?

[00:01:52] Co-Host 2: Ian Schnoor : You. Giles had a pretty major announcement, so. And a pretty major release. Yes.

[00:01:56] Co-host 1: Giles Male: So you're the music video? Yeah.

[00:01:58] Host: Paul Barnhurst: I think we'll cover that in the news.

[00:02:00] Co-Host 2: Ian Schnoor : We'll cover that in the news.

[00:02:02] Host: Paul Barnhurst: Excel had a big birthday, I heard. So welcome back, Giles. And welcome back.

[00:02:08] Co-Host 2: Ian Schnoor : Thank you. Thanks, Paul. Great to be back for another episode here.

[00:02:12] Host: Paul Barnhurst: Yeah. I'm really excited to have you. And, you know, it always was. My name is Paul Barnhurst and I get to co-host with these two amazing modelers. I almost had something else, but I can stop myself.

[00:02:22] Co-host 1: Giles Male: I don't want to know what you were going to say.

[00:02:23] Host: Paul Barnhurst: We're here again to test tools, have a little fun, share our views. And at the end of the day, we're really hopeful. This is helpful. It's one of the reasons we've added before we get into what tool we're going to test. And then the actual testing is we're adding a section each week that is called the Mod Squad. The latest news. This space is moving extremely fast, but just from the last testing we did to today, there's been a lot of changes. But before we get into the AI news, Excel had a birthday on September 30th. It turned 40 and there was this big reveal, a video that was released and I heard it was quite popular. So Giles, tell us a little bit about this video you did.

[00:03:04] Co-host 1: Giles Male: We doubled down. I did a music video a year ago, uh, when I got made MVP for the first time, and this time we went even bigger. So, yeah, to celebrate Excel's 40th, we hired a thousand seater theater for two days, got Deb Ashby back, roped Mark Proctor into it, and somehow roped Leila Gharani into flying over from Vienna. And, uh, and about 25 dancers, a children's choir. God knows what else. It was bonkers. But yeah, everybody seemed to like it. That was the only thing I wanted. I just wanted people to have a good time.

[00:03:34] Co-Host 2: Ian Schnoor : If you haven't seen it, check out Giles Male LinkedIn page and take a look. It is absolutely worth your time and well worth watching. You'll get a great laugh and enjoy it. I highly encourage everyone to take a quick look.

[00:03:47] Co-host 1: Giles Male: Thank you very much.

[00:03:48] Host: Paul Barnhurst: Fully agree. It was very well done. It was a lot of fun. It's amazing to think Excel is 40.

[00:03:54] Co-Host 2: Ian Schnoor : It's older than all of us at this point, isn't it?

[00:03:56] Co-host 1: Giles Male: No.

[00:03:58] Co-Host 2: Ian Schnoor : Maybe not, maybe not.

[00:04:00] Host: Paul Barnhurst: It's older than one of us. Maybe close. All right.

[00:04:05] Co-Host 2: Ian Schnoor : All right, Paul, let's go.

[00:04:07] Host: Paul Barnhurst: So, other news. Last week, Xcel released in beta an Excel agent. And I think that's a really big announcement. I'll share a few thoughts on that. And then we'll go around the horn, so to speak, here and get everybody's take. I've done some testing. I've seen a lot of videos. I think it's very similar to these tools we test in areas that will be stronger in areas that will be weaker. But there is one big drawback against all the other tools that will change with time. By the time this is released, it could change with how quick things move. It's currently only available on the web. And Giles, when was the last time you built a full model in the cloud version?

[00:04:46] Co-host 1: Giles Male: I've never built a model on the web version of Excel. Never. I can't use it. So yes.

[00:04:55] Host: Paul Barnhurst: What's your take? Do you ever use the cloud version?

[00:04:57] Co-Host 2: Ian Schnoor : No I don't, and I and you know, in my prior life when I was a trainer before, you know, heading up FMI, when I was a trainer and ran a training company for 20 years, and I trained thousands and thousands and thousands of bankers all over the world, I never worked with anyone who was using it. So I'm not a big fan of powerful tools like that. In a cloud version where you're really keyboard intensive, certainly Excel. I have not, so I'm not a big fan of it. And yes, for now it only works. For now. This new agent only works in the cloud version of Excel. I assume that it. I have no reason to say this, but Paul, maybe you do. I assume it will become available in the desktop version of Excel at some point.

[00:05:37] Host: Paul Barnhurst: That's the general assumption. Time will tell. I mean, usually they start with the cloud and then they release it to the desktop. So I'm with both of you. I've never built a model in the cloud. I mean, I've used it for a few little things, mostly when I need to download my forms because it opens them online and it's just a quick look at something, not building anything. Consumption. Great building.

[00:05:59] Co-Host 2: Ian Schnoor : Know there's a great purpose. There's a great reason to use spreadsheets in the cloud, collaboration is better, etc. so great. But for building truly hardcore, rigorous spreadsheet tools. I'm not aware of. I'm not aware of anyone that's largely using Excel in the cloud as opposed to a desktop version. And the other issue is, as you said, I mean it, this new agent tool looks really powerful. And it does. I would think at this point, I would think that it must threaten the numerous companies that have come up with their own add-ins to Excel. It looks like Excel is going to fill that gap and meet or possibly exceed what these existing financial modeling AI tools are doing. Time will tell. I don't know if either of you have a thought on that point.

[00:06:47] Co-host 1: Giles Male: I totally agree. I was going to say it just feels like it's interesting, isn't it? Because what we're doing with our whole focus for this podcast series was on the tools, and we're kind of one full episode, full review down. And, Microsoft has come out with the agent and, um, I think it's great. It's, you know, a step in the right direction. Uh, I think everybody's just got to be objective about what it can and can't do and how you use it. But yeah. Awesome. I love the idea that it could all be managed by Microsoft. Maybe we will be building full scale financial models with Microsoft's AI tool in the future.

[00:07:25] Co-Host 2: Ian Schnoor : What do you think about this, uh, what do you think?

[00:07:27] Host: Paul Barnhurst: I'm a little more measured.

[00:07:29] Host: Paul Barnhurst: I think it definitely could be a big issue for a lot of these tools, but I also think there is some room for some of them and that they're very much specialized. This agent is still going to be broader. Yeah. It's not like it's just modeling. So is it going to do that, those extra little things that push it over the top and make a modeler want to use it, And how long will that take? It's not the same priority, so I think there could be room for some of them. But do I think they're nervous? They better be. Do I think if Microsoft wants to, if it's a primary focus to say we want our agent to be the best in the world in modeling, it will be these other tools, right, because it has access to them. They've announced anthropic is now available for open AI. They have ChatGPT. They have the resources. They have access to any of the code to do whatever they want. Right. Once you integrate something and you decide you want to do it, it doesn't have to be on parity. I mean, it has to do most of the things they do. But if it makes other things easier because of the way they can integrate, yeah, that's hard to compete with. So if this is a real , if modeling is their focus, it's a matter of time.

[00:08:37] Co-Host 2: Ian Schnoor : And I would also say and and and how you feel about it, I would also say there's we are in an era where there's a lot of hype still, and no one really knows exactly what the future looks like. You know, I continue to be of two minds. I love the direction things are going. I'm amazed at the technology. I'm amazed at the speed and some of the benefits that will accrue as a result of these tools. And yet, I also remain firmly entrenched in my view that with modeling specifically, which means thinking and understanding and analyzing, um, that there will always be that the best and the brightest will always balance. Usage of tools with deep knowledge and deep insights themselves. Which means I don't foresee a day when you push a button, get a tool, and show up at a board meeting that day. I don't know if that day is ever happening. You'll, you know, just be wildly embarrassed if you show up at a board meeting and you haven't understood every single thing going on in your model. So you have to understand it and know how to challenge it, and you have to know how to push back on these tools and use it as your partner. Um, just like and that takes time, just like you always used to. And I don't know if the two of you have a thought on that.

[00:09:43] Co-host 1: Giles Male: I so similarly, I think we're all of the same mindset, which is a great start for for the podcast, but I'm excited for the future. I'm just really anti hype and I think people need to be really careful with the language. Yeah. Next thing you're.

[00:10:00] Host: Paul Barnhurst: Gonna tell me you don't like cheat sheets Giles.

[00:10:03] Co-host 1: Giles Male: Hype I just, I just get annoyed every now and then with the loose language. When you hear people saying, oh, you can, you can click a button and what you used to do in four hours, you can go get a coffee and it's done. So that's, that's it'll do something, it'll build something. And it might say income statement, but it's not at the level where you can just go, oh, we don't need an analyst to do that now. I'm sure it will get there. But just don't hype it up because then when people use the tools they're going to go, oh, this is not what I thought it was. This is not what I've been reading about.

[00:10:36] Co-Host 2: Ian Schnoor : But the commentary. When there's commentary around, take a six hour task and get it down to six minutes. There will never be a day when you can learn. I mean, that six hours was not just clicking away in Excel. That six hours was your thinking process and your design and understanding how you're going to frame and shape and form the data. So taking that down to six minutes is some of the build, and that can be great, but you're still going to need a lot of time to kind of get yourself caught up and up to speed. So you can be an intelligent leader on a team that's not going down to six minutes. And so I think there's some misconception over how it's going to, uh, you know, impact people's jobs. But anyway, Paul.

[00:11:13] Host: Paul Barnhurst: And I think one thing less, you know, people think we're like, hey, the agent's just going to take over. There's no space for these tools. We still think it's important to test them. We still think this testing will allow us to see where the strengths are, because I guarantee you, not every tool is going to respond the same. We'll test agents in this group before we're done to see how the Excel agent responds. So with that, why don't we jump into this week's tool? Really excited, it's Rosie AI. I think all of us had the opportunity to chat with their CEO Dennis. And I'm going to start by just sharing a little bit about the company. We'll look at their website similar to what we've done before, and then we'll jump into testing and we're going to do the same order we did last time for testing. We're going to start with our humble MVP, then the bearded guy, then the smart one of the group will, uh, close out for us.

[00:12:02] Co-Host 2: Ian Schnoor : It is not true, Paul. There's only one MVP in this group. Well.

[00:12:07] Host: Paul Barnhurst: There's only one humble MVP.

[00:12:09] Co-Host 2: Ian Schnoor : There's 100. Oh, that's right, I forgot you're an MVP as well. I forgot that Paul. You don't talk about it as much. You don't talk about being humble.

[00:12:19] Co-host 1: Giles Male: Why don't you talk about it all the time, Paul?

[00:12:23] Host: Paul Barnhurst: Because you do more than enough for both of us. Giles.

[00:12:26] Co-host 1: Giles Male: You can be the next music video. You'll be all right.

[00:12:28] Host: Paul Barnhurst: If I can. Then I'll be famous. All right, so let's jump into Rosie. Ai. So what's interesting is Rosie has now launched. It's out there. It's one of the few many are planning on doing it. It works in both Excel and Google, so I think that's a good step. There are definitely people that's really helpful, right? You got that full cloud and Google if you want a cloud spreadsheet. It's the strongest cloud spreadsheet out there, most history and so forth. So, uh, launch date, it's one of the older tools. It's been around for two years now.

[00:12:56] Co-Host 2: Ian Schnoor : It's hilarious. We're in an industry. We're two years old and.

[00:13:00] Host: Paul Barnhurst: I know it. But when you think of AI, that's a lifetime.

[00:13:05] Co-Host 2: Ian Schnoor : Absolutely.

[00:13:06] Host: Paul Barnhurst: You know, its focus is modeling in Excel. It's really looking at finance and consulting professionals. So I think ideal testing really like what it's done in its pricing. I love that it has a student version, you know, $10 a month. I'd love to see that even lower. But I like that they're trying to accommodate everybody, the individuals at a very reasonable price point at $20. And then, hey, if you want a team, contact us. Yeah, right. I appreciate the pricing. Any thoughts so far, Giles.

[00:13:37] Co-host 1: Giles Male: With the same thought, just again, with this spectrum of pricing that we're seeing ten and $20 a month for a proper license is fantastic. I think it is interesting that they're on Google Sheets and Excel. Yeah, I'm not a Google Sheets user other than just to manage things with my own team, but I'm sure that's a good bonus.

[00:13:58] Co-Host 2: Ian Schnoor : I quite like Google Sheets for that. It's funny, I almost think of them as different sorts of tools. I mean, there are certain tasks that I naturally gravitate to Google Sheets, like collaborating with our team, building, you know, collective, you know, spreadsheets to analyze, project, whatever we're doing. Um, that of course, we use sheets for, um, I think we've all I've met, uh, Dennis, the founder of Rosie, a great guy. I really admire what they're doing. Um, I'm rooting for them and for all of them. I'll be honest. I am rooting for every one of these because I think there's. It's amazing to see what's happening. I'm rooting for them all. And I really hope that, um, that there's a space for them. It's not going to be easy. Right? But as we've, as we as we seem to think. But I root for all of their success. So. Yeah. Did you anything you want to finish on the summary, Paul.

[00:14:46] Host: Paul Barnhurst: I think I'll mention if they use, uh, GPT. Yeah, they're married to one company. They have multiple different models. So I think that's interesting. You know, we'll see what they know, what they say makes them unique. And we'll get to kind of test this. Rosie combines deep semantic indexing with native integration. So you know, some of that work they've done on their own to really allow you to accomplish more. So we'll get in and test that.

[00:15:11] Co-Host 2: Ian Schnoor : I think we discovered in our last episode that having the choice sometimes, you know, I think we all find in life having too much choice is never always a good thing, right? Just, you know, just show me one paint color, you know, if red and it's perfect, right? I mean, having too many options for anything can be overwhelming. I think we discovered when we had a number of choices with the last tool. Sounds like a good idea, but I think we found that that was.

[00:15:38] Host: Paul Barnhurst: There are definitely some challenges in that right for sure.

[00:15:42] Co-Host 2: Ian Schnoor : So now we're going to see the opposite. Now we're going to see the opposite of a tool with only one. And we'll see how.

[00:15:46] Host: Paul Barnhurst: We feel about which model if it's going to use any different GPT models. So we'll see a tool that hey choice is kind of gone so to speak, and we'll see how it performs.

[00:15:55] Co-host 1: Giles Male: But even that's interesting because I remember last episode, Ian, you were saying, like, God, it would be so much better if the tool made the choice for you. So to some degree, albeit within one area, it's doing that, which is interesting. So that's it.

[00:16:08] Host: Paul Barnhurst: What that's.

[00:16:09] Co-Host 2: Ian Schnoor : What Rosie's going to do. You're saying.

[00:16:10] Co-host 1: Giles Male: Yeah yeah yeah yeah.

[00:16:12] Host: Paul Barnhurst: Yeah.

[00:16:12] Co-Host 2: Ian Schnoor : It will, but it still stays within. It still stays within one family. Correct. That's what you were saying, Paul.

[00:16:18] Host: Paul Barnhurst: Yeah, yeah it stays. It's all within open AI. So we'll take a quick peek at their websites similar to what we did before, you know, very, very plain. Not in a bad way. Very simple website. Right. You come right to the blog. Data security contact us.

[00:16:32] Co-Host 2: Ian Schnoor : Help center I love it.

[00:16:33] Host: Paul Barnhurst: You got Rosie. You got the options. You got a video, you got everybody has their testimonials. Here are some of the companies that are using it.

[00:16:42] Co-host 1: Giles Male: It's not bad company. Tesla. No it's not. And what.

[00:16:45] Host: Paul Barnhurst: I wonder.

[00:16:46] Co-host 1: Giles Male: Ikea.

[00:16:47] Host: Paul Barnhurst: I'm wondering if some of these are individuals testing a small departments. I doubt those entire companies have integrated the tool across their finance team, but if they have more power to him, he's doing awesome, right? So hey, Rosie gives you a spreadsheet, powers catch errors before they catch you. Modeling made magical. Now here we discussed this a few times. Rosie can build a new model from scratch or iterate on existing ones. Just ask Rosie and it's done. Little, little hype. But I get it. They're selling less struggle, more answers. Why you'll love Rosie. No formula errors. Save hours. Work across sheets. Excel for everyone. Smart data analysts works with your data, and then you can watch a video of it in action. And it has pricing. So pretty similar to last one. Anything you guys want to say? As I scrolled through that.

[00:17:40] Co-host 1: Giles Male: Yeah, no more formula. Errors is a bold claim. Did you say it said that?

[00:17:46] Host: Paul Barnhurst: Yeah. It said eliminate hash ref Na and other common errors with AI that understands what you want.

[00:17:52] Co-host 1: Giles Male: I okay, okay, I see what they mean. I thought it was making the claim that by using it there will no longer be any errors. It's saying it can find hash errors and remove them for you. Gotcha.

[00:18:03] Host: Paul Barnhurst: Yeah. So you can eliminate those errors from your models and. All right. So why don't we go ahead and jump into the actual, uh, testing here.

[00:18:12] Co-host 1: Giles Male: So this is the first of the two esports cases. This is the simpler one. Uh, for quick memory, it's seven levels that I'm going to get it to do. Uh, starting from simple things like pulling apart string from a text. And it gets more complicated as you go through counting characters in a text string referencing a map somewhere else. In terms of the Rosie interface, again, this is an add in. So you've got a button on the home ribbon tab. You've got a very nice friendly note that comes up saying like, hello, this is Rosie, your AI spreadsheet assistant. I haven't clicked anything and it's already basically given me a summary saying, this is what this spreadsheet does in my opinion. Here's what I can do. So very nice. Where would you like to start? I'm going to type in that same prompt again saying, look, I want you to analyze this, um, solve all the questions. So what do we got? We've got the a more concise summary of what has happened here. Um, really just just like here's what the area is, here's what I've done in a solution. Um, but I have to say, Rosie, on a quick glance, has got all seven levels. Right. And I think it's quite impressive to me is all the levels are formula driven. Yeah yeah yeah yeah. Level level two. So it's like, again, this is one area where it wasn't quite as efficient. It is referencing texturing, doing the split, taking the two largest values and summing them together. There's no hard code. Do you remember last.

[00:19:49] Host: Paul Barnhurst: And.

[00:19:49] Co-Host 2: Ian Schnoor : I remember the last and we were impressed. First of all, this is very impressive. But I remember we were very impressed. It was able to read all of the values. Of course, what you're asking people to do in this question, right, is you've got a big string of text in the numbers area. You're asking them to pluck out the individual, um, numbers. Uh, and then what were you asking? And it was very clever. That was really clever. But it managed it because it's a computer. It typed in the numbers within your formula and not something a human could learn from, or a human could do themselves. So we were impressed, but recognized. So this one is literally doing exactly as you would expect.

[00:20:32] Co-host 1: Giles Male: A lot of sports place we do it is the same level three. You got to pull out.

[00:20:37] Co-Host 2: Ian Schnoor : Something they're.

[00:20:39] Host: Paul Barnhurst: Going to add. I think one of the key differences here we'll see this with different tools is the last tool you saw. A lot of times you said it was doing it in Python and then kind of converting it. I don't see any mention of that here with Rosie. So I think there's a little bit one different model but different approach on the back end. Yeah. That's why I say there's room for a few tools. I don't think agent necessarily just takes everything because there's different approaches. And so there's going to be different strengths and weaknesses across the board. I think you'll see that throughout all this testing.

[00:21:09] Co-host 1: Giles Male: Sure. So I'll run through this quickly. But for me every level has been answered with good formulas that are linking to cells rather than putting hard coded numbers in. So this is using indirect and column to get the column number, uh, counting the number of times this character is in the string. This is exactly what we would do Len minus Len with the substitute of the character for nothing, Uh, level five, which trace light didn't interpret correctly. This has done correctly. Uh, level six, you've got, uh, indirect, of the map, level seven. Indirect. With the shift for above and below, you can see.

[00:21:46] Host: Paul Barnhurst: The OP with an offset with the simple if really. Yeah, really clean formula.

[00:21:52] Co-Host 2: Ian Schnoor : Really strong. And again you know I'm going to keep you know coming back to this point I love it. And I love the fact that that it can do this. And I also believe that someone looking at this needs to understand what it has done. Um, because someone is on your team is going to ask you, what does that offset? What is an indirect right. Indirect is one of the most feared and least used functions in all of Excel, and you need to understand it if it's using it. And as we're seeing all of these AI tools so far have at times an inclination to hallucinate, get things wrong. So you do need to be able to double check it and explain it. So I love that it can do it, save you time and you need to know exactly what it's doing. You I would I would strongly encourage anyone. Never, never hand in something like this without double checking to make sure it's working. Would you guys agree with that?

[00:22:40] Co-host 1: Giles Male: I'm going to push on to the second one. I mean, the other thing that I would just mention again is we said last week was my prompt wasn't saying too much in the way of like specific here, here, here's every bit of information. So I'm impressed that it can interpret what the challenges are and then do the right work and use the right formulas. I think that's really impressive. Uh, so okay, we're on to the what I think is a significantly harder case with the humble MVP case. Uh, let me find the same.

[00:23:09] Host: Paul Barnhurst: Who is the humble MVP? I mean, we have two MVPs on the call. Is that me or is that you? That's.

[00:23:14] Co-host 1: Giles Male: I just I just I don't like to talk about myself.

[00:23:17] Co-Host 2: Ian Schnoor : You have to ask, Paul. If you really have to ask, then you're not. You're not in with the program.

[00:23:23] Host: Paul Barnhurst: So I know I need to get in with the crowd. Alrighty. So we have the Microsoft Excel UK Championship case running and we'll be back when it's finished. It's all about our humble MVP, so stay tuned to see how Rosie does.

[00:23:37] Co-host 1: Giles Male: Okay, so we're back. And it's interesting because I tested Rosie on these cases before the recording started. And actually it did. It did this second case in full. It did take a while. This time it did levels 1 to 3 and then it just seemed to die. It said there has been an error and it stopped. I've restarted it. It has managed to do levels 1 to 4 and it's just still churning over. For the rest it.

[00:24:03] Host: Paul Barnhurst: Is.

[00:24:03] Co-Host 2: Ian Schnoor : Trying to do level. Did it do level four?

[00:24:06] Co-host 1: Giles Male: It has done level four as well. And I mean that. So this this is a hard level. The this is difficult. There are a lot of things in play here about characteristics of my mood and the coats and everything. That's not easy to come up with. Um, it's definitely still kind of working through 5 to 7. It did do this when I tested it before, but we paused for 15 minutes. I think that's too long, but I would still say that's quite impressive. Very impressive. Coming up with very complex, which is not necessarily a good thing, but it's coming up with very complex single cell solutions.

[00:24:39] Host: Paul Barnhurst: Like using let and it's naming everything you could someone could go through and trace that. That's complex though right? Momo transposed his number search index and a let. Did I miss anything at what has seven oh short by in a sequence in a row? So what is there about different formulas in that thing?

[00:25:00] Co-host 1: Giles Male: There's a lot. But but it is doing it formula based. This is more of a sort of programing approach. Rather than shoving hard coded numbers everywhere, it is leaning on the supportive information. If I f2 that it is leaning. I didn't tell it to do this necessarily exactly like this, but it's done a pretty good modeling job of those levels 1 to 4. So I would say four out of seven just ran out of time. Maybe it would have got to the others in time. We don't know.

[00:25:26] Host: Paul Barnhurst: I mean, it got there before once. So I mean overall impressive, right? I think that the result is it does a really good job here.

[00:25:32] Co-host 1: Giles Male: Yes. I'm going to move straight on to the third one. And again, I'm not going to go through the whole subject again of what we're doing. This was kind of raw trial balanced data was essentially analyze the raw data and produce a sort of comprehensive summary with some visuals in there.

[00:25:50] Host: Paul Barnhurst: And this is the one that had the two sheets where you've done your summary sheet. And if I remember right, I referenced that a lot, not a lot on the balance sheet. We would have liked to see a little more work on the transactional, so it'll be interesting to see where they focus.

[00:26:04] Co-host 1: Giles Male: Definitely. The prompt specifically says look at the trial balance data, which is this one, and use that.

[00:26:09] Host: Paul Barnhurst: We are back about 7.5 minutes of work. I think there's a couple things in there. Giles, why don't you kind of walk us through how it did and what you're what you're showing us what your take is here.

[00:26:20] Co-host 1: Giles Male: Yeah, it's a bit of a mixed bag, I guess. A little bit like trace, like so. So it has actually produced a summary that's formally driven. It's using good functions. It's looking at the source data. So that's good. I mean there's a few, you know, hard coded things like revenue. And you could have looked at the headers here, but that's not bad. It's giving me the outcome that I kind of wanted, which is to show that net income drops negative and then kind of comes back at the end. So that's the right sort of insight that you're looking for. So if you look at stuff like this like it's that's okay, it's not beautiful, but it's done an okay job, I would say the graph looks a bit horrendous.

[00:27:02] Co-Host 2: Ian Schnoor : And why do you say okay. I mean the the analysis in the columns looks bang on or is it missing. Is are you concerned about are you concerned about anything in columns A through H. Is it missing. Is it missing anything. Is it you know.

[00:27:16] Co-host 1: Giles Male: No I think that's my point. So so in terms of like a professional report, the good bit is I think it's actually done the technical modeling pretty well. Yeah. Would you show this as a final report page? No, I think it would need work done to it, but it's like it's it's okay.

[00:27:34] Co-Host 2: Ian Schnoor : It's a great it's a great start. Right? I mean, it feels like it's a great start.

[00:27:38] Co-host 1: Giles Male: Well, it's it's an improvement on what we saw last week because I think the, the approach to getting the data and looking at the source data using the right functions is good.

[00:27:48] Host: Paul Barnhurst: But one thing that's interesting, I think was better last time in my take is remember, it gave us the product breakout. Here we don't get a product breakout. I think it summaries a little better in that it took it all off the trial balance. I like that it gave us operating expenses. The last one missed that. So I would like to take some from both of them. You know, if I'm doing my final work. And that's what's interesting is there are some areas on the whole, I think this one did a little better job on this task, but there were some things I liked from the first one.

[00:28:17] Co-host 1: Giles Male: I think that's fair. I think he's done. I think he's done better. It's not perfect. We probably aren't expecting perfect, but again, pretty good use of functions and logic to get a pretty future proofed answer, I think.

[00:28:30] Host: Paul Barnhurst: Before we move on to my testing, any last takeaways from what you've tested for Rosie?

[00:28:35] Co-host 1: Giles Male: I mean, I think it's done well. I mean, especially on the esports cases. I think they've got full marks on the first one. When I trialed it before on the harder case, it actually got full marks as well. But just again, that's an interesting talking point. Same prompt, same challenge. It just broke after 3 or 4 levels a second time around.

[00:28:57] Host: Paul Barnhurst: It could also just be how busy AI is and the app versus Rosie. It could just be some of the you know, what's the volume hitting it?

[00:29:06] Co-host 1: Giles Male: I'm pretty impressed. I mean, it's done a good job.

[00:29:08] Co-Host 2: Ian Schnoor : It's only getting better. It's going to have to, you know, continue to stay a step ahead of, uh, Microsoft's built in tools. Right. But, um, but it it's pretty, pretty mind blowing that it can do this. But again like take take it with. Right. It's not a replacement yet but it's a great start. It's getting some great stuff done. And uh, it's impressive what it's able to do on on the things you've tested it for.

[00:29:35] Co-host 1: Giles Male: Cool. I think that wraps up my bit.

[00:29:37] Host: Paul Barnhurst: Well, let's go ahead and move on to some use cases. We'll be able to test 2 or 3 here. We're going to test this last time. We'll be testing it again. This is a price volume mix where it's looking at margins and saying how much did the price impact it? How much did the volume impact it, and then how much did the mix of the individual products impact it? It's called Paul's Getting Spicy. It has three different products. We won't get into the details about them, but there's one called BB or BB, PMP and boss and we can see we have budget and actuals. So what we're going to ask it to do is we're going to ask it to fill in the price right here for these three products. We'll start with that prompt. We're going to go ahead and copy it in. Basically I said hey look in this workbook we have a price volume mix. I want you to give me the answers in these cells. So let's see what it does here. You can see it already did some work. It says this workbook walks through PVM topics. I can help with variance calculations and result visualization summary. Where would you like to start? I'll give it my prompt. All right, we're back. It only took about a minute to do the first one, and then another minute to do the next two. I didn't ask it to do volume or mix.

[00:30:52] Host: Paul Barnhurst: It said, hey, do you want me to answer those as well? And I said, yes. So let's walk through it. Price did it perfectly. You can see it took the gross margin from the actual compared it to the gross margin of the budget and the units and said, here's the difference in price. So we're $30 better on price on, well, gross margin per unit, which is, you know, we're kind of calling our price part of this PVM. And that's really why BB was better. This one was worse. This one was slightly better. Makes sense. All good volume. It didn't quite get that right. So what we wanted for volume is we want actual volume in total minus budget volume times average budget price. It took the individual prices, but it didn't look like it did it quite the way it was supposed to. And so it came up with -250. The actual and we can drop the formula should have been 125. So we could ask it to redo mix. If you look at average budget price is 125 and our volume was 100 better. So 100 times 125 should have been 125. Here it gave us -250. So it did something wrong on that one. And then it got the mix right. So pretty good. Not bad. One issue I'm just going to ask it. Please calculate the volume in total using row 29 for the formula, and we'll see if it gets it right.

[00:32:34] Co-Host 2: Ian Schnoor : You're looking for it to correct.

[00:32:36] Host: Paul Barnhurst: Yeah I'm seeing if it will correct it.

[00:32:38] Co-Host 2: Ian Schnoor : Because you didn't tell. You didn't tell it that you think it made a mistake. So we'll see. Um, it.

[00:32:43] Host: Paul Barnhurst: Is saying I'm adjusting the volume formula, so we'll see.

[00:32:46] Co-Host 2: Ian Schnoor : Okay.

[00:32:47] Host: Paul Barnhurst: It says I realized. And then what's interesting is it doesn't show you all the thinking, right? You see almost everything with trace light. And I kind of like that because sometimes it felt a little overwhelming. It again, it gets back to which one, which transparency.

[00:33:00] Co-Host 2: Ian Schnoor : Which version do you like.

[00:33:01] Host: Paul Barnhurst: Just be clear. Like this a little better of seeing the thinking. Sometimes I want to see step by step. It'd be nice, but it just I don't know if there's a way to compress it or, you know, go back and see the detail behind it. It's you're saying.

[00:33:14] Co-Host 2: Ian Schnoor : You.

[00:33:14] Host: Paul Barnhurst: Like, don't see everything.

[00:33:16] Co-Host 2: Ian Schnoor : Yeah, you're like that. You're seeing dots indicating that it's thinking, but you're not. You're not seeing huge amounts of paragraphs explaining it. You're saying.

[00:33:23] Host: Paul Barnhurst: Correct. Yeah. I don't want the paragraphs to come back, but I would love to be able to audit and go see them. And I don't know if that's, you know, there we go. It got it right now. Right. It did the right formula we wanted. Now it's 220. It matches here.

[00:33:35] Co-Host 2: Ian Schnoor : Well listen again it continues to validate. You know my notion that he still needs to know what the heck you're doing. I mean you need to know how to calculate these things. If you didn't, that was a pretty impressive formula it had in there before, which you knew was incorrect. But you know, I again, I continue to be worried about, uh, about junior professionals running using AI tools to, to to to get answers and not really having the background or ability to check and audit and make sure it's being done correctly. Right. And if you hand it in the prior one, uh, I don't think there's going to be a lot of sympathy from your boss. Right, Paul, when you say, yeah, but I did it. And you know the AI tool got it wrong. I don't think there's going to be a lot of sympathy with that answer, is there?

[00:34:21] Co-host 1: Giles Male: And definitely if you think to your point and you know, you still need to know what you're looking at and what you're doing. So, so maybe, I mean, what are the options here? If you know what you're doing, maybe you would do something through Power Query. You'd have most of the logic already set up versus you go to this sort of half finished template where you're going to choose to click an AI tool. I think people that know what they're doing are probably still going to default to the way that they operate, and they can trust, I don't know, it's interesting.

[00:34:54] Co-Host 2: Ian Schnoor : I just think we're in for an interesting time. And like you said, I mean, if you do use a tool like this, you have to be able to validate whether it's correct. Right.

[00:35:03] Co-host 1: Giles Male: And then when she did, it was one check, one prompt and then it's right. So that's impressive I guess.

[00:35:09] Co-Host 2: Ian Schnoor : I mean, the reality is here's the thing. The reality is spreadsheets are still one of the few open source software applications, right? The expectation. This is why I'm always concerned when people use overly complex, you know, formulas within Excel. When you use when you use a piece of purchased software, off the shelf software, there's no expectation that you're going to audit and check the code or that's fine, but the whole notion of using a spreadsheet for anything implies that it's open source, and you need to be able to understand, check, demonstrate, present, show everything going on in it. And  I worry that some people will move away or think that that's no longer the case when I think it might be more important than ever to know how to do that.

[00:35:53] Host: Paul Barnhurst: All right. So let's do the next one. This is the deferred revenue schedule. So last time we gave it some billing data where the customers were prepaying for multiple years. And we said, hey, build the waterfall schedule of how we're going to recognize the revenue. So here's what it analyzed. Let's just take a quick look, open the file it on my topics I can help with automating calculations. Deferred revenue schedules. It already said I could do it. So instead of me giving it the prompt since it came up and said, hey, do you want that? Let's see what happens if I just say, hey, go ahead and do what you said you're going to do? We're back and we're looking at the deferred revenue schedule. I did ask it to format some of the dates, but I was editing one of the cells, so I don't think it got it all done. It told me what it wanted to do, and it told me to exit the formula mode and it would do it, but let's walk through it anyway. We could see what that looks like. But if you remember, you know, one of the others we did, and I've seen this with other tools. Often we see them doing some hard coding for dates, but here it definitely referenced it. Right. And so that's good. Now here we'll just quickly convert this to date so we can kind of see that the dates look right. We've seen others where the dates didn't quite come through. Right. That all looks good. We'll do this since I didn't give it the chance to format it all and it all looks right.

[00:37:18] Host: Paul Barnhurst: What is weird? And, you know, you and I were talking about this Giles, as we were watching it, as we were waiting is the formula it wrote here. And so let me, you know, C4 divide. So C4 is the total amount. It's dividing it by the number of months. And so it's taking months from the beginning to the end. And so instead of using date diff or just taking the dates between each other dividing months, it's taking the year times 12 plus the month. So like 2025 times 12 plus the month. So you know some crazy big number minus the other. It works. It's not wrong, but it's just it's a very not intuitive way for a modeler. Right. If I handled this to an early stage modeler, they'd look at those formulas and go, what is it doing? I mean, we did for a while, right? Neither of us just said, oh, here's what it's doing. It took us a minute versus a traditional one would be like, oh, that's easy. We know what it's doing. The checks all look right. You know, um, one thing it didn't do is it didn't give us a total at the bottom. The other one did. So the other felt like there were some areas where it was a little cleaner. But here it was better with the formulas, which I would prefer. Right. I can clean the other up. So I do think it did a really good job here. I like it, I mean this is something, within a couple minutes I have a presentation ready, I can dump in more companies and it can do the work.

[00:38:47] Co-host 1: Giles Male: Yeah, I think that's pretty good. Yeah.

[00:38:49] Host: Paul Barnhurst: So solid. I want to try one last one here real quick. I have one more here I want to test. And anyone who's worked in FP&A has built many waterfall bridges. So it analyzed the data. It's already asked me, hey, do you want me to create a waterfall, All a step by step visualization setup. I'm going to copy my prompt and give it, you know, the opportunity to see if it builds a waterfall chart. Now, you know, as we're doing this, what's interesting is different tools definitely take different approaches. Like, you know, we tested Trace Lite previously. It does all its charts in Python. Some will write it again and will do it using the objects in Excel to create a chart. So let's see how Rosie does that. I'm curious to see, you know, kind of the approach it takes on this task. It didn't complete the waterfall chart. First I copied my prompt that I wanted it to use, then it hung up on that prompt. So I closed the file, reopened it, and asked it to build the waterfall it wanted. Based on the setup. It said, hey, there's two ways I can do it. I said, yeah, I want a waterfall chart. I don't want a placeholder chart. I want you to build the real thing. Stop for a few minutes. You can see here native waterfall, please. Never built it. This is an area where I think charts and some things there's some real opportunity because let's take a look at this. If I'm an analyst and I want to build this chart, I'm just going to come up here, insert recommended charts.

[00:40:15] Host: Paul Barnhurst: It's the second one. I have a waterfall. Now am I going to clean that up? Of course I am. So then I would come in and, you know, say, hey, I need to get rid of the gridlines, change the colors, probably get rid of this, all those things. But all right, put a chart title. Can it take me what, 2 or 3 minutes at max to get this hair looking like what I want. So obviously that's an area where it didn't do well. Not sure the reasons. I've tested this on several different tools and I've never been able to get it all the way there. Even here, it got one thing wrong that I'd have to fix, right? It didn't put this. This should be a total. So I want that there. So now they're both totals. So it really shows the start and the end. So obviously an opportunity where we'd love to see some improvement there. I've played with multiple different tools, different times, and I've never been able to get the chart quite perfect. I've got some to build it, so not sure what happened there, but disappointment is what I'd say at this point and something we can do on our own pretty easily. We're going to get set up to do some financial modeling examples, and we'll come back here in a moment. But we're through the first two sections so far. And before we jump into INS. Giles, any thoughts so far after seeing you know what six tests are?

[00:41:34] Co-host 1: Giles Male: It's mixed isn't it? I think there's some really impressive things. And then there's some things that it just doesn't seem quite as capable or, you know, such a good fit to, to create the solution for. So yeah, I'm still impressed. I'm still blown away overall by what these tools can do. But they're not perfect.

[00:41:55] Host: Paul Barnhurst: To in for some financial modeling use cases here.

[00:41:59] Co-Host 2: Ian Schnoor : All right. Thanks, Paul. Fantastic. We're going to get into some financial modeling. And I have brought my favorite modeling helper, Benny the modeling dog. Great buddy. Are we going to do some modeling together here? He loves financial modeling, and, uh, he's really pretty quick on the keyboard as well. But, yeah, we've got some great tests we're going to run here. Um, I'm going to have to put him down. He was barking, so I picked him up. But I'll put you down, buddy. Okay. You can say hi to everyone. And, uh.

[00:42:26] Co-host 1: Giles Male: Benny. No.

[00:42:27] Co-Host 2: Ian Schnoor : Yeah. Benny. Um. He goes. I hope that settles him down. Alright, let's do this. Let's dive in. We're gonna do some similar things that we tried on the prior tool. And, um, you can see here I've got the standard, one of our FMI Henderson model. This is the exact type of model that, um, our AFM candidates have to build from scratch on the exam. So again, if you haven't see the last one, this model includes a cover sheet, an executive summary sheet. It's got an assumption sheet. It has scenarios. And then the full this is a vertical. This model is built using sort of a vertical orientation style, and it's got all the pieces of the model, the revenues, the costs, the financial statements are underneath it, the income statement, cash flow statement and balance sheet, and then followed by all the supporting schedules the depreciation schedule, tax, working capital, etc.. Now I'm going to start my testing by looking at an existing fully built model. There are some problems in this model, and I'm going to start by asking it to help me understand what's going on and if it can figure out the answer to these problems. And then we will, um, we will get into seeing if it can build something like this from scratch. So then what we'll do is we'll turn to another Excel file here. Uh, this is just the historical three years. This is just the historical income statement, cash flow statement and balance sheet. And we'll see if it can transform this.

[00:43:53] Co-Host 2: Ian Schnoor : This is what this is what any modeler has to start with. Right? You guys would know any modeler who's building a model has to start with just the historical statements and build it up. And we'll see what Rosie can do on that front. So let's dive in here then. Let me open up Rosie. And the first thing I'd like to do is check for some issues. So I'm going to ask Rosie a couple of questions, including this I always like to do some checks. So, um, it realizes by the way, I didn't when it turned on. When I turned on Rosie, it recognized that it's a comprehensive financial model for Henderson Manufacturing. Even. That's impressive. Um, with detailed assumptions, scenarios. It's telling me what's in here, and so it's even suggesting how it can help me. So, um, that's pretty cool, actually. But let me just start by asking, number one, are there any hidden sheets in this file? I mean, it's not one thing I'd like to know. It doesn't take very long to look for them. I did that in the last test, but it's thinking and it says, um, it did this pretty quickly a few minutes ago. I just want to know, are there any hidden sheets I should be aware of? And it says, yes. Look at how quickly that thought. It thought for seven seconds. It's impressive. It's impressive, I think. Right. It's, um. It says yes.

[00:45:09] Host: Paul Barnhurst: Check me out. I only took seven seconds. Look at that.

[00:45:13] Co-Host 2: Ian Schnoor : I know it's it's flexing right now, but I'm impressed. It knows that there's a sheet called Hidden and Clandestine. It's a very hidden sheet, which I talked about last time. Would you like me to unhide either of them? Let me. Let me see that. It's it's offering. It's offering to. Should I ask nicely? I'll say.

[00:45:30] Co-host 1: Giles Male: Yes. Definitely. Always. Good.

[00:45:31] Co-Host 2: Ian Schnoor : What if I ask nicely and say can you please unhide them? And I'm saying yes, please. Let's see if it it can take action quickly. What do you guys think? Will it be able to do.

[00:45:42] Host: Paul Barnhurst: If there is a study they did where there are two different things I've seen kind of conflicting. One says you generally get better results if you'll say please and thank you.

[00:45:50] Co-Host 2: Ian Schnoor : Oh come on, really.

[00:45:51] Host: Paul Barnhurst: Human for conversations. And another that said you got worse results. You get better results if you're mean to it.

[00:45:59] Co-Host 2: Ian Schnoor : Well, it's not my instinct.

[00:46:01] Host: Paul Barnhurst: Uh, that was really interesting. You basically need to tell it, like, just shut up. Don't give me all this extra fluff and give me what I want. It was really interesting. There's really. When you're very direct, almost curt with it, sometimes you get better results. And then someone else said your conversations tend to be better. You're willing to go further when you say yes, please and thank you, because you treat it more like a conversation with a human, and you're willing to go back and forth with the tool.

[00:46:26] Co-Host 2: Ian Schnoor : Isn't it interesting? Well, I mean, it's seemingly, uh, was not, you know, being polite, being polite has not exactly won the day here for me, has. I don't know where it's at.

[00:46:39] Host: Paul Barnhurst: Be a jerk. Say you haven't unhidden them. Get to work. I want to see these files.

[00:46:43] Co-Host 2: Ian Schnoor : How about I say yes? I said yes, how about that? Is that rude enough? Paul, is that aggressive?

[00:46:49] Host: Paul Barnhurst: I think you could do better. I guess for Canadians. That's not bad.

[00:46:52] Co-Host 2: Ian Schnoor : That's fine. Let's try it. That's pretty aggressive.

[00:46:54] Host: Paul Barnhurst: I mean, you know, us Americans are aggressive.

[00:46:57] Co-Host 2: Ian Schnoor : Let me try that. Let's see. It just and it didn't really tell me anything. It did not unhide them. And it, uh, it's clarifying. Um, and it's so it got it. Do you want me to unhide both hidden sheets or just one of them? Now you want me to? I'll say both exclamation marks and see what it does on this front. Um, let's see if it does it. Because again, it very.

[00:47:23] Host: Paul Barnhurst: Yeah. I would've probably said something like, are you stupid? I said both.

[00:47:25] Co-Host 2: Ian Schnoor : Oh, really? Oh, that's that's aggressive. That's aggressive. I don't know how I didn't want it to get annoyed at me so quickly, but it's creating a plan. It is creating a. Now if I had to unhide this script, I wouldn't need a plan. Unhide the sheet anyway. I'm just kidding it. It's creating a plan.

[00:47:43] Host: Paul Barnhurst: B super easy right? Anyone would get the first one. The second one you have the very hidden. You'd have to know to look in VBA. The first one anybody would get within.

[00:47:53] Co-Host 2: Ian Schnoor : Well it knew but but it it it it somehow it knew by looking in the VBA that there was a very hidden sheet here. So now it's it's it is aware that it is there and it's it's okay. Well, maybe we'll stop this obviously. Um, and move on. What do you think?

[00:48:09] Host: Paul Barnhurst: Because I think so. It's taking longer than one would expect.

[00:48:13] Co-Host 2: Ian Schnoor : Well, and also it's taking significantly longer than, uh, right. Than it would take. Um.

[00:48:20] Host: Paul Barnhurst: Correct. That's what I mean when I say then we would expect. Right. It's a fairly simple path.

[00:48:25] Co-Host 2: Ian Schnoor : There we go. It is now did it right. It's now done it it took a minute. It says it took 55 seconds plus the previous. So it took a couple of minutes and it managed to unhide both sheets. Perfect. Okay. So it did that. So I'm, um I'm I'm pleased it you know, it would have been faster to do that manually, but that's okay. Um it is clear that I was able to figure that out. That's great to see. Let me ask it now I have um I, I have let me try this one. I want to try this. You know, another thing that I always like to look for if there's any white values. Are there any white values on this sheet? Let's ask it that question. Um this one, it responded pretty quickly last time as well. And let's see if it gives me the same answer. Um, because it would be nice to know. And it says I'm trying to figure out what the user means by white values. Um, so somehow it didn't understand what I meant. It's says it's planning a color detection strategy here. It's actually thinking. Right. Um, it's thinking, do you mean cells with white font? Yeah. I'll say yeah. Yeah. Yes. Uh uh, are there cells with white font? Maybe it didn't like what I asked.

[00:49:38] Host: Paul Barnhurst: And it asked, is it okay to add a temporary audit font colors tab with the results? Let's see what it does. We didn't, uh.

[00:49:45] Co-Host 2: Ian Schnoor : I know.

[00:49:45] Host: Paul Barnhurst: I.

[00:49:45] Co-Host 2: Ian Schnoor : Just exploring I need to check the model for, uh. What is it saying here?

[00:49:50] Host: Paul Barnhurst: I need again this is something that as a person we could do. It's taking a lot of thinking. So is it right this gets back to is there a way we could have been more, you know, um, detailed in the prompt? Did it get it to do it, to do it faster or is it just sometimes AI has to think through it a lot to do it.

[00:50:12] Co-Host 2: Ian Schnoor : Maybe. But I think this is a pretty clear question, don't you? I don't, I don't have any other. I can't think of a simpler way.

[00:50:19] Host: Paul Barnhurst: I mean, I think you could say a simple way instead of saying white values, say there are some thoughts, there's white font. Please identify what cells the white font is in. Right, right. That's what I mean by you could potentially get a different result trying.

[00:50:35] Co-Host 2: Ian Schnoor : Identify the cells that, uh, have.

[00:50:39] Host: Paul Barnhurst: That contain white font on this sheet.

[00:50:42] Co-Host 2: Ian Schnoor : Please identify the cells that contain white font color. I mean, let's just see what that does. I'm not sure. Anyway, we're going to stop this because.

[00:50:53] Host: Paul Barnhurst: Yeah, stop that one.

[00:50:54] Co-Host 2: Ian Schnoor : It's obviously not, you know, it's doing something. Uh, previously when I ran this before we started, I ran this test. It actually said to me that it was not able to check for colors. So I don't know if that's always the answer that I was going to get. It's not told us that on this particular test, but it's still um, it was still struggling a little bit about that. Now I want to look for the next thing I want to show is there are a couple there are a couple rows in here that have problems, including this row where there is a hard coded value at the end of the formula. So what I want to ask is, um, I want to ask it is there, are there any um are there any inconsistent formulas on this sheet. And let's see what it comes back with on um, with regards to that question, because this one, it did pretty well the previous time I asked, I was looking for inconsistent formulas. And um, so it's doing something here to check the file. And yeah, it seems to be in row 13. And row 13 we know is an important row that has inconsistency. So it's already discovered something in row 13. And it was pretty quick last time at uh but I think it's helpful. But I think it's also I mean in this particular way it's not bad, you know, um, for people to see what it's doing. Right. I mean, it took, it took uh, a little while to. It is. Yeah. So it does. No, the short answer is yes. There are several spots on models where formulas differ across the horizontal series. Most look intentional. Um, but they are inconsistent relative to neighbors. Notable examples. So in row seven. Row seven is the year okay. It's pulling up something there.

[00:52:38] Host: Paul Barnhurst: The pricing the one of them. You format it differently. You have the actuals and then you don't have it right.

[00:52:43] Co-Host 2: Ian Schnoor : Right. That's pretty good. Pricing row. This is the problem row. It says M and O contain constants. So that's great. I'm really happy about that. It figured that out. It discovered that this cell has a constant, as does this one. That's excellent. Um, there's another row, though, that I didn't show you yet. That has a problem. I'm going to show you. It's a classic modeling error. Sales volume. Operating rate. Set up J20. Um, j.

[00:53:07] Co-host 1: Giles Male: Oh, it's the back calculation of the volume.

[00:53:10] Host: Paul Barnhurst: Noticing you calculated the first.

[00:53:12] Co-Host 2: Ian Schnoor : That's fine. So that one I should probably just put in a box. This is historical. So it realizes that the historical is a different formula than the forecast. That's okay. And then several first year. But there is one that did not roll up. So these are all deliberate. But there is one thing that's a classic error that is and that's here. This is a very common modeling error. This happens especially when people use their mouse. So you can see here on the income statement. The revenue has to pull from the revenue schedule. And of course this one is linking correctly. It's linking to cell K 26 I'll show you. That's correct. That's the right link. But sometimes, especially when people use their mouse or they're not copying across, this one is k26. And then, you know, I might have changed or fixed something. And this one is also linking to K instead of L, right? I might have done this one manually and then tried to link this one up. And if I was using my mouse and I'm not oriented to where I am, it's not hard to. It happens all the time. It's not hard to link on the wrong column. So now I have two years that are linked to the same one, and then the rest of them are all off. This should be M, it's L it's, etc. um, I ask because it was previously. Are there any other inconsistencies?

[00:54:24] Host: Paul Barnhurst: You could also ask it to analyze row eight since we know 83. If it doesn't get it here and just see if it picks it up. If you basically point it to it.

[00:54:32] Co-Host 2: Ian Schnoor : Yeah, it actually solved it last time on its own when I did this. So I'll see if it finds it.

[00:54:36] Host: Paul Barnhurst: It was prompts, isn't it, that that's part of what's so wild about because we know it's generative AI right. You never get the exact same thing twice.

[00:54:45] Co-Host 2: Ian Schnoor : Right. And so I know. So here's the thing I know. Sorry.

[00:54:50] Co-host 1: Giles Male: No, I was just going to say it's interesting that we've all we've all we all keep having that experience. We try something beforehand just to see how it runs. And then the same prompt gives you different answers. Um, and sometimes you just can't do it. I think that's the more surprising thing, getting slightly different answers I would expect with everything I've learned about Llms. But sometimes where it just goes, it just breaks and doesn't give you anything.

[00:55:13] Co-Host 2: Ian Schnoor : There's a lot in here, you know? I see more places where the form. Yes, I see more places where the formulas differ across rows. It thinks it looks. Can it be appropriate? Like it was deliberate. I'm looking. I thought I saw something for row 83 in my first year. It's talking about the seed versus rolling calcs. No. Yeah. It's something it hasn't seemed to recognize. So we'll leave that one at that. I have an issue where it's. Yeah. Where these two are the exact same. Let me fix this one though. This is K but I guess the point is simply asking it. And we saw this last week as well. Simply asking it to look for errors or inconsistent formulas might find something, but you're not necessarily guaranteed that it's found everything unnecessarily. And so? So you still need to be able to check things. Um, this balance sheet also does not balance for the same reason I showed last time. It does not balance because the change in working capital, uh, is the wrong way.

[00:56:09] Host: Paul Barnhurst: The formula is wrong, right?

[00:56:10] Co-Host 2: Ian Schnoor : Formula of change in working capital is wrong. The the change in working capital formula is going. A classic modeling error. It's going the wrong way. It's taking the current year minus last year whereas it should be the opposite. And I'm just going to ask about it. So again if I'm young, if I'm a new young analyst and I'm staring and my boss says, Ian, we just received this client model and the balance sheet is not balanced. Go find it. You know, I'll ask my tool here to say, why doesn't the balance sheet balance? And it's my starting point, right? It's my first go to option. Say why doesn't the balance sheet balance. And I hope that it can find something. It's inspecting. It's checking. Um, you know, what it seems to do is it shows us some of its thinking and it takes it away. Right. And then it comes back to the dancing dots here, which is fine. Um, I'm analyzing it's doing something. I mean, listen, I recognize this is a very difficult task. It would have to understand model construction, model design. It has to understand how every formula needs to work to be able to determine that that one formula on change in working capital is wrong. I recognize that this requires an enormous amount of knowledge and insight to pinpoint, um, why this is off, but it's, um, let's pause and and we'll come back in a, in a couple of minutes and we'll see where it gets.

[00:57:33] Co-Host 2: Ian Schnoor : All right. We are back. It only went for another 70 minute. And it says the short answer. The balance sheet is out of balance. Um, in the forecast columns. And it confirms that through the check. So it knows it's looking at some checks. Again this is very difficult to do which requires a fair bit of knowledge. Um, it seems to think that what's driving it is cash versus financing. And there's some explanation. There is the mismatch grows so it realizes not just rounding. Working capital lines don't look consistent. Um, but it's not quite getting to the answer. It is uh, so it's giving me some ideas and suggestions. It's not, of course. Of course. The only reason it's out of balance is because I'll go to the bottom here. Because of a classic, classic modeling error, the working capital should have been the previous year minus the current year. Right? The change in working capital represents the change in cash as a result of working capital changes. The sign was the wrong way, happens to people all the time. And simply by fixing that, uh, my balance sheet is now fully balanced and working properly. So it was one small issue that, you know, a strong modeler would be able to detect fairly quickly. So that's that wraps up that part of the test. I want to move into a build a example. Now did you want to.

[00:58:50] Host: Paul Barnhurst: Yeah, I want to add one thing. I thought it was interesting, you know, at the bottom, would you like me to create the audit sheet to show it and then fix the driver logic? You know, typically by setting the revolver as the cache plug to a target minimum cache or simply hard plug class to force the check row to zero. Interesting that it gives kind of those options.

[00:59:10] Co-Host 2: Ian Schnoor : And it's and it's really and it's yeah I, I'm glad you pointed it out. So the right one option is an audit sheet. Uh, this seems like it's going to do a lot of work, as the cache and, and my, my suspicion is it would have forced cache like it would have plugged something on the balance sheet maybe. I'm not sure. And then this one says, do you want me to do that thing? I'm never allowed to do the thing that you should never, ever do in a model. Should I just simply hard plug cache? It's like. It's like.

[00:59:41] Host: Paul Barnhurst: This is not recommended. Not recommended.

[00:59:44] Co-Host 2: Ian Schnoor : It's like a.

[00:59:44] Host: Paul Barnhurst: Little.

[00:59:45] Co-Host 2: Ian Schnoor : A little.

[00:59:45] Host: Paul Barnhurst: Uh.

[00:59:46] Co-Host 2: Ian Schnoor : Like that little devil on your shoulder that comes at two in the morning when you're like, just plug it. Just plug it. Just go to bed. Plug it. Um, so it's it's giving me that option. Uh, I probably shouldn't take it, but again, um, as a, as a financial modeler, I know that I'm not supposed to take I, you know, I worry about I worry about people without modeling skills trying to do this and not knowing what direction to go and saying, yes, but, uh, okay. Well, that that is, that is that why don't we move over then? And I'm going to actually going to clean it up here. Uh, why don't we just go back to the beginning and I'll start a new conversation, a new conversation, and I'm actually going to go into the other file here. This is so this is this is just the three years of historical financial statements. This is exactly how most modelers start their models is with three years of historical data, and then you have to build in sheets for assumptions, sheets for scenarios. And this is exactly what people have to do, you know, on the AFM exam, right, to prove that they can do it.

[01:00:50] Co-Host 2: Ian Schnoor : And it just hones your thinking. So let me take, um, a prompt here. Let me just see the starting point. Let me just see. I've said here, just like build a five year forecast model on the model sheet, this is the model sheet for the years 2025 to 2029. And I've said build separate schedules, build separate schedules for revenues, cost, depreciation income tax working capital debt and equity. So so I obviously I had to even know enough to say do that or I'm trying to give it. You know, you have to know enough to know that you need schedules. And then I've literally said make reasonable assumptions because I don't care what it uses for growth rates or margins, um, or days. I just want to see that it can understand the design and construction. So let me run this. This will take a few minutes. So I'm going to click go. Why don't we, um why don't we stop this Paul and come back in a few minutes once we've got something to report back on.

[01:01:40] Host: Paul Barnhurst: All right. Well, we'll go ahead and pause and see how it does. We'll be back. All right, we're back. It ran for about six minutes and it really struggled on this task. But it did ask us a few questions. We'll try a little bit more for here. But when we first kind of maybe end just show a little bit. What are your first thoughts. It feels like it really struggled here.

[01:02:00] Co-Host 2: Ian Schnoor : Yeah it did. It only took six minutes as you said. But you know, I deliberately asked it. As you recall, I was pretty open ended and said, just build a five year forecast model and try to build schedules for each of the key components of a model. Um, and I just said, make reasonable assumptions. So it's come back. Yeah. I mean, this is obviously needs work and it's asking me a couple more questions. I'm going to go back and run some more. I'm going to try and clean it up. But it's it's kind of, you know, broken up the income statement a little bit. It is added a bunch of rows. It's got, uh, you know, some stuff off to the sides. It put in five year forecast. Sort of worked on the income statement. The cash flow statement sort of um, and balance sheets are not complete. There's nothing underneath it yet. So why don't I come in and, um.

[01:02:50] Host: Paul Barnhurst: Maybe we can ask it to put the assumptions on a separate sheet and then complete everything.

[01:02:55] Co-Host 2: Ian Schnoor : So why don't I say, please, uh, put all the assumptions on a different sheet, then finish forecasting the. Financial statements. Okay, let's come on back in a minute, Paul, then and see what it what it does at this point. Okay.

[01:03:16] Host: Paul Barnhurst: All right. We're back. Um, the bottom line is and I'll let Ed speak to it a little bit. We found the model struggled to build the model. Right. Kind of your take there in on that part.

[01:03:27] Co-Host 2: Ian Schnoor : Yeah. And again we're not totally surprised I mean it's it's it's done some impressive things. But what we're seeing is, you know having more guardrails, more direction, having more formulas to fill in tools we've looked at so far, you know, have are doing some impressive work. Have a bit of an easier time. It was struggling to build a full three statement model with little guidance. I kind of said do what makes sense, pick reasonable assumptions. And it really was not creating anything that, uh, that we would find usable. And again, I don't. I wouldn't have cared if it used silly assumptions. I still just mean, one of us could build, could have some pretty basic assumptions and build a five year forecast pretty quickly, mechanically, logically understanding how the numbers talk to each other. Right? Understanding the discipline of the build approach to build it, link it in, get the pieces connected and then you can refine your assumptions later. So it was struggling with that and not generating anything that was worth showing. Um but again that's not not not a huge surprise. And I think we uh company even talked about that. Is that right? Um, that, that that's.

[01:04:32] Co-host 1: Giles Male: Yeah. I mean, I remember speaking to Dennis a while ago. I had a really long Dennis.

[01:04:37] Host: Paul Barnhurst: Dennis is the founder, right?

[01:04:38] Co-host 1: Giles Male: Just the founder. Yeah. So. So he was really open and honest and very approachable with all of us, I think. I think he was very open with the fact that it wasn't ready as a financial modeling tool, and he wasn't claiming it was going to do full financial models. He was actually very keen to get kind of models from me when I spoke to him, to kind of help the tool learn. So I'm not massively surprised that it's not really built as a three statement model.

[01:05:05] Host: Paul Barnhurst: What I appreciate is he admits that I've had others that like, it's just not ready to build a full model. And I think this demonstrates so far they're not ready to build work ready models. But we're early.

[01:05:18] Co-Host 2: Ian Schnoor : Early days. It's doing some great work, early days. There's nothing to suggest it won't. But as I said, even if and when it does, you have to understand everything it's doing. You're going to need to check it. I do not believe there's a day anytime soon where your managing director is calling up models, um, into AI. I mean, there's still someone who is going to have to be responsible for building models and if you can use AI to help you. Fantastic. I'm a big fan of that. Uh, and then being able to check them, double check them, AI is actually going to be great at helping you check them. So you can use it as your partner. You know, what I did though do is because it struggled with the model, but I asked it to add a. I asked it to go to the completed model because it seems to have more success with, you know, when there's already structure in place. So I asked them to build a DCF for me at the bottom by using a weighted average cost of capital of 10% and a terminal growth rate of 2%. That's all I gave it, and I wanted to see what it did. And it actually did quite a good job on this. I mean, it's not quite client ready, but it made one mistake which I had to fix. It was using the wrong cell for the tax rate. So it grabbed the wrong one. And I said, I think you used the wrong cell for the tax rate. And then it figured it out. I realized I like it. Look how friendly it is, Paul. It said, good. That's because I'm being nice to it. It said, good catch.

[01:06:31] Host: Paul Barnhurst: You're right. You're rubbing that part in that. I'm not nice. Fine. Absolutely.

[01:06:36] Co-Host 2: Ian Schnoor : So it did, though correctly. And now that's all. It's built all into one cell. Now, as a modeler, I need to understand what it's doing. It's taking the bit. It's multiplying it by one minus the tax rate. It's adding back depreciation, um, and the working capital and taking out the CapEx. And then it has properly calculated a discount factor for the appropriate number of years automated. I like that a lot. Uh, most people for the exponent in their, in their discounting and in most people's discounting exponent to discount years. They would have a hard one here. And in this one they would have, you know, a hard two or they would have a row above that had the one, two, three, four, five, which is fine. This is using Excel's column function to automate that, which is very nice, but all it's doing, this entire series of columns is just getting to the exponent, the discount years, the one two, three, four, five, it's doing it correctly and then it's calculating the present value of the cash flows. I like that of the five years in the forecast period. It calculated the terminal value by taking um.

[01:07:39] Co-Host 2: Ian Schnoor : Now, I'm not a huge fan of this approach, but I didn't give it more direction. And I think if I did, it would have done whatever I wanted. It took the fifth year's cash flow. It pushed it out by one year. It grew it forward a year into the future, and then it used a perpetuity, um, to, to discount it and, and take the present value, which gets it back one year and then it further brought it back, uh, five additional years. And then the enterprise value is adding it all up. So there's certainly some tweaks, but that's an impressive piece of work. Uh, I'd love it to ask it to do a levered cash flow for me to get to the equity value. Um, and, and compare that against the equity value that arrives using, uh, the unlevered approach. So there's a lot of things I would push it forward on, but we'll do that on another time. Uh, for now, this is, uh, something I think it did quite well at much better than its ability to build up a full model from scratch. So? So I think that's good for now then, Paul.

[01:08:31] Host: Paul Barnhurst: All right. So I think uh, that's where we've gone through. And we've tested 9 or 10 different use cases here. We've done some modeling, some FP&A There are some charting in there just analyzing data. And you know, I'll give my kind of final thoughts and then we'll go around the horn. And my final words, I think it's an impressive tool. I think it did some things really well. I think it struggled in some other areas. You know, some other tools I've seen have done better in things. But what what is clear to me is if you know what you're doing, there are areas that can do things and make your life easier. There are other areas where it's not quite ready yet, but I thought on this, especially the Excel cases, it did fabulous. That did really good, right? When it has that structure and the way it gave those formulas you could learn a lot from. So those are my takes. Um, in your final take.

[01:09:22] Co-Host 2: Ian Schnoor : Same thing. I mean it's still again, I still do give them tremendous credit, all these tools. Incredible to think that you can use English language prompts and get something to build a spreadsheet for you. But again, my strong recommendation for anyone. I would highly encourage everyone to start playing around and testing these things and getting comfortable. Uh, the future analyst will need to know how to use AI in their work, but my strong view today, and I don't think this is going to change anytime soon, is we need to view them as our partners. Um, not as our it's not a competitor to you. It is your partner that can hopefully assist you and help you. Um, and, uh, and if you think of it that way, I think there's great things in store for you and for all these tools. Charles.

[01:10:07] Co-host 1: Giles Male: I think I've got two thoughts. I think if you think of, like, the job of reviewing a financial model, I use a tool called oak all the time to, you know, help me review models. I could see myself using a Rosie or another tool. Or maybe it will be, you know, the agent in Excel to do some additional work, like a partner. Um, so I think that's great. I've forgotten what my other point was. It's really important as well.

[01:10:36] Host: Paul Barnhurst: We're not leaving until we get a second out of you guys.

[01:10:39] Co-Host 2: Ian Schnoor : It's late in the UK. We're forcing you to stay up late.

[01:10:41] Speaker7: Yeah. So what was my other point? You had an auditing.

[01:10:45] Co-Host 2: Ian Schnoor : Tool you used. You said you loved your auditing tool.

[01:10:48] Speaker7: I'll let you, um.

[01:10:50] Co-host 1: Giles Male: I think it's gone. Whatever the point was, it's gone.

[01:10:52] Co-Host 2: Ian Schnoor : Um, take one of mine. It's fine, I don't mind. You can copy. I.

[01:10:57] Co-host 1: Giles Male: I I'm I'm I'm. I guess I'm still where I was before. I think there's some really nice strong signs. And there's no part of me at the moment that is that is just ready to go, right? This is going to replace what we do. But I think that was our gut feeling before we started this. And everything we're seeing is kind of cementing that view for me.

[01:11:18] Co-Host 2: Ian Schnoor : Yeah.

[01:11:18] Co-Host 2: Ian Schnoor : It's a tool right. It's a tool. Uh, it's a powerful tool. Like Excel was a tool 40 years ago when it came onto the scene, like calculators were a tool. Um, you know, and I believe that at least for the foreseeable future, I think, like you guys, um, those who rise to the top will use it to supplement their work. Paul.

[01:11:37] Host: Paul Barnhurst: I think that's a great point to end on. Look, it's a tool. It's a tool. If you're not using your tool belt, shame on you. You should be finding ways to make it.

[01:11:45] Co-Host 2: Ian Schnoor : Yourself more effective.

[01:11:47] Host: Paul Barnhurst: Shame I'm publicly shaming you if you're not.

[01:11:50] Host: Paul Barnhurst: Finding.

[01:11:50] Host: Paul Barnhurst: Ways to be more effective. So there you go. See, the American in me is coming out like I'm gonna. I'm way too nice for that. And Giles, like, it's late at night. I don't care what they say. So on that note, we're going to call this a wrap and we'll be back again for another round of testing. So if you have questions, you're curious about what we did. If you'd want to reach out to us, please do. We'd love to hear from you. We'd love to know what you think. And thanks for joining us.

[01:12:17] music: The mod squad. We are the mod squad.