How Finance Pros Should Manage Data Context and Token Forecasting
In this episode of Future Finance, hosts Paul Barnhurst and Glenn Hopper discuss AI fatigue, better ways for finance teams to use AI, and the growing challenge of forecasting AI costs. They explore why finance leaders should focus less on chasing new models and more on building the data, context, and workflows needed to get real value from AI.
In this episode, you will discover:
Why finance teams should stop chasing every new AI model.
Why context is becoming more important than prompting alone.
How better KPIs and data foundations improve AI results.
How organizations can forecast and manage AI token costs.
Why AI success should be measured by business outcomes.
Paul and Glenn explain why effective AI adoption isn't about using the most tokens or knowing every new tool. It's about using AI to help finance make better, faster decisions.
Follow Glenn:
LinkedIn: https://www.linkedin.com/in/gbhopperiii
Follow Paul:
LinkedIn: https://www.linkedin.com/in/thefpandaguy
Disclosure: Portions of this episode (such as the introduction or promotional segments) use AI-generated voice narration produced under human editorial review.
Future Finance is sponsored by QFlow.ai, the strategic finance platform solving the toughest part of planning and analysis: B2B revenue. Align sales, marketing, and finance, speed up decision-making, and lock in accountability with QFlow.ai.
Stay tuned for a deeper understanding of how AI is shaping the future of finance and what it means for businesses and individuals alike.
In Today’s Episode:
[00:00] – Trailer
[03:17] – AI Fatigue & Model Overload
[09:38] – Beyond Prompt Engineering
[16:03] – Building Better AI Context
[24:29] – Organizing Finance Knowledge
[27:29] – Token Maxing & AI Costs
[32:06] – Forecasting Token Spend
[38:23] – Focusing on Business Value
[41:43] – Closing Thoughts
Full Show Transcript:
Host: Paul Barnhurst (00:39):
Welcome to the Future Finance Show, where we talk about treasury management on bars. Future Finance is brought to you by qflow.ai, the strategic finance platform solving the toughest part of planning and analysis. B2B revenue, align sales, marketing, and finance seamlessly, speed up decision-making, and lock in accountability with qflow.ai. Welcome to another episode of Future Finance. I'm your co-host, Paul Barnhurst, the FP&A guy. And today I decided to dress like I'm the FP&A guy. What do you think, Glen? Did I do pretty well?
Co-Host: Glenn Hopper (01:32):
Honestly, I'm glad I wore a collared shirt because I still am underdressed, but you must not have gone outside today. I don't know what the weather is in Salt Lake City, but I'm looking at a crisp 96 degrees outside here right now. So you're lucky I'm wearing a shirt at all.
Host: Paul Barnhurst (01:45):
I'll tell you where we're at here in one second. So I'll give the backstory because I think our audience will get a laugh out of it, then we'll get to the show. So it's 92 here, so not very different from you. So we did a training this morning, our best-practice FP&A course, Ron Montero and I. And he saw me dressed up. Ron's like, he saw my course. You never dress like that anymore. You need to wear that outfit next time. And so I did. That's where it came from. So that's why I'm dressed up like this today. It was more his, I was going to say, whining, but his prompting. I'll be nice.
Co-Host: Glenn Hopper (02:16):
Well, I'll say with the vest and the beard, you look like a dapper like Hell's Angel.
Host: Paul Barnhurst (02:21):
What I'm trying to look like is the old-fashioned barber or an old salon, like the Barber Quartet or maybe the bartender from 1800s Western Salon. There
Co-Host: Glenn Hopper (02:33):
You
Host: Paul Barnhurst (02:33):
Go. That's
Co-Host: Glenn Hopper (02:34):
It. Yeah.
Host: Paul Barnhurst (02:35):
I think I'm more that than Hell's Angels. I don't have the tattoos or anything to quite pull the Hell's Angels off. All right. Before we jump into what we're going to cover today, we're going to be switching our format up a little bit. So let us know what you think. We're going to go more and more toward Glen and I with occasionally bringing in a guest. It may be part of an episode versus full guests. And we want to talk more and more kind of practical finance things around AI. So we'd love to hear from you. If you like that direction, you think that sounds great, enjoy this episode. Ping Glen or I and let us know because we're always looking for feedback. That being said, we're going to start today. Talk a little bit about the news. And so I'm going to go to you, Glen, first.
(03:17):
What's some of the AI news you are seeing? Something you want to talk about on that front? Let's start there.
Co-Host: Glenn Hopper (03:23):
It's exhausting, isn't it? Trying to keep up with everything.
Host: Paul Barnhurst (03:26):
Did you see secret CFOs post on the exhausting part? You know who secret CFO is, right?
Co-Host: Glenn Hopper (03:31):
Oh yeah. Yeah.
Host: Paul Barnhurst (03:31):
So this would've been when Fable five first came out. He put out a post that said, "Here's everything you need to know about Anthropic's model, Fable five." And then there's a bunch of lines, "Nothing. Get back to work." And then he made the point like you don't need to pay attention to the latest model. Did anyone do this every time a new Excel formula comes out or any other technology? Seriously, what percentage actually would know when a new Excel formula came out? 2%? But what percentage of people are freaking out every time a new model comes out?
Co-Host: Glenn Hopper (04:03):
To me, at this point, there's so much parity across the models. It's the wrong conversation to be having. I think that's why I say it's exhausting. Chasing models and chasing whatever the latest benchmarks. We've been saying for a couple of years, if this is the best AI we ever get, it's only going to get better. Meaning if the models don't get any smarter, AI usage is just going to get better from here on. And we're seeing it in 2026 with one, yes, this is the models being smarter, but it's also architecture around them. But the fact that the models can now go and do long running tasks. So we really have true agents going and doing things in 2026. For the first time, we've been talking about agents. I think it's already crested the peak of the Gartner Hype Cycle AI agents. But we're finally seeing results from that.
(04:53):
But the problem is who's going to build them? You have to put the guardrails on, you have to do the harnesses and all that. And now every time, it's not even the model releases. It's, oh, Claude now has a skill. By the way, it's only available on a Mac, but you can screen share with Claude. It will learn a skill based on what you're doing. And all this stuff is cool, but so is the internet. All the software that we've used, it's so where we are right now to me, and we can talk about token maxing, which is probably one of the bigger AI stories this year. To me right now it is yes, learn how to use the tools, but think about all the SaaS software we use and every time it updated, or for that masker, take it into consumer land. Think about every time Facebook.
(05:41):
I'm not on Facebook, but I remember for years, every time somebody changed something with Facebook, people would say, "Well, I'm not using that anymore." They keep changing everything on me, but people figure it out.
Host: Paul Barnhurst (05:51):
Isn't in the same place anymore.
Co-Host: Glenn Hopper (05:53):
When my crazy conspiracy theorist great uncle can figure out how to use Facebook, I feel like we can figure out how to use AI. So obviously you have people who haven't used it. We want to show the tools, explain how to use it, what to do with it. But at some point that I need a trainer to come in and do this, and maybe I'm getting myself out of a big portion of my consulting work, but at some point it is, I've got the basics, there's a new feature, I need to learn how to use it. And if I don't know, maybe I could ask AI.
Host: Paul Barnhurst (06:24):
That's a great point. So I'll share a little bit of my news and I want to talk a little bit of tips. Then I want to come back to TokenMaxing because I think I want to talk about some practical things around forecasting because I guarantee you everybody's struggling with that. It's a real challenge right now. So the fatigue thing is different at the moment. The fatigue for a while was, hey, the latest new model came out. I think people now are getting a little bit of, okay, it's a new model. Yeah, maybe it's a little better. All right, great. ChatGPT has this. Claude has this. LAMA has that. Whoever it is, whatever the main players are. I think we're heading into the phase where the challenge now is how do I put together something that works and there's fatigue in, oh, there's co-work. Oh, there's chat.
(07:16):
Oh, there's this harness thing. Oh, there's these skills. Oh, there's these MD files. Oh, well, does my copilot have that? Because my coworker told me I need to do a skill, but Copilot doesn't have it. Oh, ChatGPT calls it this. Well, I hear claude code, but ChatGPT is codex. And I think there's real confusion on how to start get the benefit and bring it together. I was just teaching today students and explaining skills. And I think the idea of a harness, and at several points they're looking at me like, I'm trying to write a prompt. Who are you and what are you talking about? And then there's other students, oh yeah, tell me more. I'm at that point. And so I feel like the fatigue right now is trying to make sense of the best way to use it and really getting value versus the fatigue for a while was holy crap, this stuff just keeps moving so quick in the sense of new model and it can do what?
(08:10):
Am I off on that? That feels like that's kind of the fatigue we're at.
Co-Host: Glenn Hopper (08:14):
Yeah, no, 100%. That's exactly it. I remember I met in the early days when I was talking around AI and finance, I'd meet a lot of, I'm doing air quotes here, a lot of AI experts and there were people who - AI. Yeah. Their AI expertise meant they figured out early how to prompt and how to get the best results from it. And they were on the leading edge. But these AI experts were, all their time was spent not learning the fundamentals of AI and machine learning and everything that goes below it, but it was, let me keep up with all the latest news so I can be, every time I talk to someone, I can quote benchmarks and I can quote the features from each one and I can talk about all the latest new tools. And then you'd end up with all these people who you talk to them, they're a mile wide and an inch deep.
(09:11):
And that was exhausting, was trying to keep up with the benchmarks and all that. But the rest of us, as we're hearing, okay, AI is a thing. We understand it's going to change the nature of work. All my employees are using it. Great. I don't know what they're doing with it. It's how do we implement it? How do we just make it? It's part of our workflow. It's part of our team. How do we get the best results from it and stop chasing whatever the latest news release is?
Host: Paul Barnhurst (09:38):
Yeah. So it's funny, I'm going to share something for a minute. So those that watch us on YouTube will be able to see this. If not, I think you'll be able to follow along. But I think this really gets to something you said. Early on, prompting made a big difference. Today, the reality in some cases, AI can write your prompt. If you're struggling to write a prompt, tell AI you need help prompt using best practise about what you're trying to accomplish. And that prompt 90% of the time will be better than the prompt you would. Agree, Glen? Yes. Right? So prompting is no longer the skill people need. If you were paid 250,000 a year to be a prompt engineer, odds are your job's in trouble. I hate to break it to you. But what I want to share when you speak of that, so I did a couple models, and let me see which one this is.
(10:29):
So this model, I used a really detailed best practise prompt, like a page and a half. And so I got a cover page. I got this lease versus buy decision. All it was was option one was renew the current lease. Option two was lease a new tower. Option three is buy the tower outright. Okay? So three options. The instructions have all the details of how much the renewal and the monthly costs and everything is. So I did this big, huge, long best practise prompt. I got this and it did it all by months, not really the way you'd want a scenario done per se. But what you'll see is I did a really simple prompt. It's like one line and I got this. Okay, fine enough. I can see the options. It compares them, gives me the inputs, gives me something similar to before on separate sheets, just a little different layout, but probably as good as.
(11:24):
Not far off the best practise. All the math was the same between them. The answer was the same. It just laid it out in a different format. Then I said, "Let me give it an actual example to build off." And so I gave it personalization with Copilot. I gave it workbook rules, and I gave it at the back a sample file what my scenarios look like because you can't load a file in Excel's agent when you're doing it this way. So I couldn't load the file. And so now, all right, I can see my months. I can see it laid out. All right, here's your share drivers, your options. It's very similar to the. Let me go here. To my scenario I gave it. I said, "Hey, I want an executive summary. I want a cover sheet. I want you to always put blue up at the top." And it gave me very close to what I had.
(12:21):
And so what my point is without showing the full file is as soon as I gave it a reference, as soon as I gave it instructions, as soon as I gave it all the stuff beyond the context, it got way closer to I want. Didn't follow it exactly? No, because I kept working with it. But it did it years like I wanted. It laid it out a more typical scenario where I used the best practise prompt. I went through all kinds of detail, told you I wanted a Daniel, and I got something about the same as a two-line prompt.
Co-Host: Glenn Hopper (12:55):
Yeah. Did you check the math on all those?
Host: Paul Barnhurst (12:57):
As far as I can tell, they're all right because I've read it like six times and they all come out to the same total. And it's very close to what I originally had. I gave it all the numbers to use. It does the formulas in Excel. But have I tied out every single one? No. Am I 98% confident?
Co-Host: Glenn Hopper (13:12):
Yes.That's the thing, and we've talked about this a million times, but the difference between finance and accounting and finance, if we're directionally correct, we got it. 98% is good enough. I've got my decision here. It's not going to come down to a few dollars off in the interest calc and the M table or whatever on your lease decision versus buy. But the thing is, whether you're using Copilot or Claude or ChatGPT and Excel or whatever and building models, or if you're building it through the chatbot or through Cowork, whatever your way in, you're going to get something pretty good. Again, the point I was making about checking your numbers, but it's changing work in such tremendous ways. I'll give you an example from my world. I have a client, I'm doing kind of a mix of analytics and a little bit of a churn analysis and some basic fractional CFO work for a client.
(14:07):
They have almost 20 different unique LLC entities that all roll up into an enterprise system. And all the entities, there is a consolidation, but it's still that many different entities. So I'm doing some dashboarding for them. I ended up, I do daily polls, just rip everything down into a simple Postgres database. But I have a semantic layer over that and I have an MCP server. So their chief of staff, they're in a meeting today, is asking about one of the entities and without getting into any specific client details, he wanted to compare what this one entity was paying compared to what the others were. If I had to go into the ERP and go pull all that, it would've taken forever. Meanwhile, I'm doing something else. I have chat with my data. It goes in, rips down way too much information, I guess, because I gave it too loose a prompt.
(15:10):
Gave me way more, put together a five-page report on all this. But anyway -
Host: Paul Barnhurst (15:15):
It gave me a Warren piece novel. I get it.
Co-Host: Glenn Hopper (15:17):
Yeah. I never even opened up the ERP system. I just queried the data that I'd already pulled down. Now, I could have had it go straight through and connect through the API, through the MCP server that they had, but I already had the data. I could get it so quickly. And I think that's what's so different to me. If I'd gotten that question four years ago, I'd have been like, well, here goes my afternoon. I got to go dig through all the entities, pull individual reports on this one account, and then line them up and build all this. And it just did it in the background. Because it got very verbose, it took about 15 minutes to pull it all together. But had I prompted better and gave it more clear guardrails, it could have given me that answer in under three minutes.
Host: Paul Barnhurst (16:03):
Yeah, you could have had it in five minutes or whatever. So I think that as we talk about the news, let's bring this back to the listener. Fatigue is real. Stop focusing so much on every bell and whistle. Recognise the first thing I'll say is the models are pretty similar. All models can do a good job. Will some be better than others? Yes. I think second is we're at the stage where if you're still trying to do everything with prompting, you need to move past that. Fair statement? Yes. We're at the stage where giving it a reference file, looking at your personalization, instructions, all that matters. The next thing, which we've heard in the news again and again, and I think maybe some practical advice for people, is if your data's a mess, if you can create a data schema, a context layer, star schema is great.
(16:56):
There's a reason Power BI is in a better spot than Tableau because they required that data schema, which AI loves, build one. So if you don't have schema for your data, you're limiting how quickly you can get answers. Fair statement?
Co-Host: Glenn Hopper (17:16):
Yes.
Host: Paul Barnhurst (17:16):
Because you're pulling together from all kinds of different places and AI can help build it. But if you have a good context layer now, three minutes.
Co-Host: Glenn Hopper (17:26):
Oh yeah. And that is huge. And nobody wants to talk about doing the data. Nobody's wanting to talk about the data work for two decades. But now this is the clarion call for you have to do this. You're just going to have to take the time and the effort. And it's admin, it's grunt work, it's not sexy. Everybody knows this, but it's so painful to think about doing the work if you have siloed data and you don't have -
Host: Paul Barnhurst (17:57):
Well, I think what's important is just talk a little bit about that practically because everybody hears that and they think big, huge project. So we've talked about first practical one, learn how to do instructions, learn how to build a skill, which ask AI to help you. Learn how to give context, skills, instructions beyond your prompt, to help repeatable, to get good stuff. Next is that data layer. When you want repeatable, you want agents to work with something, you want to be able to really start to get to an intelligent layer, you need good data. And I think everybody understands the idea of cleaning data, but where does somebody start? So today I've been pulling it. I pull from QuickBooks, I pull from Salesforce, I pull from Stripe. We've all done something like that. Marry the codes all together in a way I go. So do I try to connect an MCP to each of those?
(18:54):
Say what I want? Do we have to put it all in a warehouse? Quick tip. I don't want a six-month project. What's the quick tip to get value here? How do you think about it?
Co-Host: Glenn Hopper (19:06):
Yeah, the quickest tip is not. I love a data lake. I love just ripping down and having in my own database that I'm not touching production and all that. But outside of that, I say this all the time. There's a OneStream thing I did with Chris Ortega where I just keep repeating data governance and data dictionary. But what I mean by that is keep it in your system of record, but identify,
(19:30):
Okay, this is our data dictionary. This is our data universe. This is what it means when I say billing contact. These are our KPIs. We're going to use, this is the numerator, this is the denominator. Boom, everybody's singing from the same sheet of music. These are what our KPIs are. And then identify your source of truth. You could have one truthfully, as a piece of context, if you had one markdown file that had, this is what I mean when I say churn, this is what I mean when I say DSO, this is the data that goes into it, this is where it comes from. You don't need to build anything. And you can still query it all as long as you have the connectors and everything. You can still query direct from coworker where you build your skill to do whatever you need, but you need that context layer.
(20:19):
So the simplest thing you can do today, don't worry about pulling it out. If you've got APIs and everything and you can connect in, then just make sure that AI knows just like any new employee would know where to look for the information. Pull the billing data from the ERP, not the CRM, whatever the rule is that you want to follow, but document it. Context is everything. And so we talked about prompt engineering, then we talked about context engineering. Now we're into graph engineering, which is all the relational bits between them. But you don't even have to have all that if you just have those context documents. And it doesn't have to live all in one master thing. If this is what operations uses, this is what finance uses. I mean, if you're going to be using the same KPIs, you have to do it.
(21:07):
But really just knowing that source of truth and what you mean when you say this is how we calculate whatever, DSO or whatever, that's huge for step one.
Host: Paul Barnhurst (21:19):
Ever feel like your go-to-market teams and finance speak different languages? This misalignment is a breeding ground for failure, impairing the predictive power of forecasts and delaying decisions that drive efficient growth. It's not for lack of trying, but getting all the data in one place doesn't mean you've gotten everyone on the same page. Meet qflow.ai, the strategic finance platform purpose-built to solve the toughest part of planning and analysis. B2B revenue. QFO quickly integrates key data from your go-to-market stack and accounting platform, then handles all the data prep and normalisation under the hood. It automatically assembles your go-to-market stacks, makes segmented scenario planning a breeze, and closes the planning loop. Create airtight alignment, improve decision latency, and ensure accountability across the team.
(22:27):
The first step to people is if you have systems of record, do you have a way to connect to them? If not, see if there's a way you can do it. Hopefully there's a simple MCP. If not, yes, you could custom build something, you could have AI help you with that, but start with the ones you can easily connect to that your IT allows or you have it set up. Then you don't need to boil the ocean. What are your top five or 10 KPIs? Are they finance related? Are they company-wide? So who needs to agree on that definition? Make sure they're aligned. List where the source data comes from. If there's some exceptions like you're always cleaning something or write it down so AI knows that. And then try pulling and see if it can get you there. And maybe you do a data lake later, maybe you start doing all that, but just begin with that simple context document that you can store as a markdown file and give it to any of the systems and they can read it.
(23:25):
You're going to get better results. I mean, that sounds simple, but that's really a great starting place.
Co-Host: Glenn Hopper (23:32):
Yeah. And I don't mean to be overly reductionist in that. I mean, I understand there's transformation that happens to the data when you pull it down and you've got, yes, this is the journal entry, but there's actual transactions behind it. Well, build on it. You have to start somewhere. Don't worry about that right now. You summed it up. Let's get the simplest bit that we can and figure out how to use AI with that, and then we'll layer on top of it.
Host: Paul Barnhurst (23:56):
So if you feel like you have no good context, the advice I'm hearing here is write down those top 10 KPIs, write down their sources, the ones that need to be aligned, align them, write down the systems they come from, and just start by giving that to AI, to what you can connect to, and just build. So many people feel like they got to solve everything or they don't know where to start. So that's one area on the data side. I think that's a great tip. Anything else you want to add? I know you're not trying to minimise it because we can make it way more complex than we have.
Co-Host: Glenn Hopper (24:29):
Here's another thing I'm trying out with a client right now that's been pretty interesting. I work a lot with founder-led companies that are trying to move in and pull in investment or whatever stage they are. They're trying to put on their accounting - When I
Host: Paul Barnhurst (24:42):
Investment, I can come to you.
Co-Host: Glenn Hopper (24:44):
Yes, I'll invest. I'll give you $30.
Host: Paul Barnhurst (24:46):
Oh, that's not what I meant. I meant you're going to help me find the right investors.
Co-Host: Glenn Hopper (24:49):
Oh, right. Yeah.
Host: Paul Barnhurst (24:49):
But now that I know you're rich, I'll come to you for the investment.
Co-Host: Glenn Hopper (24:52):
It's interesting when you talk to companies of this size where maybe they're using Dropbox, maybe they're using Google Drive, they've got some stuff on SharePoint. Stuff is scattered everywhere and they have this kind of oral culture where nothing's documented. We've been setting up deep research prompts and going through all of their whatever's in this folder, whatever, pulling out. We're just trying to build context
Host: Paul Barnhurst (25:15):
Right now. Through Slack or dig through this folder and think
Co-Host: Glenn Hopper (25:17):
About - Yes, and through
Host: Paul Barnhurst (25:19):
Teams. Everywhere it says the word bookings, that kind of thing.
Co-Host: Glenn Hopper (25:22):
And just build this context. So right now you don't have to have. I'd love it if somebody wants to hire me and come in to do this, but the stuff they can do themselves really just by combing those emails, combing those. If you're doing investor reports or management reports or whatever, comb through those. Just get the narrative in the past, summarise it, get it all chunked down, organise your document, your directories, and have something for AI to use. Because if you're just spending your time doing that and not investing in AI with the rate of change on technology, I could see. I mean, imagine having this conversation this time next year. I mean, AI, as they get bigger context windows, as the features are added, just by doing this, you're going to put yourself in a better position even if you don't do one thing with AI this year.
Host: Paul Barnhurst (26:14):
That's a great example. So speaking to that, I've been working on this website I'm getting ready to launch and I've talked to a tonne of software vendors. My notes are a mess. I store them in ClickUp. Finally just connected the MCP. And at first it's like, I didn't find anything. I go, no, no, you're looking in the wrong folders. Go dig through this area. Oh, we found 60 conversations and here's the summary and here's all the tools you visited with. And I'm like, that would've taken me hours to dig through all that. Now I had a lot of context for something I'm building. And so it was incredibly valuable because it allowed me to be like, okay, yeah, this makes sense. This doesn't make sense. And much quicker than, oh, let me go look at every single note. And I might've missed one. I may not have found everyone.
(27:02):
AI may have made a mistake somewhere, but it was a heck of a lot better than me spending 10 hours trying to see if I found everyone. I'm with you on the digging through fault. It's a great example. So I think that gives some tips there. Anything else you want to say on the news side before we move on to the next topic I think we want to discuss?
Co-Host: Glenn Hopper (27:20):
No, I'm going to give everyone a break from chasing benchmarks and model capabilities and all that. There's plenty of folks out there. The
Host: Paul Barnhurst (27:27):
Model's
Co-Host: Glenn Hopper (27:28):
Wrong.
Host: Paul Barnhurst (27:29):
Yeah. There's our benchmark announcement for the day. Token maxing, token forecasting, token usage. Do you cringe? Do you think, love it? When I say those words, what comes to mind?
Co-Host: Glenn Hopper (27:42):
What comes to mind is, and I knew this was going to happen, I didn't know what it was going to be. I have a book, Paul, coming out at the end of September, The AI Ready CFO. I finished writing that book in December last year, January of this year. It's been going through the whole editing process.
Host: Paul Barnhurst (28:00):
Bub review process, yeah.
Co-Host: Glenn Hopper (28:01):
Yeah. I knew nine months is an eternity in AI time, and I knew I was going to miss something. In that book, I explicitly say, "If all you're doing is giving your employees access to ChatGPT or Claude or whatever, stop whining to me about ROI. That's just a software cost. You don't try to come up with ROI on your ERP or whatever other SaaS tool that you buy. I don't want to hear about it." Well, then token. I mean, then the models get better, you're using more tokens, and suddenly that 25, 30, $60 a month is not, and it's not predictable because everybody is blowing out their token limits because the models get better. They use more tokens. And the tokens themselves are less expensive than they were a year ago. We're burning through so many more of them. So I was completely wrong on that point in the book.
(28:56):
The rest of it's fabulous though, Paul, you should pre-order a copy.
Host: Paul Barnhurst (29:00):
A new link and I'll get on that.
Co-Host: Glenn Hopper (29:03):
But a couple of things happen, and I know you've got a take on this, so I'll try to go quickly. We were incentivizing the wrong thing at the beginning of the year. Jensen Wang came out and said, "If I hire a software engineer for $500,000," sidebar, I immediately thought I have gone into the wrong profession. If a software engineer starts out at $500,000. Anyway, I digress. Jensen Wang said, "If I hire a software engineer at $500,000 and they don't spend $250,000 a year in tokens, then I am not maximising or they're not maximising their role and I've hired incorrectly." So this sets a bar for instead of thinking about output, it is how many tokens are you using? Which I've never asked, I hate time and materials billing. I've never asked anyone who worked for me how long whatever they did took. I don't care.
(29:59):
If it took Take you two minutes and you can take a two hour lunch. As long as I get what I want and it's right, awesome. If it takes you 22 hours, maybe you're doing something wrong. I don't care about the input. I only care about the output.
(30:13):
Companies were going out and rewarding people for how many tokens they used regardless of what their outcome was. So then all these big companies you're reading about, it was April, May, they had already blown through their whole
Host: Paul Barnhurst (30:24):
AI budget. Yeah. You saw that Uber blew their budget by April for the full year. I think that was the example, right?
Co-Host: Glenn Hopper (30:30):
Yeah. So we were incentivizing the wrong thing. Yes, I get it. It's an early software you want to encourage use. You want to get people to figure out their own ways to be as proficient at it as possible, but that's not a sustainable or it's not even a logical what ROI would maximise input without somehow leveraging it on the other side with what the output is. So I think we started out wrong. I get it. It's a new tech. We're trying to figure out, we're trying to encourage people to use it, but it's now token spend should be a line item in the budget by headcount. And engineers are going to use a lot more tokens than maybe right now than financial analysts are or more than sales and marketing people are. Not to say that the usage is less, but how deep they're using and just the number of tokens that they're using by the nature of their job.
(31:29):
So I think right now, if we have a team or enterprise account, we can track usage by role or by person so we could roll it up into roles. So then the problem is we didn't have anything to base a budget on before this year. So now we're collecting data for the 2027 budgets. We should know, okay, every employee at management level in engineering gets this number of. This is how we build the budget. So I do think it's going to be part of a fully loaded employee cost in the future, just like benefits are. So I don't know, what do you think? What are you saying?
Host: Paul Barnhurst (32:06):
I think that's interesting. I just saw last week, Team Ohana, their HR, human resource, people forecasting tool. So instead of being your entire forecasting tool like an Anaplan or whatever, there's several of them. And they just released their feature of, here's kind of what we recommend. You should be doing a line item for headcount engineers, 20% of their salary, whatever for tokens, kind of AI. I'm mixed on that exact approach. So I'm going to share a couple things. First, I agree with you, token maxing was completely wrong. And the example I give, you ever seen the company that the BDR sales plan is only about how many calls they make? You know what I'm talking about, right? Yeah. And then none of them ever lead to leads because they're just getting off the phone as quick as they can because they make more money. And so it has to be input and output.
(32:57):
Next, as you're trying to forecast, you need a tool that Ramp has one. If you use Ramp, there are others. Maybe you have AI do it. I've seen people build it with vibe coding that can bring together all your different usage into one place. You need to be able to look at it in a dashboard and understand it. You need to start doing some forecasting.
Co-Host: Glenn Hopper (33:16):
You know we partner with someone who builds that dashboard.
Host: Paul Barnhurst (33:19):
Yes. Go ahead. Who is it? Say their name.
Co-Host: Glenn Hopper (33:23):
It's our friends at Ramp. They rolled out their token tracking dashboard
Host: Paul Barnhurst (33:27):
That they have. I was going to say they have one. I know somebody who did some vibe coding where they built one and was doing forecasting. So first thing, if you're not able to bring it all together, figure that out, however you do that. So first you need to have a full picture, then run some trend lines. But then next, when you think of forecasting in the next year, like a friend of mine said, he goes, "This is how I feel about AI." And he's working at a company that's AI in its product, AI central. And they have a huge OpEx line because we all saw what ChatGPT did recently where they reduced their cost like 80% or whatever on some of the models for tokens. He's like, "There went my whole forecast. I'm completely off." And so I like what somebody said. You got to break it into a couple areas.
(34:11):
There has to be a certain amount of the bucket that's allowed for experimentation. So maybe your budget's 30% experimentation because this is a new software. We have to figure it out. And unfortunately there's cost. It's not like Excel where there's no cost to figure it out. There is a cost to figure out AI, whether you'll like it or not. So you need to budget a certain percent for that. Then you need to look at the rest. And then there's the management part. My friend said on this podcast, he looked at it and one guy was using like 10 times as much as all the other engineers. So instead of sending an email that says, "Stop using it. What are you doing, you idiot?" They pulled him in. He first went to his boss and said, "Hey, look at this." He said, "I'm not sure what's going on." They pulled him in and go, "We'd love to know what are you doing with AI?" It wasn't, why are you using so many tokens?
(34:58):
I'm doing this. And well, we've noticed what's the return you're getting? And had that conversation to really understand and then said, "Oh, you're using substantially more than others. Do you think maybe you could use a different model? Let's better understand it." So I think within that forecasting, I kind of think of this as almost a four-step process. There's get all the data together, nothing new, just like anything else, collect and gather your data. Understand what you're trying to accomplish as you're setting your budget. Remember there's just an experimentation portion. There's a cost of learning. Think of it like training. That's really what a portion of your token budget is. Then the other side, but then make sure you're also having the conversations on the high usage, those individual conversations to understand that people are educated to drive that cost down. Then ultimately you need to also be look at what's the return we're getting?
(35:50):
And that can be long-term. Just because you can't quantify it today does not mean you're not making a good investment. The number I saw from a report I just saw today, I wish I could remember who it was. I don't have the link in front of me, but I was using it for some training I was doing. 14% of CFOs say they can actually quantify an ROI right now with token cost, but something like 70% say they expect to be able to do it within the next two years. So there has to be a patience. And then if you are one of those that continues to just say, "Hey, you're rewarded for spending the most tokens, you're an idiot." Oh wait, sorry. I mean, you need to change that approach. I couldn't resist. So those are my thoughts. Feel free to take that wherever you want.
(36:32):
But that's kind of how I think about it. And then as far as the forecast you build, I would almost focus a little bit more on optimising usage for productivity, knowing that costs are going to change, have that overall budget, but you're going to have to be flexible because you kind of guess what it's going to cost per token over the next year. You're going to be wrong.
Co-Host: Glenn Hopper (36:56):
Someone said in a talk recently, oh, tokens are as cheap as they're going to ever be. And I though, no, they're not. You don't understand how commodities work. And as these models, the frontier models and then the Chinese models that are catching up very quickly, they're going to drive the cost per token to the bottom, which that's a whole other episode is, and we've talked about it before, but how are these AI companies ever going to be profitable with the amount of capital that's going into. Anyway, that's a whole other episode, but I don't think that's
Host: Paul Barnhurst (37:30):
Not true. These are - You should break them down next time. That would actually be a really fun episode. So I think what you're saying is, like you said, the cost is going to come down. I think the cost will go up in the short run, but in the long term, they're going to become a commodity and come down. Because as these firsts go public, they're going to have to raise prices. I think they're keeping them low to lock people in. But eventually as the competition gets bigger, as the cost to build these models come down, like the Chinese and these different approaches, that's what will ultimately derive these down. And just the fact that commodity, how long can they hold those prices when their models no better than Peter's or Paul's model or Glen's?
Co-Host: Glenn Hopper (38:07):
Or Mary's.
Host: Paul Barnhurst (38:08):
Yeah, we got to get the woman in their Mary's. We're not going to big boys club. Wait, did I say that? I'm just kidding. Okay. Anything else you want to. Any last thoughts you want to leave with our audience? Anything else you want to cover before we wrap up?
Co-Host: Glenn Hopper (38:23):
I think we've hit it today. A little bit of fatigue. Maybe the temperature outside amount of work we've done around it. Hopefully next week I'm back and I'm fresh again and I have some crisper insights. But right now I think I'm going to close with the same thing that I led with. It's exhausting.
Host: Paul Barnhurst (38:39):
I don't disagree as I spent several hours trying to put together something for training that I didn't use today because we ran out of time. They hate that. And I was like, oh, I spent forever with AI because that's what we were training today. But what I will say, invest the time, be smart, stick with it. If you're dealing with forecasting of OpEx, come up with a framework and process. I shared one here. There'll be lots of others coming out. Don't just be like, "I can't do it," or, "I'm going to let AI do it all." You got to think through this and manage these things. And then who cares if that new model came out tomorrow? You don't got to spend all your time testing it. Wow, it was better on this. Did it help you do your job any better? Did it really give you better insight?
(39:26):
Did you make any money off it? Probably not, is the usual answer, right?
Co-Host: Glenn Hopper (39:31):
Yeah, I guess actually I'll close with. It's kind of like chasing how many tokens you're using versus the output. We've been spending so much time focusing on how to better use AI. It's easy to kind of take your eye off, what is our day job? What are we trying to do here? In FP&A, it's very clear what it is. And you can be just like we could obsess on models in Excel, but you can be obsessing on ways to do it with AI. Meanwhile, the company's not hiring you to be an AI expert. The company is hiring you to provide a value or service. So we need to keep our focus there and yeah, keep incrementally improving, but the big massive leap from one AI model is not going to come there. It's these are the models we have. Yeah, I get it. They'll incrementally get better.
(40:16):
So let's make our processes incrementally better and not go chasing the whale and just look for those small steps where we can use the AI that we have today to get better at our jobs.
Host: Paul Barnhurst (40:26):
I think you led to something I'll mention, and I'll close with this. Finance's job, ultimately as an entire org CFO, is to help the business make better, faster decisions. If you're doing that and you're doing a good job of that, and you have automated things really well without AI, great. Now layer in AI and you could probably even help a little bit with the faster and the better. But don't think you have to immediately just go and run and chase it. Don't chase AI for the sake of AI. Always come back to what am I being paid to do? How can I do that better, smarter, to drive the business forward? And if you take it from that perspective, hopefully it'll help a little bit with the fatigue and it'll help a little bit with being overwhelmed versus, well, I read on LinkedIn, I can build my model in five minutes and every CFO built their budget in 30 minutes for the year and I took me eight hours.
(41:26):
What's wrong with me? Just close LinkedIn and take a breath. Maybe that's my parting advice. Close LinkedIn. Even though Glen and I are on there, close LinkedIn and take a breath.
(41:43):
All right. With that, thank you for joining us. As always, let us know how you like this episode. Reach out to us. We'd love to hear from you. And Glen, hang in there. The fatigue will get better.
Co-Host: Glenn Hopper (41:57):
All right. Thanks, Paul. We'll see you next time.
Host: Paul Barnhurst (41:59):
Thanks for listening to the Future Finance Show, and thanks to our sponsor, qflow.ai. If you enjoyed this episode, please leave a rating and review on your podcast platform of choice. And may your robot overlords be with you.