How many of us have heard an intern or a new joiner say: “This doesn’t work the way they said it would in school.” Sadly, business schools don’t teach students just how messy and disorganized business is — how entropic it is.
Processes are complex. They depend on thousands of interconnected rules for applying discounts or approving requests. Rules change based on business need and regulatory requirements. And there are exceptions — and exceptions to exceptions: send the invoice to this executive rather than that one in a local market; apply an off-book discount that someone agreed to years ago. Data is opaque. Information is inaccessible because it lives in email chains or spiral notebooks. It is incomplete, disconnected, and out of date.
Why do enterprise systems cost so much and perform so poorly? Because business is entropic. Why is system modernization so hard? Because legacy code reflects years of patches and hacks accumulated to capture the weird idiosyncrasies of our business processes. Sometimes this produces the dreaded second-system problem: a replacement system that degrades productivity because it fails to replicate some of the never-written-down, barely-understood functionality that made the first system work for users.
We will all spend considerable time in the coming years using GenAI tooling to understand legacy code. Where does it capture important process idiosyncrasies? Where does it assume an obsolete process that can be reimagined? Where does it contain years of accumulated, useless crud?
I had the chance to talk with Harrison Touati, CTO of Rhino AI, and discuss how he thinks about using GenAI tooling to interrogate legacy code — and what that means for modernizing systems and improving operational productivity. As always, Prosaic Times never endorses and company or technology, but we often find it interesting to hear from builders themselves! Here is what I took away.
Legacy systems are not technical artifacts — they are business records. A company’s processes contain more information than any one person can hold. Code is the accumulated record of how those processes evolved — including rules nobody wrote down, constraints nobody remembers creating, and idiosyncrasies that may or may not still matter. Modernization fails not because developers are slow but because nobody can comprehend the whole thing.
The right comparison is not AI versus deterministic code. Human engineers are already a stochastic process. The real question is which messy stochastic process produces cheaper outputs with a lower defect rate. Once you frame it that way, the trajectory becomes clearer.
Where you process entropy matters as much as whether you do. Enterprises can deploy AI agents at runtime to handle messy decisions as they arise, or at design time to clarify the rules and encode them in software. Design-time is almost always preferable — it lets you apply 30 years of software QA to the output. Most organizations are currently doing it backwards.
When code becomes cheap, what remains scarce is structure. Domain knowledge commoditizes. What doesn’t: the structured understanding of how a specific business actually operates — its ontology. The companies that will win are those that own the vertical model, not just the development capacity.
James Kaplan: Hi there, this is James Kaplan, joining from Rhode Island for another Prosaic Times video podcast. We have Harrison Touati, CTO of Rhino.ai, with us this morning. Harrison, welcome.
Harrison Touati: Hey, great to be here.
James: Why don’t you tell us a little bit about your journey to Rhino AI?
Harrison: I’ve always been interested. I remember even back in college, natural language, NLP.
Data science, stuff like that. And my early career was a bit of a pivot from there, but it all came back together when I worked at some low-code SaaS companies. I did a lot of what I learned was, wow, a lot of these processes are really painful and miserable. And the thought process was, okay, can we use AI to make understanding legacy systems and to make modernization a lot less painful?
And that’s what inspired the creation of the technology and the company and how I got to where I am currently.
1. The Low-Code Mirage: legacy systems are business records
Low-code promised to eliminate developer dependency; in practice it reinvented programming complexity in proprietary form.
The hard part was never syntax — it was understanding the problem.
James: Many people were convinced that low-code, no-code was going to take over the world. It didn’t. What was the promise, and what caused the gap?
Harrison: Low-code, no-code products — the promise was that business analysts could do the development. You wouldn’t need developers. And that never really manifested for two reasons.
One is a failure to actually be sufficiently no and low-code in the first place. And the other is that even if you were sufficiently such, you would still need developers anyway, because the hard part is understanding the problems. If you look at low-code products in particular, they end up reinventing the wheel. At first they’ll be like, oh, we have this really easy way to solve some problems.
You want to do as much at build time as possible because it’s cheaper, it’s faster. Runtime is what you want to do the absolute bare minimum — exception cases, extremely fuzzy amounts of judgment. That’s the only time you’re going to want to really do runtime.
But then they’ll ask: what if I need to iterate that across an array of issues? And all of a sudden they’re rebuilding for loops in their own way of doing that. You end up sometimes in a worse situation where not only are you not low-code, you’re basically in some invented programming paradigm that only your staff knows. And that sometimes makes things even worse.
So what you end up seeing is for the easy use cases, you do get savings. And then for the complex use cases, sometimes you’re like, man, it would have been easier to just run a Python script for this. The other thing is that the idea that if development was easy, that you wouldn’t necessarily want people with a developer mindset is not really true. A lot of business problems are hard and complicated and require a lot of thinking to solve.
If you’re going through and doing really complex ERP implementations and the rules are really hard, you need someone who can sit down and logically think through things. And so you end up just having developers working with weird proprietary tools, and I think that probably hurt the adoption of low-code.
James: It reminds me of the 1990s when people started coming out with visual development languages. I’m dating myself, and I remember people showing them to me and — great. So instead of typing three lines of code, I have to click and drag 40 things from a palette onto some sort of screen and then do a bunch more mouse clicks to connect those things with arrows and what have you. Why wouldn’t I just write the four lines of code or the 10 lines of code? Once you get past remembering some very simple syntax, it doesn’t buy you anything.
Harrison: Especially when you start getting into complex arithmetic calculations — you’re calculating interest rates or something like that — you end up back having to enter Excel-like formulas and then you end up with a programming language anyway. That’s the worst part — you drag and drop the boxes, you draw the arrows, you map the variables in the dropdowns, and then you end up still running some monstrosity of an Excel-like expression to close it all together and you’re like, this could have just been five lines.
James: Exactly. And in a programming language, you can just call up all the logic and look at it. Instead of having to click on 10 different things to see where the logic might be buried — which gets to an important underlying topic: the reason systems have been challenging is that there’s a lot of entropy, a lot of variation and uncertainty in business processes, and none of the low-code, no-code products either in the 1990s or more recently had any effective way of dealing with that.
Harrison: The complexity — when you think about most businesses, there are absolutely use cases, like you buy an ITSM system for ticket management, and okay, this really does just out of the box solve a lot of my problems, but mostly because ticket management is fairly consistent across many companies.
And businesses very rarely are building greenfield new applications. Even when they think they are, they run today and therefore whatever problem they need to solve, they have some way of solving it today, even if it’s not good. And that problem is often constrained by regulations, by economic realities, by human processes that nobody’s even aware of.
Whenever you try to automate these things with code — entropy is a really good framework for thinking about it. These are very high entropy, confusing, often poorly thought out processes, often emergent. They’re not an artifact of design. They evolved in the wild and you have to be able to capture these and it can be just so complex. And then what you end up having to do is taking that ITSM or whatever CRM ERP software and smashing it into some sort of monstrosity workflow engine to handle really complex use cases. It doesn’t work out very well usually.
James: And often the failure is in the understanding of the business. How does this actually work? Your point about emergent processes is a very good one — because we sit there and say, here, use the SaaS product and use the out-of-the-box workflow. That’s interesting because we don’t really know whether the emergent idiosyncrasy is accidental — just the way someone had to design it at some point — or, to use the example of Chesterton’s fence[5], did that idiosyncrasy evolve for an important business reason?
Was there some regulatory thing someplace or some set of customer requirements that nobody bothered to write down?
Harrison: That is the hard part. When modernization projects fail — and this is something I learned early in my career — it’s very rarely due to developers not being fast or smart enough. It’s almost always due to everyone — the client, the developers, the project managers — it’s not a one-person thing or one-role thing. It’s everybody not really understanding how the business operates.
And it’s not their fault. The business is in some sense an emergent super intelligence — more information than any one human mind can hold. When the individual human tries to comprehend it all, they struggle.
A lot of the idiosyncrasies are created for very good reasons, but sometimes those reasons don’t even exist anymore. They’re like vestigial organs of the enterprise.
James: But you don’t know which ones exist and which don’t.
Harrison: You don’t know that out of the box. That’s the AI use case — and why, from a career perspective, I thought it was worth pursuing. Can we apply some degree of limited super intelligence to help augment that process?
Can we pull in as much information from as many people as possible — we’re not going to understand everything, but can we at least understand more than the average person on the project? And if we can even a little bit lower the failure rate of projects from that cause, that becomes very worth it.
2. Interrogating legacy systems: ask how to use AI, not whether
Modernization projects fail not from slow developers but from everyone — client, dev, PM — not understanding how the business actually operates.
Systems are digital manifestations of business processes; what looks like a technical problem is almost always also a business problem.
Reconstructing process from code, config, logs, and documents is the triangulation problem at the heart of legacy modernization.
James: Okay. So tell us about the creation of Rhino AI and tell us how the technology works.
Harrison: The creation was born out of the pain and suffering a lot of the other co-founders and original employees and myself had — modernization projects are miserable. They require so much specific information.
James: And just one definition — when you say modernization project, can you just define that because that could mean a few different things? Just scope it for us.
Harrison: So what we normally mean is the business has some existing capability, normally already implemented in some legacy tech stack. And legacy is in the eye of the beholder very often. Of course.
James: Yeah. I knew I was in trouble when I started hearing about legacy cloud. And in a couple of years we’re probably about three months away from people starting to talk about legacy AI.
Harrison: Oh, definitely. And part of that’s due to — the analysts have to make new buzzwords. I don’t know how many analysts are listening to this, so I’m going to maybe keep it there.
So that was the business problem — born out of the pain of those modernization projects, of taking these legacy software systems and having to rebuild them to reap modern benefits: modern security, more maintainable, easier to add new features, better UX, better performance, lower cost.
James: And sometimes a modernization project can be largely technical in nature — we’re running on outdated code. Sometimes it can be business and technical in nature — it’s not integrated enough, it doesn’t have the type of capabilities that business users are demanding. Fair?
Harrison: Yes, you’ll see both of those. And often when people think they have the former, it really is the latter. As you dig into these, the system is like a digital physical manifestation of the business’s job to be done and their processes. As you dig into these things, you’ll realize that people don’t actually like them and they don’t understand them.
You’ll often find — they’ll be like, hey wait, that’s how we do approval? I didn’t even know how approval logic worked. We should definitely change that now that we know how that works.
James: Years ago, a wise old architect told me a system is just an automated process — which I think is almost true, but not quite true. A system is automated parts of a process, because often you’ll see pre-processing and post-processing. We’ve all seen the spreadsheet gymnastics required sometimes to enter data into a quarterly close, as well as the spreadsheet gymnastics on the far end of some sort of system output in order to put the data into a format that people can use or understand.
Harrison: It’s going to depend on the system. Some systems really do capture — and people get into, what is a process? Sometimes a process is a purely technical or automated job.
But usually when people talk about processes, it’s with human actors. And therefore, almost by definition, the system is only part of the process because it is not the human actors — it might tell someone that an approval has to happen, but it is not the approver.
James: In banking you have many processes that are truly — clearing and settlement, in financial services broadly, we now have many transactions that are truly no-touch.
Harrison: I agree. And with AI agents, you’re seeing some processes that people traditionally thought couldn’t be done that way — people are starting to say, well, maybe some of these approvals really can just be handled by an agent. Where that goes is yet to be determined. But I do suspect we will see more and more people try to move processes to be like that.
And in many cases, they’ll succeed. In some cases, they’ll probably fail spectacularly.
James: In banking, if you look at highly structured products like equities, those are incredibly automated. On the other hand, exotic derivatives, much less automated because you could argue there’s just a lot more entropy associated with the product.
Harrison: It’s not just the entropy. It’s probably also how many people are doing it, because there’s always that cost tradeoff — is it even worth automating?
James: Equities or credit card transactions are a great example of processes with a lot of clarity, not an unreasonable amount of variation and very, very high scale.
Harrison: And what’s probably changed due to AI is that the ability to automate things that traditionally had a lot of unstructured data and lower frequency is going to go up — and it’s probably already gone up. So I think we’re going to see a lot of attempts and probably a lot of successes in automating those things.
The structure is the interface, in a sense. It’s the method of communication between parties. Even humans need structure when they talk to each other — that’s why they make forms.
People often compare AI-based solutions to current processes as determinism versus stochastic processes. That’s probably not a good comparison. A piece of code might be very deterministic, but AI can write code. So the question really becomes who’s generating the code.
The humans who write the code are often stochastic — they’re messy. Code is filled with bugs. That’s why there are implementation budgets to go through and do all this maintenance, fix all these bugs. As you zoom out to the 20,000-foot view, everything’s a messy stochastic process.
It becomes which messy stochastic process is going to be cheaper and have a lower defect rate. There’s a nice chart I like where it’s the performance of the best chess engine and the win rate of humans over chess. The win rate of the human grandmasters at 100%. And then one day you get Deep Blue and all of a sudden the human rate is zero.[2] I think you’re going to start seeing that in more and more industries — sure, the AI might mess up, it might hallucinate, it might write bad code with a bug.
But the goal was never to write zero bugs. Human engineers can supply plenty of bugs. You only have to have less costly bugs on average than the human. And as we see the models improve, that world is going to come.
3. Design time vs. runtime: where you process entropy matters
Processing entropy at design time beats runtime: you can apply 30 years of software QA to the output. Most organizations are currently doing it backwards.
LLMs are poor at saying “I don’t know” — the truly hard edge cases may be better handled by humans, at least until the models improve on calibration.
As code gets cheaper, exception cases shrink. The agent-vs-human decision for the residual tail matters less than it appears.
James: I think about it slightly differently, which is: where do you want to turn the stochastic world into something less stochastic? Where do you want to process the entropy? One of the big decisions will be, do we want to do it at design time or runtime?
If you have a whole messy set of rules that are not well documented, you could either build an agent to apply those rules at runtime, or you could develop software using agents to clarify those rules and apply them in a more deterministic manner. The advantage of addressing stochastic processes at design time rather than runtime is that then we can apply everything we’ve learned over the past 30 years about software quality assurance to the output.
Harrison: That’s a very good point. It’s design time. And when people think about agents still, they’re thinking very much runtime agents — oh, we can have some agent do automatic approvals and stuff like that. But I think about — there’s an example of when the internet was getting big and people talked about the information superhighway.
If you remember that, I suspect I’m a little bit older than you are.
James: Yes, I remember the information superhighway. In college, I wanted to take the Brown Daily Herald and put it on the internet. I found out I couldn’t because at that point, advertising was still not allowed on the internet. They would disconnect you.
Harrison: Man. Wow.
James: Yeah. So before 1995, there was no advertising because the US government agency that managed the internet did not allow you to advertise on the internet.[1]
Harrison: I didn’t know. That’s why the internet took off in the late 1990s — because they allowed advertising on it. What I think about with the superhighway is everyone — there were people who were like, oh, we have the internet, but when are we getting the superhighway? And it’s like, well, you have the superhighway. It’s the internet, right?
And I think about that with agents now where you still have folks being like, how are we going to replace all these processes with agents? And it’s like, you did — it’s called Claude Code. If you built GPT-12 and GPT-12 is like Ultron and it can do anything — if you’re like, GPT-12, go solve ITSM — it’s not going to manually review a million tickets. It’s going to write code to do 99% of it.
You want to do as much at build time as possible because it’s cheaper, it’s faster, the AIs are probably better at coding than they are at anything else. Runtime is what you want to do the absolute bare minimum — exception cases, extremely fuzzy amounts of judgment, things that require a huge amount of natural language processing. That’s the only time you’re going to want to really do runtime. Then you might ask, if it’s that big an exception, it might be a small enough fraction that you may just want to give it to a human.
Harrison: You could just give it to a human. And that’s going to come back to what I was saying earlier — will the human on average make fewer bad calls? How often will humans outperform the machines in judgment? As time goes on, I think you’re going to see the machine judgment just continue to improve.
James: Let me offer a counter perspective. Large language models or GenAI is phenomenal at certain things — reading large amounts of text. It’s really bad at saying “I don’t know.”[3]
Harrison: It is, yes.
James: And the truly big edge cases, where there isn’t a dataset — we’ve never seen this edge case before. A lot of what you want from the human is less, oh, let me make a decision — can I even make this decision, or do I have to go ask these six other people?
Harrison: That’s a very fair point. And historically the LLMs have really struggled with that. They’re trained to maximize some objective function during reinforcement learning, and that objective function is answering math problems correctly, so it’s almost always valuable to guess. I’ve seen some interesting research about penalizing guessing.
There are many possible futures — there’s a world where that stays the case for decades.
James: From an efficiency standpoint, if you’re down to half a percent of all the transactions, whether you send those to an agent or send those to a human may not make that much of a difference.[4]
Harrison: It might not matter. And it might not be a dramatic amount of judgment either way — especially because in practice you want to get as few of those exception cases as possible. And often a huge portion of exceptions are not due to the fact that this couldn’t be done deterministically at build time, but because code was just expensive. Now that code is...
James: Code was expensive, yes. Code is now cheaper.
Harrison: And so as code becomes cheaper and cheaper, the exception cases are going to start declining.
James: The way I think about it is we’ve changed the operating leverage dynamics for automation.
4. What Becomes Scarce: When Code Is Cheap, Structure Is Scarce
When code is cheap, the traditional SaaS moat — “we understand the problem and we’ve solved it” — erodes.
Domain knowledge commoditizes; proprietary structure doesn’t.
What remains scarce: hardware and compute, unique data nobody else can access, rare human judgment that outperforms machines, and the ontology of a vertical.
Pure developer-army businesses face structural disruption. In periods of rapid change, the answer is not to wait.
Harrison: Companies in the past, especially on the software side, their value proposition was the software itself and domain knowledge — those would be the two things, especially in the SaaS space. We understand how this problem is supposed to be solved and we’ve solved it.
The latter is going to be not that important. What becomes scarce in a world where I can just give a very fuzzy spec and it just gets perfectly coded by a system that’s worked with millions of people and can really infer intent quite well? In that world, what becomes scarce?
Obviously hardware, chips, data, electric power — all of those things are scarce. And there’s also data that nobody else has access to — information that nobody else can see. That’s where software companies and companies in general are probably going to have to start consolidating around. You’re either going to need to double down on whatever rare human skills remain — good judgment that perhaps exceeds the LLMs.
Or hardware, physical resources, or unique data access. But there’s a whole world of companies that are just pure labor — our value proposition is that we have an army of developers, and we’ll go in for hire and fix your problems. That’s going to be dramatically disrupted.
James: Two thoughts for you. One is to your point about the exceptions — there are two reasons I think exceptions don’t get processed in an automated fashion. The first, which you mentioned, is it’s just expensive to write all the business rules. The second is — and we have to remember that business rules need data to operate on — the data required may be in a spreadsheet, in an email someplace, unstructured data that we haven’t been able to transact on.
Now we can, because in addition to writing code agentically, we can ingest data agentically. And then which leads me to believe there’s maybe one additional source of competitive advantage: structure. Because I think evidence indicates that models are much less likely to hallucinate when given a structure — which is why they write code better, they’re very good at writing code because it’s a structured output. When you don’t give them that type of structure, they’re more likely to have problems.
And then if you create the structure via the process of compounding, you can over time generate proprietary data. So I think getting the ontology right will be incredibly important and will be a source of competition going forward.
Harrison: I agree. The structure is the interface, in a sense. It’s the method of communication between parties. Even humans need structure when they talk to each other — that’s why they make forms.
James: Forms as ontology, I would suggest.
Harrison: And to me, when I think about ontology — I use that word a lot when we’re talking about our own software offering. It gets confusing sometimes because people mean different things. Some people mean the data ontology.
We often operate in a very process-centric world — what are the sequences of activities, business rules. But at a high level: what is the structure of your business? People call it use context now a lot.
Harrison: Like context engines. Buzzwords. But that ability to say we have a way of working with structure around it — structure is the good word — which includes things like the ontology of your data, the understanding of your processes, the context through which decisions need to be made, rules can be done deterministically. It becomes a way of operating as an enterprise. And that becomes a model that can be monetized and a competitive edge.
A lot of the idiosyncrasies are created for very good reasons, but sometimes those reasons don’t even exist anymore. They’re like vestigial organs of nopethe enterprise
Harrison: I think also it ties very closely to just scarce state in general. Because if you think about it — and I think scarce state still becomes critical for that — the reason is: how do you know your ontology is good? The only way you know it is good is to be able to use some judgment, but you probably want to have some evidence and measure it. And therefore, if you’re in a specific vertical, you’re a company dealing with that vertical, and you have access to lots of metadata, metrics, and in some cases raw data, you can start to learn what interfaces work, what didn’t work, what processes work.
And then you can build — could turn that not just into aggregates, but into ways of working. And I think that’s what we’re going to see companies start moving more and more towards and away from. Some of them are going to move sooner, some slower, many will die. Companies always die.
But I do think that’s what we’ll see.
Harrison: In the long term, even that activity is a job that I expect will eventually be automated. You will literally just have machines learning about how to build ways of working for machines to solve business problems, probably solving problems for other machines with their ways of working.
James: I’ll avoid the obligatory Skynet joke here.
Harrison: We already brought up Ultron.
James: Different generations.
Harrison: I was sitting with someone the other day and talking about AI, and they were like, the perception in society — not unjustifiably — people are, there’s some fear because even if there’s only a 25% chance it ends up like that, a 15% chance. That’s a big deal.
James: So tell us — you guys ingest lots of code and other documents in order to identify a baseline and then identify and propose a future state business process? Is that correct?
Harrison: We pull from a couple of sources — code, obviously, documents. We also focus a lot on SaaS and COTS, and that’s a big area for us. People largely ignore that because one, LLMs aren’t well trained on it, and two, a lot of people just don’t know about that space.
So we do a lot of SaaS and COTS configurations. And then also some runtime information as well — log analysis is supplemental — we’ll look into it but it’s not our focus. And the goal is to pull out data structure, business rules, process as the orchestrating engine of those things, and also what the current user experience looks like. And then to be able to work with the human users — if they have desired target end states, figure out to honor those or propose new end states, updated processes, areas of redundancy.
We’re working with companies where company A bought company B, and now we have two of everything. So where can we do process consolidation and use case consolidation? And because we’re focusing both at the technical level — mapping out every technical component, method, class, program, etc. — and then process, use case, actor, core data object, etc. — we can start to say, if you want to consolidate these processes, then that’s going to reach down through these three different systems and these different sub-components. And your roadmap to modernization is going to need to take these chunks and convert them into these chunks so that you can change this process.
James: I like the fact that you’re pointing out you can derive the business actions from the tech — you can derive a set of business actions or business processes from the technical code.
Harrison: Yes, absolutely. In fact, I think that’s one of the best places to get it.
James: There are some companies out there seeking to derive information about business processes from code, which I think is great. There are other companies seeking to derive information about business processes by instrumenting user screens and seeing what users do. How do you think about that — there’s some part of me that says it has to be both, in the sense that there is always preprocessing and there are things that happen under the covers that you can’t see from the screen.
Harrison: You can’t do it just through screen analysis. There are many lenses, and different viewpoints give you access to different data. If you really want to know exactly how a user is clicking and which screen they go to in which order, there is valuable process you can infer from that.
Likewise, if you’re looking at pure log files, there are things you can learn. But those are going to help you more if your goal is to automate the human. Think about the RPA world, really automating the system processes — they were trying to replace human swivel-chairing and human activities. This is a really important decision you have to make when you’re automating: are you trying to replace human labor hours that are using systems, or are you trying to replace systems themselves?
And if you want to replace a system, then you have to look at the parts of the process the system does. And those system processes are literally one-to-one defined by the code that executes.
James: I’d come at it differently — I want a better process end to end.
Harrison: Fair.
James: I presume some of the processes are handled by Excel gymnasts who are pre-processing and post-processing data. So therefore, if I really wanted to understand the processes just today and get to hopefully a zero-touch process in the future, if appropriate — I’d need to both do exactly what you’re describing in terms of interrogating the code and other data sets and artifacts, and at the same time instrumenting the screen.
Harrison: You can probably get away with skipping some of the screen instrumentation. But at a high level, some indication of user input or user activity is important. When we’re doing that type of analysis, we’ll take supplementary information — logs with information about who clicked what in what order, SOP documents, user forms. The companies we work with, we do ingest from their spreadsheets. And all of that can start to be correlated and consolidated together into that overall process. Usually as long as you get n minus one of the sources, you’ll be able to infer the last one — there are only so many possible sequences a user can click through these forms.
I see the log files, I see the spreadsheet they’re filling out, I see how the code’s working, I see the documentation they’ve written about it. You can start to infer a lot from there. Pure screen scraping also has its use — traditionally from our side, we’ll just look at the outputs of those. If they have some other screen scraping technology and they’re like, here’s the dump of that, we’ll just ingest the logs of it.
As opposed to trying to build our own — not worth it.
James: You can ingest data from multiple sources, and it’s a game of triangulation.
Harrison: Exactly.
James: What are the markers of a process that you think are most ripe for reinvention? I’m running a bank, a consumer package company, a pharmaceutical company — how should I think about where to apply this first versus later?
Harrison: It depends on your business problem. What’s the issue you’re having? Is it wanting things faster, costs lower?
Which of those really matter to you? If you have a company with a legacy SaaS license that’s enormously expensive and killing them, then the answer is: literally any important process in there that you care about, you should get out, because you’re paying licenses for those and you need to shed those licenses.
Now if it’s a company on a bunch of old mainframe stuff, where perhaps they’re not paying expensive SaaS licenses — then it becomes, what are you trying to get from modernizing here? Is it you want a more maintainable, updateable process? It’s often not performance, because those systems usually perform pretty well. It becomes more about process transformation.
If they’re on the mainframe, it’s very complex for them to maintain and modify those processes, and therefore as a business they’re held back from their competitors.
James: It works great so long as you don’t need to change anything.
Harrison: Exactly. So if your pain point is we need to change stuff, then it’s — which are the processes that you need to change? Is your goal more flexibility in process improvement, or is your goal cost cutting? And sometimes it’s both. That will dictate the processes.
When I work with clients, depending on your stakeholder, they’ll care about one versus the other — or depending on your industry or the situation they’re in. It’s going to be based off one of those two criteria normally.
James: So what have I not asked about? What else should viewers or readers know?
Harrison: Every business — if you’re running a software business, a consulting firm, anything like that, and your goal is not to just get out and sell immediately — you need to be running hopefully five, ten years. The world is changing very fast now. There are very different possible futures right now, and they have very different outcomes. It’s really worth thinking about the top futures you could see within the next few years and where you see your company going. And being a little almost extreme with that.
The possible deviation in futures is very extreme. If you go back 15 years, people were like, you have software, you have consulting, we’re going to bill, we’re going to charge licenses, and we’re going to just keep doing that forever. That was true till now. It might not be true anymore.
Ask yourself: what if our core value proposition at a global level evaporates in the next five years? And what does that mean for us as a company? Every company should do that.
James: What do you do with that once you’ve done it?
Harrison: You need to start changing your plans to capitalize on the main scarcity. If you’re a business, your goal is to monetize scarcity. If scarcity is changing at a global level, your business strategy needs to change at a global level.
Let’s say we are moving towards Ultron World. That means potentially everything will become difficult to monetize at some point, at least for a human. But not everything will become difficult to monetize at the same time.
You need to start thinking about what do you see evaporating and in what order? Your business roadmap and your product roadmap need to start reflecting your bet there. You might make the wrong call, but it’s probably better than just doing nothing and waiting to die a slow death. In periods of more rapid change, you need to take more risk.
Legacy systems are not technical problems. They are high-entropy records of how a business actually evolved — carrying rules nobody wrote down, idiosyncrasies that exist for reasons nobody remembers, and constraints that may or may not still apply. The AI opportunity is not to manage that entropy in real time. It is to reconstruct it at design time, convert it into something deterministic, and apply thirty years of software QA to the output. As the cost of code falls, the exception cases shrink. What remains scarce is not development capacity. It is the structured understanding of a business that nobody else can see.
Footnotes
[1]: The NSFNet Acceptable Use Policy, administered by the National Science Foundation, prohibited commercial traffic on the internet backbone until it was retired in April 1995. The commercial internet effectively began with that transition. See: NSFNet AUP history.
[2]: Deep Blue (IBM) defeated world champion Garry Kasparov in a six-game match in May 1997 — the first time a computer won a match against a reigning world champion under standard chess tournament conditions. The human win rate against top chess engines has been effectively zero at the professional level since roughly 2005–2006.
[3]: Research on LLM calibration confirms this. See Kadavath et al., “Language Models (Mostly) Know What They Know” (Anthropic, 2022), which found that while LLMs show some self-knowledge about their own uncertainty, they systematically overstate confidence. RLHF training compounds this: the objective function rewards confident correct answers, which penalizes hedging even when hedging is warranted.
[4]: The “half a percent” figure is illustrative, not empirical — Harrison and James are reasoning through the asymptotic case where AI handles the vast majority of transactions and only extreme edge cases remain. No benchmark is cited for the specific threshold.
[5]: G.K. Chesterton (1874–1936) was an English writer and social critic, best known for the Father Brown mysteries and for aphorisms that have outlasted his more topical arguments. The fence passage appears in The Thing: Why I Am a Catholic (1929): “There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, ‘I don’t see the use of this; let us clear it away.’ To which the more intelligent type of reformer will do well to answer: ‘If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.’”
A classic formulation of epistemic humility: before removing something, understand why it was put there. The reason may have been silly. It may have become irrelevant as circumstances changed. But you must understand before you act. The principle maps cleanly onto legacy code — where the original intent of a module or business rule is often invisible to later teams.
See Tom Nijhof-Verhees, “Legacy code: Chesterton Fence vs. Ólafur’s yellow box”, ITNEXT (November 2024); and Christos Galanopoulos, “Chesterton’s Fence”, Medium (September 2025), which documents a digital transformation team that stripped out “redundant” validation rules from an inventory system — they turned out to be court-mandated procedures from a legal settlement.
A related mirror concept: “Chesterton’s gap” — the constraint you should have built but didn’t, because nobody understood the problem well enough to know it was needed. The empirical version of the fence is the Scream Test: announce you’re removing something and wait to see who objects. If nobody does, maybe the fence never mattered.
This reminds me of a storage re-tiering exercise from 2007. Developers swore their applications needed the highest class of Fibre Channel storage. The head of infrastructure quietly moved their storage anyway to a lower SAN tier. Nobody noticed. Except finance, which was happy.
