“It’s a legal pad, a very fast legal pad.”
The project manager was older (must have been in his thirties!), upper class and British. [1] He had worked at Coba-MID [2] before coming to Deloitte & Touche Consulting Group. [3]
As we sat in a conference room in Denver, preparing to embark on one of the sillier CRM adventures of the 1990s, he had asked me “What can a computer do?” [4]
What form of information technology did the project manager trust? The yellow legal pad. He brought one to every meeting and carefully worked through every issue on its pages.
So I explained: “Think of a software program on a computer as a very fast legal pad. If you can do something on a legal pad, you can do it much more quickly via a software program. If you can’t do it on a legal pad, you can’t do it with a software program.”
He nodded. The products of the British public school system and Northern Valley Regional High School [5] had achieved shared context. [6]
Agentic ideologues will demand I retire my metaphor. They will argue that probabilistic reasoning allows us to build systems that we can’t reduce to a set of instructions on a legal pad. [7] And if you’ve never sat in a conference room, with worried professionals asking each other why the release can’t pass the bloody test cases, you might believe that.
Yes, generative AI allows us to do things we couldn’t do before. But we don’t have to put probabilistic reasoning in the core transaction flow.
Have a mess of unstructured data? You don’t have to interpret it afresh every time someone needs an answer. Use agents to extract and structure the information, validate the results, and load them into a database (possibly a graph). Then use fast, reliable Cypher or SQL queries at run-time.
Have thousands of pages of standard operating procedures outlining untold numbers of paths through and exceptions to a process? Use agents to derive explicit business rules, then test and validate those rules using decades’ worth of software quality assurance practice.
Users frustrated by having to tab through dozens of fields on several screens for data entry? Let them enter data as free text. Use an agent to map that input to the required fields, present the proposed transaction for approval, and let conventional software execute it. [8]
Now consider enterprise technology. Executives in other business functions marvel at how manual IT remains, despite employing armies of people whose business is automation. [9] Technology environments are incredibly intricate machines that (for most business processes) must work even in the face of a terrifying array of edge cases.
That’s why we have a whole discipline -- change control -- that traditionally involves people sitting on a videoconference asking each other “Well, what might this break?” It’s fun, even if it might breed a little neuroticism. [10]
If you’re prioritizing alerts, kinda-sorta may be good enough. If you’re automating server management, it isn’t. You have to demonstrate that the automation will work. It is a hell of a lot harder to do that if you don’t understand the internals.
I think Serval rests on this proposition. The company helps operational users — citizen developers — turn their knowledge of business processes into executable code. It also seeks to provide the testing, deployment, version control, permissions, and other infrastructure required to operate that code reliably. [11]
Many of Serval’s early implementations have been in IT, [12] itself an enormous business domain populated by thousands of operators and managers who aren’t necessarily software engineers. [13]
In today’s Prosaic Times, I discuss all this with Serval CTO Alex McLeod. We don’t endorse companies or technologies. But I like hearing directly from builders about the world they intend to create.
Why IT stayed manual
In this section:
Automating a task takes far longer than doing it by hand once, and time-pressed teams default to manual work
Handling exceptions can consume more effort than automating the core process
IT’s thousands of applications and configurations turn operating leverage against it
James Kaplan: Good afternoon. This is James Kaplan with another Prosaic Times podcast. I’m here with Alex McLeod, from — and I’m gonna try and pronounce this correctly — Serval.
And he’s the CTO there. And we look forward to having a whole bunch of discussions about technology automation of IT operations and of security. How’s that sound, Alex?
Alex McLeod: That sounds great. Looking forward to it.
James: All right, terrific. You wanna tell us a little bit about yourself and a little bit about the company?
Alex: Yeah, sure thing. So yeah, my name’s Alex McLeod. I am the CTO and co-founder of Serval.
My background is very much in startups and building companies from the ground up. And I spent a number of years at Verkada previously helping them build new product lines, until starting Serval about two and a half years ago. And Serval itself is an automation platform that helps anyone at a company automate away kind of repetitive work and toil so they can get back to meaningful work.
We do often enter and start with IT teams at businesses and help them automate employee support. It’s a big part of what we do. And then we will often move from there into other departments like HR, legal, finance, security, engineering, and so on.
And it’s been a fun journey and excited to chat about it.
James: So sometimes I joke that, with regards to IT, it’s not that the cobbler’s children don’t have shoes. They barely have feet. I think most people who have not worked in large enterprise technology organizations would be astonished by how much manual activity there has been historically.
That’s gotten better over the years. But talk to us a little bit about that. Why is there so little automation in IT historically?
Alex: Yeah. That’s a great question. The reason for that is it takes very significantly longer to automate something than to just do it manually one time.
And when people are pressed for time, they will then go and kind of choose to do the manual version of that task. So, as an example, you might have a task like resetting an MFA factor for a device. This is often a problem when new iPhones come out or new phones come out, and that may take 10 minutes, 20 minutes per case to handle manually. The automation of that historically could take a long time.
It could take days, weeks. It really depends on the organization. I’ve seen cases where the internal process, although it sounds simple on the surface, just reset the MFA factor, can be very nuanced, and there can be lots of different situations that you have to deal with and account for, and exception cases that you can’t really, maybe, require kind of thinking through up front. And so historically building those automations and deploying that can just take orders of magnitude longer than actually just manually resolving it.
The reason for that is it takes very significantly longer to automate something than to just do it manually one time. And when people are pressed for time, they will then go and kind of choose to do the manual version of that task.
James: The way I sometimes think about it is that operating leverage has been sometimes hard to come by in IT, which is maybe one way to say it. Let’s take a credit card.
In many cases, at least in consumer credit cards, one credit card provisioning action looks like the last ten thousand. But in an IT environment where you have thousands of applications and God knows how many configurations, the operating leverage dynamics can work against you. Is that a fair way of thinking about it?
Alex: I think that’s a fair way of thinking about it. I also think that when it comes to automation, the exception cases are what really kind of just take up all the time.
James: Well, exceptions are the enemy of operating leverage, right? And when it’s version four point two of the OS, we have to configure something differently than when it’s version four point one of something or other, right? Or when it’s in this location — or this country — we have to deal with the firewall rules or something, right?
Alex: Right, exactly. A hundred percent. That’s a good way to think about it.
DevOps and the operator problem
In this section:
DevOps made operational procedures repeatable but did not remove the effort of automating them
Automating takes a different mindset and skill set than operating
Agents may allow operators to encode their own process knowledge without becoming professional software engineers
James: So how did DevOps change some of that? How much did DevOps change the dynamics, right? Because in DevOps we started to push sysadmins, instead of doing things, to write scripts to do things. Now that was imperfect, but how did DevOps change that?
Alex: I think, broadly, it helps a lot to have this kind of like engineering or automation mindset to solving a lot of problems. And I think the kind of like DevOps mentality of how do we make these operational procedures repeatable and consistent and minimize kind of like one-off cases, I think that is 100% the right mentality. And it certainly helps.
It doesn’t perfectly resolve the situation of the — it takes time to kind of think through how do I automate something. And I think it takes a different kind of mindset as well, and it takes a bit of a different skill set. And so it’s directionally correct, but still there’s a lot of energy and effort required to go from more of a manual world into an automated world.
James: You imply something really interesting there. Okay, we always have debates. Who’s best positioned to think through automation?
Is it the business owner who’s close to the business process? Or is it someone who understands the discipline of automation? And you’re saying we still have that same tension inside the four walls of IT, because there may be a quote-unquote operator who is close to an operational process, but that person may not have an engineering mindset required to automate that process.
Alex: Yeah, I think that the engineering mindset can certainly be adopted and learned. I mean, one thing that we are very much doing and seeing with our platform Serval is that we’re enabling people at companies to be builders, to be engineers and to operate with that kind of mindset. And our goal is really to give the operators — the people that are closest to a business process and have the context of all the kind of like messy business rules and complexity — have that in mind.
And also understand the user as well, or the end customer, whether that’s an internal customer or otherwise. Have that deep understanding. And then we want to give them the tools to encode that into a system like ours, so that they can automate and get themselves more leverage, basically. We think that’s the ultimate best situation and direction.
And then the mindset of, well, how do you engineer this? How do you take this messy process and convert that into something repeatable or into a workflow of some kind? That is where we’re leaning on LLMs and agents to help give people that skill set and that capability. And so our view is that’s the direction.
James: Yeah, I joke that I want a large language model pre-trained on the principles of lean operations. Or a small language model pre-trained on the principles of lean operations. I also joke that I want a large language model that has only ingested good code from the internet as opposed to bad code, but that may be a little bit of an idealistic hope, if an impractical one.
Alex: I think that’s a great idea. It’s also — I think what my learning over time is that so much of this stuff is subjective, and there’s so much like taste that is still important. And that’s why it’s always a better-together situation with large language models and the human experts.
User experience beyond the chrome
In this section:
At a forum of about sixty-seven CTOs, nobody raised a hand for a beautiful IT experience and nearly everyone raised one for frustrated users
Teams can reinvest the toil automation removes in experience, not only in output
Good experience means fewer fields to tab through, and the next step is preventing the request altogether
James: Yeah. It’s funny.
We had our enterprise CTO forum yesterday; we had about sixty-seven CTOs together. [14] We were talking about various things, including user experience, and in one of the plenary sessions I asked the assembled group, how many of you think your users believe they get a beautiful experience from IT? And nobody raised their hands.
And then I said, how many people feel their users are frustrated with the experience they get from IT? Pretty much everyone raised his or her hand. And I think you make a really interesting point.
Listen, an architect can design a building that is either soul deadening or inspiring, and we can either define software that is not only productivity enhancing or productivity detracting, but also is soul deadening or, if not inspiring, at least pleasant.
Alex: Yeah, a hundred percent. And the teams that we work with, they really care.
They want to create that pleasant experience. They deeply care about that. It’s just about giving them the tools to make that possible, and giving them the time back effectively by making it as easy as possible to get to a great user experience, because that’s what’s hard.
James: Yeah, that was some of the discussion we got into, which is if you can reduce the toil associated with automation, some of that you may drop to the bottom line, some of that may be used to do more automation and automate more things, but some of that you might be able to reinvest in — let’s actually think about the user experience rather than just slamming it home in order to meet a production deadline.
Alex: Yeah, a hundred percent correct. And as I say, we’re seeing that across a lot of our customers at the moment, in many cases customers who have kind of existing automations built out in other platforms, and they’re looking at it and they’re like, “This is not it. This is not a great experience.” And they care.
James: Yeah, and so much of what we talk about great experience, I fear, sometimes focuses on the chrome. What’s the fonts, what’s the colors, as opposed to, gee, are we really asking people to tab through forty fields to automate one thing? I don’t care what the fonts look like. That’s gonna be painful, right?
Alex: Exactly. I think some of the organizations we’re working with kind of go a step further. And I think this is where it’s really going: you can make the experience of requesting something or getting help very, very easy and super simple.
And that is important. But you can also be thinking about how do I proactively prevent something or automate something away so that the user doesn’t even have to ask for it in the first place. And we’re really excited about that. We’ve built a lot of functionality around that, and we’re seeing that is a great direction.
You can make the experience of requesting something or getting help very, very easy and super simple. And that is important. But you can also be thinking about how do I proactively prevent something or automate something away so that the user doesn’t even have to ask for it in the first place.
From drag-and-drop to code
In this section:
Agentic coding is the capability that made this possible, and it arrived only a couple of years ago
Drag-and-drop builders constrain logic, while code is compact, reusable, and handles conditionals and errors
An agent abstracts the syntax and preserves the semantics
James: So would it be fair to say that what you’re trying to do would have been impossible two years ago or three years ago?
Alex: Without question. I think it just became possible. We were at a good time where it just started becoming possible.
James: And obviously we talked about the maturity of large language models, but what specifically is the capability from a large language model that allows you to do what you do that would have been impossible three years ago? What’s the specific unlock?
Alex: Agentic coding ability is basically what it comes down to. I mean, when we started Serval, the status quo for automation was — and I think largely still is — that if you want to automate something, you are building kind of like drag-and-drop workflows or automations, and you’re kind of piecing together logic visually. And maybe you’re scripting to some extent.
James: Which I will say — and I’m a little bit older than you — I hated that in the nineteen nineties. I recall in the mid nineteen nineties trying to explain to people, I can type code more quickly than I can drag and drop things onto it. And I can achieve a degree of reusability with textual code that is probably all but impossible once I’m dragging things around. Well, maybe if the drag and drop is just a front end to textual code, but stuff like Macromedia Director, there was no reusability, there was no ability for abstraction.
Everyone’s like, gee, we have a GUI. No, no, no. That doesn’t help that much. Sorry for the rant.
Alex: A hundred percent. No, I totally agree. It’s like — programming is incredibly flexible, and UI is actually kind of constraining, but maybe more approachable.
And so we had this idea early on of, well, why can’t I just chat with an agent that goes and writes code that represents my logic? Code is super flexible, it’s very compact representation of logic. I get all these great built-in primitives from the programming language, like conditionals and loops and error handling and all that good stuff. And that ability to do that — so you didn’t have to drag and drop and build a workflow visually, but you could actually just chat within something, write out a few sentences, and it would go and build it for you — became possible a couple years ago.
Code is super flexible, it’s very compact representation of logic. I get all these great built-in primitives from the programming language, like conditionals and loops and error handling and all that good stuff.
James: What I’d say is you’re preserving semantics but abstracting syntax. I don’t know about you, but I can never — it’s like, what the hell, what order do they want the parameters in? Let me go look that up, which is a giant pain in the neck, but the ability to abstract away that syntax but still retain the flexibility the code allows is quite an important thing.
Alex: 100%. I think the other thing as well that is very important is, as I mentioned up front, exception cases are where a lot of the problems come in with automation. And that’s where large language models are just incredibly valuable: you want to use code and determinism for certain parts of automation.
But you also want to have the ability for there to be kind of like reasoning and judgment for other aspects of automation. And so I’ll give you an example. If I am coming into my IT help desk and I’m like, “I need to reset my Okta password,” and I send a screenshot, and actually the screenshot is of maybe a Google Workspace login.
And the user maybe has gotten confused. You want to have a system that can have some amount of judgment to be like, “That looks like Google Workspace. I’ll trigger the Google Workspace automation instead.”
Judgment at the edges, determinism at the core
In this section:
Has the field overindexed on non-deterministic processing?
No one can predict every failure in a chaotic environment. Edge cases get judgment, and execution stays deterministic
Deterministic tools enforce authorization and rate limits, and can reject an invalid request or escalate it to a human
James: This is fascinating to me. This to me is a very fundamental question. And one of the reasons I want to interview you guys, interview you, excuse me, is I worry that we’ve overindexed on non-deterministic processing.
And what’s interesting to me, as I understand it, is that what you guys do a lot of is, in effect, okay, let’s have non-deterministic interaction between the analyst or the engineer or operator and the tool, which is constructing code. But then we construct deterministic code, and we can apply 30 years of software quality assurance practice to make sure it works. And I don’t know about you, but I sort of like that idea, especially if you’re touching systems in production.
And that to me is super exciting, but you raised something else. So I want to test the hypothesis now, because one of the joys of agentic coding is the automation of error handling. Because I don’t know about you — I mean, what’s worse than sitting down and thinking through every single error condition and trying to code those? That’s soul deadening, right?
Now, you challenged a hypothesis of mine in an interesting way, which is I would assume that at design time or at build time, you would provide an instruction to the agent to sort of say, okay, here’s the 50 possible edge cases. Go create code to trap each one of those edge cases.
Let’s say, I don’t know, browser version. Go research the zillion different outdated browser versions you might possibly have and create some error catch, error handling code for each browser condition. And that’s all deterministic, right?
But you’re saying you could even go one step further and even if the core of the script is deterministic, you might have some, I don’t know, ingestion, routing, what have you that is non-deterministic. Because even with the instruction, you may not be able to figure out all the potential errors. That’s a really interesting point.
Alex: I think that when a system becomes complex enough or chaotic enough, it becomes entropic enough.
James: Entropic enough is the term I use. Actually, the system isn’t entropic.
The environment around it is entropic. We build system complexity to deal with environmental entropy. That’s maybe the right way to describe it, right?
Alex: Exactly. And I think when a system becomes entropic enough, it becomes effectively impossible to predict every failure case and every situation that can go wrong. I remember Netflix had this popular idea of chaos engineering, maybe it was a decade ago, where they recognized that within their kind of software systems, where there’s enough services and enough interactions between services that you can no longer kind of upfront predict the different failure modes. So instead they would inject failure modes into the chaotic system and see how different parts of the system would react, right?
James: Okay. Every banking CTO watching this is now screaming at the screen: that’s fine to do with video streaming, not with high dollar volume transactions. It’s like, there was a glitch in my video stream, let me hit play again.
Alex: Totally. I totally agree. But I think that the core idea there is that as entropy increases, it becomes impossible to predict failure modes. So you do want to test resilience to failure.
It’s an interesting way of thinking about things. And I think that’s true in all kind of like IT operations and internal operations at businesses: the environment is just so chaotic and complex that you can never predict every single failure mode. And so you want to encode some of those, but you also want to give some amount of freedom or flexibility to an intelligent entity, which is an LLM, to deal with the edge cases that you’re comfortable with, right? So the example I gave you of, like, resetting a password.
You don’t want to connect the LLM directly into all of your business systems and say you can take any action you want. But if it’s got two tools, one for resetting Okta password, one for resetting a Google Workspace password, it’s okay if it picks one or the other and has some intelligence there. As long as when it executes that tool, the reset happens very deterministically.
One thing you might want to be deterministic about is, you might want to say, “A user is allowed to reset their password anytime they want — sorry, anytime as long as they haven’t reset it in the last three hours.” You want some kind of rate limiting, and you want that logic encoded into the tool. And that’s where you want code literally checking, when was the last time they did a reset? “Okay, it’s been less than three hours — fail.”
That’s kind of authorization logic or rate-limiting logic that you really want to bake into the tool. But the selection of which tool to use in which situation, it’s okay to allow some amount of reasoning and intelligence to make that selection. That’s how I think about it.
You don’t want to connect the LLM directly into all of your business systems and say you can take any action you want. But if it’s got two tools, one for resetting Okta password, one for resetting a Google Workspace password, it’s okay if it picks one or the other and has some intelligence there. As long as when it executes that tool, the reset happens very deterministically.
James: What I’m hearing from you is — what you’re suggesting is the combination of non-deterministic routing and deterministic execution can feel okay from a risk and resiliency standpoint. Because if, worst case, you make a bad routing decision, the deterministic code says, “I can’t do anything about this,” and it gets kicked back to a human operator, right?
Alex: Exactly. And you can be thoughtful about what are the boundaries around which I’m comfortable allowing a level of judgment and reasoning. And that, we see as like combining the best of all worlds together, to automate more processes kind of in a really safe manner.
James: Yeah, I mean, the other thing I’m thinking about is a bit of a tangent. I wonder if we’re going to see more modeling of complex technology environments, which is — you can think about a technology environment as a graph, where every piece of technology, every system, every application is a node, and every network link and every system call is an edge. And then, in effect, you can have a non-deterministic chaos monkey that figures out the tests. A non-deterministic virtual chaos monkey to figure out where you might possibly break the system, to sort of figure out the edge cases you need to automate around.
Alex: I think that’s a great idea. I think that, like, simulating the full environment and simulating, yeah, what goes wrong if we enable a user to have this level of flexibility, and where should we take that flexible decision and maybe make that non-flexible and kind of encode that in a deterministic system? And where is something deterministic that really could be flexible and non-deterministic? I’m seeing a lot of that today.
We’re seeing, actually, and working with a number of companies that are trying to simulate business systems. So, like Workday, Okta, Slack. So you can actually iterate and test your work in a simulated environment. So you’re not testing it in a production environment, which is exciting.
James: Yeah. I mean, I’d like to see more people even take that up a level from the operational level to the strategic level, so they can test business strategy. But now we’re really far afield.
It’s a fascinating time, right? So much less depressing than fifteen years ago, when there were like six vendors and it was a giant oligopoly, right? Or twenty years ago.
Why not just use a coding agent?
In this section:
If frontier labs absorb enterprise software, who needs a platform?
Agent-written scripts still need deployment, version control, system access, permissions, approvals, and someone on call
Writing code is a small share of running software at scale, thirty years of discipline for managing code has to evolve for agents, and Serval aims to package that discipline for people who are not engineers
James: One other thing we discuss as we talk about an oligopoly — what we discussed at the CTO forum — is how much of the world will wind up being controlled by the frontier model. So if you go back ten years or maybe eight years, there were a bunch of people saying the entire enterprise software market will disappear and will be entirely owned by the cloud service providers, and as of yesterday morning that had not happened.
Now there are other people saying, gee, the entirety of the enterprise software market will be consumed by the frontier labs. Now, this is actually maybe a little bit more plausible a hypothesis, because somebody, perhaps wrongly, might listen to this conversation and say, okay, I have my favorite coding agent.
Why do I need a third-party platform? I can take advantage of agentic software development all by myself. I could tell my coding agent to develop error trapping for 50 different browser versions and what have you. So how do you think about what will get done, in terms of, say, IT automation, via coding agents versus more specialized platforms or more targeted —
Alex: Yeah. So I mean, the way that I think about it is you need a full platform like ours to effectively deploy those automations and maintain them over time. So in the example you’re talking about there, of like, well, can I just use a coding agent and build out some scripts or things like that to solve these different cases?
James: And clearly people are doing that in various places. That is happening.
Alex: Yeah, for sure. And it’s like, yeah, you can build out those scripts, but then what do you deploy them to? How do you version control those things?
And then when they’re deployed, how do they get access to your business systems? And then if someone wants to make a modification to that script, then how does that work? And what permissioning and role does that person need to make that modification?
And then what if you need an approval procedure through to execute a particular script? Well, how is that executed? And how does that get routed to the right person?
And then what happens if, through all those things, something goes wrong in the middle of the night, right? Like, the writing of software is — I think we are all relearning, maybe — a small percentage of the kind of effort required and kind of work required to deploy and run software and maintain it over time at scale. And so I do think you need coding agents. So coding agents are incredibly powerful, but then you need all of the different kind of systems and capabilities around them to leverage that capability effectively, which is certainly what we provide at Serval and what we focus on.
You can build out those scripts, but then what do you deploy them to? How do you version control those things? And then when they’re deployed, how do they get access to your business systems?
James: Okay. That’s interesting.
That’s a little bit different from the answer I thought you were gonna give. I thought you were gonna give a more tactical answer, which is, hey, if you’re gonna automate IT stuff, you need to interrogate and control AWS, and you need to control ServiceNow and a million different things. And you guys had — and maybe you have done this — some degree of understanding of the object models and the API models, and whatever MCPs have been exposed that allow you to avoid a lot of trial and error in manipulating much of that stuff.
Alex: Yes, which is also definitely true. But I think it’s more — no, for sure, without question, that is definitely true. It is just like there’s so many pieces around the deployment and execution of software that are critical. And so for a platform like — sorry, go on.
James: And non-trivial, I would add. What you’re saying is really interesting.
It’s quite a big thing, because what you’re saying, and I would agree with this, is we’re starting to do software development differently, right? And much of the discipline that the world developed over the past 30 years is going to need to evolve. Not all of it.
We’re gonna build on top of it, it’s not all gonna get thrown out, but we’re gonna need to build on top of it, because contexts, skills — how do you version skills that can invoke scripts? I mean, a million things. And what you’re saying is you guys aspire to provide some of the discipline that the world has developed in managing code over the past 30 years to the process of managing agentic development and all the stuff that lives above code in this case. Okay.
Alex: Yes. Exactly.
And we’re actually taking a lot of the — I guess the tools and techniques and capabilities that software engineers have relied on for a long time. We’re packaging that all up into our platform. And then we’re actually making that available to anyone at a company, in IT or beyond, so that they can make use of those different capabilities.
So we’re kind of packaging up all these capabilities that we have come to rely on as software engineers, and techniques we’ve come to rely on as software engineers, into a platform that lets people who are not traditionally software engineers automate things at their company, to reduce that toil and get back to meaningful work. And that’s very much how we think about the entire Serval platform: it’s a software development system. You can have an entire software development lifecycle on top of Serval that includes code, it includes skills, it includes all configuration. It’s just that we’re making that available to maybe people who wouldn’t traditionally go through that software development lifecycle.
Guardrails for citizen developers
In this section:
Builders range from vibe coders to Wall Street strats. A personal automation is low risk, but a password reset can hurt people
The critical skill is architecting a system in natural language, and SQL is a precedent for declarative programming
Generated code gets a constrained SDK, deterministic type checking, and LLM review
James: It’s interesting because there’s two flavors of what we’ve been discussing. One flavor is for software engineers who need something, because it’s not like they’re doing the same thing that they were doing two, even three years ago.
Then in addition you have what — I hate the term vibe coder, but let’s say citizen developer, right — who is going to try and build things with some degree of discipline, as opposed to just sitting down and saying, give me a report, and I don’t care how good or bad the underlying code is. And you guys are more focused on the second use case, the citizen developer? Is that a true statement?
Alex: Yeah, we’re certainly enabling that, that kind of individual. Again, often those people maybe are in an IT department, but they may be in other departments as well. We’re enabling them to have automation capabilities so they can get to the outcome they want and kind of can encode the business rules and context that is important for their domain, without tapping a software engineer on the shoulder.
James: So let me ask this question. How much do you think a citizen developer needs to understand in order to be competent? At one end of the spectrum is the vibe coder, right? At the other end of the spectrum — you know what a strat is?
Alex: A strat.
James: Yeah, they’re the folks in investment banks who sit on the trading floor and code trading algorithms. And they are people who deeply, deeply know capital markets and also in many cases are very good software engineers. Sometimes they have computer science backgrounds, sometimes they have finance backgrounds, and I think the history there is they put the capital markets person sitting next to the software engineer, and after all nobody could tell which was which. And that is often a fairly sophisticated person.
As you think about a citizen developer, where on the continuum do you think that person needs to be? How much does that person need to know to avoid hurting him or herself and people in the vicinity?
Alex: Yeah. I think it depends on what you’re building and the scope of that.
If I’m building an automation for my personal use, I think anyone can do that, and you can iterate on it, and it’s just gonna use your credentials and your access that you have anyway. And there’s less of a long-term maintenance challenge. I think when you are —
James: Fair. But if you’re building a password reset application, you could hurt people with that, right?
Alex: Yeah, I think you need to have — again, I think it comes back to a bit of an automation mentality of how you think about what actions should be allowed, what shouldn’t be allowed, and less about, to your point, an ability to kind of take that mentality and the vision and convert that into syntax and code. That is less important now.
But you do need the mentality of being able to kind of architect a vision of how a system should work in your head and document it in natural language. I think that’s the critical skill.
But you do need the mentality of being able to kind of architect a vision of how a system should work in your head and document it in natural language. I think that’s the critical skill.
James: Yes. Sometimes the metaphor I use is — and this is before even I remember — in the early nineteen eighties, when people started using SQL to build databases.
Before that, people actually hand coded the database, using procedural language, and then SQL, which I think we can describe as a declarative language, right? [15] You just describe the query you want. You’d still need to know SQL, but you ripped about, what, ninety percent of the effort out of constructing a database and what have you. [16] So I wonder if that’s the metaphor, that we’re moving more things to a declarative programming model, which is more democratic in certain respects.
Alex: Absolutely. Exactly right.
And you declare in a language they’re all familiar with. And as, I guess, with declarative programming, you’re less concerned about how that gets executed under the hood, how it gets to that outcome. And so you need the system underneath to be robust, and that execution logic, I guess, to be very well thought through and tested internally.
James: So let me ask this question. One thing I’ve noticed about coding agents — some of you may have a different experience — is they will hard code anything. They’re like the laziest software.
Unless you explicitly have instructions not to hard code things, they will hard code things. So how do you think about that enforcement of reasonable programming practice? Now obviously that’s a doable thing, and you can include checks for architectural smells and what have you, but is that something you do? How do you think about that?
Alex: Yeah. The way that we approach that, in a lot of cases: for our system, we’ve built our own SDK. It’s a TypeScript SDK, and that is the library that basically the agent uses to write all of its code.
And as a result, the type of code that we’re expecting is relatively constrained. That enables us to do, one, a reasonable amount of sanity checking of the output, and that could be done by an LLM. But also, two, we can use fully deterministic checks, like just type checking, for example, which won’t necessarily catch hard coding, but it does catch misuse of an API, for example, where you pass a parameter that doesn’t actually exist, or a type for a value that is incorrect.
James: Actually, large language models are pretty good at catching coding errors if you instruct them to look for the coding errors, which obviously many people neglect to do.
Alex: Yes. And we can kind of encode those opinions into — and I wouldn’t say that’s necessarily opinion, but we can encode tastes and opinions into our system and our agent that runs on top of the Serval platform, which isn’t just a coding agent, because it’s writing code in a library that we have constructed, and we can see over time how best to write code that operates with our system. So that has a lot of advantages.
Adoption and the automation engineer
In this section:
Customers range from “anyone can build anything” to tightly controlled, and the platform lets them set the level
A new role is emerging: the internal automation engineer who works as a forward deployed engineer for other departments
The role pairs platform expertise with business context
James: One final question is, as you engage with people looking at your platform — and we’ve talked about a lot of complexity here, important stuff — to what extent do you think the people considering implementing your platform get that? And to what extent, at the other end of the continuum, is it like, hey, we’re gonna give users a platform and goodness will emerge? What’s the level of recognition that citizen development is good, but it’s not thoughtless, right?
Alex: Yeah, for sure. And I think the idea of complete — anyone at a company can build anything — I think there are varying degrees of comfort with that or interest in that. In some businesses, that just is not going to make any sense.
And that’s not gonna be the direction they want. And for those companies, we are an incredible solution. We let you control the level of flexibility you have across an org.
I think that’s kind of a continuum, I guess, of comfort or interest in having automation being spread across the business. For a platform like ours, we give you all the controls and the capabilities and enable you to make the judgment calls of how those are kind of leveraged around the org. The thing we’re seeing as well, which I think is incredibly interesting, is the kind of rise of this automation engineer.
Which maybe that’s somebody who is in an IT department or otherwise at a business who becomes an expert in a platform like ours and uses it to automate password resets and the long tail of kind of IT issues, but then they become effectively like an internal forward deployed engineer for other parts of the business to go out and help them automate the capabilities or the issues in their environment. So they can then work with other people who have the deeper context that maybe aren’t as interested in being the automation engine themselves. They can work with those people to create tremendous amounts of value and give a lot of time back. And we’re very excited about that direction, and seeing that happen across our customer base.
The thing we’re seeing as well, which I think is incredibly interesting, is the kind of rise of this automation engineer. Which maybe that’s somebody who is in an IT department or otherwise at a business who becomes an expert in a platform like ours and uses it to automate password resets and the long tail of kind of IT issues, but then they become effectively like an internal forward deployed engineer for other parts of the business to go out and help them automate the capabilities or the issues in their environment.
James: Terrific. Anything that I neglected to ask about?
Alex: I think we covered a lot of the areas I thought might be interesting in this conversation. So nothing springs to mind.
James: Right. Thank you so much for joining us. This was great.
Alex: This is fantastic. Thank you, James. I really appreciate the time.
Not really a legal pad
Okay, you can’t really do everything you can do in software on a legal pad, at least not without an infinite amount of time and a capacious supply closet. But legal pads have had some terrific advantages over software. Using a legal pad to perform an analysis or keep track of a process has almost no fixed costs, is flexible and requires no ongoing technical support — even if it’s prone to error and impossible to scale efficiently.
Traditionally, CIOs have faced a brutal tradeoff in enterprise technology, especially in entropic business domains with messy business rules and data. Every incremental unit of automation or analytics functionality has required substantial investment to build and maintain. Which is worse? The Scylla of users frustrated by incomplete automation with a narrow happy path? Or the Charybdis of a CFO appalled by spiraling technology costs?
Agentic technologies offer the possibility of shifting the efficient production frontier outward: more automation and analytics for the same expenditure, or the same functionality at lower cost. This changes the minimum efficient scale for any given requirement -- you can automate exceptions that you would have previously forced onto users for manual intervention.
Not a small thing for companies where the pace of digitization has been a rate-limiting factor for their business strategy.
CIOs and CTOs might ponder two questions.
Where in the digitization value chain and technology stack should they deploy agents? What combination of design time or run time? In the presentation layer, the application layer or the data layer?
Can they give the cobbler’s children feet? Can enterprise technology provide a proof point in using agentic technologies to achieve extreme automation? Could you aspire to:
Roughly 90% of routine user IT requests fulfilled without human intervention.
95%+ of standard infrastructure changes executed automatically.
What changes in governance, platforms, risk management and operating models would that require?
Footnotes
[1] One of the other associates reported the manager claimed descent from a buccaneering sea captain who danced on the line between commerce and privateering. Sadly, a common surname precludes confirmation of this detail.
[2] In the 1990s, we listened to the Gin Blossoms on cassette in our cars and sought interviews at a million boutique consulting firms: Marakon, CDI, Mitchell Madison, none of which I could get hired at. Yes, the Marakon folks were smart, but given the firm’s subsequent history, I’m lucky I blew the final-round case interview in 1996. Couldn’t figure out that holding too much cash depressed returns for a manufacturing company.
Very little available on Coba-MID, but there are enough references to it in CVs and the like to confirm I am not engaging in carbon-based hallucination here.
I’m (much) less cynical about consulting than the author of Consulting Demons, but it’s an amusing memoir of a more buccaneering, less risk-averse and often worrisome era in the industry’s history.
[3] Later Deloitte Consulting. Then Braxton for a little bit. Then Deloitte Consulting again. Some of the consulting partners of the era just wanted to call it Touche Ross.
[4] Historians call the Monroe presidency (1817–1825) the Era of Good Feelings. Their successors will refer to the 1990s as the era of silly CRM adventures.
[5] Admittedly by way of Brown University and the Wharton School.
[6] Reuse is essential. You build a function that works, you keep invoking it. I have used the fast legal pad metaphor hundreds of times in the ensuing decades.
[7] You might respond that given enough time and enough legal pads you could replicate the operation of a GPT manually. You shouldn’t, but you could.
More interestingly, Goodfire researcher Nick Cammarata argues, as a working hypothesis, that most neurons in trained networks do one identifiable job, and that the apparent tangle was largely a misreading of the evidence.
If he is right, a model’s internals are more legible than I assumed. His results are preliminary, and he works at an interpretability company. Understanding individual neurons is not the same as demonstrating the reliability of an entire system.
[8] Prompt injection is the risk this leaves. Scoped permissions and least privilege limit what a manipulated agent can reach, and a program can check the agent’s plan against security policy before it acts. Assume eventual penetration.
[9] They should have seen what it was like 20 years ago, before infrastructure as code became widespread.
[10] Change control can be a test, not a meeting: a change proves itself in a sandbox, against an adversary agent, before it rolls out in stages. Rich Isenberg: “Autonomy without governance is not innovation; it’s negligence.” An agent can’t lose its job. A named person stays on the hook.
[11] Business users want use cases quickly, and someone has to build the shared layer once: enterprise technology provides the platform and citizen developers build on top. Some companies will build their own platform. Others will buy one.
[12] Here are some of the case examples Serval provides on its website: Cribl reports building in twenty minutes an automation that took consultants three weeks. SeatGeek reports a 50% automation rate within sixty days. Abridge reports clearing a two-month ticket backlog in two weeks.
[13] Skills have boundaries. Mindsets compound. A good developer pairs curiosity with intellectual precision, and asks “What do I mean by X?” as well as “How do I do X?” Five habits for a citizen developer: automate what you understand, remember that models aren’t magic, make agents write things down, keep business logic deterministic and tested, and probe the agent as you would a new joiner.
[14] The Technology Leadership Forum convenes 50–70 CIOs, CTOs, CISOs and enterprise technology executives every spring and fall.
[15] I overstated this in the interview. SQL did not remove the work of designing databases. It separated what you want from how the system gets it: you write the query, and the database decides how to run it.
[16] Thomas Haigh on the tape era: “Every time a field was added or changed all the programs working with the file would need to be rewritten.” And on relational systems: “deleting all records more than a year old or adding 10% to the sales price of every item could be performed with a single command.” Demand for applications was outrunning what programmers could build, and specialists often wrote whole programs to fetch one piece of data. I have no source for ninety percent.
