Getting the message across: from fax machines to using LLMs to create an audience panel
Also: what a future CEO taught me about stapling the right way signals care
In today’s issue of Prosaic Times
Early thoughts: initial perspectives from my travels -- not fleshed out enough to be a full article (but may go that way in the future)
The main argument: how to use a LLM audience panel to make your board presentations, employee town halls and RFP responses more effective
Throwback Thursday: a future CEO teaching me about getting faxes and stapling right — as we did a multi-channel strategy!
Early thoughts
Some observations from this weeks travels. These may become articles in the future, but I haven’t had a chance flesh them out yet. (I’m frenetic, but not the frenetic.) [1]
Non-deterministic cybersecurity
Really enjoyed being on a panel with Kong CTO Marco Palladino (maybe a future interviewee on a Prosaic Times podcast? We’ll see!). We talked about how agentic workflows will challenge existing cybersecurity models.
Our cybersecurity controls are deterministic -- encryption at rest for all customer PII, never let anyone email a social security number outside the organization. But much of information protection is much fuzzier. Is this strategy document baked enough for someone to email it to an important vendor-partner? It depends.
So not only do we need controls fit-for-purpose for agentic workflows. We also may have the opportunity to create agentic controls. And, as a result of that, think more in terms of business risks than “encryption” or “multi-factor authentication.”
Will we talk about this in the podcast with my colleagues Charlie Lewis and Rich Isenberg that should go live Sunday night? We just might!
The trust challenge in enterprise SaaS
Yes, (like everyone else) I’m bearish on enterprise SaaS because of the intersection of the “one size fits none” problem with a coming secular reduction in the cost of software engineering (which will reduce the replacement value of existing SaaS code).
There is also an interesting trust dynamic here. Most of the cloud infrastructure breaches we know of are misconfiguration rather than the result of an underlying technical compromise. The major CSPs make it their business to take security very seriously. The market insists on it.
SaaS is much more fragmented. Some SaaS providers take security very seriously. Others less so. How easy is it for any enterprise to figure out which is which? Not very!
The main argument: Using an LLM panel to get your message across to your audience
Communicating is part of the job for CIOs and CTOs: board presentations, strategy documents, employee halls and, at least in B2B markets, customer meetings. Effectiveness depends on audience. You wouldn’t communicate to the board the way you communicate in an all-hands meeting.
Historically, you’ve tried to align message, tone and audience by guesswork. Now you ask ChatGPT (or Gemini or Grok) to review an important email or presentation. It gives you generic feedback — “put the message up front,” “be more concise.”
You can get dramatically better input: create a panel of synthetic readers who mirror the people you’re trying to reach.
What you need to know: A panel of LLM-based potential readers can provide more insightful input into your business communications
What you need to do: Create relevant panels (e.g. board members, employees, customers) to give you a range of feedback on high stakes communications
What you need to decide: Which characteristics matter for your audience — role and seniority? Personality type? Tenure and experience? I used demographics (role, sector, company size) plus OCEAN personality traits; you might weight different factors for a board deck vs. an all-hands script.
I want each Prosaic Times article to resonate with readers, especially readers who have to run large, messy, complicated enterprise technology organizations. As you would expect, I ask multiple LLMs “will this resonate with a senior enterprise technology executive?” But what does that mean -- a CIO who has spent 20 years at an industrial manufacturer will respond to different messages from a CISO who spends two years at a company and then moves on to fix the next program. I so created a panel of 500 senior technology executives to road test my articles.
Learnings in setting up an audience panel
A few learnings if you want to do something like this at home
1. Tune you panel model to your format. I don’t know all my readers (but I know many of you!), so I created a large synthetic panel. A CTO I know created a much personalized panel based on the governance committee he has to work with. He captured detailed insights about each member and uploaded materials they had created.
2. If you create a synthetic panel, use a wide range of characteristics. I used a combination of
Personal demographics -- e.g. age, location, time in role
Personal characteristics based on OCEAN model. [2] -- openness, conscientiousness, extraversion, agreeableness, sensitivity
Company demographics (e.g. size, sector, employees, IT budget)
Tech strategy and environment (e.g. cloud adoption, outsourcing,)
3 . Be specific in the criteria you provide. I used relevance, insight, credibility, clarity, enjoyability, and I provided bullets about what each of these mean. The rubric I used is below, but I think I’m going to update this (in the next few days) to a more structured scoring sheet.
4 . Test your criteria and panel against materials you have written already. The first scoring I did resulted in very little variation across members of the panel, so I increased the variability of opinions in panel members. This yielded more interesting results.
5. Aggregate you panel into segments. I also asked the model to segment the panel by personal preferences and characteristics (using the OCEAN model as a structure). It suggested:
Creative Visionary — Connects dots others miss, though sometimes the connections feel like a stretch. Comfortable with ambiguity, and pushes back on conventional approaches
Diligent Traditionalist — Asks for the data behind the recommendation. Colleagues trust them to do the homework, follow the process, and catch what others miss.
Outgoing Collaborator — Works the hallways, building coalitions and making introductions. Thrives in meetings and leaves with action items that involve connecting people.
Calm Skeptic — Doesn’t react to the crisis du jour. Has seen enough cycles to know that most things that feel urgent aren’t. Needs to be convinced.
Anxious Achiever — Always preparing for the next thing. Prepares for meetings, anticipates objections before they’re raised, and carries mental checklists of everything that could go wrong. A high performer with high standards
Independent Challenger — Asks the uncomfortable question -- the person in the room who probes when others nod. Doesn’t need to be liked, but needs to get to the right answer.
Quiet Observer — Built their career on listening more than talking. Processes ideas privately, tests them against experience, and influences through quiet credibility rather than volume.
Steady Generalist — Just gets things done. Colleagues see someone reliable, even-keeled, and adaptable — not wedded to any particular philosophy or approach. Easy to work with, low drama.
This provided a lot of insight (to me) I might push this a bit further and play around with segments that combine personal, demographic and company information. Will revert with how that works out.
Creative Visionaries and Quiet Observers are big Prosaic Times fans -- calm skeptics less so!
6. Get quotes. I asked my panel to give me 20 positive comments and 20 negative comments. For example, I asked for feedback on an earlier draft of this article:
“The meta-move of using an LLM panel to write about using LLM panels is delightfully self-aware. This is how you demonstrate a concept.” — CIO, Tech, age 47
“The stapler story isn’t about staplers — it’s about signaling theory. Visible effort as communication. That’s the real insight buried in the nostalgia.” — CIO, Healthcare, age 53
“The honesty about which articles worked and which didn’t is refreshing. Most people wouldn’t publish their own mediocre scores.” — CTO, Tech, age 47
“How do we know the LLM panel responses are valid? Couldn’t you tune the prompts to get whatever answer you wanted?” — CTO, Manufacturing, age 53
“The fax machine nostalgia feels like pandering to older readers. Not everyone finds pre-digital inefficiency charming.” — CTO, Aerospace & Defense, age 61
What I took away
I’m going to try to use this feedback to make Prosaic Times better and more relevant.
Personal narrative and professional development resonate -- The highest performing articles (“The first thing I ever fixed,“ “Things I was too stupid to know“) addressed being a technologist rather than technology itself; these were popular across segments. I decided to include the story below on faxes and staplers based on this input!
So do cross-domain analogies -- “Moneyball for business writing” and “Let’s not have software that looks like Tyson’s Corner” both take something from outside enterprise tech (baseball analytics, suburban architecture) and use it to illuminate a familiar problem
Articles on technology architecture and economics felt dense and inaccessible -- though the “what you need to know/do/decide” helps here
As much how as what -- some of the quotes asked for more guidance on how to apply the ideas Prosaic Times discusses. I structured and re-framed this article based on that feedback. [3]
This matters even if you don’t have a newsletter
Everyone reading this article writes in some way for professional reasons -- board decks, all-hands meetings, RFP responses, business cases, etc. I’m sure that everyone asks a LLM for feedback. Many ask multiple LLMs, as I have. We can push this further.
If you, for example, hold staff town halls should you create a panel of employees (based on the professional demographic of your organization) and use feedback from it to tune your script? If you present to the board, should you store context for each of your board members and use that to red-team your presentation?
Throwback Thursday: A future CEO teaching me about getting faxes and staples right
I will try to pay attention to some of the panel’s advice — yes, I should be careful about length. Some advice I will ignore — I couldn’t be the “here’s five tactical things you should do Monday” guy if I wanted to. And I don’t.
I will lean into personal narrative. Stories stick in ways that frameworks don’t. They encode tacit knowledge that bullet points can’t capture. The reminded me to live the advice I provide about how to use narrative to influence people.
Today we can use LLMs to provide detailed feedback on how effective our documents are. Not too long ago we were happy if we could get the document faxed to the right person or printed out correctly. Doing that could be a huge victory (and fount of learning) for a young professional like myself at the turn of the century.
A painfully enthusiastic college student asked me about what I was most proud of in my time at McKinsey & Company. Naturally, I told him about my experience helping Jon Beyman, Jim Rosenthal and Phil Venables set up Sheltered Harbor. Relatively rarely do you get to work on something that people write thriller novels about. Power in the Blood by Hiawatha Bray may not be Eric Ambler (or Alan Furst), but it’s still pretty good!
But that was not my greatest triumph, which happened much earlier in my tenure at the Firm.
Very quickly after joining the Firm I received a battlefield promotion to Engagement Manager on a multi-channel strategy for a call center outsourcer. We had weekly check-ins with CEO and head of strategy.
I was very excited for our first check in — I never met a CEO at the previous consulting firm I worked at. Not sure I ever met someone who had met the CEO. It was a phone call with the partners and me in the Chicago office, and the clients on the road somewhere.
I brought printouts of the deck into the partner’s office. (For our young readers, before there was a big screen in every office…oh, never mind.) We joined the audio-conference.
I started to describe the analysis. Tim Gokey stopped me and asked the CEO if he had the deck. He did not. I said I had emailed it to him this morning, but this was an era before iPads, wireless internet or even Blackberries. The mid-level clients I worked with at my previous employer took their laptops everywhere. CEOs did not.
So I took my printout and went to a fax machine. Twenty minutes later I managed to get the fax through to the CEO, and I returned, shame-facedly, for the last five minutes of the call.
We had another check-in call a week later. At the very start of the call, Tim asked the CEO, “Do you have the deck?”
The CEO chuckled and said, “It’s funny you ask. When I went into my home office and the deck was sitting in my fax machine. Then I look in my personal email, and the deck was there. I went to the office. As I walked to my desk, my assistant handed me a copy of the deck. When I sat down, I saw the deck sitting in my personal fax machine. Then I logged into my work email and there I saw the deck. So, yes, I think I can say I have the deck.”
“Ah,” Tim responded, “the multi-channel approach.”
I smiled in satisfaction, very glad that I had personally shelled out $10 per month for an efax account. (For our younger readers, awareness of third-party cybersecurity exposure…oh, never mind.)
In twenty-five years at Firm, I have done many bigger things, but I don’t think I’ve ever done anything as perfectly as making sure the CEO had that dang deck. Hell, I didn’t even manage to meet Tim’s bar in every circumstance. I couldn’t always get the right meeting on the right partners calendar. I hadn’t quite figured top-down document structure. Sometimes I even struggled with the copy machine.
For the final progress review, we had a deck, an appendix, a vertical business case and an excel model. When Tim arrived at the client site, he saw me racing between copy machines to make sure we had 30 copies of each document in time for the meeting. (For our younger readers, we used to have physical printouts...oh, never mind.) Tim looked at my burgeoning pile of copies and said: “The staple is in the wrong corner.”
In this era, copy machines could only affix a staple to the top right corner of a horizontal document. I explained to Tim that I didn’t want to be late getting all the copies ready for the meeting, but if I had time I would go back and re-staple the decks. A few minutes later I saw Tim, tenured McKinsey partner and future Broadridge CEO, with a staple remover and a stapler, re-stapling the decks in the top left hand corner.
I watched this and said to myself, “When you care about something, no detail is too small.” Effort is a signal. The CEO didn’t remember my analysis — he remembered getting the deck five different ways. Tim didn’t care about the staple — he cared that the client would notice we cared.
Footnotes
[1] Jacob Neusner was a famously prolific Brown University professor. (Sadly I never took any of his classes). A colleague (I understand) joked: “I called Jack’s office and his assistant said he couldn’t come to the phone because he was writing his latest book. I replied that I would hold.”
One of Neusner’s former students compared my writing pace to his. I think it was intended as a compliment. I think.
[2] Yes, I know Prosaic Times has many readers outside of enterprise technology organizations -- especially in the tech, venture and PE communities. I will augment the panel with a wider range of reader profiles in the future.
[3] Yes, a little post-modern. No, I am not entirely comfortable with that. Also I can’t stand Frank Gehry.