Everyone who makes things has a folder on their computer with all their creative ambitions and ideas waiting to come to life. I’m extremely guilty of this — unfinished articles, unedited photographs, half-edited videos, and my most prolific project graveyard… songs.

To add to the mess, when I was working as a teacher I began to develop plenty of ideas for learning courses. I could sit with a problem and see the learning path through it — what to cut, what to put first, where a learner would trip, the one example that would make the whole thing click. This problem-solving process is a creative act in exactly the same way that writing a song is a creative act. When it’s complete, the whole point of the thing is that you share it with someone. I can play the song live, and I can teach the course in a classroom — but wouldn’t it be nice if I could record my music and distribute it globally? Wouldn’t it be nice to provide access to my learning courses globally?

The problem is, when I went looking for the process that would let me do that, I found it behind a door with a price on it. A four-figure-a-year price. For one person, teaching themselves a new craft in the evenings, that isn’t a purchase — that’s a bloody mortgage payment for software you’re not even sure you can use yet.

So the folder continued to collect course ideas that could sit alongside my unfinished songs. The barrier to entry was successful.

The song you can’t release

Let me be precise about the barrier, because it’s easy to point at the wrong thing.

The barrier was never the ideas. Teachers have the ideas. Trainers, coaches, the brilliant subject expert two desks over who explains things better than any course you’ve ever taken — they all have the ideas. The barrier was never the craft, either; that you can learn, and half of it people already have from teaching a room full of tired teenagers on a wet Tuesday.

The barrier was the studio — and, more precisely, the release. The bit that takes what you can already do — the writing, the sequencing, the clever little interaction that makes someone think rather than nod — and turns it into something you can actually get to more than a classroom of people.

For nearly everything else in the mystery ‘ideas’ folder on the computer, the world sorted the release problem out. You can put music on Spotify or YouTube for free. You can post your photographs and your sketches to social media. You can even teach for nothing — upload a video, or just shout from your balcony! One craft after another had its gate quietly lifted: creation was democratised, and so was distribution.

Then there’s eLearning. Try to get a proper course — tracked, assessed, the full experience — out to other people for free, and you hit a wall the other crafts tore down years ago. Your options were two. Use a platform that hosts it for you, but ties you inside its walls and takes a cut of anything you earn. Or buy your own authoring tool and keep control — for which the going rate, per seat, sits in the four-figures-a-year range. These are genuinely good tools — Articulate’s Storyline and Rise, Adobe Captivate, Lectora — and that price is aimed, quite reasonably, at professionals with an employer’s budget. But for a classroom teacher who’s merely curious about scaled digital learning, four figures a year isn’t something you try on a whim. It’s something you talk yourself out of.

Which means the person it prices out is the exact person we should most want in: the teacher wondering whether instructional design might be their second act, the freelancer with no company card, the domain expert who’d make superb learning if the cost of trying weren’t a month’s rent.

There was a free option, and I want to be fair to it because it’s a lovely bit of kit. H5P is open source, gives you dozens of interactive content types, and drops neatly into a lot of learning management systems. If the paid suites are the pro studio, H5P is the free four-track recorder your mate doesn’t need anymore — brilliant for laying down an idea. But it was built to make interactive content, not to close the loop. Getting solid, consistent data back out — the “did it work, and for whom?” — is where it strains: its richer tracking leans on you wiring up a separate learning record store, not every content type reports, and scores don’t flow to your platform unless you go and switch it on. To continue the music analogy, H5P lets you lay down the track. Mastering it and getting it onto the shelf, so other people can reliably play it, is another matter.

So that’s the state of the industry. Releasing a course — by which I mean consistently, measurably, to anyone — cost a fortune. That’s not a talent problem. That’s an access problem. And access problems have a habit of quietly deciding who gets to become what, and I don’t like that much.

Who gets to make the record?

Here’s the question underneath all of this, and it’s bigger than eLearning: who gets to build the thing?

The cost of the equipment has historically always been a barrier to entry… until it isn’t. Photography was gatekept by the darkroom, then by the price of a decent camera — until the phone in your pocket became good enough that a kid could shoot something beautiful for nothing. Music was gatekept by the studio, until a cheap laptop and a copy of GarageBand meant a teenager in a bedroom could make a record that charted.

The barriers always come down the same way: the tools get cheap, and the crowd rushes in. It’s not because the crowd suddenly becomes talented (the talent was always there hiding in billions of computer folders around the world!), but because the price of trying falls to something a normal person can afford. The result isn’t only more stuff (though it certainly is that too), it’s people discovering they can do a thing they’d never have found out they were good at, because the cost of finding out used to be prohibitive.

Learning design has been waiting for its GarageBand moment. And although that’s an exaggerated claim, it’s what I’m trying to achieve here with my open-source learning design platform.

The studio moved into the laptop

I am not a software engineer. I want to be completely clear about that, because it’s the whole point. I’m a learning designer who can write a bit of code in the same way I can cook a bit — enough to feed myself and impress my [more polite] friends, but not enough to run a restaurant.

And yet I built Folio: a proper eLearning authoring tool. It runs on your own machine, builds a course out of blocks, and exports a standards-compliant package that any mainstream learning management system can play and, crucially, report on — completion, scores, time spent on each question, the lot. I didn’t do that by finally learning to be an engineer. I did it by describing what I wanted, in plain English, to an AI, and then doing the part that was actually mine: the judgement, the design.

A couple of years ago, Andrej Karpathy — one of the more sober voices in AI — put it in a line that’s become iconic: the hottest new programming language is English. More recently he gave the working style a name, “vibe coding”, and the term spread far enough that Collins made it their word of the year for 2025. Strip away the hype and there’s a real shift underneath: for a growing class of software, you no longer need the formal language to instruct the machine. You need to know what you want, and you need to be able to tell whether what came back is any good.

That second half is the bit that matters. Building Folio with an AI was not magic, and it was absolutely not effortless. The pattern looked like this: I set the direction and brought the domain knowledge — what a course is, what data a learning management system needs, what makes an assessment fair. The AI did the building, the boilerplate, the first drafts. And then everything — everything — got checked, because the single most dangerous sentence in this entire way of working is “it ran without errors.” “It ran without errors” is not the same as “it worked.” A thing can run perfectly and do the wrong job with total confidence. Catching that is judgement, and judgement was still mine.

So the AI didn’t replace the craft. It removed the drag — the setup, the plumbing, the fiddly export formats, the first ugly draft of everything — so that far more of my time went to the decisions that actually needed a learning designer. That’s the democratisation that interests me. Not “the machine does it for you.” More like: the machine does the studio engineering, so you can focus on getting the music right.

Why not just use the preset drum machine?

A fair challenge: authoring tools already exist, so why build another? Why not just pick up Rise and go?

Because I kept hitting the edges, and the edges taught me something.

Rise is excellent at what it’s for. It’s the slick preset drum machine of eLearning: pick a groove, drop in your content, and you get a clean, responsive course in an afternoon. For a great many jobs that’s exactly right, and I still use it daily. But the presets are the positive and the negative. You get to change the theme colour and the header image; you don’t get to change the whole kit. When I wanted a specific interaction the templates didn’t offer, or full control of how the course carried a brand, or — the one that really stung — proper data from more than one quiz in a course, I was politely shown the wall.

I didn’t want a bigger drum machine. I wanted a studio — somewhere the constraints came from good design and the medium, not from what the template would allow.

What Folio actually is (and what it deliberately isn’t)

So here’s what I made, and the two ways you can use it, because the second one is what excites me the most.

The first way: use it as it comes. Folio is a ready-to-go authoring tool. You build a course as lessons of blocks — text, images, video, knowledge checks, a few richer interactions — preview it exactly as a learner will see it, and export a package your learning management system can run and track. It’s free, it’s open source under an MIT licence, and it runs locally, so nothing you make leaves your machine. Open the page and start.

The second way: use it as a starting point and customise like crazy. Because Folio reads a plain, simple text format — Markdown — a whole course is just a text file. That sounds like a technical footnote, but it is quietly the most liberating thing about it. Here’s the trick that makes it genuinely effortless: Folio ships with a short brief that explains its format. You paste that brief into any AI, tell it what you want your course to cover, and it hands you back a correctly-formatted text file you import in one step — then refine with a designer’s eye. The AI does the formatting donkey-work; you never have to learn the syntax yourself. It means the writing, the reviewing, and the building stop being three separate jobs in three separate apps (no block-by-block copy-paste, which was killing my productivity in Rise). And because the whole thing is open, if you can code, or you fancy learning with an AI beside you, you can take the framework and extend it — add the interaction you wish existed, bend it to your world. It’s a solution and a platform to build on. Ready to play, or ready to rebuild.

Remember, even though AI is involved, Folio absolutely does not do your thinking. It closes the packaging gap — the record, the release, the “any venue can play this and tell me how it went”. What it very deliberately hands straight back to you is the part that was always the real work: the song.

Because here’s the thing about a studio. Once recording is free, the difference between a track that moves someone and a track that doesn’t has nothing to do with the studio. It’s the writing. It’s the arrangement. It’s taste. In our world, that’s the clever, plain-spoken explanation that finally makes a hard idea land; the assessment that tests whether someone can actually do the thing rather than recite it; the video that’s genuinely worth watching; the scenario that lets a learner feel a consequence safely. That’s the seasoned instructional design, and no tool — free or four-figure — supplies it.

Let me make that concrete, because I promised a studio and I should show you inside it. Say you’re a teacher trainer, and you want to build a short course on giving good feedback — something you know cold from a decade of doing it. An afternoon with Folio might go like this. You paste in Folio’s format brief to your AI of choice and spend twenty minutes telling an AI what you want — the objective, the three things people get wrong, the tone — and it hands you back a first-draft course as a properly-formatted text file. You import it to Folio in one click. And now the real work starts, which is the good work: you rewrite the flabby AI phrasing into something with a pulse, you bin the recall question and replace it with a short scenario where the learner has to choose how to open a hard conversation, you drop in a two-minute video of you actually doing it, you add some Actionable Experience Survey questions (my other hobby-horse — I’ve written about the method and why I built a tool for it) so the feedback you get back is worth having. Then you export, upload it to a learning management system, and — this is the part that used to cost a fortune — you can see how it went. The loop closes. On a budget of… nothing!

Are you sure that the democratisation of eLearning is a good thing?

There’s a critique to my plan here — if you make the tools free, don’t you just get an ocean of terrible courses? More clicky-clicky-next rubbish, now at scale, now written by a robot?

Well, yes, probably. Every democratised medium went through its flood of rubbish. Cheap cameras gave us a billion bad holiday photos; they also gave us photographers who’d never have picked up a camera otherwise. The flood is real, and it is the price of the gate coming down. But notice what the flood isn’t: it isn’t evidence that access was a mistake. It’s evidence that access worked, and that the next scarce thing isn’t the tool — it’s the judgement to use it well.

That’s the line I’d underline for anyone nervous about where this goes. Democratisation lowers the cost of entry. It does not lower the value of expertise — if anything it raises it, because when everyone can press record, taste becomes the whole game. Access is not the same as ability. But access is how you find out whether you have the ability, and then how you build it. The teacher who imports their first rough course and feels the itch to make it better — that itch is the beginning of a designer. They just needed to get in the room.

So no, I don’t think a free studio makes everyone a great learning designer, any more than GarageBand made everyone a great producer. What it does is stop the equipment being the thing that decides. It moves the bottleneck from your wallet to your work, which is precisely where a bottleneck ought to be.

Where else the door is opening

None of this is really about eLearning, which is why I think it’s worth your time even if you’ll never build a course in your life.

The pattern — a craft gated by the cost of its tools, then thrown open when the tools got cheap — is playing out across nearly everything right now, and AI is the crowbar. It’s in software, obviously, where people who can’t formally code are shipping small, useful, personal tools. It’s creeping into data analysis, design, film, research — anywhere the “packaging” layer used to demand either money or a specialist you had to become or hire. Each time, the same two things happen at once: a flood of the mediocre, and a quiet rush of people discovering an ability they’d have died never knowing they had.

If you lead a team, there’s a version of this for you too. The most valuable thing you can do in the next couple of years probably isn’t buying everyone a licence for the expensive thing. It’s asking which of your people have the biggest folder of ideas on their computer — a process they could turn into a tool, an idea they could turn into a thing — and whether the only reason it’s still in the ‘ideas’ folder is that the studio used to cost too much. Because that reason has expired.

What Folio is for, and where it’s going

Firstly, Folio is for other people. I wanted to democratise eLearning development — to make it something a curious person can pick up and play with, an accessible extra in the toolbox when someone becomes curious about scaled digital learning, without betting a month’s rent to find out whether they even like it. And for designers already in the field, I wanted a platform to build on: something open enough to inspire your own customisations, your own tools, your own idea of what an authoring studio should be.

Secondly — and this reason is purely selfish — I was tired of the limits of the software I was using, and I wanted a studio that did what I told it to, rather than the other way round.

I should point out that Folio is in version one. I’ll be using Folio in my own work over the coming months and improving it as I go, so if you’d like to follow where it goes — and get the proper technical write-up when it lands — that’s your cue to subscribe.

Subscribe now

And I’d genuinely like your help shaping it. If you use it and something’s missing, drop a comment: tell me what you’d build, and let’s see if we can make it happen. Or, better still, take the code, build the thing yourself, and share it back. That’s rather the whole point.

The studio’s open, and it’s free — so let’s make something.

A note on tooling, because the lesson shouldn’t hang on my tool. Folio is one free option among a growing few; H5P is another well-loved open one, and if you’ve an employer’s budget, Articulate’s Rise and Storyline, Adobe Captivate, and Lectora are the polished paid studios and earn their keep. The point isn’t “use mine”. The point is that I’m providing an option to those who don’t want to invest. The tool is the studio; the song is still yours. Folio is free and open-source: try it at andyrogers.design/tools/folio, or take the code from GitHub and make it your own.

Sources and further reading

· Karpathy, A. (2023, X/Twitter). “The hottest new programming language is English.” And (2025) the post widely credited with coining vibe coding. Overview and dating: Wikipedia, “Vibe coding”.

· Vibe coding named Collins English Dictionary Word of the Year, 2025.

· Raval, S. (2026). The Vibe Coding Playbook: Building Your Tech Business with AI. (On building real things with AI-assisted development.)

· Unqork. Code No More: The Power of Visual Programming. (The no-code / “who gets to build” case.)

· McDonald, J. K., & West, R. E. (2021). Design for Learning: Principles, Processes, and Praxis (open access) — for the craft that no tool supplies.

· H5P documentation on tracking and LRS integration, for the “measure” limitations described: help.h5p.com.

After the article went out, a message came in that I could see coming: “OK, but how do I actually run one?”

A few weeks earlier I’d written about Actionable Experience Surveys (AESurveys) – the method I use to replace flat “lovely!” satisfaction scores with something that actually tells you what to do next. Regarding tooling I mentioned explicitly that “the design is the asset, not the software.” Indeed, any survey platform can create AESurveys – Qualtrics, Typeform, Google Forms, a well-built Storyline slide … a form on a napkin.

I stand by the fact that this is a mental model and therefore tooling agnostic – but also, I can appreciate that if you’ve never done it before, it’s not the shallowest of learning curves. Someone wishing to switch to AESurveys has the mental load of designing the survey in addition to deploying the thing, wrangling the responses back out again, and working out how to display the information in a way that serves the unique insights AESurveys are designed to elicit. The method doesn’t stop being the method when the tools get in the way; it just … well, puts an effort barrier there.

So Andy, build a product! Give it a catchy name, set up a business, charge a subscription, market it! Hang on, the whole point of AESurvey design is that it’s portable. AESurveys are a design process that belongs to whoever reads about it, not to me and not to one platform or product. But I do want people to be able to get started easily, so I’ve built the smallest and simplest tool I could get away with – and it’s here for anyone to use, for free, right now.

Subscribe now

Making a recipe card, not a restaurant

AESurveys are a method, like a recipe. If a product gets associated to that recipe suddenly I’ve made a restaurant. It’s the difference between a hamburger and a Big Mac.

That was the framing that decided the development. If the goal is a recipe card, then:

· No backend. No account, no login, no database. Every kitchen has an oven; just as every laptop has a browser. That’s enough.

· No data on my hardware. If I hold nobody’s responses, I can’t lose them, leak them, or misuse them. GDPR becomes a paragraph in a README rather than a lifetime of compliance. I’ll stay completely out of any data generated.

· No moat, no vendor lock-in, no pricing page. MIT licence, source on GitHub, fork away – it’s genuinely free.

· Small enough to hold in your head. The whole thing is a handful of self-contained HTML files. You can read it in an afternoon. Anyone who can write a form can extend it however they desire.

What’s in the recipe?

Three pieces, each of which works on its own and hands off cleanly to the next.

A survey builder. It’s a form in the browser. Type your prompts, drop in four (or more) statements per question, pick a band for each – Alarming, Concerning, Acceptable, Superior – and tick whether you want a comment box. When you’re done with your survey design, hit Copy shareable link. The link contains your whole survey, encoded into the URL. No upload, no save-to-cloud, no “here are the terms of service and here is our privacy policy.” Just a URL.

A participant page. Your respondents click the link. They see the survey as any respondent would, pick their statements, and – if they want – leave a comment. At the end they can save the whole thing as a PDF, email it back to you, or copy a response link they can paste to you separately. The link opens the analyser with their single response already loaded, so a busy person can send you a link and be done in ten seconds.

A report analyser. Drop in the response files, paste the response links, or open one directly. You get full visualisations including Below/Above the Bar per question, hover-tooltips showing the statement each band represents, and every free-text comment grouped by band and paired to the exact statement the respondent picked. Print it out and you’ve got a PDF-ready report.

There’s a fourth mode for anyone who wants to run this through their Learning Management System (LMS): the survey design can be uploaded via SCORM zip export to Totara, Moodle, SCORM Cloud, whatever LMS you’re running. It records the response category, the exact statement chosen, and any comment as SCORM interactions, so you can export the standard “Track details” CSV and drop it into the report analyser above.

Nothing here is clever, technically. The whole tool is a handful of HTML files a browser understands. But the constraints – no backend, no data on my hardware, no vendor lock-in – are quietly doing a lot of work. By taking those choices seriously, the design of the tool almost wrote itself.

Try it (with a worked example)

Open the builder at andyrogers.design/tools, build from scratch or click Load demo to populate a four-question AESurvey about a feedback workshop, wired up and ready. Edit to your heart’s desire. Ten minutes and you’ve got a survey.

Now click Copy shareable link. That URL – nothing else needed – goes to your colleagues. Two days later, you have your replies in your inbox. You open the analyser, paste their response links into the panel, and there it is: the Below/Above the Bar picture, the split per question, and the comments grouped so you can see which frustrated respondents said what.

Total infrastructure involved: none. Total accounts anyone had to create: none. Total data sitting on my servers: none (there are no servers).

What I very deliberately left out

· No account system. Because there’s no data to protect, there’s nothing to authenticate against.

· No aggregation on a shared server. For small cohorts (a team, a class, a research pilot), an inbox is a fine aggregation layer. For hundreds or thousands of respondents, you probably want something with a real backend – and that “something” isn’t this tool (unless you integrate your LMS and have AESurveys in SCORM). Otherwise, reach for a proper survey platform, run the AESurvey method inside it, then come back to the analyser to view the export.

· No public URL directory of surveys. No listing, no discovery, no “trending”. A shareable link is enough.

· No branding options. The look is fixed. I’d rather you spend the ten minutes you’d waste on colour pickers to write better questions.

Feature choices should say something. These say: the recipe is the thing. Anything the tool adds beyond carrying the recipe is a distraction from it.

Where else this pattern applies

The shape of what I built here – a small, portable, no-backend enabler around a method rather than a product – isn’t specific to surveys. It’s a pattern worth stealing wherever you’ve got a design you want to give away.

Wherever a method depends on a specific piece of infrastructure – a login, a subscription, a shared database – reach is quietly gated. Wherever a method fits in a self-contained tool that runs in a browser, or a spreadsheet you can email around, or a printable one-pager, reach is quiet, wide, and permanent. You’d be surprised what fits:

· Design frameworks. A prompt library, a critique checklist, a card-sort exercise. Web page, no login.

· Facilitation aids. A retrospective template, a decision-log form, a role-clarifier. Static HTML.

· Research protocols. Consent forms, interview guides, coding schemas. Version-controlled Markdown someone else can fork.

Each of these is a “recipe” – an idea with a shape – that someone else could use if it were sitting on a card. It doesn’t need a product around it. It just needs the card.

The takeaway

If you’ve designed something useful and you want it to spread, the temptation is to wrap infrastructure around it – apps, subscriptions, dashboards, growth loops. That machinery does spread things; it also gets between the reader and the idea, and it makes the idea contingent on your company staying alive.

There’s another route. Make the idea portable enough that it doesn’t need you. Give it away in a form small enough to travel, cheap enough that no one has to justify the spend, and open enough that anyone can pick it up.

In the end the design is the asset. The tool is the recipe card. Sometimes a recipe is enough.

Try it yourself

· Build your own in the browser and copy a shareable link.

· Explore the report with sample data – twelve respondents, four questions, comments grouped by band.

· Source on GitHub – MIT-licensed.

The method it enables – Actionable Experience Surveys – is written up here: A call for the resignation of the customer satisfaction survey.

Subscribe now

Subscribe now

I was preparing dinner for some friends last month. My partner’s best friend was coming over, and so was a last-minute addition: the new boyfriend, someone neither of us had met. So, what to cook?

Does he eat meat? “I don’t know.” Vegetarian? Any allergies? “Pfff, no idea.” Does he like a bit of spice? “Pass …” What my partner can tell me, in surprising detail, is that he’s deep into science fiction (the Alien franchise especially), yet he’s confusingly also a devout flat-earther (this should be an interesting evening …), and he fanatically follows a football team whose name “I think begins with an E, or something” (the detail dropped off on that last point). Not exactly useful information in this context.

Wait … I’ve got it – pizzas! Everyone likes pizza (especially flat-earthers?) and if I make a few little ones I can improvise and adapt to tastes, swapping ingredients in and out. Foolproof! Andy, you’re a genius!

Pizza 1: margherita. Classic, not adventurous, a safe baseline. The new guy’s response … “Lovely!”

Pizza 2: prosciutto e funghi. Still classic, slightly more risk. The review … “Lovely!”

Pizza 3: anchovies. Divisive, dangerous, but my personal favourite, so I’ll happily eat it all if the group isn’t keen. I get one slice, because … “Lovely!”

And on it went. The pizza plan paid off, and everything was indeed “lovely”. Fast forward to our next gathering and I’m staring into the fridge again wondering what to cook. I know he likes pizza, but I can’t repeat myself – that’d be boring. Italian, then? That’s an assumption, not a certainty. And which pizza did he actually prefer? No idea. He just told me they were all lovely.

This is exactly the level of information that Likert-based customer satisfaction scores provide (you know, those strongly-agree-to-strongly-disagree ladders). They hand you a marker of ‘loveliness’ with no depth beneath it, so the next time you sit down to cook, you have no concrete information to work with.

Customer satisfaction scores: the good, the bad, the ugly, and the misleading

Whenever you’ve completed some form of professional education, you’ve very likely been handed a survey about your learning experience. The course was useful. I understood the material. The length was about right. You tap “agree”, maybe “strongly agree” and you close the tab.

For a year, a learning program I joined was defaulting to exactly this type of survey. At the end of the year we mapped strongly disagree to 1 and strongly agree to 5 and took the averages. We hit 4.5 out of 5 for learner satisfaction across thousands of responses, comfortably over the 4.2 target. Leadership loved it, we loved it, it seemed like such a triumph … at surface level.

On a deeper level, the surveys were bloody useless. The average for every question, on every course, sat somewhere between 4.3 and 4.7. When nearly everyone agrees with nearly everything, you haven’t measured the experience; you’ve measured people’s reluctance to be unkind on their way out. I knew the learners were content (and, victoriously, much more content than last year) but I had no specific insight into why, what exactly had I done to drive this change, or, crucially, what can I change to make things even better and more effective in the next design. In short, I had no ability to meaningfully distinguish between the customer’s experience of all the learning products I’d made throughout the year because they were all … lovely.

Learning designers lean on satisfaction scores because we’re desperate to quantify our impact, and they’re the simplest, most accepted way to do it. But when you deliver something holistic and human, and then try to squeeze it into a number to prove it worked, you often show nothing at all – and instead lead yourself into the dangerous trap of believing your own positive publicity.

The good news is that you can have both the numbers and the information you need to drive improvements. You just need to structure the right questions.

Poor questions give poor answers

Two things were sabotaging my survey design.

The questions were leading. For example, “How much do you agree with the following statement: the e-learning was useful and improved my understanding.” That’s the survey equivalent of leaning across the table and asking, “Hey! New guy! The pizza was delicious, right?!” Nobody’s going to say no. It isn’t a question, it’s a nudge in a lanyard, and agreeing is the path of least resistance.

The scale was mush. Like most satisfaction surveys, we used a Likert scale – the strongly-agree-to-strongly-disagree ladder that asks you to mash a real, textured opinion into one of five options. Then I’d turn the agreement statement into a number, average all the numbers, and report one figure. Meaning leaks out at every step, until “4.5/5” has almost none of the experience left inside it.

I got so frustrated with how little my own surveys were telling me that I set out to build a solution from scratch – something that would give me the information I needed, while avoiding stakeholders asking “Where did the CSATs go? We liked those. We need numbers.”

Thankfully I’m not the only one out there who is dissatisfied with customer satisfaction surveys. Will Thalheimer is an expert in how we evaluate learning, and he makes a convincing case against Likert-based satisfaction surveys. The research he pulls together is genuinely damning: conventional satisfaction scores barely correlate with how much people actually learn or how they later perform (Alliger et al., 1997; Sitzmann et al., 2008). Agreement scales, he argues, are ambiguous, tiring, and tend to produce the same weary tap straight down the page – agree, agree, strongly agree, agree … done.

It was time to stop chasing a better score and start designing for insightful answers. Less “was the meal lovely?”, more “how was the amount of salt for you?”.

Good questions give good answers

The redesign – which I’ve coined Actionable Experience Surveys – rebuilt survey design from the ground up based on three key principles.

1. Bin the leading statement. Ask something neutral and open. Not “the practice questions were useful (agree?)” but “which of these statements best describes how the practice questions worked for your learning experience?” No thumb on the scale.

2. Swap the agreement ladder for real sentences. Instead of five shades of “agree”, the learner picks one statement that concretely says something – from “the practice questions were too complicated” to “the questions were useful, but I need more of them”. Every option is there to direct you to an action, e.g., reduce complexity, or add more questions.

3. Tag each sentence to a standard. Behind the scenes, every statement that the customer can select is marked as Alarming, Concerning, Acceptable, or Superior. This is the move that lets you keep the numbers and the meaning all at once:

   • The quantitative half: I could still report “72% at or above the acceptable/superior bar”, track it over time, and set targets – the numerical benchmarking leadership rightly wants.

   • The qualitative half: because the options are specific sentences, the spread itself is a diagnosis. You don’t just learn how many were unhappy. You learn exactly what the issues are.

Why it works: two quiet bits of behavioural science

Here’s where the design earns its keep.

It forces metacognition – thinking about your own thinking. A Likert ladder lets you freewheel; in fact, it invites you to skip evaluation altogether. In AESurveys, four or more distinct, plausible sentences make you stop and genuinely weigh up what you got from the thing. That small cognitive friction isn’t a bug; it’s the entire point. It pulls a considered judgement out of people instead of a reflex.

It primes the free text response. Every question ends with an optional comment box – nothing new there, we did that with the satisfaction surveys too. But what comes before the box decides what goes in it. This is priming: earlier input shaping later response. After a vague “agree”, you get a vague comment (if you get one at all). After choosing between four or more sharp, specific sentences, the reader’s mind is already running in the right channel, so the comment comes out focused and on topic. I didn’t get richer free text by begging for it. I got it by making people think properly about the question first, so they’re ready and willing to give their opinion which has formed naturally.

No design is free though. Making people think harder is more work, and more work means some will bail before finishing. I manage that by keeping surveys short and scaled to the course – maximum five minutes of survey for an hour of learning. I haven’t seen any drop off in participation, but you’ll have to judge this for your use case.

Subscribe now

Actionable Experience Surveys: the good, the bad, the ugly, and the informative

Rebuilt this way, my surveys stopped flattering me and started feeding me. Same people, same broad goodwill – but now I could truly understand their experience and act to improve it (hence “Actionable Experience Surveys”, or AESurveys). For example, I wanted to dive deep on how the examples embedded in a learning program were serving my learners. The customer satisfaction survey would have told me they were ‘lovely.’ The AESurvey showed me that a strong majority (64%) believed the examples had directly improved their work processes (my ‘Superior’ AESurvey marker). The remaining 36% didn’t simply contradict the “superior” responses, instead they pointed to specific fixes. For example, 12% were asking for more examples, 10% were asking for greater complexity in the examples. So, alongside the pat on the back from the 64%, I received a prioritised to-do list which I can act on to push the 64% higher.

What you can take away

When you build AESurveys from scratch you can use the answer statements to truly zoom in on what you need feedback about (it’s particularly effective to use learning objectives to structure AESurvey questions for learning programs). And if you already have surveys, you can also rebuild almost any question into the AESurvey format. For example, let’s imagine I’ve gone pro with my pizza, and I’m now running a focus group for my pizza restaurant. My customer satisfaction survey could look like this…

“The pizza crust has the perfect balance of crispiness and chewiness.”

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly agree

… but that would have put me back in the ‘lovely’ void. If I’m not hitting 100% strongly agree, I’m left with an ambiguous problem to deal with – too crispy, or too chewy?

Let’s make it an AESurvey question by avoiding bias — “What did you make of the pizza crust?” — and structuring the responses so I can read crispiness against chewiness. Each response maps to a category (Alarming, Concerning, Acceptable, or Superior), the respondent only sees and selects the statement:

Negative

• Alarming — “I’ve nothing positive to say about the crust” · 5%

• Concerning — “Too chewy” · 5%

• Concerning — “Too crispy” · 20%

Positive

• Acceptable — “Good, but I’d prefer it crispier” · 15%

• Acceptable — “Good overall, but I’d prefer it chewier” · 25%

• Superior — “Nicely balanced between crispy and chewy” · 30%

Note how I’ve aligned the information I need (the statements the respondents choose) against the AESurvey’s four categories — this is so I can give the appropriate audiences the right levels of information they need. I can tell my investors that 70% of customers are happy with the pizza crust (all the positive answers in my ‘Acceptable’ and ‘Superior’ categories). But I can go into the details and tell myself that there’s an issue with crispiness. Even in the ‘Acceptable’ range, it’s biased towards being too crispy. And in the ‘Concerning’ range, it’s heavily biased towards excessive crispiness. What’s the learning? Reduce crispiness to improve the customer experience.

I get a spectrum of detail within one question that I can present in full detail or in highlights according to my stakeholder requirements. By the time I’ve asked about the sauce, the toppings, the wine pairing and so on I will have a rich tapestry of data-driven action points, or I can reduce it to “65% of customers are responding positively to the restaurant and we have a plan in place to reach 75% by Q4.”

A quick recipe for an AESurvey question:

• Delete the leading statement. If your question signals the “right” answer, rewrite it as a neutral one. Bias isn’t welcome.

• Write real sentences, not a scale. Four or more options a real person might actually say. If picking one doesn’t give you the information you need, it’s too vague. When you’re reviewing the answers, you want to discover at least one clear action item.

• Tag each response option to a standard. Alarming / Concerning / Acceptable / Superior. Two negative, two positive. Now you can count and diagnose, delivering the headline metrics to the business folks.

• Add free-text boxes. The questions prime deeper thought, and those specifics are goldmines. (If you’re fielding thousands of responses, use AI to spot trends and summarise.)

No matter your industry, there’s always an ask from a higher power to reduce a rich, human experience to a number (whether that’s consuming education or pizza). It’s easy to get lost in the qualitative-versus-quantitative divide and believe you can only have one or the other. That’s a false choice. Design the questions well enough, and the number stops standing in for the meaning and starts carrying it. It is entirely reasonable (sensible, actually) to ask your learners directly, and specifically, for the insight you need. That’s not an imposition, it’s the whole opportunity. And if you close the loop, showing your customers how their input is improving their own experience, you start to build a genuinely healthy improvement ecosystem.

Beyond learning experience

The Actionable Experience Survey method isn’t tied to learning design evaluation. It’s about the gap between measuring a vague feeling and gathering specific, actionable insight – and that gap is everywhere the humble agreement scale has crept in:

• Customer feedback

• Product and UX research

• Employee engagement and pulse surveys

• Patient-reported experience, event feedback, 360 reviews

A caveat, because no method is a cure-all: AESurveys earn their keep when you need actionable depth. If you just want a demographic fact or a quick temperature check, a plain question still wins. Reach for AESurveys when the answer has to tell you what to do next, not merely how people felt.

And that’s that – next time you’re building a survey, or hunting for a clean read on how your content is being experienced, consider AESurveys. Done well, it’s a method of investigation that turns a polite shrug into a plan – and keeps the richness of meaning in, even while it’s handing you a number. Customer satisfaction survey, it’s time to retire.

Subscribe now

A quick note on tooling: I originally built these as self-contained, portable survey modules so they weren’t welded to one platform. I happened to use Articulate Storyline, but nothing here depends on it – Adobe Captivate, Lectora, or a well-built form in Qualtrics, Typeform, or Google Forms will do the same job. The design is the asset, not the software.

Sources and further reading

• Thalheimer, W. (2016). Performance-Focused Smile Sheets: A Radical Rethinking of a Dangerous Art Form. Somerville, MA: Work-Learning Press. (Later updated and retitled as Performance-Focused Learner Surveys, 2022.)

• Thalheimer, W. (2018). The Learning-Transfer Evaluation Model (LTEM). Work-Learning Press. Available at worklearning.com.

• Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(2), 341–358.

• Sitzmann, T., Brown, K. G., Casper, W. J., Ely, K., & Zimmerman, R. D. (2008). A review and meta-analysis of the nomological network of trainee reactions. Journal of Applied Psychology, 93(2), 280–295.

Note: the meta-analyses above find the correlation between end-of-course reaction scores and actual learning or job performance to be very weak rather than literally zero; the exact figure varies by study.