Language Matters: The Make-or-Break Role of Readability in the Age of AI

Hello fellow ResOps Nerds,
Coming up for air briefly, from our full immersion with AI.
A little over a year ago, we began training Meta’s Llama 3 on how to translate MS Word surveys into our proprietary Survey Programming Language (SPL). We chose Llama 3 because it was open source and could run locally.
Early results were…mixed. The AI accurately translated ~60% of the survey’s content into SPL.
Neat.
But not of much practical use.
Increasing Accuracy
In early February, mindful of the chatter around AI’s advances, we ported the Llama training efforts into a larger model. The results were astonishing. More than 90% of the content of moderately complex survey instruments was correctly translated into SPL – not just text, but term points. Quotas. Skip patterns. Option conditioning. Question / section looping. Max diffs. And more.
This includes programming notes like this:
[SHOW Q17 if Q16 IS MORE THAN 5 GREATER THAN Q15]
[CREATE A SUMMARY VARIABLE, QUSED, FOR ALL BRANDS >5%]
and complex constructs like this:
15. What was your share of spend between the following retailer types in 2024? How is it tracking for 2026? What do you expect it to be in 2028?
| 2024 | 2026 | 2028 | |
|---|---|---|---|
| AI-directed | N/A | ||
| Online | |||
| Supermarkets | |||
| Drugstores | |||
| Warehouse / Big Box | |||
| Other: ____________ | |||
| Total (must be 100%) |
AI is still imperfect. But once AI has done the heavy lifting, a project manager can quickly review, correct where needed, and fill in gaps. For moderately complex surveys, this cleanup often takes less than an hour – even if the survey content is sprinkled with advanced constructs.
Consider the “share of spend” question (Q15) above. This is written as follows in SPL:
15. What was your share of spend between the following retailer types in 2024? How is it tracking for 2026? What do you expect it to be in 2028?
class: add_to_100_percent
cell: R[1]C[1] {text: N/A}
columns:
1. 2024
2. 2026
3. 2028
rows:
1. AI-directed
2. Online
3. Supermarkets
4. Drugstores
5. Warehouse / Big Box
6. Other: {auto other: y}
Above, the “cell” function places an “N/A” for the cell at Row 1, Column 1 in the grid.
A Little Background on SPL
SPL itself was birthed and shaped from decades-long dialogues – What words best encapsulate functions? What are the natural extensions? How can they be framed intuitively?
The majority of people who use our platform day-to-day are researchers, not engineers. They don’t speak programming. Operationally, that means the system has to be readable to regular humans. Intuitiveness is critical, which is why SPL reads like plain English.
Why does this matter?
At their heart, today’s “AI” systems are large language models (LLMs). They’re remarkably good at processing language. Our SPL is, at its heart, a structured version of plain English for surveys.
A colleague recently inquired about how we are “training” our AI. Although this training involved many steps and lots of experimentation, at its core, it was made vastly simpler by the fact that our SPL reads like English. That meant that the LLM was already primed to understand it. Much of our work then became just as if we were onboarding a new team member to our merry band.
Getting it into the system is just a first step in the battle.
Consider how Q15 is coded in Forsta XML by a seasoned Forsta programmer.
<number label="Q15" size="3" optional="0" range="0,100" uses="autosum.6" grouping="cols" amount="100" autosum:postText="%">
<title>15. What was your share of spend between the following retailer types in 2024? How is it tracking for 2026? What do you expect it to be in 2028?</title>
<comment>Please ensure each column sums to 100%.</comment>
<col label="c1">2024</col>
<col label="c2">2026</col>
<col label="c3">2028</col>
<row label="r1">AI-directed</row>
<row label="r2">Online</row>
<row label="r3">Supermarkets</row>
<row label="r4">Drugstores</row>
<row label="r5">Warehouse / Big Box</row>
<row label="r6" open="1" openSize="25">Other:</row>
<style copy="hideElements" cols="c1" rows="r1" name="question.element"/>
</number>
<style name='question.element' label="hideElements">
<![CDATA[
\@if ec.simpleList
<div class="element $(rowStyle) $(levels) $(extraClasses)
${col.group.styles.ss.groupClassNames
if col.group
else
(row.group.styles.ss.groupClassNames if row.group else "")}
$(col.styles.ss.colClassNames) $(row.styles.ss.rowClassNames)
${"clickableCell" if isClickable else ""}"$(extra)>
${v2_insertStyle('el.label.start')}
${v2_insertStyle('el.label.end')}
</div>
\@else
<$(tag) $(headers) class="cell nonempty element $(levels)
${"desktop" if this.grouping.cols else "mobile"} border-collapse $(extraClasses)
${col.group.styles.ss.groupClassNames
if col.group
else
(row.group.styles.ss.groupClassNames if row.group else "")}
$(col.styles.ss.colClassNames) $(row.styles.ss.rowClassNames)
${"clickableCell" if isClickable else ""}"$(extra)>
${v2_insertStyle('el.label.start')}
${v2_insertStyle('el.label.end')}
</$(tag)>
\@endif
]]></style>
While the IntelliSurvey language is more compact and elegant (please forgive the bias), a skilled Decipher user can presumably deploy Q15 in roughly the same amount of time. Though there’s more instruction in Decipher, it’s not crafted from scratch – it usually incorporates and adjusts a previous solution. In this case, the speed of an experienced Decipher user is unlikely to be impacted.
But now consider what happens when an AI programs a survey. AI-generated code can be imperfect. Granular human review is essential. And that’s where language matters.
- SPL is compact and closely aligned with plain English
- XML is more complex and verbose, and hard to read for non-engineers
With SPL, a researcher can scan and validate the content rapidly. With XML, even experienced programmers may need to dig deeper — and a single misplaced character deep in the code can require hours of troubleshooting.
For some, this may seem a distinction without a difference. But, if you’ve ever had to debug why Q73B isn’t getting through tests on Thursday night, and your significant other is looking askance at the prospect of being late for your dinner reservation, you’ll appreciate the value of clarity and simplicity.
Where things stand now
In 2025, deploying more rigorous surveys typically took days, with hours of team input.
In 2026 and beyond, delivery times for moderately complex surveys will be measured in hours, and team inputs in minutes. Survey programmers, and ResOps colleagues, will increasingly shift their focus to consultative activities – helping clients manage iterations, identifying and remedying survey trouble spots, and designing and integrating data cleansing frameworks.
The question remains: Can AI truly accelerate the deployment of complex and often idiosyncratic surveys, considering the unique challenges substantive groups present in their work streams? Or are these claims too fanciful?
Certainty is impossible in these rapidly changing times, but we’re optimistic. If you give us a few hours, we can advise whether our magic would work for you by running one of your surveys through our machinery.
We welcome a challenge!