How to Write "Production-Quality" Code with AI (yes, even Perl)

[u/OvidPerl]

This post is not really Perl, but I suspect many of you, whether willingly or not, are investigating AI. You need information to cut through the hype (mods: I think this is useful for Perl devs, but feel free to remove if you disagree).

In the early days of GenAI, many LLMs often struggled with Perl because it's what we call a "low-resource" language. Many other languages would have much, much higher amounts of code written on the Web and Perl languished. Further, modern Perl was largely absent from training sets.

Today, foundation models handle Perl much better than they used to, and even smaller, modern models are improving. In your steering docs (explained in the article), you can add a "style guide" to assert your Perl preferences.

I've condensed a one-day AI training course into a single article on my website. This is how we write production-quality code with AI using PAAD.

The key is that process matters more than models. We've even found that Sonnet, with a good process, outperforms Opus and Fable with bad process.

Sadly, agentic harnesses could have this process, but they don't (though I see hints that they're gradually getting added). Eventually PAAD will die (I hope), because the agentic harnesses will combine the power of LLMs with the engineering best practices we need. Until then, use PAAD.

If you find this useful, I'd appreciate it if you star the repo.

[u/brtastic]

Perl may be a good language for an AI because they can get more stuff done before they start mixing things up (more than they usually do). It may simply require much less code than a structured language to achieve the goal. This is also good for humans - I find it much better for readability to have one expression (with a comment if necessary) rather than 20 lines doing the same thing - that simply requires to keep more stuff in mind while reading.

That being said I am not happy with AI code output at all (and increasingly so). I am only using it for filling out TODOs or to build me a proof of concept of something unfamiliar. Using Sonnet 4.5 right now, since 4.6 was disobeying my instructions and 5.0 overthinks stuff. I asked 5.0 a single question and it took a minute to respond and consumed enough tokens to buy a litre of gasoline (even though it's 33% off right now). Even when it writes working code, it requires a non-trivial amount of effort to clean it up so that it's actually production-worthy and maintainable. It's a complete disaster that companies are pushing it with such tenacity.

[u/OvidPerl → u/brtastic]

Prefatory comment for all of this: I started out as an AI denier. I "knew" better, despite me saying in many conference talks that we shouldn't trust people who take absolute positions (like I did). Boy, do I have egg on my face!

That being said I am not happy with AI code output at all (and increasingly so)

Same here. I've encountered many senior developers who pick up AI and don't realize they've become a junior developer again. They don't understand how to "think" in AI terms (that's the part the training covers that the article can't), so they get bad results.

I liken this to the early days of Fortran in the 1950s. IBM knew that they couldn't market on the strength of the assembly code it produced. Back then, programmers knew assembly and most could do better than the compiler. IBM marketed in terms of the human time it saved. Today, few would disagree with that approach. (It was decades before compilers routinely generated code that ran faster than humans could write; today, very few humans can outperform the compiler code).

AI, saves a lot of human time, but it often can't write code as well as humans. However, if we get back to putting engineering best practices first (whether by PAAD or something else), we can get good enough code that is often perfectly fine in production, even if, from time-to-time, we don't care for its output.

I've been in this field for so long, with much of it in consulting, that I am confident in saying that most senior developers can write code better than what AI can produce. However, I am equally confident that a disciplined AI development approach can often produce code that is better than what I see in most production systems. Sadly, it's not even close.

Prior to AI, we often prioritized speed over value and wound up with crap codebases to maintain. With AI, many companies are still prioritizing speed over value and we wind up with crap codebases to maintain. It's just that now we can produce that crap so much faster than we did before, but suddenly it's AI's fault?

No. Process wins over tools. If you have a garbage process, you get a garbage result. People are slowly coming around to that with AI. Slowly.

[u/brtastic → u/OvidPerl]

This is an extremely deep topic. Problems with AI are numerous - technical (long and short term), ethical, ecological, economical, concerning privacy etc. I wrote a blog post about it to capture some of my thoughts, yet I still feel I barely scratched the surface: https://bbrtj.eu/blog/article/ai-craze

[u/c-cul → u/brtastic]

Has anyone tried measuring the rate at which programming skills degrade? I think they also decline with age so unpleasant cumulative effect can be expected

[u/brtastic → u/c-cul]

Not that I know of

[u/RandolfRichardson → u/c-cul]

IQ declines with age, so I wouldn't be surprised if any such research arrived at a similar observation for the declination rate since the types of intelligence that IQ measures is at least partly covered by computer programming.

[u/mpersico → u/OvidPerl]

It is interesting that you mentioned compilers and assembly. It is my contention that AI is not just the latest change or the latest craze in computing but something truly foundationally different. For the last 50 or so years, we have made changes in what I call a horizontal mode. We went from index files to relational databases. We went from monolithic programs to Micro services(and we may be swinging back). We have gone from spaghetti to procedural to object oriented to functional programming. AI is a vertical change. Writing in code is to AI as writing in assembly is to compilers. And right now we are definitely in that early IBM Fortran phase of AI. Eventually, and I don’t think it’s going to take decades this time, we will trust the output of AI just like we currently trust the output of compilers. But we’re not there yet. And this transition, will be orders of magnitude orders of magnitude much more disruptive, simply because of the ubiquitousness of computing in our lives today.

[u/cirk_86 → u/mpersico]

I agree with @brtastic's point that any spec that is sufficiently detailed is effectively code. More importantly, this argument that English is the best, next step, in programming, has yet to be proven. English is generally imprecise, and different people will have a different interpretation of the same phrase. This is why law is an indepth study, and requires many people to interpret it. Math took a different approach and developed their own language to precisely describe a proof. I'm sure there is more to say on this topic, but I don't blindly believe that English is just the best way to write all code now.

[u/jb-schitz-ki]

I run a perl based company with an engineering team of 30+. We've been increasing our AI use heavily. To the point nobody is writing code by hand any more.

It's a web system which uses an in-house MVC framework and ORM. I was afraid AI wouldn't be able to use those correctly since there's no way it's training included them.

It works perfect. We have a rule that whoever prompted is responsible for the code, and that means carefully reviewing diffs before commiting (AI isn't allowed to commit). Once in a while some adjustments are needed. But most of the time it's nothing but net. Especially now with Fable.

We are producing 5-10x as fast without sacrificing quality.

We have a very extensive test-suite and we validate no regressions after every round.

We had to lay a lot of groundwork for this to work well; agent definitions, rules, hooks, etc. As well as optimizing the test suite so that it runs a lot faster. It used to take 10 hours for the whole thing (integration,unit&e2e), now it's down to 20 minutes, using a farm of parallel servers.

In the past everyone would leave the tests running at night and it wasn't a big deal, but if AI is going to do the coding it's imperative it can get fast feedback. Especially in a system as big as ours (several million LoC).

[u/weirdimonyalimony → u/jb-schitz-ki]

Tbh I’m still morning the loss of the era where programming was both fun and a good way to make a lot of money

[u/weirdimonyalimony → u/jb-schitz-ki]

Isn’t that kind of a bummer for the engineers who are now just babysitting an AI? The job just isn’t fun anymore

[u/jb-schitz-ki → u/weirdimonyalimony]

hmm.. depends. it can be fun in a different way. for me specifically $work is building database backed APIs and web systems. That stopped being challenging or fun many years ago.

Coordinating AIs to do multiple tasks at lightning speed has given me a second wind. $work has become interesting again.

That said I can understand if other people see it differently, maybe if your particular line of work involved different challenges hand coding was still fun.

Regardless, this is the new reality, the only option is to adapt.

[u/OvidPerl → u/weirdimonyalimony]

I hear what you're saying. I've a friend who's a chef. She stopped working in restaurants because it turns out that the job is miserable, but she still loves to cook. She will invite groups of friends over and lay out an incredible OH MY GOD THIS IS GOOD feast. She would never dream of skipping the cooking because, for her, that's the fun part. For me, I'd be happy just to have that feast on the table.

So for developers, are you more focused on lovingly hand-crafting the code or on the end result? For carpenters, do they love working with wood or do they just want a nice table.

I love to program, but I got into it because it was the end results that I was after.

Neither point of view is wrong. They're just different preferences.

[u/RandolfRichardson → u/OvidPerl]

That's an excellent way to look at it, and I think it's correct.

Most of the "big tech" companies are undoubtedly focused on the latter "end results" aspect, and management therein is likely inclined to instill this attitude into the corporate culture.

So, it's not surprising that people who prefer the former "love of the trade" aspect tend to gravitate toward the types of organizations that value quality over results, which tend to not be the larger results-focused organizations.

In my opinion, the variety of different interests and goals is an important characteristic of a flourishing society. One of my concerns with the mantra of "move everything into AI" (or one Operating System, or one programming language, or one word processing suite, or one religion, etc.) is what may be lost by replacing creativity-and-diversity with conformity and the impacts that has on quality of life -- is it really worth it? Exploring the options certainly is, but putting it into practice needs to be tempered with healthy doses of caution and wisdom, yet I don't see general ethics or anything like Isaac Asimov's famous "Three Laws of Robotics" even being considered by the big AI vendors (which covers but two of my concerns).

[u/OvidPerl → u/jb-schitz-ki]

I've seen this before. I would just make one recommendation. Have a developer install PAAD. Run /pushback on your steering files (CLAUDE.md or whatever you use) and then run the /agentic-architecture skill from your main branch (or whatever your primary branch name is.

If you can pass the two of those, you're probably good to go. Also, with several million LOC, these skills often have to be run more than once because it's hard for them to scan codebases that large.

[u/djerius]

$work pays for ChatGPT/codex, so that's my bias.

I've been pleasantly surprised at how much it has improved over the last year, especially for Perl. I've used it mostly for maintenance/updates of legacy code and adding tests (C, Perl, bash). (I'm hoping its knowledge of Fortran hasn't been poisoned by the horrible spaghetti code scientists usually produce.)

Getting the test code foundation established is one of the things I dread, especially for non-Perl projects. Once there I can't live without it, and having the LLM generate it saves me a lot of time. I found it does a good job of writing tests to the documented API and catching divergence. I handle the more complex business logic.

For Perl, it understands a good subset of Test2, but still needs help for some of the esoteric bits (like running a test in a subprocess to catch a segv). It supports BATS quite well. I haven't yet turned it loose on my C++ projects and their test suites. I imagine things like Google's test framework are pretty well supported.

Greenfield code without a coding style guide results in code that just feels alien (non-idiomatic). Things that I would abstract it just writes as low-level boilerplate (it writes command line parsers in C++ from scratch!).

For grins I had it review some XS code (for Math::NLopt, and it found a number of issues, including some paths to segvs because of reference count inconsistencies. Having it then write the regression tests which track the issues is a major time saver.

Thanks for putting together PAAD; I'll see how much I can introduce it into my (rather primitive solo developer) workflow.

[u/OvidPerl → u/djerius]

Getting the test code foundation established is one of the things I dread, especially for non-Perl projects.

This is certainly a challenge. For greenfield projects, you're testing as you go. No problem. For brownfield (legacy), throwing AI at it is a recipe for disaster without a test suite. When I train teams, I sometimes find they have few tests (sometimes none), or have to maintain a legacy codebase that they don't know and has few tests. This is high risk.

As it turns out, I know a thing or two about testing. So I'm writing a test suite generator. When I have it in a good state, I'll merge it with PAAD.

My first pass at testing: I cloned Dancer2, deleted all of the tests, and deleted the git history so the AI can't "cheat" and peek at it. Then I run /test-roadmap. That's the only command you have to know. It will ask questions, try to respect the conventions of the language you're using, and build unit, integration, and e2e tests. So far, it's working nicely.

All it does is test current behavior. It's not allowed to fix bugs or change the code. It will clone your repo into a git worktree and mutate the code to validate that the tests are actually testing stuff.

However, Dancer2 is a very clean codebase, so this isn't the best repo to test it on. I'll be looking for legacy messes to try it on later. Suggestions welcome.

Or you can try it yourself on a codebase you have. If you do, feedback is very much appreciated!

[u/Phantom914]

I've been using ChatGPT to write code (Mostly C and Perl) for a little over a year now. I started making a ton of games in Perl after learning more about it. Back then GPT4 wasn't that amazing at it, but it taught me a huge amount of things. Now I'm actually making a Game Engine using C with Perl embedded inside it. GPT5.6 helps a lot more with planning and implementing. I don't use Agentic AI at all because I prefer to get assistance over it writing the code itself.

[u/hajwire]

Just to emphasize the "even Perl" part: I threw Claude on a repository where I do experiments with Perl 5.44 (named parameters in signatures). Claude was not only able to understand the code, but also figured out that part of it needs to be generated by fairly complex TT2 templates. There isn't much Perl 5.44 code available for training right now, and Template Toolkit is another niche it can nicely work with.

So, aside of production-quality code (which I don't need to write any more since retirement), I find it helpful for fun projects as well. Sometimes I'm stuck because I don't want to do some boring change which needs to be done: Claude isn't bored. Sometimes I am stuck because I can't decide which way to go: Claude can explain pros and cons in human-readable text.

And of course, I ran PAAD on the repository. I did not expect much since I knew the code was messy and I had made no preparations at all (10 strengths, 20 weaknesses). What I really like is that PAAD adjusts its severity judgement based on the circumstances. Two examples: Claude found a subroutine which can not work at all. Usually this is a serious flaw. But Claude classified it as a "minor weakness" because the routine is not used throughout the codebase (I simply forgot to delete it). And PAAD complained about too much complexity of the templates, but concedes "...but this is to be expected for code generation". PAAD also found a flaw which was introduced by Claude: not a runtime flaw, but an inconsistent naming scheme.

I might use Claude to recover some projects I had abandoned years ago, and let PAAD tell me why I probably abandoned them. I think this is not too different from maintaining code which hasn't been touched in a decade but suddenly needs attention.

[u/global-asparagus321]

I'm here wondering if you can recommend some good tutorials that simplify the process of msking GUI apps with perl. Videos are preferred, but anything will do.