The cost of a vibe‑coded automation

A pattern showed up in three different engagements last quarter. In each case, a marketing team had used an LLM to generate a multi‑step automation - branching workflow, lead scoring overlay, list‑hygiene routine - and it had been running in production for between four and seven months before anyone noticed the consequences. In each case, the cleanup took longer than the original build would have taken. In one case it took longer than the original build would have taken if I had done it twice.

I want to give that pattern a name because I think we are going to be cleaning it up for the next several years. I am calling it vibe‑coded automation. The name is borrowed; the shape is familiar.

What vibe‑coded means in our world

In software, "vibe coding" is the term that has stuck for using an LLM to produce code you do not fully understand and shipping it anyway. The output looks plausible. It runs. The author cannot quite explain what it does, and is not going to be the one debugging it at three in the morning.

In marketing operations, vibe‑coded automation looks like this. A marketer asks a model to produce a workflow that does something they could roughly describe in a sentence. The model produces a flow with eleven steps, four conditional branches, and a custom field nobody else knew was there. The marketer ships it because it looks reasonable, the test send went out, and they are eight other things behind on their week. Six months later, the lead routing is wrong, the customer segment is mis‑tagged, and the source of the bug is a branch nobody read carefully when it was first deployed.

This is not new - marketers have been deploying things they did not fully understand since the first drag‑and‑drop builder shipped. What is new is the throughput. Pre‑LLM, a marketer who could not explain a flow usually could not build it either. The platform was the gating function. With AI on top of the platform, the gating function is gone, and the only check left is the marketer's own discipline.

What it actually costs

The visible cost is the broken thing - wrong messages, wrong scores, wrong segments. Nobody likes finding those, but they are recoverable. The cost that hurts is everything around the broken thing.

It costs operational confidence. Once a team has discovered one vibe‑coded automation in their stack, they assume there are others. The next quarterly audit takes twice as long because everything is suspect. We have inherited instances where the team was so unsure of their own scoring rules that they ran the next campaign in a brand‑new list rather than trust the segmentation that already existed.

It costs analytical credibility. If the workflow is silently mis‑tagging contacts, every report downstream of that tag is wrong. The wrongness is invisible until somebody compares the dashboard to the CRM and finds a population delta of fifteen percent. Now leadership does not trust the dashboard, and the marketing team gets to spend two weeks rebuilding a number that should not have been broken.

It costs deliverability. Generated workflows tend to be generous with sends. They re‑enter contacts into nurtures, fire dormant‑reactivation sequences against truly dead lists, and trigger transactional templates as marketing. The inbox providers notice this faster than humans do. We have seen sender reputations move two full grades in a week from a vibe‑coded re‑engagement program. (For the underlying mechanics of why this happens, the DMARC article and sub‑domain isolation post are the technical baseline.)

And it costs the marketer. The person who shipped the original flow rarely owns it on the cleanup side. By then they have moved on, switched roles, or left. Somebody else has to reverse‑engineer a workflow they did not write, in a tool they may not specialize in, with no accompanying notes.

The honest test

Before any automation goes to production, generated or otherwise, somebody on the team should be able to answer four questions in plain language without opening the platform.

What is this workflow doing, and to which contacts? If you cannot describe the audience without naming a segment ID, you do not understand the audience.

What does this workflow stop doing, and when? Most damage from automation is not what it sends; it is what it forgets to stop sending.

What other systems read or write the fields this workflow touches? CRM sync, BI pipelines, sales hand‑off - every one of those is a downstream consumer that may break silently when a field changes shape.

What happens if this workflow is wrong? If the answer is "we send a wrong email to a thousand contacts," the cost is small and recovery is fast. If the answer is "the entire customer health dashboard goes sideways," the workflow needs more than one set of eyes before it ships.

We run those four questions on every automation we build. Generated or not. The test takes ten minutes. It is the cheapest insurance in marketing operations.

What we recommend

Use AI to draft the workflow. Use the marketer to ship it. Do not let the model close the loop end‑to‑end without a human review that is honestly capable of catching the failure modes. If your team does not have someone who could catch them, the workflow is not ready to be in production yet, and no amount of LLM speed is going to change that.

If you are wrestling with an instance that has a few of these in it already, that is the kind of cleanup we do for a living. The starting point is usually a list‑hygiene and deliverability audit, because that is where the damage tends to surface first. From there it is unwinding the workflows. It is not glamorous work, but most of marketing operations is not.

If you want to talk through it, we are reachable.