How to know an AI feature will flop before you build it

You can usually call the flop in planning. The signals are loud if you are listening for them instead of for applause. A feature is likely to flop when it cannot name an outcome, when the value depends on the user trusting an output they did not shape, or when it adds a step to a job that was already one step.

That last one is feature creep wearing an AI costume: complexity added beyond the core function, dressed up as progress. The deeper version is older than this AI wave. A feature shipped is not an outcome moved, and teams that are handed features to build rather than problems to solve tend to ship a lot of the former. AI makes this trap more expensive, because the feature costs more to run and looks more impressive while doing nothing.

Score the idea before it enters a sprint. This is lighter than a full ai feature prioritization pass and catches the obvious refusals fast:

# Flop-risk scorecard (answer honestly, sum the risk)
metric_named:        true        # false = +3 flop risk
baseline_measured:   false       # false = +2 flop risk
beats_simpler_tool:  unknown     # unknown/no = +2 flop risk
trust_required:      high        # high = +2 flop risk (users distrust unshaped output)
adds_a_step:         true        # true = +1 flop risk
 
# total >= 4  → refuse, or rescope to the simpler tool
# total 1-3   → de-risk: ship the non-AI version first, measure, then decide
# total 0     → this one earns a build slot

The scorecard is not the point. The honesty is. If you fill it in the way you would defend the feature to your boss, it lies. If you fill it in the way a skeptical finance lead would read it, it tells the truth.

When to use AI in product, and when a plain feature wins

The clean rule for when to use AI in your product: use it as supportive AI sitting above a deterministic core, never as the core itself. Supportive AI suggests, drafts, ranks, and summarizes on top of a system that still works perfectly when the model is wrong or offline. That is how you de-risk the build. The product does not depend on the model being right; it gets a little better when the model is right and loses nothing when it is not.

A plain feature wins whenever the job is deterministic and the user needs to trust the result. Filtering, sorting, validation, calculation, routing with clear rules. Running the cost-and-return test shows exactly when a simpler feature beats an AI feature on ROI, so the comparison is a number rather than a preference. Putting a probabilistic model in front of a job that demands a correct answer is how you manufacture distrust. The model will be confidently wrong some percentage of the time, and that percentage is the whole story for the user.

This is also where the difference from killing a feature matters. If the feature already shipped and is underperforming, that is a separate decision: how to kill an AI feature you already shipped cleanly, after launch, once the data is in. Refusing to build is cheaper than killing, because nothing was spent. The earlier you say no, the smaller the cost of the wrong AI feature is.

Which AI features should I avoid building first

Rank refusals by the cost of being wrong, not by how exciting the idea is. This is a one-line ai cost benefit analysis: the features to avoid building first are the expensive-to-run, low-confidence, trust-dependent ones, because they fail loudest and cost most while failing.

Refusal cost = (run_cost_per_month + maintenance_drag) × months_before_you_admit_it
             − (metric_delta_if_it_worked × confidence_it_will_work)
 
If confidence_it_will_work is a guess, treat it as 0.
Then Refusal cost is pure loss, and the feature ranks first to cut.

The ordering falls out cleanly. Cut the open-ended generative features with high run costs and no named metric before you cut the small supportive nudges that are cheap and reversible. Excitement inverts this ranking, which is exactly why excitement is a bad sorting key.

A refusal applied: scoring one AI idea (Concept Demo)

Take a common request: an AI assistant that writes a customer's project setup for them during onboarding. It demos wonderfully. Run the test.

Metric: activation rate, the share of new accounts reaching first value. Baseline (projected, for illustration): 38% today. The flop-risk scorecard reads metric_named true, baseline_measured true, beats_simpler_tool unknown, trust_required high, adds_a_step false. Trust is the killer here, because users do not trust a setup they did not shape and tend to redo it, which adds time instead of removing it. Projected flop risk lands at 2, in the de-risk band.

So the refusal is not a flat no. It is a sequence: ship the non-AI guided flow with strong defaults first, measure the activation delta, and only then decide whether a generative layer earns a build slot. That is the whole discipline. You did not kill an idea, you refused to build the expensive version of it before the cheap version proved the metric. This is a Concept Demo of the method, not a client result.

Most of the value in deciding which AI features to build is in the refusals, because a roadmap that ships only features tied to a metric and a baseline is a roadmap that compounds instead of bloats. The teams that win the next two years will not be the ones with the longest AI feature list. They will be the ones with the shortest, and the clearest reason for every name on it.