When to Use AI & Judging Feasibility
Is AI the right tool, a one-day feasibility test and the levels of automation.
AI is not always the right tool. Rules, search or people may be better. Test feasibility in a day with 30 to 50 real cases and a rubric. Start at the lowest level of automation that creates value, match it to risk, and do the cost sums early.
- Feasibility
- Whether the idea can work well enough in practice.
- Level of automation
- How much the AI does without a person, from suggest to full automation.
- Reversible
- An action that can be undone.
Not every problem needs AI, and not every AI idea can be built well today. A good AI product manager asks whether AI is the right tool, whether it is good enough, and what happens when it fails. This lesson gives you a way to judge feasibility in days, not months, and to choose the right level of automation.
Is AI the right tool?
- A fixed rule or a form may do the job better, cheaper and with perfect repeatability.
- Search may be enough if people mainly need to find existing information.
- A person may be best when stakes are high and cases are rare.
- AI shines with messy text, images or speech, with many variations, and with tasks where a good draft saves time and a person can check it.
The feasibility questions
VALUE Is the problem frequent and costly enough to matter? How much? QUALITY How good must the output be? What error rate is acceptable? CHECKING Can a person (or a rule) check the result quickly and cheaply? ERRORS What happens when it is wrong? Is it reversible? Who is harmed? DATA Does the information the AI needs exist, and are we allowed to use it? VARIETY How different are the inputs? Are there many rare cases? COST What would one task cost to run? What is the value of one task? SPEED How fast must the answer come? Is a few seconds fine? RISK Privacy, fairness, safety, legal or reputation concerns? ALTERNATIVE What is the simplest non-AI solution, and why is it not enough?
A one-day feasibility test
- Step 1: collect 30 to 50 real examples of the task, with the right answers from an expert.
- Step 2: try a strong prompt on a capable model. Score the results with a simple rubric.
- Step 3: read every failure and group them. Are they fixable with better instructions or data?
- Step 4: try a second approach, such as adding documents or examples. Compare.
- Step 5: decide whether it is promising, needs more work or is not ready.
Levels of automation
You do not have to jump to full automation. Choose the level that fits the quality you can reach and the cost of errors.
LEVEL 1 SUGGEST The AI offers an idea; the user decides what to do. LEVEL 2 DRAFT The AI writes a draft; the user edits and sends. LEVEL 3 ACT WITH APPROVAL The AI prepares the action; a person approves each one. LEVEL 4 ACT, THEN REVIEW The AI acts alone on safe cases; people review samples. LEVEL 5 FULL AUTOMATION No review. Only for low-risk, well-tested, reversible tasks. Start at the lowest level that creates value. Move up only when evidence supports it.
Match risk to level
- Low risk and reversible: higher levels are fine after testing.
- Money, health, jobs, legal position or safety: stay at draft or approval level, with strong checks.
- Public and irreversible actions, such as sending to many customers: require approval.
Check the economics
Estimate cost per task: model use, search, review and support. Compare it with the value of the task: time saved, errors avoided or revenue gained. A product that costs more to run than it saves will not survive. Do the sums early, with a range.
- ✓AI is not always right. Rules, search or people may do the job better. AI shines with messy input and drafts that people can check.
- ✓Ask about value, quality, checking, errors, data, variety, cost, speed, risk and the simple alternative.
- ✓Test feasibility in a day with 30 to 50 real cases, a rubric and a failure review.
- ✓Choose the lowest automation level that creates value, match it to risk, and do the cost sums early.
Judge an idea's feasibility
- Pick an idea, such as an AI that drafts replies to customer complaints.
- Fill in the ten feasibility questions in one or two lines each.
- Choose the automation level you would launch at, and what evidence would move you up a level.
- Estimate cost per task and value per task. Write whether the sums work.
Q1When might a fixed rule beat AI?show answer
When the logic is clear and must repeat exactly. Rules are cheaper, faster and perfectly consistent.
Q2Why start at a lower level of automation?show answer
It creates value while limiting harm from errors, and gives evidence before you give the AI more freedom.
Q3Why is a good demo not enough for feasibility?show answer
Demos use the best cases. Feasibility depends on how the system does on typical and hard real cases.