Day 3 - Problems Where ML Fits vs. Problems Where It Doesn't

📌 Key Takeaways

• Traditional programming is data + rules → results, while machine learning is data + results → rules. The input and output are swapped.
• ML is a good fit when the rules are too complex for humans to write down, there is enough data, and a small margin of error is acceptable.
• ML is a poor fit when the rules are simple and clear, there is almost no data, or 100% accuracy and a complete explanation are mandatory.
• Choosing the simplest method that is still sufficient is good design. A more complex technology is not automatically the right answer.
• In practice, clear rules are often written as code and only the complex judgments are handed to ML, so the two are used together.

Traditional Programming vs. Machine Learning

Machine learning is powerful, but it is not a cure-all. In practice, cases are common where ML was adopted without any real need, so costs went up and the results were worse.

• Traditional programming: a person writes the rules, and the computer executes them exactly as written.
• Machine learning: you give the computer data and example answers, and it produces the rules (a model).

Traditional programming is data + rules → results, and machine learning is data + results → rules. It is easy to remember if you think of it as the input and output being swapped.

Problems Where Machine Learning Fits

The more of the following conditions are met, the better a choice machine learning is.

1. The rules are too complex, or too hard for a person to write out one by one — writing code rules to recognize a cat in a photo is practically impossible.
2. Patterns exist but are hard to express explicitly — the characteristics of spam email keep changing and are subtle, so they are hard to pin down as fixed rules.
3. There is enough data of sufficient quality — you need plenty of historical data (examples) to train on.
4. A small margin of error is acceptable — machine learning is not 100% accurate; it gives probabilistically good answers.
5. The rules must change when circumstances change — the model can be retrained on new data to adapt to change.

Examples of a Good Fit

• Image, speech, and natural language recognition
• Recommendation systems (products, movies)
• Spam and fraud detection
• Demand and price forecasting
• Customer churn prediction

Problems Where Machine Learning Doesn't Fit

Conversely, in the following cases machine learning is overkill or inappropriate.

1. The rules are clear and simple — "VAT is 10% of the amount" is just a multiplication.
2. There is almost no data — with too few examples to learn from, it will not work properly.
3. 100% accuracy and a perfect explanation are mandatory — when legal or safety requirements allow no error and the basis for every decision must be explained, machine learning, which is probabilistic and sometimes a "black box," can be risky.
4. Simple calculation or lookup is enough — database lookups and fixed formula calculations are more accurate and cheaper with the traditional approach.

Examples of a Poor Fit

💡 Related theory: Avoid the misconception that "machine learning is always smarter." Applying machine learning to a problem with clear rules can raise costs and even lower accuracy. Choosing the simplest method that is still sufficient is good design.

Decision Flowchart

This is a quick sequence for checking whether to adopt machine learning.

Can a person write the rules easily and clearly?
   └ Yes → Use traditional programming (ML is unnecessary)
   └ No ↓
Is there enough data to train on?
   └ No → Get data first (ML is difficult for now)
   └ Yes ↓
Can a small margin of error be tolerated?
   └ No → Review carefully (ML may be risky)
   └ Yes → ML is likely a good fit

Two "Poor Fit" Signals the Exam Guide Points To

The AIF-C01 exam guide gives cost-benefit analysis and cases that need a fixed outcome rather than a prediction as examples of situations where AI/ML is not a good match.

1) You need a "fixed outcome," not a prediction

• A machine learning answer is a prediction. It means "probably this," and even for the same kind of input it is occasionally wrong.
• Traditional code, on the other hand, is deterministic. The same input always gives the same result, and you can point to a single line of code to show why.

• A rule of thumb: if you can say the answer is "calculated or looked up," use the traditional approach; if you have to say it is "estimated or singled out," it is a candidate for machine learning.

2) The cost-benefit does not add up

Machine learning does not cost only the price of building a model. You have to add up everything below and weigh it against the benefit you get.

• If you build a model to auto-classify 20 inquiries a month, it is far cheaper for a person to just read them. When the volume is small, the benefit of automation is also small.
• Even for a problem where a few lines of rules already solve 95%, adding a model to raise that to 97% means checking whether those 2 percentage points are worth more than the operating cost.

Conversely, Three Ways AI/ML Adds Value

The exam guide gives assisting human decision-making, scalability of solutions, and automation as examples of where it adds value.

Judging by Scenario

Striking a Realistic Balance

• In practice, the two approaches are often used together. Clear rules go into traditional code, and only the complex judgments are left to machine learning.
• The important perspective is not "only one of the two is right" but "pick the tool that fits the problem."

📚 Case: In a card payment approval system, "decline if over the limit" and "decline a card reported lost" are policies that must be enforced without exception, so they are put in as code. "Is this an unusual, suspicious payment?" is hard to write as rules, so machine learning scores it. Cases where the score is ambiguous are checked by a person. Three layers sit one on top of the other in a single system.

Traditional ML Models vs. Foundation Models

Even after deciding to use machine learning, there is one more choice left. The exam guide asks you to tell which of a traditional ML model and a foundation model (FM) fits, based on regulation, explainability, and operational constraints.

• "The basis for a loan screening result must be explained to a regulator" → the explainable traditional ML side.
• "We have to answer customer inquiries in natural sentences" → the foundation model side.
• Foundation models themselves are covered in Week 3. For today, just remember that it is not "generative AI is the latest, so always choose it."

Correcting Common Misconceptions

• "If you have lots of data, machine learning will work" → Before the amount, look at whether answers are attached and whether the data is skewed. One million records you cannot use are the same as zero.
• "Once a model is built, you're done" → When the world changes, the model goes stale too. Monitoring and retraining costs keep coming.
• "If you use machine learning, you don't need rules" → Policies that must be followed are still fixed in code. The model takes the ambiguous judgments outside of them.
• "If you can't tolerate error, you can never use machine learning" → It can be used in a decision-support form, where the model only narrows down candidates and a person makes the final decision.

📖 Glossary

• Traditional programming: An approach where a person writes rules as code and the computer executes them exactly as written.
• Model: The bundle of rules machine learning finds in data. It is the product of training.
• Training data: The past data used to teach a model, together with its answers.
• Data quality: How accurate, complete, and unbiased the data is. It matters as much as the amount of data.
• Probabilistic: Does not guarantee the same correct answer every time, and answers "this is likely."
• Black box: A state where an answer comes out but a person has difficulty explaining why.
• Explainability: The property of being able to give the basis for why a model made a decision. Especially important in finance, healthcare, and law.
• Churn prediction: The task of predicting in advance whether a customer will stop using a service. A typical ML use case.
• Fraud detection: The task of picking out abnormal transactions, such as in card payments.
• Foundation model (FM): A large model pretrained on huge amounts of data so it can be used broadly across many tasks. It is the basis of generative AI.

📝 Practice Questions

Click a choice to reveal the answer and explanation.

Question 1

For which of the following problems is traditional programming more appropriate than machine learning?

• ARecognizing cats in millions of photos
• BDetecting ever-changing spam email
• CCalculating tax by multiplying an input amount by a fixed 10% rate
• DRecommending products based on a customer's purchase history

Question 2

Which statement most accurately describes the input/output relationship of traditional programming and machine learning?

• ABoth take data and rules as input and output results
• BThe traditional approach produces results from data + rules, while machine learning produces rules from data + results
• CMachine learning takes rules as input and generates data
• DThe traditional approach produces results without any data

Question 3

In a certain business task, there are only a handful of historical records available for training, and the results must be legally 100% accurate and fully explainable. Which judgment is most appropriate for this situation?

• AAdopt machine learning immediately, since it is always more accurate even with little data
• BMachine learning may be unsuitable due to the lack of data and the requirement for complete accuracy and explainability
• CUsing reinforcement learning solves it perfectly even without data
• DThere is no problem since machine learning always guarantees explainability

Question 4

Which of the following is hard to consider a characteristic of problems suited to adopting machine learning?

• AThe patterns are so complex that humans struggle to write explicit rules
• BSufficient quantity and quality of training data is available
• CSome prediction error is acceptable
• DThe rules are very simple and unchanging, and a simple calculation suffices