Day 3 - Prompt Engineering Basics: Good Prompts, Zero/Few-shot, and the Limits

📌 Key Takeaways

• Prompt engineering is the technique of refining the input text to draw out the desired answer without retraining the model. It is the fastest and cheapest improvement method to try.
• A good prompt clearly contains the instruction, context, input data, and output format, and raises quality through role assignment and examples.
• Zero-shot shows 0 examples, One-shot shows 1, and Few-shot shows a few. All are In-Context Learning that does not change weights, so they differ from fine-tuning.
• Format, tone, and behavior are solved with prompts, information the model doesn't know with RAG, and deep internalization of a domain with fine-tuning. The order to try is "prompt → RAG → fine-tuning."
• The limits of prompts are information the model doesn't know (hallucination), the context window and token cost, non-determinism, and security risks such as prompt injection.

Components of a Good Prompt

Even with the same model and the same question, the quality of the answer varies enormously depending on "how you ask." "Write something for me" and "Explain photosynthesis in 5 sentences, in a friendly tone, so a 4th grader can understand" produce completely different results. This "skill of asking a model well" is prompt engineering.

• With generative AI, you can get a considerable share of the results you want just by writing a good prompt, without training a new model.
• Training and fine-tuning are expensive and slow, whereas a prompt can be changed immediately and has no additional training cost. That is why, when the AIF exam asks for "the fastest and lowest-cost improvement method," prompt engineering is the answer.

A good prompt comes from structure, not luck. It usually clearly contains the following elements.

A prompt that fills in the elements above produces far more stable results than a vague "analyze this." There are four key principles.

1. Be specific and clear: "Within 3 sentences" rather than "short."
2. Assign a role (persona): Giving a role such as "You are a friendly customer service agent" sets the tone.
3. Specify the output format: Stating the desired shape, such as JSON, a table, or bullets, makes post-processing easy.
4. Provide examples: Showing a model of the desired answer raises the hit rate (Few-shot).

Prompt engineering is effective because LLMs have been further trained to "follow instructions (instruction-tuned)." A raw model that has only been pretrained simply appends the next token, but a model aligned with human feedback (RLHF, reinforcement learning from human feedback) has learned the behavior of "understanding and following requests." So the clearer the instruction and format you give, the more the model aligns its response to that pattern. A prompt does not change the model's weights; it merely sets conditions on "which output distribution to draw out."

The Process of Rewriting a Vague Prompt

To see what the four elements actually change, we fix one prompt step by step. The situation is classifying shopping mall reviews and routing them to the responsible department.

• The "only one word" in step 3 also reduces output tokens. Narrowing the format often improves quality and cost together.
• Specifying the default answer for when the model doesn't know, as in step 4, is a representative prompt technique for reducing hallucination.
• Explicitly stating "don't do this" is called a negative prompt. It is an instruction that removes unwanted elements, such as "don't mention price," or, in image generation, "don't include text or watermarks." The exam guide lists this, together with context and instruction, as a component concept of prompts.

Prompt Templates and Management

In a real service, the user does not write the entire prompt. Developers build a frame in advance and only the user input is inserted.

Classify the sentiment of the following {{document type}} as one of {{choices}}.

Text: {{user input}}
Answer:

• A frame that leaves blank only the parts that change, like this, is a prompt template. The instruction, context, examples, and format are fixed and only the input is swapped in, so results are consistent.
• Like code, changing a template changes the results. That is why you need to keep versions and compare them. Prompt Management in Amazon Bedrock is a feature that creates and stores prompts, inserts variables (placeholders), compares the outputs of several variants side by side, and keeps versions to deploy to an application.
• When the exam says "a team wants to reuse prompts and manage change history," that is Prompt Management.

Zero-shot vs Few-shot: How Many Examples You Give

Among prompting techniques, a representative classification covered by the AIF exam is "how many examples (shots) you give." Here, one shot means one pair of "an example input and its desired output."

Zero-shot: Give only the instruction, with no examples. The model solves it with its pretrained knowledge alone.

Answer the sentiment of the following sentence as either positive or negative.
Sentence: "This movie was truly the best!"
Sentiment:

Few-shot: Show a few models of the answer first, then ask the real question. The model sees the pattern and imitates it.

Sentence: "The food was terrible" → negative
Sentence: "The service was friendly" → positive
Sentence: "This movie was truly the best!" →

One-shot is the case of giving exactly 1 example. The exam guide writes this as single-shot.

Few-shot does not change the model's weights. It merely shows examples within the prompt, so it is also called In-Context Learning. Don't confuse it with real learning (fine-tuning).

• Examples disappear when the conversation ends. To get the same effect in the next request, you have to include the examples again, and each time they are billed as input tokens.
• Give examples evenly. If you show three positive examples only, the model leans toward positive.
• The format of the examples becomes the output format. If you ended the examples with "→ negative," the answer comes out in that shape too.

🔍 Going Deeper: A technique one step further is Chain-of-Thought (CoT) prompting. When you guide the model to write out its reasoning process, as in "Let's think step by step," the accuracy rises substantially, especially on math and logic problems. This is because the model "spits out" intermediate reasoning as tokens and uses it as the basis for predicting the next step. When the AIF exam says "we want to raise the accuracy on complex multi-step reasoning problems with prompts alone," think of CoT. There is a cost too. Output tokens increase by the length of the working-out, so cost and latency grow.

Prompt vs Fine-tuning vs RAG: How Far Can Prompts Solve It?

There are several layers of ways to change a model's behavior. The standard approach is to try them in order from lowest cost and difficulty.

Key judgments:

• Information the model doesn't know is needed → Prompts can't do it. You must supply knowledge through RAG (or fine-tuning).
• Only the "format, tone, behavior" of the output needs to change → Prompt engineering is enough.
• You want to permanently internalize vast domain patterns → Fine-tuning.

By analogy: a prompt is writing a good work instruction sheet for a new employee, RAG is spreading a reference binder on their desk, and fine-tuning is sending them to training to change the person themselves. The instruction sheet and the binder can be changed today, but training costs time and money.

As a hypothetical case, a legal advisory startup initially considered fine-tuning a model right away for "more accurate answers." The cost estimate came to tens of millions of won. On a consultant's advice, they first tried prompt engineering (role assignment + a fixed output format + Few-shot examples) and RAG using the company's own case law, and most of the target quality was met. Fine-tuning was never needed in the end. "Cheap methods first, expensive methods later" is the golden rule of generative AI practice.

Limits of Prompt Engineering

Prompts are not omnipotent. Knowing the limits matters both for the AIF exam and in practice.

1. It cannot produce (or it makes up) information it doesn't know: However well you write the prompt, up-to-date information the model hasn't learned or private in-house data will not come out accurately. Instead, hallucination, where plausible-sounding content is invented, occurs (more tomorrow).
2. Context window limit: If you include too many examples, you hit the token limit and the cost grows as well.
3. Non-determinism: Even with the same prompt, answers can differ slightly each time, so there are limits for tasks that need 100% consistency.
4. Vulnerability to prompt injection: If malicious phrases such as "ignore the previous instructions..." are mixed into user input, the model can behave differently from what was intended (a security issue).
5. Sensitive or deep domain expertise: When consistent professional tone/regulatory compliance is the key, prompts alone have limits and fine-tuning may be needed.

⚠️ Pitfall: "The longer and more detailed the prompt, the unconditionally better" is a common misconception. If it is excessively long, token cost and latency grow, and the key instruction gets buried so the model can actually get confused. Also, giving contradictory instructions ("write briefly but include everything") makes results unstable. A good prompt is not long but clear and consistent.

Four Attacks Aimed at Prompts

The exam guide lists exposure, poisoning, hijacking, and jailbreaking as risks of prompt engineering. Because the names are similar and confusing, tell them apart by "what they target."

• The collective term for the four is prompt injection. AWS documentation describes this as an application-level security issue, like SQL injection. AWS protects the underlying infrastructure, but designing to prevent injection is the responsibility of the customer building the application.
• Defenses: validating user input, not putting secrets in the system prompt, minimizing the permissions given to the model, and the prompt attack filter of Amazon Bedrock Guardrails.
• "Writing 'never ignore the instructions' in the prompt blocks it" is a misconception. The instruction is ultimately just text too, so it can be overpowered by a more cunning input. Safeguards outside the prompt must be in place as well.

Choosing a Technique by Scenario

Tomorrow we face head-on the risks of generative AI, such as hallucination, bias, and non-determinism, which are the core of those limits.

📖 Glossary

• Prompt: The entire input given to the model. It includes the instruction, context, examples, and the data to process.
• Persona (role assignment): A way of setting the tone and perspective of the answer by fixing a role with "You are ~."
• shot: One pair of "an example input + the desired output" placed within a prompt. With 0 it is Zero-shot; with a few it is Few-shot.
• In-Context Learning: Following a pattern on the spot by looking at examples in the prompt. The weights do not change.
• Chain-of-Thought: A technique that raises reasoning accuracy by having the model write out the working step by step rather than giving only the answer.
• Prompt template: A prompt frame that leaves only the changing parts as blanks. The same frame is reused for many inputs.
• Negative prompt: An instruction that tells the model what should not go into the result.
• Prompt injection: An attack that mixes malicious instructions into the input to make the model act differently from its original intent.
• Jailbreaking: An attempt to bypass the model's safeguards and draw out forbidden answers.
• RLHF: A training method that uses human evaluations as the reward to refine the model so it follows instructions better.

📝 Practice Questions

Click a choice to reveal the answer and explanation.

Question 1

Which of the following is the most accurate explanation of Few-shot prompting?

• AA learning method that permanently changes model weights
• BA technique where a few example answers are shown in the prompt to make the model follow the pattern
• CA method where no examples are given, only instructions
• DRetraining the model from scratch

Question 2

Which of the following is NOT a recommendation for writing a good prompt?

• ASpecify the desired output format (e.g., JSON, one word)
• BAssign a persona (role) to establish tone
• CWrite as ambiguously and generally as possible
• DGive specific and clear instructions

Question 3

You need to reflect the contents of the latest internal documents the model wasn't trained on in your answers. What is the most appropriate approach?

• ASolve it by strongly instructing "answer accurately" in the prompt
• BUse RAG (Retrieval-Augmented Generation) to find relevant documents and provide them with the prompt
• CLowering temperature to 0 will make unknown information accurate
• DIncreasing Few-shot examples will make the model know internal documents

Question 4

What is the general standard order to try methods of adjusting model behavior from lowest to highest cost/difficulty?

• AFine-tuning → RAG → Prompt engineering
• BPrompt engineering → RAG → Fine-tuning
• CRAG → Fine-tuning → Prompt engineering
• DFine-tuning → Prompt engineering → RAG

Question 5

Which of the following is NOT a limitation of prompt engineering?

• AInformation the model wasn't trained on is hard to answer accurately, and hallucinations can occur
• BThe same prompt can produce different outputs each time (non-determinism)
• CThere is prompt injection vulnerability from malicious user input
• DOutput format and tone cannot be adjusted