- Meri Nova
- Posts
- AI Product Engineer Day 6
AI Product Engineer Day 6
Day 6: Prompt Engineering and Context management for Agents
Meri Nova
May 13, 2025
Day 6: Prompt Engineering and Context Management for Agents
Welcome to Lesson #6!
In the previous lesson, we covered evals, tracing, and guardrails. Today, you will learn about prompt engineering fundamentals for Agents - the foundation for building smarter, more reliable agents.
Agenda:
Prompt Craftings vs Prompt Engineering
Context Management for Agents
The 5 Tiers of Memory (How We Teach It)
If you missed previous lessons, find them here:
👉 Day 1: Intro to Agents
👉 Day 2: Agents vs Workflows
👉 Day 3: RAG and Tool Use
👉 Day 4: Memory
👉 Day 5: Evals for Agents
🎓 Why Join AI Product Engineer Bootcamp?
Go beyond demos and build agentic systems that work in the real world, join our next AI Product Engineer Cohort in June, 2025!
Here’s what you’ll get:
Vibe Coding sessions 5 Days a Week. Watch senior engineer code right in front of your eyes every day to 10x your learning speed.
Community Support: Join a network of hungry and ambitious innovators to keep you accountable (hosted on our private community platform).
Build with Confidence: Don’t just learn the latest tools like MCP, and Agent SDK, master the SWE principles behind them so you can build, debug, and scale your own systems from scratch.
Personalized Coaching: Get direct access to Hai and Meri, who’ve helped 100s non-tech professionals land their first jobs in tech.
Ready to turn your AI curiosity into high-income skills?
👉 Reserve your spot in the Bootcamp now!
🚨 Prompt Crafting vs Prompt Engineering
Most beginners stop at:
“Write me a summary…”
“Classify this text.”
“What are 5 ways to...?”
That’s prompt crafting.
But for agents to reason, act, and handle complexity, you need to go further:
✅ Prompt engineering is about building systems that:
Work across edge cases
Handle ambiguous inputs
Think step-by-step
Maintain output structure
And improve with evaluation and refinement
AND here’s the challenge:
LLMs are stateless by default.
If you don’t feed them the right context, they don’t know who they’re talking to — or what just happened!
🧠 Context Management: The Real Bottleneck
When most people try to build agents, this is where they fail:
They hardcode everything into a single system prompt
Or dump the full message history into every request
Or worse, they assume the LLM “just remembers”
But memory isn't magic. It’s system architecture.
To build reliable and adaptive agents, you need structured, multi-tiered context management.
🔁 State Management
You need to remember that LLMs don’t remember past interactions — any sense of continuity must be manually designed. State management means tracking key information across sessions:
task progress,
user preferences,
prior responses,
and behavioral cues.
Short-term state can be managed in memory or with lightweight stores like Redis, while long-term state is best handled using relational databases (Postgres) or vector stores (PGVector, Pinecone) with semantic retrieval.
If you want to build stateful agentic systems, join our next cohort in June!
📊 The 5 Tiers of Memory (How We Teach It)
One way to manage context is building multi-tiered memory for agents. In our bootcamp, we train AI engineers to design context systems using a proven tiered approach.
Watch the full video from our instructor, Hai.
F Tier – No Memory
Each prompt is treated independently. No state is preserved. You send all relevant info in every request.
🪫 Stateless, zero-shot interaction — suitable for isolated queries or one-off completions.
D Tier – System Prompt Memory
You hardcode persistent context (e.g., “You are a career coach”) into the system prompt.
📌 This works like a static initialization script. It grounds the assistant but doesn’t adapt dynamically.
C Tier – Message History
You append a running chat log into the prompt, simulating memory.
📚 Useful for maintaining conversational flow, but inefficient at scale due to token limits and noise accumulation.
B Tier – Entity Memory
Extract and store structured facts (e.g., likes/dislikes) as key-value pairs. Re-inject them selectively at runtime.
🧩 Enables lightweight personalization with minimal context window usage. Usually stored outside the LLM in a local cache or database.
A Tier – Embedding + Vector Store Retrieval
Facts about the user or conversation history are embedded and indexed (e.g., via Pinecone or PGVector).
⚙️ At query time, semantically relevant facts are retrieved via cosine similarity and added to context. Supports personalization at scale with efficient memory recall.
Our instructor, Hai 🙂
Congratulations on completing Day 6!
You’ve just leveled up your agents with better context management. Tomorrow, we’ll explore how to build multi-agent frameworks.
If you want to build production-grade agentic systems, join our next cohort in June! You won’t learn how to write simple prompts. You will learn to build reliable AI systems with Python or TypeScript track.
🔑 Unlock new opportunities:
Build cutting-edge portfolio projects
Unlock Applied AI Engineering roles (with $150k+ compensation)
Build a new business or side hustle
Build a lifelong professional network
Spots are limited!
If you have any additional comments, suggestions, or feedback, respond to this email directly. We’d love to hear from you!
How was today's newsletter? |