I've learned about context engineering

Context Engineering is the discipline of designing and building dynamic systems that provides the right information and tools, in the right format, at the right time, to give a LLM everything it needs to accomplish a task.

One of the biggest challenges I faced when creating AI agents is not necessarily creating the prompt, but creating the context around the prompt in order for the agent to give the best possible response.

It's one thing to create an agent with a set of instructions, but I found that the real complexity comes when you have to handle real-time data, dynamic user states, tool availability, and retrieved knowledge, all in a way that the agent can make sense of in a single, coherent shot.

In a practical sense, a very basic example is where

const prompt = "My order hasn’t arrived. What can I do?";
const response = await callLLM({ prompt });

Essentially becomes:

const retrievedDocs = await searchHelpCenter("order hasn’t arrived");
const customerData = await getCustomerOrderStatus("order123");

const systemPrompt = `
You are a support agent. Base your answers on the order data and help center.
Use informal but helpful tone.
`;

const context = {
  prompt: "My order hasn’t arrived. What can I do?",
  documents: retrievedDocs,
  userOrder: customerData,
  tools: ["issue_refund", "resend_order"],
};

const response = await callLLM({ system: systemPrompt, context });

It's also essential that you retrieve data that is very relevant to the input of the user due to the current limitations of the context window and to minimise the risk of hallucinations.

Must read:

The New Skill in AI is Not Prompting, It’s Context Engineering

Context Engineering is the new skill in AI. It is about providing the right information and tools, in the right format, at the right time.

Philipp SchmidPhilipp Schmid