Prompt injection occurs when malicious or misleading instructions are placed in user input or content the model reads, with the aim of changing its behaviour. A prompt can help define intended behaviour, but it cannot create a reliable security boundary by itself. Secure design combines clear instruction hierarchy with application-level controls.
Treat retrieved and uploaded content as untrusted
Documents, emails, webpages, issue descriptions and transcripts can contain instructions that are unrelated to the user’s task. Tell the model to treat such material as data to analyse, not as authority to change system behaviour. Delimit retrieved content and identify its source, but do not assume delimiters alone prevent injection.
Keep permissions outside the prompt
The application must decide which user can access a document, which tools are available and what actions are allowed. Apply authorization before retrieval and before every action. A model saying that a user is authorised is not proof of identity or permission.
Use narrow tools and explicit confirmation
Expose only the tools needed for the task. Prefer read-only operations when possible. Validate arguments against a schema and require an authorized human to approve sensitive, external or irreversible actions. Tool descriptions should explain intended use and limits, but the backend must enforce them.
Avoid leaking sensitive context
Do not place secrets, API keys or unnecessary personal data in prompts. Redact or minimise content before sending it to a model provider. Restrict what tool results are returned to the model and make sure logs do not become an uncontrolled copy of confidential data.
Test direct and indirect injection scenarios
Create tests where the user asks the model to ignore rules, where a document contains hidden instructions, where a webpage requests a tool call and where retrieved data attempts to expose another user’s content. Verify that the model keeps its task boundaries and that the application independently blocks unauthorized actions.
Further reading and resources
The article is original GrapeCrux editorial content. The following official documentation and technical resources provide deeper reading on the concepts discussed:
Related GrapeCrux services and articles
Need help evaluating or implementing a similar solution? Start with the business requirement and discuss the appropriate scope with the GrapeCrux team.
Discuss a project ↗ Back to all blogs