AI & LLM Security interview questions

Unbounded autonomy plus tool access is OWASP LLM 'excessive agency': a manipulated or mistaken model can take destructive actions. Constrain it with least-privilege tool scopes, require human confirmation for irreversible operations, and keep permissions narrow and audited. Trusting it or granting admin widens the blast radius, and hiding a UI button does nothing about the model's underlying capability to call the action.

Untrusted content the model ingests can carry instructions — indirect prompt injection. You can't fully prevent the model from being influenced, so isolate fetched content as data, constrain what tools/permissions the agent has, require confirmation for sensitive actions, and avoid giving it secrets it could be coerced into leaking. Assuming the model will simply ignore injected instructions is exactly the failure mode being exploited.

A third-party model is a supply-chain dependency: verify it comes from a trusted source with matching checksums/signatures, that its license permits your use, and that the file format won't execute arbitrary code on load (prefer safe serialization over pickle-style formats). 'It loads' and 'download speed' say nothing about trust, and assuming public models are safe ignores real poisoning and deserialization risks.

This is OWASP LLM 'improper output handling': model output influenced by user input can become command injection when fed to a shell or eval. Treat it as untrusted — map intents to a small allow-list of parameterized actions, validate strictly, and sandbox execution rather than running raw generated strings. Trusting the output, raising token limits, or only logging does nothing to stop the injection.

Sending customer PII to an external API exposes it to a third party's processing and retention and can breach privacy obligations. Minimize and redact what's sent, confirm the provider's data-use/retention terms and a data-processing agreement (or no-training guarantees), or move to a private deployment for sensitive data. Key length is irrelevant, and sending more PII increases the exposure.

If retrieval pulls any indexed document regardless of who's asking, the model will faithfully surface data the user shouldn't see — this is an authorization flaw, not hallucination. Enforce the user's document-level permissions at retrieval time so only authorized chunks enter the context. A bigger system prompt is bypassable and doesn't implement access control, temperature is unrelated, and another model has the same gap.

Training on unvetted user data lets an attacker poison the model — implanting backdoors, triggers, or skewed behavior that surfaces later. Control it with data vetting and filtering, provenance tracking, anomaly detection on the dataset, and evaluation of model behavior after training. 'More data is better' ignores integrity, and the real concern is poisoning, not speed or disk space.

Prompt injection can't be fully solved with more prompt text; assume the model can be subverted and constrain what it's allowed to DO. Give tools least privilege, gate high-impact actions behind human confirmation, and validate or sandbox tool calls before acting (OWASP LLM 'excessive agency' and 'improper output handling'). System-prompt pleading is bypassable. Higher temperature only adds randomness, and logging alone records the damage without preventing the injected action.

AI helps the SOC by triaging and de-duplicating alerts, summarizing incidents, enriching context, drafting detections, and accelerating analyst onboarding — reducing fatigue and dwell time. The risks: hallucinated or confidently wrong conclusions, automation bias where analysts stop verifying, prompt injection through attacker-controlled log or alert data, sensitive data leaking into third-party models, and adversaries using the same tools. Keep a human in the loop, verify outputs, and isolate untrusted inputs.

Direct prompt injection is when a user types adversarial instructions straight into the prompt to override the system prompt or safety rules. Indirect prompt injection hides malicious instructions inside external content the model later ingests — a web page, email, PDF, or RAG document — so the attack fires without the victim ever typing it. Indirect injection is the bigger risk because the attacker and the victim are different people, and the payload rides in on data the app implicitly trusts.

Insecure output handling is trusting what the model returns and passing it to a downstream system without validation or encoding. Because model output is attacker-influenceable, rendering it as raw HTML causes XSS, feeding it to a URL fetcher causes SSRF, and passing it to a shell or SQL query causes command or SQL injection. The fix is to treat model output exactly like untrusted user input: context-aware output encoding, allowlisting, sanitization, and parameterization before it reaches any sink.

A jailbreak targets the model's safety alignment — it tricks the model into producing content the provider tried to prohibit, like instructions for harm. Prompt injection targets the application's instruction hierarchy — it overrides the developer's system prompt or hijacks the model's behavior within an app, often via untrusted data. Jailbreaks attack the model; prompt injection attacks the surrounding system. They overlap, but the goal and the trust boundary they cross are different.

The LLM supply chain spans base models, fine-tuned variants, datasets, embeddings, plugins, libraries, and the hosting platform — each a place to introduce risk. Threats include downloading tampered or backdoored model weights, malicious fine-tunes, poisoned or license-tainted datasets, vulnerable or over-permissioned plugins, and typosquatted model repos. Defenses: source models from trusted registries, verify integrity and provenance, maintain an AI bill of materials, scan and pin dependencies, vet plugins, and apply least privilege to anything the model integrates with.

The NIST AI Risk Management Framework (AI RMF 1.0) is a voluntary, risk-based framework for governing trustworthy AI across its lifecycle. Its core is four functions: Govern (culture, policy, accountability — and it runs through the others), Map (context and risk identification), Measure (assess and track risks), and Manage (prioritize and respond). It also defines trustworthiness characteristics — valid and reliable, safe, secure and resilient, accountable and transparent, explainable, privacy-enhanced, and fair. It complements technical lists like the OWASP LLM Top 10 at the program level.

The OWASP Top 10 for LLM Applications is the consensus list of the most critical risks when building with large language models. The 2025 edition covers prompt injection, sensitive information disclosure, supply chain, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, and unbounded consumption. It exists because traditional appsec lists don't capture LLM-specific failure modes, and it gives teams a shared vocabulary and a checklist to prioritize controls.

RAG security means treating every retrieved document as untrusted input. Key risks: indirect prompt injection hidden in retrieved content, poisoning of the knowledge base or embeddings, and missing per-user authorization so the model returns data the user can't access. Defenses include access control enforced at retrieval, content provenance and ingestion vetting, treating retrieved text as data not instructions, output validation, and isolating the vector store per tenant.

An LLM agent turns text into actions via tools and function calls, so a prompt injection becomes a real-world action — the excessive-agency risk. Secure it by giving each tool the least privilege and scope it needs, validating and constraining tool arguments, requiring human confirmation for sensitive or irreversible actions, sandboxing execution, rate-limiting and budgeting calls, and logging every tool invocation. Never let the model's untrusted-data-influenced output directly authorize a high-impact action.

LLM apps leak data several ways: the model memorizes and regurgitates sensitive training or fine-tuning data, the system prompt (which may hold secrets or logic) gets extracted, retrieved RAG documents expose data the user shouldn't see, and context from one user or session bleeds into another. Prevention means data minimization before training, never putting secrets in prompts, enforcing per-user authorization on retrieval, output filtering and PII redaction, and tenant isolation.

Training-data poisoning is when an attacker tampers with the data used to pre-train, fine-tune, or embed a model so the resulting model behaves maliciously — embedding a backdoor trigger, injecting bias, or degrading accuracy. It exploits the fact that models scrape and trust large, often web-sourced datasets. Defenses include curating and signing data sources, provenance and integrity checks, anomaly detection on training data, dataset versioning, and limiting who can contribute to training and RAG corpora.