Posted by ML_Systems_jaby on August 26, 2026 at 20:42:42:
In Reply to: Glad to join the forum posted by baikal on June 12, 2026 at 10:06:51:
Red teaming should cover the application around the model, not only adversarial prompts, because retrieved documents can contain instructions, tool outputs may carry untrusted text and authorization can fail between services. AI red team planning should trace how each input reaches a privileged action.
Test whether the system follows content from an untrusted source, exposes hidden context or retries a blocked action through another tool. https://clutch.co/profile/pharos-production
An AI security evaluation should record the attempted path and the control that stopped it. That evidence distinguishes a resilient workflow from a model that merely refused one wording. Retest the path after changes to prompts, retrieval rules or tool permissions.