AIdata leakage& safetychallenge

Share a solution that stops an assistant or agent from exposing private data.

Partner prizes being added

Fast-track interviews, fellowships, cash and more

In March 2023, a ChatGPT bug let some users see other people’s chat titles, and a smaller set of Plus users had payment details exposed.

* March 20 ChatGPT outage: Here’s what happened ↗

Assistants and agents carry private text into places the user did not choose. A pasted document, a retrieved email, or a tool result can reappear in an answer, a log, or the next tool call, and a later refusal cannot pull that data back.

Nov 15 26’

Challenge start

Dec 15 26’

Deadline

Meet your jury

Your work will be evaluated
by industry experts.

The jury will be announced soon.

Top solutions

Winners

Best overall data-leakage prevention

Keeps private data inside the boundary the user intended.

  • Project name

    Author’s names

    1st

  • Project name

    Author’s names

    2nd

  • Project name

    Author’s names

    3rd

  • Project name

    Author’s names

    Finalist

  • Project name

    Author’s names

    Finalist

  • Project name

    Author’s names

    Finalist

  • Project name

    Author’s names

    Finalist

  • Project name

    Author’s names

    Finalist

  • Project name

    Author’s names

    Finalist

  • Project name

    Author’s names

    Finalist

Best in least-privilege data access

Retrieves only the sensitive data needed for the task.

Best in preventing reply and tool-call leaks

Keeps secrets out of answers and tool arguments.

Best in privacy-safe memory and logs

Prevents private data from resurfacing in later turns or stored logs.

Best in preserving useful output

Protects sensitive data without unnecessarily blocking legitimate work.

Winners will be featured on Drawerrr’s Winners page.
The Top 10 projects will be recognised across award categories.

Top solutions among

Partner categories

To be decided

2nd place

To be decided

1st place

To be decided

3rd place

Extra awards from our partners: fast-track interviews, fellowships, cash and more. Exact categories and prizes will be announced soon.

Nov 15 26’

Challenge start

Dec 15 26’

Deadline

How to succeeded

Master the Challenge in 5 days

Day 1

Identify high-risk scenarios

Pick an assistant or agent you can run, and name one place private text can leave: a reply, a tool call, a log, or a later turn.

Tip:

Start with a pasted note, a retrieved email, or a secret in a tool result.

Day 2

Set up a test environment

Recreate it with fake personal data in an inbox, document, or tool you control. Do not use real customer data.

Tip:

A test mailbox with dummy names, emails, and card fragments is enough.

Learn more:

Day 3

Choose a safeguard stack

Stop the private text from leaving. Redact it before the model sees it, block it in the reply, or keep it out of logs and the next tool call.

Tip:

Check the answer, the tool arguments, and the log. A leak in any one of them counts.

Learn more:

Day 4

Document failure states

Try a secret in the prompt, in retrieved text, in a tool result, and in a follow-up turn. Note what still appears.

Tip:

Also try a normal request that should still be answered, so the safeguard is not only a blanket block.

Learn more:

Day 5

Make it reproducible

Share the dummy data, the check, and a short record of what leaked and what did not.

Tip:

Publish the test set and the expected redactions so someone else can run them.

Learn more:

FAQ

Learn more about the challenge

1. What is the Data Leakage & Privacy Protection challenge?

This challenge asks participants to prevent assistants and agents from exposing private data in replies, tool calls, logs, or later turns. The goal is to stop sensitive information before it leaves the boundary the user intended. Share a solution that stops an assistant or agent from exposing private data. Potential areas include: • Redaction or data-loss prevention before content reaches the model • Output and tool-argument checks • Access controls and least-privilege retrieval • Log filtering and retention controls • Tests for leaks across tools and later turns

2. Who can participate in the challenge?

The challenge welcomes students, career switchers, working professionals, independent researchers, engineers, and others interested in practical AI safety. You do not need to be affiliated with a university or school. The affiliation field is optional. If your submission involves an employer’s intellectual property or confidential information, make sure you have permission to share it.

3. Can I participate individually, or do I need a team?

You may participate individually or as a team. An individual submission can suit a focused prototype, evaluation, or case study. A team may help when the work spans engineering, security, testing, hardware, or user experience. There is no maximum team size; we recommend up to five members.

4. What can I submit?

You can submit an existing project, an improvement to an existing solution, or a new approach that clearly addresses data leakage & privacy protection. Example approaches: • Redaction or data-loss prevention before content reaches the model • Output and tool-argument checks • Access controls and least-privilege retrieval • Log filtering and retention controls • Tests for leaks across tools and later turns You may submit a reusable agent, Skill, Bot, plugin, MCP, custom build, test harness, implementation, or article/case study.

5. How is it judged?

Every submission is evaluated against the following criteria:

CriterionWeightWhat judges evaluate
Safety effectiveness35%Does the safeguard reliably prevent the targeted unauthorized or harmful action?
Testing and robustness25%Has the solution been tested against realistic failure scenarios and bypass attempts?
Real-world relevance20%Is the problem meaningful, and does the proposed safeguard address it?
Reproducibility and reuse15%Can another team understand, test, or adapt the solution?
Clarity of communication5%Are the implementation, evidence, and limitations clearly presented?
Total100%—

6. What should I include in my submission?

Strong submissions define a clear safety problem, explain how the approach works, provide evidence, test relevant failure scenarios, and document limitations honestly. Depending on maturity, include a project overview, technical explanation, demonstration, tests and results, reproducibility instructions, and known limitations. Early-stage ideas and prototypes are welcome when you clearly separate what is implemented, tested, and still theoretical.

7. Do I need to open-source my project?

No. You may submit a plugin, connector, app, or an article or case study describing work in a closed system. If you share code or materials, remove credentials, personal data, and security-sensitive details. For a closed system, provide enough evidence and instructions for judges to understand and evaluate the approach.

8. When is it due?

Submit by Dec 15, 2026. Winners are announced Dec 31, 2026.

9. What prizes or recognition are available?

The strongest projects will be featured on Drawerrr’s Winners page, and the Top 10 projects will be recognised across award categories. Extra awards from our partners: fast-track interviews, fellowships, cash and more. Exact categories and prizes will be announced soon.

10. Will submissions be reviewed by experts?

Yes. Submissions are evaluated by a jury of industry experts using the published criteria. Jury members will be added to the challenge page as they confirm.

11. Can I update my submission after submitting it?

Yes. You can submit again with updated materials before the deadline. Each submission creates a new record, so use the latest submission for your final project link and details.

12. Who owns the intellectual property in my submission?

You retain ownership of your work. By submitting, you grant Drawerrr permission to display and evaluate the submission for challenge operations, mentoring, and jury review. Do not include confidential material you are not authorized to share, and confirm you have rights to employer, collaborator, or third-party content.

Subscribe to challenge results

Our mission is to accelerate practical solutions to real-world AI safety challenges by connecting people, ideas, and expertise to build AI systems that are safer, more reliable, and easier to govern. Drawerrr’s AI safety challenge is informed by leading institutions in AI safety, standards, and public policy.

European AI OfficeAISINIST