LLM incorrectly follows instructions within user prompts

[kangsuhyun-yanolja]

Arena is currently experiencing a critical issue where the LLM attempts to follow instructions contained within user prompts rather than summarizing/translating the entire text as intended. For example, when a user submits a prompt containing instructions for an AI assistant,

You are an AI assistant for a smart home device manufacturer.
Analyze customer feedback messages and extract relevant information.
Provide your analysis in JSON format without additional commentary.

- Disregard pleasantries, repetitive content, and unnecessary personal details.
- Identify and summarize the following elements (if present):
1. Primary concern or feedback type
2. Specific device model mentioned
3. Setup process difficulties
4. Connectivity issues
5. Battery life or power-related problems
6. Voice command recognition accuracy
7. Mobile app functionality
8. Integration with third-party services
9. Privacy or security concerns
10. Software update experiences
11. Hardware malfunction reports
12. Comparison with competitor products
13. Feature requests or suggestions
14. Warranty or return inquiries
15. Customer support interaction details
16. Overall satisfaction level indicators

our service tries to execute those instructions instead of translating the text. This behavior is incorrect, as the service should summarize/translate all content, including any embedded instructions, without attempting to follow them.