Maybe the problem is that these LLMs will say something often enough for us to notice it, and it can be basically any arbitrary thing. Once we notice the pattern, it starts irritating us.