No, I mean that when constrained (like discussing a specific bug in code) the options are limited.
SynthID specifically mentions something similar: "It is harder to watermark factual answers because the model has fewer alternative word choices available without altering accuracy."
In the codebase, I do see the models finding alternative word choices -- and I hate them. Already in a highly technical latent space, it reaches for other highly technical word choices (which may be more accurate, but ones I am unfamiliar with -- like terms in ERP systems as it felt that was close but bring just technical jargon that i have to google what it is saying because i don't understand -- and i have to google the phase sentence as the words themselves are ok, just not how they are put together).
Anthropic in particular has been moving the level of watermarking since the beginning of the year. You didn’t think it just went from off to on one day did you? It needs time to get plastered over the internet, see what people do that break it, how Google summaries erase it (or not, or add their own). All of this takes at least months if not a year.
SynthID specifically mentions something similar: "It is harder to watermark factual answers because the model has fewer alternative word choices available without altering accuracy."
In the codebase, I do see the models finding alternative word choices -- and I hate them. Already in a highly technical latent space, it reaches for other highly technical word choices (which may be more accurate, but ones I am unfamiliar with -- like terms in ERP systems as it felt that was close but bring just technical jargon that i have to google what it is saying because i don't understand -- and i have to google the phase sentence as the words themselves are ok, just not how they are put together).