Me again! I couldn't figure out why I was getting transient errors on that Poe2wiki sitemap, so I tried looking at other sitemaps on the site. I'm getting transient errors on all of them.
Then I realized - the sitemap files are being overwritten in real time. Every edit on the Wiki is causing the sitemap file to be edited in real time. That's why the sitemaps sometimes stop right in the middle of a filename when I access it - the sitemap file is in the process of being rewritten.
That's a behavior unique to a Wiki, and might explain the entire phenomenon.
Makes sense. Whatever generates those site maps should use the classic Unix trick of writing a separate file then renaming. Also maybe don’t update lastmods on each individual update, it’s a waste of resources since the granularity of lastmod is usually the day (though you can make it the second if you want).
Though weirdly it makes the subdomain result even more interesting: presumably those wikis are rewriting their sitemaps too, yet Google manages to index them normally
Then I realized - the sitemap files are being overwritten in real time. Every edit on the Wiki is causing the sitemap file to be edited in real time. That's why the sitemaps sometimes stop right in the middle of a filename when I access it - the sitemap file is in the process of being rewritten.
That's a behavior unique to a Wiki, and might explain the entire phenomenon.