I think some research on trypophobia found that humans biologically see the formations as dangerous, and the shivers we get are warnings from our bodies to stay away.
Think about it: Ant hills, wasp nests, beehives, all of these things have the same types of patterns, and all of them are things you generally want to avoid.