How would it put stress on infrastructure? And would a surplus of unskilled workers be an issue?
Immigrants typically make a net contribution in taxes, relative to how much they take out in transfers or services from the government, yet things like schools and hospitals take time to build. The taxation, and accumulation of public funds is immediate, but it takes a significant amount of time to actually translate into public infrastructure.
As for unskilled workers--I don't think so. Although we should place the most emphasis on freeing up educational and high-skilled immigration. All the literature I've read doesn't suggest a significant negative impact (like wage compression) from unskilled immigration which isn't incredibly localised to one part of the native population or temporary.