Don’t hesitate to plant hardy plants – perennials, shrubs, trees, conifers, etc. – early, well before the last frost date. After all, these plants can withstand significant frost (always possible in May) and even better, recover more rapidly from transplant shock when the soil is still cool than from plantations at the more traditional planting dates in late May.
Wait while the soil is well drained before planting, though: working in the garden when the soil is still wet can destroy its structure.
For bare-root shrubs and trees, an early planting is even more important: they have to be planted before the foliage starts to expand significantly.