伊始 follows the thing that begins:
新年伊始 (at the start of the new year),
上任伊始 (just after taking office). The set phrase
下车伊始 (on first arriving at a new post) is often critical: it describes an official who starts giving orders before he has looked around.
伊 is an old particle with no meaning of its own here.
