Ignoring the slowdown ability makes the game Hotline Miami levels of hard, especially the last two sets of levels. Luckily, you only need to do it once and can earn all other stars seperately after that. Sometimes I get lucky and do a par time no slowdown in one go, sometimes I can only manage the no slowdown star. After that, it becomes fairly easy to complete the par time and/or the perfect aim achievement using said ability.

But I don't feel the game becomes impossible if you don't utilize the slowdown, just pretty difficult.