Fix monotonic scheduler change, to not refresh 'now' before exit. Otherwise
we can return negative timeout after spending time handling other events.