As I initially suspected overflow in time handling, I took a detailed
look at lib/time.c. This adds comments about units being used, reduces
amount of type casting being done, and makes __udelay() always wait at
least one tick. (Current code could do no delaying at all for short
delays).

Signed-off-by: Pavel Machek <pavel@denx.de>