For light sleep the system clock is reset but the RTC clock is not.
For modem sleep neither clock is reset.
The built in RTC is actually not very accurate - it is affected a lot by temperature, which is why there is a calibration function in the SDK. The problem is that the moment the chip goes into deep sleep it starts to cool down and the timing starts to drift.
If you really need accurate timing then you need to use an external RTC, or connect to an sntp server when you wake up. Even when the chip is awake I use the system clock for my timing as it is way more accurate than the RTC.