Rather than requiring a precise 3.3V as most people imagine, the chip is more "interested" in the "stiffness" of the supply. That's why so many linear regulators fail in this role as they have a relatively long response time (delay between sudden increase in load current and output voltage regulation). This is why "Big" capacitors can solve issues - but even then there's a catch because capacitors also have a certain amount of series resistance (ESR) which drops volts "on their way out". So only exceptionally low ESR caps are going to help.
It's also worth noting that switch-mode regulators are a better bet as they generally respond faster than linear regulators but they come wither a higher parts count and price-tag. On the other hand, a 1N4001 Si. diode represents a parts count of one, and also protects from reverse-supply connection if your battery happens to be removable