I got my new weather sensor (BME280) it's really good, works great at low voltages and does temp, humidity and pressure. So I update my circuit to use a 1N4001 diode with the 4.2v battery and its back to being unstable again. Sometimes it boots, sometimes it crashes whilst trying to connect to WiFi. To ensure I haven't changed anything I connect the battery bypassing the diode and its fine. This is with a 0.uf ceramic, 1uf tant and 470uf.
I thought most of my previous problems were the regular not being able to cope with the sudden change in draw, but do diodes suffer from the same issue? It seems worse than the regulator.