The issue is when using an LDO. I have tried using various LDOs, MCP1700 and MCP1825s which is rated at 500mA. Neither of these are stable enough. Looking at the serial output I get all sorts of crashes, memory dumps and resets when powering up. I am using the caps recommended on the spec sheets for the regulators, and I tried placing in 470uf cap and 1uf decoupling cap close to the ESP vin/gnd pins. These helped, but still not completely stable, but still sometimes fails to boot or resets when connecting to wifi. This is in a breadboard if it makes any difference.
Any ideas? I am at the point where I will probably just run it straight from the battery as I am so fed up