This shield is also able to charge your battery.
Your battery has a nominal voltage of 3.7V, but it varies from 3.0V to 4.2V depending on whether it is empty or fully charged. So the 4.2V will probably kill your ESP is you connect it directly.
A method I have used before is boosting the battery voltage to around 5V using a step up converter module.
https://www.ebay.com/itm/DC-3-24V-to-5V-28V-9V-12V-24V-2A-Adjustable-Boost-Step-Up-Converter-Power-Module/112619032791?hash=item1a389e10d7:g:NE8AAOSwvDpbBoGe
And then a 3.3v linear regulator to make a nice and steady 3.3v (for example LD1117v33) with plenty of power. Maybe not the most efficient way, but it works fine.