The GPIO2 needs to be HIGH at Reset/PowerUp for normal boot.
If you wish that your software do something different when button is pressed, don't press button at power up but simply after, for example, during the first 5 seconds following the normal boot.
EDIT : I've look back to your wiring diagram, I see that the button is at VCC and there is a PullDown to GND. It should be reversed instead : PullUp to VCC and button to GND.
Simply switch resistor and button on your original diagram, making the button connected to GND and the resistor to VCC, acting as a PullUp.