1. Are there some fundamental difference in the way the esp8266 works, which conflicts with the mainstream Arduino?
The answer to the first question is no, it's not necessary. In fact, you can download the latest Arduino IDE and then use AdaFruit's proxy to install the ESP8266 library's.
Instructions can be found here: https://learn.adafruit.com/adafruit-huz ... rduino-ide
However, if you meant to ask, why is it necessary to fork the library? Then the answer (as was eluded to) is that the ESP8266 is a completely different architecture (It's a RISC chip rather than an AVR or an ARM).