Once unpacked the IDE, make a "portable" folder inside the IDE folder and start the IDE.
Preferences and additional tools and Boards will be installed into that folder.
I got it from there:
http://forum.arduino.cc/index.php?topic=313236.15
See post #25 by Federico Fissore.
After that you have to reload all not native Boards like the esp in Boards manager to get it placed into the new "portable* folder. You also have to redo your Preferences.