On the loading issues, by virtue of being on a card doesn't mean it doesn't need to load. Depending on the class of SD card you have, loading games off of the SD card can be slow, at least compared to things like the 3DS cards. Once you begin downloading a lot of apps you may find it takes a few seconds for things from the 3DS's SD card to appear.
On the Vita, games load faster off the game cards then off of the memory stick, though I hear the difference there is minimal. Sony says they use high speed memory cards, so they're probably faster then SD cards, at least the one you own.