But they didn't sell that well to begin with..wasn't this the main issue?
That's not entirely true; at least, it wasn't here in the states. Both the SEGA CD and 32X sold well their first month or two of release. They just didn't have lasting appeal due to several factors. From what I've read, as well as experienced, there weren't enough games released for SEGA CD that really demanded purchase. Many titles were FMV-based, or expanded cartridge games, with very few titles that really took advantage of the CD format or SEGA's hardware.
The 32X was sort of a cluster-fuck. I mean, the Saturn was right around the corner. With SEGA jumping it's North American launch, not to mention the PSX and N64 following soon after, how could the 32X compete? On top of that, not having a steady stream of new titles to support it didn't help matters. It just shouldn't have been made. I love the thing, but SEGA really should've focused their efforts on Genesis, Game Gear and Saturn.
At one point, that were supporting 6 pieces of hardware; Genesis, SEGA CD, 32X, Saturn, Game Gear and Pico. That's a lot of mouths to feed.