To be fair, it wasn't entirely speculation that nearly killed the industry, but a particular type of speculation, or speculation paired with other things.
• Speculating by ordering multiple copies in advance didn't prevent potential readers from buying a book. But the go-out-and-grab-them-all type of speculator may have
• Speculation paired with Extremely late books (pun intended) was a problem. Many retailers tied up their capital in Image books that were solicited and took forever to ship, if they shipped at all, preventing retailers from recouping their costs, because they had no product to move, or from ordering more books that did ship on time
• When the speculator market fell apart, many retailers were left with books that customer requested but never bothered to come back for. Less of a problem if a store took deposits, but in my area, I don't think any did that. They just ordered what they were expecting to sell based on customer pull lists and previous sales but then the books sat on the shelves