Speculators happen. There are good speculators and bad speculators (the ones Fastball mentioned that lie and create websites in order to manipulate the market to their own ends). In the end though, for speculators to survive, collectors must outnumber them by a decent ratio. If the ratio drops or, god forbid, speculators outnumber collectors, then speculators are in for a shock. Right now, I think there are still more collectors than speculators but the ratio has shrunk significantly over the last few years. in an oversimplified nutshell, the 90's crash happened because the speculator dollar vastly outnumbered the collector dollar.
The relationship between speculators and collectors can be mutually beneficial, like a symbiotic relationship but if the balance is thrown off, it can become parasitic. Collectors are the host body, and the host can only maintain a certain number of parasites before it becomes ill. I don't think we are there yet, but speculators seem to multiply faster than collectors, like weeds versus an old growth forrest. This analogy may be offensive to some, but I don't mean it to be because I don't have a problem with ethical speculators.
Oh, and a guy that sells a GA book and upgrades is not necessarily a speculator. Collectors have long preferred high grade books, hence the demand. I think you can trace the multiples of guide value for high grade books to the Edgar Church collection. Many of those books have not changed hands since Chuck sold them. The people that bought those books were and still are old growth collectors.