When you think back investment-wise, many that have been buying comics for years probably got them from spinner racks or LCS and paid cover price or less, so any money made is really "found." Owing to the general way of things, most do not have an unlimited supply of money so books were obtained to read and give pleasure. Owing to those same limited funds there might have been several pitfalls that were avoided by limiting the collectible books (speaking for myself, I might have missed out on a TMNT #1 but I also missed out on spending money on the avalanche of worthless paper that was also generated at that time). Selling books for what seemed to be a good price, only to find there was a higher ceiling further on down the road? What about other books that collectors hang on to for a little bit too long, only to find that the ship has sailed and they're best used to line birdcages? You can never really know how long to keep a collectible if you want to obtain the ever-elusive, just-out-of-reach max value. Sometimes these things just balance out.
It would annoy anyone but, as in any collection, there have been bargains found and goals reached that made you happy also. You cope by looking over the books you still have that are worth a lot to you - both in money and pleasure in collecting or reading. And if all else fails, you tell yourself that these are only material things that we're all just (basically) renting out for a while and upon reflection there are things in life that are way more valuable.