Exactly. I have a restored book, although I think if I sent it in for a reholder it would be called "conserved" now, and a green label qualified book in my collection because I either could not afford a copy as nice or just could not find another copy, period. Then there's page quality.
I went from White only, to well, ok off white to white, to cream to white, to, gasp!, all the way down to slightly brittle. I have 2 books in my collection with slightly brittle notations: a USA Comics #8 and an Archie #23. I read and reread the Archie #23 so I'm not even sure where the brittleness is. It was not apparent to me. I've never had the USA #8 out of its case but I'm told by its previous owner that it was perfectly readable. As has been said, sometimes with GA you must turn a blind eye.
Still, rust is definitely a turn off for me but, depending on the book, it would come down to price. Would anyone here turn down a complete Bat #1, in any readable condition, with rusty staples for $10,000? If money wasn't the problem, then of course not. Its all relative to your comfort level. Buy what makes you happy. If you are going to fret and sweat about it, then walk away. Otherwise, enjoy the rarity and keep your eyes open for a nicer, affordable, copy.