Subjectivity of Grading and Deciding what to Accept

[Guest]

thats a good price on those Ricky....Id buy them if I were into Transformers. Hows about sellin me the Joe #12??

[ArnoldTBlumberg]

"However, you're right in that this example contradicts the chart on page 126 which indicates that a NM- 9.2 comic would have no more than 4 defects."

 

Ah, ah, but remember - every quantitative bit of info in there, from the numbers I suggested for our vague terminology (minor, moderate) to that allowable defects chart - is a suggestion, NOT a hard and fast rule. That chart was simply meant to finally provide a more visual and detailed look at how the number of defects MIGHT break down from one grade to the next. Doesn't mean it always works that way.

 

By the way, having been on these boards for a while - you DO put aside all this grading stuff and just relax now and again, right? I can't remember ever caring this much about the condition of anything I bought. You guys are so dedicated, wears me out just reading about it .

 

Arnold

[PovertyRow]

I can't remember ever caring this much about the condition of anything I bought.

 

Well, over 20 years ago that sentiment would have been true. Nowadays? When the diff between a VG+ and a Fine can mean 25% or more? You better start caring!

[fantastic_four]

Ah, ah, but remember - every quantitative bit of info in there, from the numbers I suggested for our vague terminology (minor, moderate) to that allowable defects chart - is a suggestion, NOT a hard and fast rule. That chart was simply meant to finally provide a more visual and detailed look at how the number of defects MIGHT break down from one grade to the next. Doesn't mean it always works that way.

Then why use numbers, and why use a chart? That's science. Every five or ten years Overstreet has been getting more and more precise, using more and more exacting numbers and descriptions to describe grade, becoming more and more scientific, trying more and more to polish the standards he has helped create...yet at the same time, he excuses what I consider to be the youth of grading (especially when you compare it to evaluation of older media such as artwork, stamps, and coins) by saying that "no absolute standard is possible." Well that's true--aesthetics are whatever we all decide them to be--but there is a considerable amount of science behind aesthetics, and much more to come. And it's very pessimistic! It's not so much that a standard isn't practical, it's more that for a hobby like comic book collecting, nobody kills themselves over it. We all just pluck a few new cherries from the tree of science every decade or so and apply them to this fun little pasttime. And why not--it's supposed to be leisure! It's hard to ditch the simplistic attitude we approach hobbies with, apparently even when you're making your living off of that hobby.

 

There is a limited set of defects that paper commonly displays. It's a large set, but still limited. There is also a limited surface area on the exterior and interior of a comic. Defects can be both counted and individually measured. Numerical methods can be applied to anything that can be counted and measured. I see experienced graders do this in their heads--take both the count and severity of defects into account simultaneously--and it's something that the vast, vast majority of beginning graders don't do. It's what's missing from that one-dimensional chart on page 126; that chart tracks defect counts, but it's missing the dimension of defect severity. Which is why it's only useful as a suggestion at the present time. It points us in the right direction, but there's still more to be improved on the road ahead.

 

I'm really glad that you guys finally began to standardize defect severities for the grades--like I said before, it's the biggest published advance to grading since CGC brought tight grading to the high-end masses.

 

By the way, having been on these boards for a while - you DO put aside all this grading stuff and just relax now and again, right?

WHAT? Of course not! I spend every waking hour evaluating the precise condition of every physical object around me, OF COURSE! I identify and appreciate the quality of every single molecule I come in contact with, don't you?

 

Talking about grading and restoration is the main reason I continue coming to this web site. A lot of people use this forum as 2/3 entertainment and 1/3 learning...I use it for 2/3 learning and 1/3 entertainment, which is why my tone is serious in my messages more often than not. When my girlfriend wants me to stop reading messages, she tells me to "stop working." Writing develops your own thought process and draws ideas out of others.

[ArnoldTBlumberg]

Since I haven't bought a back issue in about ten years, it's a moot point. I long since tired of collecting comics to be honest. I read new stuff but I'm nowhere near the financial level I would need to be to collect the Spider-Man issues I'm missing. So I just stopped and moved onto other things. And the more I write about comics, the less I mind not collecting them .

[ArnoldTBlumberg]

We put in numbers for a simple reason - you always say you want more detail! And frankly, I thought it would be fun to see if I could come up with numbers that fit, and surprisingly, they seemed to make a sort of sense. But I adhere strongly to the belief that while you can claim to apply scientific principles to grading, it will forever remain a subjective art that cannot be truly quantified. However, with *suggested* numerical guides, you can approximate something resembling a consensus among most participants in the collecting game.

[fantastic_four]

Why so pessimistic about the possibility of an exacting grading standard?

 

Did you ever talk much to that guy Gifflefunk on the EBay boards? He claimed he had written that automated grading software after spending around 6 weeks reverse-engineering the defect types and severities from the 1992 edition of the Overstreet Grading guide. He claimed that it worked rather well for grades from Fine to Mint, but that he had trouble getting his linear equations to work right below the VG level. He said he kept tweaking his equation until it worked for all the comics in that guide, and that he found it worked rather well for other comics also. I believe what you did as a user was to manually enter all the defects and severities into the software, and it spit out a grade for you.

 

I fully believe this type of software is possible, although it obviously remains to be seen how accurate it could be. I've already seen a demonstration of software that assigns a 10-point beauty grade to the human face in a totally automated way by using image processing to weigh metrics such as size and proportional placement of eyes, nose, mouth, ears, hairline, etc.

[C3Galaxia-migration]

Well, I have to agree that I treat grading as a mixture of art and science. The art being the procedure of applying a defined set of qualitative standards to arive at a quantitative result. I for one find that the hard part about grading isn't finding the defects, but trying to categorize and nail down a specific grade. It's not hard for me to obtain a reasonable grade range (i.e. 8.5-9.0 or 9.0-9.4) but trying to determine the exact grade is where the art comes into play the most. I usually try to keep myself conservative so I will automatically steer to a lower side of a range, especially if my gut feeling tells me to.

 

I for one appreciate everything that the Overstreet 2003 guide has provided because now it gives me a more standard set of standards to apply the art of grading. And the pictures make it soo much easier and so far it has helped in nailing down a few grades! Thanks, Arnoldt for your time and dedication!

It takes dedication to deal with the most anal of this hobby.

 

fantastic_four, I would love to try out grading software just to see how accurate it is. Also, from the perspective of a Software Engineer, writting a piece of software to accomplish this would not be a monumental task but it wouldn't be trivial either. A ton of QC would be needed and evaluation from the hobby experts. I think it would be a good tool for an estimate at best. And I also come to these boards for the more educational aspect, but recently there has been some funny "stuff" going on...

 

Dungeon, the only GI Joes I have are the Transformers/GI Joe crossover mini. And I already have a copy of 76 and 77 that I am happy with. I do not own any CGC graded comics and am happy with keeping my treasured comics mylared.

 

[awe4one]

I believe what you did as a user was to manually enter all the defects and severities into the software, and it spit out a grade for you.

 

Kind of takes out some of the fun of collecting comics doesn't it? Seems too clinical and uninvolving to just put in numbers and wait for the condition of your comic to pop out.

 

At least with CGC, we can argue that the 9.2 really looks like a 9.0 or that 7.0 should be a 9.0 if some stuff wasn't stuck to the back. Letting a machine decide for something as subjective as the condition of a comic (and everyone has at least small differing opinions on condition/no two people have exact opinions on the subject) is the kind of "advancement" that would be more of a detrement to the hobby than an enhancement.

 

 

Jim

[ArnoldTBlumberg]

Yeah, Gifflefunk provided some much needed feedback.

 

I'm pessimistic for two major reasons. One is that I'm a naturally cynical [!@#%^&^] , the other is that grading comics is NOT a science - it's a highly subjective evaluation of something that by definition MUST involve an emotional response. Collecting is not a clinical pursuit - it's a psychologically ingrained passion. You can construct the finest evaluation software imaginable, but you will never be able to remove the human factor from grading. Add to that the fact that there is simply no way to construct a finite list of defects - tears alone would go on forever. Try tearing a piece of paper precisely the same way twice, look at the resulting fiber pull, line of the tear, length, resulting creasing, etc. - it can't be done. No two tears are alike...ever.

 

Basically I think we can create the *illusion* of exactitude by making up lists of these things and having everyone agree on them. But that's all.

 

Read more in my next book, ZEN AND THE ART OF COLLECTING COMICS...just kidding.

[tb]

FF wrote:

 

He claimed that it worked rather well for grades from Fine to Mint, but that he had trouble getting his linear equations to work right below the VG level. He said he kept tweaking his equation until it worked for all the comics in that guide, and that he found it worked rather well for other comics also. I believe what you did as a user was to manually enter all the defects and severities into the software, and it spit out a grade for you.

 

I fully believe this type of software is possible, although it obviously remains to be seen how accurate it could be.

 

---

 

Interesting thought . I've spent most of my career doing research in processing of 2d and 3d data sets and am just writing up a publication on feature extraction from models acquired by a 3d laser scanner. There is a large body of literature on automatic identification of features in image processing, but IMO more interestingly, researchers are starting to integrate geometric (3d) info. into the feature analysis. With both geometry and texture hints available, you can do a much better job than purely working with images. How this might be applied to comics is kind of interesting. 3d scanners are accurate enough that you could detect creases, scuffs, and other "geometric" defects automatically. There is also a lot of research in reflection models that could be applied; the basic idea is to shine light at a book from different angles and measure the reflected light. I suspect a lot of defects could be inferred this way alone (this is the only way that I can think of that might possibly detect dirt and cleaning). By combining the 3 methods (color, geometry, and reflection), I suspect such a system could do a pretty ok job at the feature detection, although nowhere near as any human with the most rudimentary knowledge of grading. Then, you would have to do a mapping from the features to a numerical scale; sounds like a pretty straightforward job for neural networks trained on a database of CGC graded comics.

 

All this is academic, of course: the technology is just not quite up to it yet. Quality assurance in just about every type of manufacturing (from toys to cars and airplanes) involves an inspection that conceptually is very similiar to comic book grading: you measure the part with a 3d scanner and compare it to an ideal 3d computer model. Although this inspection is a lot easier to quantify than the comic book grading process, there is no automated software available. If you could come up with a system such as the one you describe, you would very quickly become a billionaire.

 

 

 

 

 

 

[fantastic_four]

Collecting is not a clinical pursuit - it's a psychologically ingrained passion.

Collecting is, yes, but grading shouldn't be. That's the problem with most graders--they let their personal attachment and bias cloud their objectivity in grading. Lacking an exact grading standard, the bias of evaluating defects without objective reasoning is prevalent; luckily, the Overstreet Grading Guide is starting to set that exacting grading standard.

 

You can construct the finest evaluation software imaginable, but you will never be able to remove the human factor from grading.

You're right, depending upon what you mean. If you mean that the idea of what beauty is can only be determined by humans and not computers, then you're right; a Near Mint comic is whatever we humans think it should be. That's the art inherent in grading; beauty is in the eye of the beholder, and we beholders are human. If you mean that a computer will never be able to consistently determine a grade given a set of defects, then you're wrong; computers already do that for other types of products every single hour of every single day. Automated quality assurance testing has been growing and growing for decades; nobody's applied it to comics yet because there isn't enough money in it.

 

Add to that the fact that there is simply no way to construct a finite list of defects - tears alone would go on forever. Try tearing a piece of paper precisely the same way twice, look at the resulting fiber pull, line of the tear, length, resulting creasing, etc. - it can't be done. No two tears are alike...ever.

There are no two defect instances that are the same, no. The universe is chaotic and infinitely complex, yes. But we're talking about human aesthetics, here; we can only evaluate so many criteria. There are an infinite number of defect instances, but there aren't an infinite number of defect attributes that we'll always take into account when grading. No two tears are alike, no. But they do have attributes in common--they have a length, they have angularity (some tears are jagged, some straight and clean), and they could have dozens of other attributes that we could measure if we really wanted to. But since grading is an art, we won't go through every scientific measurement on a defect attribute that could ever be collected; humans will only focus on the ones that detract from aesthetic appeal. And we all make that up based upon the way the objects make us feel.

 

Basically I think we can create the *illusion* of exactitude by making up lists of these things and having everyone agree on them. But that's all.

Well, yea, aesthetics are an illusion, but once somebody teaches us what "beauty" really is--whether it be an impressionist painting or a defect-free comic book--the criteria used for measuring aesthetic beauty are based upon real, tangible physical attributes, not illusion. Visual beauty is based upon physical critera. Physical criteria can be measured. Is the "perfection" of that criteria an illusion? Yea, it is. Most people date and marry within their own race because they find the physical characteristics (skin color, size of nose, size of eyes, cheek structure, cranial structure, etc) of people from other races to be "displeasing" to the set of aesthetics they grew up with in their household. Does that mean only people of their race are "beautiful"? No; beauty is learned. But once the criteria for beauty become established, they're measurable. It's not all smoke and mirrors. People don't always think supermodels are hot simply because other people told them so; they learn to gauge physical features and apply them to new instances.

 

Since I was snowed in for a few hours today, I started reading the EBay board again. I may be posting less here because it looks like the grading discussions over there have really progressed over the last few months. Do you know what the deal is with that "Longbox Grader's Association" I saw a few people mention?

[fantastic_four]

A ton of QC would be needed and evaluation from the hobby experts. I think it would be a good tool for an estimate at best.

Yea, you're right, a TON of feedback would be needed. It would be a good tool for an estimate only until the hobby identifies a larger set of standard defect attributes, but once you do that, the amount of time it takes to manually enter all that data in becomes time prohibitive. But then again, any grade determined by a human is only an estimate also when using the pre-2003 Overstreet Grading Guide criteria. I'm not familiar with all the new criteria in the 2003 Grading Guide to make a call on how the current set of defect categories and attributes are; haven't pored over the individual grade descriptions yet.

 

Unless computer vision gets much, much better to allow defect discovery to be automated, grading software would only likely be useful to help learn grading or to act as a second opinion on expensive books.

[fantastic_four]

If you could come up with a system such as the one you describe, you would very quickly become a billionaire.

Nah, there's not enough money in comic book grading to go through the expense of developing that much technology particular to it. CGC does pro grading, and I don't think they'll end up correcting me if I say that they're many, many decades off from earning anywhere close to a billion dollars in revenue, much less profits.

[fantastic_four]

Kind of takes out some of the fun of collecting comics doesn't it?

I dunno, I don't find grading to be fun. I like hunting for comics and buying them, but the process of grading makes my head hurt when I do it all day at conventions. I just like to learn grading so I don't get ripped off! And because I recognize it as being in its youth; I'm a curious person and like pushing the envelope in areas where it needs to be pushed.

[tb]

> I don't think they'll end up correcting me if I say that they're many, many decades off from earning anywhere close to a billion dollars in revenue, much less profits.

 

---

 

With the 10^9, I was referring to an industrial strength automatic inspection package. A comic grading system would be much harder and worth much, much less. Would be a fun project, though.

 

[ArnoldTBlumberg]

Back when I enjoyed collecting comics, my grading criteria was simple - if it looked nice, I bought it, and if it was banged up so much that I didn't like it, I didn't buy it. I read the Price Guide every year, kept up on what was going on in comics, even looked up some of my stuff from time to time to see what it was worth - and never did I bother reading the grading section or look at any comic I bought for "staple stress lines" or "color break." I just looked at the whole thing and if I liked it, that was that. I was happy enough with that system.

 

Of course, whenever I found a Marvel Value Stamp missing, I knew I had to replace that issue. Brackin-frazzin-razzin MVSes...

[fantastic_four]

The rise in attention to grading has risen right along with the prices collectors are paying for these older issues. Popular demand for scarce goods leads to competition...competition leads to money...and money leads to the Dark Side of the comic-book-grading-Force. Ah, we adult males, our disposable incomes, and our massive egos! It makes things so complex.

 

Part of the reason I started collecting high grade was to artificially extend my collecting. I like the Silver Age Marvels, and I began to realize that with E-Bay, I could have a complete mid-grade set within a few months. So I said to myself, "I'll just buy VF+ or better, that'll make it more challenging." After I got half of my runs complete buying VF+ or better, the end of those runs was in sight, so I tightened up and said, "I'll just buy VF/NM or better now, that'll make the hunt last longer." And now I'm paying multiples of Overstreet for 9.4s and 9.6s; don't think I'll ever finish THOSE runs! I'll be collecting forever!

[tb]

FF: Part of the reason...

 

---

 

Agreed: it's easy and quick to gather a pile of junk of any particular run - much more fun and challenging to put together a high grade set as a long term project. I've had to put my own threshold around FN/VF in order to be able to find any of the pre-Barks Disney's that I collect. I've now hit a point where it is so rare that I find anything worth upgrading that my collection has basically come to a halt. That's why I'm looking to start on an entirely different series (More Fun and/or Adventure). I envy you ASM collectors a bit: it seems that you can always find nicer copies if you want to pay the price - that keeps raising the bar as you are describing. With GA, there's just very little you can do except wait and hope... Hopefully putting high quality scans of my entire collection on the web will help bring me in contact with more people who are willing to sell.

[COI]

"Part of the reason I started collecting high grade was to artificially extend my collecting. I like the Silver Age Marvels, and I began to realize that with E-Bay, I could have a complete mid-grade set within a few months. So I said to myself, "I'll just buy VF+ or better, that'll make it more challenging." After I got half of my runs complete buying VF+ or better, the end of those runs was in sight, so I tightened up and said, "I'll just buy VF/NM or better now, that'll make the hunt last longer." And now I'm paying multiples of Overstreet for 9.4s and 9.6s; don't think I'll ever finish THOSE runs! I'll be collecting forever! "

 

The concept of the imossible goal is one of the many ironic characteristics of human nature. You can only relish an acomplishment for so long before you begin looking for the next challenge. This is simutaneously the reason for getting up in the morning, and the source of unhapiness and frustration for many people.

 

I lift weights, and when I was a teenager my goal was to get to a solid 175lbs. When I acomplished that, I wanted to gain another 20 lbs, and so on...... and it seems there is no end in sight.

 

As for comics, I'm in the same boat as FF. But now, I find myself wanting to own multiple copies of keys in every grade from VF+ to NM+, as well as runs of my favorite titles in similar grades. I've set up a collecting goal that seems to have no end.