2022 CGC Grading Contest Season 1 Winter Edition (#1) Final Round 7 7

[sledgehammer]

On 2/21/2022 at 4:07 AM, zzutak said:

On 2/20/2022 at 10:41 PM, Squirrel Guy said:

My wife is suggesting we move to Florida and I could get a job grading at CGC.

An excellent idea!   

Go fund me for the moving expenses?

[CGC Mike]

On 2/20/2022 at 10:28 PM, Axe Elf said:

What would be interesting would be to see the range and distribution of responses--whether the majority of people over-graded or under-graded or if there were pretty normal distributions of misses around the correct answers.

I will do a final round results thread later today.  

[JollyComics]

On 2/20/2022 at 10:41 PM, Squirrel Guy said:

Thanks so much everyone. Thanks @CGC Mike, I imagine this has been a lot of work and time. Much appreciated. It is my first contest and I am very happy that I got to participate. I am surprised with my own results. I have been handling, selling and grading my own comics on a small level on and off for over 30 years (since I was 13). Back when I was collecting McFarlane Spidermans off of the rack. Very excited, towards the end it was getting a bit stressful. That Atom #1 with the stains was really giving me a hard time. My wife is suggesting we move to Florida and I could get a job grading at CGC and she could go to Disney and the Beach while I'm at work lol.

The mystery guy has answered!  Congrats!

[JollyComics]

On 2/21/2022 at 6:06 AM, sledgehammer said:

It's really not that large of a sample to draw conclusions. Were the better scores missing lower as much as the bottom scores.

The larger the sample size, the more that it will probably just look like a big probability curve.

I kept track of my grades, and I missed over as much as I missed under.

The only 2 times that I missed badly, by 3 grades, ( 11 bullseyes, 4 by one grade, and 3 by 2 grade not included) both times I missed under.

Well, thanks to that fake crease on UF 4.

Mine has only two bullseyes and two grossly overgraded and the rest are by 1 point.

[Axe Elf]

On 2/21/2022 at 6:06 AM, sledgehammer said:

It's really not that large of a sample to draw conclusions.

There were 2,025 grades observed.  Usually a sample size of 30 or more is considered adequate for statistical analysis.

On 2/21/2022 at 6:06 AM, sledgehammer said:

I kept track of my grades, and I missed over as much as I missed under.

The only 2 times that I missed badly, by 3 grades, ( 11 bullseyes, 4 by one grade, and 3 by 2 grade not included) both times I missed under.

Almost half of the observed grades were under-grades.  It appears that your experience would reflect the overall findings (your two worst misses were under-grades), though having just cautioned me against a small sample size, your sample size of 1 doesn't really tell us anything about the overall population.

I'm not trying to put anyone down here.  It has just been my impression that a lot of the grades given on this site err on the conservative side--and this data would bear that out.  Knowing that the bias exists can help us all get better at this.

[Point Five]

On 2/21/2022 at 11:25 AM, Axe Elf said:

There were 2,025 grades observed.  Usually a sample size of 30 or more is considered adequate for statistical analysis.

Almost half of the observed grades were under-grades.  It appears that your experience would reflect the overall findings (your two worst misses were under-grades), though having just cautioned me against a small sample size, your sample size of 1 doesn't really tell us anything about the overall population.

I'm not trying to put anyone down here.  It has just been my impression that a lot of the grades given on this site err on the conservative side--and this data would bear that out.  Knowing that the bias exists can help us all get better at this.

All good thoughts! The thing to remember though is that we're trying to guess the CGC grades, not to grade the books ourselves. And there is a difference! CGC standards have usually been more conservative than average, though they have seemingly bobbed and weaved a bit from year to year. So what seems like undergrading is (sometimes, at least) an attempt to move where CGC is or has been on a particular set of flaws.

 

[Axe Elf]

On 2/21/2022 at 10:45 AM, Point Five said:

All good thoughts! The thing to remember though is that we're trying to guess the CGC grades, not to grade the books ourselves. And there is a difference! CGC standards have usually been more conservative than average, though they have seemingly bobbed and weaved a bit from year to year. So what seems like undergrading is (sometimes, at least) an attempt to move where CGC is or has been on a particular set of flaws.

 

Excellent analysis!

So maybe the message isn't that people are being too conservative, but that people expect CGC to be too conservative...  But people are still being more conservative than CGC actually is, and the data fits the prediction I made just from eyeballing grades given on this site--that they tend to err on the conservative side in general.

Edited February 21, 2022 by Axe Elf

[grendelbo]

On 2/21/2022 at 9:45 AM, Point Five said:

The thing to remember though is that we're trying to guess the CGC grades, not to grade the books ourselves.

Exactly.

And it messes with my head sometimes. 

On 2/21/2022 at 9:50 AM, Axe Elf said:

So maybe the message isn't that people are being too conservative, but that people expect CGC to be too conservative... 

Exactly. Add a grade to whatever you think a book would be, especially mid-range and higher. Definitely if it presents very well. Absolutely if it's Golden Age. Most likely if it's a Key.

[Point Five]

On 2/21/2022 at 11:53 AM, grendelbo said:

Exactly.

And it messes with my head sometimes. 

Same! I was thinking about this too. I wonder if, for us old-timers on the boards, there's maybe a tipping point at which having detailed knowledge of the minutae of CGC standards from (say) 2009-2019 becomes more of a hindrance than a help in these contests. (Or at least gives us more fodder for second-guessing our initial thoughts.)  

And as I always say in these contests, you shouldn't beat yourself up if you can't match the impossible standard of all bullseyes. A lot of good graders here ended up in the middle or even the back of the pack. If you can look at the posted books and get even 'in the ballpark' on CGC grades for half of them, you are way, *way* ahead of the average comic grader.

 

[grendelbo]

On 2/21/2022 at 10:07 AM, Point Five said:

And as I always say in these contests, you shouldn't beat yourself up if you can't match the impossible standard of all bullseyes. A lot of good graders here ended up in the middle or even the back of the pack. If you can look at the posted books and get even 'in the ballpark' on CGC grades for half of them, you are way, *way* ahead of the average comic grader.

Exactly.

Bulleyes are great then I can .

But if I'm within a grade off what CGC says, or even two grades off, I feel good about my grading skills. 

We all know a book graded let's say 8.0 on a Friday could be a 7.5 on a Monday and an 8.5 if it's an important book.

Heck, I have even regraded books that were posted on PGM and that I commented on. I regrade the same only about 3/4 of the time.

[CGC Mike]

Here's the results page for the final round.  Look for the spring edition contest in a few months.

 

[sledgehammer]

On 2/21/2022 at 10:25 AM, Axe Elf said:

There were 2,025 grades observed.  Usually a sample size of 30 or more is considered adequate for statistical analysis.

Almost half of the observed grades were under-grades.  It appears that your experience would reflect the overall findings (your two worst misses were under-grades), though having just cautioned me against a small sample size, your sample size of 1 doesn't really tell us anything about the overall population.

I'm not trying to put anyone down here.  It has just been my impression that a lot of the grades given on this site err on the conservative side--and this data would bear that out.  Knowing that the bias exists can help us all get better at this.

Honestly, sorry to bother you with my thoughts.

I didn't notice that you had been here since Friday, and that one of your first posts was that you hadn't graded a comic book before two weeks ago.

I'm sure that you're the guy to run thoughts about a change in grading strategy by before the next contest.

 

[sledgehammer]

On 2/21/2022 at 8:55 AM, JollyComics said:

On 2/21/2022 at 6:06 AM, sledgehammer said:

It's really not that large of a sample to draw conclusions. Were the better scores missing lower as much as the bottom scores.

The larger the sample size, the more that it will probably just look like a big probability curve.

I kept track of my grades, and I missed over as much as I missed under.

The only 2 times that I missed badly, by 3 grades, ( 11 bullseyes, 4 by one grade, and 3 by 2 grade not included) both times I missed under.

Well, thanks to that fake crease on UF 4.

Mine has only two bullseyes and two grossly overgraded and the rest are by 1 point.

One guy had 5 points after the first book, and finished round 3 with 14 points.

Looking forward to the spring contest.

I will donate a prize.

[Axe Elf]

On 2/21/2022 at 2:58 PM, sledgehammer said:

Honestly, sorry to bother you with my thoughts.

I didn't notice that you had been here since Friday, and that one of your first posts was that you hadn't graded a comic book before two weeks ago.

I'm sure that you're the guy to run thoughts about a change in grading strategy by before the next contest.

 

If you're suggesting that I'm not an expert in grading comics, you'll get no argument from me.

I do, however, have an advanced degree in neuropsychology, so I have some insight into how people's biases affect their decisions, and I'm pretty good at analyzing a set of observational data, like the distribution of grades given in the grading contest.  So I can say with some authority that the distribution of grades is not normal, but rather skewed toward under-grading.

Now, what that analysis MEANS is going to require input from those of you with more grading experience.  It was informative, for instance, to recognize that some contestants may not be giving their own grading analysis, but rather what they expect to be the CGC grading analysis--so more studies would have to be done to control for the effect of grading expectations on base grading skill--but for whatever reason, more people are currently under-grading comics than are over-grading them, which is consistent with what my newbie eye observes on the PGM board as well (which should remove the effect of CGC expectations that could be present in the contest).

Individually, your results may vary.  Even if the community as a whole under-grades comics, you may still be an egregious and consistent over-grader; the cumulative results can't speak to any one participant.

Edited February 21, 2022 by Axe Elf

[CGC Mike]

Hey all:

I have started a thread here seeking feedback for any changes you would like to see happen in future grading contests.