CGC Grading Contest Awards Based on Cumulative Performance Through Contest #6 4 4

[Superman2006]

On 5/13/2023 at 8:15 PM, CGC Mike said:

Great thread, @Superman2006

Thanks, Mike 

[DrMitchJ]

I don't understand anything that is going on here, can someone explain the contest?? 

[Superman2006]

On 5/13/2023 at 8:24 PM, ADAMANTIUM said:

On 5/13/2023 at 7:58 PM, Axelrod said:

31st eh?

Expand  

 at least you made "a list" haha I'll see myself out  

You're in some lists, just not the ones published in my first post 

You are in the group of 99 participants that qualified for the lists in my first post above by completing 2.75+ contest. In your case, as you mentioned, you have participated in all the contests. In fact, you are one of just 14 graders that has submitted grades for every round so far (ignoring the 4th round of the first contest for those that didn't qualify for that round).  I missed the signup deadline for the first contest by a day or two, or I'd be in that club too. 

Perfect Attendance Group of 14:

jbpez

Motor City Rob

WilliamLunt

Point Five

zzutak

grendelbo

pastandpresentcomics

silverseeker

Axelrod

davidtere

comicginger1789

jas1vans

musicmeta

ADAMANTIUM

[Superman2006]

On 5/13/2023 at 9:11 PM, DrMitchJ said:

I don't understand anything that is going on here, can someone explain the contest?? 

Hey DrMitchJ,

See this thread for CGC Mike's explanation of contest #6 that just wrapped up a week or so ago:

As Mike explains in the thread above, the top 3 winners of each contest win prize money from CGC.

This thread that you're reading now just provides some additional stats for those that have partaken in multiple contests to date.

I believe CGC Mike starts threads in the Comics General forum and in this "Hey buddy, can you spare a grade?" forum a few weeks or so before the start of each contest, so keep any eye out for Mike's advance notice thread(s), and join in the fun in the next contest!

 

 

[TheGeneral]

Awesome recap, thanks for the analysis! @Superman2006

I thought there have only been 5 contests? Was the very first one a different format or limited entry or something? Never mind, I took a look back. Missed the first contest. Also got confused because I think I've done 5, but it says 4 on your score breakdown.

Edited May 14, 2023 by TheGeneral

[Superman2006]

On 5/13/2023 at 11:29 PM, TheGeneral said:

Awesome recap, thanks for the analysis! @Superman2006

I thought there have only been 5 contests? Was the very first one a different format or limited entry or something? Never mind, I took a look back. Missed the first contest. Also got confused because I think I've done 5, but it says 4 on your score breakdown.

Thanks General!

I took a look back and you are correct; you participated in contests 2 through 6. For contest #3 the results showed you as "The General" rather than "TheGeneral", and when I sorted by name before finalizing my spreadsheet, there were a couple of other board names between those two names, so I didn't catch that (I did catch 6 other such variances between contests; hopefully I was 6 for 7 overall in catching such typos). I will fix that in my spreadsheet, but it won't really impact your ranking or average score, as you averaged 17.8 in the 4 contests where you were listed as "TheGeneral" vs. 18.0 in the 1 contest (contest #3) where you were listed as "The General".

Edited May 14, 2023 by Superman2006
typo

[Superman2006]

@TheGeneral I fixed it in my spreadsheet (your average rounded to one decimal place is still 17.8), so it should be good to go for any further analysis / future contest updates. 

[Superman2006]

One tool that can be used to improve one's CGC grading contest results is analyze your scores to see if you're consistently undergrade or overgrade compared to CGC. zzutak has mentioned this concept in some of his past posts: "So, just for kicks, recalculate your total contest score using "signed deltas": -2 for your guess being 2 grade increments too low, -1 for your guess being 1 grade increment too low, 0 for a bullseye, +1 for your guess being 1 grade increment too high, +2 for your guess being 2 grade increments too high, and so on.  What's your cumulative 20-book tally now?"

In other words, if you graded a book an 8, and CGC graded it a 9, you would have a signed delta score of -2 score for that book. Now if you graded another book a 4.0, and CGC graded it a 3.0, then you would have a signed delta score of +2 for that book. If you summed up your results across those 2 books, you would have a total signed delta score of -2 + +2 = 0, meaning that on average, you didn't undergrade or overgrade books relative to CGC.

Here are the signed delta results for the award winners listed in my first post:

Amongst the top 10 ranked participants:

EastEnd1 has an average signed delta score of -0.2, meaning on average he just slightly undergrades compared to CGC.

jbpez has an average signed delta score of -7.2, meaning he tends to undergrade compared to CGC. This means that you'll want to buy raw books from jbpez, lol. This also means that it is possible that jbpez could improve his already impressive #3 ranking if he loosened up on his grades a tab in future CGC grading contests.

Get Marwood & I has an average signed delta score of 4.0, meaning that he tends to overgrade a bit compared to CGC during grading contests. Keep in mind that this is an averaged signed delta score per 20 book contest, so it's really not that high, and it translates to just a 0.2 averaged signed delta score per book, and with most grading increments equal of 0.5 (or less when you get into grades in the 9's), this has very little impact on the average grade assigned, so I'd caution anyone against over compensating when assigning grades in future grading contests if you have a relatively small average signed delta score.

There are average signed delta scores close to zero amongst those in the top 10% and in the bottom 10% of average scores, so an average signed delta score close to zero doesn't necessarily translate to being an accurate overall grader, but if your average signed delta score is a big positive, or a big negative number, it might be worth adjusting your future scores a bit.

I'll probably post everyone's signed delta score at some point (sorted alphabetically, without overall score) in case anyone else wants to see how much they tend to overgrade or undergrade on average.

[Superman2006]

If someone has experience successfully copying and pasting from Excel into a boards message, please let me know; the formatting gets all messed up on me when I try to do it (hence the screen shots that I've been including in my earlier posts).

Thanks : )

[CJ Design]

On 5/13/2023 at 4:15 PM, Superman2006 said:

@Point Five @JollyComics @zzutak @Frederic9494 @MBFan @grendelbo @Morganmi @pastandpresentcomics @mytastebud @Yorick @JohnH19 @Warlord @batcollector @flashlites @PINWHEEL @Mars76 @silverseeker @JWKyle @CJ Design @thehumantorch @Axelrod @universal soldier @Hudson @Kramerica @davidtere @JustJimN @sledgehammer @Old Fashion PB and J @AJD @Mr. Zipper @KryptoMayor @stynxno @Cerebus3000 @scburdet @comicginger1789 @Hibou @toro @skypinkblu @Wipple @mysterymachine 

Special thanks to @CGC Mike for the great contests. Great job working with all the data too; I didn't have to make very many data tweaks in order to pull this all together (most tweaks were pretty minor like the occasional typo on boardie names in the results summaries, that I could easily identify and fix).

If anyone else not included in the top 50 rankings above would like their score, just post in this thread, and I'll reply in the thread with your score and contests completed (or PM me and I'll PM your score and contests completed to you).

Thanks For doing this..cool to see...I'm in the middle of the pack...but hey, glad I'm even in the pack. Being with such esteemed company that is.

[sledgehammer]

I've been going downhill.  I'm blaming my eyes.

I played along for the first contest, and ended up with what would have been 5 points behind Squirrel Guy, at third place with 16 points (11 bullseyes).

Finished 3rd with 20 points once, but that feels like eons ago.

Thanks for the info!

 

 

[Superman2006]

After each contest CGC posts the grade distribution from all graders. zzutak then analyzes the CGC distribution and provides various grade distribution statistic (including the mean, median, and mode), and shares a bunch of other interesting commentary.  Based on the grade distribution info from CGC and grade distribution statisics from zzutak, there are some books where the average board grade is off by one or two increments from the CGC grade. Which is the more accurate grade in those cases, the grade assigned by 1 (or 2 or 3?) CGC graders, or the average grade assigned by a hundred or so boardies?  Perhaps the CGC grade would approach the boardie average grade if CGC took that average grade from a higher number of graders. Due to economics, one can't really fault CGC for not doing so. Also, the more CGC graders physically handling a book, rather than simply grading from scans like we're doing the greater the risk of handling damage to book.

With that said, I used my spreadsheet to calculate the mean grade for every in the contest, and then rounded that grade to the nearest actual CGC grade(e.g. if the average boardie grade was 1.6, I rounded that to a grade of 1.5, since 1.6 is closer to 1.5 than 1.8). I then measured everyone's performance against the average boardie grade and came up with the following results:

Under this ranking approach, jbpez is the overall champ by a wide margin, with an average score of just 12.8!!!

Congrats @jbpez !!!

[Superman2006]

While preparing the summary above, it hit me that maybe an even more accurate way to measure outstanding boardie grading performance (at least in my mind) might be to look at the average scores assigned by some number of top graders as measured by some preliminary criteria (I used the 50 award winners from my first post of this thread) in order to assign the "average top ranked boardies grade" for each book.

I then recalculated all boardies scores against the "average top ranked boardies grade" across all contests in order to come up with the following results:

Using this approach, jbpez once again ranks first overall with an average grade of just 12.5!

TheGeneral and Electron follow fairly close behind with average scores of just 13.8 and 14.8 respectively!

Although not shown in the summary above, just for kicks, I came up with a score for the CGC grader(s) across all books graded over the 6 contests, and the CGC grader(s) had a very impressive average of 14.0, which is just a bit behind the average score for jbpez and TheGeneral, when measured against the "average top ranked boardies grade".

Edited May 16, 2023 by Superman2006
a few less words

[Superman2006]

The rankings based on collective averages in my posts above follow Aristotle's "wisdom of the crowd" concept, whereby "large groups of people are collectively smarter than individual experts..."

[Superman2006]

Here is my last planned ranking summary through contest #6 (I may post some other analysis at some point, such as most improved).

This post just provides a consolidated view of the various rankings that I analyzed, sorted in order of the CGC ranking from my first post in this thread.

The first two screenshots below show the same top 50 as in my first post. The 3rd screenshot below shows some additional contestants that are in the top 50 based on one the two alternative "wisdom of the crowd" ranking systems that I described in my posts above.

Congrats to @spidermanbeyond @Black_Adam @Rumler @jcjames @frozentundraguy @PeterPark @Ron C. @Stefan_W and @Gman65NJ for top 50 grader finishes based on the wisdom of the crowd ranking systems shown in the last 2 columns below (and described in more detail in my posts above)! 

Edited May 16, 2023 by Superman2006
formatting fix

[Superman2006]

Just a reminder that by only listing the top 50 ranked graders according to the requirements outlined in my posts, my intent isn't to exclude anyone from the rankings. I just don't want to anyone near the bottom of the rankings to feel bad about their average scores, so I cut off the rankings pretty high up (i.e. just the top 50 out of the 99 participants that have competed in 2.75 rounds or more out of the first 6 contests).

For anyone else that's interested in their scores outside of the top 50 (whether you've competed 2.75 rounds, or not), don't hesitate to shoot me a PM, and I'll share your scores, ranking, etc. with you via PM.

[Squirrel Guy]

On 5/13/2023 at 5:03 PM, Superman2006 said:

There are different ways that one could slice and dice the grading contest results over the last 6 grading contests to come up with cumulative rankings. The approach I took was to come up was to rank all participants based on their average score over 4 rounds.

If all 6 contests to date allowed everyone to participate in all rounds, I would have set the minimum participation level equal to 3 contests worth of participation, however, given the limitations of the 1st contest, I decided to set the minimum participation level equal to 2.75 contests in order to qualify for the various award categories below.

Without further adieu, following are the "MVP", "Diamond Club", and "Honor Roll" award winners based on performance through the first 6 CGC grading contests:

 

 

 

 

You'll notice that in the rankings above, I also show the ranking based on an alternative ranking approach shared by zzutak in another thread, where zzutak applied the following ranking approach that only gives weight to top 10 finishers: "... I decided to compare the Top-10 finishers in each of the first six CGC Quarterly Grading Contests. I awarded 10 pts for a 1st Place, 9 pts for a 2nd Place, and so on (down to 1 pt for a 10th Place).  When there were multiple-player ties, I awarded each player in the tied group the number of points corresponding to the best place occupied.  For example: a three-player tie for 4th-6th Place would earn each player 7 points; a four-player tie for 6th-9th Place would earn each player 5 points; a five-player tie for 8th-12th Place would earn each player 3 points...". The results from zzutak's points ranking system are generally in line with the more inclusive (and data intensive) ranking system that I applied (especially for the top 10 ranked finishers, as one would expect). 

 

Congrats to Squirrel Guy and all other award winners!

 

A few additional thoughts based on contest results through the first 6 rounds:

1. 391 participants have submitted grades for 1 or more rounds

2. 99 participants have submitted grades for 2.75+ contests, and thus qualified for the awards above. The average grade for those 99 participants ranges from 16.7 points per contest (i.e. per 4 rounds) for Squirrel Guy to 46.7 points per round for the 99th ranked qualifying participant.

3. To summarize, the average grades at or around the bounds of each award (and non-award) category: for the 1st, 11th, 51st, and 99th qualifying participants, the average grades were 16.7, 22.3, 29.4, and 46.7.

 

spoon, was there another contest I missed? I appreciate the hard work. I definitely need to do the next one.

[Superman2006]

On 5/19/2023 at 12:54 AM, Squirrel Guy said:

spoon, was there another contest I missed? I appreciate the hard work. I definitely need to do the next one.

Ha ha, yeah, be sure to join the next one, so you can retain your MVP title!

I'll probably increase the minimum number of contests to qualify for the updated rankings through contest #7 to 3.25 contests (Not everyone could participate in the 4th round of the 1st contest, so the max contests most participants could have participated in through contest #7 will be 6.75, and 6.75 / 2 = 3.375, which rounds down to 3.25 contests).

[DrMitchJ]

Still, this is about as confusing as calculus ... even with the attempts to explain it ... appreciated the attempts all the same. 

I have more books to list on my registry but not enough time to do so. 

[CGC Mike]

On 5/19/2023 at 12:54 AM, Squirrel Guy said:

spoon, was there another contest I missed? I appreciate the hard work. I definitely need to do the next one.

Signups for the contests should start:

January 1st

April 1st

July 1st

October 1st