Claimed:The Grading Problem

[probably not noah 346]

Study: The VHL's Grading Problem

 

The VHL has a grading problem.

 

There is a massive discrepancy between graphics grades and writing grades. This has brought on tension in the community. Many graphics makers feel that the criteria for a six is far too high, and they are unfairly losing marks.

 

As a red team mission, I created a quick study of the grading trends since the start of my discovery. The data can be found here, and my findings follow.

 

Graphics Grading:

 

Fifty-two graphics have been collected on 27 sigs since the start of February. Of these grades, only three sigs have been awarded a full six. Each of these sixes was awarded to a different member by a different grader. Below are the three sigs that have received full marks since the beginning of the month:
 

 

The vast majority of grades fall between 5.5 and 5.75, with the average reaching 5.48. This means that on average, a graphic posted in the last 2 weeks did not reach a final grade of six. Below is a plot of the frequency of grade occurrence. There was no distinct pattern in the grades, but instead a large imbalance in grade distribution.

 

 

Fifty-two grades is a relatively small sample size, but it still illustrates what many of the graphics community seem to be upset about. There is an incredibly high number of 5.75s given out, and a much smaller rate of full sixes. This was especially true in the grading of 8Ovechkin8. He gave out 18 grades in this time period, eight of which were 5.75. Five of Sherriflobo’s grades were also 5.75. Below is a plot of each grader’s own grade distribution.

 

 

Media Spots

 

I planned to plot the information of media spot grading as well, and compare data points and charts. However, when I collected all of the data from the media spots, I learnt that every single media grade given so far this month has been a six. The number of point tasks graded is consistent with graphics, as 24 media spots have been graded, yet not a single one has been less than a perfect six. This also means that the grading sample size is cut in half because a six doesn’t warrant another grade.

 

Conclusions

 

In my research, I came to several conclusions. First, and most obviously, there needs to be consistency in grades between media spots and graphics. It’s not possible that every media spot is perfect, or almost every sig is flawed enough to deduct. However, the grades are completely different entities. This creates inconsistency in the league and may sway people away from doing point tasks they are more comfortable with in favour of seeking easier grades.

 

Secondly, I think that a solid understanding of grade distribution needs to be established. Right now, nobody is quite sure what to expect from grades, or how frequently sixes should be awarded. I think that the VHL’s grading team needs to adopt a frequency that they want, and to find some sort of grading curve. This will help people better understand their grades and create a more consistent grading system.

 

I also believe that many graphics grades are being awarded arbitrarily, particularly those rated 5.75. Often the 5.75 grades have no feedback as to why they gave that grade, and it seems as if the grade is granted simply so that they are not giving out a full six. The high occurrence of 5.75 against any other grade backs this up, as do some of the rubrics marking 5.75 sigs. This is problematic, and I think is the source of many of the frustrations with grading. I think that graphics graders need to put a lot more justification behind their grading in order to create the best system.

 

I think that a similar thing needs to be said about media spot grading. Sixes are being too often, and it seems that graders are not always taking time to truly consider the content of the story. It seems as though if a story is nicely presented and has decent grammar, it will get a six. Much like the graphics grading, this system also needs to be looked at again.

 

These conclusions are by no means ultimate, they are just my thoughts. Feel free to take the data I’ve collected and do what you want with them. 

Edited February 14, 2014 by Noah!!

[8Ovechkin8 262]

one possible reason for the increased amount of 5.75 marks imo may also be due to the fact that if a sig gets a 6 it doesn't require a 2nd grade. Is it fair to say that giving a 5.75 is ok so that other team members can get a grade in as well? Or is the "get a 6 and lock" rule something that should be looked at as well.

And on a side note, I love how my ass gets chewed apart the most for being too harsh yet I give out the most marks resulting in an overall grade of 6. smh.

[probably not noah 346]

one possible reason for the increased amount of 5.75 marks imo may also be due to the fact that if a sig gets a 6 it doesn't require a 2nd grade. Is it fair to say that giving a 5.75 is ok so that other team members can get a grade in as well? Or is the "get a 6 and lock" rule something that should be looked at as well.

And on a side note, I love how my ass gets chewed apart the most for being too harsh yet I give out the most marks resulting in an overall grade of 6. smh.

 

I think that the "get a 6 and lock" should be looked at. I see the merits, but I don't know if it's worth faster grades. Especially in the GFX community, grades are one of the only places where you can be ensured feedback, and the insta-lock stops some of that. Also, perhaps more relevant, it's really easy for someone to skim over some things or not realize errors in a media spot or sig, especially because graders usually bulk grade. I know that when you get to the end of a massive bulk grade, your mind is usually on autopilot and you can miss a lot of things. 

[Fire Tortorella 2,653]

I think that the "get a 6 and lock" should be looked at. I see the merits, but I don't know if it's worth faster grades. Especially in the GFX community, grades are one of the only places where you can be ensured feedback, and the insta-lock stops some of that. Also, perhaps more relevant, it's really easy for someone to skim over some things or not realize errors in a media spot or sig, especially because graders usually bulk grade. I know that when you get to the end of a massive bulk grade, your mind is usually on autopilot and you can miss a lot of things. 

 

Get rid of the 6 and lock rule and people start complaining that grading is taking too long again. I'd rather have a grade mistakenly be a 6/6 instead of a 5.75/6 and get locked right away than have a bunch of 6/6 grades just sitting there waiting for a 2nd grade.

[Guest]

one possible reason for the increased amount of 5.75 marks imo may also be due to the fact that if a sig gets a 6 it doesn't require a 2nd grade. Is it fair to say that giving a 5.75 is ok so that other team members can get a grade in as well? Or is the "get a 6 and lock" rule something that should be looked at as well.

And on a side note, I love how my ass gets chewed apart the most for being too harsh yet I give out the most marks resulting in an overall grade of 6. smh.

 

You have a good point. But you also give out the lowest scores too. So you're all over the board whereas other graders don't find them quite as bad as you do.

[8Ovechkin8 262]

You have a good point. But you also give out the lowest scores too. So you're all over the board whereas other graders don't find them quite as bad as you do.

You've misread the graph is seems.

Anyways you have to take into account I've graded more as well.

[CowboyinAmerica 2,889]

Just a note: If we're really going to take these individual grader numbers at complete face value (and you shouldn't, because individually the sample sizes are too low, little bit better as a whole), then 808 doesn't have a dissimilar proportion of his grades at below 5.5 as compared to the rest of the graders.

 

Sherriff: 31%

808: 28%

Jackim: 25%

Bushito: 25%

Coach: 20%

[probably not noah 346]

Yeah, the sample size for a lot of these is far too small to take at face value. However, I don't have the time or motivation to run a giant sample size right now, and I think these stats do a decent job of representing trends in grading. Stats on graders shouldn't be pulled too much from this. 

[AwfulHomesick 148]

if a story is nicely presented and has decent grammar, it will get a six

Yeah, you're right. If there's a good story and has grammar that any reader can understand, then it will most likely get a 6. Each media spot doesn't have to be a work of art.

Edited February 14, 2014 by AwfulHomesick

[frescoelmo 1,276]

Yeah, you're right. If there's a good story and has grammar that any reader can understand, then it will most likely get a 6. Each media spot doesn't have to be a work of art.

 

You can lose a 6 pretty easily if you are careless about it though considering how readily accessible online word processors are nowadays (at least that was my rationale when I graded for 5-6 years).  Media spots are graded from a quantitative aspect, and graphics from a qualitative aspect.  That's why there will always be conversation about it...  It's one's perception of what they see and think about it (graphics), whereas if you surpass a minimum requirement and have less than a certain amount of errors you should have no problem with a perfect grade (articles).

Edited February 14, 2014 by frescoelmo

[iRockstar 307]

if people are making graphics that deserve a final of a 6, but aren't getting it, I'd be pissed. If I get 5.5 two times and get a 6 final, then I'm fine. People need to realize, you're getting full marks despite not getting a 6/6...

[8Ovechkin8 262]

if people are making graphics that deserve a final of a 6, but aren't getting it, I'd be pissed. If I get 5.5 two times and get a 6 final, then I'm fine. People need to realize, you're getting full marks despite not getting a 6/6...

FUCK, THANK YOU

[Squinty 116]

this is so fucking cool

[Squinty 116]

Maybe make it a rule to not allow shitty comments like "yes" and "obvious" and "duh" to be given towards sigs, and instead require at least 2 pros and a con for each grade

[Victor 10,940]

I do think a lot of writing graders don't search much for grammar mistakes. There would probably be a few more 5.75 and 5.5 then but Media Spots over 550 words have always been almost guaranteed 6s.

[sherifflobo 137]

If I give someone below a 5.5, I usually explain in more than two sentences why I'm not giving them that mark.  If it's a 5.5 or 5.75, I'll give them a brief explanation, but won't go into as much detail.  A 6 shouldn't need any explanation as to the grade I'm giving them.  I've never said Obviously or yes in Look.  Since Effort and Creativity are basically gimme unless the effort or creativity clearly isn't there, I'll say Yes, Yeah, etc.

 

 

The 6/6 thing is something that needs to be looked at for Graphics.  It's fine for Media Spots, because graders only need 4 grades to get 1 TPE, but Graphic Graders need 8. Plus graphics usually don't sit in the Point task section waiting for a grade, which was the reason this was implemented.  

[Edgar 283]

0.5/6

[Seth 92]

Personally speaking, I believe media spot grading is perfectly fine. Individuals have simply improved their grammar, and stories dramatically over time. In regards to graphics, the criteria for a six is clearly too high, and must be readjusted. The goal for point tasks is for every member to garner a six on a weekly basis. There is no need to impede that by being anal retentive with grades.

[iRockstar 307]

If people are telling me how to improve my graphic, I take their advice and then I go out and get a 6/6. Some people have seen that my graphic shit is not consistent, I don't make Koradek like graphics ever, maybe once or twice I've come close but a lot of them are not close to it. As long as I'm getting full points each week, what the fuck does it matter? Coach your sigs are great, but overall you need to realize you're walking out with full points, half the time I don't even care for the 1st grade I just move on to when the topic is locked and I can claim 6. 

 

If you're not all about getting criticism from someone else's point of view, NO one should be posting their work to get graded. I'm pretty sure the majority of us are old enough to realize this. 

[sterling 1,997]

Do one on our simming problem next.

[sterling 1,997]

An obvious easy solution to me is to get rid of the auto lock for graphics.

[probably not noah 346]

Do one on our simming problem next.

 

I would, but the media spot would only be one word: Jardy.

[Victor 10,940]

An obvious easy solution to me is to get rid of the auto lock for graphics.

But if someone gets a 6 they won't get 4.75 as a second grade.

[probably not noah 346]

But if someone gets a 6 they won't get 4.75 as a second grade.

 

But they might still get feedback on a second grade, and it allows for more grading opportunities. 

[Mr. Grumpy Bear 347]

Auto lock is no good.