Jump to content

Claimed:The Grading Problem


Recommended Posts

Study: The VHL's Grading Problem

 

The VHL has a grading problem.

 

There is a massive discrepancy between graphics grades and writing grades. This has brought on tension in the community. Many graphics makers feel that the criteria for a six is far too high, and they are unfairly losing marks.

 

As a red team mission, I created a quick study of the grading trends since the start of my discovery. The data can be found here, and my findings follow.

 

Graphics Grading:

 

Fifty-two graphics have been collected on 27 sigs since the start of February. Of these grades, only three sigs have been awarded a full six. Each of these sixes was awarded to a different member by a different grader. Below are the three sigs that have received full marks since the beginning of the month:
 

2zsa5xe.png

esJoXJM.png

Nx1mp6q.png

 


The vast majority of grades fall between 5.5 and 5.75, with the average reaching 5.48. This means that on average, a graphic posted in the last 2 weeks did not reach a final grade of six. Below is a plot of the frequency of grade occurrence. There was no distinct pattern in the grades, but instead a large imbalance in grade distribution.

 

wFqlk6X.png

 

Fifty-two grades is a relatively small sample size, but it still illustrates what many of the graphics community seem to be upset about. There is an incredibly high number of 5.75s given out, and a much smaller rate of full sixes. This was especially true in the grading of 8Ovechkin8. He gave out 18 grades in this time period, eight of which were 5.75. Five of Sherriflobo’s grades were also 5.75. Below is a plot of each grader’s own grade distribution.

 

qlpqwMv.png

 

Media Spots

 

I planned to plot the information of media spot grading as well, and compare data points and charts. However, when I collected all of the data from the media spots, I learnt that every single media grade given so far this month has been a six. The number of point tasks graded is consistent with graphics, as 24 media spots have been graded, yet not a single one has been less than a perfect six. This also means that the grading sample size is cut in half because a six doesn’t warrant another grade.

 

Conclusions

 

In my research, I came to several conclusions. First, and most obviously, there needs to be consistency in grades between media spots and graphics. It’s not possible that every media spot is perfect, or almost every sig is flawed enough to deduct. However, the grades are completely different entities. This creates inconsistency in the league and may sway people away from doing point tasks they are more comfortable with in favour of seeking easier grades.

 

Secondly, I think that a solid understanding of grade distribution needs to be established. Right now, nobody is quite sure what to expect from grades, or how frequently sixes should be awarded. I think that the VHL’s grading team needs to adopt a frequency that they want, and to find some sort of grading curve. This will help people better understand their grades and create a more consistent grading system.

 

I also believe that many graphics grades are being awarded arbitrarily, particularly those rated 5.75. Often the 5.75 grades have no feedback as to why they gave that grade, and it seems as if the grade is granted simply so that they are not giving out a full six. The high occurrence of 5.75 against any other grade backs this up, as do some of the rubrics marking 5.75 sigs. This is problematic, and I think is the source of many of the frustrations with grading. I think that graphics graders need to put a lot more justification behind their grading in order to create the best system.

 

I think that a similar thing needs to be said about media spot grading. Sixes are being too often, and it seems that graders are not always taking time to truly consider the content of the story. It seems as though if a story is nicely presented and has decent grammar, it will get a six. Much like the graphics grading, this system also needs to be looked at again.

 

These conclusions are by no means ultimate, they are just my thoughts. Feel free to take the data I’ve collected and do what you want with them. 

Edited by Noah!!
Link to comment
Share on other sites

one possible reason for the increased amount of 5.75 marks imo may also be due to the fact that if a sig gets a 6 it doesn't require a 2nd grade. Is it fair to say that giving a 5.75 is ok so that other team members can get a grade in as well? Or is the "get a 6 and lock" rule something that should be looked at as well.

And on a side note, I love how my ass gets chewed apart the most for being too harsh yet I give out the most marks resulting in an overall grade of 6. smh.

Link to comment
Share on other sites

one possible reason for the increased amount of 5.75 marks imo may also be due to the fact that if a sig gets a 6 it doesn't require a 2nd grade. Is it fair to say that giving a 5.75 is ok so that other team members can get a grade in as well? Or is the "get a 6 and lock" rule something that should be looked at as well.

And on a side note, I love how my ass gets chewed apart the most for being too harsh yet I give out the most marks resulting in an overall grade of 6. smh.

 

I think that the "get a 6 and lock" should be looked at. I see the merits, but I don't know if it's worth faster grades. Especially in the GFX community, grades are one of the only places where you can be ensured feedback, and the insta-lock stops some of that. Also, perhaps more relevant, it's really easy for someone to skim over some things or not realize errors in a media spot or sig, especially because graders usually bulk grade. I know that when you get to the end of a massive bulk grade, your mind is usually on autopilot and you can miss a lot of things. 

Link to comment
Share on other sites

I think that the "get a 6 and lock" should be looked at. I see the merits, but I don't know if it's worth faster grades. Especially in the GFX community, grades are one of the only places where you can be ensured feedback, and the insta-lock stops some of that. Also, perhaps more relevant, it's really easy for someone to skim over some things or not realize errors in a media spot or sig, especially because graders usually bulk grade. I know that when you get to the end of a massive bulk grade, your mind is usually on autopilot and you can miss a lot of things. 

 

Get rid of the 6 and lock rule and people start complaining that grading is taking too long again. I'd rather have a grade mistakenly be a 6/6 instead of a 5.75/6 and get locked right away than have a bunch of 6/6 grades just sitting there waiting for a 2nd grade.

Link to comment
Share on other sites

one possible reason for the increased amount of 5.75 marks imo may also be due to the fact that if a sig gets a 6 it doesn't require a 2nd grade. Is it fair to say that giving a 5.75 is ok so that other team members can get a grade in as well? Or is the "get a 6 and lock" rule something that should be looked at as well.

And on a side note, I love how my ass gets chewed apart the most for being too harsh yet I give out the most marks resulting in an overall grade of 6. smh.

 

You have a good point. But you also give out the lowest scores too. So you're all over the board whereas other graders don't find them quite as bad as you do.

Link to comment
Share on other sites

You have a good point. But you also give out the lowest scores too. So you're all over the board whereas other graders don't find them quite as bad as you do.

You've misread the graph is seems.

Anyways you have to take into account I've graded more as well.

Link to comment
Share on other sites

Just a note: If we're really going to take these individual grader numbers at complete face value (and you shouldn't, because individually the sample sizes are too low, little bit better as a whole), then 808 doesn't have a dissimilar proportion of his grades at below 5.5 as compared to the rest of the graders.

 

Sherriff: 31%

808: 28%

Jackim: 25%

Bushito: 25%

Coach: 20%

Link to comment
Share on other sites

Yeah, the sample size for a lot of these is far too small to take at face value. However, I don't have the time or motivation to run a giant sample size right now, and I think these stats do a decent job of representing trends in grading. Stats on graders shouldn't be pulled too much from this. 

Link to comment
Share on other sites

if a story is nicely presented and has decent grammar, it will get a six

Yeah, you're right. If there's a good story and has grammar that any reader can understand, then it will most likely get a 6. Each media spot doesn't have to be a work of art.

Edited by AwfulHomesick
Link to comment
Share on other sites

  • Head Moderator

Yeah, you're right. If there's a good story and has grammar that any reader can understand, then it will most likely get a 6. Each media spot doesn't have to be a work of art.

 

You can lose a 6 pretty easily if you are careless about it though considering how readily accessible online word processors are nowadays (at least that was my rationale when I graded for 5-6 years).  Media spots are graded from a quantitative aspect, and graphics from a qualitative aspect.  That's why there will always be conversation about it...  It's one's perception of what they see and think about it (graphics), whereas if you surpass a minimum requirement and have less than a certain amount of errors you should have no problem with a perfect grade (articles).

Edited by frescoelmo
Link to comment
Share on other sites

if people are making graphics that deserve a final of a 6, but aren't getting it, I'd be pissed. If I get 5.5 two times and get a 6 final, then I'm fine. People need to realize, you're getting full marks despite not getting a 6/6...

FUCK, THANK YOU

Link to comment
Share on other sites

If I give someone below a 5.5, I usually explain in more than two sentences why I'm not giving them that mark.  If it's a 5.5 or 5.75, I'll give them a brief explanation, but won't go into as much detail.  A 6 shouldn't need any explanation as to the grade I'm giving them.  I've never said Obviously or yes in Look.  Since Effort and Creativity are basically gimme unless the effort or creativity clearly isn't there, I'll say Yes, Yeah, etc.

 

 

The 6/6 thing is something that needs to be looked at for Graphics.  It's fine for Media Spots, because graders only need 4 grades to get 1 TPE, but Graphic Graders need 8. Plus graphics usually don't sit in the Point task section waiting for a grade, which was the reason this was implemented.  

Link to comment
Share on other sites

Personally speaking, I believe media spot grading is perfectly fine. Individuals have simply improved their grammar, and stories dramatically over time. In regards to graphics, the criteria for a six is clearly too high, and must be readjusted. The goal for point tasks is for every member to garner a six on a weekly basis. There is no need to impede that by being anal retentive with grades.

Link to comment
Share on other sites

If people are telling me how to improve my graphic, I take their advice and then I go out and get a 6/6. Some people have seen that my graphic shit is not consistent, I don't make Koradek like graphics ever, maybe once or twice I've come close but a lot of them are not close to it. As long as I'm getting full points each week, what the fuck does it matter? Coach your sigs are great, but overall you need to realize you're walking out with full points, half the time I don't even care for the 1st grade I just move on to when the topic is locked and I can claim 6. 

 

If you're not all about getting criticism from someone else's point of view, NO one should be posting their work to get graded. I'm pretty sure the majority of us are old enough to realize this. 

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...