probably not noah 346 Posted February 14, 2014 Share Posted February 14, 2014 (edited) Study: The VHL's Grading Problem The VHL has a grading problem. There is a massive discrepancy between graphics grades and writing grades. This has brought on tension in the community. Many graphics makers feel that the criteria for a six is far too high, and they are unfairly losing marks. As a red team mission, I created a quick study of the grading trends since the start of my discovery. The data can be found here, and my findings follow. Graphics Grading: Fifty-two graphics have been collected on 27 sigs since the start of February. Of these grades, only three sigs have been awarded a full six. Each of these sixes was awarded to a different member by a different grader. Below are the three sigs that have received full marks since the beginning of the month: The vast majority of grades fall between 5.5 and 5.75, with the average reaching 5.48. This means that on average, a graphic posted in the last 2 weeks did not reach a final grade of six. Below is a plot of the frequency of grade occurrence. There was no distinct pattern in the grades, but instead a large imbalance in grade distribution. Fifty-two grades is a relatively small sample size, but it still illustrates what many of the graphics community seem to be upset about. There is an incredibly high number of 5.75s given out, and a much smaller rate of full sixes. This was especially true in the grading of 8Ovechkin8. He gave out 18 grades in this time period, eight of which were 5.75. Five of Sherriflobo’s grades were also 5.75. Below is a plot of each grader’s own grade distribution. Media Spots I planned to plot the information of media spot grading as well, and compare data points and charts. However, when I collected all of the data from the media spots, I learnt that every single media grade given so far this month has been a six. The number of point tasks graded is consistent with graphics, as 24 media spots have been graded, yet not a single one has been less than a perfect six. This also means that the grading sample size is cut in half because a six doesn’t warrant another grade. Conclusions In my research, I came to several conclusions. First, and most obviously, there needs to be consistency in grades between media spots and graphics. It’s not possible that every media spot is perfect, or almost every sig is flawed enough to deduct. However, the grades are completely different entities. This creates inconsistency in the league and may sway people away from doing point tasks they are more comfortable with in favour of seeking easier grades. Secondly, I think that a solid understanding of grade distribution needs to be established. Right now, nobody is quite sure what to expect from grades, or how frequently sixes should be awarded. I think that the VHL’s grading team needs to adopt a frequency that they want, and to find some sort of grading curve. This will help people better understand their grades and create a more consistent grading system. I also believe that many graphics grades are being awarded arbitrarily, particularly those rated 5.75. Often the 5.75 grades have no feedback as to why they gave that grade, and it seems as if the grade is granted simply so that they are not giving out a full six. The high occurrence of 5.75 against any other grade backs this up, as do some of the rubrics marking 5.75 sigs. This is problematic, and I think is the source of many of the frustrations with grading. I think that graphics graders need to put a lot more justification behind their grading in order to create the best system. I think that a similar thing needs to be said about media spot grading. Sixes are being too often, and it seems that graders are not always taking time to truly consider the content of the story. It seems as though if a story is nicely presented and has decent grammar, it will get a six. Much like the graphics grading, this system also needs to be looked at again. These conclusions are by no means ultimate, they are just my thoughts. Feel free to take the data I’ve collected and do what you want with them. Edited February 14, 2014 by Noah!! CowboyinAmerica 1 Link to comment Share on other sites More sharing options...
8Ovechkin8 262 Posted February 14, 2014 Share Posted February 14, 2014 one possible reason for the increased amount of 5.75 marks imo may also be due to the fact that if a sig gets a 6 it doesn't require a 2nd grade. Is it fair to say that giving a 5.75 is ok so that other team members can get a grade in as well? Or is the "get a 6 and lock" rule something that should be looked at as well. And on a side note, I love how my ass gets chewed apart the most for being too harsh yet I give out the most marks resulting in an overall grade of 6. smh. Link to comment Share on other sites More sharing options...
probably not noah 346 Posted February 14, 2014 Author Share Posted February 14, 2014 one possible reason for the increased amount of 5.75 marks imo may also be due to the fact that if a sig gets a 6 it doesn't require a 2nd grade. Is it fair to say that giving a 5.75 is ok so that other team members can get a grade in as well? Or is the "get a 6 and lock" rule something that should be looked at as well. And on a side note, I love how my ass gets chewed apart the most for being too harsh yet I give out the most marks resulting in an overall grade of 6. smh. I think that the "get a 6 and lock" should be looked at. I see the merits, but I don't know if it's worth faster grades. Especially in the GFX community, grades are one of the only places where you can be ensured feedback, and the insta-lock stops some of that. Also, perhaps more relevant, it's really easy for someone to skim over some things or not realize errors in a media spot or sig, especially because graders usually bulk grade. I know that when you get to the end of a massive bulk grade, your mind is usually on autopilot and you can miss a lot of things. Link to comment Share on other sites More sharing options...
Fire Tortorella 2,653 Posted February 14, 2014 Share Posted February 14, 2014 I think that the "get a 6 and lock" should be looked at. I see the merits, but I don't know if it's worth faster grades. Especially in the GFX community, grades are one of the only places where you can be ensured feedback, and the insta-lock stops some of that. Also, perhaps more relevant, it's really easy for someone to skim over some things or not realize errors in a media spot or sig, especially because graders usually bulk grade. I know that when you get to the end of a massive bulk grade, your mind is usually on autopilot and you can miss a lot of things. Get rid of the 6 and lock rule and people start complaining that grading is taking too long again. I'd rather have a grade mistakenly be a 6/6 instead of a 5.75/6 and get locked right away than have a bunch of 6/6 grades just sitting there waiting for a 2nd grade. Link to comment Share on other sites More sharing options...
Guest Posted February 14, 2014 Share Posted February 14, 2014 one possible reason for the increased amount of 5.75 marks imo may also be due to the fact that if a sig gets a 6 it doesn't require a 2nd grade. Is it fair to say that giving a 5.75 is ok so that other team members can get a grade in as well? Or is the "get a 6 and lock" rule something that should be looked at as well. And on a side note, I love how my ass gets chewed apart the most for being too harsh yet I give out the most marks resulting in an overall grade of 6. smh. You have a good point. But you also give out the lowest scores too. So you're all over the board whereas other graders don't find them quite as bad as you do. Link to comment Share on other sites More sharing options...
8Ovechkin8 262 Posted February 14, 2014 Share Posted February 14, 2014 You have a good point. But you also give out the lowest scores too. So you're all over the board whereas other graders don't find them quite as bad as you do. You've misread the graph is seems. Anyways you have to take into account I've graded more as well. Link to comment Share on other sites More sharing options...
CowboyinAmerica 2,880 Posted February 14, 2014 Share Posted February 14, 2014 Just a note: If we're really going to take these individual grader numbers at complete face value (and you shouldn't, because individually the sample sizes are too low, little bit better as a whole), then 808 doesn't have a dissimilar proportion of his grades at below 5.5 as compared to the rest of the graders. Sherriff: 31% 808: 28% Jackim: 25% Bushito: 25% Coach: 20% Link to comment Share on other sites More sharing options...
probably not noah 346 Posted February 14, 2014 Author Share Posted February 14, 2014 Yeah, the sample size for a lot of these is far too small to take at face value. However, I don't have the time or motivation to run a giant sample size right now, and I think these stats do a decent job of representing trends in grading. Stats on graders shouldn't be pulled too much from this. Link to comment Share on other sites More sharing options...
AwfulHomesick 148 Posted February 14, 2014 Share Posted February 14, 2014 (edited) if a story is nicely presented and has decent grammar, it will get a six Yeah, you're right. If there's a good story and has grammar that any reader can understand, then it will most likely get a 6. Each media spot doesn't have to be a work of art. Edited February 14, 2014 by AwfulHomesick Link to comment Share on other sites More sharing options...
Head Moderator frescoelmo 1,275 Posted February 14, 2014 Head Moderator Share Posted February 14, 2014 (edited) Yeah, you're right. If there's a good story and has grammar that any reader can understand, then it will most likely get a 6. Each media spot doesn't have to be a work of art. You can lose a 6 pretty easily if you are careless about it though considering how readily accessible online word processors are nowadays (at least that was my rationale when I graded for 5-6 years). Media spots are graded from a quantitative aspect, and graphics from a qualitative aspect. That's why there will always be conversation about it... It's one's perception of what they see and think about it (graphics), whereas if you surpass a minimum requirement and have less than a certain amount of errors you should have no problem with a perfect grade (articles). Edited February 14, 2014 by frescoelmo Link to comment Share on other sites More sharing options...
iRockstar 307 Posted February 14, 2014 Share Posted February 14, 2014 if people are making graphics that deserve a final of a 6, but aren't getting it, I'd be pissed. If I get 5.5 two times and get a 6 final, then I'm fine. People need to realize, you're getting full marks despite not getting a 6/6... CowboyinAmerica 1 Link to comment Share on other sites More sharing options...
8Ovechkin8 262 Posted February 14, 2014 Share Posted February 14, 2014 if people are making graphics that deserve a final of a 6, but aren't getting it, I'd be pissed. If I get 5.5 two times and get a 6 final, then I'm fine. People need to realize, you're getting full marks despite not getting a 6/6... FUCK, THANK YOU Link to comment Share on other sites More sharing options...
Squinty 116 Posted February 14, 2014 Share Posted February 14, 2014 this is so fucking cool Link to comment Share on other sites More sharing options...
Squinty 116 Posted February 14, 2014 Share Posted February 14, 2014 Maybe make it a rule to not allow shitty comments like "yes" and "obvious" and "duh" to be given towards sigs, and instead require at least 2 pros and a con for each grade Link to comment Share on other sites More sharing options...
Admin Victor 10,911 Posted February 14, 2014 Admin Share Posted February 14, 2014 I do think a lot of writing graders don't search much for grammar mistakes. There would probably be a few more 5.75 and 5.5 then but Media Spots over 550 words have always been almost guaranteed 6s. Link to comment Share on other sites More sharing options...
sherifflobo 137 Posted February 14, 2014 Share Posted February 14, 2014 If I give someone below a 5.5, I usually explain in more than two sentences why I'm not giving them that mark. If it's a 5.5 or 5.75, I'll give them a brief explanation, but won't go into as much detail. A 6 shouldn't need any explanation as to the grade I'm giving them. I've never said Obviously or yes in Look. Since Effort and Creativity are basically gimme unless the effort or creativity clearly isn't there, I'll say Yes, Yeah, etc. The 6/6 thing is something that needs to be looked at for Graphics. It's fine for Media Spots, because graders only need 4 grades to get 1 TPE, but Graphic Graders need 8. Plus graphics usually don't sit in the Point task section waiting for a grade, which was the reason this was implemented. Link to comment Share on other sites More sharing options...
Edgar 283 Posted February 14, 2014 Share Posted February 14, 2014 0.5/6 Link to comment Share on other sites More sharing options...
Seth 92 Posted February 14, 2014 Share Posted February 14, 2014 Personally speaking, I believe media spot grading is perfectly fine. Individuals have simply improved their grammar, and stories dramatically over time. In regards to graphics, the criteria for a six is clearly too high, and must be readjusted. The goal for point tasks is for every member to garner a six on a weekly basis. There is no need to impede that by being anal retentive with grades. Link to comment Share on other sites More sharing options...
iRockstar 307 Posted February 14, 2014 Share Posted February 14, 2014 If people are telling me how to improve my graphic, I take their advice and then I go out and get a 6/6. Some people have seen that my graphic shit is not consistent, I don't make Koradek like graphics ever, maybe once or twice I've come close but a lot of them are not close to it. As long as I'm getting full points each week, what the fuck does it matter? Coach your sigs are great, but overall you need to realize you're walking out with full points, half the time I don't even care for the 1st grade I just move on to when the topic is locked and I can claim 6. If you're not all about getting criticism from someone else's point of view, NO one should be posting their work to get graded. I'm pretty sure the majority of us are old enough to realize this. Link to comment Share on other sites More sharing options...
sterling 1,997 Posted February 14, 2014 Share Posted February 14, 2014 Do one on our simming problem next. Link to comment Share on other sites More sharing options...
sterling 1,997 Posted February 14, 2014 Share Posted February 14, 2014 An obvious easy solution to me is to get rid of the auto lock for graphics. Link to comment Share on other sites More sharing options...
probably not noah 346 Posted February 14, 2014 Author Share Posted February 14, 2014 Do one on our simming problem next. I would, but the media spot would only be one word: Jardy. Link to comment Share on other sites More sharing options...
Admin Victor 10,911 Posted February 14, 2014 Admin Share Posted February 14, 2014 An obvious easy solution to me is to get rid of the auto lock for graphics. But if someone gets a 6 they won't get 4.75 as a second grade. Link to comment Share on other sites More sharing options...
probably not noah 346 Posted February 14, 2014 Author Share Posted February 14, 2014 But if someone gets a 6 they won't get 4.75 as a second grade. But they might still get feedback on a second grade, and it allows for more grading opportunities. Link to comment Share on other sites More sharing options...
Mr. Grumpy Bear 347 Posted February 14, 2014 Share Posted February 14, 2014 Auto lock is no good. Link to comment Share on other sites More sharing options...
.png.ed69a7260a477fcf4feead7a0e4b7506.png)
Recommended Posts