Login

Welcome, Guest. Please login or register.

May 03, 2024, 05:00:28 pm

Author Topic: I don't think that COVID will affect VCE subject scaling - let me tell you why  (Read 1129 times)  Share 

0 Members and 1 Guest are viewing this topic.

keltingmeith

  • Honorary Moderator
  • Great Wonder of ATAR Notes
  • *******
  • Posts: 5493
  • he/him - they is also fine
  • Respect: +1292
Firstly - a lot of this information is not new, and things that us older members have been saying all year. For a while, though, we also said we weren't sure how COVID will affect scaling. This was us playing it safe, because really - everything I'm saying is conjecture, and I wouldn't be surprised if other members come in and think that I'm wrong. After a good, long, think - I've finally reached a verdict on how I think scaling will be affected. None of this information is because I've had a VCAA birdie whisper in my ear, none of this is because I'm clairvoyant, and I may be wrong - but there's a logic behind my opinion, that I want you all to listen to, and hopefully it will ease fears for people who are scared.



So firstly - what is scaling, and how does it work? Some of you know, or think you know, but this is going to get super technical. Like, insanely technical. But it's important to get technical, or else we can't make inferences on what might happen.

A study score is a point on a bell curve. Where you are on that bell curve depends on how well everybody else did. If you did better than 16% of the state, but the rest scored higher than you, you'll get a 23. If you did better than 50% of the state, but other 50% scored higher, you'll get a 30. If you did better than 84% of the state, you'll get a 37. These numbers have nothing to do with what exact score you get, only what you get in relation to other people.

Let's say I'm doing maths methods and psychology, and my results come in. The results aren't given to me as a study score - they're given as a percentage. For methods, this is 80%, and for psychology, it's 60%. Now, VCAA wants to turn this to a study score, but how do they do that? They can't just say that for every subject, an 80% means a study score of 37, a 60% means a study score of 32, because it depends on how many people did better or worse. So, let's say that the average mark for methods was 70%, and the average mark for psychology was 30% - that would mean my methods score would be close to 30, and my psych score would be up near the 40-50 range since I scored so much higher than everyone else

But that doesn't seem fair, does it? For example, if normally people doing psychology score so low, is it because psychology is hard? What if I can get 50% in psych? Even if it's hard, that's still way higher than everyone else, so I could get a really good score. Meanwhile, people universally know that methods is really hard, so why should I be penalised for doing a harder maths, just because everyone else is really good at it? This is where scaling comes in.

Now, some people might think that psychology scoring lower means it's harder and that you should scale up - but what if everyone in psychology scores so low because the people who take it think it's easy, and so don't try? And what if the methods average is so high not because it's easy, but because everyone knows it's hard and studies extra hard for it. Because of all of this, difficulty is really subjective, so it's not how VCAA does scaling - like so many believe.

The way VCAA decides scaling isn't to look at how well you did in the subject being scaled - it's to see how well you did in all your OTHER subjects. For example, I scored really low in psychology, but really high in methods. This indicates that I can score highly, but found psychology harder than methods. From this, VCAA will slightly scale psychology up, and slightly scale methods down. But then, what if I got full marks in English, media, and biology? Well, then VCAA might see that I did much worse in methods, and put methods slightly up a bit, and the rest down a bit.

And that's what VCAA does - they look at the scores of every individual student, and then looks at how well they did in all of their OTHER subjects. How relatively the student does in each of those other subjects will decide how the subject you're looking at gets scaled in relation to those subjects. So, if everybody doing psychology also does media, and everybody in psychology scores higher in psychology than they did in media, you can bet that psychology will have better scaling than media.



So with all that in mind, why do I think COVID won't impact scaling? Because it doesn't matter what your exact score is, only what your score is relative to all your other scores. So, it doesn't matter if everybody in the state scores lower in all of their assessments for all of their subjects - if the difference between their subjects is still relatively the same, it won't make a difference.

Which brings us to an important question - if scaling depends so heavily on how individual students do on individual subjects each year, and their related subjects, and every individual student is different to every other student, why does scaling rarely change? The reason for this is because everything works on averages. If you have a large enough sample size (and note: there are A LOT of year 12 students getting ATARs), then you'll see a lot of similar trends stay, even if every individual is very different to every other individual. On average, if the numbers are big enough, everything will start to look the same.

What I've seen is that, in general, if someone did poorly in one subject because of COVID, they did poorly in all subjects because of COVID. In-school moderation is also being used to fix that. Teachers are using previous years data to predict how that student should've done. All of these fixes mean that everything from this year should look roughly like it has in previous years - and even if it doesn't, the relative score difference in each subject likely won't change much. If you scored 10% lower in media than you did in English, even if they're both 20% lower than you could've scored, you're still scoring with a 10% difference in media and English. And that's the same for everyone else - maybe their scores only dropped by 15%, or maybe they dropped by 30%, but it's likely that the relative gap between subjects is exactly the same - and that's what you need to pay attention to.



So, why is it important that scaling doesn't change? Because it means that the trends we've seen in previous years should mostly hold. A lot of people are worried that they can't trust the AN calculator, they can't trust study score predictions, etc. Firstly, I want to make this clear - if you're too worried by COVID to trust a calculator to predict your score, then a person isn't going to be able to predict it better. This is because the volunteers predicting your scores are doing this based on past trends - that's how we do it, because again, it depends on how well everybody else does.

Now, I'm not guaranteeing that everything stays the same - for example, there's a rumour that the psych exam was way harder than it normally is. If this is true, you'll likely see people scoring (on average) far lower in psych than in previous years, which will likely lead to a slight increase in psych scaling (I'm also not predicting that this WILL happen, so psych students, please don't assume you'll get better scaling - but do take solace that if everybody has done worse, then at the very least, a lower score on the exam won't necessarily mean a lower study score, since it's all position based, not grade based). Scaling trends do change each year, and I'm not saying they won't - I just don't think that any differences we see this year will be due to COVID.

I know people are scared, and I can't assuage anybody's fears, but the thing that usually has worked in keeping VCE students comfortable in previous years is knowing the trend from the year before. A lot of people are scared that COVID is going to change everything, and that you can't predict everything, and that everything is a coin flip - I don't think it is. I think we will see the same amount of trend retention that we normally see.

Any questions, please direct them below - I'm not the only person here qualified to answer, too, and I'm slowly becoming more inactive because personal life. But there are plenty who know just as much as me (and probably a few who will disagree with me - open discussion welcome) who can still help calm your fears.

Bri MT

  • VIC MVP - 2018
  • Administrator
  • ATAR Notes Legend
  • *****
  • Posts: 4719
  • invest in wellbeing so it can invest in you
  • Respect: +3677
+15
I wouldn't be surprised if the gap between high and low SES students and schools is higher this year with resultant impacts on scaling or if there is some general disparity in how well different subjects are taught/learnt online influencing scaling but the key message to not be scared of scaling is definitely something I can stand behind.

I think students often lose sight of the fact that the vast majority of scaling is not there to "reward" or "punish" certain subjects (languages being an obvious exception) but to do the opposite and keep things fair

chem-nerd

  • Victorian
  • Forum Obsessive
  • ***
  • Posts: 368
  • Respect: +13
I agree with keltingmeith that COVID should have minimal impact on the moderation and scaling.

Keep in mind they are two separate processes:

VCAA
- sets curriculum and exams
- MODERATES internal SAC marks ( short version - converts the cohorts exam marks to an External score on the same scale as the Internal SAC score, adjusts the cohorts Internal score to match the level and spread of the External score)
- calculates raw Study Scores - these are used for tertiary course prerequisite scores
- implements the new Consideration of Educational Disadvantage (CED) - allowing schools/ teachers to account for bushfire/COVID impacts. This happens prior to moderation and calculation of Study Scores

VTAC
- connects applicants and tertiary providers
- undertakes the SCALING process to convert raw Study Scores prior to ATAR calculation
(short version - eg Subject A SS 35 - compared to the average of the average of the top three other Study Scores for each student who achieved a 35. If that average average is higher than 35, means it was harder to get a 35 in Subject A so it is scaled up.  Each Study Score undergoes this process, and so the scaling is different at each point)
- scaling really doesn’t change much from year to year, unless there is a specific change to the demographic of the subject cohort (usually results in a slow shift in scores over a few years eg English group) or a change in the scaling method (eg SM between 2011 - 2012)
- implements the Special Entry Access Scheme (SEAS) - each tertiary provider individually applies adjustments to your ATAR to create a Selection Rank