December 08, 2019, 09:14:52 pm

### AuthorTopic: VCE Science Research Methods Question Thread  (Read 898 times) Tweet Share

0 Members and 1 Guest are viewing this topic.

#### vox nihili

• National Moderator
• Great Wonder of ATAR Notes
• Posts: 5289
• Respect: +1373
##### VCE Science Research Methods Question Thread
« on: March 01, 2019, 02:18:49 pm »
+8
After changes to the VCE study designs a couple of years ago, research methods are now central to all of the science curricula. Since the introduction of research methods, this has continued to be a challenging area of the course for science students.

This question thread is your opportunity to ask questions about research methods from any of the VCE sciences. From time to time, we will also post questions for you to answer that will illustrate the basic principles of research methods!
MED INTERVIEW TUTORING PM to secure your place early, as they fill up quickly!

Join ATARNotes Footy Tipping

2013-15: BBioMed (Biochemistry and Molecular Biology), UniMelb
2016-20: MD, UniMelb
2019: MPH, UniMelb
Year I: BIOL10002 BIOL10003 CHEM10006 MAST10011 MAST10016 PHYC10007 SPAN10001 SPAN10002
Year II: BCMB20005 BIOM20001 BIOM20002 CLAS10022 GENE20001 SPAN20020 SPAN30014
Year III: BCBM30001 BCMB30002 BCMB30010 BIOM30001 BIOM30002 PHRM30008

#### vox nihili

• National Moderator
• Great Wonder of ATAR Notes
• Posts: 5289
• Respect: +1373
##### Re: VCE Science Research Methods Question Thread
« Reply #1 on: March 16, 2019, 03:40:38 pm »
+8
Introduction to practice questions

From time to time moderators will post questions here for you to have a go at. The purpose of the questions is not to search for narrow definitions in research methods, but to encourage you to think more deeply about what good research looks like and where it might go wrong. A lot of what you learn at school is very much about the rote of research (how to write a poster, what an IV and DV are); however, questions on the exam expect you to be able to think about experiments critically. These questions aim to develop your skills in critical analysis of research.

Active participation is encouraged! We want to hear your ideas and hear you justify them. Once a good discussion has happened, we will add our own answers and give you some pointers about your own.

Practice question #1

Jerry, a researcher at the National University of Victoria, is really interested in pain. Even more than that, he's interested in making lots of money, so he sets out to design a pain drug and test it. Jerry sets about designing the drug and once he's done, he recruits 10,000 people to a study to test it. He splits the people into three different groups:

Group 1: 5mg of drug
Group 2: 10mg of drug
Group 3: 20 mg of drug

Each group is followed for six months and instructed to take the drug whenever they feel pain that they rate more than 5/10 on a severity scale. They are then asked to rate their pain at 30 minutes, 1 hour and 3 hours after taking the drug.

Once study is concluded, Jerry is very excited about the results! The results are summarised below by average reduction in pain score

Group 1
30min: 17%
1 hour: 20%
2 hours: 21%

Group 2
30 min: 24%
1 hour: 28%
2 hours: 30%

Group 3
30 min: 27%
1 hour: 31%
2 hours: 34%

Jerry then sets about selling his drug on the basis of these results, as he concludes that the drug is making a significant difference to people's pain scores.

QUESTIONS:

-Should the drug be sold? Why or why not?

MED INTERVIEW TUTORING PM to secure your place early, as they fill up quickly!

Join ATARNotes Footy Tipping

2013-15: BBioMed (Biochemistry and Molecular Biology), UniMelb
2016-20: MD, UniMelb
2019: MPH, UniMelb
Year I: BIOL10002 BIOL10003 CHEM10006 MAST10011 MAST10016 PHYC10007 SPAN10001 SPAN10002
Year II: BCMB20005 BIOM20001 BIOM20002 CLAS10022 GENE20001 SPAN20020 SPAN30014
Year III: BCBM30001 BCMB30002 BCMB30010 BIOM30001 BIOM30002 PHRM30008

#### DBA-144

• MOTM: APR 19
• Forum Obsessive
• Posts: 206
• Respect: +33
##### Re: VCE Science Research Methods Question Thread
« Reply #2 on: March 17, 2019, 01:56:59 pm »
+1
Are we allowed to ask further information? I have a pretty good idea of what I think, but I want to ask some more questions to get a better idea of how the research was conducted.

If we can, can you please answer these (sorry if I am taking this way too seriously, I have some time so I thought it would be good to participate in this):

1.What age groups were the participants and what was the selection process?
2. Was there any incentive for the participants to take part in the research? (eg. \$, better grades for other uni students, etc.)
3. Did the drug have any side effects?

My thoughts:

There was no control group (i.e there was not a group that was given a placebo drug, for example) and there was also no indication, or from what I gather, there was no minimum level of pain for which participants were told to take the drug. Hence, those who took part in the study could have been taking the drug for minor headaches or for much worse cases of pain. This could have changed how they ranked their pain socre. For instance, the pain of a headache may subside after 3 hours or half an hour, depending on the person and severity, and it is likely participants take additional steps, other than taking the drug, to minimise their pain. These actions, depending on what they were, may have caused them to provide a lower pain score than if they only took the drug. This could indicate that the effects of the drug are minimal relative to other actions that participants might have taken.

More importantly, the results do not seem to suggest that any conclusions can actually be made about the research. Like I said earlier, there isn't a control group to compare results to, so we do not know what the actual effects of the drug are. Furthermore, the results simply show that after time, the pain score is reduced by a greater percentage- yet, changing the concentration of the drug seems to have a small, if any, influence on pain score reduction after the first 30 minutes. So, while this may suggest that higher concentrations of the drug cause greater pain score reduction earlier, this can also be because the person taking the drug feels that as there is more drug, they will get better sooner. IN this way, it seems likely that there is also a psychological effect of the drug.

Therefore, as the research has results which a) do not highlight what the effects of the drug are and b) by extension of point a), the results may be purely because of a placebo effect, then no conclusions that support the sale of the drug can be made and hence the drug should not be sold.

(Hopefully I didn't say anything too stupid )

#### vox nihili

• National Moderator
• Great Wonder of ATAR Notes
• Posts: 5289
• Respect: +1373
##### Re: VCE Science Research Methods Question Thread
« Reply #3 on: March 17, 2019, 08:04:51 pm »
+1
snip

1. Patients were referred by their doctors to the study. Any one over the age of eighteen was eligible for the study. The ages of the participants was balanced in the groups.
2. No.
3. All drugs have side effects
MED INTERVIEW TUTORING PM to secure your place early, as they fill up quickly!

Join ATARNotes Footy Tipping

2013-15: BBioMed (Biochemistry and Molecular Biology), UniMelb
2016-20: MD, UniMelb
2019: MPH, UniMelb
Year I: BIOL10002 BIOL10003 CHEM10006 MAST10011 MAST10016 PHYC10007 SPAN10001 SPAN10002
Year II: BCMB20005 BIOM20001 BIOM20002 CLAS10022 GENE20001 SPAN20020 SPAN30014
Year III: BCBM30001 BCMB30002 BCMB30010 BIOM30001 BIOM30002 PHRM30008

#### vox nihili

• National Moderator
• Great Wonder of ATAR Notes
• Posts: 5289
• Respect: +1373
##### Re: VCE Science Research Methods Question Thread
« Reply #4 on: March 26, 2019, 04:21:10 pm »
+1
Before I provide answers to the above, does anyone else want to have a crack?
MED INTERVIEW TUTORING PM to secure your place early, as they fill up quickly!

Join ATARNotes Footy Tipping

2013-15: BBioMed (Biochemistry and Molecular Biology), UniMelb
2016-20: MD, UniMelb
2019: MPH, UniMelb
Year I: BIOL10002 BIOL10003 CHEM10006 MAST10011 MAST10016 PHYC10007 SPAN10001 SPAN10002
Year II: BCMB20005 BIOM20001 BIOM20002 CLAS10022 GENE20001 SPAN20020 SPAN30014
Year III: BCBM30001 BCMB30002 BCMB30010 BIOM30001 BIOM30002 PHRM30008

#### Comet striker

• Forum Regular
• Posts: 64
• Respect: +12
##### Re: VCE Science Research Methods Question Thread
« Reply #5 on: April 07, 2019, 11:32:04 am »
0
3. All drugs have side effects
Hopefully no substantial or permanent side effects
My thoughts:
A rating of pain is a hard scale to measure. Some people may simply experience pain differently and with no control groups(as DBA said). Therefore, the study's results cannot be measured with accuracy and is unreliable. Hence, drug shouldnt be sold.
Leaves from the vine
Falling so slow
Like fragile tiny shells
Drifting in the foam
Little soldier boy
Come marching home
Brave soldier boy
Comes marching home

• Posts: 14
• Respect: 0
##### Re: VCE Science Research Methods Question Thread
« Reply #6 on: April 08, 2019, 04:51:35 pm »
0
Just some thoughts:

- Since the results are summarised and only average scores were shown, there could be data that is not compatible with the average or affected the average in a great deal but are hidden.  For example, the results of each person could be plotted on a graph and a trend line drawn in the middle, where the results are in fact zigzagged and irregular. The results should be shown more elaborativly.

- The scientist's desire to gain money could have possibly lead him to alter some results to fit his hypothesis, and it seems like he is working on his own and not in a team, so his results might not be reliable, since no other scientists were there to make sure the results are true. It is also unclear who summarised the results and put them toegther this way.

#### briv01

• Trailblazer
• Posts: 26
• Respect: +1
##### Re: VCE Science Research Methods Question Thread
« Reply #7 on: April 08, 2019, 06:24:48 pm »
0
Just a few thoughts-

1) The selection process of the participants was unspecified. The selection process may not have been random sampling and experimentor bias may have been present during sampling ( eg convenience sampling ). The participants may not be representative of the overall population meaning the study can not be generalised

2) There was no control group to contrast such results of the drug to, therefore the study is unreliable

3) As the info has emphasised that he valued money a lot and has not stated any co experimentors, the experimentor May have experimentor bias and chose to not state undesirable results to promote his drug, affecting the accuracy and validity of the results

4) people have different pain tolerates and may rate pain differently. Furthermore, the type of pain experienced varies and so this study is unreliable. An individual may consider a headache worthy of taking the drug whilst another may not. Furthermore, some participants may be prone to more pain than others ( eg if they have medical issues ). A better method would have been to register the same type of pain in an ethical manner to all participants

5) there is no mention of how the results were collected. It may have been a simple question of if the participant had experienced reduction in pain. In this case, participant bias may be present as they may feel inclined to say yes as they took the drug. This is reinforced by the small  chnage in the percentage after increasing concentrations of the drug, suggesting participant bias may have taken place- a double blind procedure would have been good.

6) the actual experiment process may not have met ethical guidelines meaning the study itself cannot be reliable. It is unsure if informed consent and withdrawal rights as well as debriefing about the drug and it’s side effects took place. Furthermore, beneficence is important to consider. It is unsure if this drug has allowed for a benefit to the research of pain medications or to the individual and can compete with other drugs.
« Last Edit: April 08, 2019, 06:39:28 pm by briv01 »

#### vox nihili

• National Moderator
• Great Wonder of ATAR Notes
• Posts: 5289
• Respect: +1373
##### Re: VCE Science Research Methods Question Thread
« Reply #8 on: April 09, 2019, 01:44:58 pm »
+6
Hopefully no substantial or permanent side effects
My thoughts:
A rating of pain is a hard scale to measure. Some people may simply experience pain differently and with no control groups(as DBA said). Therefore, the study's results cannot be measured with accuracy and is unreliable. Hence, drug shouldnt be sold.

Plenty of drugs have substantial and indeed permanent side effects. What makes a side effect acceptable or not is the benefit of the drug weighed against the side effects. For that reason, drugs that are more likely to save your life are often the kinds of drugs that have quite severe side effects. Chemotherapy tends to come to mind here. The side effects are terrible, but the results of not using the drug often (but not always) more terrible.

You're right in a sense that a pain scale isn't hugely accurate, but we're not necessarily bothered by that. As long as the results are precise and you can demonstrate a difference between the groups, then using such a scale should be fine. Pain is a subjective experience. Using a scale that acknowledges that subjectivity is ok here.
Just some thoughts:

- Since the results are summarised and only average scores were shown, there could be data that is not compatible with the average or affected the average in a great deal but are hidden.  For example, the results of each person could be plotted on a graph and a trend line drawn in the middle, where the results are in fact zigzagged and irregular. The results should be shown more elaborativly.

- The scientist's desire to gain money could have possibly lead him to alter some results to fit his hypothesis, and it seems like he is working on his own and not in a team, so his results might not be reliable, since no other scientists were there to make sure the results are true. It is also unclear who summarised the results and put them toegther this way.

1. This is a reasonable point. Normally results are reported with a measure of their spread. I didn't include that because it would have over complicated the question, but suffice to say that in a sample of 10000 participants it is highly unlikely that any outliers skewed the result. This is well beyond VCE science, but methods students might appreciate the effect that a small sample size could have on the accuracy of a data set.
2. This could be true but is reasonably uncommon. There are some issues about blinding that might be more worthwhile considering here than the possibility of research fraud. If you're interested in researcher fraud, Benford's law is an interesting concept you might enjoy reading about! Very much beyond VCE though.

Just a few thoughts-

1) The selection process of the participants was unspecified. The selection process may not have been random sampling and experimentor bias may have been present during sampling ( eg convenience sampling ). The participants may not be representative of the overall population meaning the study can not be generalised

2) There was no control group to contrast such results of the drug to, therefore the study is unreliable

3) As the info has emphasised that he valued money a lot and has not stated any co experimentors, the experimentor May have experimentor bias and chose to not state undesirable results to promote his drug, affecting the accuracy and validity of the results

4) people have different pain tolerates and may rate pain differently. Furthermore, the type of pain experienced varies and so this study is unreliable. An individual may consider a headache worthy of taking the drug whilst another may not. Furthermore, some participants may be prone to more pain than others ( eg if they have medical issues ). A better method would have been to register the same type of pain in an ethical manner to all participants

5) there is no mention of how the results were collected. It may have been a simple question of if the participant had experienced reduction in pain. In this case, participant bias may be present as they may feel inclined to say yes as they took the drug. This is reinforced by the small  chnage in the percentage after increasing concentrations of the drug, suggesting participant bias may have taken place- a double blind procedure would have been good.

6) the actual experiment process may not have met ethical guidelines meaning the study itself cannot be reliable. It is unsure if informed consent and withdrawal rights as well as debriefing about the drug and it’s side effects took place. Furthermore, beneficence is important to consider. It is unsure if this drug has allowed for a benefit to the research of pain medications or to the individual and can compete with other drugs.

1) You're quite right that the sampling method wasn't specified here. Again, I omitted this detail to simplify the question. I'm glad that you're thinking about sampling though, as it can have a big impact on the results and potentially distort them. Convenience sampling doesn't generate experimenter bias, it generates selection bias. It's a very broad term and one that isn't really necessary to know in VCE!

2) YES

3) Potentially, but lots of people value money a lot. Fraud happens but we can assume that in this case it hasn't. Any trial involving 10,000 people would have a very large team working on it.

4) You're the second person to raise the point about the pain scale. See above for my response to it. tl;dr: I disagree that it's a problem. We're interested in whether the drug reduces pain broadly. To measure it broadly is entirely appropriate.

5) Again, omitted to simplify the question but, also again, a very worthy point to make.

Overall thoughts

This was a tricky first question. I'm pleased that many of you are looking for the important elements of research, although I think some might have tried to be a little too smart with their answers and missed some of the more obvious problems. The question was deliberately designed to encourage you to do that, as this was something I struggled with a lot during VCE and certainly cost me a lot of marks over the years.

A lot of what is discussed above goes beyond VCE. This is ok. The purpose of this thread isn't to rigidly stick to the course, but to improve our understanding about research more generally so that you develop a framework for thinking about research.

Whenever we ask a question about an experiment, we need to consider what the experiment is for. In this case:

Does drug X cause a reduction in pain?

We now need to consider whether the independent and dependent variables of the experiment have been set up to answer that question. The IV is the drug and the DV is the reduction in pain, so yes we have tried to test this question. (In some cases an appropriate question at this point might be whether the original question is the right question but this is probably taking things too far).

Now that we've ticked this box, we start asking some more detailed questions about our experiment. I always like to think about the people first: do we have enough and did we select them appropriately? In the above example, I didn't go into much detail about this. But suffice to say, we have 10000 people (which is heaps) and they were randomised into three different groups. So this is good!

The next thing we need to ask is whether we have something to compare our results to. The complex answer is sort of, but for the purposes of VCE we can say the answer is no. We only compare the drug to itself, we don't compare it to a placebo (looks like the drug but no active ingredient). Without a point of comparison, how can we tell whether our drug is truly reducing pain, or whether patients' pain is only being reduced because of their "belief" in the drug? Certainly, if you give someone a tictac and tell them that it's morphine, any pain they're experiencing will likely go away. This effect is called the placebo effect. In this case we can't be sure whether our results are such because the drug works or because of the placebo effect. Therefore, it's not ethical to sell the drug.

Well done to everyone who also asked questions about ethics and about blinding of the participants. These are also important considerations, but the obvious answer here was a lack of a clear control group.
MED INTERVIEW TUTORING PM to secure your place early, as they fill up quickly!

Join ATARNotes Footy Tipping

2013-15: BBioMed (Biochemistry and Molecular Biology), UniMelb
2016-20: MD, UniMelb
2019: MPH, UniMelb
Year I: BIOL10002 BIOL10003 CHEM10006 MAST10011 MAST10016 PHYC10007 SPAN10001 SPAN10002
Year II: BCMB20005 BIOM20001 BIOM20002 CLAS10022 GENE20001 SPAN20020 SPAN30014
Year III: BCBM30001 BCMB30002 BCMB30010 BIOM30001 BIOM30002 PHRM30008