Follow us: Twitter Behance Linked In Facebook

iD30 Blog

All the latest news, events and projects from iD30...

Some pitfalls to avoid when researching

If you have ever had to research anything, whether it be for a blog, school project, training or just personal interest, you may have accidentally picked up information that is wrong.

You may not have known it was wrong at the time. But later on someone made sure you knew that you were wrong. They might have made a big deal out of it or been really understanding.

Either way you need to make sure you don’t get in that situation again because it affects your credibility, and more importantly it means you could have picked up the wrong information in other, more important places too.

For example, you could be on a new diet that promises drastic weight loss. But what you don’t realise is that the weight you will lose on this diet is muscle and as soon as you stop following the diet you put on fat easier.

So, what is making you pick up wrong information?

I’m not a psychologist, far from it in fact. So I can’t explain all the things at play when we judge whether some information is wrong or right.

But what I can do is suggest some things to be aware of when learning and researching.

Avoid confirmation bias.

Confirmation bias is when a person favours information that confirms their preconceptions.

Whichever side a person is rooting for will always look like it has better evidence to support it, even if this is not the case from a nonbiased standpoint.

Either knowingly or not, information to support the other side of the argument is given less significance than it deserves or is ignored entirely.

Take Mathew. He has always sworn by his favourite chocolate brand, Fudgy Comets. But his best friend, Charlotte, buys the cheaper Coco Moons.

Mathew decided he is going to try and convince Charlotte to buy the superior brand of chocolate. After all, he just wants her to have the best chocolate.

“They just taste way better than anything else, I’m telling you!” Mathew starts his convincing argument.

“But they both list the same ingredients on the back” Charlotte provides some valid evidence.

“Hmm, it can’t be that, oh I know! We should search the internet for answers” Mathew ignores Charlotte’s evidence.

Mathew did a something search for ‘Why Fudgy Comets are the best chocolates’

“See, it says here that they are made in the best way which is the reason they taste the best.” Mathew found evidence to support his favourite brand.

Later on, Mathew found out his uncle works at a Fudgy Comets factory and decides to take Charlotte for a tour. Upon arrival Mathew sees something and his world is turned upside down.

The sign for the factory says ‘Fudgy Comets and Coco Moons factory’

After all that, it turns out that they are both the best… if Mathew had avoided confirmation bias, he would have saved himself the embarrassing moment.

Confirmation bias when searching

So if a person is searching for information to back up their point in an argument, it is confirmation bias. What should happen is searching for information which is correct.

If you are using a search engine, you can help it give you less biased results by searching more neutrally.


“Oh, I knew it! I knew dinosaurs were fake. See, I searched here and all these websites prove it.”

How could you argue with that? The top results are all about dinosaurs being fake. Though, you have to remember it is less likely that evidence for the existence of dinosaurs will appear for the keyword ‘fake’.

Using leading keywords will narrow your search results down to just one side of the argument. You can’t get a clear picture without understanding both sides of the argument.

Confirmation bias with memory

Confirmation bias can also happen when in your memory too. And it can be just as damaging to your impartial knowledge.

In memory, confirmation bias occurs by recalling only pieces of information which support your favoured standpoint. This ‘cherry-picking’ of memories will reinforce and strengthen only the ones recalled… which, with confirmation bias involved, are only one side of the argument.

Your memory might be bias corrupt!

This might be because those memories made you right and being right made you feel good, which is everyone’s goal right? No one wants to remember the things that make them feel bad, including those which make you wrong.

So even if at one point in time you researched the subject from neutral searches and found evidence for both sides, confirmation bias could mean you forget the other side of the argument over time.

If you are just recalling some information you have previously researched or had experience with you could also have favoured the experiences in your memory that confirm your initial belief.

How do I avoid confirmation bias?

Confirmation bias can happen with any subject. If you don’t search from a neutral standpoint you risk being victim to confirmation bias. Everybody has done it at some point and it is easier to avoid if you are aware of it.

So the checklist is:

- Be the least stubborn you can.

- Ask neutral questions.

- Be prepared to be wrong.

- Don’t rely on memories.

Being aware of what confirmation bias is will be a big help towards avoiding it. Maybe you will revisit things you learnt long ago to make sure the information was correct? Maybe you won’t? Just make sure the big things are correct so you don’t get on the wrong diet or something.

Check your sources

Sources, references, a bibliography; these are all things you should be checking to see if the place you are getting your information from has credible sources.

What do I mean by credible sources?

Well obviously peer reviewed scientific sources are your safest bet. But there won’t always be research carried out scientifically and then peer reviewed on every subject.

Just to make sure we are all on the same page. Science is a method for discovery where a hypothesis is set, then through testing and research is proven right or wrong through the use of evidence. The hypothesis is then adjusted and more testing is carried out to prove the new hypothesis right or wrong.

It doesn’t stop there though, if a hypothesis is proven to be right, another scientist (peer) will attempt to replicate the test results the same way and prove the hypothesis correct. This is usually done a few times and so you can understand why they are the best sources of information.

Anyone can be a scientist if they know the correct scientific method.

This high quality of source is not available for everyday subject discussions, so what would your next best thing be?

Try to find multiple sources that all agree and seem to have some sort of unbiased reasoning.

Without a peer reviewed scientific study, I wouldn’t recommend staking your reputation on any big claims no matter the source. This doesn’t really apply to things like news stories, but you must have heard of claims that some news companies over exaggerate stories for ratings.

Caution is your weapon of choice here, well more of a shield. You shouldn’t just jump in and agree with someone else’s claims without understanding the reasoning used. Even your own reasoning will be wrong sometimes.

Being careful with the importance you give evidence from sources that aren’t fully trustworthy should protect you from diving head first onto what you were told was a comfy bed only to find out that it is a 30 foot drop masked with a sheet.

Correlation or causation

Knowing the difference between correlation and causation can mean that you suspect something isn’t right when you look at this chart below

Does cheese make people spend more on their pets? Do happy pets emit an aura which gives humans a craving for cheese? The answer is nope.

This is just a correlation I chose from this website: http://tylervigen.com/discover give it a try, you can find some pretty funny correlations.

But to get back to the point, a correlation between two sets of information does not mean that either is linked. If they were linked it would be called causation.

So when you are out in the wild web collecting evidence, be sure to scrutinise graphs that use the correlation as conclusive evidence to support their point.

However, correlations could actually have a link between each other that you or they have not mentioned, so don’t write off all correlations straight away.

Human superstition

This is basically when humans see correlations as causation without any evidence to support their conclusion.

You’ve all heard of crossing your fingers? Well, I’m afraid there is no evidence to support that crossing your fingers might somehow channel some power into changing the results of something you cannot affect.

Now now, I’m not saying that superstitions should go away, I have some of my own I enjoy. All I am saying is that they should not be taken seriously when it comes to things that matter.

For example, there are some superstitions which are about counting birds. Depending on the number of birds you get in one place you are meant to get some luck or money or sadness etc. But I would hope that you wouldn’t count the amount that is supposed to give you gold, quit your job and then charge out with a pickaxe to go searching.

I am hoping that now that you are aware of some of these concepts you’ll avoid some situations which could be quite embarrassing.

What to look out for when researching a topic

If you have ever had to research anything, whether it be for a blog, school project, training or just personal interest, you may have accidentally picked up information that is wrong.

You may not have known it was wrong at the time. But later on someone made sure you knew that you were wrong. They might have made a big deal out of it or been really understanding.

Either way you need to make sure you don’t get in that situation again because it affects your credibility, and more importantly it means you could have picked up the wrong information in other, more important places too.

For example, you could be on a new diet that promises drastic weight loss. But what you don’t realise is that the weight you will lose on this diet is muscle and as soon as you stop following the diet you put on fat easier.

So, what is making you pick up wrong information?

I’m not a psychologist, far from it in fact. So I can’t explain all the things at play when we judge whether some information is wrong or right.

But what I can do is suggest some things to be aware of when learning and researching.

Avoid confirmation bias.

Confirmation bias is when a person favours information that confirms their preconceptions.

Whichever side a person is rooting for will always look like it has better evidence to support it, even if this is not the case from a nonbiased standpoint.

Either knowingly or not, information to support the other side of the argument is given less significance than it deserves or is ignored entirely.

Take Mathew. He has always sworn by his favourite chocolate brand, Fudgy Comets. But his best friend, Charlotte, buys the cheaper Coco Moons.

Mathew decided he is going to try and convince Charlotte to buy the superior brand of chocolate. After all, he just wants her to have the best chocolate.

“They just taste way better than anything else, I’m telling you!” Mathew starts his convincing argument.

“But they both list the same ingredients on the back” Charlotte provides some valid evidence.

“Hmm, it can’t be that, oh I know! We should search the internet for answers” Mathew ignores Charlotte’s evidence.

Mathew did a something search for ‘Why Fudgy Comets are the best chocolates’

“See, it says here that they are made in the best way which is the reason they taste the best.” Mathew found evidence to support his favourite brand.

Later on, Mathew found out his uncle works at a Fudgy Comets factory and decides to take Charlotte for a tour. Upon arrival Mathew sees something and his world is turned upside down.

The sign for the factory says ‘Fudgy Comets and Coco Moons factory’

After all that, it turns out that they are both the best… if Mathew had avoided confirmation bias, he would have saved himself the embarrassing moment.

Confirmation bias when searching

So if a person is searching for information to back up their point in an argument, it is confirmation bias. What should happen is searching for information which is correct.

If you are using a search engine, you can help it give you less biased results by searching more neutrally.

“Oh, I knew it! I knew dinosaurs were fake. See, I searched here and all these websites prove it.”

How could you argue with that? The top results are all about dinosaurs being fake. Though, you have to remember it is less likely that evidence for the existence of dinosaurs will appear for the keyword ‘fake’.

Using leading keywords will narrow your search results down to just one side of the argument. You can’t get a clear picture without understanding both sides of the argument.

Confirmation bias with memory

Confirmation bias can also happen when in your memory too. And it can be just as damaging to your impartial knowledge.

In memory, confirmation bias occurs by recalling only pieces of information which support your favoured standpoint. This ‘cherry-picking’ of memories will reinforce and strengthen only the ones recalled… which, with confirmation bias involved, are only one side of the argument.

Your memory might be bias corrupt!

This might be because those memories made you right and being right made you feel good, which is everyone’s goal right? No one wants to remember the things that make them feel bad, including those which make you wrong.

So even if at one point in time you researched the subject from neutral searches and found evidence for both sides, confirmation bias could mean you forget the other side of the argument over time.

If you are just recalling some information you have previously researched or had experience with you could also have favoured the experiences in your memory that confirm your initial belief.

How do I avoid confirmation bias?

Confirmation bias can happen with any subject. If you don’t search from a neutral standpoint you risk being victim to confirmation bias. Everybody has done it at some point and it is easier to avoid if you are aware of it.

So the checklist is:

· Be the least stubborn you can.

· Ask neutral questions.

· Be prepared to be wrong.

· Don’t rely on memories.

Being aware of what confirmation bias is will be a big help towards avoiding it. Maybe you will revisit things you learnt long ago to make sure the information was correct? Maybe you won’t? Just make sure the big things are correct so you don’t get on the wrong diet or something.

Check your sources

Sources, references, a bibliography; these are all things you should be checking to see if the place you are getting your information from has credible sources.

What do I mean by credible sources?

Well obviously peer reviewed scientific sources are your safest bet. But there won’t always be research carried out scientifically and then peer reviewed on every subject.

Just to make sure we are all on the same page. Science is a method for discovery where a hypothesis is set, then through testing and research is proven right or wrong through the use of evidence. The hypothesis is then adjusted and more testing is carried out to prove the new hypothesis right or wrong.

It doesn’t stop there though, if a hypothesis is proven to be right, another scientist (peer) will attempt to replicate the test results the same way and prove the hypothesis correct. This is usually done a few times and so you can understand why they are the best sources of information.

Anyone can be a scientist if they know the correct scientific method.

This high quality of source is not available for everyday subject discussions, so what would your next best thing be?

Try to find multiple sources that all agree and seem to have some sort of unbiased reasoning.

Without a peer reviewed scientific study, I wouldn’t recommend staking your reputation on any big claims no matter the source. This doesn’t really apply to things like news stories, but you must have heard of claims that some news companies over exaggerate stories for ratings.

Caution is your weapon of choice here, well more of a shield. You shouldn’t just jump in and agree with someone else’s claims without understanding the reasoning used. Even your own reasoning will be wrong sometimes.

Being careful with the importance you give evidence from sources that aren’t fully trustworthy should protect you from diving head first onto what you were told was a comfy bed only to find out that it is a 30 foot drop masked with a sheet.

Correlation or causation

Knowing the difference between correlation and causation can mean that you suspect something isn’t right when you look at this chart below

Does cheese make people spend more on their pets? Do happy pets emit an aura which gives humans a craving for cheese? The answer is nope.

This is just a correlation I chose from this website: http://tylervigen.com/discover give it a try, you can find some pretty funny correlations.

But to get back to the point, a correlation between two sets of information does not mean that either is linked. If they were linked it would be called causation.

So when you are out in the wild web collecting evidence, be sure to scrutinise graphs that use the correlation as conclusive evidence to support their point.

However, correlations could actually have a link between each other that you or they have not mentioned, so don’t write off all correlations straight away.

Human superstition

This is basically when humans see correlations as causation without any evidence to support their conclusion.

You’ve all heard of crossing your fingers? Well I’m afraid there is no evidence to support that crossing your fingers might somehow channel some power into changing the results of something you cannot affect.

Now now, I’m not saying that superstitions should go away, I have some of my own I enjoy. All I am saying is that they should not be taken seriously when it comes to things that matter.

For example there are some superstitions which are about counting birds. Depending on the number of birds you get in one place you are meant to get some luck or money or sadness etc. But I would hope that you wouldn’t count the amount that is supposed to give you gold, quit your job and then charge out with a pickaxe to go searching.

I am hoping that now that you are aware of some of these concepts you’ll avoid some situations which could be quite embarrassing.

STAY UP TO DATE...

Get the latest goodies and special treats from the iD30 camp