3. Evaluating Information

To create works of high quality you need high-quality information to build upon. Using for example unreliable or outdated work could lead to wrong conclusions or design decisions. To contribute to science and progress it is crucial to:   

• Evaluate the quality of information published by others before you use it
• Evaluate the quality of your own work before you publish it

Objective evaluation can be challenging, as you do not always have access to the information you need to properly assess a source. You can use the following (CRAAP Test) criteria for evaluating any type of source, bearing in mind that it differs per source how much data is available to assess its quality. 

Currency

Identifying the time of publication can help you eliminate outdated sources, or sources that do not apply to the specific timeframe you are interested in. 

• When was the information published or posted?
• Has the information been revised or updated?
• Do your need current information, or sources published within a specific timeframe?

Relevance

Determining the importance of sources in relation to your information need will prevent you from using sources that are off-topic or lack comprehensiveness. 

• Does the information relate to your topic or answer your question?
• Is the information at an appropriate level (not too elementary or advanced for your needs)?
• Are you aware of more or better sources you could make use of?

Authority

Knowledge about the people or organizations behind the sources you find can be useful to identify a certain level of expertise.

• Who is the author/publisher/source/sponsor?
• What are the author's credentials or organizational affiliations?
• What are the author’s qualifications?

Accuracy

A proper judgement of the reliability, truthfulness and correctness of sources is crucial to prevent usage of inferior sources, potentially leading to wrong conclusions or decisions.

• Is the information supported by evidence?
• Has the information been reviewed or refereed?
• Can you verify any of the information in another source or from personal knowledge?
• Are there spelling, grammar or typographical errors?

Purpose 

Knowing the reason why information exists is important to identify for example bias or greenwashing. If you suspect that such factors are at play, you should mention and/or discuss them when using the sources in your own work.

• Do the authors/sponsors make their specific intentions or purposes clear?
• Does the point of view appear objective and impartial?
• Can you determine if the goal is to for example teach, sell, entertain or persuade?
• Are there political, ideological, cultural, religious, institutional or personal biases?

Evaluation can be performed on various levels, from a quick assessment of the reliability of a news source to an in depth-analysis of published scientific research. Presented below are some commonly used methods for evaluation within the scholarly community.

• CRAAP Test: The evaluation of non-scientific sources in particular, to assess the currency relevance, authority, accuracy and purpose of these sources. A useful practice during the gathering of sources in every phase of an assignment or project.
• Critical review: A critical analysis of a research article, in which you summarize its content, discuss its strengths and weaknesses (using other sources to support your statements) and formulate a conclusion restating your overall opinion. A practice that usually comes in the form of a separate assignment.
• Peer-review: The assessment of a submitted manuscript for a research article by researchers in the same field of expertise (peers). This process is usually double-blinded, to reduce the chance of a bias in the assessment. Although peer-review is used as a term in the context of student assignments, the practice is mainly applied in the publication process for journal and conference articles. 
• Bibliometric analysis: The statistical evaluation of scientific works, making use of so-called metrics (for example the number of citations a published work received). The aim is to measure the influence of scientific publications, institutes or authors within the scientific community. Some metrics are controversial when used to determine the quality of an authors or institute’s published works, as they can be misleading. Increasingly, weighted metrics are used to evaluate works within the context of their own field. This gives a more realistic idea of the impact of a work, but it remains important not to rush to conclusions based on such a statistical evaluation.