Which of the following should you do when interpreting and analyzing a graphic presented within text?

Working out how to interpret and then present your research material and data is probably the most creative aspect of research, but also an area where it is easiest to compromise integrity. The rules for interpretation and presentation are usually very field-specific and often unwritten. For example, no clear universally accepted standards exist to distinguish acceptable manipulation of digital images. See the box at the end of this section for guidance in this area, but if you have further questions:

• Check carefully with your supervisor, colleagues or publication editor before making any changes
• Be honest and upfront in letting others know what changes you have made.

Data interpretation and presentation is a crucial stage in conducting research, and presents three key challenges:

• Selecting which material will be used for drawing conclusions about your work
• Establishing the significance (or otherwise) of material and identifying potential weaknesses and limitations
• Deciding how to present your findings and observations.

These three challenges form the subject of this section. Before we launch into further detail, some experts talk about some of the ways in which data can be manipulated.

Media - Video

Data selection and bias

Dr. Daniele Fanelli, Research Fellow, The University of Edinburgh: In my research, there is pretty good evidence that the frequency of positive results, as opposed to results that do not support the hypothesis that was tested in the study, have been dramatically increasing over the last twenty years. The problem behind this has partly to do with probably how journals select results. Presumably they want-, they're increasingly selecting studies based on the outcome, and this in turn, however, clearly will put a pressure on researchers to get those positive results, to get the publication.

So it's quite well acknowledged that the temptation, and it's a temptation very few researchers resist, including probably myself, is that once you have your data set, you will look for the kind of patterns you suspect are there. The tragedy, if you like, nowadays is that you have so many ways to do that, so many statistical techniques at your disposal, and so many technologies that allow you to be more and more clever at mining your data for results, that the risk is obviously that then you end up just seeing whatever you wanted to see in the first place, without actually being anything there.

Adding to that risk is the fact that usually when you do research, you're not only looking and getting one result, but you're looking at several different aspects of a problem. Then, if you then only choose some for publication, you will only discuss some and ignore the rest, then again the risk is that you're unduly selecting what is the evidence. The extent to which this is perfectly legitimate, or it is an unconscious form of bias, or it is even a dishonest practice, is controversial.

What steps can researchers take to reduce the risk of bias?

Dr. Daniele Fanelli: The way out of this, generally speaking, is to be transparent about what you did. I'm not naive enough to think that this is going to be the whole story, because publication space in journals is limited, and you will never be allowed to tell precisely everything that you have done. So in part, the system does need other ways also to allow researchers to make fully public their data, you know, all the results they obtained, etc.

Again the ideal to follow, I think, in any kind of research, is as much as possible, be transparent of the whole procedure. What were your original research questions, how you collected the data, what eventually was the data that went into this particular study, and so on.

Fabrication and falsification

Dr. Melissa S. Anderson, Professor of Higher Education, University of Minnesota: If you think about it, what's the most important aspect of research and new knowledge? It's that it's right, it's correct, it's true. Now, it may be wrong because of a mistake or an error, and if that happens, you go back and you fix your mistake, but if it's wrong because someone has intentionally introduced false information, that's inexcusable. That's exactly what happens in the case of falsification or fabrication. If, in fact, somebody introduces false information into the research record, it can be there for a long time, and people may be making bad decisions on the basis of wrong information.

Which has the greatest impact?

Dr. Nick Steneck, Director of the Research Ethics and Integrity Program of the Michigan Institute for Clinical and Health Research, University of Michigan: The assumption has been that falsification, fabrication and plagiarism or, kind of, the very serious offences, are the ones that we ought to pay the most attention to. Those are serious offences. They need to be investigated when they occur. They actually, in my view, don't have the biggest impact on the research record, because although they're more common than we thought, they still are few in number. It's other practices, such as bias and conflict of interest, kind of small manipulation of the data, improper authorship, those sorts of things that ultimately turn out having the biggest impact on the research record, and then as we use that research record, actually having the biggest impact on society's use of research.

Since you want your work to turn out to be important and well-received, it can be tempting to manipulate results. In fact, studies have suggested that misinterpretation and over-interpretation may be the most significant sources of error in the research record and of bad advice for policy makers (Al-Marzouki, 2005). In this section, we consider the different stages involved in data interpretation/presentation, and the temptations involved at each one.

Reference - Al-Marzouki, 2005
Al-Marzouki, S. et al. (2005) 'The effect of scientific misconduct on the results of clinical trials: a Delphi survey', in Contemp Clin Trials 26(3): pp.331–7.

Interpreting the ideas of others

Analysis in humanities disciplines generally involves engaging with the texts and ideas of others to define and discover themes and issues. The researcher enters into an ongoing 'conversation' to contribute to the furthering of knowledge about ourselves, our history and our cultural milieu. Two potential problems arise here which are partly due to the complexity of the phenomena being studied:

• That you will over- or misinterpret the ideas of others
• That misinterpretation becomes misrepresentation of the ideas of others, used to build up your own argument on false grounds.

The main problem is drawing the line between creative interpretation and misrepresentation. It is important to reflect on the implications of this for your own work.

The typical arts researcher works in a significantly different way. The aim is usually to produce artefacts or creative work (e.g. texts, media, performances) where the boundaries between 'data', research material, analysis and one's own creative powers are in tension. Research in these kinds of projects means that your own or others' creativity is often the object of the research. In arts research, then, what counts as 'data' or 'research material' is more ambiguous than in most other disciplines. This means you should think carefully about what counts as 'data' in your work when reading through the rest of this section.

Selecting data

As a first step, you need to determine which data are suitable for further analysis and which should be discarded. This is sometimes referred to as quality assurance and/or quality control.

In the following section, consider a straightforward request regarding data that appears to deviate from the expected trend. Make a note of your ideas, then move on to our feedback.

The scenario

You have been analysing a set of newspaper articles on the portrayal of contemporary poets. You have discovered that one particular arts correspondent in a leading Sunday newspaper is both female and very supportive of women poets. This undermines your own argument that women poets are mostly ignored and that when they are covered in the media, it is mostly in very negative terms.

These 'positive' articles constitute around 5% of the total number of articles. Your supervisor suggests you don't include them, justifying the exclusion on the grounds that you could reframe your research question to only focus on weekday newspapers. What should you do?

How will you respond to your supervisor's suggestion?

Feedback:
Some of the most common questionable research practices (QRPs) centre on the analysis and interpretation of data. In this case, it may have been tempting to ignore the data which bucked your expectations, made your own argument more difficult, or which you found difficult to align with your own theoretical or personal position. Responsible researchers should have solid and unbiased justification for ignoring data which presents such problems.

Researchers should be mindful of the bias that their perspectives and goals bring to the research setting. This can be a bias toward our own ideas, career pressures or external pressures (for example, funding agencies) that influence our decision-making. Being conscious of these influences is a first step towards addressing them.

Establishing significance and limitations

Another important aspect of interpreting and presenting findings is estimating their significance. For example, consider the following questions:

• Are the participants in my dataset representative of the population?
• Are my findings applicable in the real world?
• Does my theory properly account for the phenomena I am attempting to interpret?
• Am I collecting data in the appropriate range of conditions?
• Are my findings significant enough to warrant publication as a new and unique contribution to the field?

It is important to recognise the limitations of any research study and to interpret the findings within these constraints. In the following section, you will be presented with a list of research components and a list of potential limitations. For each component, make a note of the limitation you think it might cause, then continue to our suggested answers.

Research components:

• Funding and time constraints
• Number of participants
• Type of participants
• Data collection setting
• Analysis methods or models
• Your expertise and abilities

Potential limitations:

• Can bias or limit findings and may require collaboration or education
• Can limit applicability of the findings to other populations
• Can limit the scope of the research project or study
• Can bias findings due to influence of environment on research participants
• Can limit confidence in the findings
• Can bias or limit the findings due to assumptions used to develop the techniques/tools

Our suggestions:

• Funding and time constraints can limit the scope of the research project or study.
• Number of participants can limit confidence in the findings.
• Type of participants can limit applicability of the findings to other populations.
• Data collection setting can bias findings due to influence of environment on research subject.
• Analysis methods or models can bias or limit the findings due to assumptions used to develop the techniques/tools.
• Your expertise and abilities can bias or limit findings and may require collaboration or education.

Presenting data

A final aspect of data interpretation involves making decisions on how to present and explain your findings to others. Data presentation overlaps into the subjects of reporting and publishing, covered elsewhere in this course – however, we mention it here because some of the decisions you make in relation to presentation will be critical to your analysis.

A great deal of freedom and creativity can be employed in data presentation in order to convey information that seems to suggest a particular conclusion. In the following section, consider the alternative versions of the same information, and in each case reflect on the significance of this difference in relation to interpretation.

Example 1 of 3

Manipulation:

'...it is clear that the influence of Bloggs and Smith... has been considerable'.

Original:

'Although their work is now universally agreed by scholars to be unreliable, it is clear that the influence of Bloggs and Smith in the past has been considerable.'

Our thoughts:
If you use citations, you need to consider the full context in which they appear.

Example 2 of 3

Manipulation:

Original:

Our thoughts:
Inserting, removing or enhancing certain elements of an image is a form of misinterpretation and would be considered falsification.

Example 3 of 3

Manipulation:

Original:

Our thoughts:
Manipulation of the vertical range can exaggerate change.

Whilst these are relatively simple examples, it is clear that the tools available to researchers open the door to a variety of manipulations that can overstate the significance or underplay the limitations of findings. Data interpretation and presentation raise many challenges for responsible behaviour.

To minimise these challenges it is important for you to:

• Recognise your own bias and how it might influence your choices
• Represent your data objectively
• Identify and acknowledge the limitations of your study.

All meanings, we know, depend on the key of interpretation.

George Eliot (1876)

In surveys, approximately 14% of researchers admit knowing colleagues who have falsified data (Fanelli, 2009).

Reference - Fanelli, 2009
Fanelli, D. (2009) 'How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data', in PLoS ONE 4(5): e5738.

How do you analyze graphic information in a text?

When including graphics in a document what does interpreting mean?

What best determines if a graphic presentation is effective?

Which type of bar chart would work best for tracking progress toward completing a series of events over time?