Category Archives: Method

Beyond survey design: take survey data to the next level

Posted on by

by Carolyn Doi
Education and Music Library, University of Saskatchewan

You’ve designed a survey, found the right participants, and waited patiently while responses come streaming in. The initial look at responses can be thrilling, but what happens next? I’ve used questionnaires as a data collection technique, and made the mistake of thinking the work is over once the survey closes. Kelley, Clark, Brown and Sitzia warn us about treating survey research as a method requiring little planning or time:

“Above all, survey research should not be seen as an easy, ‘quick and dirty’ option; such work may adequately fulfil local needs… but will not stand up to academic scrutiny and will not be regarded as having much value as a contribution to knowledge.”1

Let’s consider some steps to explore once data collection has been completed.

1) Data cleaning and analysis
Raw survey data is usually anything but readable. It takes some work to transform results into meaningful and shareable research findings. First of all, familiarize yourself with some of the relevant terminology, before moving on to actually working with the data. Before touching the dataset, you’re going to need to create four worksheets, one for raw data, one for cleaning in progress, one for cleaned data, and one for data analysis. Each worksheet shows a stage in the process, which will allow you to backtrack, or find errors. If you haven’t taken a stats class recently, I like this introductory Evaluation Toolkit, which clearly describes the processes of cleaning, tabulation, and analysis for both quantitative and qualitative data.

2) Visualization and reporting
Consider data visualization to bring your survey data to life, but remember to choose a visualization tool that makes sense for the data you’re trying to represent. The data visualization catalogue is a handy tool to learn more about the purpose, function, anatomy, and limitations of a wide range of visualizations. It includes links to software and examples of each visualization. There are lots of free or inexpensive programs to help create visualization including Microsoft excel, Google sheets, or Tableau Public. If you’re looking for some inspiration, take a browse through the stunning work of Information is Beautiful for ideas.

Likely you will want to share the outcomes of your research, either at your institution or in a paper or presentation. Kelley, Clark, Brown, and Sitzia provide a great checklist of information to include when reporting on any survey results, including research purpose, context, how the research was done, methods, results, interpretation, and recommendations.2 Clarity and transparency in the research process will help your audience to better understand and evaluate the research and its applicability to their context.

3) Data preservation and access
Consider an open data repository such as the Dataverse Project to make your data discoverable and accessible. Sharing your data comes with benefits such as “web visibility, academic credit, and increased citation counts.” You may also be required to archive your data to satisfy a data management plan or grant funding requirements, such as those from the Tri-Council. When archiving in a repository, remember to share your data in an accessible file format, and include accompanying files such as a codebook, project description, survey instrument, and outputs such as the associated report or paper. As a rule of thumb, aim to provide enough documentation that another researcher would be able to replicate your study. A dataset is a publication that you can cite in your CV, ORCID profile, in a paper, or presentation. Doing so is a great way encourage others to learn about your research or to build on your research project.

Getting your hands dirty and working directly with survey data is where you’ll be able to explore and eventually tell a compelling story based on your research. Be curious, persistent, and enjoy the process of research discovery!

1KATE KELLEY, BELINDA CLARK, VIVIENNE BROWN, JOHN SITZIA; Good practice in the conduct and reporting of survey research, International Journal for Quality in Health Care, Volume 15, Issue 3, 1 May 2003, Pages 261–266, https://doi.org/10.1093/intqhc/mzg031

2Ibid. p. 265.

This article gives the views of the author(s) and not necessarily the views of the Centre for Evidence Based Library and Information Practice or the University Library, University of Saskatchewan.

Futures Studies: What is it, and how can it be ‘evidence-based’ research?

Posted on by

by Tegan Darnell
Research Librarian
University of Southern Queensland, Australia

In March 2015, I started as a student in the Doctorate of Professional Studies (DPST) program. I wanted to find out why librarians are ‘doing’ information practice so far behind what is relevant in the current information environment. Obviously, we are all at different places and have different strengths in regards to our professional practice, but generally, as a group, librarians are, well, behind the information use of our clientele. Just admit it.

Scholarly communication has been transformed. The world in which information professionals operate has been disrupted, and embracing these changes allows for a much broader scope for the roles we play. I wanted, really, to shake things up. After reading tonnes of the literature, debating with myself, and arguing with the DPST Program Director about how I was going to address the problem, I was introduced to causal layered analysis (CLA).

CLA is a ‘futures studies’ methodology which was introduced by Sohail Inayatullah in 1998. The original paper can be found here. Professor Inayatullah is a practitioner of futures studies, the interdisciplinary study of postulating possible, probable, and preferable futures. But how can this possibly be scientific? I mean, how can it be possible to collect evidence from a future that hasn’t happened yet? It is a paradox which has not been ignored by practitioners.

Futures studies is a growing transdisciplinary field which has embraced such fields as systems thinking, education, hermeneutics, macrohistory, sociology, management, ecology, literature, ethics, philosophy, planning and others. It is an integrated field ‘with many lines of inquiry weaved together’ to create a complex whole (Ramos 2002).
The discipline uses a systematic and pattern-based approach to analysing the sources, patterns, and causes of change and stability in the past (history, economics, political science) and present (sociology, economics, political science, critical theory) in an attempt to develop foresight and determine the likelihood of future events and trends.

De Jouvenel (1965), an early futures theorist, likened forecasting or ‘the art of conjecture’ to the science of the meteorologist. Weather forecasts can be prepared reasonably accurately for each of the next few days. A forecast for more than a month in advance can be based on patterns, such as normal temperatures and precipitation, and other factors which may affect these in relation to the average. There is no way for a meteorologist to, with any certainty, say what the minimum and maximum temperature and precipitation levels on a particular day one month in the future will be. The meteorologist may, however, be able to say that it is likely that we will have above average rainfall, or that temperatures will be below average. A futures study considers patterns of power and privilege, social institutions, religion, and history, to postulate possible future states that may recur.

The causal layered analysis method, specifically, is not used to predict the future, but rather to create ‘transformative spaces for the creation of alternative futures’ (Inayatullah 1998). It is an action research method for increasing the probability of a preferred future by examining the problems, systems, worldviews and myths of the present. It is about human agency – using what we know about the past, to act in the present, in order to create/shape the future we would like to see.

Just imagine librarians in your own workplace, critically examining their own current problems, existing systems, worldviews, and subconscious myths and mythologies, to transform their practice. Perhaps you are starting to see why I decided to use the causal layered analysis method in my research.

I’m currently preparing for Confirmation of Candidature. Professor Inayatullah has agreed to be one of my supervisors. I think that makes me a *ahem* futures theorist.

If you are interested in finding out more I recommend this article by Professor Inayatullah on Library Futures published in The Futurist magazine.

References:

Inayatullah, S 1998, ‘Causal layered analysis: Poststructuralism as method’, Futures, vol. 30, no. 8, pp. 815–829.

De Jouvenel, B 1965, The Art of Conjecture, Trans. by Nikita Lary. Weidenfeld and Nicholson, London.

Ramos, JM 2002, ‘Action Research as Foresight Methodology’, Journal of Futures Studies, vol. 7, no.1, pp. 1-24.

This article gives the views of the author(s) and not necessarily the views of the Centre for Evidence Based Library and Information Practice or the University Library, University of Saskatchewan.