Research updates

This is the rough text prepared for UCC College of Arts, Celtic Studies and Social Sciences Postgraduate Conference December 2014. It is mostly informal because it is an oral presentation. Some of the paragraphs are left hanging because they lead into the next slide/section of text. A pdf of slides and notes are uploaded to my academia.edu profile (link here).

Introduction

Today I’m going to give you a quick run through of my research project, now in its second year. I’m going to go through what inspired it in the first place, how things are beginning to shape up now that I am this far into the research, and give a quick outline of what (at this stage) I hope the outputs will be.

My topic is about assessing the online dissemination of cultural heritage projects, looking particularly at those with very niche or local interests: how do they operate within the context of the global forum that is the world wide web?

Starting point

One of my inspirations at the start of this project was the paper: “If you build it, will they come?” (Warwick et al. 2008)

This paper was one of the outputs of a project called LAIRAH (Log Analysis of Internet Resources in the Arts and Humanities).

It looked at log results from arts and humanities sites. Anecdotal evidence suggested that lots of these sites became very well known, but that also others were quickly forgotten about. The suggestion was that this wasn’t about academic merit or about public interest, so they wanted to find out whether they could pinpoint the reasons why some were not doing as well as others, and make suggestions to help some resources improve performance.

Some of the suggestions that this project came up with (finding a good name and writing a good description) can be done by any project with a bit of common sense and some resolve.

But there were deeper issues of expectations – people expect slick commercial interfaces, even if they are accessing small, cheaply produced resources – and projects that don’t have a large amount of funding to begin with, or that don’t have ongoing funding, are immediately at a disadvantage.

I wanted to look at how small scale cultural heritage groups function in this kind of environment.

My research case study

So I started working with the Cork Folklore Project (CFP) – an oral history group that works as a collaboration between university and the local community.

CFP was founded in 1996 and their aim is to collect stories of everyday life in Cork city and county. The Project is staffed by researchers working on training and employment schemes and operates on a not for profit basis. Over the past 18 years CFP has amassed a record of the voices and traditions of Cork City, in the past and the present, with c. 500 audio recordings stored in its archive.

CFP operates as a public oral history archive, and it has also produced books, radio programmes, websites and worked on Heritage week events and exhibitions, as well as producing an annual magazine that is freely distributed across Cork city.

Small scale digital projects from the CFP

I am looking at the digital outreach of the CFP as part of my research. These include:

  • social media outreach (some details about this in other posts here and here)
  • Cork Memory Map (some more blog posts about this here)

I am using the CFP as an example of a small, local cultural heritage group, looking at how they operate in the digital realm, and beginning to think about how this experience can be used to provide guidance and ideas for other small groups who operate within a similar context, be they a small local group, or a cultural heritage organisation that works within very limited time, personnel and financial constraints.

For today, I am going to show some results so far from the Cork Memory Map, which includes audio clips (excerpts from oral histories), photographs and transcripts pinned to relevant locations on a map of Cork city.

This is an extremely local project…often people who are not from Cork don’t understand the voices that make up this project because they have strong accents, or they use slang.

In order to combat gaps in understanding, the Memory Map has transcriptions of the excerpts that are there to help “outsiders” but it is ultimately a niche local project.

My questions are – What sort of digital audience does this attract? And how do you measure this?

Using a toolkit

To investigate this I decided to follow some guidance from TIDSR – Toolkit for the Impact of Digital Scholarly Resources

This toolkit was built by researchers in Oxford it follows on from the “If you build it, will they come?”/(LAIRAH) research (and other research on similar themes, see here for details). It provides some handy guidelines about what tools you can use, whether they are free, and what to concentrate on when you do use them.

Google Analytics – results and some problems

So, as recommended in the Toolkit, the free version of Google Analytics was implemented on the Cork Memory Map, and to look at where our users were coming from, to investigate this idea of local and global use of a niche resource. These are some of our use figures for the Memory Map over the past few months.

Picture1

One of the things the toolkit recommends is that you look at how the resource is used over a long period, and compare similar time periods (say March 2014 and March 2015). I don’t have a full year of figures yet, but (from the figures I have collected so far) the fall off in use over July and August suggests to me that there is an academic use of the site.

However, what is more confusing is the fact that I often can’t find some sustained periods when I know the Memory Mao was in use. For example, I presented a poster and the Memory Map at a large digital humanities conference in Switzerland this summer (the poster can be found here). I had a table and my laptop was set up in front of the poster; headphones were attached – so that people could listen. I was logged in to the Memory Map for c. two hours. And I can’t find this session in the metric. What am I to make of that?

In fact, TIDSR recommends not to get too bogged down in the actual number of session counts using metrics like this because….

What do page views measure?

“page views must be viewed with a grain of salt”  (Meyer 2013 in TIDSR Google Analytics tutorial)

It also said that the first question to ask is whether the results make sense, “If the answer is no, you may need to dig deeper to figure out why” (Ibid.)

For me, these figures didn’t make sense. This leads me to question how reliable it is to base search results on material from Analytics, as we don’t know what criteria have put in place to harvest these figures.

I also have to bear in mind the experience of other researchers who have used Google Tools as the basis of their research in the past, for example Jucker et al. (2012) who wrote a journal article based on results from Google’s Ngram viewer. They had to submit a significant addendum because the algorithim changed between when they had researched the paper in 2009 and when it was published live in 2012, and these hidden change meant that they could not replicate their findings (although they did manage to find a way to work around this – see here for details).

Similarly for me, Google Analytics is a black box – I don’t know how they gather these figures and I have begun to question whether I should be using it as part of my research.

So digital formats can offer the means to gather lots of nice data that seems like a great opportunity for study – but it is also a means of data deception. Algorithms, working behind the scenes on websites, unseen, can affect how empirical knowledge is created and transmitted. To counteract this,  following Wendy Hsu (2014) we need to think about

“software as infrastructure and materiality at the sites of …research”. (Hsu 2014)

Big Data needs a context

For me, thinking about how to use Big Data tools like Google Analytics – and thinking about how they impinge on my research is an important step – because these tools and the information that they produce needs to be contextualised. And ethnography is a good means of producing the context for Big Data – what Wang (2013) calls “Thick Data”.

However, given the size and scale of the projects that I am looking at, what I am really looking at is what Hsu calls “Boutique data”. With Big Data, the voice of the little person gets lost, or dehumanised, but you can use small amounts of quantitative data to trigger questions that are then investigated using qualitative methods.

Mixed methods approach for “boutique” datasets

And this approach, using mixed methods (quantitative and qualitative) is an important foundation of my research –  metrics were only ever meant to be a small part of it. I also want to look at qualitative results, particular user studies, and ethnographic interviews with interested parties.

The idea here is to tie themes and ideas from lots of different sources together using mixed methods, to try to get an holistic view of how online resources are used and how they are appreciated (or not!).

And my aim is to use the results to develop some practical outcomes, including sharing the results, insights and pitfalls with similar small and local cultural heritage groups, emerging as a set of guidelines for small organisations who wish to engage in an online world. Because these will be born out of the work of a community organisation, rather than an academic community, they will operate in tandem with things like the TIDSR toolkit, but they will have a slightly different emphasis, as the aim is not the promotion of academic careers.

On a more abstract level, I want to use the results to look at ideas about the perception of cultural heritage in general, exploring links with this and ideas of community, place, identity and value.

Value and price, cynicism and sentimentality

A character in an Oscar Wilde play (Lady Windemere’s fan) says:

 “What is a cynic? A man who knows the price of everything and the value of nothing. And a sentimentalist….. is a man who sees an absurd value in everything, and doesn’t know the market place of any single thing.”

Frequently now people working in the arts and heritage sectors are confronted by the cynics, who wish to put a monetary value on the work that they do and don’t appreciate that the idea of value could be extended beyond the economic. At the same time, there is no point in being a sentimentalist about arts and humanities projects – if people don’t appreciate them then there is a genuine reason to question whether time and resources should be ploughed into them. What I am hoping to elicit in my interviews is a sense of whether people feel these is point in engaging with local projects online, and to walk the fine line between cynicism and sentimentality.

 

References

Jucker, A. H., Taavitsainen, I. & Schneider, G. (2012). Semantic corpus trawling: Expressions of “courtesy” and “politeness” in the Helsinki Corpus. Varieng 11.
http://www.helsinki.fi/varieng/journal/volumes/11/jucker_taavitsainen_schneider/

Meyer, E. (2013). Using Google Analytics. Available online at: http://microsites.oii.ox.ac.uk/tidsr/sites/microsites.oii.ox.ac.uk.tidsr/files/2013-06-15_GoogleAnalytics.pdf

Wang, T. (2013). Big Data Needs Thick Data, Ethnography Matters. (accessed on February 20, 2014) http://ethnographymatters.net/blog/2013/05/13/big-data-needs-thick-data/

Warwick, C., Terras, M., Huntington, P., & Pappa, N. (2008). If you build it will they come? The LAIRAH study: quantifying the use of online resources in the arts and humanities through statistical analysis of user log data. Literary and Linguistic Computing, 23(1), 85–102.

 

Some more stuff about metrics

Seeing Ourselves Through Technology (free ebook) (http://jilltxt.net/?p=4086) and link to a related TedX talk (http://jilltxt.net/?p=4151)

Van Dijck, J. (2012). Facebook as a Tool for Producing Sociality and Connectivity. Television & New Media, 13(2), 160–176. doi:10.1177/1527476411415291