Citation Metrics & the Freedom to Share

Citation Metrics probably aren’t the most exciting thing you’ll hear about this week but they’re incredibly important if you’re an academic, or if you’re concerned about the freedom of knowledge. In the UK the performance of departments in universities is assessed based on an audit that occurs every five or so years called the Research Assesment Exercise (RAE). The result of the RAE translates into how much government money that department will recieve until the next RAE.
So far the RAE has worked through a system of peer review. Every department submits a certain amount of articles
from various staff members along with a statement about the make-up of the department and other academics in the country are assigned to review the work and rank it. At the end all the ranks are tallyed up and a grant is worked out. The whole process can be rather expensive if you consider the cost of administration and dealing with staff having to be taken off their normal schedules to evaluate their peers.
So, bring in the citation metrics. Whenever someone writes an article and submits it to an academic journal it also becomes indexed by a citation index, the best known of these being the ‘Web of Science’ from Thomson Reuters. This database keeps track of which articles cite which articles and so provides a massive resource for ranking academic work based on how many people considered it worth citing. From these individual rankings academics can be valued based on how many other articles have cited their work. This gives them a number value which is often consulted when they apply for jobs. Departments routinely look up their applicants on the index to see how influential their work has been. If they are ranked highly it’s more likely the department will recieve more audit money.
The government has already expressed interest in the idea of moving from an audit based on peer review to one based on the data provided by the citation index. The massive costs required to run a peer reviewed audit certainly makes an audit based on mining already compiled data rather enticing.
To get to the point consider this scenario. Due to certain political aspirations in line with the Creative Commons movement, I feel that if I were to publish my work, it should be done openly, available to anyone to utilise. Most academic journals require you either pay for access to the content, or that you are a member of an institution that pays for your access. If you’re not an academic or willing to fork out money, a massive chunk of intellectual endeavour is closed off to you. If I want to publish outside this system I can, I’m totally free to give my work away for free however I choose. However these articles will never be indexed and so never count towards my ‘value’ as an academic. I shoot myself in the foot for my principles.
There is of course the option of a book, which is my current contingency plan. Over the course of my thesis I’ll have to lock my work away into those closed systems to secure some vague semblance of value. However at the end I can recompile these articles into a book (which i wanted to do anyway) and give that away for free as a seperate piece. As long as I can find a publisher willing to just do physical distribution and be happy to distribute free pdfs I should be set. Although not all publishers are a part of the index either…

Citation Metrics probably aren’t the most exciting thing you’ll hear about this week but they’re incredibly important if you’re an academic, or if you’re concerned about the freedom of knowledge. In the UK the performance of departments in universities is assessed based on an audit that occurs every five or so years called the Research Assesment Exercise (RAE). The result of the RAE translates into how much government money that department will recieve until the next RAE.

So far the RAE has worked through a system of peer review. Every department submits a certain amount of articles from various staff members along with a statement about the make-up of the department and other academics in the country are assigned to review the work and rank it. At the end all the ranks are tallyed up and a grant is worked out. The whole process can be rather expensive if you consider the cost of administration and dealing with staff having to be taken off their normal schedules to evaluate their peers.

So, bring in the citation metrics. Whenever someone writes an article and submits it to an academic journal it also becomes indexed by a citation index, the best known of these being the ‘Web of Knowledge’ from Thomson Reuters (a closed system similar to Google Scholar). This database keeps track of which articles cite which articles and so provides a massive resource for ranking academic work based on how many people considered it worth citing. From these individual rankings academics can be valued based on how many other articles have cited their work. This gives them a number value which is often consulted when they apply for jobs. Departments routinely look up their applicants on the index to see how influential their work has been. If they are ranked highly it’s more likely the department will recieve more audit money.

The government has already expressed interest in the idea of moving from an audit based on peer review to one based on the data provided by the citation index. The massive costs required to run a peer reviewed audit certainly makes an audit based on mining already compiled data rather enticing.

To get to the point consider this scenario. Due to certain political aspirations in line with the Creative Commons movement, I feel that if I were to publish my work, it should be done openly, available to anyone to utilise. Most academic journals require you either pay for access to the content, or that you are a member of an institution that pays for your access. If you’re not an academic or willing to fork out money, a massive chunk of intellectual endeavour is closed off to you. If I want to publish outside this system I can, I’m totally free to give my work away for free however I choose. However these articles will never be indexed and so never count towards my ‘value’ as an academic. I shoot myself in the foot for my principles.

There is of course the option of a book, which is my current contingency plan. Over the course of my thesis I’ll have to lock my work away into those closed systems to secure some vague semblance of value. However at the end I can recompile these articles into a book (which i wanted to do anyway) and give that away for free as a seperate piece. As long as I can find a publisher willing to just do physical distribution and be happy to distribute free pdfs I should be set. Although not all publishers are a part of the index either…

2 thoughts on “Citation Metrics & the Freedom to Share

  1. You’re also forgetting the problem with all so-called metrics – it’s an indirect measure of what you’re actually trying to find out, i.e. the quality of the academic’s work. I’m concerned that this system implies more citations = good, with no consideration for the context or quality of those citations. In fact, I just get concerned whenever the word ‘metric’ is mentioned.

    • Indeed one simple issue is the article that is frequently cited for being so incredibly wrong is classified as fantastic due to the amount of citation. In more general terms it is of course difficult to make any sort of objective judgement on quality of academic work. The peer review system used up till now is in no way objective either however, as your comment implies, metrics are often taken to be more objective judgements of quality and therefore truth, which of course is nonsense.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s