2026-01-12 lecture
Course overview, describing what bibliometrics and scholarly communication entails. Counts as a tech credit for MI program. Offered hybrid. Last two weeks of class will not be in the regular classroom Rowe 3089. Class period includes lecture, labs, and work sessions, and office hours. Taught by Geoff and Maddie, and Summer is our TA. Bibliomagician, a forum? for bibliometric practitioners.
ScholComm: scholarly communication.
Lectures will by hybrid. Tutorials starting in week 4, slides and other materials posted in brightspace, textbook is online.
Different techs will be used. Bring laptop to the tutorial sessions.
Part 1 introduction and history. Part 2 of the class is analyzing and evaluating research, measuring output and impact. IWB conference around March 2nd. Part 3, improving research systems.
What is a research intelligence librarian?
5 assignments: article review, research plan, network analysis, research evaluation, research report (and participation). All assignments are individual, not group work. For assignment 1, be sure to pick an article that uses bibliometric methodology, not theoretical. Book a meeting to discuss topic for the research plan and project/report.
Follow the course schedule as defined in brightspace, not what's listed on the front page of the textbook website.
Bibliometrics: applying statistics and maths to publication and reading of books and documents. Publication counts, collaboration analysis, venue analysis, keywords and topic trends, geographic and institute differences, disciplinary differences, funding acknowledgements, etc.
Related terms: scientometrics, informetrics, altmetrics, webometrics, science of science, research on research, metaresearch, and many more. Often quantitative, different focuses/methods. Lots of overlap.
Why: to understand, to evaluate and improve, to make decisions. Don't use quantitative evaluations alone when making decisions, they are not the whole picture. Reflective and critical use of bibliometrics shape the culture of research.
Assumptions: publication is contribution to knowledge, peer review gatekeeps what counts in research. Citations link publications together, and thus can measure how much a publication contributes (scholarly attention). Citation as proxy for use, not direct observation. Don't necessarily suggest a positive contribution. Favoritism. Doesn't necessarily suggest quality if lots of people cite it to critique it. Lots of areas of bias and potential bias. Don't conflate measurement with value.
Bibliometric papers have a short turnaround because the data is already there.
History: started in libraries, developed for collection management and Collection development process. Different librarian specialization roles (research impact librarian, scholarly communication librarian). Origin of bibliometrics disputed, some say psychology etc. Eugene Garfield the "father of citation analysis" started SCI science citation index. Web of Stories has interviews with Garfield.
Journal index: or bibliographic index, or bibliographic database. List of journals organized by some criteria (field, topic, region, etc). Separate from citation index.
ScholCom: mostly about published journal articles, but also includes other publishing-related artifacts.
Journals were born out of the exchange of letters between scientific scholars. Started in 1665 in London (according to the Brits). Shifted to subscription by institution rather than individuals.
Peer review is a quality process for journals. Independent assessment of research. Formalized around 1970. Connected to the journal, but it doesn't inherently have to be that way. Used as a proxy for quality. The journal itself (eg Nature) is also used as a proxy for quality. Problematic.
What happened to Journal of Informetrics in 2020?
Dal has published more about bibliometrics recently, not currently shown in analyses because of citation window, but also as a result of Phil being here. Who is here impacts what the institution is known for.
Guest lecture by Bertrum MacDonald: applications of bibliometrics in historical and contemporary science communication
Dawson 1882 inadequate opportunities for publication in Canada. What was contained in the correspondence of scientists at the time? So many letters. Darwin wrote 15k letters. For old sources, we have a bibliography but we can't confirm the original works existed. Risk of perpetuating ghosts. Lots of cross-referencing to try to merge records. Hand writing on 5x7 index cards. Tapes 2x3cm ish. Source data mostly in libraries across Canada, some in archives, other places. Britain had more accurate records than Canada because Canada didn't have a central library until mid 20th century. This is stories less than lecture. The result was a microfiche database of bibliographic primary sources, 58k records. And then used them for research, eg contributions of women scholars of science and technology prior to ww1. Reconstructing a personal library. Finding the geographic sources of citations. "Read the article." Correspondence network topology. GESAMP publication citations to prove UN ocean protection group is valuable and worth funding, but difficulties because they were published by different organizations, index not including grey literature and policy documents. Overton helps fill some of that gap.
Questions that you want to answer determine a lot about what data you need to look at. Why does someone cite something? Citation metadata about positive/negative/criticism?
Chapter 3, the social structure of science
Based on Merton and Bourdieu sociologists. Merton considered science's main goal to be to extend the boundaries of certified knowledge. How do people work together to achieve that goal?
- Communism. The knowledge produced by science is a public good available equally to everyone.
- Universalism. Knowledge should be judged on its merits, and people who want to be scientists should be inducted based on their potential. Not who did the research/sociodemographics of the applicant.
- Disinterestedness. Altruism, for the good of humanity not for personal gain.
- Organized skepticism. Claims should be validated empirically and approached with skepticism. Research should be peer reviewed before being accepted into the body of scientific knowledge.
1 is currently troubled by capitalism. 2 assumes we all start on equal footing and are free from bias. 3 is troubled by capitalism. 4 is troubled by the publisher oligarchy. Rawls would argue that identifying people based on their supposed "potential" assumes we all share an equal starting point, which is obviously false. And that assumption would preclude inclusion of the experiences and viewpoints of people who have had more challenges getting to the starting line.
Supposedly scientists are in it for the recognition they receive from their peers, because there's no financial or private interest as a reward. Whatever the supposed altruistic values of science, those practicing it exist within a late-stage capitalist hellscape. The recognition of my peers isn't going to fill my belly. Recognition comes in various denominations, from Planck's constant to nobel prizes to tenured professors and journal editors down to just being cited. People with more recognition get more resources to fulfill their potential, Merton argues.
High potential gets more resources which results in more prestige which accumulates power and further stratifies science. "When it works optimally, the divide between the haves and the have-nots grows exponentially." optimally for whom? Zuckerman (1998) argues optimal for the scientific enterprise. This is the Matthew effect. Rossiter noted the Mathilda effect (gender gap), which extends to all minority groups and equity-disadvantaged groups (and originated before a Western understanding of non-binary gender experience, which I argue has preceded and existed throughout Western history).
Switching to Bourdieu, who examined individuals acting within power dynamics. Agents (people, universities) struggle to accumulate capital (power) within their fields (and also exert influence outside of them). Agents internalize and act on the values of the field in order to become members of that field. Capital takes the form of money, social recognition and influence, cultural (eg diploma from prestigious university), symbolic (one's position within the field), scientific (based on contributions to science, form of symbolic?).
Habitus is the conceptualization of the structure of the field and their position in it within the mind of an agent. The actual structure relies on how people perceive each other, so the collective habitus to an extent defines the field. Circular. Agents behave practically, not ruled by rules but by their own pragmatic self-interests.
Both Merton and Bourdieu depend on a stratification of status and accumulation of peer recognition.
I think they both have a lot of embedded assumptions about capitalism. Merton would completely miss out on all of the epistemic diversity of anyone who had more to overcome on their way to the starting line, and Zuckerman seems like a straight up supremacist. I don't understand how you get from "science is a public good" to "capitalism in science is good, actually". Bourdieu is at least more realistic with agents working in systems of stratified power but doesn't seem interested in exploring why that's bad and how under capitalism it's inevitable.
Chapter 2, the organization and evaluation of research
We used to have groups of amateur scientists in "academies" instead of research universities. First scientific societies in 1600s, non-egalitarian. France started government stipends for scientists. Science as hobby to science as profession in the wake of WW2. Letters were primary vehicle of science, societies kept letterbooks. Visualization of scholarly communication letter networks. Letterbooks became journals. Merton liked organized skepticism and so we get peer review. Pre-publication vs post-publication review, single and double blind vs open, all have pros and cons. Peer review is arbitrary (reviewers might disagree), biased, misses things, exploitative, slow. Journal prestige as proxy for quality, impact factor abused for tenure deliberations (was originally to help librarians in collection development). Corporate oligarchy capitalist exploitation.
Lecture 2026-02-02 Peer review and data sources
US researchers donated 1.5 Billion dollars in free labour for peer review to for-profit journals. Peer review can be single or double blind, or open. Pros and cons to each. Predatory and fraudulent journals charge lots of fees to authors and readers, may not do any peer review.
https://dal.ca.libguides.com/c.php?g=257122&p=5259250
Review of different data sources, show export formats. Basically do a search to get a subset of the database's contents and then export to csv. What do you do if you want to analyze more than that? Cry I guess.
Lecture 2026-02-09 Fields/disciplines
No one agrees what these are. Lots of people have lists of them, sometimes databases consider them exclusive and sometimes not. Lots of ways to try to figure out what fields exist. What you do with them depends on your research aims.
Lecture 2026-02-23 Authorship
This is titled "measuring research output" but it's not, it's measuring authorship and credit. It's not measuring the articles or the datasets. Author ordering varies by discipline and context. Sometimes order matters, sometimes it doesn't. Sometimes leftmost is best, sometimes rightmost. Credit is calculated different ways, eg inverse harmonics. Some papers have thousands of co-authors.
Lecture 2026-03-16 Altmetrics
They're earlier indicators than citations, extend outside academia. What are "academic novels"?
Pages that link here: