My latest at NITLE’s Techne blog: Digital Humanities Now, A Videoconference Discussion
November 11, 2011 § 2 Comments
Originally posted on November 11, 2011 at NITLE’s Techne blog, http://blogs.nitle.org/2011/11/11/digital-humanities-now-a-videoconference-discussion/
Last week Digital Humanities Now (@dhnow) was relaunched. This experiment in how we evaluate scholarship begs the question, how will our colleagues outside the digital humanities evaluate our digital work? How can we make our work legible to them? This was the subject of yesterday’s impromptu videoconference discussion.
I was joined by Joan Fragaszy Troyano, Managing Editor and Sasha Boni, Editor, Digital Humanities Now, as well as Ryan Cordell, Assistant Professor of English at St. Norbert College and a member of NITLE’s Digital Humanities Council; Gabriel Hankins, University of Virginia Scholars’ Lab Fellow; and Chris Dickman, Ph.D. candidate in English Composition and Rhetoric at Saint Louis University, and co-organizer of THATCamp Pedagogy.
Part of our discussion focused on how Digital Humanities Now (DHNow) works in practice. Their about page explains:
Digital Humanities Now aggregates and selects material from our Compendium of the Digital Humanities, drawing from hundreds of venues where high-quality digital humanities scholarship is likely to appear, including the personal websites of scholars, institutional sites, blogs, and other feeds. It also seeks to discover new material by monitoring Twitter and other social media for stories discussed by the community, and by continuously scanning the broader web through generalized and specialized search engines. Scholarship—in whatever form—that drives the field of digital humanities field forward is highlighted in the Editors’ Choice column. Digital Humanities Now also lists news items of interest to the field—jobs, calls for papers, conference and funding announcements, reports, and recently-released resources.
In practice, that means editors spend about an hour a day sifting through and nominating content. Then the editor in chief reviews those recommendations and chooses the final editors’ choices for the day. They try to limit themselves to 1-2 choices a day. When looking at the site, be sure to check out the older archives of the editors’ choice because they ran the site in test mode before officially launching and there are some hidden gems, like Lisa Spiro’s October 17 blog post, “Getting Started in the Digital Humanities” or Ryan Cordell’s blog post on “’The Celestial Railroad’ and the 1861 Railroad.” The Editors’ Choice forms one level of review that goes into this project.
A second layer of evaluation comes in open peer review that will lead to a quarterly peer-reviewed publication. Joan and Sasha explained what and how they are measuring to determine what makes the publication. Currently, they are trying a variety of measures but expect to refine their practice over time. Some of those measures include, tracking clickthrough rates on twitter (from @dhnow) and their own link shortener through bitly. Their RSS feeds are all feedburner links so they can track views and clickthroughs of people reading in their own RSS reader. They are using ThinkUp to track social media. “ThinkUp is a free, open source web application that captures your posts, tweets, replies, retweets, friends, followers and links on social networks like Twitter and Facebook.” And each viewers of the Digital Humanities Now website can participate in a star ranking system for each item. All of this automated tracking will be combined with editorial oversight to determine which items have the most impact in the current conversation about digital humanities.
So, how do you get credit for the kind of work you are doing that would get picked up on DHNow? Ryan Cordell ran into this issue when he was trying to capture the fact that one of his blog posts had been an editor’s choice. When you click on the post, it leads you to the original site. He finally settled on a screenshot. Prompted by Ryan’s tweets about this issue, DHNow is working on alternative methods. This technical issue leads us to a larger issue—does this type of publication count for tenure and promotion. The consensus of the group was that it would count in some circles. But in many cases, the committee won’t have a nuanced understanding of electronic scholarship or of any of these new forms of publication. Chris noted that graduate students face similar hurdles (both technical and institutional) in wanting to publish their multimedia dissertations openly online. Overall, we need to work on how to talk about this type of work and make it legible for review committees. Laura Mandell shared a CommentPress Site with me that attempts to answer that question by answering a request from the Promotion and Tenure Committee at Texas A&M University, Department of English for information about how to evaluate digital work for promotion and tenure. So, two answers seem to be educating our colleagues and experimenting with new models, as DHNow does.
Conversation then moved to a discussion of exactly what value DHNow demonstrates. Ryan regards the blog post chosen for DHNow as unformulated thoughts put online so people would give him feedback. In other words, this is a contribution to the public scholarly conversation. Traditionally, we measure the worth of humanities scholarship based on the unspoken prestige of the journal or press, etc. By that rule, if DHNow gains more and more prestige, then simply being chosen might convey some sense of value. But, the second layer of evaluation in DHNow seeks to get at a different measure of worth, i.e., scholarly impact. In the sciences, citation analysis can help demonstrate impact, and Joan suggested that in Europe, even in the humanities, citation metrics are taken into account when evaluating a scholar. Among other measures, DHnow hopes to establish which items generate more conversation and promote those to their quarterly journal.
We also discussed a few other aspects of measuring the scholarly conversation. Gabriel noted that journals typically capture the finished project of that conversation, and wondered if we had a print form that captures the impact of a conversation in process. We asked, how can we keep our finger on the pulse of the conversation and not just individual contributions? The journal roundtable is one method. The DHNow editors have already noted that some topics seem to hit all at once and generate conversation, even if all the participants aren’t aware of each other’s contributions. The recent Digital Humanities and Theory Round Up and today’s Digital Humanities and Theory Round Up Part 2 editors’ posts attempt to capture that conversation and connect all the participants. In this case, when a topic has found its kairos should we be awarding credit to individuals? Do we give out bellwether credit for those who wrote on the issue first? Or does it go to the one who said it best? Gabriel also argued that it would raise legitimacy and the level of the conversation to actively seek out and include representatives of the straw men views discussed in some of the recent digital humanities and theory articles.
Finally we discussed risk-taking and innovation for the journal form. Would would it look like to take a risk and be willing to fail? Some suggestions included:
- Putting the comments first
- Having all contributors video blog
- Items that are not primarily text—one recent DHNow editors’ choice was a youtube video, Digital Humanities – MCN-2011 in Atlanta, which was itself a contribution to Neal Stimler’s (@nealstimler) crowdsourced panel at Museum Computer Network, 2011.
Here are some other sources that came up for Evaluating Digital Scholarship
- Laura Mandell’s ComentPress Site: Promotion and Tenure for Digital Scholarship
- NINES: http://www.nines.org/about/scholarship/peerReview.html
- 18th connect: http://www.18thconnect.org/18th_about/peerReview.html
Previous NITLE impromptu videoconference discussions:
- Mozilla badges for education (part 1, part 2)
- Google+ for education
- Recent developments in open education
- Unbundling Education
- Digital StorytellingEbooks in Education