As librarians, we invest a great deal of time and effort instructing researchers on how to use our materials. This is especially true for special collections librarians, as we attempt to familiarize researchers with our unique resources and intricate collection arrangements. At the end of that instruction investment, we often wonder if we have been effective and what our students have truly learned. Have we taught them lasting research skills? If so, how do we illustrate the value of this service to cost-cutting administrators? How do we quantify the skills gained from working with our materials? Most importantly—how do we know if our instruction is making a difference for the researcher?

University Archives Photograph Collection, NCSU Special Collections Research Center

Last year, we had the opportunity to collaborate with the Association of Research Libraries (ARL) in the production of SPEC Kit 317: Special Collections Engagement. SPEC Kits, produced annually, survey the 124 ARL member institutions and collect data on current practices and policies of libraries.

We surveyed member institutions about the ways special collections are engaging students, faculty, and researchers through exhibits, events, and curricular involvement, and found that over 95% of respondents are involved in these activities (Berenbak et al., 2010, 16). A core component of many of these outreach efforts was instructional engagement in the use of special collections materials.

As we began the work of analyzing the survey results, a recurrent theme surfaced: the inconsistency of instructional engagement assessment. We began to ask ourselves questions about the concepts of evaluation and assessment of instruction, and how those terms are articulated and understood in the context of special collections. For example, when conducting a one-time instruction session, should evaluation focus on the librarian’s presentation skills, the use of archival collections by participants after a session, or the number of participating students or classes?

Although special collections are attempting to assess their instruction in a variety of ways, these efforts are not consistent, not standardized, and often not driven by a “need for information that fosters targeted change” (Ariew, 2007, 508). Many special collections would like to move assessment beyond use counts and anecdotal feedback, but the majority of ARL special collections have no plan or policy for outreach or engagement, and few have dedicated outreach staff (Berenbak et al, 2010).[1] Under these circumstances, how do special collections conceptualize what success looks like, or what measurements will convey when success has been achieved?

What IS being assessed?

University Archives Photograph Collection, NCSU Special Collections Research Center

Currently, most research libraries contribute annual statistics to government agencies and organizations such as the Association of Research Libraries (ARL).[2] These statistics include the size of each library’s collections, circulation, staff. Additionally, ARL asks libraries to describe instructional engagement efforts, reporting on the number of presentations that are given to groups, the number of participants in those groups, and the number of reference transactions. ARL provides the following definitions for its categories:

Presentations to Groups. Report the total number of sessions during the year of presentations made as part of formal bibliographic instruction programs and through other planned class presentations, orientation sessions, and tours . . . Presentations to groups may be for either bibliographic instruction, cultural, recreational, or educational purposes . . . the purpose of this question is to capture information about the services the library provides for its clientele. (Kyrillidou, & Bland, 2009, 100, emphasis added)

Participants in Group Presentations. Report the total number of participants in the presentations. For multi-session classes with a constant enrollment, count each person only once. Personal, one-to-one instruction in the use of sources should be counted as reference transactions (Kyrillidou, & Bland, 2009, 100).

Reference Transactions. A reference transaction is an information contact that involves the knowledge, use, recommendations, interpretation, or instruction in the use of one or more information sources by a member of the library staff. The term includes information and referral service. Information sources include (a) printed and nonprinted material; (b) machine-readable databases (including computer-assisted instruction); (c) the library’s own catalogs and other holdings records; (d) other libraries and institutions through communication or referral; and (e) persons both inside and outside the library . . . . (Kyrillidou, & Bland, 2009, 100).

Special collections departments are asked to contribute their numbers to their library’s general pool; ARL does not differentiate between general library instruction and the instructional efforts of special collections departments, a practice that makes the compiled statistics less useful for both ARL and the responding institutions.

It is clear from the results of our SPEC Kit findings that few institutions are doing any assessment of instructional engagement beyond what is required by ARL. Most responding institutions do not have formal evaluative measures. [3] Instead, these institutions tend to rely heavily on feedback and conversations with students, faculty, and researchers (Berenbak et al, 2010, 78, 79,91,92). [4] Special collections tend to either quantify the usefulness of their instruction when patrons mention they “learned” or “got something” from the instruction, or when they count how many items were checked out to patrons. And while counting items is arguably important for certain kinds of assessment, without measuring against a desired and stated outcome, what does a number like this really tell a special collections about its practices?

We know that very few special collections departments have any sort of formalized planning or policies guiding their instructional programming (Berenbak et al, 2010, 15). Different circumstances in each special collections contribute to this situation. In some cases, staff are short on the time and energy to devote to this activity (or, more commonly, staff tasked with this activity are a luxury most special collections cannot afford). In others, the responsibility of instruction is delegated at the time of need to the staff person whose background most closely aligns with the subject area of the instruction, limiting the consistency of the instruction. Sometimes the institution simply has not considered or not yet formally developed a plan for instructional engagement that fits into the overall activities of that special collections.

Whatever the circumstances, the results of our survey showed that most (80%) special collections are engaging in instructional sessions on a steady basis, and will likely continue to do so in the future — perhaps at an even greater frequency than their current rates (Berenbak et. al, 2010, 13).[5] If special collections are going to direct more focused efforts at planning their instructional engagement, they will need articulated and useful assessment metrics. After all, we cannot know if our engagement planning is a worthwhile investment if we are not assessing the outcomes of that engagement.

Though we recognize a need for better assessment, we are struggling to respond to this need. Determining which metrics will provide useful information about instruction is a conundrum that is keeping many special collections frustrated or hesitant to try assessment at all. A few institutions provide evidence that assessment is not daunting for everyone — one special collections, for example, looks for citations of materials from their holdings in student papers as an indication of the success of their instruction; some look for any citations of primary source materials; and some have undertaken short surveys and faculty interviews. [6] But by and large, most special collections seem uncertain as to what to collect or how to collect it.

What are we teaching?

University Archives Photograph Collection, NCSU Special Collections Research Center

When we do an instruction session with patrons in special collections, what are our objectives? Aware that specific objectives will vary from one session to another — informed by the needs, topics, or other parameters that may frame a session — there are still general objectives that we, as instructors, are always hoping to meet. Helping patrons find exactly what they need is possibly the most successful outcome we can achieve, but the many steps along the path to discovery are the components of instruction that perhaps most need to be measured in order to gauge the effectiveness of our instruction. Before patrons can find exactly what they are looking for, they first have to learn how to find it. From our perspective as instructors, a successful journey is more indicative of our instructional impact than arrival at the destination.

Why is the journey so important in special collections? Elizabeth Yakel, in her article “Listening to Users,” describes archives as a tabula rasa for researchers (Yakel, 2002, 122). She makes the important point that, unlike libraries where the “paradigm for assistance, access tools, and rules” has been learned by users from childhood at their public and school libraries, archives are considered a great unknown (Yakel, 2002, 122). The intricacies of the different rules, different materials, and different access tools often stump even the most experienced library user or researcher. Some archivists have correctly compared a successful special collections instruction session to an “archaeological dig” (Schmiesing & Hollis, 2002). Since the majority of special collections materials are not reflected on an item-by-item basis in either the library catalog or a finding aid, researchers must “dig” through boxes of materials, digital images, or artifacts. Because of the nature of this type of research, and because materials are not individually pre-selected for consumption, users must constantly reformulate their queries as they discover new materials. This often necessitates close collaboration with the special collections staff throughout the research process.

Unfortunately, this type of instruction is not accurately reflected in our measurements. Certainly limited head counts and use statistics do not paint an accurate picture of this work, nor do brief reactionary evaluations. ^[7] These evaluations are important and necessary, especially when reporting to organizations outside the library, but they fail to assess whether or not learning objectives are being met.

What’s out there now?

Academic libraries recognize that the reactionary evaluation of instruction often falls short, and have developed tools to help libraries make sure students and users are meeting learning objectives. These include guidelines such as ACRL’s “Information Literacy Competency Standards for Higher Education,” “Standards for Proficiencies for Instruction Librarians and Coordinators: A Practical Guide,” and skills tests such as Project SAILS. These guidelines give a framework for conducting meaningful evaluation for instruction librarians. And although there is no shortage of literature to be found on the subject of library instruction and assessment, we are only beginning to see similar literature and tools dealing with evaluating instruction in archives and special collections. A good example of this emerging interest can be found in Michelle McCoy’s article, “The Manuscript as Question: Teaching Primary Sources in the Archives – The China Missions Project.” McCoy details methods for the planning, instruction, and innovative assessment of a collaborative effort between the special collections and archives department at DePaul University and Professor Warren Schultz’s undergraduate History 199 Historical Concepts and Methods class.

Arguably the most important current project appearing in the assessment literature for archives and special collections is the Mellon-funded Archival Metrics Project, which includes models for assessing instruction (discussed at length below). In addition to the products themselves, Archival Metrics investigators have produced papers detailing initial studies for the project such as Duff and Cherry’s “Archival orientation for undergraduate students: An exploratory study of impact.“

Although we appear to be making progress, current assessment practices of efforts to instruct patrons on the use of special collections resources — both the materials themselves and the many discovery tools we’ve created (finding aids, databases, and subject guides) — would probably not receive a passing grade. Measuring and quantifying the journey is a daunting task.

While we should not stop collecting the statistics that are needed by ARL, the general library community — and especially special collections — should have a clear understanding of what these numbers actually represent. Any instruction or reference librarian will tell you that a headcount for their curricular sessions or a tally mark for a reference transaction does not adequately measure what they do or the instruction they provide. Especially when tally sheets obscure the difference between a quick question lookup and an hour-long research consultation at the desk.

We face a number of difficulties in achieving the goal of both establishing and collecting useful assessment metrics. In addition to a lack of policies or plans regarding curricular outreach and engagement, special collections often do not have positions designated to conduct instructional outreach. As discussed earlier, these duties often fall to the person in the department with the greatest subject knowledge, or the most available time.[8] It will be difficult to take on additional duties — especially when there are no easy answers and many special collections are short on staff and funding — but we offer some suggestions for ways that special collections might start.

Assessing the Journey

University Archives Photograph Collections, NCSU Special Collections Research Center

First, we must share. Some special collections are reaching students and evaluating their work with them in innovative ways, and the success of these efforts needs to be promoted.

Some of these innovations include:

• Making early contact with graduate student instructors so that they have experience working with special collections before they enter faculty positions;
• Working with subject librarians to incorporate relevant material into their teaching efforts;
• Giving awards to undergraduate research projects that make extensive use of the collections;
• Working with students to create virtual and physical exhibits highlighting materials used in special collections.

Assessment examples include:

• Monitoring use statistics of particular collections after an instruction session;
• Asking classes to donate copies of student papers to review the citations as a tool for better understanding the effectiveness of instruction;
• Using student focus groups to evaluate video tutorials;
• Monitoring books and articles published, performances given, and theses written;
• Tracking number and value of grants received;
• Examining web server statistics;
• Feedback forms and surveys;
• Monitoring number of graduate and practicum students using the collections;
• Soliciting and compiling one-on-one feedback from professors and students. [9]

The assessment practices that generate the most useful results are multipronged in their approach. The China Missions Project, for example, was organized in such a way that the students included a self-assessment of their experience using archival materials and research methods as part of their class research papers (McCoy, 2010, 55). Copies of these papers were deposited with the University Archives, and then staff conducted a qualitative survey of the papers to assess their responses. Recognizing that self-assessment in a graded paper might encourage students to write positive responses regardless of actual understanding, staff further scrutinized the papers’ citations. “Students who used a total of four citations or fewer or relied heavily on Wikipedia or other Web sources whose reliability cannot be verified were moved to a neutral position and not included in the positive total” (McCoy, 2010, 55). This approach—as well as other methods listed above—have plusses and minuses, but becoming aware of what other special collections are trying gives the rest of us a jumping off point.

A variety of special collections have noted their relationships with outreach and subject liaison librarians. Developing these close relationships can be beneficial for everyone involved. Understanding the holdings in a special collections, and illustrating how those materials might be incorporated into the curriculum, creates a great opportunity for instructing students in the value of primary sources. Drawing on the skills and backgrounds of subject specialists and instruction librarians can help special collections staff (often untrained in these areas) to develop sound instruction techniques.

Additionally, our colleagues in outreach and instruction have done an extraordinary amount of work related to best practices for evaluating instruction. In a 2007 article, Ariew and Lener state that one of the main insights gained in their study “Evaluating instruction: developing a program that supports the teaching librarian” was that teaching evaluation forms should be “tailored to specific classes, objectives and learning outcomes.” Most importantly, the group learned that “effective assessment requires a variety of assessment procedures be used” (Ariew, & Lener, 2007, 512). From teaching portfolios to 3-2-1 cards to surveys, the literature yields a great deal of information about what works and what doesn’t for each type of instruction. Although not all of these practices can be used to evaluate special collections instructional engagement practices, they provide guideposts to start from.

Fortunately, some people are starting to address the problem of how to assess the engagement work being done by special collections departments. The Archival Metrics Toolkits[10], for example, attempt to standardize evaluation in archives. This work recognizes that the “administration and use of primary sources are sufficiently different from libraries that they deserve tools that appropriately measure service to users” (Yakel, & Tibbo, 2010, 221). This creation of a standardized survey tool for archives could relieve a large part of the assessment burden, which is particularly important for archives with small staff. It also begins to answer the call for standardized evaluation that was so apparent in the results of our SPEC Kit survey.

The Archival Metrics Toolkit is particularly useful in laying out a set of standard questions about archives use, and they provide clear instructions on how to gather, compile, and analyze the data from the surveys. This information provides a basis for making comparisons across institutions, and could give special collections a better chance of identifying best practices and trends.

However, even the best of surveys have drawbacks such as rate of completion (particularly difficult in archives due to small numbers)[11], survey fatigue, and a focus on perceptions. Supplementing surveys by seeking evidence of skills mastered, such as citation analysis or testing, seems a more well rounded method to determining “what students have learned as opposed to how they feel about what they have learned” (Barclay, 1993, 198).

Special collections must clearly state engagement goals in order for any type of evaluation to be meaningful. Good practice in evaluating instructional engagement starts “with the learning objectives of the instructor” (or the department), and uses those to shape the tools being applied for evaluation (Areiw, & Lener, 2007, 512). As the libraries at Virginia Tech discovered, evaluation, when possible, should be unique to specific classes and desired student and faculty outcomes and will likely require that a variety of assessment procedures be used (Ariew, 2007, 512).

Conclusions

Today more than ever, library administrators are being asked to describe in a quantifiable way the value of their academic libraries and their practices. Therefore, special collections must be able to articulate to administrators why current evaluation methods are insufficient. Simple forms, tally marks, and baseline ARL statistics will never be able to get at the information we need to improve our practices. Specials collections need to make the case for developing more appropriate evaluation methods — even though this will require a commitment of valuable staff resources — and then make the commitment to using the results of these evaluations to enhance services. Ultimately, more meaningful data will help us provide better service to the students, faculty, and researchers who rely on special collections, and it will better equip us to tell their story and our own.

University Archives Photograph Collection, NCSU Special Collections Research Center

Huge thanks to our editors and advisors: Kathy Brown, Hyun-Duck Chung and Brett Bonfield. Your thoughtful comments have made this a much better post,` and sparked ideas for future avenues of exploration. And of course, thank you so much to all of our SPEC Kit co-authors, Adam Berenbak, Claire Ruswick, Danica Cullinan and Judy Allen-Dodson.

[2] The ARL states that these data “describe collections, staffing, expenditures, and service activities” of the 114 university libraries and 10 public, governmental, and nonprofit research libraries that collectively form ARL (Association of Research Libraries, 2008).

[3] This is not to imply that no one is attempting to assess instruction, but it is not standardized, and based on the survey responses, in general, it is fairly ad-hoc (Berenbak et al., 2010, 78, 79, 90, 91).

[4] Question 35 in the ARL Spec Kit did not specifically ask about the evaluation of instructional engagement but more broadly inquired, “What measure(s) have been used to evaluate special collections engagement with faculty/scholars/researchers who are affiliated with your institutions.” Many of the responses were similar to the more directed question 28 “What measure(s) are used to evaluate student use of unique materials in research projects.” The following types of statements made up the bulk of the responses: “no evaluation,” “much to few [sic],” “no particular measures have been used,” “nothing systematic,” “little evaluation has been done,” and “none to date.”

[5] The number of respondents actively working to engage students for curricular purposes is even higher at 99% (Berenbak et. al, 2010, 62).

[6] These examples are taken from the responses to the question “What measure(s) are used to evaluate student use of unique materials in research projects?” Responses include examining the “extent and breadth of primary resources and collections in any format,” a “learning outcomes survey,” and “discussion with faculty of results” (Berenbak et. al., 2010, 78, 79 and 80).

[7] Reactionary refers to a short survey after a presentation that often focuses on a students’ perception of the presentation rather than on whether or not new skills have been developed.

[8] When respondents were asked who had primary responsibility for coordinating curricular engagement, 15% had one individual who held primary responsibility, 15% said one individual leads a team or staff, 31% stated that all (or most) special collections staff shared the responsibility, and 39% noted that it varied depending on the project (Berenbak et al, 2010, 64).

[9] These examples are drawn from the responses to questions 28 and 35 of the ARL SPEC Kit 317 “What measure(s) are used to evaluate student use of unique materials in research projects” and “what measure(s) have been used to evaluate special collections engagement with faculty/scholars/researchers who are affiliated with your institution”(Berenbak et. al, 2010 78, 79, 80, 91). The respondent’s institutions are kept anonymous in SPEC Kit publications, so although these are specific examples, we are unable to point out specific schools for the purposes of this post.

[10]The toolkit includes sections for “Researchers” (A user-based evaluation tool for on-site researchers to evaluate the quality of services, facilities, and finding aids in university archives and special collections), “Online Finding Aids” (A user-based evaluation tool for visitors to evaluate the quality and usability of online finding aids in university archives and special collections), “Websites” (A user-based evaluation tool for visitors to evaluate the quality and usability of websites in university archives and special collections), ”Student Researchers”(A user based evaluation tool for students use the archives or special collections as part of a class and participate in archival orientations), and a “Teaching Support” section (A user-based evaluation tool for instructors who have used the university archives and special collections to evaluate its services.)

[11] Small numbers can make it difficult to obtain an appropriate sample size.

Resources

Ariew, S., & Lener, E. (2007). Evaluating instruction: developing a program that supports the teaching librarian. Research Strategies, 20. Retrieved from http://www.sciencedirect.com/science/article/B6W60-4MWXT97-2/2/e3d2a22ec51f17a15bc53a77240d49e7 doi: 10.1016/j.resstr.2006.12.020

Association Of Research Libraries, (2010). Association of Research Libraries: SPEC Kits. Retrieved Sep. 15, 2010, from http://www.arl.org/resources/pubs/spec/index.shtml.

Association of Research Libraries. (2008, February 4). Association of research libraries: annual surveys. Retrieved from http://www.arl.org/stats/annualsurveys/index.shtml

Barclay, D (1993). Evaluating library instruction: Doing the best you can with what you have, RQ 33 (2), pp. 195–202.

Berenbak, Adam, Putirskis, Cate, O’Gara, Genya, Ruswick, Claire, Cullinan, Danica, Dodson, Judy Allen, Walters, Emily, & Brown, Kathy (2010). Spec kit 317 special collections engagement. Washington, DC: Association of Research Libraries.

Knight, L. (2002). The Role of assessment in library user education. Reference Services Review, 30(1), Retrieved from http://www.emeraldinsight.com/journals.htm?articleid=861677&show=html

Kyrillidou, Marth, & Bland, Les. (2009). Arl statistics 2007-2008 Washington, DC: Association of Research Libraries. Retrieved from http://www.arl.org/stats/annualsurveys/arlstats/arlstats08.shtml

McCoy, M. (2010). The Manuscript as question: teaching primary sources in the archives – the china missions project. College and Research Libraries, 71(1), Retrieved from http://crl.acrl.org/content/71/1/49.full.pdf+html

Schmiesing , Ann, & Hollis, Deborah. (2002). The Role of special collections departments in humanities undergraduate and graduate teaching: a case study Libraries and the Academy, 2(3), Retrieved from http://muse.jhu.edu/journals/portal_libraries_and_the_academy/v002/2.3schmiesing.html

Yakel, E. (2002). Listening to users. Archival Issues, 26(2), 111-127.

Yakel, E., & Tibbo, H. (2010). Standardized survey tools for assessment in archives and special collections. Performance measurements and metrics, 11(2), Retrieved from http://www.emeraldinsight.com/journals.htm?articleid=1871188&show=abstract doi: 10.1108/14678041011064115

Photo by Flickr user Hammer51012

Picture it, a higher education institution, 2009. The sun is shining. It’s a warm summer day. Your iced coffee perspires on the desk in front of you. You are a faculty librarian participating in a workshop with other faculty members on outcomes-based assessment for teaching and learning. You’re excited to make the leap from routine library orientations to in-class assignments centered around information literacy concepts, which will help improve your instructional sessions and place students on the continuum towards mastery of information literacy concepts. Suddenly, the conversation turns to the topic of the learning outcome for information literacy.

“How is information literacy any different from critical thinking?”

“Couldn’t we just get rid of information literacy since it shares similar outcomes with critical thinking?”

Wait, what?!?

Immediately, your head starts reeling with the national standards of the Association of College and Research Libraries, various statewide initiatives that have mobilized to embed information literacy into k-12 and higher education curriculum, and individual faculty with whom you have worked with to cover these very same standards as part of their learning outcomes for their students.

As you witness this debate unfolding, you think to yourself, what is the difference between critical thinking and information literacy? Do they share some common characteristics? Is it possible for one to exist without the other?

How would you respond to this challenge, where there are shared outcomes among information literacy and critical thinking?

The goal of the assessment-based learning outcome is to identify one necessary skill, such as teamwork, critical thinking, or communication, the student will use on the job, in their daily lives, or in the next stage of their educational process. This learning outcome is then embedded in an assignment or throughout the curriculum, and a scoring rubric is devised to focus on increasing student competency in this skill. This rubric allows for a quantitative value to be given to completed assignments representative of the student’s application of the skill and helps place them on the competency continuum somewhere between a basic, on the low-end of the scale, and advanced, on the high-end, which provides a great opportunity to make sure students are “getting it.” This process also engages both instructors and students in a teaching and learning partnership. By emphasizing the mastery of key learning outcomes at your institution students will be prepared for a post-graduate workplace environment.

It is important to ask this question: Is it possible for critical thinking to take the place of information literacy in today’s digital information universe? In this post I’ll be looking at information literacy and critical thinking as components of assessment. How is critical thinking and information literacy intertwined in the assessment of student learning? What other types of literacies are required for today’s student? Why is information literacy critical to student learning?

Information literacy and critical thinking: An accidental marriage?

To begin, we should establish some common ground on the definition of information literacy. The Association of College & Research Libraries (ACRL) outlines the major competency areas for the information literate individual:

• Determine the extent of information needed
• Access the needed information effectively and efficiently
• Evaluate information and its sources critically
• Incorporate selected information into one’s own knowledge base
• Use information effectively to accomplish a specific purpose
• Understand the economic, legal, and social issues surrounding the use of information, and access and use information ethically and legally [[1]]

These standards outlined by ACRL have been integrated into evaluation rubrics used by instructional librarians across the nation in levels K-12 through higher education. While there is variation in the language and presentation used to express these competencies, information literacy assessment closely follows these standards. One such example of a higher education institution is McKendree University, which has its very own on-line, interactive information literacy rubric. [[2]] Developing and using a rubric for information literacy can help place an individual or group of students on the information literacy continuum, where a student may come into the instructional session scoring low, or beginning, and leave the instructional session scoring somewhere in the middle to high range, or mastering. Through this process, we can gauge a student’s mastery of information literacy concepts and measure the effects of the instructional session on student learning. It is important to remember that when students hear the word “assessment,” this naturally inspires a lot of fear. They are concerned about having to take a test, turn in a paper, or engage in some form of an academic assignment. While fear may be a reality for most students, it is equally important to remember that these evaluation rubrics are also an assessment of the instructional librarian. They are tools for improving both learning and teaching, and provide very useful metrics for identifying future teaching opportunities.

Similar to information literacy, critical thinking has its own set of competencies. One of the leading organizations in providing a foundation for the assessment of critical thinking, Foundation for Critical Thinking breaks this down into eight discrete elements:

• Generates purposes
• Raises questions
• Uses information
• Utilizes concepts
• Makes inferences
• Makes assumptions
• Generates implications
• Embodies a point of view [[3]]

Critical thinking, as its own unique form of assessment, aims to get students to distinguish between empirical and factual evidence by applying higher order thinking to their own mental processes of receiving, taking apart, and synthesizing information. In addition, students balance all of this with an awareness of their own subjective judgment. Assignments created with the learning outcome of critical thinking in mind strive to create a fair and balanced outcome and parallels similar skills that will be required for future practical application. In their article on the importance of this outcome for graduate and experienced nurses, Fero, et. al directly link critical thinking to patient safety: “Nurses must have the ability to recognize changes in patient condition, perform independent nursing interventions, anticipate orders and prioritize.” [[4]] Focusing on the critical thinking characteristics of “interpretation, analysis, evaluation, inference, explanation, and self-regulation,” Fero, et al., argue that using a critical thinking framework in nursing education will bear directly on the nurse in their post-graduate nursing experiences. [[5]] This study included an exercise that presented nursing students with videotaped scenarios in a clinical setting and then asked them to record, in writing, their identification of the problem and necessary action steps for the resolution. Even though this case study incorporates situations aimed to exercise the critical thinking skills of its student membership, there are points at which information literacy competencies come into play. This is a perfect example of an inherent collusion between critical thinking and information literacy, as the scenarios presented in these videotapes require evaluation of visual information and critical thinking, which will then lead to an understanding of the information needed to resolve these problems.

If professional disciplines, such as nursing, look to specific competencies for graduates entering the workforce, then educational institutions should meet this same need by embedding competencies in the curriculum. For example, Washington State University has codified critical thinking for its student and faculty membership, by creating an online Critical and Integrative Thinking Rubric. It serves as an institution-wide foundation for learning assessment across disciplines. [[6]] Similar to the information literacy rubric, instructors use these templates as competency models, and modify existing critical thinking rubrics tailored to specific assignments. A class taught at WSU on investigations into the arts, manipulated the existing baseline of the Critical and Integrative Thinking Rubric to achieve its own critical thinking outcome for its course. These malleable rubrics are very useful in standardizing learning outcomes and setting clear guidelines for students to follow. Presenting and following a scoring rubric with an assignment can also take a lot of the subjective evaluation out of assessment and point students toward clearly stated goals.

Until death us do part

Clearly, there is a shared relationship between information literacy and critical thinking. Critical thinking comes into play when getting students on the path to looking at information and using it judiciously in light of their topic. Library information instruction sessions have been following a trend which seeks to blend these two together, getting students to look at websites, articles, media items, and other online content with a critical eye toward evaluating these sources for credibility.

Library instruction incorporates brainstorming or concept mapping into sessions, providing students with a creative approach to generating a purpose for their assignment. Increasing numbers of instructional librarians also devote time to the evaluation of information, which ultimately takes students out of the pre-packaged box of research databases and into the frontier of the freely available information online. Ellie Collier in her post, In Praise of the Internet: Shifting Focus and Engaging Critical Thinking Skills, touches upon this very relationship existing between information literacy and critical thinking. She encourages instructional librarians everywhere, “to shift our primary focus away from teaching how to find information and towards engaging critical thinking skills.” [[7]] This does not mitigate the need for the information literacy competency, when in fact both of these competencies live in perfect harmony. Both critical thinking and information literacy work together in a partnership, each leaning on and supporting the other. Li Zhang points to this very relationship between these two competencies, stating:

An information literate student will be able to formulate research queries and create search strategies that reflect an understanding of information sources and their organization, analyze the data collected for value, and ultimately incorporate the data to solve problems. This literacy or competency goes beyond simply acquiring knowledge; it involves the process of critical thinking, which emphasizes reasoning, forming judgment about the evidence, and determining when new information must be generated. Since information literacy and critical thinking are so closely related, it is the job of librarians who are also educators to go beyond merely providing lectures, but strive to cultivate students’ thinking skills in order to equip them with necessary strategies to cope with complex problems. [[8]]

Working with students to foster a cohesive, give-and-take relationship between critical thinking and information literacy will reinforce their post-graduate skills. As Zhang suggests, engaging students in that grey area between information literacy and critical thinking will foster direct engagement with information, and help them make connections between their research needs and the information available to meet those needs. Specifically, Zhang focuses on the following: comparing and critiquing websites for credibility; encouraging students to come up with their own standards of evaluation; using specific search strategies for online and database searching; and ultimately focusing on transference of skills between web and database searching. If you are an instructional librarian who loves to get your students thinking and talking about information, then you already know it is impossible to draw a clear line between information literacy and critical thinking.

Opening up the relationship

Increasingly, information literacy has become a national concern, stemming from a rapidly changing information and technology landscape. This includes print and electronic content, photographs, videos, podcasts, blogs, government documents, corporate records, institutional archives, and information formats yet to be defined. The concern for information literacy has had a long history, punctuated by different modalities: media, technology, computer, and cultural literacy. [[9]] To this list could be added multimedia, digital, communications, and social media literacy. Information literacy’s best friend, ACRL, draws a clear line between information literacy and information technology, stating that “[information] technology skills enable an individual to use computers, software applications, databases, and other technologies to achieve a wide variety of academic, work-related, and personal goals.” [[10]] Similar to the shared relationship between critical thinking and information literacy, much of these alternate literacies can reinforce and provide added dimension to the information literacy competency and create critical skills for the 21st century student. As information rapidly changes in appearance and content, it is of import for information literacy to be a part of the conversation regarding other literacy modalities.

The landscape of information literacy is changing, and these 21st century skills will also change the way students access, evaluate, incorporate, and use information effectively. Perhaps now and in the future, writing research papers may not be the primary method of student assessment at every higher education institution, and may not always coincide with every institutions mission for its student membership; however, in an increasingly networked world the necessity for an information literacy learning outcome is paramount. Beyond equipping students to interact with and use technologies effectively, being able to navigate information-rich environments is critical.

Information literacy: All by myself…

The necessity for classroom embedded information literacy skills is unavoidable. When navigating different user groups (e.g., community college associations, library associations, local library councils, and national library associations) the feelings on the topic of information literacy is similar: “To be information literate, a person must be able to recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information.” [[11]] Even as information literacy bears strong ties to critical thinking, it must be able to stand alone as its own competency, by holding its own and be given equal representation in the assessment of teaching and learning.

Higher education institutions have taken a stand on the importance of information literacy, emanating standards and guidelines for the information competent individual from state and local government. The American Association of Community Colleges (AACC) has made their stance on information literacy very clear:

Information literacy, which encompasses information fluency and information technology mastery, is critical to success in higher education and lifelong learning. Rapid and continual changes in technology and the proliferation of information resources present students with an abundance of information through a variety of vetted and unvetted formats. This wide variety of choices raises questions about the reliability, authenticity, and validity of content and poses challenges for students trying to evaluate, understand, and apply the information.  The Association of College and Research Libraries, in its Information Literacy Competency Standards for Higher Education, notes that information literacy is considered a key outcome by several regional and discipline-specific accreditation bodies because of its close ties to students’ competency with evaluating, managing, and using information . . . . An important element of both teaching and learning in today’s information age is information literacy — the set of skills needed to find, access, retrieve, analyze, synthesize and use information effectively and ethically. [[12]]

As discussed, information and technology is changing, as is the terrain of teaching and learning in higher education. Among the current changes, peer-reviewed and trade journals are consistently moving to an online format, newspapers are scaling back print production to pursue an electronic future, mobile devices are being used to access and navigate online information environments, iPhone apps are being created daily from a variety of creators, blogs and wikis are rapidly being created by both individuals and large-scale organizations, and educational content is shifting to open source environments. Students encounter information in their daily lives when they drop by the supermarket to pick up groceries, when they sit down to watch television or stream media content on their computers, or when they engage in the task of media creation. First and foremost, this is all information.

Critical thinking and information literacy rubrics, as they are typically laid out, are very similar. However, critical thinking is rooted in developing a thought process that occurs around a variety of life skills related to interacting and engaging with information. Information literacy has a marked departure from critical thinking. This happens when educating students about the types and formats of information and technologies that exist, and getting them to think about information and technology concepts in the aggregate. For example, we might emphasize, this is word processing, not, this is Microsoft Word 2007. Or this is a search interface, not, this is a research database. Increasing information competency equips students with the skills to efficiently navigate and gather requisite information they will need in the workforce. Out in the “real world,” students are not likely to be presented with a Google search box during every work task. Instead they will be required to use a variety of search interfaces to get at the information they need to address the problem at hand, and be able to evaluate the credibility of those sources. They will need to be able to create workarounds when their information strategies fail them. They must be able to locate, gather, evaluate, synthesize, and responsibly use information. This process is markedly different from critical thinking, which often emphasizes thought processes around information, situations, interpersonal communication, project-based work, and other aspects.

Information comes in a variety of formats and is needed not only for writing research papers, creating presentations, but is a critical skill in dealing with day-to-day personal problems and issues. In an important study by Project Information Literacy (PIL) facilitated by the Information School at University of Washington, researchers found that students were frustrated equally when “conducting research, whether for course assignments or everyday life problems, [presenting] its own set of challenges that are usually exacerbated in digital environments. Challenges are often deep-seated frustrations tied to finding resources students know exist, somehow, somewhere, but are unable to access.” [[13]] One remedy can be found in a problem-based approach to the information gathering process, a process that can successfully be carried out through information literacy. By involving higher order thinking skills students see the connection between the skills they acquire in the educational setting and how they can be applied in their everyday lives. Guided instruction on how to navigate a complex information landscape would greatly improve a student’s ability to appropriately identify and navigate information, and better equip them to amass information if a definitive information gap is found. Developing information seeking strategies that meet real-life, everyday situations creates a natural bridge to workforce training, providing students with an invaluable set of skills similar to critical thinking, but independently valuable in its own right.

Rekindling the old flame

Library instruction sessions need to remain open to integrating critical thinking skills, as well as other learning outcomes and literacy modalities, in the k-12 and higher education setting. This practice can make information literacy relevant and test the mettle of this skill set in the educational environment. In the process of engaging students, we can find out what they think about information on a personal level as they interact with it in their daily lives. Once we push students to “figure it out,” begin to stir their creative and intellectual faculties, we can start moving the conversation forward. While information literacy and critical thinking share a contiguous and inseparable relationship, one cannot exist without the other. Without information literacy, students would find themselves equipped to think about situations and ideas, but incapable of recognizing and understanding the vast information network or how to access this network. Alternately, without critical thinking we would have vast amounts of information with no way to filter, gather, or synthesize this information.

It is also important for instructional librarians in an educational context to be involved in the development of an information literacy rubric. In the course of developing such a rubric, we can provide our instructional peers with a template for evaluating information literacy in their own classrooms, as well as our own. In addition, modifying existing rubrics at your institution can be an invaluable test of the impact of your teaching and show you if students are truly “getting it.” Such a venture might also help integrate you more into instruction and allow students and instructional peers to see you as more than just the cool librarian with the book cart.

Along with critical thinking, information literacy should be offered with the same frequency as other teaching and learning assessments in the educational institution. It touches every aspect of practical skills provided at higher education institutions: automotive core students use Mitchell 1® OnDemand™ to create estimates and look up information on various types of automobiles; allied health and nursing students use MedlinePlus® to locate current, reliable information on topics of health and wellness; psychology students use search interfaces to review current literature on mental health topics; and students find themselves using these information strategies out in the world without realizing it. Embedding information literacy into instruction with the same frequency as other outcomes-based assessments promotes an emerging population capable of working with various technologies, as well as an aptitude for finding the right information to meet a need in a timely and efficient manner. There is an everyday use for information literacy that would be lost if it were to disappear into or merge with critical thinking and obscure its importance by calling it anything other than information literacy.

A very special thank you to Emily Ford, Edward Sargent, Cheyenne Roduin, and Merinda Kaye Hensley for their invaluable insight and suggestions for this post.

[[1]] “Information Literacy Competency Standards for Higher Education.” Association of College & Research Libraries (ACRL), 2009. http://www.ala.org/ala/mgrps/divs/acrl/standards/informationliteracycompetency.cfm.
[[2]] McKendree University. “Information Literacy Rubric.” Information Literacy Rubric, n.d . http://www.mckendree.edu/academics/Information_Literacy_Rubric.aspx.
[[3]] Foundation for Critical Thinking. “Critical Thinking Model 1.” The Thinker’s Guide to Analytic Thinking, 2007. http://www.criticalthinking.org/CTmodel/CTmodel1.cfm#.
[[4]] Fero, Laura J., Catherine M. Witsberger, Susan W. Wesmiller, Thomas G. Zullo, and Leslie A. Hoffman. 2009. Critical thinking ability of new graduate and experienced nurses. Journal of Advanced Nursing 65, no. 1 (January): 139-148, p. 140. doi:10.1111/j.1365-2648.2008.04834.x.
[[5]] Ibid., p. 141.
[[6]] Washington State University. “WSU’s Critical and Integrative Thinking Rubric.” Critical and Integrative Thinking Rubric, 2006. https://my.wsu.edu/portal/page?_pageid=177,276578&_dad=portal&_schema=PORTAL.
[[7]] Collier, Ellie. 2009. In Praise of the Internet: Shifting Focus and Engaging Critical Thinking Skills. In the Library with the Lead Pipe. January 7. http://inthelibrarywiththeleadpipe.org/2009/in-praise-of-the-internet-shifting-focus-and-engaging-critical-thinking-skills/.
[[8]] Zhang, Li. 2007. Promoting Critical Thinking, and Information Instruction in a Biochemistry Course. Issues in Science and Technology Librarianship 51, no. 2 (Summer). http://www.istl.org/07-summer/refereed.html.
[[9]] Horton, Jr., Forest Woody. “Understanding Information Literacy: A Primer.” United Nations Educational, Scientific, and Cultural Organization (UNESCO), 2007. http://unesdoc.unesco.org/images/0015/001570/157020E.pdf.
[[10]] Association of College & Research Libraries. 2000. Information Literacy and Information Technology. Information Literacy Competency Standards for Higher Education. January 18. http://www.ala.org/ala/mgrps/divs/acrl/standards/informationliteracycompetency.cfm#iltech.
[[11]] Association of College & Research Libraries. “Presidential Committee on Information Literacy: Final Report.” ACRL | Presidential Committee on Information Literacy, January 10, 1989. http://www.ala.org/ala/mgrps/divs/acrl/publications/whitepapers/presidential.cfm.
[[12]] American Association of Community Colleges. 2008. AACC Position Statement on Information Literacy. American Association of Community Colleges: Position Statements. May 4. http://www.aacc.nche.edu/About/Positions/Pages/ps05052008.aspx.
[[13]] Head, Alison J., and Michael B. Eisenberg. “Finding Context: What Today’s College Students Say about Conducting Research in the Digital Age.” Seattle, WA: The Information School, University of Washington, February 4, 2009. http://projectinfolit.org/publications/.