In the Library with the Lead Pipe is pleased to welcome guest author Derek Rodriguez. Derek serves as a Program Officer with the Triangle Research Libraries Network where he supports collaborative technology initiatives within the consortium and is project manager for the TRLN Endeca Project. He is a Doctoral candidate at the School of Information and Library Science at The University of North Carolina at Chapel Hill, and is the Principal Investigator of the Understanding Library Impacts project.

Value for money in higher education

These are challenging times for colleges and universities. Every week it seems a new article or book is published expressing concerns about college costs,[1] low graduation rates, and what students are learning.[2] We also don’t have to look very hard to find reports computing the economic benefits of a college education to individuals.[3] Clearly, U.S. colleges and universities are under pressure to demonstrate that the value of an undergraduate education is worth its cost.

Graduation rates are important measures.  Personal income is  certainly a measure that hits home for most of us during these difficult economic times. However, stakeholders in higher education have had their eyes on a different set of metrics for many years: student learning outcomes. A recent example is A Test of Leadership, better known perhaps as the Spellings Commission report in which the U.S. Department of Education raised concerns about the quality of undergraduate student learning. The report called for measuring student learning and releasing “the results of student learning assessments, including value-added measurements that indicate how students’ skills have improved over time.” [4] In recent years, higher education has responded with new tools to assess and communicate student learning such as the Voluntary System of Accountability.[5]

As colleges and universities grapple with this challenge, academic libraries are also seeking ways to communicate their contributions to student learning. The recently revised draft Standards for Libraries in Higher Education from the Association of College and Research Libraries (ACRL) signals the importance of this issue for academic libraries.[6] The first principle in the revised standards, Institutional Effectiveness, states that:

“Libraries define, develop, and measure outcomes that contribute to institutional effectiveness and apply findings for purposes of continuous improvement.”[7]

And an accompanying performance indicator reads:

“Libraries articulate how they contribute to student learning, collect evidence, document successes, share results, and make improvements.”[8]

While libraries have made significant progress in user-oriented evaluation in recent decades, libraries still lack effective methods for demonstrating library contributions to student learning. Unless we develop adequate instruments (and generate compelling evidence) libraries will be left out of important campus conversations.

In this post I review current approaches to this problem and suggest new methods for addressing this challenge. I close by introducing the ‘Understanding Library Impacts’ protocol, a new suite of instruments that I designed to fill this gap in our assessment toolbox.

The challenge of linking library use to student learning

Demonstrating connections between library use and undergraduate student achievement has proven a difficult task through the years.  Several authors have suggested outcomes to which academic libraries contribute such as:  retention, grade point average, and information literacy outcomes.[9] I review a few of these efforts below.

Retention

Retention is a measure of the percentage of college students who continue in school and do not ‘drop out.’ A handful of studies have investigated relationships between library use and retention. Lloyd and Martha Kramer found a positive relationship between library use and persistence as students who borrowed books from the library dropped out 40% less often than non-borrowers.[10] Elizabeth Mezick explored the impact of library expenditures and staffing levels on retention and found a moderate relationship between expenditures and retention.[11] Several authors report a different ‘library effect’ on retention: holding a job in the library.[12] This finding is supported by evidence that holding a campus job, especially in an organization that supports the academic mission, is related with “higher levels of [student] effort and involvement”[13] in the life of the university and should logically lead to increased retention. Those of us who have worked in academic libraries have probably observed this mechanism at work with students we have known.

However, I believe relying exclusively on this measure is problematic. First, numerous factors influence retention and it can be difficult to isolate library impact on retention without extensive statistical controls. Second, retention is an aggregate student outcome; it is not a student learning outcome. Retention is an important metric in higher education and we should seek connections between library use and this measure, but it does not satisfy our need to know how libraries contribute to student learning.

Grade point average

Several authors have attempted to correlate student use of the library with grade point averages (GPA). Charles Harrell studied many independent variables and found that GPA was not a significant predictor of library use.[14] Jane Hiscock, James Self, and Karin de Jager, among others switched the dependent and independent variables in their studies and found limited positive correlation between library use and GPA.[15] Shun Han Rebekah Wong and T.D. Webb reported on a large-scale study with a sample of over 8,700 students grouped by major and level of study. In sixty-five percent of the groups, they found a positive relationship between use of books and A/V materials borrowed from the library and GPA.[16]

However, GPA-based studies have their problems. As Wong and Webb note, studies that use correlation as a statistical method cannot assure causal relationships between variables; they can only show an association between library use measures and GPA. As the old adage goes, ‘correlation does not imply causation.’ Do students achieve higher GPAs because they are frequent users of the library? Or do students who make better grades tend to use the library more? Without adequate statistical controls it is impossible to conclude library use had an impact on GPA. Also, as noted by Wong and Webb, it can be difficult to gain access to student grades to carry out this type of study.

Information Literacy Outcomes

Information literacy outcomes assessment is the most fully developed approach we have for demonstrating library contributions to undergraduate achievement. Broadly speaking, information literacy skills encompass competencies in locating and evaluating information sources and using information in an ethical manner. Instruction in these skills is a core offering in academic libraries and findings from Project Information Literacy suggest there is still plenty of work to do![17] ACRL has also created a suite of information literacy outcomes[18] to guide the design and evaluation of library instruction programs. Numerous methods have been used to assess information literacy skills including fixed-choice tests, analysis of student work, and rubrics.[19]

It is tempting to rely solely on student achievement of information literacy skills to demonstrate library contributions to student learning. However, a recent review of regional accreditation standards for four-year institutions suggests there is uneven support for doing so. Laura Saunders found three of six regional accreditation agencies specifically name information literacy as a desired outcome and assert the library’s prominent role in information literacy instruction and assessment of related skills. [20] Others rarely use the term “information literacy” in their standards. Instead, competencies such as “evaluating and using information ethically” appear in these standards as general education outcomes to be taught and assessed throughout the college curriculum. In part, I think this reaffirms for us that many in higher education associate information literacy outcomes with general education outcomes such as critical thinking.

While it may be encouraging for information literacy outcomes to be integrated into the college curriculum, I think this poses real difficulties when we attempt to isolate library contributions to these outcomes. If information literacy and critical thinking skills are inter-related, how are we to assess one set of skills, but not the other?  Heather Davis thoughtfully explored this issue in her post “Critical Literacy? Information!” finding that these competencies are intricately related and it is extremely difficult to teach (and assess) them independent of one another. If information literacy skills are taught across the curriculum, when, where, and by whom should they be assessed?  Where does faculty influence stop and library influence begin?

Information literacy outcomes are integral to undergraduate education, but these are not the only learning outcomes that stakeholders are interested in. And information literacy is not the library’s sole contribution to student learning.

I believe we need to shift course in our assessment practices and tackle ‘head on’ the challenge of connecting library use in all its forms with learning outcomes defined and assessed in courses and programs on college and university campuses. We should also link our efforts to the learning outcomes frameworks used in the broader academic enterprise. Broadening our perspective will provide a better return on our assessment dollar.

Where to begin

We can improve our ability to detect library impact on important student learning outcomes by carefully choosing our units of observation. Fortunately we can look to the literature of higher education assessment for clues.[21] My conclusion is that a one-size-fits-all approach to assessment is not likely to work for higher education or for library impact. Instead, our instruments should respect differences in students’ experiences. We should focus on the ‘high-impact’ activities in which faculty expect students to demonstrate their best work. Capstone experiences and upper level coursework within the academic major seem to fit the bill for four year institutions.

The academic major

Students majoring in the arts and humanities, the sciences, and the social sciences acquire different bodies of knowledge and learn different analytical techniques. We also know that learning activities, reward structures, and norming influences vary by discipline. This suggests the academic major plays a significant role in shaping expectations for student learning outcomes and the pathways by which they are achieved.[22] Shouldn’t student information behaviors vary by academic major as well? Our assessment tools should be sensitive to these differences.

The capstone experience and upper level coursework

Capstone courses are culminating experiences for undergraduate students in which they complete a project “that integrates and applies what they’ve learned.”[23] I think we should be studying information use during these important times for several reasons. First, there is ample evidence that the time and energy students devote to college is directly related to achieving desired learning outcomes.[24] Students who work hard learn more. Furthermore, students exposed to high-impact practices such as capstone experiences are more likely to engage in higher order, integrative, and reflective thinking activities.[25] Finally, there is strong evidence that student learning is best detected later in the academic career.[26]

If faculty expectations are at their highest and student effort is at its peak during the capstone experience and in upper-level coursework, shouldn’t studying student information behaviors during these times yield valuable data about library impact?

Speaking the language of learning outcomes

Assessing information use during upper-level and capstone coursework in the academic major is only part of the puzzle. We also need to link library use to student learning outcomes that are meaningful to administrators and policy-makers. I’d like to share two frameworks for student learning outcomes which I think hold great promise.

The Essential Learning Outcomes and the VALUE Rubrics

The Association of American Colleges & University’s (AAC&U) Liberal Education and America’s Promise (LEAP) project[27] defined fifteen ‘Essential Learning Outcomes’ needed by 21^st century college graduates such as critical and creative thinking, information literacy, inquiry and analysis, written and oral communication, problem solving, quantitative literacy, and teamwork. These outcomes are applicable in all fields and highly valued by potential employers. A companion AAC&U project called VALUE (‘Valid Assessment of Learning in Undergraduate Education’) generated rubrics that describe benchmark, milestone, and capstone performance expectations for each outcome. These rubrics are intended to serve as a “set of common expectations and criteria for [student] performance” to guide authentic assessment of student work and communicate student achievement to stakeholders.[28]

Tuning

Some student learning outcomes are discipline-specific. For instance, one would expect students majoring in chemistry, music, or economics to acquire different skills and competencies. A process called Tuning is intended to generate a common language for communicating these discipline-specific outcomes.[29] First developed as a component of the Bologna Process of higher education reform in Europe,[30] Tuning is a process in which teaching faculty consult with recent graduates and employers to develop common reference points for academic degrees so that student credentials are comparable within and across higher education institutions.[31] Expectations are set for associate, bachelor, and master degree levels. Generic second cycle or bachelor degree level learning expectations as defined by the European Tuning process are noted below. Recent work funded by the Lumina Foundation has replicated this work in three states to test its feasibility in the U.S.[32]

Subject-specific learning expectations for second cycle graduates[33]

• Within a specialized field in the discipline, demonstrates knowledge of current and leading theories, interpretations, methods and techniques;
• Can follow critically and interpret the latest developments in theory and practice in the field;
• Demonstrates competence in the techniques of independent research, and interprets research results at an advanced level;
• Makes an original, though limited, contribution within the canon and appropriate to the practice of a discipline, e.g. thesis, project, performance, composition, exhibit, etc.; and
• Evidences creativity within the various contexts of the discipline.

The VALUE rubrics are currently being evaluated in several studies[34] and colleges and universities have begun using them internally to articulate and assess student learning outcomes.[35] While the Tuning process hasn’t yet ‘taken off’ in the U.S., the Western Association of Schools and Colleges recently announced a new initiative to create a common framework for student learning expectations among its member institutions.[36] As colleges and universities experiment with and adopt these frameworks, we should incorporate them into our library assessment tools.[37]

New tools for generating convincing evidence of library impact

As part of my doctoral research I created the Understanding Library Impacts (ULI) protocol, a new suite of instruments for detecting and communicating library impact on student learning outcomes. The protocol consists of a student survey and a curriculum mapping process for connecting library use to locally defined learning outcomes and the VALUE and Tuning frameworks discussed above. Initially developed using qualitative methods[38] the protocol has been converted to survey form and is undergoing testing during 2011. I illustrate how it works with a few results from a recent study.

A pilot project was conducted during the spring of 2011 with undergraduate history majors at two institutions in the U.S., a liberal arts college and a liberal arts university. Faculty members provided syllabi and rubrics regarding learning objectives associated with researching and writing a research paper in upper-level and capstone history courses. History majors completed the online ULI survey after completing their papers.

First, students identified the types of library resources, services, and facilities they used during work on their research papers, including:

• Electronic resources, such as the library catalog, e-resources and databases, digitized primary sources, and research guides.
• Traditional resources, such as books, archives, and micro-formats.
• Services, such as reference, instruction, research consultations, and interlibrary loan.
• Facilities and equipment, such as individual and group study space, computers, and printers.

The forty-one students who participated in the pilot project collectively reported 590 types of library use during their capstone projects ranging from e-journals, digitized primary sources, books, archives, research consultations, and study space. Electronic resource use dominated, but traditional resources, services, and facilities made a strong showing.

Students then identified the most important e-resource, traditional resource, service, and library facility for their projects and when each was found most useful.  At one study site over 60% of students said library-provided e-resources were important when developing a thesis statement.  And 90% of students said both library-provided e-resources and traditional resources were important when gathering evidence to support their thesis.  Over 80% of this cohort reported library services were important during the gathering stage.  These services included asking reference questions, library instruction, research consultations, and interlibrary loan.  These data help link library use to learning outcomes associated with capstone assignments and to the VALUE and Tuning frameworks.

Students reported next on helpful or problematic aspects of library use. For instance, students at both study sites extolled the convenience of electronic resources and the virtues of interlibrary loan, while several complained of inadequate quiet study space and library hours. Information overload and ‘feeling overwhelmed’ were also frequent problems. Time savings and ‘learning about sources for my project’ were mentioned often in regard to library services.

A series of open-ended questions ask about a challenge the students faced during the project. Almost fifty percent of the student-reported challenges were related to finding and evaluating sources and almost as many were related to managing the scope of the paper and issues with writing. Faculty and librarians can ‘drill down’ into these rich comments to understand challenges students face and shape collaboration faculty-librarian collaboration to meet the needs of future student cohorts.

Open ended questions also elicit powerful stories of impact. When asked what she would have done without JSTOR, one student replied:

“I honestly have no idea. I may have been able to get by with just the books I checked out and Google searching, but those databases, JSTOR specifically, really helped me.”

I hope these glimpses of recent pilot study results demonstrate the value of focusing our attention on important and memorable academic activities in students’ lives. Using both quantitative and qualitative methods helps us understand how and why libraries support students when the stakes are highest. Authentic user stories coupled with links between library use and student learning outcomes serve as rich evidence of library impact to support both advocacy efforts and internal improvements.

Future uses

The Understanding Library Impacts protocol is not designed to assess student learning; teaching faculty and assessment professionals fulfill this role. The protocol is intended to link library use with existing assessment frameworks. ULI results can then be used in concert with other assessment data enabling new partnerships with teaching faculty and assessment professionals. For example:

• The AAC&U Essential Learning Outcomes map well to general education outcomes at many colleges and universities.  The protocol’s use of the VALUE rubrics creates a natural vehicle for articulating library contributions to these outcomes.
• Understanding Library Impacts results may also integrate with third-party assessment management systems (AMS). As Megan Oakleaf noted in the Value of Academic Libraries Report, integrating library assessment data with AMSs allows the library to aggregate data from multiple assessments gathered across the library and generate reports linking library use to a variety of outcomes important to the parent institution.[39]

It is critical to find ways to connect library use in all its forms with learning outcomes important to faculty, students, and stakeholders. Doing so will bring the library into campus-wide conversations about support for student learning.

Thanks to Ellie Collier, Hilary Davis, and Diane Harvey for their comments and suggestions that helped shape and improve this post.  Thanks also to Hilary and Brett Bonfield for their help preparing the post for publication. I also want to thank the librarians, faculty members, and students at the study sites for their support and participation in this pilot study.

Librarians are always struggling to convince someone of something: convincing voters to say ‘yes’ to a library bond; persuading a library director to invest in a text-messaging reference tool; trying to get students to use library resources instead of Google. One of the most effective ways to be successful is to learn the art of “making it their idea.”

In his book, The Education of an Accidental CEO, David Novak (2009) illustrates a crucial idea in advertising a product:

You can tell people to go out and buy something, but that doesn’t make them do it. But if you appeal to both the head and the heart in a compelling and relevant way, then people will come up with the idea to buy of their own accord (p. 44).

Novak goes on to describe how Nike uses minimal language in its commercials, never telling viewers to buy their shoes. Instead, they fill the screen with images of professional athletes performing amazing feats in their products. The idea is to let the customer come to the conclusion that Nike shoes will help them accomplish their athletic goals.

In fact, very few advertisements tell people explicitly to do anything. They present information that leads customers to come up with the idea of buying their product on their own.

Convincing people by “making it their idea” isn’t unique to marketing. In Throwing the Elephant: Zen and the Art of Managing Up (Bing, 2003), there’s a chapter devoted to “convincing the elephant that it was the elephant’s idea” (followed by “Getting Drunk with the Elephant” and “Frightening the Elephant with Mice”). Though done with a little more tongue-in-cheek panache, this book highlights the usefulness of the concept in leadership and management.

Why does this approach work so well? Business people might argue that “making it their idea” is an ego boost managers need in order to act on something. However, educators have long understood the value in letting people come to their own conclusions, and it has less to do with ego than it does with the way the brain learns. People feel a rush of pride when they come up with ideas, solutions and concepts for themselves and see the value in what they have just learned much more clearly than if they had simply been told a good idea. When it comes to seeing the value in libraries and their resources, we need to leverage a mode of teaching that allows students to experience information literacy concepts in this way.

The Learning Cycle

In The Learning Cycle, Ann Cavallo and Edmond Marek (1997) describe a teaching technique used in science education that presents students with deliberately confusing or confounding situations. With minimal instruction, students try to make sense of these situations based on prior knowledge, observation, and experimentation. At its core, the learning cycle method embodies the nature of science and helps students develop critical thinking skills.

Cavallo (2008) describes an example of the learning cycle, illustrating how it works. In an activity called “The New Society,” a small subset of a class is sent outside while the instructor tells the remaining students that they are a new society with three simple rules:

1. They can only say the words ‘yes’ and ‘no.’
2. They only respond to people of the same gender and ignore those of opposite gender.
3. Regardless of the question, they always respond ‘yes’ if the questioner is smiling and always respond ‘no’ if the questioner is not smiling.

The students sent outside (called ‘the anthropologists’) are asked to find out as much information as they can about this new society that has recently been discovered on a remote island.

As the anthropologists move about the classroom, they are confronted with confounding answers. They quickly discover the first rule through their initial observations. The second rule takes more time – often students will develop hypotheses and test them on students leading to the discovery of the second rule.

The third rule is much more difficult to figure out. Students feel frustration, anxiety and impatience. Proclamations of “They’re lying!” and “They answer randomly!” are flung about until they finally figure out the third rule.

Throughout this process, the teacher simply provides guiding questions when people get stuck, on occasion reminding them how scientists find things out by making hypotheses and testing them out.

Two results of this lesson for students: utter joy or relief from solving a frustrating problem and experience working in a confusing environment but inventing a solution to a problem themselves (without the instructor providing the answer).  The joy or relief is what builds a love of learning into the experience, and the act of inventing a solution is critical thinking in action.

How People Learn

While many of us have been told that active learning and critical thinking are vital for our information literacy programs, very few of us understand the ‘how’ and even fewer the ‘why.’

Active learning is important because it more closely models the way that humans learn. Experiments carried out by Piaget (1973) and other noted educational psychologists (Renner & Marek, 1988; Inhelder & Piaget, 1969) indicate that all learning begins with data collection (called assimilation in Renner & Marek, 1990).

This assimilation can be the observation of a phenomenon or reading of new materials. In many cases, the new data is incongruous with the learner’s current view of the world, and they can’t make sense of it.

The next step in learning is trying to make sense of the new information (called accommodation in Renner & Marek, 1990). Critical thinking skills are developed during this phase as learners make sense of the new information by inventing rules, testing hypotheses, and changing their world view in light of this new data.

In this stage, they are no longer just memorizing information or learning a series of clicks; rather, they are actively inventing new ways of understanding the world and taking ownership of the knowledge they’re creating.

The final step is called organization (Renner & Marek, 1990), and this is when they use their newly created knowledge and skills to solve other problems, and figuring out where else their new knowledge can be applied.

The learning cycle instructional method – giving students a new situation, asking them to make sense of it, and serving merely as a guide in their process – models the way people learn, and as a result, generates authentic, meaningful learning experiences for students. Compared to lectures or demonstrations where students are told what the answers are and then perform exercises that verify that what they are told is correct, they are making the new knowledge out of their own ideas.

Library Instruction as Science?

Modeling instructional activities after the way people learn is vital for making learning experiences that ‘stick.’ Typical library instruction involves copious amounts of “click here, then click here, and once you’re there, click here.” There’s little discovery or invention of core information literacy concepts. Students are told how to use information resources, told how to use citation styles, and told the consequences of unethical use of information. How can we make discovery of information literacy concepts more… scientific? Can students invent information literacy concepts on their own, given a scenario and a librarian as a guide?

Let’s take peer reviewed journals as an example. At its worst, library instruction on this topic is equivalent to “Check this box for peer reviewed articles in your results. It’s what your professors want.” This kind of instruction not only goes against the way people learn new ideas, but also undermines the importance of the peer review process by reducing it to “because your professor wants it.”

Active learning can be used to get students to explore issues of peer reviewed journals and have them compare them to magazine or popular literature. While this introduces the element of discovery and active learning, it’s only discovering the difference between the two types of publications, not the importance of the peer review process. If a librarian in this class room tells them why peer review is important, even after this activity, it’s still telling, not students discovering.

Instead, I develop learning cycles that reflect how people learn. In this instance, I give students a situation where they don’t have an answer but must work together to solve a problem. I tell students they have decided to start a magazine and they want to publish the best, newest research done in educational psychology (or whatever field they’re majoring in). Unlike TIME or Newsweek, their articles should be useful for researchers who are pushing the boundaries of knowledge in their field. They plan on sending out a call across the Internet asking for people to send in their best papers for the magazine.

I then ask the students to come up with a method for judging how good a paper they receive is and let them go to it. As they come up with criteria (e.g., “It has to be undiscovered knowledge” and “It must be based on sound evidence”), I ask how they, as college students, will be able to tell what’s good and what’s not. Who is qualified to answer those questions? How will they, as the editors, use these people?

As they work to create this new publication, they will be inventing peer review. Peer review will be an idea that they came up with themselves. They may call it something else, but the core purposes of peer review will be in their responses. As a library instructor, my goal is to guide them with questions that challenge their thoughts, and finally, give it the label of ‘peer reviewed’ once they’ve established the concept.

This lesson models how the mind actually works.

There Isn’t Time!

Learning cycles, like the one described above, take lots more time. It would have taken at most two or three minutes to explain peer review and have students tell you why it is an important feature of scholarly research. However, if students don’t invent it, it’s much less likely to stick.

The learning cycle on the other hand would take twenty or thirty minutes. Librarians don’t have the luxury of time!

There are some solutions. In an article for the Texas Library Journal, Jeremy Donald suggests a model of library instruction that offloads most of the technical details to online tutorials and learning modules (see “Step 6” in Donald, 2010). This enables library instruction to devote needed time to the learning cycle.

Donald’s model requires librarians to think about the instructional needs of student in a different way. Rather than think linearly about what skills and knowledge students need to have, think about the tasks they need to do in order of difficulty or complexity. What parts of the lesson will be most confusing and most important? Identify one or two concepts, and plan on spending at least half of your time on those topics, including time students explore new tools and ideas independently and running learning cycle-style lessons.

The rest of the time is devoted to brief introductions and answering questions. This type of model not only creates the time needed to run meaningful, engaging lessons on key topics, it forces library instructors to identify what those core topics are, the first step in developing good learning cycle lesson plans.

Developing Learning Cycles

With that said, the first step in developing a learning cycle lesson plan is to identify those core concepts students should learn. For example, for a lesson on plagiarism, some of the topics that may come up are:

• What is plagiarism?
• What are the penalties for academic dishonesty?
• How do you effectively use quotes or paraphrasing?
• How do you cite articles using a specific citation style?

Of these, I see the second (academic dishonesty policies) and the fourth (mechanics of citation) as topics that could easily be off-loaded to online tutorials or even printed brochures. There’s no need to spend time in class covering these topics, short of connecting students with resources to learn more about them and their importance.

The other two are great topics for learning cycles. I usually approach these topics from a personal perspective: how did I come to understand these concepts myself? What’s important about them? How can I create situations or activities that will lead students to invent the concepts on their own?

At its core, avoiding plagiarism means giving credit for someone else’s work. How can I get students to come to understand this concept without simply telling it to them?

Before I tell students what the class is about, I ask them to take out a sheet of paper and be prepared to write down the first word or phrase that comes to mind after I say a secret word. When students are ready, I shout, “Plagiarism!” They scribble words and phrases down then I ask them to hold up their papers. Words associated with malicious cheating usually crop up: stealing, dishonest, and sometimes lazy.

I then ask them to take on the role of summer school teacher with an imaginary group of low-performing students in an English class. They are told they’ve received a paper from a student written fairly poorly, but right in the middle, a sentence or two of pure academic gold. What happened? When they say “Plagiarism!” I ask them to describe the actual events and student actions that led up to this. I ask them to think about student motivation and behavior, and I prompt them with questions like, “What was going through the student’s mind when they pulled in these sentences into this document?”

What results is astounding. Students describe quite innocent situations: perhaps the student didn’t know that copy-and-pasting information without quotes was wrong; maybe they couldn’t find an author on the website and assumed you didn’t need to cite anonymous sources; or perhaps it was malicious cheating.

Usually students don’t view this situation as the latter. Instead, they’re forced to revise their own definitions of plagiarism based on the critical examination of the scenario they were presented with. Plagiarism is no longer cheating or stealing… so what is it?

Again, these discussions take time, but they’re valuable experiences that students will be able to apply in more situations.  In these scenarios, students are employing critical thinking skills – they are working through problems by discussing them with peers, proposing potential solutions, and evaluating their own and others’ responses. There’s more to a learning cycle than rote memorization of the concepts the instructor intends to teach; instead, it’s problem solving.

Conclusion

Library instructors should develop a “less is more” philosophy. There is real value in spending time on learning cycles because it does more than just pay lip service to active learning and critical thinking – it helps students develop them.

Faculty members and students alike may be anxious if they don’t get the step-by-step instructions they’re used to from the library session. Combating this expectation is our challenge. Donald (2010) also addresses buy-in and collaboration as a way of preparing faculty members for these kinds of drastic changes to the typical library session.

Appropriately, Donald says, “They are likely to wait to hear your ideas before introducing their own, and they may re-state an idea of yours as one of their own. This is to be encouraged, as it signals their investment in the collaboration and its outcome” (2010, 129). How’s that for “making it their idea?”

For a visual representation of Jeremy Donald’s instructional design model, see his slides from a recent Texas Library Association webinar, titled “Technology & Information Literacy Instruction: A Model for Active Learning Environments” at http://bit.ly/cpt6ON.

Thanks to Michelle Millet, Ellie Collier, and Kim Leeder for their feedback on this post.

References

Bing, S. (2002). Throwing the elephant: Zen and the art of managing up. New York: HarperBusiness.

Cavallo, A. M. L. (2008). Experiencing the nature of science: An interactive, beginning-of-semester activity. Journal of College Science Teaching, 37(5), 12-15.

Donald, J. (2010). Using technology to support faculty and enhance coursework at academic institutions. Texas Library Journal, 85(4), 129-131. Retrieved from http://www.txla.org/ce/Collaboration/Donald.pdf.

Inhelder, B. & Piaget, J. (1969). The psychology of the child. New York: Basic Books, Inc.

Marek, E. A. & Cavallo, A. M. L. (1997). The learning cycle: Elementary school science and beyond. Portsmouth, NH: Heinemann.

Novak, D. (2009). The education of an accidental CEO: Lessons learned from the trailer park to the corner office. New York: Three Rivers Press.

Piaget, J. (1973). Psychology of intelligence. Totowa, NJ: Littlefield, Adams and Co.

Renner, J.W. & Marek, E.A. (1988). The learning cycle and elementary school science teaching. Portsmouth, NH: Heinemann Educational Books.

Renner, J.W. & Marek, E.A. (1990). An educational theory base for science teaching. Journal of Research in Science Teaching, 27(3), 241-246.

Photo by Flickr user Nathan deGargoyle

Recently I was lucky enough to come across the publication of a commencement speech given by David Foster Wallace in 2005 to a group of wide-eyed graduates from Kenyon College. While it’s difficult to sum up what one takes away from a four-year-degree, this particular rumination helps to qualify the value of a liberal arts education by hitting home a simple metaphor.

Wallace starts with a joke about fish. One looks to the other and says, “So, how’s the water?” The other fish replies, “What’s water?” The speech goes on to point out that a liberal arts education opens our eyes to the world around us by providing experiences that help us move beyond our assumptions. Situations and phenomena in our daily lives become more nuanced and complicated.

Helping students “see the water” is at the heart of the information literacy teaching that librarians-as-educators do. When I think globally about information literacy and what’s outlined by the ACRL Information Literacy Competency Standards, it seems we want students to open their eyes to the world of information. We want them to recognize that finding and using information isn’t as straightforward as it seems. Not so that they’ll shy away, but instead, graduate with the abilities and understandings they need to shed assumptions, ask questions, and navigate an ever-changing information landscape.

The individual goals and means of library instruction programs may vary, but some form of teaching happens at most academic libraries. Librarians’ teaching takes the form of hosting workshops, teaching courses, and being there for students when and where they need us, from reference desks to chat rooms. And our teaching efforts are driven by goals for student learning, with the hope that we can play an active role in graduating information literate students–eyes wide open in the fish bowl.

But this is a lot to accomplish in a never-ending stream of one-shot library workshops. At this collective realization a few years ago, the librarians at my library decided to face it head on. Not that we thought one-shots weren’t effective, just that we weren’t sure what they were accomplishing, exactly. Plus we were spending a lot of time teaching freshman how to find a book and an article and releasing them into the research paper abyss, and we wanted to consider other forms of teaching.

I know we’re not alone in grappling with this conundrum and I hope to hear how you and your library are working to address our shared challenge: how to design an instruction program that meets our learning goals for students.

A bit of background

To begin addressing the learning goals we had for students, we first looked to the first-year writing courses that streamed through the library, English 101 and English 102. As is most likely the case in academic libraries across the country, we had been actively trying to reach as many students with foundational information literacy know-how in their first years of academic work. No matter how tailored our instruction was to a given assignment, we still felt a bit like broken records; each workshop needed to cover the “basics” and we rarely got past the book-and-article routine. We were left unsure of the impact of our efforts. Like ducks in a pond, we appeared calm atop while our feet below paddled furiously to keep up.  So we began dreaming up our ideal instructional opportunity: a foundational information literacy course that gave students the time and space to meet the learning goals we set for them.

Though Boise State University’s library has been teaching a one-credit library research skills class for the past decade or so, it had yet to reach its potential. Titled University 106: Library research, it has historically been a self-paced course that has students complete a series of question-and-answer worksheets, for instance: “go to the Library of Congress Subject headings and find a narrower term for sports accidents.” More recently it has evolved into a project-based course where students continuously work towards several small or one more substantial culminating paper, bibliography or presentation.

In the past two years, we’ve thrown the course into perpetual beta, ever evolving the curriculum, and have been testing the waters by experimenting with how the course is taught: in-person, online, and as a themed course (for example, one semester we “themed” two courses by focusing on Business Resources and Diversity). We’ve also continuously expanded our offerings of Univ106 from one large section to 23 sections this spring semester, taught by 13 librarians.

The in-person sections of Univ106 typically meet once a week for an hour in one of the computer classrooms in the library. There is also variation in how the online sections are offered. We have one large stand-alone, self-paced course that is capped at 150 students and team-taught by two librarians. We host this online course through Blackboard, our campus course management system, in order to use the time-saving features such as automatic grading for quizzes. We also offer smaller sections online, capped at 25 students, that make use of a variety of tools, including wikis, blogs, Google Sites and Blackboard.

While all Univ106 courses share standard learning outcomes, librarians have free rein to design experiences, activities and assignments that map how students get there. This freedom has led to a lot of creativity and experimentation with teaching techniques–from active learning to building video tutorials. Here’s an example of a typical weekly assignment in the revamped University 106. A student is first asked to find a newspaper article that mentions research on their topic. As a next step, the student is asked to track down the original research article mentioned in the newspaper article. The student then answer a series of questions about the authorship, audience and kind of information they find in each article. This exercise would be supported with how-to instructional videos, step-by-step directions, and worksheets that scaffold the process. As an instructional team, Boise State librarians have shared with one another while developing their own course content. I’ve learned a great amount from my colleagues as we’ve rolled up our sleeves and mucked around in the messy art of teaching.

Student learning has been the focus throughout all of this experimentation. The first semester I taught Univ106 I had my 25 students work towards creating or editing a Wikipedia entry of their choice. They were to add significant content with the support of at least 10  information sources–their justified “top 10” resources on a topic. A lot of things went well that semester: students showed up for our hour of class each week, performed the research-related tasks I asked of them, and even seemed to get excited when it came time to edit Wikipedia live. But at the end of the semester I was left with a sinking hunch that students weren’t making connections between what they learned in Univ106 and the research they would need to do for future courses; a hunch I’ve yet to confirm, but about which I am still curious. We’ll get to more on assessment in a bit.

The problem was I spent much of that first semester fabricating a reason for my students to do academic research. By choosing Wikipedia as the genre for their final project, I’d tried to create a context that was meaningful for them (beyond, “because I said so”), but I still felt as if a majority of the students were a bit too complacent about the work. I was left wondering how to better tap into their innate curiosity; I wanted my students to have genuine questions, an authentic information need to satisfy. But was I asking too much? University 106 is a one-credit pass/fail class, after all. That’s a lot of enthusiasm and engagement to expect for one credit.

Pairing University 106 with English 102

Armed with a renewed enthusiasm for teaching, and with our eyes on the prize–laying a foundation of information literacy in the first years at Boise State –we looked to trends and best practices in the profession. Embedded librarianship has received a lot of interest in recent years, and seems to have had some success as a method of teaching information literacy skills to students at the point of need (Bowler & Street, 2008). The basic idea is to teach more than a one-time workshop in support of a project. Instead, the embedded librarian has an ongoing  instructional presence in a course or project-based situation, either online or in person through a series of tailored workshops. With embedded librarianship in mind, we embarked on a series of conversations with the First Year Writing Program to explore possibilities.

As in many academic libraries, our instruction program had for several years been targeting our teaching efforts towards English 102: Research Writing for a variety of reasons. As the course title indicates, the focus of course is to develop research-based writing abilities, and so is a good fit for library research instruction. Engl102 is also a course all students are required to take and usually take in their first year at Boise State, which opens to the door to the possibility of reaching most incoming students with meaningful information literacy instruction. So librarians set out to proactively explore how to partner with Engl102 faculty in the development of our instructional offerings so that we could identify and meet student needs.

Targeting collaborations with Engl102 also made sense because of existing partnerships with faculty in the First Year Writing Program. Thomas Peele, First Year Writing Program Assistant Director, had already been leading a curricular change to emphasize research (Peele & Phipps, 2007). Based on annual assessments of student work, the First Year Writing Program had identified students’ limited research skills as needing additional instruction. When I started at Boise State University I had assumed that building relationships with key campus partners would take years, but instead I was able to hit the ground running. Within a year of my arrival, we were already discussing possibilities for co-teaching courses or pairing English 102 with University 106 as co-requisites, and the more we talked, the more the doors kept opening wider. It’s been an instruction librarian’s dream come true; a collaboration and mutual goal to support student learning.

So, we’d found our match. The next step was to align the work librarians had been doing to redesign Univ106 with the instructional needs presented in Engl102.

PoWeR-up!

In spring 2009 Kim Leeder and I embarked on teaching four sections of linked Univ106/Engl102 courses, taking two each. Students co-enrolled in paired courses of Engl102 and Univ106. Of course, we needed to come up with a catchy way for students to recognize this new offering, and so it became Project Writing and Research (PoWeR). We pitched it to students as a combined four-credit experience that would strengthen their research-writing skills. University 106 would act as a research lab for writing assignments in English 102 and the curricula would align so that the courses would be mutually supportive at the day-to-day level. Kim and I met individually with our English faculty counterparts and designed a series of weekly activities and developed shared assignments that directly supported the research-based papers and projects students were working towards in Engl102.

Right away I felt a different level of engagement from my students. I didn’t have to spend as much time introducing the “why” for research; the context existed in the paper writing of English 102. I could instead spend more time helping students explore and refine a topic and make it interesting for themselves and their intended audience. Through their reflections and performance on assignments, it was clear students were seeing the applicability of the research side of things. I often received comments from a student who expressed in amazement that they were able to find articles for a biology assignment and other coursework. It was working! Students were becoming better researchers and beginning to understand how these skills could be applied beyond University 106.

Since then we’ve expanded PoWeR course offerings from 4 to 20 sections. This growth has been supported by a state funded grant aimed to integrate technology into teaching in higher education. We’ve spent the past fall leading a series of collaborative institutes in which librarians and English faculty worked together to build the combined English 102/University 106 curriculum and content. The institutes resulted in a series of University 106 modules of research instruction, including content, activities and assessment. The modules currently number 22 in all and cover topics from image and video searching to field research to crafting search terms. Librarians and English faculty also worked to create a combined course schedule in order to ensure the Univ106 modules directly supported the weekly writing and research expectations for Engl102.

This push towards offering 20 sections has been quite an effort for everyone involved. Collaborating closely with English faculty has made our course design that much richer and, well, more fun–certainly for us, and we hope for students as well.

Photo by Flickr user Nathan deGargoyle

But, did they really get it?

Assessment has been a key tool to aid our decision-making processes, from deciding on course offerings to how we deliver and design course content. It was clear early on that if we were to put intensive efforts towards Engl102 instruction, we needed to know if students were actually learning what we intended.

At the end of each semester we’ve collected portfolios of student work from PoWeR sections. The portfolios typically consist of final drafts of their major papers and a reflective letter in response to prompts on both their growth as a research and a writer. Librarians and English faculty developed a rubric to assess the quality of student work in terms of source variety, source appropriateness, citation use, and research strategies employed. The assessment of student work has proven to be an insightful lens into what they’re learning and what they’re not, and this has directly informed the development of course content. It’s also forced us to articulate what proficient research looks like.

The spring 2009 assessment made clear that PoWeR students were using a wider variety of higher quality sources in their work. They were also significantly more able to discuss their research strategies. Students in both PoWeR and non-PoWeR sections of English 102 struggled with citations. In response, we created an annotated bibliography assignment for use during the fall 2009 semester in order to provide formative feedback for students on citations prior to submitting a final draft. The upcoming portfolio assessment this spring will show us whether the added assignment improves student performance.

Course evaluations have also proven useful when considering course delivery and activities. Students made it clear the first semester we taught PoWeR that they would prefer a combined course schedule and course site. This seems like a logical consideration now, but it was reflective of librarians and English faculty still thinking of the courses as separate in that first semester. I think the steps we’ve taken in the last semester to build on our collaborative efforts with English faculty while growing the PoWeR program has helped to create a one-course experience for students.

Opportunities & challenges

Although I’m hopeful heading into the spring semester, I’m also aware of the challenges and opportunities ahead.

First and foremost teaching Univ106, in all of its many forms, has proven a wonderful opportunity for librarians to grow as educators. We have learned to see through the water along with our students, and will continue to learn how to teach in a way that students learn. It’s felt like a cultural shift in librarian identity; my colleagues and I have truly seen ourselves as responsible for students becoming information literate, and therefore had to fully embrace our role as campus educators. Having instructional partners in the English Department, and seeing our teaching from their perspective, has also positively influenced the way we see ourselves as educators.

But with the ultimate goal of reaching all incoming freshman, the task is a bit daunting with finite resources; good teaching takes time and effort. I’m not sure that we’ll ever be able to match Univ106 with all 70 sections of Engl102, but the challenge is there. We would need to develop a scaleable model of course design and delivery that doesn’t take us backwards when it comes to student learning.

Some librarians have expressed interest in matching a Univ106-like-course to key courses in their disciplines. This is a wonderful idea, one that would tier the library instruction program to reach our goal of graduating information literate students. But we can’t be everywhere and do everything, so our course offerings will need to grow and balance over time. The ultimate goal is to have the academic library remain at the heart of teaching and learning on campus to ensure our relevancy as an academic unit and support student success in meaningful ways.

As a next step, we’ll begin an assessment project this spring to follow Univ106 students into future courses and beyond. We’ll be curious to see if students are able to transfer the foundational information literacy skills into their upper division coursework. The hope is to be better informed about what research abilities they’re expected to have in future courses, and we’ll use this insight to inform our course learning outcomes. We’ll see if they’re in fact able to see the water.

Instruction librarians are faced with the challenge of how to design and deliver an instructional program that meets information literacy learning goals. I’d like to hear about the efforts librarians are making at your own institution to address the information literacy needs of your students. I look forward to learning from your comments.

• Bowler, M. & Street, K. (2008). Investigating the efficacy of embedment: Experiments in information literacy integration. Reference Services Review, 36(4), 439-449.
• Peele, T. & Phipps, G. (2007). Research instruction at the point of need: Information literacy and online tutorials. Computers and Composition.

I’d like to thank Ellie Dworak, Emily Ford, Kim Leeder, Ellie Collier and Derik Badman for their insightful comments and helpful suggestions for this post.  And a special thanks to Kim Leeder for offering the opportunity to reflect on our work.

Photo by Flickr member Bill Gracey

Recently I found out that it could be happening to all of us: Google and the Internet as a medium could indeed be changing the ways our brains function and process information. “As Marshall McLuhan pointed out in the 1960s,” writes Nicholas Carr in The Atlantic, “media are not just passive channels of information. They supply the stuff of thought, but they also shape the process of thought. And what the Net seems to be doing is chipping away at my capacity for concentration and contemplation.” Carr’s article in the July/August issue of The Atlantic, “Is Google Making Us Stupid?,” received some attention for accusing its readers of not being able to accomplish deep, sustained reading in the age of the Internet. According to the article, the Web is reprogramming our brains in a fundamental, biological way. (Note: for a smart, satirical look at the issue, check out Stephen Colbert’s interview with Carr).

The responses to Carr’s article came from both sides of the fence: those who agreed with with him and those who objected to the perceived insult to their intelligence. The Chronicle of Higher Education came out with three articles that expressed concern and agreement: “Your Brain on Google,” a compilation of somewhat ironic quotes from the Web, “On Stupidity,” an extended book review of “a cartload” of recent books on anti-intellectualism, and “On Stupidity, Part 2,” an English professor’s response to the problem. Meanwhile, The New York Times Technology section printed a counterpoint by Damon Darlin, “Technology Doesn’t Dumb Us Down. It Frees Our Minds,” that accused Carr of being a technophobe and insisted that “writing, printing, computing and Googling have only made it easier to think and communicate.”

The irony of the entire argument is encapsulated in the first two lines of the New York Times article: “Everyone has been talking about an article in The Atlantic magazine called ‘Is Google Making Us Stupid?’ Some subset of that group has actually read the 4,175-word article.” Darlin builds the satire by attempting to sum up Carr’s article in a Twitter “tweet” of less than 140 characters, but only skims the surface of the real irony: the likely truth that very few of the people discussing Carr’s article had been able to read the whole thing. There’s something amazing and a bit disturbing about a culture in which everyone’s opinion is equally important and valid, no matter whether or not one has even a basic knowledge of the subject.

As an academic librarian, I’m particularly interested in the implications for libraries of Carr’s article. Hand in hand with Carr’s concern about a growing inability to engage in deep reading is the equal possibility of a growing inability to engage in sustained research. Google leads us to believe that searching for information is easy when library research is complex, often frustrating, and full of twists and turns. So the next question is: does it have to be that way? It’s a given that library systems tend to be overly complicated, even for simple searches. The common refrain is: how can we be more like Google?

The followup question is: do we want to?

These days academic libraries are grasping at every possible product—from federated searching to LibraryThing—that might ease our students’ apparent impatience with the challenges of research. After all, the 2002 Pew Internet & American Life report, “The Internet Goes to College,” made it clear that our students rely on the Web first when they’re doing research, and generally use the library only as a latter resort. If academic libraries don’t make it easier for students to find relevant information for their course projects, they may not come at all. We may as well just hand Google Scholar the keys.

On the other hand, a recent study of the research practices of college students in the humanities and social sciences offered more heartening results. Alison J. Head’s article, “Beyond Google” in First Monday (later written up for September 2008’s College & Research Libraries) found that students are using libraries in greater numbers—and earlier in their searches—than the Pew Research Center would have us believe. Granted this was a study at a single, small, liberal arts college that doesn’t necessarily reflect the situation everywhere. But we can glean some optimism from the study, along with the requisite grain of salt.

On the positive side, academic libraries have the benefit of a captive audience of students whose professors often require the use of library resources. While we may hope that these requirements train students in the ways of deep research, the day-to-day interactions at any academic reference desk would indicate otherwise. Instead, a majority of students reflect a desire to find adequate sources for a given project as soon as possible, even if those sources are not ideal. Is it Google that has raised their expectations for how quickly an information search can be accomplished? A study from the British Library calls this a “truism in the age in which we live” that “crosses all generational boundaries in the digital environment…. The speed of new media has cultivated a lowered tolerance for delay.” The study goes on to say:

There is considerable evidence to support the view that many students do not explore information in any deep or reflective manner. The lack of any evaluative efforts on the part of information users has been documented…. According to Levin and Arafeh (2002) most students stop searching at ‘good enough’ rather than trying to find the best source etc. Some ‘view the Internet as a way to complete their schoolwork as quickly and painlessly as possible, with minimal effort and minimal engagement.’

English professor Thomas H. Benton’s personal observations are nearly identical. In “On Stupidity, Part 2,” he writes:

Essentially I see students having difficulty following or making extended analytical arguments. In particular, they tend to use easily obtained, superficial, and unreliable online sources as a way of satisfying minimal requirements for citations rather than seeking more authoritative sources in the library and online. Without much evidence at their disposal, they tend to fall back on their feelings, which are personal and, they think, beyond questioning.

The echo of Carr’s article in both of these quotes is unmistakable. Whether or not Google is actually changing the biology of our brains it is difficult to say, but it does seem possible that Google could be damaging our students’ ability or inclination to conduct real research.

I’m not blaming our students. It is not the fault of anyone in particular if they are losing the interest and ability to conduct complex research. They are products of their culture, just as we all are. Just as I am.

In fact, those of us currently in our early to mid-thirties are in a unique position to address this issue. You see, I didn’t grow up with computers, but computers and I grew up together. I can remember, back in grade school, Atari and I bumbling our way through Asteroids. In high school, America Online and I had our first heady experiences in online chat rooms. When I went to college my library’s young OPAC was incomplete and I still had to use the card catalog to find certain items. Computers were leaking into my research in college, but their effect was fragmented. Google was founded the year I graduated from college.

I grew up with computers, but I grew up knowing that they were fickle, fallible, and constantly changing. I still have a collection of old floppy disks with files I will never be able to access again. I greatly enjoy technology, but I maintain a certain skepticism about it.

That said, I had to make a conscious effort to read Nicholas Carr’s article all the way through. The first time I linked to it, I skimmed the first few paragraphs and bookmarked it. The second time, I skimmed further into the text. I didn’t actually read the whole thing until I chuckled at Darlin’s observation on how few had read it and realized that I was not one of them.

What happens to our libraries in a culture where sustained reading and deep research are skills that our students and patrons increasingly do not value? There is no easy answer, but the most critical thing we can do is reflect passion for our work and share it with our students. Benton writes, “Effective teaching requires embodying the joy of learning — particularly through lectures and spirited discussions — that made us become professors in the first place. It’s extremely hard, but teachers have been doing it for generations.”

Notice his admission that playing such a role is “extremely hard”; we can all appreciate his honesty there. It is hard to be an intellectual in a culture that values actors over educators. It is hard to face a constant onslaught of superficial research when we know how much richer and more inspiring information can be. But the payoff comes when we open the door and a student steps through, leaving Google aside for the moment, to consider the wealth of research tools at their disposal that they never knew existed.

If only it happened more often.

Many thanks to my ITLWTLP colleagues Derik and Brett, and to Rick Stoddart, Tom Hillard, Ellie Dworak, and Elaine Watson for offering feedback that helped shape this post.