[assessment in the digital age]

Via hastac:




How to grade, assess, teach, learn and structure the learning experience for students in the digital age?

Many interesting projects are working on this question, and we invite you to share others with us below. For example:

- The Learning Record, a portfolio-based evaluation system designed to emphasize student learning, not product-based outcomes
- Nils Peterson and his colleagues at the Center for Teaching, Learning, & Technology (at Washington State University) have been working on developing new assessment strategies and forms of classroom engagement
- Pecha Kucha in the classroom - reframing the presentation from the unstructured long-form speech to the conversation-starting breakdown
- Digital Youth Research was a 3 year project to investigate how kids use technology and media in their everyday learning. They have reports available on their site, and the group recently published a book, Hanging Out, Messing Around, and Geeking Out. 
- Re-mediating assessment, a blog considering participatory assessment models in education, authored by Daniel T. Hickey, Michelle Honeyford, and Jenna McWilliams (Indiana University).
 - The DML Research Hub, funded by a MacArthur grant, is supporting two projects. One, lead by Mimi Ito, is called Distributed Learning Research Network, and works on distributed learning that happens in social environments. The other, lead by Joseph Kahne, is called Youth, New Media, and Public Participation Research Network, and investigates the ways that youth, through social and political participation in online communities, affects their capacity and motivation to engage in social and political issues.
 - Cathy Davidson and David Theo Goldberg's report, The Future of Learning Institutions in a Digital Age (also available as a free PDF). The report found that students are learning in deeply collective and innovative ways, and that learning institutions - schools - have to keep up or risk obsolescence. They offer ten principles for redesigning learning institutions and pedagogical systems to better reflect the way students learn today. The book-length version of the project, The Future of Thinking: Learning Institutions in a Digital Age will be coming out in 2010.

 Note: Image on flickr by violet.blue

[university 2.0 approaches]

The future of the university is set to change, we all know that. But how rapidly and in what ways? Peer 2 Peer University is an example of how to "hack education" and upgrade teaching and learning especially for those who cannot afford the more traditional books, laptops and professor time. Note: the future is just beginning, there is a long way to go.

The Peer 2 Peer University (P2PU) is an online community of open study groups for short university-level courses. Think of it as online book clubs for open educational resources. The P2PU helps you navigate the wealth of open education materials that are out there, creates small groups of motivated learners, and supports the design and facilitation of courses. Students and tutors get recognition for their work, and we are building pathways to formal credit as well.

For more information:

Introduction

• About P2PU
• Course list
• Team
• Advisory Group
• Frequently Asked Questions

Courses

• Behavioral Economics and Decision Making
• Copyright for Educators
• Introduction to Cyberpunk Literature
• Land Restoration and Afforestation
• Open Creative Nonfiction Writing
• Poker and strategic thinking

Unless otherwise noted, all content on the P2PU site is licensed under:

[assessment in the 21st century]

Along with helping students learn and develop skills suitable for the 21st century, our assessment techniques should also correspondingly change.

"In too many schools, too many students suffer an education of drill and memorization but are deprived of high-level thinking activities, of intellectual discussions, of opportunities to synthesize information and respond creatively—elements that form the basis of education for other students in other schools."

[17 year old drives web app development]

How amazing is this? A 17 year old student was faced with a problem: how to keep track of school work? Well, Andrew Shaper just went and created an amazing free online resource that allows students (it is geared toward secondary school and undergrad. students) to add assignments, classes and, the best bit (it's also the best bit for Christopher Dawson at ZDNet) is the ability to upload assignments via e-mail or text message to the site. Very well connected. I can imagine asking my own students to read Chapter 5 "Encoding and Retrieval from Long-Term Memory" in Cognitive Phsychology: Mind and Brain, and then they'd text this assignment to Soshiku. A really excellent example of filling a gap and being creative.


From the site:
"What is Soshiku?

Soshiku is a simple but powerful tool that manages your high school or college assignments. Soshiku keeps track of when your assignments are due and can even notify you via email or SMS.

And it's totally free."

[creative writing and new media campus week]

This week, four weeks into the Online Masters in New Media and Creative Writing, is an opportunity for all the students to get together and meet each other in real life. Yesterday was their first day, a chance for all to catch an English breath and today they're all hard at work giving presentations. I've had the lucky chance to participate as a second marker on the presentations which have been incredible. As we break for lunch, I'm able to grab a moment of thought to ruminate on the presentations and then after lunch we'll finish with the final two presentations.

This morning I've learned about writers. Not writers in general, but writers, dreams and creators who are very specific entities. Thinking about the presentations is making me reconsider my previous thinking that I might be able to group "writers" and "readers" and individual groups (though of course some may blend between both groups). Based on the the writers/creators this morning, there is no such thing as "writers" but rather "a writer" in a singular and sense unique to each creator. Everyone today has been influenced by different people, occasions, thoughts and feelings. Poignant, for Barrington Salmon, is the role his mother (mother, worker, creator, chef, inspiration) in his poetry and stories. Leo, instead, finds creativity in the work of Rollo May, Daniel Pink, Banksy, Ken Robinson and more.

Melodie Daniels spoke about not liking The Old Man and the Sea, but interestingly she doesn't like it precisely because of Hemingway's gift with language. She, like me, doesn't want to be stuck out on the boat with the old man who was "thin and gaunt with deep wrinkles in the back of his neck" (http://www.scribd.com/doc/21616/The-Old-Man-and-the-Sea). Even though Hemingway's language, at least in this story, is "spare and compact," everything is so vivid. Hemingway's language makes the reader feel there, in the boat with Santiago.

"The Old Man and the Sea could have been over a thousand pages long and had every character in the village in it and all the processes of the way they made their living, were born, educated, bore children etc. ...I have tried to do something else....I have tried to eliminate everything necessary to conveying the experience to the reader so that after he or she has read something it will become part of his or her experience and seem actually to have happened."

nb. the image on the right of this post is a scanned in version of Melodie's first poem.

Sukai Bojang is also interested in language but she's focusing more on the oracular version. Recovering folk talks and translating them into English, Sukai is hoping to not only reach a different set of readers, but also to pass on cultural artifacts and help literacy rates in The Gambia. One of her inspirations is Chinua Achebe.

Still to present are Tia Azulay and Jaka Železnikar. I'm looking forward to hearing how and if South Africa has had an impact on Tia and her writing. I'm thinking of Andre Brink, J.M. Coetzee, Breyten Breytenbach, Nadine Gordimer, Mongane Wally Serote and and and...

[gender and literacy]

Sure women and men are different and sure our brains work differently but I didn't realise how drastically different. In a presentation on boys' writing and ict that I found at the Nottinghamshire Primary ICT Framework site there is a really interesting image of a girl's brain *at rest* and a boy's brain *at rest*:



"In the resting female brain, we find just as much neural activity as in the male brain that is solving problems."

[digital literacy: what is it and do we really need it?]

I'm reading "Literacy Debate: Online, R U Really Reading?" and while I'm scrolling through the article I'm googling some of the researchers mentioned (Rand J. Spiro, Elizabeth Birr Moje and Linda A. Jackson) and looking up some of the reports and studies. I'm also skimming through Research in Research Quarterly and Journal of Research in Reading (and complaining to myself loudly because of the 12 month embargo) and examining brain scan images. Obviously I'm reading and obviously I'm doing it in a manner different from print. But is it better? Better than what exactly? I think this is where my difficulty lies. It seems, as with the NYT article, that this is a vs matter. Print vs digital. Reading vs surfing. Literacy vs adequacy. But it isn't a simple vs issue is it? The video included in the NYT article shows a white affluent family. Each family member enjoys reading but the mum says reading for her is *quiet* and requires a comfy chair: "I can't curl up with my computer." But is this part of a quantitative assessment of online reading? Is it a feature of literacy per se? I wouldn't disagree with anyone that reading online and reading in print are different. But can we generalise and say that all reading online is different from all reading in print? Can we compare Manga online to its offline sibling? I think we could even find suitable comparisons between some early more text-based hypertext stories and print novels. Maybe instead of citing the differences we should be looking at the similarities as that might form part of the base of new literacies education and assessment. Ken Pugh says that reading in print encourages a more reflective stance, allowing time for rumination. Well, would that not only hold if students/readers are encouraged to do so. I know I've skipped to the good bits in books before {of course this is firmly in my past :)}. Do we reflect on what we read *only* when we read in print? Reading online is not always just about the "short bits" that Pugh refers to. Take a look at the project "Evaluating The Development of Scientific Knowledge and New Forms of Reading Comprehension During Online Learning" run by Dr. Donald J. Leu and Dr. Douglas Hartman. Their main research questions addressed the effects that "varying levels of intensity of Internet integration into seventh grade classroom science instruction." Their general findings suggest that:

"Internet integration generates greater online reading comprehension ability. Our results suggest it is better to have no integration or high-intensity integration of the Internet for developing concept knowledge, but not low or moderate intensity integration. Our study also provides preliminary data that suggests online and traditional reading achievement tests are not correlated."

• internet integration in a seventh grade science classroom resulted in higher achievement levels in online reading comprehension. This was true for both the ORCA-IM and ORCA-Blog; two assessment instruments with good psychometric properties. Each assessment required students to locate, evaluate, synthesize and communicate information on the Internet.
• Conceptual knowledge development in science was greater among students in the high-intensity Internet integration group and the control group.
• Consistent with new literacy predictions, we found no association between either of the measures of traditional reading comprehension (January and June DRP)and the measure of online reading comprehension (ORCA-Blog). No evidence of gains on a test of traditional reading comprehension following treatment.

Of course there are different kinds of reading too. Sometimes we read for information (and then maybe on the 'net we have quicker access to more resources) and sometimes maybe we're reading for the whole tactile and sensory experience and then we want our comfy chairs and crisp pages. But as educators, parents and leaders we need not only to address the different reasons our students/children etc... might read but also how. As Gay Ivey, a professor at James Madison University says “I think they need it all.”

[job opportunity - multimodal learning]

Research Fellow in Educational Design and Evaluation (0.5fte), London Knowledge Lab, Faculty of Culture and Pedagogy Salary in the range £28,290 – £30,013 per annum, pro rata

plus pro rata of £2,323 London Allowance

Fixed term for three years.

The London Knowledge Lab invites applications for a Research Fellow to work on the design and evaluation of a technology enhanced multimodal learning environment designed to scaffold children’s social interaction and communication skills.

The TLRP TEL funded project “ECHOES II: Improving Children’s Social Interaction through Exploratory Learning in a Multimodal Environment” involves partners from the University of Edinburgh, Sussex University, University of Strathclyde, University of Birmingham, Dundee University, and University of Wales Institute Cardiff, and the position will involve collaboration with all project partners and associated research fellows.

You will collaborate in the design of the ECHOES II environment with other researchers, focusing particularly on the design and testing of the learning activities and the evaluation of the educational impact of the environment.

A Masters or a PhD in Educational Evaluation, Educational or Developmental Psychology, Cognitive Science, or a related discipline or equivalent experience is essential, as are knowledge of empirical design methods, ability to organise and run studies involving children, parents and professionals and ability to analyse and report the results of the studies up to international research standards.Enquiries should be addressed to Dr. Kaska Porayska-Pomsta (K.Porayska-Pomsta@ioe.ac.uk).

This appointment will be subject to an enhanced CRB Disclosure.

Reference: 7AC-CPLKL-4664

Closing date:22^nd August 2008

To apply online please visit http://jobs.ioe.ac.uk or tel 020 7612 6159

Further Details

Click here for Employer Profile

[assessing digital/new literacies]

I'm still trying to craft a pedagogically-sound rubric for the assessment of digital narratives (ones undergraduate students create) and am finding it really tricky. Bryan Alexander has been updating his finds on web 2.0 storytelling and education and he also wonders whether there are any rubrics out there tackling both the medium and the content. Since I originally blogged about this, I have come across interesting resources but my latest find is from Sheila Webber at Information Literacy. She shares with us a fantastic resource for developing a module on information literacy. It sounds somewhat similar to the Education Pack I've made for teaching Inanimate Alice.

"there are five 'chapters' with titles like "Learning Theories and Information Literacy" which provide summaries of some theories and issues, and short reading lists. There are also supporting documents, including an example assessment briefing and mark sheet for an assignment "Design and prepare a training event to develop information literacy skills."

Though this resource by Barbara Chivers is aimed at those teaching librarians, I'm sure all educators will find cross-overs and aspects they can use in other learning environments.

It might be more interesting to read Chivers' IT Literacy pack alongside Lynne Spichiger website assessment. (via Bryan Alexander) Lynne says:

"In developing a website that depicts a controversial subject from a variety of perspectives, we wondered if visitors to the site would be interested in exploring each of the perspectives, or if they would be partial to just one or two perspectives. Would they be partial to the European viewpoint that has predominated for hundreds of years, or would they explore competing views? Although we do not know the backgrounds of our visitors, we do know something about their behavior on the site:

Visitors to the attack scene viewed each culture's tab in roughly equal numbers.
Most of the visitors who viewed each of the non-English cultural tabs, also viewed the English tab: 1,211 of the 1,367 who viewed the French tab viewed the English tab; 1,104 of the 1,272 who viewed the Kanienkehaka tab also viewed the English tab; 997 of the 1,174 who viewed the Wendat tab also viewed the English tab; 942 of the 1,068 who viewed the Wobanakiak tab also viewed the English tab.
866 visitors viewed all of the cultural tabs.
This data suggests that many of the visitors to the website were indeed open to viewing multiple perspectives. But did they learn anything about the event and its competing viewpoints, and did they change in any way as a result of their experience?"

Read the case-study here and see the site in question here.

Don't know your learning style or want students to figure out theirs? Have a go at testing yourself at VARK. After doing the test myself, VARK rightly concluded that I'm a multimodal learner though I didn't realise I heavily favour kinesthetic learning...hrmm...will give it a think:

Your scores were:

Visual: 10
Aural: 6
Read/Write: 5
Kinesthetic: 12

You have a multimodal (VARK) learning preference.

[grading digital storytelling]

For ages now I've been on the hunt for some rubrics geared towards grading digital stories...I mean, how do we mark for both narrative (and all the aspects including point of view, plot, character, language etc...) AND the digital medium (images used, html, sound, user-interaction etc...). Bryan Alexander has been keeping track of web 2.0 storytelling and education and he also wonders whether there are any rubrics out there tackling both the medium and the content. I've found Meg Ormiston's rubrics at tech teachers and another rubric at the bottom of the "Educational Uses of Digital Storytelling" page. The latter is based on rubrics found here, Dr. Helen Barrett's work and Scott County, Kentucky Schools.









Does anyone know of any more?

[web 2.0 + storytelling = education]

Creating Lifelong Learners has an interesting post on a "Digital Storytelling Blog Carnival" featuring links to everything educators might like to know about digital storytelling. A link from his (Matthew Needleman's) post leads to an EduCause Connect conversation featuring Bryan Alexander, Director for Research at the National Institute for Technology and Liberal Education (NITLE) and Gail Matthews-DeNatale, Associate Director for Academic Technology at Simmons College.

A blurb about the discussion:

"Digital storytelling merges leading-edge technology with age-old storytelling processes. Digital stories are typically in video format but can also include Web pages, digital maps, and other emerging technology mashups. With the addition of a Web 2.0 focus, audience also becomes co-author. How do these concepts apply to pedagogy and how can instructors evaluate and assess the process and final product?"

The discussion begins with the question: "What is Web 2.0 storytelling and how is it different from multi-media?" Bryn responds: "Web 2.0 storytelling is the combination of web 2.0 platforms and practises with storytelling, the desire to tell a story and narrative structure." He also add that web 2.0 is based on the social and micro content, both these ideas have a big impact on how students can use the web.

Gail: "With the 2.0 experience there is a much lower barrier to use..."

Bryn also makes the point that educators shouldn't try to stop students from using wikipedia or googling for answers but should encourage students how to "search more broadly."

"How do you access digital story telling production?"

Gail: "I'm a very big fan of the process...the power of story as this kind of conversational iterative process is the power of assessment (formative assessment)...I give them a rubric and they give feedback according to the rubric."

Bryan: "This is the problem with the audio, you can't tell if I'm agreeing or disagreeing...it's important to recognise that we've been composing in multimedia for a long time...it's hard for us to recognise the history of technology, we tend to define tech. as the most recent thing. We can draw on how people were asssessing hypertext in the 80s and how people were assessing web pages in the 90s. You have to select evidence and materials and assess them and that process (of selection) can be assessed."

Gail: "I would add to this that there is a context, what I'd have a first year, first semester student do would be very different for a final year communication student...Sometimes it's useful to have two rubrics, one for the subject matter and one for the media literacies."

I like Gail's idea of having two rubrics...that would certainly make it clear to students exactly how their work was being assessed...but, for transliteracy or digital literacy or new media literacy etc...should we be working towards rubrics (and other strategies) that can more fully *intertwingle* form (process) and content?

Listen to the entire podcast here but I've tried to embed it below:

This discussion took place at the ELI 2008 Annual Meeting in San Antonio, Texas where Gail Matthews-DeNatale presented a session at ELI 2008 called "Digital Story Making: Understanding the Learner's Perspective" and Bryan Alexander presented a workship at ELI 2008 on "Web 2.0 Storytelling".

[blog comments vs peer review]

"What if scholarly books were peer reviewed by anonymous blog comments rather than by traditional, selected peer reviewers?

That's the question being posed by an unusual experiment that begins today. It involves a scholar studying video games, a popular academic blog with the playful name Grand Text Auto, a nonprofit group designing blog tools for scholars, and MIT Press.

[...]

The blog is read by many of the same scholars he sees at academic conferences, and also attracts readers from the video-game industry and teenagers who are hard-core video-game players. At its peak, the blog has had more than 200,000 visitors per month, he says.

"This is the community whose response I want, not just the small circle of academics," Mr. Wardrip-Fruin says.

So he called up the folks at the Institute for the Future of the Book, who developed CommentPress, a tool for adding digital margin notes to blogs (The Chronicle, September 28, 2007). Would they help out? He wondered if he could post sections of his book on Grand Text Auto and allow readers, using CommentPress, to add critiques right in the margins.

The idea was to tap the wisdom of his crowd. Visitors to the blog might not read the whole manuscript, as traditional reviewers do, but they might weigh in on a section in which they have some expertise.

[...]

Each day he will post a new chunk of his draft to the blog, and readers will be invited to comment. That should open the floodgates of input, possibly generating thousands of responses by the time all 300-plus pages of the book are posted. "My plan is to respond to everything that seems substantial," says the author."

From Blog Comments and Peer Review Go Head to Head to See Which Makes a Book Better
By JEFFREY R. YOUNG in The Chronicle of Higher Education.