When the IDB plans to "evaluate its performance from a quantitative standpoint," it's a good sign that they mean to do just that. The XO project in Haiti, discussed here with a cost breakdown here is bearing a ton (1 pages, to be precise) fruit, with the recent IDB report (PDF).

It reveals some promise, some best practices, and also reminds us of some common problems.

From the "I toldya so" files

Finding an XO laptop spark

Somewhat unsurprisingly, there were some hassles in the basic deployment and daily use of the XOs which have been common topics of debate around OLPCNews.com for some time now, from dealing with shipping, hardware and infrastructure limitations, and the importance of teacher training.

First, classroom usage in the Haiti project started out as laptop sharing among students due to "an unforeseen shortage," later explained with slightly more detail as "logistical barriers and shipping delays"). The project team made lemonade from these lemons and studied the sharing dynamic, concluding that students more comfortable with the laptop and/or students who already had an stronger academic skills tended to dominate laptop usage; furthering the educational divide.

Green and Cute does not mean Safe
The strong branding of the laptop, ideally destroying the market for stolen XOs may or may not work as a theft deterrent, but most of the Haitian children aren't willing to gamble with theirs: "However, more than half of the fourth-grade students interviewed reported feeling afraid to take the XO laptop home because they might be robbed."

Battery life correlates with attention span:

The evaluation also found great variation in student attention span per XO Camp session, with a rising attention span from 9:00 am until approximately 10:30 am, and declining attention span thereafter. One of the explanations for this variation, as
provided by observational data, was the low battery life of the XO laptops, which led to student fights over electrical outlets at approximately 10:30 am.

The actual problem then is not some mysterious mid-morning-onset Attention Deficit Disorder, but the real-world impacts of the reality of the XO battery life. It's great in black-and-white screen mode with wifi off and being used mainly for reading, but in real-life classroom situations; "the fully charged batteries were depleted within 3–5 hours, depending on usage. It is unclear whether this significant difference is attributable solely to a technical default or if the XO laptops were distributed without first being properly charged."

Even when working in a constructivist paradigm, teacher involvement and training helps:

School and MENFP staff also commented on the limits of constructivist pedagogy within the framework of the OLPC pre-pilot project... 42 percent believed that teachers and students were simply not used to a constructivist model of education and needed additional guidance to ensure its success. However, 100 percent of school and MENFP staff interviewees believed that the constructivist model was a good one to follow.

Clearly, however, the path from rote memorization to a more exploratory framework still had some ground to cover, based on not only the above sentiments, but some choice student quotes like "When I use it, I’ll understand what I’m working on much better, which will make it easier to memorize."

Haiti also reveals a valuable role for constructivist style, student-led learning in low-resource environments:

Particularly in settings of extreme poverty, where
educational materials and resources are scant and teacher quality is lacking, the implementation of child-centered learning can increase students’ ability to think independently and to develop problem-solving skills more rapidly. Technology that is explicitly child-centered and designed for individual use has the potential of accelerating educational development in the short term.

The question remains, though, where (if ever?) does that change? With unlimited funds, few would argue that better trained teachers and high-quality teaching materials would not be the best path to improve education, but in the vast middle ground, at what level is do you throw in the towel, and stop focusing on teachers and start finding other tools to improve education?

Training and Localization

Project recommendations focused on training - both technical and pedagogical training for the teachers, including more training on constructivism, with other recommendations towards safety, methodology, and ideas to deal with the power problems.

Some insightful recommendations from the teachers themselves include some of the hardware hassles OLPCNews.com readers will be familiar with (*cough*touchpad*cough*), some clear UI improvements that more recent software updates have begun to address, and (mainly) localization challenges:

Include a map of Haiti.
Include a game in which sentences appear out of order, thus requiring students to arrange the words in proper order.
Include the Cut, Copy, and Paste features typically available on computers.
Improve the touchpad, which quickly loses sensitivity and proved difficult for students and staff to use effectively.
Include a longer-lasting battery, since the current battery lasts only about 2 hours.
Include activities specific to Haitian culture. For example, when teaching about science, it would be useful to have an XO laptop activity that compared the traditional or natural medicines of several countries, including Haiti.

Some Good News

Haitian kids ♥ XO laptops

The software was not a problem - core Sugar Activities like Record, Write, Browse and Paint were well woven into curricula and represented the vast majority of classroom usage. Students would explore the other programs, but this exploration declined when activities were used in class. Also, introduction to the Internet "drastically reduced usage of the Paint and Record programs after being formally introduced to the Internet and receiving an essay-writing assignment." If you think that's a steep drop off, just wait till you show them Facebook, Twitter, and LOLCATS.

The decline in exploration is unfortunate, but could indicate student comfort and expertise levels increasing in programs they use more often. I certainly am a bigger fan of Excel than Access, and Photoshop more than MSPaint:

"when a clear activity was presented to the students and the students were given projects to work on using specific software programs, use of those programs increased steadily, while exploratory usage declined."

Actual skill and teaching improvements
The role of technology - and how to evaluate it - is the topic of lively debate. The Haiti OLPC project team did an impressive job gathering a mixture of hard numbers, survey responses and basic observations. Even so, educational progress with the laptop remains hard to measure. Some high points were clear improvement in spelling (possibly thanks in part to spellcheck) and improved writing (perhaps simply a better tool to write with?).

Regardless, there were clear process improvements for teachers:

Students wrote daily journal entries on the XO laptop. Like their written assignments, these were edited by the teachers. During interviews, most teachers said it was easier to edit students’ work done on the XO laptop. Thus they were able to spend more time working one-on-one with students and less time lecturing. Increased individual attention may have thus been a significant factor in the perceived improvement of student’s reading and writing skills.

Still, the potential value was best stated by the students themselves:

Students believed the XO laptop could be used to facilitate the learning process through the use of tools such as the Internet and a calculator because, as student #40 stated, "Everything you need is on the laptop."

Well, almost everything.