LabintheWild has joined forces with TestMyBrain and GamesWithWords with the goal of Making Science Less WEIRD. Together, we are hosting a panel at the SXSWedu Conference and Festival in Austin, TX titled “Taking Research Into the Wild”. It is scheduled for Tuesday, March 4, at 9am. Come and see us there!
In the spring we conducted an experiment, in which participants clicked on several dozen dots (like the ones below) and at the end of the experiment our system made a prediction of each participant’s age based on their performance.
We were overwhelmed by the amount of interest this experiment received, by how many people were sharing it, and how many newspapers around the world reported on it. In fact, even our servers were sometimes overwhelmed: On several occasions I had to turn back to the office to restart the server after it had silently crashed underneath my desk.
When we started analyzing the data in early June, only a few months after launching the experiment, more than 500,000 people from 218 countries and territories participated. The map below shows where our participants came from (darker color means more participants).
If you were one of the participants, we thank you! With your help, we were able to gain new insights into how clicking performance differs depending on age, gender and country. Over the next several weeks, I plan to write up some of the most interesting results and share them with you.
Today I will start by telling you how the overall clicking performance changes with age. This is only one of many measures that we recorded from the experiment to “guess” people’s age, but arguably a very important one. And if you are really interested in how exactly we measured the performance, we have the geeky details at the bottom of this post.
But let’s talk about the results. The graph below shows the average performance per age between 5 and 85. The error bars show the 95% confidence intervals – the smaller the error bars, the more reliable the result.
And here we are already with some bad news for you. If you are over 17 years of age, you have seen your best times, at least in the mouse-clicking world. On average, 17-year olds are the most efficient clickers. Starting in late twenties, we all begin to very steadily get a little slower every year.
If you are under 17 years of age, the news is great. Children and adolescents get faster very quickly from year to year. This is actually very important for those of us building software for children because it means that even in a single classroom we are likely to see children with very different clicking speeds. Finally, you can see that the clicking efficiency of young children (5 through 7 year old) is very similar to the clicking efficiency of our participants aged 80 and older.
Some technical details
How exactly do we measure the clicking performance? You might have noticed that it is much easier (and therefore faster) to click on big things than on small things. It is, of course, also faster to click on things that are nearby than on those that are far away. Because our experiment was designed to adapt to the size of the screen on which it was displayed, everybody saw a slightly different version of the experiment: for some, all the dots were close together, while for others the dots were further apart.
To make it possible to compare how different people did, even though each completed a slightly different task, we used the concept of Throughput (sometimes also called Index of Performance) developed in 1954 by a psychologist named Paul Fitts. Paul Fitts developed an equation for measuring a difficulty of a clicking task that depended on how far one had to move and how big the target was. Of course, there were no personal computers in 1954 —- what Paul Fitts was studying was how quickly and accurately people could point with their fingers, but, as it turns out, his findings apply to mouse clicking as well. His throughput measure is simply the difficulty of the task divided by the time it takes a person to complete it. Throughput is measured in bits per second. We realize that saying that a person has a “throughput” and measuring it in bits per second is very geeky, but it works. Think of it as speed: the higher the better.
A vast amount of research findings out there relies on studies that were conducted with American undergraduates. In fact, most of what we know about people’s perception, cognitive skills, or their behavior is derived from such studies. Yep, we know pretty well how American undergraduate students work.
But we have only found out recently that American undergrads are not necessarily representative of the world’s population. In fact, they are quite a WEIRD species: Western, Educated, Industrialized, Rich, and Democratic. (See Henrich et al.’s paper (PDF) for some excellent examples on how WEIRD some of us are.)
We totally agree that science is a little bit WEIRD. This is also why we have proposed a panel presentation at the SXSW conference next year to show how much better we can do by collecting larger, and more diverse samples on LabintheWild.org, TestMyBrain.org, and GamesWithWords.org.
In order to be selected, however, we need your vote!
Thanks for your support and for helping science become a little less WEIRD. :)
We design studies for LabintheWild based on this central idea: we trust you. Whether you are a new arrival or a long-time participant, we trust your responses to be an honest representation of your beliefs and perceptions of the Web. We also hope that you trust us - to use your information carefully and to provide a fun experience where you can learn more about yourself.
But what about other websites? Trust is a decisive factor for engagement when users go online, so it’s important to understand what in a website makes users decide - even within the first few moments of visiting that site - whether to stay or leave in search of greener (and perhaps safer) pastures.
With this question in mind, the LabintheWild team is delighted to announce our new Trustworthiness Test, where you can use your intuition to rate websites based on their trustworthiness. We’re interested in testing out claims from previous studies that we can make judgments about websites within milliseconds (!) of seeing them for the first time. We are also interested in investigating what people perceive as (un)trustworthy and how this differs around the world. At the end of the test, you’ll see how well your intuition of trustworthiness matches reality. Click here to take the test, and stay tuned for our findings!
This just in: a new study that focuses on how you use information to predict the future. Our LabintheWild summer team - Michelle, Willy, Rishav, Jon, Dianna, Katharina, and Krzysztof - has been hard at work over the past few weeks, and we’re very excited to present our new Graph Prediction study, which analyzes the way you make predictions and compares your tendencies to others around the world. Participate now!
LabintheWild has a new design! Check out the Wild Facts on our homepage to see how much we’ve grown over the past few months - it’s incredible that we now have participants from more than 200 countries and regions all over the world! With such a growing number of participants, we want to make sure that everyone can stay updated on the latest and greatest from LabintheWild. You can now subscribe or follow us on Facebook, Twitter, or Tumblr to hear about new studies and results!
Internet users make lasting judgments about a website’s appeal within a split second of seeing it for the first time. This first impression is influential enough to later affect our opinions of a site’s usability and trustworthiness.
Because the first impression counts, we developed an approach to predict whether users will find a website appealing. Given a website’s screenshot, we are now able to
In the future, we would like to enhance this approach to work on a more individual level, for example to enable a comparison of the preferences in various cultures. But of course, we need your help. Please participate in our ongoing study!
Interesting in reading more? Have a look at our recently published paper:
Katharina Reinecke, Tom Yeh, Luke Miratrix, Rahmatri Mardiko, Yuechen Zhao, Jenny Liu, and Krzysztof Z. Gajos. Predicting Users’ First Impressions of Website Aesthetics With a Quantification of Perceived Visual Complexity and Colorfulness, in Proceedings of Human Factors in Computing Systems (CHI), 2013. Download paper
Check out our new Facebook page! We’ll make sure to provide updates there if a new experiment comes online or if we have results for our existing experiments. Thanks to all of you who have already participated!
Many people have asked me why LabintheWild is mainly about exploring how we are different and similar in our cultural preferences. I guess one of the reasons is that I have always liked to travel and see how people around the world differ from each other. What I find fascinating, and what my research has shown, is that these differences also translate to the online world.
I first noticed this around 7 years ago when I lived in Rwanda for a while to develop an e-learning platform for agricultural advisors in the country. Being completely ignorant about cultural preferences, I designed the user interface the way I thought was nice and intuitive. I guess my German background must have convinced me that “nice” means plain, very few colors, and very little information. Whatever it was, my Rwandan co-workers didn’t like it at all. It took me a while to understand that they preferred more colors, a higher complexity of the user interface, and a linear structure that would lead them through each lesson step by step. But why? Why doesn’t everyone in the world have the same preferences? One reason might be that everything in Rwanda is a little bit more colorful. At least in comparison to Northern Germany! Their clothes, the nature, … the colors make everything look a little more cheerful.
In an attempt to explain some more preferences, I also visited a few local schools and noticed that in those schools, the education was mostly teacher-centered and students focused on following instructions. Could this maybe explain why my co-workers favored more guidance in the e-learning platform?
Well, years have past and answering these questions has proved to be a little more complicated than I originally thought and has evolved to be my central research interest. What I have found so far is that online cultural preferences are quite complex—but they give a unique and fascinating view into who we are and how we experience today’s global village called the Internet.
We’ve just launched LabintheWild with three exciting experiments revolving around culture and how it influences people’s interaction with computers. Enjoy learning about your own cultural particularities, how other cultures see the world, and make sure to check back to read about our first results for these studies.
Copyright © by the President and Fellows of Harvard College
Brought to you by the Intelligent Interactive Systems Group at Harvard University.