Our first #CountryOfTheMonth is #Brazil in honor of our fellow LabintheWild team member Nigini! Here is an interesting fact about the “Copacabana beach city”: Researchers found that the city of Rio de Janeiro in Brazil has a slow pace of life (ranked 29 out of the 31 cities measured around the world).
You can listen to the RadioLab episode talking about the pace of life concept here: https://goo.gl/JRRMpa. You can also read the full research paper by Levine and colleagues here: https://goo.gl/Pbp91w.
Sharing food photos is a popular way of engaging with others on social media. Taking web-based quizzes to learn about ourselves is also a popular social media-based pastime. We wondered how these two phenomena might be leveraged to help people improve their nutritional literacy through casual learning online.
Nutrition literacy represents how well someone can find, understand, and use information about nutrition to make decisions about the foods they buy and eat. Even though knowledge about nutrition isn’t the only factor that determines what people eat, it’s an important piece of the puzzle and improving nutrition literacy is part of health promotion.
One way that humans learn is by observing others. Albert Bandura’s Social Cognitive Theory describes how this vicarious learning works. People often engage in behavior that they have not explicitly been taught, but rather learned from observing others’ (models’) actions and their consequences; they are motivated to imitate behaviors when they observe models being rewarded for these behaviors or when models are authority figures . Social media is a great place to observe others’ behaviors and how others reward them for it.
We conducted two studies using the Nutrition Knowledge Test on LabintheWild.org to see how food photos and observational learning might help people learn about the nutrition composition of different foods and improve their nutrition literacy.
In our first study, we found that when we presented people with an expert’s explanation of which ingredient in a pair of food photos made one meal higher in a certain macronutrient (i.e., carbohydrate, protein, fat, or fiber) than the other and why, they learned the ingredient-macronutrient pairs more often than when they just got feedback telling them if they were right or not. But experts aren’t usually hanging around on social media sites to teach people about the amount of protein in a food photo! Fortunately, many other people were taking the Nutrition Knowledge Test and making similar judgments. We wondered whether explanations provided by other test-takers could be as effective as those provided by experts. For our second study, we asked other test-takers to provide explanations for their choices. After we filtered out explanations from people who answered incorrectly, we presented people with peer explanations of which food in a pair of food photos had more of a macronutrient. They learned just as much as when they saw an expert explanation.
The results of our studies make sense given a previous study showing that feedback improves knowledge of nutrient composition of a single food among paid crowd workers (like those who participate in Amazon Mechanical Turk) . Our study extends these findings in two ways: 1) we used photos of complex meals rather than single ingredients, and 2) our participants were volunteers who were just interested in learning how much they know about nutrition. Another important aspect of our findings supported by theory is that observational learning does not privilege information from experts .
Our findings suggest that online quizzes could help people improve nutrition literacy through observational learning by using crowdsourced feedback with some computational assistance. On top of that, they’re engaging – almost 2,000 people took the Nutrition Knowledge Test during our two 3-month studies.
Detailed results can be found in our forthcoming paper: Burgermaster, M., Gajos, K. Z., Davidson, P. & Mamykina, L. (2017). The Role of Explanations in Casual Observational Learning about Nutrition. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
 Albert Bandura,
Richard H. Walters. 1977. Social learning theory. Prentice
Hall, Oxford, England.
 Lena Mamykina, Thomas N. Smyth, Jill P. Dimond and Krzysztof Z. Gajos. 2016. Learning from the crowd: Observational learning in crowdsourcing communities. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. http://doi.acm.org/10.1145/2858036.2858560
Software applications are getting ever more complex and so are the user interfaces. With hundreds of features and options, it can take a long time to find the thing you want to click on. To somehow rein in this complexity, many user interfaces use what we call a “split” design like in these examples:
In these interfaces, in addition to the usual way of navigating to the font you want to use or to the folder you want to open, you can also use dynamically generated shortcuts (the “Recent Fonts” section of the menu on the left or the “Recent Folders” section of the interface on the right). Sometimes, these shortcuts will have the items you are looking for and you can save time. Sometimes, the items you are looking for will not be among the shortcuts, but you are no worse off than you would have been without them — you can still find the font, folder, or whatever, the usual way.
To make use of these shortcuts (and to save yourself some clicks), one has to exert a little bit of extra mental effort to monitor the ever-changing contents of the part of the interface that is showing the shortcuts. If your mind is too busy, you can ignore the shortcuts and click through the interface the usual way. In other words, these shortcuts introduce a trade off between clicking and thinking: you can get the same job done by doing more clicking while paying little attention to the interface, or you can exert some extra mental effort and save yourself some clicks. Most people use these shortcuts some of the time, but some people use them more than others. Are these individual differences in how people use the shortcuts random, or is there a pattern?
RESULTS: The results of the Multitasking Test on LabintheWild provided some answers. On average, introverts used the shortcuts more than extraverts in our study. Also, people with high need for cognition (a personality trait reflecting the extent to which a person is inclined to engage in effortful cognitive activities) used the shortcuts more than people with low need for cognition.
The first result is interesting: prior work in psychology showed that extraversion matters on tasks where one has to maintain attention over an extended period of time. However, it is not yet clear why extraversion matters or even what impact it will have on performance: in some studies, introverts pay closer attention and in others extraverts do. Some psychologists posit that the results depend on how exciting a task is: the more exciting and engaging the activity, the more attention the extraverts pay. Perhaps this means that our Multitasking Test wasn’t exciting enough?
The second result is as expected: when given a choice between mental and physical effort, people with high need for cognition will typically exert extra mental effort.
These results indicate that not everybody benefits equally from the effort-saving innovations in modern interfaces. Perhaps the best way to keep our software usable is to keep it simple.
PAPER: Detailed results can be found in our upcoming paper:
Krzysztof Z. Gajos and Krysta Chauncey. The Influence of Personality Traits and Cognitive Load on the Use of Adaptive User Interfaces. In Proceedings of ACM IUI'17, 2017.
For quite some time now, LabintheWild has been featuring a color vision test with the slogan “Can we guess your age?”. More than 30,000 of you have participated in this test, and we thought it is about time that we thank you for adding to our data!
So what did we learn from all your hard work? We suspected that there is a relationship between color vision and age. And there is! Those of you who are between 20 and 30 years old were best at differentiating colors in our test. But it’s not so bad for those of us who are older than 30: While color vision seems to gradually decline with age, you will hardly notice a difference in daily life. This is due to the fact that our color vision declines very slowly, with enough time for us to get accustomed to it.
We were also curious whether there are any gender differences in how well we can differentiate colors. Sorry to say, but our male participants performed on average worse in this test. This also confirms previous research showing that the average female person is better at perceiving changes in colors .
Finally, we looked at those 10% of participants who scored worst in our test. And what we found is this: The 10% who scored lowest are, on average, ten years older than other participants, more likely to be male, and much more likely to be color blind. But their color vision is also affected by how they chose to set up their work environment or where they took the test. In our experiment, the bottom 10% of participants were significantly more likely to have taken the test outdoors compared to the remaining 90% of participants. They also had a lower monitor brightness than others, as well as higher ambient brightness levels, such as the case if you are outdoors in bright sunlight. As a result, this group of participants is not able to differentiate roughly 50% of the colors in an average website and infographic!
What does this mean for you? If you want to be able to see colors well on your computer, we recommend that you avoid the outdoors, or at least bright sunlight, and that you set your monitor to a higher brightness level. Unfortunately, you will not be able to make yourself younger or less male, but keep in mind that you might have a better color vision than the average person.
If you are interested in more detailed results, please contact us at firstname.lastname@example.org. We are happy to send you a draft of the academic paper that we wrote based on the data.
 Israel Abramov, James Gordon, Olga Feldman, and Alla
Chavarga. 2012. Sex and vision II: color appearance of
monochromatic lights. Biology of Sex Differences 3, 1
This is what Jim Perry, HT Morse Distinguished University Professor at the University of Minnesota, was interested in finding out. In this guest post, he describes how he used LabintheWild’s social intelligence test to see whether he could foresee group performance in his classes:
“We all are familiar with the concept of IQ. We each work in groups many times and we find group
performance to be highly variable. We might predict that group performance would be predictable
based on IQ of group members. In fact, that seems not to be the case. Recent work has shown that
there is a factor, known as c, that serves as a good predictor of a group’s ability to perform tasks. That
factor is not highly correlated with average or maximum IQ of group members. In contrast, it seems to
be well-reflected by the ability of the group members to read the emotions of other members. That
ability can be estimated by a simple, free, 10 minute test called Reading the mind in the eyes.”
“I was interested in knowing the degree to which student performance on various group tasks in classes could be predicted by student score on that “reading the mind” test. Therefore, I offered students in three classes a chance to participate. The approach was to correlate student score on that test with scores on things like draft and final papers. I invited 135 to participate by taking the test and sending me a screen shot of the result. Students who participated received a small amount of extra credit in the class, and we each thought this knowledge might be a benefit to a person in future group experiences (e.g., other classes) because a person might become more sensitive to other members’ emotions and therefore improve group performance.”
Here is what Professor Jim Perry found:
“Eighty four students chose to participate. Some people had multiple scores because they participated in different group efforts. Therefore, the final sample size was 281 observations. I used simple linear regression analysis to ask if a person’s score on the “reading the mind” test was a significant predictor of grade on group projects. The regression was not significant (p>0.05). In fact, r2 was 0.00.”
“In this study, the test score did not predict group performance. That might indeed be due to lack of such a relationship. Alternatively, it is possible that group performance was not adequately reflected in the grade assigned to the product of group performance. In academic settings, we use grades as a measure of performance but recognize that they are a weak measure. Other measures of group performance, such as completing an objective task might produce different results. Alternatively, it is possible that the variance among individuals as expressed by the test is not sufficiently large to capture the predictive capability within the population.”
“I am grateful to all the students who participated from my classes, and to Professor Katharina Reinecke.”
And we are grateful to Professor Jim Perry for testing this hypothesis! Thank you for this guest post.
LabintheWild turned two today! Thanks to all of you who have participated in our studies, sent us comments and suggestions, and have contributed to our experiments by spreading the word. We are excited to see how much LabintheWild has grown in its first two years!
During the summer, we launched an experiment to examine graph prediction tendencies amongst different cultures, particularly the contrast between Eastern and Western individuals. This study was based on “Culture, Change, and Prediction,” a 2001 paper published by Li-Jun Ji, Richard E. Nisbett and Yanjie Su. Participants in the study were shown a series of graphs, each with three points plotted and accompanying information, and they were then asked to predict the following two points on the path. According to the results in the original paper, Chinese students had a tendency to reverse the existing trend of the graph, while American students had a tendency to continue the trend, as shown in the figures below.
In our own study, we hoped to extend these results with other countries. After gathering data from about 600 participants, we focused on analyzing participants from the US, Romania, and several Asian countries.
We began by analyzing our data at the participant level. For each participant, we graphed both trials of each type of graph on one plot, to check for consistency amongst their responses.
After evaluating the American, Romanian, and Asian data sets against each other, we found that overall, there is a significant difference between the data points provided by US and Romanian participants. US Americans seem to envision graphs to follow a more predictable trend than Romanians do. However, we did not find a significant difference in the data when comparing the US to Asia or Asia to Romania.
When examining our existing data set in detail, we discovered a few compelling trends that could imply cultural differences in graph prediction trends. We are currently working on translating this study into different languages to be able to better simulate the original study by Ji et al.
How do you feel about Naver.com, a search engine shown in the screenshot below. Would you say it is badly designed?
Most Americans would feel alarmed at the sight of Naver.com. But many people prefer this design over the simplistic look of Google. In fact, Naver’s colorful design and high information density help make it the most popular search service in South Korea.
Research has repeatedly found that people’s visual preferences vary widely. This is just as true in the online world as it is in the offline world. Challenging the notion of a one-size-fits-all universal design, we find that some people like colorful, complex websites, while others prefer simple and clean designs, and still others prefer something in between.
You might argue that it doesn’t matter what a site looks like. But it does matter! If we find a website appealing, we are more likely to buy one of its products , and we perceive it as more trustworthy [4, 5]. A more fitting design can even increase our satisfaction with the site, and our work efficiency .
But who likes what? Answering this question is difficult. Our online preferences seem to depend on individual and demographic differences, such as personality, gender, or age [2, 3, 6, 7].
To find out how our visual preferences differ, we launched a study on LabintheWild. We asked people to rate a set of websites based on visual appeal. Over the course of a year, we collected approximately 2.4 million ratings from almost 40 thousand people around the world. We used this data to study how people from different countries, educational backgrounds, ages, and genders respond to a website’s colorfulness and visual complexity.
And here is what we found:
Website preferred by male participants more than by females
Website preferred by female participants more than by males
We also looked at how aesthetic preferences vary across different countries. Prior to this analysis, my stereotypical understanding was that people from Germany and Switzerland prefer simple and colorless websites, while people from China and other Asian countries prefer colorful and complex ones. But I had to correct my stereotypes, at least to some extent. For example, we found that people in Finland, Russia, and Poland liked websites with even fewer colors than the Germans and the Swiss. Similarly, people from Malaysia, Chile, and Macedonia preferred much more colorful sites than the Chinese, whose preferences actually don’t seem to be that different from those of people in the U.S.
You can have a look at the country overview below to see more details about each country’s preference for levels of colorfulness (red lines) and visual complexity (blue lines).
We also found that countries close to each other seem to often share similar preferences. For example, the neighboring countries Finland and Russia preferred the lowest visual complexity and colorfulness of all countries, followed by Sweden. Participants from Macedonia, Serbia, and Bosnia and Herzegovina—all countries that were part of former Yugoslavia—had very similar preferences for highly colorful websites. In addition, the Northern European countries in our dataset (Denmark, Switzerland, France, Germany, Sweden, Austria) preferred a lowerer colorfulness than Southern European countries, such as Italy, Spain, Greece, or Romania. Northern European countries also preferred a lower colorfulness than Asian countries, such as China, Singapore, and Malaysia. Hong Kong and Japan preferred a lower colorfulness than other Asian countries, but higher than Northern European countries. All of the English-speaking countries (Australia, New Zealand, Canada, United States, Ireland, and the United Kingdom) also preferred a higher colorfulness than Northern European countries.
These results could suggest that countries with a regular exchange of cultural values, perhaps due to migration, share similar website preferences. We are looking forward to investigating this further.
Thanks to all our participants for taking part in this study! We cannot thank you enough for your contribution to science and to making it a little less WEIRD!
If you are interested in more details about the study design and analysis, you can download the paper here (see also ).
1. Bloch, P. Seeking the Ideal Form: Product Design and Consumer Response. The Journal of Marketing 59, 3 (1995), 16–29.
2. Cyr, D., Head, M., and Larios, H. Colour Appeal in Website Design Within and Across Cultures: A Multi-method Evaluation. Int. Journal of Human-Computer Studies 68, 1-2 (2010), 1–21.
3. Hsiu-Feng, W. Picture Perfect: Girls’ and Boys’ Preferences Towards Visual Complexity in Children’s Websites. Computers in Human Behavior (2013).
4. Lindgaard, G., Dudek, C., Sen, D., Sumegi, L., and Noonan, P. An Exploration of Relations Between Visual Appeal, Trustworthiness and Perceived Usability of Homepages. ACM ToCHI 18, 1 (2011).
5. Lindgaard, G., Fernandes, G., Dudek, C., and Brown, J. Attention Web Designers: You Have 50 Milliseconds to Make a Good First Impression! Behaviour & Information Technology 25, 2 (2006), 115–126.
6. Moss, G., and Gunn, R. Gender Differences in Website Production and Preference Aesthetics: Preliminary Implications for ICT Education and Beyond. Behaviour & Information Technology 28, 5 (2000), 447–460.
7. Reinecke, K., and Bernstein, A. Improving Performance, Perceived Usability, and Aesthetics with Culturally Adaptive User Interfaces. ACM ToCHI 18, 2 (2011).
8. Reinecke, K. and Gajos, K., “Quantifying Visual Preferences Around the World”, Human Factors in Computing Systems (CHI), (2014).
LabintheWild has joined forces with TestMyBrain and GamesWithWords with the goal of Making Science Less WEIRD. Together, we are hosting a panel at the SXSWedu Conference and Festival in Austin, TX titled “Taking Research Into the Wild”. It is scheduled for Tuesday, March 4, at 9am. Come and see us there!
In the spring we conducted an experiment, in which participants clicked on several dozen dots (like the ones below) and at the end of the experiment our system made a prediction of each participant’s age based on their performance.
We were overwhelmed by the amount of interest this experiment received, by how many people were sharing it, and how many newspapers around the world reported on it. In fact, even our servers were sometimes overwhelmed: On several occasions I had to turn back to the office to restart the server after it had silently crashed underneath my desk.
When we started analyzing the data in early June, only a few months after launching the experiment, more than 500,000 people from 218 countries and territories participated. The map below shows where our participants came from (darker color means more participants).
If you were one of the participants, we thank you! With your help, we were able to gain new insights into how clicking performance differs depending on age, gender and country. Over the next several weeks, I plan to write up some of the most interesting results and share them with you.
Today I will start by telling you how the overall clicking performance changes with age. This is only one of many measures that we recorded from the experiment to “guess” people’s age, but arguably a very important one. And if you are really interested in how exactly we measured the performance, we have the geeky details at the bottom of this post.
But let’s talk about the results. The graph below shows the average performance per age between 5 and 85. The error bars show the 95% confidence intervals – the smaller the error bars, the more reliable the result.
And here we are already with some bad news for you. If you are over 17 years of age, you have seen your best times, at least in the mouse-clicking world. On average, 17-year olds are the most efficient clickers. Starting in late twenties, we all begin to very steadily get a little slower every year.
If you are under 17 years of age, the news is great. Children and adolescents get faster very quickly from year to year. This is actually very important for those of us building software for children because it means that even in a single classroom we are likely to see children with very different clicking speeds. Finally, you can see that the clicking efficiency of young children (5 through 7 year old) is very similar to the clicking efficiency of our participants aged 80 and older.
Some technical details
How exactly do we measure the clicking performance? You might have noticed that it is much easier (and therefore faster) to click on big things than on small things. It is, of course, also faster to click on things that are nearby than on those that are far away. Because our experiment was designed to adapt to the size of the screen on which it was displayed, everybody saw a slightly different version of the experiment: for some, all the dots were close together, while for others the dots were further apart.
To make it possible to compare how different people did, even though each completed a slightly different task, we used the concept of Throughput (sometimes also called Index of Performance) developed in 1954 by a psychologist named Paul Fitts. Paul Fitts developed an equation for measuring a difficulty of a clicking task that depended on how far one had to move and how big the target was. Of course, there were no personal computers in 1954 — what Paul Fitts was studying was how quickly and accurately people could point with their fingers, but, as it turns out, his findings apply to mouse clicking as well. His throughput measure is simply the difficulty of the task divided by the time it takes a person to complete it. Throughput is measured in bits per second. We realize that saying that a person has a “throughput” and measuring it in bits per second is very geeky, but it works. Think of it as speed: the higher the better.
A vast amount of research findings out there relies on studies that were conducted with American undergraduates. In fact, most of what we know about people’s perception, cognitive skills, or their behavior is derived from such studies. Yep, we know pretty well how American undergraduate students work.
But we have only found out recently that American undergrads are not necessarily representative of the world’s population. In fact, they are quite a WEIRD species: Western, Educated, Industrialized, Rich, and Democratic. (See Henrich et al.’s paper (PDF) for some excellent examples on how WEIRD some of us are.)
We totally agree that science is a little bit WEIRD. This is also why we have proposed a panel presentation at the SXSW conference next year to show how much better we can do by collecting larger, and more diverse samples on LabintheWild.org, TestMyBrain.org, and GamesWithWords.org.
In order to be selected, however, we need your vote!
Thanks for your support and for helping science become a little less WEIRD. :)
We design studies for LabintheWild based on this central idea: we trust you. Whether you are a new arrival or a long-time participant, we trust your responses to be an honest representation of your beliefs and perceptions of the Web. We also hope that you trust us - to use your information carefully and to provide a fun experience where you can learn more about yourself.
But what about other websites? Trust is a decisive factor for engagement when users go online, so it’s important to understand what in a website makes users decide - even within the first few moments of visiting that site - whether to stay or leave in search of greener (and perhaps safer) pastures.
With this question in mind, the LabintheWild team is delighted to announce our new Trustworthiness Test, where you can use your intuition to rate websites based on their trustworthiness. We’re interested in testing out claims from previous studies that we can make judgments about websites within milliseconds (!) of seeing them for the first time. We are also interested in investigating what people perceive as (un)trustworthy and how this differs around the world. At the end of the test, you’ll see how well your intuition of trustworthiness matches reality. Click here to take the test, and stay tuned for our findings!
This just in: a new study that focuses on how you use information to predict the future. Our LabintheWild summer team - Michelle, Willy, Rishav, Jon, Dianna, Katharina, and Krzysztof - has been hard at work over the past few weeks, and we’re very excited to present our new Graph Prediction study, which analyzes the way you make predictions and compares your tendencies to others around the world. Participate now!
LabintheWild has a new design! Check out the Wild Facts on our homepage to see how much we’ve grown over the past few months - it’s incredible that we now have participants from more than 200 countries and regions all over the world! With such a growing number of participants, we want to make sure that everyone can stay updated on the latest and greatest from LabintheWild. You can now subscribe or follow us on Facebook, Twitter, or Tumblr to hear about new studies and results!
Internet users make lasting judgments about a website’s appeal within a split second of seeing it for the first time. This first impression is influential enough to later affect our opinions of a site’s usability and trustworthiness.
Because the first impression counts, we developed an approach to predict whether users will find a website appealing. Given a website’s screenshot, we are now able to
In the future, we would like to enhance this approach to work on a more individual level, for example to enable a comparison of the preferences in various cultures. But of course, we need your help. Please participate in our ongoing study!
Interesting in reading more? Have a look at our recently published paper:
Katharina Reinecke, Tom Yeh, Luke Miratrix, Rahmatri Mardiko, Yuechen Zhao, Jenny Liu, and Krzysztof Z. Gajos. Predicting Users’ First Impressions of Website Aesthetics With a Quantification of Perceived Visual Complexity and Colorfulness, in Proceedings of Human Factors in Computing Systems (CHI), 2013. Download paper
Check out our new Facebook page! We’ll make sure to provide updates there if a new experiment comes online or if we have results for our existing experiments. Thanks to all of you who have already participated!
Copyright © by LabintheWild.org