Wandering the Virtual Street
Creating an Objective Measure of Navigation Ability
Navigation research is a rich and intriguing field for those interested in the natural differences between individuals, but research in this area is restricted by the need to bring people to a physical location to do testing. For my senior thesis research in Psychology, I designed, developed, and tested a web-based test that sought to measure an individual’s ability to create and update a “mental map” of new environments by asking them to find the shortest path to various points around different virtual environments.
Link to research paper: https://repository.wellesley.edu/object/ir1209
Link to play the Virtual Navigation Test: https://simmer.io/@CrownOfStars/virtual-navigation-test
Skills: Psychological Research, Interviews, Survey Design, Unity, C#, Game Design, Statistical Analysis
Project Description and Goals
Navigating familiar areas and learning to recognize new places is a common human experience, but not everyone has equal skill in this area. One of the major drawbacks psychologists encounter when trying to study these individual differences in navigation ability is gathering enough participants in the real world so they can learn to navigate a new space.
I therefore set out to explore the feasibility of piloting, creating, and testing a new, easy-to-use, accessible, openly shareable online measure of navigation ability. My ultimate goal was to create a measure that could be hosted online and distributed to as many participants and researchers as possible, using up-to-date technology and psychological theory to create a virtual world for participants to explore and be tested on. This new measure would allow researchers to have a validated, easily accessible test of navigation ability that could be used for future research, increasing our understanding of individual differences and navigation ability alike.
Initial Research & Pretest
I began my project by diving into past research, learning about mental maps and their formation, small-scale vs. large-scale spatial abilities, and above all discovering the methods the researchers used to measure and quantify the spatial navigation abilities of their participants. The measures, strategies and theoretical grounding I uncovered gave me the building blocks to construct my measure.
After gathering the theoretical background I needed, I decided to create an initial version of the measure that used videos in place of a virtual environment so that I could learn about the aspects of the measure that the literature couldn’t tell me about: how difficult to make the questions, what landmarks and environments I should include, and so on. I filmed myself exploring various environments around the Boston area, then asked 37 student participants to choose the correct map of my path and where landmarks from the video appeared on the map.
The results of the pretest and the interviews I conducted after the test gave me many valuable insights to incorporate into later versions of the measure. Many participants said they do not think in terms of a map or cardinal directions while navigating in the real world, and they found it difficult to think in these terms when watching the videos. They also mentioned having trouble with the landmark tasks because they did not know what they were supposed to look for, and so felt that the test measured their memory more than their navigation abilities. This feedback let me refine my ideas and plans for the virtual navigation test even before I started coding, giving me a solid base to build on.
Test Design
Armed with the knowledge I gained from the pretest, it was time to begin designing the 3D virtual environment for the true measure. The results of the pretest showed me that difficulty was one of it’s biggest issues, so I decided to make two environments instead of four: a city environment and a wild/rural environment. I chose to create the test using Unity as I had worked with the game design platform in the past and was familiar enough with the system that I could quickly debug and create multiple versions of the 3D environments as I learned what worked and what didn’t
Learning how to sculpt a totally new environment and code everything from player movement to lighting, scene transitions and UI was a huge challenge. One of the biggest issues I faced was figuring out how to lead users around the virtual environments I created. There were only 14 unique buildings in the city environment, so I decided to use signs and trees to make each section of the city unique, and once the city was done, I had to figure out how to show participants where to go. After testing multiple ideas, I found highlighting areas where participants needed to go with Unity’s Spotlight asset was the most successful strategy. Another challenge I had to overcome was managing the switch from timed way-finding challenges to multiple-choice questions controlled by the user interface. Once all the environments were complete and connected, it was time to test the measure with actual participants.
Test Design In-Progress Gallery
User Testing & Results
To determine the reliability and validity of the Virtual Navigation Test I conducted a study with a sample of 151 participants recruited using the Amazon Mechanical Turk and CloudResearch platforms. This study consisted of five measures: The Virtual Navigation Test (VNT) created for this study, three self-report measures of mental mapping and navigation ability used in prior studies, and a demographics questionnaire. The data was collected in an online survey and was completed anonymously. All participants were from the United States and had an average age of 34. The average completion time was 53 minutes and 30 seconds.
In order to be a good test of mental mapping ability, and, more generally, of navigation ability, the Virtual Navigation Test needed to be a reliable and valid measure. The results of the study showed the VNT had good reliability and uncovered unexpected but intriguing validity results. It had no correlation with the self-report questionnaires of mental mapping and navigation ability, but these results may be because people cannot accurately predict their own mental mapping or general navigation abilities (as noted by prior studies on navigation), and the VNT is an objective measure of navigation ability.
The results also showed that environment had a substantial effect on VNT scores; this may be because early testers indicated that the city environment was more difficult to navigate than the rural environment due to shorter sight lines, fewer open spaces and similar buildings. Another result from the new Virtual Navigation Test that is suggestive of its validity is the replication of sex differences in navigation noted by prior researchers in the field. While future validation efforts will be needed, especially convergent validity data involving other performance-based navigation tests, these results provide preliminary evidence that the VNT may be a valid measure of mental mapping and, more generally, navigational ability
What's Next?
My data provides preliminary evidence that the Virtual Navigation Test may be a reliable and valid measure of mental mapping and navigation in a virtual environment, but future studies are needed to confirm the validity of the VNT and to explore the differences between mental mapping in the real world and the virtual world, as well as the differences between navigating and mental mapping in the city vs. a rural environment. Further iteration of the Virtual Navigation Test could be used to produce an increasingly valid, usable, and, potentially, expandable and adaptable new tool for navigation researchers to use when seeking to understand individual differences in navigation ability.
The code and environments used for the test can also be improved in the future by making it more modular and easier to edit, especially for those researchers who are not familiar with the Unity game platform or the C# coding language. Professor Jeremy Wilmer at Wellesley College, who served as my academic advisor for the duration of this project, retained my source code and will continue to iterate on and improve the test until it is ready for testing in future studies and hopefully distribution to other researchers for use in their studies. Once the test is made more adaptable, future researchers may be able to change the measure to suit the needs of their specific research projects without having to do extensive coding. I hope that future versions of this measure will be made widely available to help other researchers investigating individual differences and conducting research into navigation ability.