C-IMAGE : city cognitive mapping through geo-tagged photos

(1)

C-IMAGE : City Cognitive Mapping Through Geo-Tagged Photos By Liu Liu BE in Urban Planning Tongji University Shanghai, China (2012)

Submitted to the Department of Urban Studies and Planning in partial fulfillment of the requirements for the degree of

Master in City Planning

at the

MASSACHUSETTS INSTITUTE OF TECHNOLOGY

MASSACHUSETTS INSTMNE OF TECHNOLOGY

JUN 19 2014

June 2014

The author here by grants to MIT the permission to reproduce and to publicly paper and electronic copies of the thesis document in whole

any medium now known or hereafter created.

distribute or in part in

Signature redacted

Department orUrban Studies and Planning

May 20, 2014

Signature redacted

Certified by

Associate Professor Brent D. Ryan Department of Urban Studies and Planning Thesis Supervisor

Signature redacted

Accepted by

Associate ProfeskV P. Christopher Zegras Chair, MCP Committee Department of Urban Studies and Planning Author

(2)

(3)

C-IMAGE : City Cognitive Mapping Through Geo-Tagged Photos Author: Liu Liu, Advisor: Brent D. Ryan

ABSTRACT

Traditional research categorizes people's perceptions towards the city environment with Kevin Lynch's five elements: node, path, edge, district, and landmark. His method has been a keystone in guiding both urban design and urban study for decades. However, enabled by the proliferation of crowd sourcing technology, this thesis tries another angle to detect, measure, and analyze people's perceptions through geo-tagged photos.

Using Python scripting language, the project downloads photos of 26 cities (an average of

100,000 photos per city) with geographic information, in North America, Europe, and

Asia, from Panoramio and partially from Flickr. The process of image analysis is built on a computer vision technique - scene understanding, which categorizes the photos into 102 scenes by content. This paper aims to introduce photos collected at the massive scale as an additional method of city image. The project name of C-IMAGE reflects the interaction between city computation and city cognition.

This thesis implies different roles of C-IMAGE in three applications for urban planning: a monitor for city forms, an evaluator for planning strategies, and a reference for urban functions. Important discoveries through these applications include that 1) C-IMAGE can partially confirm Kevin Lynch's city image efficiently; 2) C-IMAGE can disclose both agent-led and agent-less urban changes; 3) There are mainly four prototypes among the tested 26 cities, based on a seven-category C-IMAGE patterns; 4) C-IMAGE can

evaluate planning suitability because its indicators are associated with the real activity; 5)

Part of land use is statistically predictable in the case of Manhattan.

Three impacts on urban planning are highlighted: First C-IMAGE is a method to pull data from social media to describe vividly the collective perception from the public. Second it computationally extracts information from photos at a deeper level for the study of urban space and activities. Third, C-IMAGE benefits from the multidisciplinary cooperation between planning and computer science to open up a new channel and provide analytical tools for further research on the cognitive mapping of urban spaces.

(4)

KEY WORDS

City Image, Cognitive Mapping, Urban Computing, Geo-tagged Photos, Crowd Sourcing

(5)

ACKNOWLEDGEMENTS

The completion of this thesis is built on a group of wonderful people!

I would like to start by thanking my partner Bolei Zhou from CSAIL. He provided

technical support in image content analysis, which enables me to continue my research on the perception of citizens. Such a cooperative working style greatly broaden we planners

capability in the 21' century - the era of Big Data.

I would also like to thank my thesis advisor Prof. Brent D. Ryan, because he encouraged

me with various instructions particularly in the structure and application part. Many excellent ideas just came out every time after discussing with him. He guided me to think deeper, suggested me with good ideas, and pushed me to make it the best I could.

Prof. Jinhua Zhao as my thesis reader also offered me a large number of insightful ideas and powerful arguments that enriched my argument to a great extent. Since every week I will attend Jinhua's JTL meeting, I learnt not only enormous ideas to extend the

boundary of C-IMAGE, but also the skills of delivering convincing presentation and thinking critically.

I would like to thank Prof. Eran Ben-Joseph, who gave me suggestions in research

direction during the preliminary stages. And I also thank Prof. Emily Talen, who provided me guidance in organizing ideas in the class of thesis preparation.

I thank all the other attendees of JTL: Shan Yu, Liqun Chen, Xiaoyun Chang, Andrew

Lai, Corinna Li, Zelin Li, Luxi Lin, Qianqian Zhang, and Zhan Zhao.

Finally, I would like to thank Wenting Liu, who supports me all the time and gives me courage to face one and another challenge

(6)

C-IMAGE : City Cognitive Mapping Through Geo-Tagged Photos Author: Liu Liu, Advisor: Brent D. Ryan

Chapter

1

INTRODUCTION

"The city is a fact in nature, like a cave, a run of mackerel, or an ant heap. But it is also a conscious work of art, and it holds within its communalframework many simpler and

more personal forms of art. Mind takes form in the city; and in turn, urban forms

condition mind"

- Lewis Mumford (1938)

"But what we have to express in expressing out cities is not to be scorned Their intricate

order - a manifestation of the freedom of countless numbers ofpeople to make and carry

out countless plans - is in many ways a great wonder."

- Jane Jacobs (1961)

What does a city look like? For centuries, researchers, scientists, architects, planners, sociologists have been trying to answer this question. Besides, it seems everyone can have his own opinion to this simple question. This situation looks very like a famous traditional Chinese idiom called "Blind man patting an elephant". This saying comes from a story that four blind men wanted to know what an elephant looked like by touching it. However, no one got the correct answer since he only received incomplete information. The guy who touched its leg thought it looked like a pillar, but the one who patted its ear believed the elephant should be like a big fan. Back to the initial question, the city is just like a large elephant that is hard to generalize. It is not appropriate to make an analogy from human to blind men, yet everyone's understanding of the city is limited, so it becomes difficult for him to give a complete judgment about the city. On the flip side, the real look of the city is separately buried in everyone's knowledge, and collecting more information from the public can approach the truth closer.

(9)

Facing the same question, Kevin Lynch offered his answer from an innovative angle by using his method of mapping city images (Lynch, 1960). Based on social surveys and interviews, he collected a large number of perceptions from the public. After absorbing all these perceptions he mapped the image of the city by compiling and synthesizing all the data he collected.

It has been around 60 years since Lynch delivered his answer, but his theory is still influential to many aspects in contemporary planning area. Inspired by the proliferation of crowd sourcing technology, this research noticed a new opportunity to tackle this question again through cognitive mapping partially similar to Lynch's work with an updated method. To support this method, this thesis attempts different ways of applying it for many urban issues.

The technological trigger comes from the explosive growth of the Internet. It has facilitated an unprecedented social network so that people are connected very tightly. Everyone in the city is somehow transformed to a "sensor" through his mobile client, receiving and sending information all the time. Correspondingly, computation nowadays has obtained sufficiently powerful speed to collect and process large-scale data. From the perspective of urban researchers, learning the information sent from those "sensors" can provide tremendous convenience in understanding the people's cognition of their city. Therefore to establish the channel between the crowd-sourcing data and cognitive mapping becomes the key to access the real city image.

The channel promoted in this thesis is named "C-IMAGE", which highlights the interaction between city cognition and computing technology. The capital letter of "C" stands for a comprehensive meaning of city, cognition, and computing techniques. Borrowing from Lynch's concept of "city image", the "IMAGE" in this project name is the fundamental research object. As a purely fabricated word, C-IMAGE also emphasizes its independence and respect compared to the orthodox "city image".

(10)

1.1 Research Questions and Objectives

Simply starting from this initial purpose of digging out the scattered city cognition from the public using modem technology, every following purpose or research target is

elevated by the accomplishment of the previous research stage. Because this is an entirely new attempt in understanding the city image, it is hard to predict any outcome of every exploration at the beginning. Therefore, the research questions in different stages have also been adjusted accordingly. Finally, throughout this entire thesis, this research is trying to tackle these questions:

1) How to extract the holistic cognition and collect the scattered perceptions from the

public through modem techniques?

2) How to apply such extracted knowledge to urban problems about detecting shifts in city changes, planning strategies, and urban land use?

3) What are the implications from the findings, such as comparability to city image and

judging planning suitability, after applying this new method?

Given the rapidly developed data collection and processing technology, this thesis is aiming at proposing a new method for extracting city perception and its potential usage for urban issues. It is quite clear that, as the premise of this thesis, method statement is the core objective, but when it comes to the application part, the research objective becomes various to show its diverse possibility. This is different from a normal research paper, which, in contrast, tends to focus on one or two specific aspects along with the

development of the research. In fact, the more aspects demonstrated in this paper means the more potentials this method may possess.

(11)

1.2 Thesis Outline

Before the exploration of the new method, in Chapter 2 this thesis delivers the literature review, which is divided into the cognitive mapping stream and the urban computing stream. Cognitive mapping as one of the important methods to explicit internal entities (Marr & Vision, 1982) becomes a powerful tool in urban research after Lynch's success, and it is also the foundation of C-IMAGE. The urban computing stream introduces the capability of related technology in contemporary world. After these two streams, the chapter also provides a brief look and evaluation of some similar works that attempt to combine these two streams together.

Coming along with the first research question, Chapter 3 demonstrates the framework of this method - C-IMAGE - step by step. Starting from a short discussion about data selection, it promotes the basic concepts and steps of collecting data from Panoramio and Flickr2. Following that, the study introduces a vision technology named "scenes

understanding" that can classify visionary data by its content. Because achieving the image content requires complex skills data processing from vision computation, this project invited Bolei Zhou, a Ph.D. student from CSAIL, to assist me to realize this step.

Chapter 4 starts with the second research question and ends with the third one. To answer how to apply the collaborated knowledge from Chapter 3, Chapter 4 offers three major applications of this C-IMAGE method, including a partial verification of Lynch's city image theory, planning suitability about green space in Shanghai and Tokyo, and urban indicator analysis in Manhattan. Some other supplementary applications, such as the correlation of different image based on image contents, are also mentioned in section

4.1.1. The third research question comes from some unforeseen findings after those

applications. From a limited scope of experience and knowledge, this thesis managed to offer implications accordingly. None of them has been tested, but their existence

indirectly supports the diversity of C-IMAGE and the potential capability of using such a

2 The operable and duplicable data collection process is included in the Appendix.

(12)

tool to deal with practical urban issues. In addition, by the end of Chapter 4, section 4.4 discusses the limitation of C-IMAGE.

Chapter 5 concludes the research results in section 5.1 by highlighting the findings in designing, developing, and exploring C-IMAGE in the urban study area. In section 5.2, this thesis sheds light on the impact of this research in two aspects: the significance of multidisciplinary cooperation in planning research, and a discussion about the future position where C-IMAGE should be put in the entire system of city planning.

1.3 Work Distribution

In respect to individual researcher's effort and contribution, it is necessary to clearly state the work distribution between the author Liu Liu and his cooperator Bolei Zhou to

explicit their contribution to this research.

First, Liu is the only initiator for this C-IMAGE project, and he develops the concept of

C-IMAGE including its name. The participation of Bolei Zhou comes about four months

after the birth of C-IMAGE, when Liu has prepared most part of the data and is looking for the method of computationally reading photos by contents.

Second, for the written part, this thesis is completely written by Liu. The content of section 3.4 and the supplement 1 from section 4.1.1 have referred to another paper3

written by Bolei and co-authored by Liu. There is also a clear statement at the beginning of these two sections.

Third, for the technical part of this thesis, Bolei supports in two aspects: classification via the algorithm developed by him and the similarity calculation between different groups of photos. The contribution of the former part is to computationally classify photos into 102

' This paper is named "Recognizing City Identity via Attribute Analysis of Geo-tagged Images ", and it has been submitted the 2014

ECCV conference. However, given the fact that the paper is under review, the complete content is not provided for this moment. It

may be searchable after October 2014 if approved.

(13)

C-IMAGE : City Cognitive Mapping Through Geo-Tagged Photos

attributes that is discussed in section 3.4. The latter part is applied in supplementl form section 4.1.1 and the prediction task in section 4.3.1. Liu is in charge of all the other technical parts, including data acquisition via Python scripts and API, plotting C-IMAGE through scripts in R, and the statistical calculation such as the Average Nearest Neighbor and Moran's I Z-scores. Liu has promised to provide all the scripts that are written by him in the address provided in Appendix I.

Finally, for those diagrams, if no source information is given by the footnote, it indicates the author has created the figure or the table by himself.

(14)

Chapter 2

LITERATURE REVIEW

This chapter offers a brief review of the development of the two related areas and the mixed area of both in this research: the application and development of the cognitive mapping stream and the urban computing stream. Because as an essential expression of individual's internal entities (Marr & Vision, 1982), cognitive mapping is also an ideal form for C-IMAGE. As to the urban computing stream, it covers modern technology about the acquisition and analysis of Big Data, which provides the technical support for

C-IMAGE. Both of them are tools borrowed from fields other than urban planning to

describe behaviors, summarize structures, and generalize patterns of cities, and quite often such description enhances the understanding of a particular structure or system. Examples can be found throughout the following parts of this chapter.

However, in many other aspects, these two streams do not have much similarity, and even some of their underlying principles conflict. As a sociological research method, mapping from cognition is usually an abstraction from individual feelings or knowledge, which is subjective and qualitative. In contrast, computing is a scientific method that is

highly reliant on objective statistics or quantifiable observation.

0.0000110% 0.0000100% 0 0000090% cognytive mapping 0 0000080%0.0000070% -0 0000060% 0 0000050% 0.0000040% 0.00000(30% 0.0000020% 0 0000010% g data 0.0000000% 800 120 1840 1860 1880 1900 1920 1940 1960 1980 2000

Figure 2-1 word frequency count of "cognitive mapping", "big data", and "urban computing" (0 before 2000)4

4 It is from Google Books Ngram Viewer:

https://books.google.com/ngrams/graph?content-cognitive+mapping%2C+big+data%2C+urban+computing&year start= 1800&year

end=2000&corpus=15&smoothing=3&share=&direct urltl%3B%2Ccognitive%20mapping%3B%2Cc%3B.tl%3B%2Cbig%2Odat a%3B%2CcO

(15)

2.1 Cognitive Mapping Stream

The term "cognitive mapping" does not originate from urban study. This label comes from the simple idea to find an alternative to the stimulus-response model of humans in rats and men experiments (Tolman, 1948). The phrase of "cognitive maps" is first mentioned in describing the nervous system and later is mainly mentioned in research

from environmental psychology. The traditional widely accepted definition of "cognitive mapping" is "a process composed of a series ofpsychological transformations by which an individual acquires, stores, recalls, and decodes information about the relative

locations and attributes of the phenomena in his everyday spatial environment" (Downs

& Stea, 1973). This is a purely psychological definition, but it also highlights individual

psychological knowledge and its relationship with spatial information.

"Cognitive mapping research seeks to comprehend how we come to understand spatial relations gained through both primary experience and secondary media" (Kitchin &

Freundschuh, 2000). According to this broad definition, a cognitive map is usually an internal reflection of individual feeling towards the external information from spatial cognition. For example, as it is mentioned in Kitchin's book, some typical research questions in this area may look like "how are new routes learned?" or "how are routes between two locations remembered?" Cognitive mapping has been widely applied in a variety of fields, including fields such as psychology, education, archaeology, planning, geography, cartography, management, architecture and urban planning.

The brief recapitulation of cognitive mapping history in this thesis mainly focuses on the utilization of those cases related to the urban studies.

- The Image of the City (1960s)

Over half a century ago, Kevin Lynch (1960) illuminated how people perceive their environment and summarized five basic elements that are operable and referable in urban design practice. To capture perceptions from the public, he applied a method - inviting

(16)

people to draw their mental maps - which was an innovation without any theoretical support in urban study (LeGates & Stout, 2011, p 499). After compiling all the sketches, he used various symbols to distinguish different percentages of recognition for each part of the city. Figure 2-2 shows two typical Lynch's city images of peninsular Boston based on sketches and interviews. For example, if more than 75% of the respondents draw a line corresponding to Massachusetts Avenue on the map of Boston, he will use a fairly thick line for the position of Massachusetts Avenue of the final map to indicate a 75%

recognition from people and thus reflect the significance of this street in their perception. Finally by counting different percentages, his team created comprehensive mental maps of Boston, Jersey City, and Los Angeles. Based on the definition of cognitive mapping, Kevin Lynch successfully transplanted its concept to the research of city image.

Through this innovative approach, he overcame the barriers to detecting human perception of a city. He synthesized empirical information into a single clear map and also quantified the influence level of different parts of the city. Meanwhile, he applied a similar method using another process: interviews. As a result, for each pair of city and data source, a single map was clearly enough because it combined multiple information sources.

According to these cognitive maps, he concluded the well-known five elements - paths, edges, districts, nodes, and landmarks - that enhance a city's identity. They have been widely used as principal rules for improving city legibility.

(17)

; % . * %:1 .N ... .-... IA VAN

4-/ ~ /

*

V IV

A

PATH EDGL NODE rM6TK3C

t2 IL-251 '"

V

Figure 2-2 the image of Boston from sketches (top), the image of Boston from interviews (bottom), and the symbols of the five elements (bottom left)5

' The top figure: page 21, and the bottom figure: page 19 (Lynch, 1960).

C-IMIAGE : City Cognitive Mapping Through Geo-Tagged Photos Author: Liu Liu, Advisor: Brent D. Ryan

=C=

(18)

During the following decades, many related studies to some extent confirmed the stability of those five elements (Appleyard, 1970; de Jonge, 1962; Francescato, n.d.; Harrison & Howard, 1972). For example, Appleyard conducted research by requiring local residents to draw a map of the whole city between the steel mill and San Felix in Ciudad Guyana. Through analyzing those "inhabitant maps", he pointed out that such subjective maps provide rich information about urban perception correlated with the visible, functional, and social character of the city.

Other research slightly questioned the "five-elements" schema of urban perception (Gulick, 1963; Klein, 1967; Rapoport, 1977). These disagreements came from the diversity of soci-cultural and physical context of different areas and populations. For example, Gulick proposed that urban imageability was "a product ofthe perception of visualform and of the conception ofsocial significance " based on the mental map from

35 Tripolitan students in Lebanon. He argued that to understand the visual experience of

laymen required knowledge beyond treating the elements purely as visual concepts. It is because that people's defining criteria are social and behavioral (Gulick, 1963).

w The Psychological Maps (1970s)

Psychologist Stanley Milgram created several psychological maps of the city of Paris, which is also a process that is based on sketches and then finalized as a comprehensive map. The major difference in the research method is the design of the survey. They added questions such as recognizing and locating photos within the city and finding places of most possibility to encounter friends while waiting (Milgram, 1976). These questions were more specific to answer: on the one hand they were subject to personal preference, on the other hand, the quality of their answers was following a series of objective principles. For instance, almost no one would choose a very closed location to increase the chance of encountering his friends.

(19)

C-IMAGE : City Cognitive Mapping Through Geo-Tagged Photos Author: Liu Liu, Advisor: Brent D. Ryan 68 69 7 71 72 7 67 66 73 65 32 33 36 37 40 6 75 304 35 30 63 ₆₄ ₂₉ ₃₁ ₄₅ ₃₆ 7 9 3 41 7 62 1~~ 2 12 10 42 7 28 26 13 11 4 59 27 25 24 21 1 4 43 s 61 23 22 20 15 5 17 48 60 57 53 19 i46 5 56 52 49 47 55

Figure 2-3 perceived areas of danger in Paris (Milgram, 1976)

a The Evaluative Image of the City (1990s)

Taking one more step, Jack L. Nasar and his team developed the concept of urban likability and created the evaluative maps of city by directly asking respondents for their evaluation about the city (Nasar, 1998). Some questions they designed were fairly straightforward. For example, apart from the basic information of interviewers, they required visitors to identify up to five most visually pleasant areas and most visually

unpleasant ones. They also made a multiple-choice survey for interviewers to select the elements that contribute to visual improvement (Nasar, 1998, p 83-84).

The final result of the research was not only a map of like or dislike areas, but also a list of element features that had positive influence to visual environment (Figure 2-4). From the perspective of research question design, the team was actually trying to give more power to interviewers for evaluation than its precedents, because asking an individual about like or dislike of an area was much highly subjective than whether he can remember that area or not.

(20)

4' Ue~

-r

iyt=34~

2NZ

/

Figure 2-4 the daytime evaluative map of the University Area, Columbus, Ohio (left), and the nighttime evaluative map of the University Area, Columbus, Ohio (right), (Nasar, 1998, p 120, 121)

Therefore the answer of this evaluative map produced stronger motivation to trigger direct changes in urban form in some areas if those districts were mapped as "dislike" area. However, accounting for the varied personal value of visual comfort, it is of high risk and difficulty to unify their answers, and even it is possible, the conclusions of that

may become quite general and even pointless. Just like the five likeable features, "naturalness", "upkeep/civilities", "openness", "historical significance", "order", mentioned in that book are too broad and hard to manipulate under a practical planning

framework (Nasar, 1998, p 62).

- Recent Case

Recently in other books, cognitive mapping tended to be less like a research method, and instead it showed up in coffee-table books and became more close to a tool of recording people's perceptions of their city. Becky Cooper invited all kinds of people, including police officers, homeless people, fashion models, and senior citizens during her journey

/

(21)

in Manhattan to "map their Manhattan " and to mail these maps back to her. In the end she collected 75 maps from both anonymous mapmakers and notable New Yorkers to publish a book (Cooper, 2013). Without strictly predetermined questions for interviews, which means when respondents were freed to map, just like Figure 2-5 the outcome of cognitive maps carried more aesthetic expression.

0y '.0c. , OA 111Njfz A yk, A A -A (Vw N.lr R t7~

Figure 2-5 the cognitive maps from the book Mapping Manhalan - A Love Story in Maps (Cooper, 2013)

br 75 New Yorkers

- Summary

From the beginning of Kevin Lynch's pioneering work, cognitive mapping as the

fundamental method to detect the urban perception from the public has been set. Its basic process is to translate the information from surveys and interviews into maps, and then to discuss findings based on these maps. During the following decades, the contributions from similar cases have been mostly dedicated to the design of interviews.

Most, if not all, of these cases in the cognitive mapping stream have inherited a form of

"interview - map - compile - structure" to conduct urban research. The first two steps

usually require a relatively long time and human resources to complete for sufficient

(22)

sample size. In the perspective of sample size, Kevin Lynch's research about the image of the city has visited around 30 people in Boston and about 15 in Jersey City and Los Angeles, covering an area of 1.5*2.5 miles respectively (Lynch, 1960). The number of respondents is relatively small comparing the one used in the evaluative maps case, which is 300 residents and visitors in two U.S. cities6 _{(Nasar, 1998).}

2.2 Urban Computing Stream

Like the application of cognitive mapping, urban computing is not an independent research field derived from urban study. It is a cross-platform field in studying a variety of urban issues such as transportation, sustainability, and economy via computing

technology. This term was first appears in a study of "the familiar stranger ", but just like the outcome of "cognitive mapping", that paper fails to describe how computation

technology is embedded in research. The term only shows up in the keywords part, and the definition is quite unclear (Paulos & Goodman, 2004). Compared to the branch of cognitive mapping, the history of this area is simply no more than 10 years.

Although urban computing area is a newly born field, it has already attracted plenty of attention. In the era of Big Data, it takes the advantage of the explosive development of information and communication technologies (ICT) when mobile phones are widely used through all societal layers (Palmgren, 2008). The application of this new approach is basically located in areas such as transportation, sustainability, which can be described, evaluated, and codified in quantitative ways.

n Urban Computing on the Macro Level

One of most appropriate aspects is transportation. Based on a large number of sensors, forecasting transportation volume and flows becomes an ideal test bed for computing techniques. A team from Microsoft research center detects "flawed urban planning",

6 Knoxville and Chattanooga, TN

(23)

which means a long-term traffic overload in specific areas7 as displayed in Figure 2-6, using the GPS trajectories of taxicabs traveling in urban areas. They arrive at the result which consists of pairs of regions with salient traffic problems and the linking structure as well as correlation among them (Zheng, Liu, Yuan, & Xie, 2011). Another team takes a case study from the city-state of Singapore using the records from public transport card "EZ-Link" and local events information from the Internet to predict public transport arrivals under special events scenarios (Pereira et al., 2013).

r

Rest days

4

+

4

Figure 2-6 the "flawed urban planning" detected based on the taxi trajectories of 2009 and 2010 in the workdays and rest days (Zheng et al., 2011, p 96)

Other aspects such urban air quality, common behavioral patterns could also be generalized through this novel technology. The project U-Air proposed a new methodology of comprehensive utilization of existing sensors, infrastructure for a detailed prediction of urban air quality (Zheng, Liu, & Hsun-Ping, 2013). An European media team reports a novel deployment of UBI-hotspots in a city center to establish an

In their paper, the author partitions a city into some disjoint regions using major roads. And they project the taxi trajectories of each day into these regions and detect the salient region pairs having traffic beyond its capacity. When the overloaded region pairs are repeatedly detected across many days, they define such a situation as a "flawed planning". Here the word "planning" they used is somehow weird to the normal one in urban planning area.

2009

A

Workdays

2010

(24)

ecosystem infrastructure for conducting diverse urban computing research and business in authentic urban setting (Ojala et al., 2010).

The MIT's Reality Mining project successfully abstracted common behavioral patterns from the activities of 70 students and faculty issued with Nokia phones carrying specially designed logging software (Eagle & Pentland, 2009). Besides, some research teams are focusing specifically time data. The Senseable City Lab from MIT develops a real-time representation and pattern analysis of the dynamic of human behavior at the city-region scale through aggregate mobile phone data from Roma (Reades, Calabrese, Sevtsuk, & Ratti, 2007).

m Urban Computing on the Micro Level

Those cases above are making full use of large-scale dataset in tracking millions of trajectories to analyze on the macro level. Because the process of collecting information is from individual behavior to general patterns, therefore it can be called a "bottom-up" method. However, many more cases in urban computing area are actually "top-down" ones that are focusing more on the distribution of information, the quality of interaction, and the change in personal urban experience.

Probably because transportation is such a unique aspect that can be detected by sensors and calibrated in numbers, it occupies a large number of the applications on the micro level. For example, the software Navitime used by nearly by two million Japanese people in 2007 is a mobile phone-based navigation service that incorporates various modes of transportation. User experiences reveal implications for designing urban computing services (Arikawa, Konomi, & Ohnishi, 2007). Another team in Australia conducts a study about the effectiveness of the real-time passenger information system deployed by the Translink Transit Authority across all of South East Queensland in Australia. They suggest using urban screens and mobile applications to improve public transportation awareness (Foth & Schroeter, 2010).

(25)

Despite the diversity of the applications, most of the interactive ones can be simply realized through personal mobile devices. Projects such as the "mobile location

bookmarking" is trying to let residents use their mobile phones to leave virtual notes at places of interest and share their experiences (Calabrese et al., 2007). The "Social Patchwork project" is working to bring a cross section of new residents together to map where they have lived in the past in a process they called "urban history lines", which is as they highlighted new media in community engagement and urban planning (Calabrese et al., 2007). A chapter from the book Urban Informatics presents a shift in mobile phone usage from communication tool to "networked mobile personal measurement instrument" via their invention of mobile computing usage called citizen science (Foth & Schroeter, 2010).

In comparison with the development of the conventional cognitive mapping methodology, the urban computing stream has its own uniqueness in four aspects: 1) most of the studies are highly practical and are on-going or even at their preliminary stages; 2) nearly all of them are led by computer science research group or backed by IT company; 3) particular innovations become their key products to sell; 4) most of the research is built on

computing mega database.

2.3 Urban computing in Cognition (mixed stream)

Following the brief introduction of cognitive mapping and urban computing streams, these two tools are products of different times aiming at different aspects of urban problems. Via social surveys and interviews, cognitive mapping is to disclose individual perception towards spatial environment that is formless, while urban computing tends to analyze the digitalized world today. And considering the popular time of cognitive mapping is before 2000 and that of urban computing is after 2000, not many research cases combine these two streams. This section chooses two projects that are related with

C-IMAGE: "City Pulse" and "What makes Paris look like Paris".

(26)

' "City Pulse" Project

The project named "City Pulse" developed by a team from the Media Lab at MIT is using thousands of geo-tagged images to measure the perception of safety, class and uniqueness in the cities of Boston and New York in the United States, and Linz and Salzburg in Austria. The research finds that the range of perceptions elicited by the images of the US cities is larger than that of the other two European cities, which was interpreted by the team that cities of Boston and New York are more contrasting, or unequal, than those of Linz and Salzburg. Besides they also validate a significant correlation between the perceptions of safety and class and the number of homicides in a NYC zip code, after controlling other dependent variables (Salesses, Schechtner, & Hidalgo, 2013).

The dataset of their research is built on 4,136 geo-tagged photos cropped from Google Street View and the evaluation of those images are achieved through an on-line survey from a website where they uploaded all the photos for the project. For each vote, there is a pair of photos put together along with a question such as "Which place looks safer?" on the screen (Figure 2-7), and all the respondents need to do is to click the picture that they believe is of higher security. The statistical analysis and modeling are based on a total number of 208,738 votes.

B

-'4

A

Which place looks safer?

photos ae equal

Figure 2-7 the locations of street views collected from Google Street Views in New York City (left), and a typical

questions for online voting (right) (Salesses et al., 2013)

(27)

D

R=68.9% safety

E

R=35.3% F4 cc0 sAle 00 2

F

R2=37.9o I 8, 40 tonue Perceptions of Safety h-74' -73 83i

Perceptions of Social Class Perceptions of Uniqueness

E -4 F - -34

G Z-Score (Standard Deviations)

.45 96 46 48 45 48 485 19 %5 16

a -258

Figure 2-8 identifying places associated with different urban perceptions (top), and map of NYC showing statistically significant clusters of high and low Q-scores8 for the perception of safety/class/uniqueness (bottom) (Salesses et al., 2013)

8 The Q-score is calculated as a weighted score through an algorithm defined by the researchers based on the result of photo votes.

They scale the score to a 0 to 10 range, which means the score of 10 represents highest degree of safety/class/uniqueness and the score of 0 stands for the lowest degree.

(28)

- What makes Paris look like Paris

A team from CMU demonstrates that geographically representative image elements can

be discovered automatically from Google Street View imagery in a discriminative manner and these elements are visually interpretable and perceptually geo-informative, which can support a variety of computational geography tasks (Doersch, Singh, Gupta, Sivic, & Efros, 2012).

The dataset of the research is based on a large image collection also from Google Street View. The team adopts a "discriminative clustering approach" to treat each photo separately into different small patches, and then extracts features from them to find that the contents of those features are both significant in being identified on perception level and being geo-located by image content. Figure 2-9 shows some of the features they have extracted from European cities. As one of the key conclusions, they demonstrate that geographically representative visual elements can be detected from street view photos in a discriminative manner, and these elements are visually interpretable.

In the application part, they use the method as a tool to detect the distinctive building styles for different cities. According to geographic coordinates of the visually unique patches, they plot the elements in the map of Paris by category as shown in Figure 2-10, pointing out that each category of the elements clustered in a unique pattern. Then they also apply the same algorithm to five European cities and make a cross-city comparison at a larger scale as shown in Figure 2-11. From that regional comparison, the team generalizes five different architectural patterns of windows in fagade from different locations in Europe. Besides, their method can even detect slight difference in

architectural styles: double-arches are only rare in London while and balcony railings in Paris, Barcelona and Milan are all made of cast iron which is different from London and Prague.

(29)

C-IMAGE City Cognitive Mapping Through Geo-Tagged Photos Author: Liu Liu, Advisor: Brent D. Ryan

Random images for Paint SreeL ntracted Vsa Efement fom Pas Random imagirs for Prague Street- tew Extracted V nual E ements from Prague

B ndm mage for L ndorr Street ve -n d trrerc r L o Random mages for Barc na Street miew Extracted Eenments Prom B reon

Ra,~ ae orSa rro o FEntrace Efem ntf from SF Random Images For Boton Extactd Efements from Boston Figure 2-9 Google Street View and informative elements for six cities, as the team suggested, the

geo-informative elements (right) are able to provide better stylistic representation of a city than randomly sampled Google Street View images (left)

(30)

PMace des

- s voswes

a mmarket

Figure 2-10 examples of geographic patterns in Paris (red dots on the maps) for three discovered visual elements

(shown below each map): balconies with cast-iron railings concentrated on the main boulevards (left), windows

with railings mostly occur on smaller streets, and arch supporting columns are concentrated on Place des Vosges

and St. Germain market (right) (Doersch et al., 2012)

A SCM

3

D E

0 M b *PamU~MI

Figure 2-11 architectural patterns across Europe (Doersch et al., 2012)

n Other Related Works

The method proposed in this thesis is also inspired by some analogous study that is also using quantitative methods to measure and identify architectural design styles.

Conducting a morphological investigation based on typical characteristics of premodern city defined by traditional advocating designers, a recent research evaluates how close today's redesigned traditional neighborhoods are to the real old versions by score. And

ironically, the scores indicate a failure in recapturing conventional building environment even in those HOPE IV communities that were originally designed for that goal. (Ryan,

2013)

Other projects also from Senseable City Lab such as using cell phone data in urban analysis are also examples for the extension and application of planning tools with the help of the advanced communications platforms (Ratti, Pulselli, Williams, & Frenchman,

2006). The research of this thesis is also motivated by attempts of discussing the

validation of using image-based analysis for urban audits. For example, Eran Ben Joseph

(31)

et al. have compared the usage of three different web-based tools of Google Maps,

Google Street View and MS Visual Oblique as audit tools with the traditional on-site observation in evaluating and measuring street-scale built environment characteristics (Ben-Joseph, Lee, Cromley, Laden, & Troped, 2013).

Some other precedents about image processing support the image content classification techniques in section 3.4. In particular, the algorithm from these papers is inspiring for the "scenes understanding" technique used in this thesis. Since this thesis is not focusing on the specific technology, here offers just a brief of them: establishing a worldwide urban landmark library via image detection technology (Yan-Tao Zheng et al., 2009), extracting the haziness9 _{of single image (He, Sun, & Tang, 2011), retrieving the photos}

with required content by characterizing layouts of outdoor scenes (Lin & Xiao, 2013), and detecting the blue sky from pictures (Zafarifar & With, 2006).

9 Visually speaking, the haziness is the degree of vagueness of a photo, and it can reflect the depth of focus.

(32)

Chapter 3

METHOD STATEMENT

This Chapter states the method of C-IMAGE including data acquisition and content analysis to the first research question: "how to extract and collect the scattered

perceptions from the public through modem techniques?" Section 3.1 discusses about the selection of research data source on three levels: text vs. photos, machine-taken photos vs. human-taken photos, and Panoramio vs. Flickr. Section 3.2 provides the definition of

C-IMAGE throughout this thesis with its three basic characters.

The following two sections brick the fundamental method of C-IMAGE, which is also the foundation of all the applications in Chapter 4. To be clear, the author of this thesis Liu completes all the contents, including data acquisitions scripts and storage in section 3.3. In order to have a deeper understanding of the data, which means classifying the images

by their contents automatically, Liu invited Bolei Zhou, Ph.D. from CSAIL in MIT to

offer the visionary computation techniques in section 3.4, and most of the work in this section except those plotting to city maps is conducted by him.

Section 3.3 provides the basic concept and complete steps of downloading photo libraries from Panoramio and Flickr, and attaches all the related scripts in the Appendix so that making the entire process duplicable. Section 3.4 offers the mechanism of using scenes understanding techniques to classify photos into 102 attributes.

3.1 Selection of Cognition Representatives

The advancements of modem technology in communications have produced

overwhelmingly large amounts of information, many of which come from personal mobile devices and contain geographic tags. For researchers, it is gradually no longer a problem of where to collect massive individual data. However, choosing what type of data is appropriate for the purpose of understanding public perception of cities is yet to be discussed.

(33)

3.1.1 Texts vs. Photos

Typically texts and photos are two forms of information carrying information that are subjective and pervasive. There are a great variety of platforms providing geo-located texts and photos all over the world, but usually texts seem to be a more direct channel to convey the sender's opinion and can usually offer extra information beyond just a preference. For example, a text message of "I don't like the restaurant, because ... " not only discloses the visitor's own preference to the restaurant but also explains the reason in the second half. Meanwhile, it is difficult to clarify whether the customer likes or not only from a photo he takes in the same restaurant. With the advantage of having

straightforward meaning and further information, texts appear to edge out photos.

However, texts can raise a series of problems if we hire computer instead of human beings to read and classify them. First, the diversity of language usage increases the difficulty in text parsing. For example, it is relatively easy to detect "like" or "dislike", but for those words having vague meanings, parsing text by computer science is not easy. Second, the language barrier makes the task much harder in comparing information from people using different languages. Third, the extra messages can sometimes be

troublesome on the translation of information. Because in respect of storage, it is relative easy to convert the degree of preference or evaluation into numbers, but the extra

explanation is hard to deal with if choosing to keep them.

Because of those problems raised by choosing texts as research objects, photos as a media conveying information are confirmed for this research. Although there are problems such as the uncertainty in translating the purpose of a picture, they have their

own strengths that overweigh the other option. On the one hand, photos have no geographic limits, which means people from any western countries can read a picture from the Middle East without any language knowledge. On the other hand, photos can be relatively more objective in recording personal perception than language. Given these two major reasons, photos are selected as the research objects and also this selection leads to the final project name of "C-IMAGE" instead of "C-TEXT".

(34)

3.1.2 Machine-taken Photos vs. Human-taken Photos

When it comes to the decision of using picture for the study about city cognitive mapping, there are two different image sources. One is from Google Street View and the other is from social media platform such as Flickr and Panoramio. The difference between these two sources lies in the photo takers. For Google Street View, cameras on Google Street View cars automatically capture all the photos. However, photos from other source are all taken and shared by humans. Below is a brief comparison between these two sources.

Figure 3-1 two different online photo sources: Google Street View standing for machine-taken photos (left), and Panoramio and Flickr standing for human-taken photos (right) 10

Google Street View is a technology combined with services of Google Maps and Google Earth. Launched on May 25, 2007, it provides panoramic views from positions along

streets all over the world. Through scripting those image can be requested and

downloaded from clients after registering and achieving Google developer API" keys. Many of previous similar cases such as the "City Pulse" project and the "What makes Paris look like Paris" project select this dataset as their starting point, because usually the Google Street View can cover nearly most of the views of the public spaces in a city and the evenly distributed images. Since each photo has its geolocation information,

researchers can know exactly where a specific landmark is located in a city even without

'0 The left figure:

http://4.bp.blogspot.com/-KzOUEJkbZiM/T9owOJTT48I/AAAAAAAAA8A/S5RhH8zoCrQ/sl600/untitled-600-2 1-http://4.bp.blogspot.com/-KzOUEJkbZiM/T9owOJTT48I/AAAAAAAAA8A/S5RhH8zoCrQ/sl600/untitled-600-2 http://4.bp.blogspot.com/-KzOUEJkbZiM/T9owOJTT48I/AAAAAAAAA8A/S5RhH8zoCrQ/sl600/untitled-600-2-http://4.bp.blogspot.com/-KzOUEJkbZiM/T9owOJTT48I/AAAAAAAAA8A/S5RhH8zoCrQ/sl600/untitled-600-2.jpg, and the right figure: http://www.gettyimages.cn/http://4.bp.blogspot.com/-KzOUEJkbZiM/T9owOJTT48I/AAAAAAAAA8A/S5RhH8zoCrQ/sl600/untitled-600-28576http://4.bp.blogspot.com/-KzOUEJkbZiM/T9owOJTT48I/AAAAAAAAA8A/S5RhH8zoCrQ/sl600/untitled-600-26

" API, Application Programming Interface, which specifies how some software components should interact with each other

(Wikipedia, http://en.wikipedia.org/wiki/API)

(35)

having been there before. In addition, most of the street views in the cities from western countries and well-developed areas have been taken by Google and are freely accessible to everyone.

Unfortunately, as the current coverage of Google Street View is shown in Figure 3-2, large areas in developing countries such as China and India are not covered. More importantly, taking Google Street View is somehow contradicting the underlying principle of "choose subjective information", because those images from Google Street View are photos taken by machines instead of human beings. Although this technology brings back a large quantity of elegant images even from unpopulated places, the photos from this source are not representing anyone's motivation and the only premise of taking a Google Street View photo is the existence of a street.

Figure 3-2 Google Street View Coverage2

However, collecting photos from social media platform is different, because the data from this source is purely subjective information, which exactly reflects individual preferences. Many such platforms as Flickr and Panoramio have already built up a huge

2 The reference link: https://support.google.com/maps/answer/68384?hl=en

(36)

database of photo collections covering most areas in the world. At least every city in the

C-IMAGE has a substantially large number of geo-tagged photos.

3.1.3 Panoramio vs. Flickr

Currently the largest two photo providers are Panoramio and Flickr. The former is a geolocation-oriented photo sharing website, which is also highlighted while comparing the two interfaces in Figure 3-3 and Figure 3-4. Due to the cooperation with Google, accepted photos uploaded to the site can be accessed as a layer in Google Earth and Google Maps, and therefore, it becomes the biggest photo venders for Google. As the other photo provider, Flickr is the image hosting and video hosting website created in 2004 and acquired by Yahoo in 2005. The photos from Flickr are more diverse and personal, and the service is widely used by photo researchers and by bloggers to host images. U UP 0..~ 7, -7 4; 09 J, a 4 : ti 4s

Figure 3-:h

e itraeofPnrr

U

-L.

Author: Liu Liu, Advisor: Brent D. Ryan

C-IMAGE : City Cognitive Mapping Through Geo-Tagged Photos

.V 1009

(37)

Figure 3-4 the web interface of Flickr

Both channels of photos have advantages and disadvantages. For example, since Flickr has its own powerful platform which is independent any other web services, it is more popular and thus having larger number of photos than Panoramio. However, the quality and the content of the photos from Flickr are of great diversity, which is hard for later data processing. As for Panoramio, especially after being purchased by Google in 2007, the goal of this website becomes providing Google Earth users with more visionary information about a given area via photos taken by others at that place. Accounting for this purpose, the total number of photos from Panoramio is smaller, most of the contents of the photos are about landscape, or in other words, are more similar to city perception from humans eyes. And photos from Panoramio tend to be typical as a reflection of its location and accurate in its x-y coordination.

Compared to Flickr, downloading data from Panoramio consumes less time and encounters fewer'3_{. Therefore the dataset picked up by most parts of the C-IMAGE} project come from Panoramio. However, given the fact that Flickr has a much larger

" Based on the practice in conducting this research via Liu's scripts, the time for completed downloading a medium size city such as

Toronto is about 1 hour while that time of Flickr may exceed 5 hours.

(38)

database, photos from Flickr are also included and analyzed in some applications such as section 4.1.

3.2 Definition of City Cognitive Map

Because "cognitive mapping" is a phrase borrowed from the psychological field and there are different applications in urban study using the approach of cognitive mapping, it is necessary to define the meaning of the "city cognitive map" specifically within the context of this thesis before further discussion.

Generally, three characters of this term are highlighted throughout the research:

First, the geographic research scope of city cognitive map is refined to the physical boundaries of cities and city is also the basic unit of how image datasets are all collected and stored. Second, conforming to the essence of cognitive map, the information

prepared for the maps should stay subjective, and the process of cognitive map is a translation of subjective information to a readable format. Third, collecting the

knowledge for city cognitive map is a "bottom-up" choice, which means the data source should come from the public so that the results are reflection of the perception from the public.

3.3 Acquisition of Data

As discussed above, Panoramio is more ideal for this research, while Flickr obtains a much larger database covering a wider time span. In the most parts of this research, photos from Panoramio are the major objects, but some image data like the peninsular Boston case in section 4.1 has also been captured from Flickr to increase the sample size and double check for some conclusions.

(39)

Figure 3-5 is the complete map of San Francisco after plotting all the downloaded Panoramio photos, which is appropriate for explaining the downloading process. For the purpose of any further study by other interested institutes or professionals, this thesis provides guidance about how to achieve the huge photo gallery from Panoramio and Flickr in the following section 3.3.1, 3.3.2, and 3.3.3. For the purpose of quick scan about the key findings of this paper, readers may just skip this part and jump to section 3.4 on page 51 and it will not be troublesome in understanding the following content without reading this part.

Figure 3-5 the complete map after downloading the photos in San Francisco from Panoramio, the 500*500 meter cell means the basic unit for downloading, each black dot stands for a photo, and the color of the cell from light yellow to dark yellow indicates the density of photos within each cell

(40)

All the scripts in Python mentioned below can be found from Liu Liu's website, and the

link towards them are provided in the Appendix I14

3.3.1 Preparations for Downloading

Before starting the download, the preparation of identifying the boundary of cities location is needed. To be simplified, the municipal boundaries of cities are selected for this research15_{. But according to the different definition of the word of "city" from}

different institutional background, the area and scale of them varies greatly. There are several more steps before starting to download the data from Panoramio and Flickr. The steps in this section are all finished in ArcGIS, and to process all the 26 cities in batch, this research combines all the following steps by the model builder of the software.

First a net composed by 500 * 500 meter cells are created to mask the city via the "Grid Index Features" tool from the Cartography toolbox of ArcGIS. As it is shown in Figure

3-6, the data acquisition of each city treats these cells as the basic units for downloading.

ArcGIS Resources - I

ArcGIS Help 10.1

Grid Index Feetur artogrpy)

Figure 3-6 dividing the research city into 500*500 meter cells (left) via the "Grid Index Features" tool from the Cartography toolbox in ArcGIS (right)

Second, for each unit the vertices are extracted through the "Feature Vertices To Points" from the Data Management toolbox in ArcGIS. Figure 3-7 shows the result after running the tool.

" Please do not take it for any commercial usage, thanks!

(41)

C-IMAGE City Cognitive Mapping Through Geo-Tagged Photos Author: Liu Liu, Advisor: Brent D. Ryan

ArcGIS Help 10.1

G F- %,m fte oi

F-re 10 -e

Feature Vertices To Points (Data Management)

~777~~ ~: TV

"I

Figure 3-7 converting the vertices of the cell into points (left) via the "Feature Vertices To Points" tool from the Data Management toolbox in ArcGIS (right)

Third, the longitude and latitude of all the vertices are calculated through the "Add XY Coordinates" from the Data Management toolbox from ArcGIS, and the attribute table including these coordinates information is exported to an external file in dbf format. Figure 3-8 shows the table ready for export in ArcGIS.

C-IMAGE : city cognitive mapping through geo-tagged photos

JUN 19 2014

Signature redacted

Signature redacted

Signature redacted

ABSTRACT

KEY WORDS

ACKNOWLEDGEMENTS

TABLE OF CONTENTS

Chapter

1

1.1 Research Questions and Objectives

1.3 Work Distribution

Chapter 2

*

A

-r

2NZ

2.2 Urban Computing Stream

r

2.3 Urban computing in Cognition (mixed stream)

B

A

Which place looks safer?

D

E

F

3

Chapter 3

Figure 3-:h

e itraeofPnrr

U

-L.

3.2 Definition of City Cognitive Map

3.3 Acquisition of Data