Fundamentals of Data Processing

Data processing is a critical component of working with geospatial data, as it enables users to transform raw data into meaningful information that can be used for a wide range of applications.

In order to understand geospatial data, it is important to understand the basics of data processing. In this section, we will cover the fundamentals of data processing, including data collection, organization and analysis, and provide some examples of how data processing can be used in mapping and geospatial analysis.

Data processing is the process of collecting, organizing/transformation, and analyzing data. The first step in data processing is collecting data. There are many different ways to collect data, depending on the type of data you are collecting and the resources available to you. Some common methods of data collection include:

• Surveys: Surveys are a common method of data collection, particularly for social and environmental data. They can be conducted in person, over the phone, or online, and can collect a wide range of data, from demographic information to opinions and attitudes.

• Remote sensing: Remote sensing refers to the collection of data from a distance, using tools like satellites or drones. It can be used to collect data on the Earth's surface, such as land cover, temperature, and vegetation.

• Field data collection: Field data collection involves collecting data in the field, often using specialized equipment like GPS devices or environmental sensors. It can be used to collect data on physical features like elevation, soil characteristics, and water quality.

• Existing data sources: Existing data sources, such as government databases or scientific studies, can also be used as sources of data. This can be particularly useful when collecting data on a large scale, such as when analyzing global trends.

Once data has been collected, it needs to be organized in a way that makes it useful for analysis. This involves several steps, including:

• Cleaning: Cleaning involves removing any errors or inconsistencies in the data. This can include removing duplicate entries, fixing spelling errors, and checking for missing values.

• Formatting: Formatting involves organizing the data in a way that is consistent and easy to read. This can include converting data to a standard format, such as CSV or Excel, and creating labels or categories for the data.

• Storing: Storing involves saving the data in a secure and accessible location. This can include saving data to a local computer, a cloud storage platform like Google Drive or Dropbox, or a specialized database.

In education, understanding how to process and analyze data can help both teachers and students make informed decisions, draw conclusions from complex information, and gain insights into various subjects.

One common type of data in the education process is tabular data. Tabular data is organized into rows and columns, much like a spreadsheet. Each row represents an individual record, and each column represents an attribute, or characteristic, of that record. For example, a table of student information might have columns for student ID, name, grade level, and test scores.

Understanding attributes is important for organizing and analyzing data. Attributes can be numerical (e.g., age or test score), textual (e.g., name), or categorical (e.g., gender or class subject). By properly categorizing and analyzing attributes, teachers can identify trends, make comparisons, and evaluate progress.

There are different ways to process and analyze tabular data, such as:

• Sorting: Organizing data in ascending or descending order based on a specific attribute, like sorting students by their test scores.

• Filtering: Selecting records that meet certain criteria, such as displaying only students who scored above a specific threshold on a test.

• Aggregating: Combining records to generate summary statistics, like calculating the average test score for a class.

• Pivot tables: Rearranging and summarizing data in a more compact and organized format, which can help identify patterns or relationships between attributes.

By understanding the fundamentals of data processing and incorporating them into the classroom, teachers can better analyze student performance, identify areas for improvement, and tailor their instruction to meet the diverse needs of their students.

To illustrate the concepts discussed in this chapter, let's consider a table representing the population and area of different cities:

City	Population	Area (sq km)
New York	8,398,748	783.8
Los Angeles	3,990,456	1,302
Chicago	2,705,994	606
Houston	2,325,502	1,651
Phoenix	1,660,272	1,340
Philadelphia	1,584,138	347

Now, let's apply the sorting and filtering techniques to this table:

• Sorting: To sort the table by Population in descending order, we would rearrange the rows as follows:

City	Population	Area (sq km)
New York	8,398,748	783.8
Los Angeles	3,990,456	1,302
Chicago	2,705,994	606
Houston	2,325,502	1,651
Phoenix	1,660,272	1,340
Philadelphia	1,584,138	347

• Filtering: To filter the table to display only cities with an Area greater than 1,000 sq km, we would keep only the following rows:

City	Population	Area (sq km)
Los Angeles	3,990,456	1,302
Houston	2,325,502	1,651
Phoenix	1,660,272	1,340

By applying these data processing techniques, we can easily organize and analyze the information in the table to answer various questions or identify patterns. For instance, sorting by population can help us understand which cities have the largest populations, while filtering by area can help us focus on cities with specific characteristics, such as a larger landmass.

To present the population table on a map, it should include geographic coordinates (latitude and longitude) or other location-based attributes (e.g., addresses, postal codes, or administrative boundaries) that can be geocoded or linked to spatial data. We can modify the previous city population table by adding latitude and longitude columns to represent the city locations:

City	Population	Area (sq km)	Latitude	Longitude
New York	8,398,748	783.8	40.7128	-74.0060
Los Angeles	3,990,456	1,302	34.0522	-118.2437
Chicago	2,705,994	606	41.8781	-87.6298
Houston	2,325,502	1,651	29.7604	-95.3698
Phoenix	1,660,272	1,340	33.4484	-112.0740
Philadelphia	1,584,138	347	39.9526	-75.1652

Tabular data with geographic attributes will thus become the spatial data, which can then be visualized on a map (Fig. 1).

Fig. 1: Spatial dataset on a map

Data processing is essential for creating effective maps and understanding geospatial data. Here are some examples of how data processing can be used in mapping and geospatial analysis:

• Data visualization: Data processing can be used to create maps that visualize the effects of climate change on a specific region. For example, a map could show the extent and vulnerability of permafrost (Fig. 2). Permafrost is thawing at an alarming rate, releasing greenhouse gasses into the atmosphere and creating hazards such as collapsing infrastructure and increased wildfire risk. Mapping can help policymakers and scientists better understand the risks and plan for adaptation strategies.

Fig. 2: Terrestrial and Submarine Permafrost in the Northern Hemisphere (Source: Earth.Org. 2020b)

• Spatial analysis: Spatial analysis can be used to identify areas that are vulnerable to natural disasters or humanitarian crises. By analyzing population density, topography, and other factors, a map could identify areas that are at high risk of flooding, landslides, or other hazards. For example, a map could show the predicted impact of sea level rise on coastal communities in the next 80 years (Fig. 3). This can help policymakers and residents understand the potential risks and plan for adaptation measures.

Fig. 3: Sea Level Rise Projection Map – Hamburg. Sea level rise projections by 2100 for two scenarios with the amount of rise in meters indicated (mild = 2m; extreme = 3m). Percentage and total population displacement indicated bottom right. (Source: Earth.Org. 2020)

• Modeling: Data processing can be used to create models that simulate the impact of environmental challenges on ecosystems. For example, a model could simulate the effects of climate change on coral reefs in a specific region (Fig. 4). This can help researchers and policymakers understand the potential impact of environmental changes and develop strategies to protect and preserve vulnerable ecosystems.

Fig. 4: Exploring Threats to Coral Reefs. The top map (A) shows reefs classified by present integrated threats from local activities. In the middle (B) and bottom (C) maps, reefs are classified by integrated local threats and projections of thermal stress and ocean acidification for 2030 and 2050, respectively. (Source: Fraczek, W., Reytar, K. 2012)

For high school teachers and students, it's important to emphasize the practical applications of data processing in mapping and geospatial analysis. For example, data processing can be also used to create maps of a school campus or local community, analyze patterns in traffic flow or pedestrian activity, or track the spread of a disease. It's also important to highlight the various tools and platforms available for data processing, and to encourage students to experiment with these tools in order to develop their data analysis and visualization skills.