Mastering Data with Pandas: An Introductory Guide

Introduction

In this lab, we will introduce you to the basics of pandas, a powerful data manipulation library in Python. We will guide you through various tasks such as importing pandas, creating and viewing data, data selection, operations and much more.

VM Tips

After the VM startup is done, click the top left corner to switch to the Notebook tab to access Jupyter Notebook for practice.

Sometimes, you may need to wait a few seconds for Jupyter Notebook to finish loading. The validation of operations cannot be automated because of limitations in Jupyter Notebook.

If you face issues during learning, feel free to ask Labby. Provide feedback after the session, and we will promptly resolve the problem for you.

Skills Graph

%%%%{init: {'theme':'neutral'}}%%%% flowchart RL python(("`Python`")) -.-> python/BasicConceptsGroup(["`Basic Concepts`"]) pandas(("`Pandas`")) -.-> pandas/ReadingDataGroup(["`Reading Data`"]) pandas(("`Pandas`")) -.-> pandas/WritingDataGroup(["`Writing Data`"]) pandas(("`Pandas`")) -.-> pandas/DataSelectionGroup(["`Data Selection`"]) pandas(("`Pandas`")) -.-> pandas/DataManipulationGroup(["`Data Manipulation`"]) pandas(("`Pandas`")) -.-> pandas/DataCleaningGroup(["`Data Cleaning`"]) pandas(("`Pandas`")) -.-> pandas/DataVisualizationGroup(["`Data Visualization`"]) python(("`Python`")) -.-> python/DataStructuresGroup(["`Data Structures`"]) python(("`Python`")) -.-> python/ModulesandPackagesGroup(["`Modules and Packages`"]) python(("`Python`")) -.-> python/PythonStandardLibraryGroup(["`Python Standard Library`"]) python(("`Python`")) -.-> python/DataScienceandMachineLearningGroup(["`Data Science and Machine Learning`"]) python(("`Python`")) -.-> python/FunctionsGroup(["`Functions`"]) python/BasicConceptsGroup -.-> python/comments("`Comments`") pandas/ReadingDataGroup -.-> pandas/read_csv("`Read CSV`") pandas/WritingDataGroup -.-> pandas/write_csv("`Write to CSV`") pandas/DataSelectionGroup -.-> pandas/select_columns("`Select Columns`") pandas/DataSelectionGroup -.-> pandas/select_rows("`Select Rows`") pandas/DataManipulationGroup -.-> pandas/sort_data("`Sorting Data`") pandas/DataCleaningGroup -.-> pandas/handle_missing_values("`Handling Missing Values`") pandas/DataCleaningGroup -.-> pandas/data_mapping("`Data Mapping`") pandas/DataVisualizationGroup -.-> pandas/line_plots("`Line Plots`") python/BasicConceptsGroup -.-> python/variables_data_types("`Variables and Data Types`") python/BasicConceptsGroup -.-> python/booleans("`Booleans`") python/DataStructuresGroup -.-> python/lists("`Lists`") python/DataStructuresGroup -.-> python/tuples("`Tuples`") python/ModulesandPackagesGroup -.-> python/importing_modules("`Importing Modules`") python/ModulesandPackagesGroup -.-> python/standard_libraries("`Common Standard Libraries`") python/PythonStandardLibraryGroup -.-> python/math_random("`Math and Random`") python/PythonStandardLibraryGroup -.-> python/data_collections("`Data Collections`") python/DataScienceandMachineLearningGroup -.-> python/numerical_computing("`Numerical Computing`") python/DataScienceandMachineLearningGroup -.-> python/data_analysis("`Data Analysis`") python/FunctionsGroup -.-> python/build_in_functions("`Build-in Functions`") subgraph Lab Skills python/comments -.-> lab-65440{{"`Introduction to Pandas`"}} pandas/read_csv -.-> lab-65440{{"`Introduction to Pandas`"}} pandas/write_csv -.-> lab-65440{{"`Introduction to Pandas`"}} pandas/select_columns -.-> lab-65440{{"`Introduction to Pandas`"}} pandas/select_rows -.-> lab-65440{{"`Introduction to Pandas`"}} pandas/sort_data -.-> lab-65440{{"`Introduction to Pandas`"}} pandas/handle_missing_values -.-> lab-65440{{"`Introduction to Pandas`"}} pandas/data_mapping -.-> lab-65440{{"`Introduction to Pandas`"}} pandas/line_plots -.-> lab-65440{{"`Introduction to Pandas`"}} python/variables_data_types -.-> lab-65440{{"`Introduction to Pandas`"}} python/booleans -.-> lab-65440{{"`Introduction to Pandas`"}} python/lists -.-> lab-65440{{"`Introduction to Pandas`"}} python/tuples -.-> lab-65440{{"`Introduction to Pandas`"}} python/importing_modules -.-> lab-65440{{"`Introduction to Pandas`"}} python/standard_libraries -.-> lab-65440{{"`Introduction to Pandas`"}} python/math_random -.-> lab-65440{{"`Introduction to Pandas`"}} python/data_collections -.-> lab-65440{{"`Introduction to Pandas`"}} python/numerical_computing -.-> lab-65440{{"`Introduction to Pandas`"}} python/data_analysis -.-> lab-65440{{"`Introduction to Pandas`"}} python/build_in_functions -.-> lab-65440{{"`Introduction to Pandas`"}} end

Importing Pandas and Numpy

First, we need to import pandas and numpy packages. Pandas is a powerful data manipulation library and numpy is used for mathematical operations.

## Importing necessary libraries
import numpy as np
import pandas as pd

Creating Objects

We will create a Series by passing a list of values, and pandas will create a default integer index.

## Creating a pandas series
s = pd.Series([1, 3, 5, np.nan, 6, 8])
s

Creating Dataframes

We can create a DataFrame by passing a numpy array, with a datetime index and labeled columns.

## Creating a pandas dataframe
dates = pd.date_range("20130101", periods=6)
df = pd.DataFrame(np.random.randn(6, 4), index=dates, columns=list("ABCD"))
df

Viewing Data

We can view the top and bottom rows of the dataframe using head() and tail() methods respectively.

## Viewing top rows
df.head()

## Viewing bottom rows
df.tail(3)

Data Selection

We can select data using labels or by position.

## Selecting a single column
df["A"]

## Selecting via position
df.iloc[3]

Data Operations

We can perform operations on dataframes like sorting, applying functions, etc.

## Sorting by an axis
df.sort_index(axis=1, ascending=False)

## Applying a function to the data
df.apply(np.cumsum)

Handling Missing Data

Pandas provides methods to handle missing data in the dataframe.

## Filling missing data
df.fillna(value=5)

## Getting the boolean mask where values are nan
pd.isna(df)

Plotting Data

Pandas uses matplotlib for plotting data.

## Plotting data
df.plot()

Saving and Loading Data

Pandas provides methods to save and load data in various formats like csv, excel, hdf5, etc.

## Saving data to a csv file
df.to_csv("foo.csv")

## Loading data from a csv file
pd.read_csv("foo.csv")

Summary

In this lab, we covered the basics of pandas, including how to create and view data, how to select and manipulate data, and how to save and load data. We also learned how to handle missing data and how to plot data. This should provide a solid foundation for further exploration of pandas for data analysis.