Pandas DataFrame Dropna | Data Cleaning Tutorial

Introduction

In this lab, we will learn about the pandas dropna() method for DataFrame. This method is used to remove missing values from a DataFrame by dropping rows or columns that contain null or NaN values.

VM Tips

After the VM startup is done, click the top left corner to switch to the Notebook tab to access Jupyter Notebook for practice.

Sometimes, you may need to wait a few seconds for Jupyter Notebook to finish loading. The validation of operations cannot be automated because of limitations in Jupyter Notebook.

If you face issues during learning, feel free to ask Labby. Provide feedback after the session, and we will promptly resolve the problem for you.

Skills Graph

%%%%{init: {'theme':'neutral'}}%%%% flowchart RL python(("`Python`")) -.-> python/FileHandlingGroup(["`File Handling`"]) pandas(("`Pandas`")) -.-> pandas/DataSelectionGroup(["`Data Selection`"]) pandas(("`Pandas`")) -.-> pandas/DataCleaningGroup(["`Data Cleaning`"]) python(("`Python`")) -.-> python/DataStructuresGroup(["`Data Structures`"]) python(("`Python`")) -.-> python/ModulesandPackagesGroup(["`Modules and Packages`"]) python(("`Python`")) -.-> python/DataScienceandMachineLearningGroup(["`Data Science and Machine Learning`"]) python(("`Python`")) -.-> python/FunctionsGroup(["`Functions`"]) python/FileHandlingGroup -.-> python/with_statement("`Using with Statement`") pandas/DataSelectionGroup -.-> pandas/select_columns("`Select Columns`") pandas/DataCleaningGroup -.-> pandas/handle_missing_values("`Handling Missing Values`") python/DataStructuresGroup -.-> python/lists("`Lists`") python/DataStructuresGroup -.-> python/tuples("`Tuples`") python/ModulesandPackagesGroup -.-> python/importing_modules("`Importing Modules`") python/DataScienceandMachineLearningGroup -.-> python/numerical_computing("`Numerical Computing`") python/DataScienceandMachineLearningGroup -.-> python/data_analysis("`Data Analysis`") python/FunctionsGroup -.-> python/build_in_functions("`Build-in Functions`") subgraph Lab Skills python/with_statement -.-> lab-68614{{"`Pandas DataFrame Dropna Method`"}} pandas/select_columns -.-> lab-68614{{"`Pandas DataFrame Dropna Method`"}} pandas/handle_missing_values -.-> lab-68614{{"`Pandas DataFrame Dropna Method`"}} python/lists -.-> lab-68614{{"`Pandas DataFrame Dropna Method`"}} python/tuples -.-> lab-68614{{"`Pandas DataFrame Dropna Method`"}} python/importing_modules -.-> lab-68614{{"`Pandas DataFrame Dropna Method`"}} python/numerical_computing -.-> lab-68614{{"`Pandas DataFrame Dropna Method`"}} python/data_analysis -.-> lab-68614{{"`Pandas DataFrame Dropna Method`"}} python/build_in_functions -.-> lab-68614{{"`Pandas DataFrame Dropna Method`"}} end

Import the pandas library

Before we start, we need to import the pandas library. We will use the pd alias for this library.

import pandas as pd

Create a DataFrame

Let's create a DataFrame that contains some missing values.

df = pd.DataFrame([
    ['Abhishek', 100, 'Science', None],
    ['Anurag', 101, 'Science', 85],
    ['Chetan', 103, 'Maths', None]
], columns=['Name', 'Roll No', 'Subject', 'Marks'])

Use the dropna() method to drop rows with missing values

We can use the dropna() method to drop rows that contain missing values. By default, it removes any row that has at least one null or NaN value.

df_dropped = df.dropna()

Display the result

Let's print the original DataFrame and the resulting DataFrame after dropping rows with missing values.

print("Original DataFrame:")
print(df)

print("DataFrame after dropping rows with missing values:")
print(df_dropped)

Summary

In this lab, we learned about the pandas dropna() method for DataFrame, which is used to drop rows or columns with missing values. By dropping these missing values, we can clean our DataFrame and make it suitable for further analysis. Remember that this method modifies the DataFrame and returns a new DataFrame by default, but we can also choose to modify the DataFrame in-place by setting the inplace parameter to True.

Pandas DataFrame Dropna Method