Pandas Correlation Analysis | Data Science Tutorial

Introduction

In this lab, we will learn how to use the corrwith() method in the Pandas library to compute pairwise correlation between two DataFrames.

VM Tips

After the VM startup is done, click the top left corner to switch to the Notebook tab to access Jupyter Notebook for practice.

Sometimes, you may need to wait a few seconds for Jupyter Notebook to finish loading. The validation of operations cannot be automated because of limitations in Jupyter Notebook.

If you face issues during learning, feel free to ask Labby. Provide feedback after the session, and we will promptly resolve the problem for you.

Skills Graph

%%%%{init: {'theme':'neutral'}}%%%% flowchart RL pandas(("`Pandas`")) -.-> pandas/DataSelectionGroup(["`Data Selection`"]) python(("`Python`")) -.-> python/DataStructuresGroup(["`Data Structures`"]) python(("`Python`")) -.-> python/ModulesandPackagesGroup(["`Modules and Packages`"]) python(("`Python`")) -.-> python/DataScienceandMachineLearningGroup(["`Data Science and Machine Learning`"]) pandas/DataSelectionGroup -.-> pandas/select_columns("`Select Columns`") python/DataStructuresGroup -.-> python/lists("`Lists`") python/DataStructuresGroup -.-> python/tuples("`Tuples`") python/DataStructuresGroup -.-> python/dictionaries("`Dictionaries`") python/ModulesandPackagesGroup -.-> python/importing_modules("`Importing Modules`") python/DataScienceandMachineLearningGroup -.-> python/numerical_computing("`Numerical Computing`") python/DataScienceandMachineLearningGroup -.-> python/data_analysis("`Data Analysis`") subgraph Lab Skills pandas/select_columns -.-> lab-68600{{"`Pandas DataFrame Corrwith Method`"}} python/lists -.-> lab-68600{{"`Pandas DataFrame Corrwith Method`"}} python/tuples -.-> lab-68600{{"`Pandas DataFrame Corrwith Method`"}} python/dictionaries -.-> lab-68600{{"`Pandas DataFrame Corrwith Method`"}} python/importing_modules -.-> lab-68600{{"`Pandas DataFrame Corrwith Method`"}} python/numerical_computing -.-> lab-68600{{"`Pandas DataFrame Corrwith Method`"}} python/data_analysis -.-> lab-68600{{"`Pandas DataFrame Corrwith Method`"}} end

Import the necessary libraries

To use the corrwith() method, we need to import the pandas library.

import pandas as pd

Create the DataFrames

Let's create two DataFrames that we will use for this lab.

chart_1 = {'Name':['Chetan','yashas','yuvraj'],'Age':  [20,25,30],'Height': [155,160,175],'Weight': [55,60,75]}
df1 = pd.DataFrame(chart_1)

chart_2 = {'Name':['Pooja','Sindu','Renuka'],'Age':  [18,25,20],'Height': [145,155,165],'Weight': [45,55,65]}
df2 = pd.DataFrame(chart_2)

Compute the correlation using the `corrwith()` method

We can use the corrwith() method to compute the correlation between the two DataFrames.

df1.corrwith(df2)

Specify the correlation method

By default, the corrwith() method uses the Pearson correlation coefficient. However, we can specify the correlation method by using the method parameter.

df1.corrwith(df2, method='kendall')

Summary

In this lab, we learned how to use the corrwith() method in the Pandas library to compute pairwise correlation between two DataFrames. This method is useful for finding the correlation between different columns in two datasets. By specifying the correlation method, we can compute Pearson, Kendall, or Spearman correlation coefficients. This helps us understand the relationships between variables and make data-driven decisions.

Pandas DataFrame Corrwith Method