Introduction
This lab will guide you on how to use sparse data structures in the pandas library. This is useful in scenarios where we have large volumes of data, most of which are similar (like zero or NaN), hence can be represented more efficiently in memory. We will learn about the SparseArray
, SparseDtype
, sparse accessor, sparse calculation, and interaction with scipy sparse matrices.
VM Tips
After the VM startup is done, click the top left corner to switch to the Notebook tab to access Jupyter Notebook for practice.
Sometimes, you may need to wait a few seconds for Jupyter Notebook to finish loading. The validation of operations cannot be automated because of limitations in Jupyter Notebook.
If you face issues during learning, feel free to ask Labby. Provide feedback after the session, and we will promptly resolve the problem for you.
Skills Graph
%%%%{init: {'theme':'neutral'}}%%%%
flowchart RL
python(("`Python`")) -.-> python/BasicConceptsGroup(["`Basic Concepts`"])
python(("`Python`")) -.-> python/FileHandlingGroup(["`File Handling`"])
pandas(("`Pandas`")) -.-> pandas/DataSelectionGroup(["`Data Selection`"])
pandas(("`Pandas`")) -.-> pandas/DataManipulationGroup(["`Data Manipulation`"])
pandas(("`Pandas`")) -.-> pandas/DataAnalysisGroup(["`Data Analysis`"])
python(("`Python`")) -.-> python/DataStructuresGroup(["`Data Structures`"])
python(("`Python`")) -.-> python/ModulesandPackagesGroup(["`Modules and Packages`"])
python(("`Python`")) -.-> python/PythonStandardLibraryGroup(["`Python Standard Library`"])
python(("`Python`")) -.-> python/DataScienceandMachineLearningGroup(["`Data Science and Machine Learning`"])
python(("`Python`")) -.-> python/FunctionsGroup(["`Functions`"])
python/BasicConceptsGroup -.-> python/comments("`Comments`")
python/FileHandlingGroup -.-> python/with_statement("`Using with Statement`")
pandas/DataSelectionGroup -.-> pandas/select_columns("`Select Columns`")
pandas/DataSelectionGroup -.-> pandas/select_rows("`Select Rows`")
pandas/DataSelectionGroup -.-> pandas/conditional_selection("`Conditional Selection`")
pandas/DataSelectionGroup -.-> pandas/slicing("`Slicing`")
pandas/DataManipulationGroup -.-> pandas/change_data_types("`Changing Data Types`")
pandas/DataAnalysisGroup -.-> pandas/basic_statistics("`Basic Statistics`")
python/BasicConceptsGroup -.-> python/variables_data_types("`Variables and Data Types`")
python/BasicConceptsGroup -.-> python/numeric_types("`Numeric Types`")
python/DataStructuresGroup -.-> python/lists("`Lists`")
python/DataStructuresGroup -.-> python/tuples("`Tuples`")
python/DataStructuresGroup -.-> python/dictionaries("`Dictionaries`")
python/ModulesandPackagesGroup -.-> python/importing_modules("`Importing Modules`")
python/ModulesandPackagesGroup -.-> python/using_packages("`Using Packages`")
python/ModulesandPackagesGroup -.-> python/standard_libraries("`Common Standard Libraries`")
python/PythonStandardLibraryGroup -.-> python/math_random("`Math and Random`")
python/DataScienceandMachineLearningGroup -.-> python/numerical_computing("`Numerical Computing`")
python/DataScienceandMachineLearningGroup -.-> python/data_analysis("`Data Analysis`")
python/FunctionsGroup -.-> python/build_in_functions("`Build-in Functions`")
subgraph Lab Skills
python/comments -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/with_statement -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
pandas/select_columns -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
pandas/select_rows -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
pandas/conditional_selection -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
pandas/slicing -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
pandas/change_data_types -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
pandas/basic_statistics -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/variables_data_types -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/numeric_types -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/lists -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/tuples -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/dictionaries -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/importing_modules -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/using_packages -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/standard_libraries -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/math_random -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/numerical_computing -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/data_analysis -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
python/build_in_functions -.-> lab-65454{{"`Using Sparse Structures in Pandas`"}}
end