Python JSON and YAML

Introduction

In modern software development, data is often stored and transported in standardized formats. JSON (JavaScript Object Notation) and YAML (YAML Ain't Markup Language) are two of the most popular formats for everything from web API responses to application configuration files.

In this lab, you will learn the essential skills for working with these formats in Python. You will start by using Python's built-in json module to read and write JSON data. Then, you will explore YAML as a more human-readable alternative and learn how to parse it using the ruamel.yaml library. Finally, you will discover the anyconfig library, a powerful tool that simplifies configuration management by allowing you to load data from various file formats without worrying about the underlying implementation.

Working with JSON

In this step, you will learn to serialize and deserialize JSON data. JSON stands for JavaScript Object Notation and is a lightweight format for storing and transporting data. It's commonly used when data is sent from a server to a web page.

Note: If you are not familiar with using WebIDE (VS Code like), please learn it first at this lab.

First, let's write a Python dictionary to a JSON file. This process is called serialization.

In the file explorer on the left, open the json_handler.py file and add the following code. This script will create a new file named output.json.

## Write JSON file: json.dump() writes Python object as JSON
import json

content = {"name": "Joe", "age": 20}
with open("output.json", "w") as f:  ## Open file in write mode
    json.dump(content, f, indent=2)  ## Write JSON with 2-space indentation

Let's break down the code:

• import json: Imports Python's built-in JSON module.
• content = {"name": "Joe", "age": 20}: Defines a Python dictionary that we want to save.
• with open("output.json", "w") as f:: Opens a file named output.json in write mode ("w").
• json.dump(content, f, indent=2): Writes the content dictionary to the file f in JSON format. The indent=2 argument formats the JSON with a 2-space indentation, making it more readable.

Now, run the script in the terminal to create the file:

python3 json_handler.py

You won't see any output, but a new file output.json has been created. You can view its contents with the cat command:

cat output.json

You should see the formatted JSON content:

{
  "name": "Joe",
  "age": 20
}

Next, let's read a JSON file into a Python object. This process is called deserialization. We will read the data.json file that was prepared for you.

Modify the json_handler.py file with the following code:

## Read JSON file: json.load() parses JSON from file object
import json

with open("data.json", "r") as f:  ## Open file in read mode
    content = json.load(f)  ## Parse JSON and return Python dict/list
    print(content)

Here's what the new code does:

• with open("data.json", "r") as f:: Opens the existing data.json file in read mode ("r").
• content = json.load(f): Parses the JSON data from the file and converts it into a Python dictionary.
• print(content): Prints the resulting dictionary to the console.

Run the script again:

python3 json_handler.py

This time, you will see the Python dictionary printed to the terminal:

{'username': 'labex', 'id': 12345}

Handling YAML

In this step, you will explore YAML, a data serialization format designed for human readability. Compared to JSON, YAML is often easier to maintain by hand and allows for comments, making it a popular choice for configuration files.

Python does not have a built-in YAML parser, so you need to install a third-party library. Two main libraries are available: PyYaml and ruamel.yaml. The ruamel.yaml library is more compliant with the YAML specification and is better at preserving comments and formatting when modifying files. We will use it in this lab.

First, install ruamel.yaml using pip:

pip install ruamel.yaml

You will see output indicating a successful installation.

Now, let's use this library to read the config.yaml file. Open the yaml_handler.py file in the editor and add the following code:

## Read YAML file using ruamel.yaml library
from ruamel.yaml import YAML

with open("config.yaml") as f:
    yaml=YAML()  ## Create YAML parser instance
    data = yaml.load(f)  ## Parse YAML and return Python dict/list
    print(data)

Let's review the code:

• from ruamel.yaml import YAML: Imports the necessary YAML class from the library.
• with open("config.yaml") as f:: Opens the config.yaml file for reading.
• yaml=YAML(): Creates an instance of the YAML parser.
• data = yaml.load(f): Parses the YAML content from the file and converts it into a Python object (in this case, a dictionary).
• print(data): Prints the loaded data.

Execute the script from the terminal:

python3 yaml_handler.py

The output will be the Python dictionary representation of the YAML file's content:

{'user': {'name': 'Alex', 'email': 'alex@example.com'}}

Using Anyconfig

In this step, you will discover the anyconfig package, a very handy library that abstracts the underlying configuration file format. It allows you to load a Python dictionary from various formats like JSON, YAML, TOML, and more, using a single, simple function. This is extremely useful when your application needs to support multiple configuration formats.

First, you need to install anyconfig. Since anyconfig needs a backend to parse YAML, and we already installed ruamel.yaml, it will automatically use it.

Install anyconfig using pip:

pip install anyconfig

After the installation is complete, you can use anyconfig to load configuration files. Let's use it to load the same config.yaml file from the previous step.

Open the anyconfig_handler.py file and add the following code:

## anyconfig: load configuration files in various formats (JSON, YAML, TOML, etc.)
import anyconfig

## anyconfig.load() auto-detects the file format based on the extension.
conf = anyconfig.load("/home/labex/project/config.yaml")
print(conf)

This code is much simpler:

• import anyconfig: Imports the library.
• anyconfig.load(...): This single function opens, reads, and parses the file. It automatically detects that config.yaml is a YAML file and uses the appropriate parser. Note that we are using an absolute path here, which is a good practice when the script's location might vary.

Now, run the script:

python3 anyconfig_handler.py

Because this example still loads config.yaml, the output is identical to the previous step, but the code required is more concise and flexible.

{'user': {'name': 'Alex', 'email': 'alex@example.com'}}

With anyconfig, you can also replace "config.yaml" with "data.json" and rerun the script. The code still works without any other changes, but the printed dictionary will reflect the contents of data.json instead of the YAML file.

Summary

In this lab, you learned how to handle two common data serialization formats in Python: JSON and YAML. These skills are fundamental for tasks such as working with web APIs, managing application configurations, and data interchange.

You started with Python's built-in json module, using json.dump() to serialize a Python dictionary into a JSON file and json.load() to deserialize a JSON file back into a Python object.

Next, you explored YAML as a human-readable alternative to JSON. You learned how to install the ruamel.yaml library and use it to parse a YAML file.

Finally, you were introduced to the anyconfig library, a powerful tool that simplifies configuration management. You saw how its anyconfig.load() function can read data from different file formats without requiring format-specific code, making your applications more flexible and easier to maintain.