In this lab, we will learn how to import data using the numpy.genfromtxt function. This function allows us to read tabular data from various sources and convert it into NumPy arrays. We will explore different options for defining the input, splitting the lines into columns, choosing columns, setting the data type, and tweaking the conversion.
Note: You can write code in
03-io-genfromtxt.ipynb. Some printing operations are omitted in the steps, and you can print output as needed.
First, let's import the necessary libraries. We will use the numpy library for creating arrays and the io module from the io library for creating file-like objects.
import numpy as np
from io import StringIOThe numpy.genfromtxt function requires the source of the data as the only mandatory argument. It can be a string, a list of strings, a generator, or an open file-like object with a read method.
data = "1, 2, 3\n4, 5, 6"The delimiter argument is used to define how the lines should be split into columns. By default, numpy.genfromtxt assumes delimiter=None, meaning that the line is split along white spaces (including tabs).
np.genfromtxt(StringIO(data), delimiter=",")The usecols argument is used to select which columns to import. It accepts a single integer or a sequence of integers corresponding to the indices of the columns to import.
np.genfromtxt(StringIO(data), usecols=(0, -1))The dtype argument is used to control how the strings are converted to other types. It can be a single type, a sequence of types, a comma-separated string, a dictionary, a sequence of tuples, an existing numpy.dtype object, or None to determine the type from the data itself.
np.genfromtxt(StringIO(data), dtype=float)The converters argument allows us to define conversion functions to handle more complex conversions. It accepts a dictionary with column indices or column names as keys and conversion functions as values.
convertfunc = lambda x: float(x.strip(b"%"))/100.
np.genfromtxt(StringIO(data), converters={1: convertfunc})The missing_values and filling_values arguments are used to handle missing data. The missing_values argument is used to recognize missing data, and the filling_values argument is used to provide a value for missing entries.
np.genfromtxt(StringIO(data), missing_values="N/A", filling_values=0)The numpy.lib.npyio module provides shortcut functions derived from numpy.genfromtxt. These functions have different default values and return either a standard NumPy array or a masked array.
from numpy.lib.npyio import recfromtxt
recfromtxt(StringIO(data), delimiter=",")In this lab, we learned how to import data using the numpy.genfromtxt function. We explored different options for defining the input, splitting the lines into columns, choosing columns, setting the data type, and tweaking the conversion. We also learned about shortcut functions provided by the numpy.lib.npyio module.
