quantdata.utils.pd#

Panda Utilities#

quantdata.utils.pd.create_empty_df(schema)[source]#

Creates an empty pandas.DataFrame from a schema.

A schema is a dictionary of the form:

{
    "col_a": np.dtype("float32"),
    "col_b": np.dtype("int64"),
}
quantdata.utils.pd.verify_schema(df, schema, header=None, *, allow_extra_columns=False)[source]#

Validate a pandas.DataFrame against a schema of the form:

{
    "col_a": np.dtype("float32"),
    "col_b": np.dtype("int64"),
}

Checks: - all schema columns exist - no extra columns, unless allow_extra_columns=True - exact dtype match

Returns df unchanged, so it can be used inline. Raises ValueError if validation fails. The error message will be preeded by header which can either be a string or a callable returning a string (to avoid expensive message construction if validation passes).

Parameters:
  • df (pandas.DataFrame) – Data frame to validate.

  • schema (Mapping[str, Any]) – Mapping of column name to expected dtype.

  • header (str|callable[[],str]| None, optional) – Header for error messages. Can be a string or a callable returning a string. Using a callable delays string composition to avoid unnecessary overhead when validation passes.

  • allow_extra_columns (bool, default False) – Whether to allow columns in the DataFrame that are not in the schema.