Skip to content

lythonic.frame

Library-agnostic tabular data, with one conversion facade per DataFrame library.

Library-agnostic tabular data model.

FrameData is a Pydantic model that stores columnar data in a simple row-oriented format (columns + data). It serializes to/from JSON without any DataFrame library dependency.

Conversions live behind one facade per library, named for the library's conventional import alias. Class access gives the inbound constructors and instance access the outbound conversions:

fd = FrameData.pd.from_frame(df)    # pandas DataFrame in
df = fd.pd.frame()                  # pandas DataFrame out

fd = FrameData.pl.from_frame(df)    # polars DataFrame in
df = fd.pl.frame()                  # polars DataFrame out

fd = FrameData.pa.from_table(tbl)   # pyarrow Table in
tbl = fd.pa.table()                 # pyarrow Table out

Each facade lazily imports its library on first access, so importing this module costs nothing when no optional dependency is installed. Reaching for a library that is not installed raises ImportError naming the package.

>>> fd = FrameData(columns=["a", "b"], data=[[1, 2], [3, 4]])
>>> fd.columns
['a', 'b']
>>> fd.data
[[1, 2], [3, 4]]

FrameData

Bases: BaseModel

Serializable tabular data container. Library-agnostic wire format for passing table data between components.

One facade per DataFrame library: pd, pl, pa. Class access gives the inbound constructors (FrameData.pd.from_frame(df)), instance access the outbound conversions (fd.pd.frame()). Each facade lazily imports its library and raises ImportError if it is not installed.

Source code in src/lythonic/frame.py 
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
class FrameData(BaseModel):
    """
    Serializable tabular data container. Library-agnostic wire format
    for passing table data between components.

    One facade per DataFrame library: `pd`, `pl`, `pa`. Class access gives
    the inbound constructors (`FrameData.pd.from_frame(df)`), instance access
    the outbound conversions (`fd.pd.frame()`). Each facade lazily imports its
    library and raises `ImportError` if it is not installed.
    """

    columns: list[str]
    data: list[list[Any]]

    pd: ClassVar[LibAccess[FrameData, PdIn, PdOut]] = LibAccess(PdIn, PdOut)
    """Pandas facade. Class access gives constructors, instance access conversions."""

    pl: ClassVar[LibAccess[FrameData, PlIn, PlOut]] = LibAccess(PlIn, PlOut)
    """Polars facade. Class access gives constructors, instance access conversions."""

    pa: ClassVar[LibAccess[FrameData, PaIn, PaOut]] = LibAccess(PaIn, PaOut)
    """Pyarrow facade. Class access gives constructors, instance access conversions."""

pd = LibAccess(PdIn, PdOut) class-attribute

Pandas facade. Class access gives constructors, instance access conversions.

pl = LibAccess(PlIn, PlOut) class-attribute

Polars facade. Class access gives constructors, instance access conversions.

pa = LibAccess(PaIn, PaOut) class-attribute

Pyarrow facade. Class access gives constructors, instance access conversions.

PdIn

Class-access pandas facade: constructors that take a pandas DataFrame.

Source code in src/lythonic/frame.py 
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
class PdIn:
    """Class-access pandas facade: constructors that take a pandas DataFrame."""

    _owner: type[FrameData]
    _pandas: ModuleType

    def __init__(self, owner: type[FrameData]) -> None:
        self._owner = owner
        self._pandas = require("pandas")

    def from_frame(self, df: pd.DataFrame) -> FrameData:
        """Build `FrameData` from a pandas DataFrame."""
        if not isinstance(df, self._pandas.DataFrame):
            raise TypeError(f"Expected pandas.DataFrame, got {type(df).__name__}")
        t: dict[str, Any] = df.to_dict(orient="tight", index=False)
        return self._owner(columns=t["columns"], data=t["data"])

from_frame(df)

Build FrameData from a pandas DataFrame.

Source code in src/lythonic/frame.py 
64
65
66
67
68
69
def from_frame(self, df: pd.DataFrame) -> FrameData:
    """Build `FrameData` from a pandas DataFrame."""
    if not isinstance(df, self._pandas.DataFrame):
        raise TypeError(f"Expected pandas.DataFrame, got {type(df).__name__}")
    t: dict[str, Any] = df.to_dict(orient="tight", index=False)
    return self._owner(columns=t["columns"], data=t["data"])

PdOut

Instance-access pandas facade: pandas views of the data.

Source code in src/lythonic/frame.py 
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
class PdOut:
    """Instance-access pandas facade: pandas views of the data."""

    _fd: FrameData
    _pandas: ModuleType

    def __init__(self, owner: FrameData) -> None:
        self._fd = owner
        self._pandas = require("pandas")

    def frame(self) -> pd.DataFrame:
        """The data as a pandas DataFrame."""
        fd = self._fd
        return self._pandas.DataFrame.from_dict(
            {
                **fd.model_dump(mode="json"),
                "index": list(range(len(fd.data))),
                "index_names": [""],
                "column_names": [None],
            },
            orient="tight",
        )

frame()

The data as a pandas DataFrame.

Source code in src/lythonic/frame.py 
82
83
84
85
86
87
88
89
90
91
92
93
def frame(self) -> pd.DataFrame:
    """The data as a pandas DataFrame."""
    fd = self._fd
    return self._pandas.DataFrame.from_dict(
        {
            **fd.model_dump(mode="json"),
            "index": list(range(len(fd.data))),
            "index_names": [""],
            "column_names": [None],
        },
        orient="tight",
    )

PlIn

Class-access polars facade: constructors that take a polars DataFrame.

Source code in src/lythonic/frame.py 
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
class PlIn:
    """Class-access polars facade: constructors that take a polars DataFrame."""

    _owner: type[FrameData]
    _polars: ModuleType

    def __init__(self, owner: type[FrameData]) -> None:
        self._owner = owner
        self._polars = require("polars")

    def from_frame(self, df: pl.DataFrame) -> FrameData:
        """Build `FrameData` from a polars DataFrame."""
        if not isinstance(df, self._polars.DataFrame):
            raise TypeError(f"Expected polars.DataFrame, got {type(df).__name__}")
        return self._owner(columns=df.columns, data=[list(row) for row in df.rows()])

from_frame(df)

Build FrameData from a polars DataFrame.

Source code in src/lythonic/frame.py 
106
107
108
109
110
def from_frame(self, df: pl.DataFrame) -> FrameData:
    """Build `FrameData` from a polars DataFrame."""
    if not isinstance(df, self._polars.DataFrame):
        raise TypeError(f"Expected polars.DataFrame, got {type(df).__name__}")
    return self._owner(columns=df.columns, data=[list(row) for row in df.rows()])

PlOut

Instance-access polars facade: polars views of the data.

Source code in src/lythonic/frame.py 
113
114
115
116
117
118
119
120
121
122
123
124
125
126
class PlOut:
    """Instance-access polars facade: polars views of the data."""

    _fd: FrameData
    _polars: ModuleType

    def __init__(self, owner: FrameData) -> None:
        self._fd = owner
        self._polars = require("polars")

    def frame(self) -> pl.DataFrame:
        """The data as a polars DataFrame."""
        fd = self._fd
        return self._polars.DataFrame(fd.data, schema=fd.columns, orient="row")

frame()

The data as a polars DataFrame.

Source code in src/lythonic/frame.py 
123
124
125
126
def frame(self) -> pl.DataFrame:
    """The data as a polars DataFrame."""
    fd = self._fd
    return self._polars.DataFrame(fd.data, schema=fd.columns, orient="row")

PaIn

Class-access pyarrow facade: constructors that take a pyarrow Table.

Source code in src/lythonic/frame.py 
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
class PaIn:
    """Class-access pyarrow facade: constructors that take a pyarrow Table."""

    _owner: type[FrameData]
    _pyarrow: ModuleType

    def __init__(self, owner: type[FrameData]) -> None:
        self._owner = owner
        self._pyarrow = require("pyarrow")

    def from_table(self, table: pa.Table) -> FrameData:
        """Build `FrameData` from a pyarrow Table."""
        if not isinstance(table, self._pyarrow.Table):
            raise TypeError(f"Expected pyarrow.Table, got {type(table).__name__}")
        col_dict = table.to_pydict()
        columns = table.column_names
        data = [[col_dict[c][i] for c in columns] for i in range(table.num_rows)]
        return self._owner(columns=columns, data=data)

from_table(table)

Build FrameData from a pyarrow Table.

Source code in src/lythonic/frame.py 
139
140
141
142
143
144
145
146
def from_table(self, table: pa.Table) -> FrameData:
    """Build `FrameData` from a pyarrow Table."""
    if not isinstance(table, self._pyarrow.Table):
        raise TypeError(f"Expected pyarrow.Table, got {type(table).__name__}")
    col_dict = table.to_pydict()
    columns = table.column_names
    data = [[col_dict[c][i] for c in columns] for i in range(table.num_rows)]
    return self._owner(columns=columns, data=data)

PaOut

Instance-access pyarrow facade: Arrow views of the data.

Source code in src/lythonic/frame.py 
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
class PaOut:
    """Instance-access pyarrow facade: Arrow views of the data."""

    _fd: FrameData
    _pyarrow: ModuleType

    def __init__(self, owner: FrameData) -> None:
        self._fd = owner
        self._pyarrow = require("pyarrow")

    def table(self) -> pa.Table:
        """The data as a pyarrow Table."""
        fd = self._fd
        col_dict = {c: [row[i] for row in fd.data] for i, c in enumerate(fd.columns)}
        return self._pyarrow.table(col_dict)

table()

The data as a pyarrow Table.

Source code in src/lythonic/frame.py 
159
160
161
162
163
def table(self) -> pa.Table:
    """The data as a pyarrow Table."""
    fd = self._fd
    col_dict = {c: [row[i] for row in fd.data] for i, c in enumerate(fd.columns)}
    return self._pyarrow.table(col_dict)