Skip to content

Schemas API

AdjacencyListConf dataclass

Config for adjacency list structure.

Attributes:

Name Type Description
track_counts bool

If a counts column should be made.
counts_dtype type[T_counts] | None

Dtype of the element count column. Defaults to np.uint8
Source code in src/packed_data_structures/schemas/foreign_key_col.py 
21
22
23
24
25
26
27
28
29
30
31
@dataclass
class AdjacencyListConf[T_counts: np.generic]:
    """Config for adjacency list structure.

    Attributes:
        track_counts: If a counts column should be made.
        counts_dtype: Dtype of the element count column. Defaults to np.uint8
    """

    track_counts: bool = False
    counts_dtype: type[T_counts] | None = None

AsciiStringColSchema dataclass

Bases: ColSchemaLike[bytes_]

Schema for a fixed-length ASCII byte-string column.

Attributes:

Name Type Description
name str

The string identifier for the column.
max_length int

The maximum allowed length for the string (in bytes).
default bytes | str

The default byte-string value.
Source code in src/packed_data_structures/schemas/ascii_string_col.py 
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
@dataclass(eq=False)
class AsciiStringColSchema(ColSchemaLike[np.bytes_]):
    """Schema for a fixed-length ASCII byte-string column.

    Attributes:
        name: The string identifier for the column.
        max_length: The maximum allowed length for the string (in bytes).
        default: The default byte-string value.
    """

    name: str
    max_length: int
    default: bytes | str = b""
    dtype: np.dtype[np.bytes_] = field(init=False)

    def __post_init__(self):
        self.dtype = cast(np.dtype[np.bytes_], np.dtype((np.bytes_, self.max_length)))

    def init_array(self) -> PackedAsciiStringArray:
        return PackedAsciiStringArray(
            self.parent_table.pre_allocate,
            self.dtype,
            self.default,
        )

    def __hash__(self) -> int:
        return hash(id(self))

ColSchemaLike dataclass

Bases: ABC

Base class for all column schemas.

Column schemas are treated as object-identity singletons (id(self)) when used as dictionary keys or accessors. They define a single column within a TableSchema.

Attributes:

Name Type Description
name str

The string identifier for the column.
parent_table TableSchema

The TableSchema instance this column belongs to.
Source code in src/packed_data_structures/schemas/col_schema_like.py 
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
@dataclass(eq=False)
class ColSchemaLike[T: np.generic](ABC):
    """Base class for all column schemas.

    Column schemas are treated as object-identity singletons (`id(self)`)
    when used as dictionary keys or accessors. They define a single column
    within a TableSchema.

    Attributes:
        name: The string identifier for the column.
        parent_table: The TableSchema instance this column belongs to.
    """

    name: str
    parent_table: TableSchema = field(init=False)

    def set_parent(self, parent: TableSchema):
        """Bind this column to a parent TableSchema.

        Args:
            parent: The TableSchema that will own this column.
        """
        self.parent_table = parent

    @abstractmethod
    def init_array(self) -> PackedArray[T]: ...

    def __hash__(self) -> int:
        return hash(id(self))

    def __eq__(self, other: object) -> bool:
        return self is other

set_parent 

set_parent(parent: TableSchema)

Bind this column to a parent TableSchema.

Parameters:

Name Type Description Default
parent TableSchema

The TableSchema that will own this column.

 required
Source code in src/packed_data_structures/schemas/col_schema_like.py 
30
31
32
33
34
35
36
def set_parent(self, parent: TableSchema):
    """Bind this column to a parent TableSchema.

    Args:
        parent: The TableSchema that will own this column.
    """
    self.parent_table = parent

DataColSchema dataclass

Bases: ColSchemaLike[T]

Schema for a standard data column containing raw values.

Defines the data type, default values, and shape of elements within the column. Maps directly to a PackedArray buffer at runtime.

Attributes:

Name Type Description
name str

The string identifier for the column.
dtype type[T]

The numpy data type of the column's elements.
default Any | tuple[Any, ...]

The default value used to fill empty or newly allocated slots.
shape tuple[*T_shape,]

The shape of individual elements. An empty tuple indicates scalar values.
Source code in src/packed_data_structures/schemas/data_col.py 
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
@dataclass(eq=False)
class DataColSchema[T: np.generic, *T_shape](ColSchemaLike[T]):
    """Schema for a standard data column containing raw values.

    Defines the data type, default values, and shape of elements within
    the column. Maps directly to a PackedArray buffer at runtime.

    Attributes:
        name: The string identifier for the column.
        dtype: The numpy data type of the column's elements.
        default: The default value used to fill empty or newly allocated slots.
        shape: The shape of individual elements. An empty tuple indicates scalar values.
    """

    name: str
    dtype: type[T]
    default: Any | tuple[Any, ...] = 0
    shape: tuple[*T_shape] = field(default_factory=tuple)

    def __post_init__(self) -> None:
        if len(self.shape) != 0 and not isinstance(self.default, tuple):
            self.default = tuple(np.full(self.shape, self.default))

    def init_array(self) -> PackedArray[T]:
        return PackedArray(
            self.parent_table.pre_allocate,
            self.dtype,
            self.default,
            element_shape=self.shape,
        )

    def __hash__(self) -> int:
        return hash(id(self))

FksOnDeleteStyle

Bases: Enum

Various ForeignKeySchema on referenced row delete behaviors.

Source code in src/packed_data_structures/schemas/foreign_key_col.py 
34
35
36
37
38
39
40
41
42
class FksOnDeleteStyle(Enum):
    """Various ForeignKeySchema on referenced row delete behaviors."""

    CASCADE = auto()
    """Cascade the deletion by also deleting the FK row"""
    RESTRICT = auto()
    """Block the deletion and raise an exception"""
    SET_NULL = auto()
    """Set FK field to missing index value"""

CASCADE class-attribute instance-attribute 

CASCADE = auto()

Cascade the deletion by also deleting the FK row

RESTRICT class-attribute instance-attribute 

RESTRICT = auto()

Block the deletion and raise an exception

SET_NULL class-attribute instance-attribute 

SET_NULL = auto()

Set FK field to missing index value

ForeignKeySchema dataclass

Bases: ColSchemaLike[T]

Schema for a column of foreign keys that all point into one specific table.

When registered, this schema dynamically injects internal adjacency list columns: - adj_head (in target table) - adj_next (in source table) - adj_prev (in source table) - adj_count (optional, in target table)

NOTE: The true column names are more verbose

Class Type Parameters:

Name Bound or Constraints Description Default
T integer[Any]

The index dtype of the target table.

 required
T_parent integer[Any]

The index dtype of the parent table. And by extension the adj_next and adj_prev columns.

 required
T_counts integer[Any]

The dtype of the counts column (if applicable).

 required

Attributes:

Name Type Description
name str

The string identifier for the column.
target_table TableSchema

The TableSchema of the table the keys to.
on_delete FksOnDeleteStyle

The policy to apply when something tries to delte a row referenced by this column.
adjacency_conf AdjacencyListConf[T_counts]

Configuration for the injected adjacency list columns.
adj_next AdjNextIdxColSchema[T_parent]

The injected next-pointer column in the parent table.
adj_prev AdjPrevIdxColSchema[T_parent]

The injected previous-pointer column in the parent table.
adj_head AdjHeadIdxColSchema[T]

The injected head-pointer column in the target table.
adj_count AdjCountColSchema[T_counts]

The injected count column in the target table (if enabled).
Source code in src/packed_data_structures/schemas/foreign_key_col.py 
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
@dataclass(eq=False)
class ForeignKeySchema[
    T: np.integer[Any],
    T_parent: np.integer[Any],
    T_counts: np.integer[Any],
](ColSchemaLike[T]):
    """Schema for a column of foreign keys that all point into one specific table.

    When registered, this schema dynamically injects internal adjacency list columns:
    - `adj_head` (in target table)
    - `adj_next` (in source table)
    - `adj_prev` (in source table)
    - `adj_count` (optional, in target table)

    NOTE: The true column names are more verbose

    Type Parameters:
        T: The index dtype of the target table.
        T_parent: The index dtype of the parent table. And by extension the `adj_next` and `adj_prev` columns.
        T_counts: The dtype of the counts column (if applicable).

    Attributes:
        name: The string identifier for the column.
        target_table: The TableSchema of the table the keys to.
        on_delete: The policy to apply when something tries to delte a row referenced by this column.
        adjacency_conf: Configuration for the injected adjacency list columns.
        adj_next: The injected next-pointer column in the parent table.
        adj_prev: The injected previous-pointer column in the parent table.
        adj_head: The injected head-pointer column in the target table.
        adj_count: The injected count column in the target table (if enabled).
    """

    name: str
    target_table: TableSchema
    on_delete: FksOnDeleteStyle = FksOnDeleteStyle.CASCADE
    adjacency_conf: AdjacencyListConf[T_counts] = field(
        default_factory=AdjacencyListConf
    )

    adj_next: AdjNextIdxColSchema[T_parent] = field(init=False)
    adj_prev: AdjPrevIdxColSchema[T_parent] = field(init=False)
    adj_head: AdjHeadIdxColSchema[T] = field(init=False)
    adj_count: AdjCountColSchema[T_counts] = field(init=False)

    def __post_init__(self):
        self.target_table.subscribe(self)

    def set_parent(self, parent: TableSchema[T_parent]):
        """Bind this foreign key column to a parent table and inject adjacency columns.

        This actively mutates both the parent and target table schemas by
        registering the hidden adjacency list management columns.

        Args:
            parent: The TableSchema that will own this foreign key column.
        """
        super().set_parent(parent)

        target = self.target_table
        parent = self.parent_table

        full_name = f"{parent.name}_{self.name}"

        self.adj_head = AdjHeadIdxColSchema(
            f"_adj_head_{full_name}",
            parent.index_spec.dtype,
            parent.index_spec.missing,
        ).set_parent_col(self)
        self.adj_next = AdjNextIdxColSchema(
            f"_adj_next_{full_name}",
            parent.index_spec.dtype,
            parent.index_spec.missing,
        ).set_parent_col(self)
        self.adj_prev = AdjPrevIdxColSchema(
            f"_adj_prev_{full_name}",
            parent.index_spec.dtype,
            parent.index_spec.missing,
        ).set_parent_col(self)

        target.register_new_column(self.adj_head)
        parent.register_new_column(self.adj_next)
        parent.register_new_column(self.adj_prev)

        if self.adjacency_conf.track_counts:
            assert self.adjacency_conf.counts_dtype is not None

            self.adj_count = AdjCountColSchema(
                f"_adj_count_{full_name}",
                self.adjacency_conf.counts_dtype,
                0,
            ).set_parent_col(self)

            target.register_new_column(self.adj_count)

    def __hash__(self) -> int:
        return hash(id(self))

    def init_array(
        self,
    ) -> PackedArray:
        return PackedArray(
            self.parent_table.pre_allocate,
            self.target_table.index_spec.dtype,
            self.target_table.index_spec.missing,
        )

set_parent 

set_parent(parent: TableSchema[T_parent])

Bind this foreign key column to a parent table and inject adjacency columns.

This actively mutates both the parent and target table schemas by registering the hidden adjacency list management columns.

Parameters:

Name Type Description Default
parent TableSchema[T_parent]

The TableSchema that will own this foreign key column.

 required
Source code in src/packed_data_structures/schemas/foreign_key_col.py 
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
def set_parent(self, parent: TableSchema[T_parent]):
    """Bind this foreign key column to a parent table and inject adjacency columns.

    This actively mutates both the parent and target table schemas by
    registering the hidden adjacency list management columns.

    Args:
        parent: The TableSchema that will own this foreign key column.
    """
    super().set_parent(parent)

    target = self.target_table
    parent = self.parent_table

    full_name = f"{parent.name}_{self.name}"

    self.adj_head = AdjHeadIdxColSchema(
        f"_adj_head_{full_name}",
        parent.index_spec.dtype,
        parent.index_spec.missing,
    ).set_parent_col(self)
    self.adj_next = AdjNextIdxColSchema(
        f"_adj_next_{full_name}",
        parent.index_spec.dtype,
        parent.index_spec.missing,
    ).set_parent_col(self)
    self.adj_prev = AdjPrevIdxColSchema(
        f"_adj_prev_{full_name}",
        parent.index_spec.dtype,
        parent.index_spec.missing,
    ).set_parent_col(self)

    target.register_new_column(self.adj_head)
    parent.register_new_column(self.adj_next)
    parent.register_new_column(self.adj_prev)

    if self.adjacency_conf.track_counts:
        assert self.adjacency_conf.counts_dtype is not None

        self.adj_count = AdjCountColSchema(
            f"_adj_count_{full_name}",
            self.adjacency_conf.counts_dtype,
            0,
        ).set_parent_col(self)

        target.register_new_column(self.adj_count)

IndexSpec dataclass

Specification for integer indices used to address rows.

Defines the underlying numpy data type, the sentinel value indicating a missing or null link, and the maximum valid row index.

Attributes:

Name Type Description
dtype type[T]

The numpy integer data type for the index.
missing int

The sentinel integer value representing a missing index.
max_value int

The maximum valid integer value for an index.
Source code in src/packed_data_structures/schemas/index_spec.py 
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
@dataclass(frozen=True, slots=True)
class IndexSpec[T: np.integer[Any]]:
    """Specification for integer indices used to address rows.

    Defines the underlying numpy data type, the sentinel value indicating
    a missing or null link, and the maximum valid row index.

    Attributes:
        dtype: The numpy integer data type for the index.
        missing: The sentinel integer value representing a missing index.
        max_value: The maximum valid integer value for an index.
    """

    dtype: type[T]
    missing: int
    max_value: int

    @classmethod
    def from_dtype(cls, dtype: type[T]) -> IndexSpec:
        """Create an IndexSpec from a numpy data type.

        By standard convention, the maximum representable value of the given
        integer type is reserved as the missing/sentinel value.

        Args:
            dtype: A numpy integer data type.

        Returns:
            A new IndexSpec instance.

        Raises:
            TypeError: If the provided dtype is not an integer type.
        """
        if not np.issubdtype(dtype, np.integer):
            raise TypeError(f"Index dtype must be integer, got {dtype}")

        info = np.iinfo(dtype)
        # Standard convention: Max value is the missing value sentinel
        return cls(dtype=dtype, missing=info.max, max_value=info.max - 1)

    def new_array(self, size: int) -> np.ndarray[Any, np.dtype[T]]:
        """Helper to allocate raw numpy arrays with correct initialization."""
        arr = np.empty(size, dtype=self.dtype)
        arr[:] = self.missing
        return arr

from_dtype classmethod 

from_dtype(dtype: type[T]) -> IndexSpec

Create an IndexSpec from a numpy data type.

By standard convention, the maximum representable value of the given integer type is reserved as the missing/sentinel value.

Parameters:

Name Type Description Default
dtype type[T]

A numpy integer data type.

 required

Returns:

Type Description
IndexSpec

A new IndexSpec instance.

Raises:

Type Description
TypeError

If the provided dtype is not an integer type.
Source code in src/packed_data_structures/schemas/index_spec.py 
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
@classmethod
def from_dtype(cls, dtype: type[T]) -> IndexSpec:
    """Create an IndexSpec from a numpy data type.

    By standard convention, the maximum representable value of the given
    integer type is reserved as the missing/sentinel value.

    Args:
        dtype: A numpy integer data type.

    Returns:
        A new IndexSpec instance.

    Raises:
        TypeError: If the provided dtype is not an integer type.
    """
    if not np.issubdtype(dtype, np.integer):
        raise TypeError(f"Index dtype must be integer, got {dtype}")

    info = np.iinfo(dtype)
    # Standard convention: Max value is the missing value sentinel
    return cls(dtype=dtype, missing=info.max, max_value=info.max - 1)

new_array 

new_array(size: int) -> np.ndarray[Any, np.dtype[T]]

Helper to allocate raw numpy arrays with correct initialization.

Source code in src/packed_data_structures/schemas/index_spec.py 
48
49
50
51
52
def new_array(self, size: int) -> np.ndarray[Any, np.dtype[T]]:
    """Helper to allocate raw numpy arrays with correct initialization."""
    arr = np.empty(size, dtype=self.dtype)
    arr[:] = self.missing
    return arr

ObjectColSchema dataclass

Bases: ColSchemaLike[object_]

Schema for a data column that uses numpy C-level object pointer arrays.

Attributes:

Name Type Description
name str

The string identifier for the column.
Source code in src/packed_data_structures/schemas/object_col.py 
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
@dataclass(eq=False)
class ObjectColSchema[T, *T_shape](ColSchemaLike[np.object_]):
    """Schema for a data column that uses numpy C-level object pointer arrays.

    Attributes:
        name: The string identifier for the column.
    """

    name: str

    def init_array(self) -> PackedObjectArray[T]:
        return cast(
            PackedObjectArray[T], PackedObjectArray(self.parent_table.pre_allocate)
        )

    def __hash__(self) -> int:
        return hash(id(self))

PolymorphicForeignKeySchema dataclass

Bases: ColSchemaLike[integer[Any]]

Schema for a column of polymorphic foreign keys.

Each key can link to one of a set of tables. Just like the non-polymorphic FK schema, it automatically injects adjacency list columns: - adj_head (in target table) - adj_next (in source table) - adj_prev (in source table) - adj_count (optional, in target table)

NOTE: The true column names are more verbose

It also injects a _type_id column, this column encodes which table each key points to. The smallest possible uint dtype is chosen for the _type_id columm.

Class Type Parameters:

Name Bound or Constraints Description Default
T_parent integer[Any]

The index dtype of the parent table. And by extension the adj_next and adj_prev columns.

 required
T_counts integer[Any]

The dtype of the counts column (if applicable).

 required

Attributes:

Name Type Description
name str

The string identifier for the column.
target_tables Sequence[TableSchema]

The ordered collection of TableSchemas that the keys can point to.
on_delete FksOnDeleteStyle

The policy to apply when something tries to delete a row referenced by this column.
adjacency_conf AdjacencyListConf[T_counts]

Configuration for the injected adjacency list columns.
keys_idx_spec IndexSpec[unsignedinteger[Any]]

IndexSpec that holds the dtype and sentinel for the keys. Optmized to be only as big as needed to link into any row of the target tables.
type_id_dtype type[unsignedinteger[Any]]

The dtype used for the _type_id column.
type_id_sentinel int

The sentinel value for the _type_id column.
type_id_mapping dict[TableSchema, int]

Mapping from table to its type id. This is just a direct mapping to the index of each target in target_tables
type_id_col PmFkTypeIdColSchema[unsignedinteger[Any]]

The injected _type_id column.
adj_next AdjNextIdxColSchema[T_parent]

The injected next-pointer column in the parent table.
adj_prev AdjPrevIdxColSchema[T_parent]

The injected previous-pointer column in the parent table.
adj_head_columns list[AdjHeadIdxColSchema[T_parent]]

The injected head-pointer columns in the target tables, ordered by type_id.
adj_count_columns list[AdjCountColSchema[T_counts]]

The injected count columns in the target table (if enabled). Also ordered by type_id
Source code in src/packed_data_structures/schemas/pm_foreign_key_col.py 
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
@dataclass(eq=False)
class PolymorphicForeignKeySchema[
    T_parent: np.integer[Any],
    T_counts: np.integer[Any],
](ColSchemaLike[np.integer[Any]]):
    """Schema for a column of polymorphic foreign keys.

    Each key can link to one of a set of tables.
    Just like the non-polymorphic FK schema, it automatically injects adjacency list columns:
    - `adj_head` (in target table)
    - `adj_next` (in source table)
    - `adj_prev` (in source table)
    - `adj_count` (optional, in target table)

    NOTE: The true column names are more verbose

    It also injects a `_type_id` column, this column encodes which table each key points to.
    The smallest possible uint dtype is chosen for the `_type_id` columm.

    Type Parameters:
        T_parent: The index dtype of the parent table. And by extension the `adj_next` and `adj_prev` columns.
        T_counts: The dtype of the counts column (if applicable).

    Attributes:
        name: The string identifier for the column.
        target_tables: The ordered collection of TableSchemas that the keys can point to.
        on_delete: The policy to apply when something tries to delete a row referenced by this column.
        adjacency_conf: Configuration for the injected adjacency list columns.
        keys_idx_spec: IndexSpec that holds the dtype and sentinel for the keys.
            Optmized to be only as big as needed to link into any row of the target tables.
        type_id_dtype: The dtype used for the `_type_id` column.
        type_id_sentinel: The sentinel value for the `_type_id` column.
        type_id_mapping: Mapping from table to its type id.
            This is just a direct mapping to the index of each target in `target_tables`
        type_id_col: The injected `_type_id` column.
        adj_next: The injected next-pointer column in the parent table.
        adj_prev: The injected previous-pointer column in the parent table.
        adj_head_columns: The injected head-pointer columns in the target tables, ordered by `type_id`.
        adj_count_columns: The injected count columns in the target table (if enabled).
            Also ordered by `type_id`
    """

    name: str
    target_tables: Sequence[TableSchema]

    on_delete: FksOnDeleteStyle = FksOnDeleteStyle.CASCADE
    adjacency_conf: AdjacencyListConf[T_counts] = field(
        default_factory=AdjacencyListConf
    )

    keys_idx_spec: IndexSpec[np.unsignedinteger[Any]] = field(init=False)

    type_id_dtype: type[np.unsignedinteger[Any]] = field(init=False)
    type_id_sentinel: int = field(init=False)
    type_id_col: PmFkTypeIdColSchema[np.unsignedinteger[Any]] = field(init=False)
    type_id_mapping: dict[TableSchema, int] = field(init=False)

    adj_next: AdjNextIdxColSchema[T_parent] = field(init=False)
    adj_prev: AdjPrevIdxColSchema[T_parent] = field(init=False)
    adj_head_columns: list[AdjHeadIdxColSchema[T_parent]] = field(init=False)
    adj_count_columns: list[AdjCountColSchema[T_counts]] = field(init=False)

    def __post_init__(self):
        n_targets = len(self.target_tables)

        optimal_power = n_targets.bit_length()
        self.type_id_dtype = BITS_TO_UINT_DTYPE[optimal_power]
        self.type_id_sentinel = np.iinfo(self.type_id_dtype).max

        self.type_id_mapping = {
            target: i for i, target in enumerate(self.target_tables)
        }

        required_max_value = 0

        for target in self.target_tables:
            required_max_value = max(required_max_value, target.index_spec.max_value)
            target.subscribe(self)

        self.keys_idx_spec = IndexSpec.from_dtype(
            BITS_TO_UINT_DTYPE[required_max_value.bit_length()]
        )

    def set_parent(self, parent: TableSchema[T_parent]):
        """Bind this polymorphic foreign key column to a parent table.

        This actively mutates both the parent and target table schemas by
        registering the hidden tag column and adjacency list management columns.

        Args:
            parent: The TableSchema that will own this foreign key.
        """
        super().set_parent(parent)

        targets = self.target_tables
        parent = self.parent_table

        full_name = f"{parent.name}_{self.name}"

        self.type_id_col = PmFkTypeIdColSchema(
            f"_type_id_{full_name}",
            self.type_id_dtype,
            self.type_id_sentinel,
        ).set_parent_col(self)

        self.adj_head_columns = [
            AdjHeadIdxColSchema(
                f"_adj_head_{full_name}",
                parent.index_spec.dtype,
                parent.index_spec.missing,
            ).set_parent_col(self)
            for _ in targets
        ]
        self.adj_next = AdjNextIdxColSchema(
            f"_adj_next_{full_name}",
            parent.index_spec.dtype,
            parent.index_spec.missing,
        ).set_parent_col(self)
        self.adj_prev = AdjPrevIdxColSchema(
            f"_adj_prev_{full_name}",
            parent.index_spec.dtype,
            parent.index_spec.missing,
        ).set_parent_col(self)

        for target, adj_head in zip(targets, self.adj_head_columns, strict=True):
            target.register_new_column(adj_head)

        parent.register_new_column(self.type_id_col)
        parent.register_new_column(self.adj_next)
        parent.register_new_column(self.adj_prev)

        if self.adjacency_conf.track_counts:
            assert self.adjacency_conf.counts_dtype is not None
            self.adj_count_columns = []

            for target in targets:
                adj_count = AdjCountColSchema(
                    f"_adj_count_{full_name}",
                    self.adjacency_conf.counts_dtype,
                    0,
                )
                target.register_new_column(adj_count)
                self.adj_count_columns.append(adj_count)

    def __hash__(self) -> int:
        return hash(id(self))

    def init_array(
        self,
    ) -> PackedArray:
        return PackedArray(
            self.parent_table.pre_allocate,
            self.keys_idx_spec.dtype,
            self.keys_idx_spec.missing,
        )

set_parent 

set_parent(parent: TableSchema[T_parent])

Bind this polymorphic foreign key column to a parent table.

This actively mutates both the parent and target table schemas by registering the hidden tag column and adjacency list management columns.

Parameters:

Name Type Description Default
parent TableSchema[T_parent]

The TableSchema that will own this foreign key.

 required
Source code in src/packed_data_structures/schemas/pm_foreign_key_col.py 
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
def set_parent(self, parent: TableSchema[T_parent]):
    """Bind this polymorphic foreign key column to a parent table.

    This actively mutates both the parent and target table schemas by
    registering the hidden tag column and adjacency list management columns.

    Args:
        parent: The TableSchema that will own this foreign key.
    """
    super().set_parent(parent)

    targets = self.target_tables
    parent = self.parent_table

    full_name = f"{parent.name}_{self.name}"

    self.type_id_col = PmFkTypeIdColSchema(
        f"_type_id_{full_name}",
        self.type_id_dtype,
        self.type_id_sentinel,
    ).set_parent_col(self)

    self.adj_head_columns = [
        AdjHeadIdxColSchema(
            f"_adj_head_{full_name}",
            parent.index_spec.dtype,
            parent.index_spec.missing,
        ).set_parent_col(self)
        for _ in targets
    ]
    self.adj_next = AdjNextIdxColSchema(
        f"_adj_next_{full_name}",
        parent.index_spec.dtype,
        parent.index_spec.missing,
    ).set_parent_col(self)
    self.adj_prev = AdjPrevIdxColSchema(
        f"_adj_prev_{full_name}",
        parent.index_spec.dtype,
        parent.index_spec.missing,
    ).set_parent_col(self)

    for target, adj_head in zip(targets, self.adj_head_columns, strict=True):
        target.register_new_column(adj_head)

    parent.register_new_column(self.type_id_col)
    parent.register_new_column(self.adj_next)
    parent.register_new_column(self.adj_prev)

    if self.adjacency_conf.track_counts:
        assert self.adjacency_conf.counts_dtype is not None
        self.adj_count_columns = []

        for target in targets:
            adj_count = AdjCountColSchema(
                f"_adj_count_{full_name}",
                self.adjacency_conf.counts_dtype,
                0,
            )
            target.register_new_column(adj_count)
            self.adj_count_columns.append(adj_count)

StringColSchema dataclass

Bases: ColSchemaLike[str_]

Schema for a fixed-length Unicode string column.

Attributes:

Name Type Description
name str

The string identifier for the column.
max_length int

The maximum allowed length for the string.
default str

The default string value.
Source code in src/packed_data_structures/schemas/string_col.py 
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
@dataclass(eq=False)
class StringColSchema(ColSchemaLike[np.str_]):
    """Schema for a fixed-length Unicode string column.

    Attributes:
        name: The string identifier for the column.
        max_length: The maximum allowed length for the string.
        default: The default string value.
    """

    name: str
    max_length: int
    default: str = ""
    dtype: np.dtype[np.str_] = field(init=False)

    def __post_init__(self):
        self.dtype = cast(np.dtype[np.str_], np.dtype((np.str_, self.max_length)))

    def init_array(self) -> PackedStringArray:
        return PackedStringArray(
            self.parent_table.pre_allocate,
            self.dtype,
            self.default,
        )

    def __hash__(self) -> int:
        return hash(id(self))

SupportsGetTableSchema

Bases: ABC

Interface for objects that can provide a TableSchema.

Source code in src/packed_data_structures/schemas/table.py 
19
20
21
22
23
24
25
26
27
28
29
class SupportsGetTableSchema[T_idx: np.integer[Any]](ABC):
    """Interface for objects that can provide a TableSchema."""

    @abstractmethod
    def get_table_schema(self) -> TableSchema[T_idx]:
        """Get the underlying TableSchema.

        Returns:
            The TableSchema instance.
        """
        ...

get_table_schema abstractmethod 

get_table_schema() -> TableSchema[T_idx]

Get the underlying TableSchema.

Returns:

Type Description
TableSchema[T_idx]

The TableSchema instance.
Source code in src/packed_data_structures/schemas/table.py 
22
23
24
25
26
27
28
29
@abstractmethod
def get_table_schema(self) -> TableSchema[T_idx]:
    """Get the underlying TableSchema.

    Returns:
        The TableSchema instance.
    """
    ...

TableSchema dataclass

Bases: SupportsGetTableSchema[T_idx]

Schema definition for a flat, column-oriented table.

A TableSchema aggregates multiple ColSchemaLike definitions and dictates how the PackedArrayTable initializes its raw buffers.

Attributes:

Name Type Description
name str

The string identifier for the table.
index_spec IndexSpec[T_idx]

The specification defining the table's index type and capacity.
cols list[ColSchemaLike[Any]]

The list of column schemas defining the table structure.
pre_allocate int

The initial element capacity to allocate for the table's arrays.
Source code in src/packed_data_structures/schemas/table.py 
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
@dataclass(slots=True, eq=False)
class TableSchema[T_idx: np.integer[Any]](SupportsGetTableSchema[T_idx]):
    """Schema definition for a flat, column-oriented table.

    A TableSchema aggregates multiple `ColSchemaLike` definitions and
    dictates how the `PackedArrayTable` initializes its raw buffers.

    Attributes:
        name: The string identifier for the table.
        index_spec: The specification defining the table's index type and capacity.
        cols: The list of column schemas defining the table structure.
        pre_allocate: The initial element capacity to allocate for the table's arrays.
    """

    name: str
    index_spec: IndexSpec[T_idx]
    cols: list[ColSchemaLike[Any]]
    pre_allocate: int = 0

    subscribers: list[
        ForeignKeySchema[np.integer[Any], T_idx, np.integer[Any]]
        | PolymorphicForeignKeySchema[np.integer[Any], np.integer[Any]]
    ] = field(init=False, default_factory=list)
    col_ids: dict[ColSchemaLike, int] = field(init=False, default_factory=dict)
    _finalized: bool = field(init=False, default=False)

    def __post_init__(self):
        for col in self.cols:
            col.set_parent(self)

        for i, col in enumerate(self.cols):
            self.col_ids[col] = i

    def subscribe(self, new_subscriber: ForeignKeySchema | PolymorphicForeignKeySchema):
        """Register a foreign key that targets this table.

        Args:
            new_subscriber: The foreign key schema pointing to this table.
        """
        if new_subscriber not in self.subscribers:
            self.subscribers.append(new_subscriber)

    def init_arrays(
        self,
    ) -> tuple[PackedArray, ...]:
        self._finalized = True
        return tuple(col.init_array() for col in self.cols)

    def register_new_column(self, col: ColSchemaLike):
        """Dynamically add a new column to the table schema.

        This is primarily used by overlay features and foreign keys to inject hidden
        management columns prior to initialization.

        Args:
            col: The column schema to add.

        Raises:
            RuntimeError: If the schema has already been initialized.
        """
        if self._finalized:
            raise RuntimeError(
                f"Cannot register column '{col.name}' to table '{self.name}': "
                "Schema is already finalized/initialized"
            )

        self.col_ids[col] = len(self.cols)

        self.cols.append(col)
        col.set_parent(self)

    def get_table_schema(self) -> TableSchema:
        return self

register_new_column 

register_new_column(col: ColSchemaLike)

Dynamically add a new column to the table schema.

This is primarily used by overlay features and foreign keys to inject hidden management columns prior to initialization.

Parameters:

Name Type Description Default
col ColSchemaLike

The column schema to add.

 required

Raises:

Type Description
RuntimeError

If the schema has already been initialized.
Source code in src/packed_data_structures/schemas/table.py 
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
def register_new_column(self, col: ColSchemaLike):
    """Dynamically add a new column to the table schema.

    This is primarily used by overlay features and foreign keys to inject hidden
    management columns prior to initialization.

    Args:
        col: The column schema to add.

    Raises:
        RuntimeError: If the schema has already been initialized.
    """
    if self._finalized:
        raise RuntimeError(
            f"Cannot register column '{col.name}' to table '{self.name}': "
            "Schema is already finalized/initialized"
        )

    self.col_ids[col] = len(self.cols)

    self.cols.append(col)
    col.set_parent(self)

subscribe 

subscribe(
    new_subscriber: ForeignKeySchema
    | PolymorphicForeignKeySchema,
)

Register a foreign key that targets this table.

Parameters:

Name Type Description Default
new_subscriber ForeignKeySchema | PolymorphicForeignKeySchema

The foreign key schema pointing to this table.

 required
Source code in src/packed_data_structures/schemas/table.py 
65
66
67
68
69
70
71
72
def subscribe(self, new_subscriber: ForeignKeySchema | PolymorphicForeignKeySchema):
    """Register a foreign key that targets this table.

    Args:
        new_subscriber: The foreign key schema pointing to this table.
    """
    if new_subscriber not in self.subscribers:
        self.subscribers.append(new_subscriber)