serdio package#

Submodules#

Module contents#

Serdio: Automatic serialization of function inputs and outputs using MessagePack.

The primary goal of Serdio is to lift arbitrary functions into useful “byte-handlers”, i.e. functions mapping bytes to bytes, that otherwise preserve their original behavior. Serdio also comes with a MessagePack-based serialization format that allows one to seamlessly convert Python values to and from Serdio-bytes. serdio.lift_io() is responsible for decorating and lifting functions, whereas serdio.serialize() and serdio.deserialize() implement an encoder and decoder for the Serdio serialization spec.

Usage

@serdio.lift_io
def my_cool_function(x: int, y: float, b: float = 1.0) -> float:
    z = x * y
    z += b
    return z

bytes_handler: Callable[bytes, bytes] = my_cool_function.as_bytes_handler()

z = my_cool_function(2, 3.0)
assert z == 7.0

We can now use the bytes_handler function to map Serdio-bytes to Serdio-bytes:

xyb_bytes = serdio.serialize(2, 3.0, b=2.0)
zbytes = bytes_handler(xyb_bytes)

z = serdio.deserialize(zbytes)
assert z == 8.0
class serdio.SerdeHookBundle(encoder_hook, decoder_hook)[source]#

Bases: object

An encoder-decoder hook pair for user-defined types.

The encoder_hook and decoder_hook specify how to convert from a user-defined type into an equivalent collection of Python-native values and back. Thus for any object X of user-defined type T, the following relationship should hold:

hook_bundle = SerdioHookBundle(f, g)
native_X = hook_bundle.encoder_hook(X)  # f(X)
Y = hook_bundle.decoder_hook(native_X)  # g(native_X)
assert X == Y

Note that native_X above needs to be some collection of native Python values, e.g. a simple dataclass can be represented as a dictionary of attributes mapping to values.

Parameters:

  • encoder_hook (Callable) – An encoder function specifying how serdio.serde.serialize() should break down any custom types into Python native types.

  • decoder_hook (Callable) – The inverse of encoder_hook, specifying how serdio.serde.deserialize() should re-assemble the encoder_hook output into user-defined types.

decoder_hook: Callable#
encoder_hook: Callable#
to_dict()[source]#

Return the encoder-decoder hook pair as a dictionary.

Return type:

Dict

unbundle()[source]#

Return the encoder-decoder hook pair as a tuple.

Return type:

Tuple

serdio.bundle_serde_hooks(hook_bundle)[source]#

Helper to lift an encoder-decoder hook pair into a SerdeHookBundle.

Parameters:

hook_bundle – A tuple, list, dict or SerdeHookBundle containing an encoder-decoder hook pair. If a tuple or list, the encoder_hook must come first. If a dictionary, must have exactly two keys "encoder_hook" and "decoder_hook".

Returns:

A SerdeHookBundle encapsulating the encoder-decoder hook pair.

Raises:

ValueError – if the hook_bundle dictionary is malformed.

serdio.deserialize(serdio_bytes, decoder=None, as_signature=False)[source]#

Unpacks serdio-serialized bytes to an object

Parameters:

  • serdio_bytes (bytes) – Byte array to deserialize.

  • decoder (Optional[Callable]) – Optional callable specifying Messagepack decoder for user-defined types. See SerdeHookBundle for details.

  • as_signature (bool) – Optional boolean determining return format. If True, unpack the serialized byte array into an args tuple and a kwargs dictionary. This argument is most useful when the user is trying to serialize the inputs to a function of unknown arity.

Return type:

Any

Returns:

The deserialized object. If as_signature=True, assumes the resulting object is a dictionary with an args tuple and kwargs dict for values, and returns these two instead of the full dictionary.

serdio.lift_io(f=None, *, encoder_hook=None, decoder_hook=None, hook_bundle=None, as_handler=False)[source]#

Lift a function into an IOLifter.

The resulting IOLifter Callable is nearly identical to the original function, however it can also easily be converted to a bytes-handler with as_bytes_handler(). The bytes-handler expects Serdio bytes as input and returns Serdio bytes as its output.

This decorator expects at most one of these sets of kwargs to be specified:

  • encoder_hook and decoder_hook

  • hook_bundle

Parameters:

  • f (Optional[Callable]) – A Callable to be IO-lifted into a bytes-handler.

  • encoder_hook (Optional[Callable]) – An optional Callable that specifies how to convert custom-typed inputs or outputs into msgpack-able Python types (e.g. converting MyCustomClass into a dictionary of Python natives). See serdio.SerdeHookBundle for details.

  • decoder_hook (Optional[Callable]) – An optional Callable that specifies how to invert encoder_hook for custom-typed inputs and outputs. See serdio.SerdeHookBundle for details.

  • hook_bundle – An optional tuple, list, or SerdeHookBundle that simply packages up encoder_hook and decoder_hook Callables into a single object.

  • as_handler (bool) – A boolean controlling the return type of the decorator. If False, returns an IOLifter wrapping up f and the hook bundle specified by the supplied combination of encoder_hook, decoder_hook, and hook_bundle. If True, returns the result of applying lambda x: x.as_bytes_handler() to the IOLifter.

Returns:

An IOLifter wrapping up f, encoder_hook, and decoder_hook with the machinery needed to convert arbitrary functions into bytes-handlers. If as_handler=True, returns the bytes-handler version of f.

Raises:

  • ValueError – if user supplies wrong combination of encoder_hook, decoder_hook, and hook_bundle.

  • TypeError – if hook_bundle is not coercible to SerdeHookBundle, or if encoder_hook/decoder_hook are not Callables.

serdio.serialize(*args, encoder=None, **kwargs)[source]#

Serializes a set of args` and ``kwargs into bytes with MessagePack.

Parameters:

  • *args (Any) – Positional arguments to include in the serialized bytes

  • encoder (Optional[Callable]) – Optional callable specifying MessagePack encoder for user-defined types. See SerdeHookBundle for details.

  • kwargs (Any) – Keyword arguments to include in the serialized bytes

Return type:

bytes

Returns:

Dictionary of args and kwargs, serialized with MessagePack and optional custom encoder.

Raises:

TypeError – if encoder is not callable. Other errors can be raised by MessagePack during packing.