Skip to content

wrmsr/omnibus

Repository files navigation

Omnibus is my personal collection of portable, general purpose python tools. Although documentation is.. lacking.. this code is intended for (and is actively seeing) production use - this is not experimental or hack code.

Its modules include:

  • lang, a large collection of language-level helpers including:
  • caches, containing a pretty beefy cache implementation inspired by Guava's CacheBuilder.
  • check, non-optional assertions inspired by Guava Preconditions.
  • collections, including:
    • Sorted collections powered either by sortedcontainers or a builtin fallback skiplist
    • topological sorting (added to functools in python 3.9 but not present in 3.7)
    • Identity and Frozen collections
  • defs, a self-awarely relatively unpythonic set of helpers for defining common boilerplate methods (repr, hash_eq, delegates, etc) in class definitions. Should be used sparingly for methods not directly used by humans (like repr) - @property's should remain @property's for type annotation, tool assistance, debugging, and otherwise, but these are still nice to have in certain circumstances (the real-world alternative usually being simply not adding them).
  • dynamic, dynamically scoped variables (implemented by stackwalking). Unlike threadlocals these are generator-correct both in binding and retrieval, and unlike ContextVars they require no manual context management. They are however slow and should be used sparingly (once per sql statement executed not once per inner function call).
  • inject, a dependency injection system heavily inspired by Guice (specifically MiniGuice). Supports annotation powered introspection, private modules, set and dict binders, and guice-style scopes and child injectors, with intended support for proxies / circular injection, type converters, and overriding modules as needed. Being aware of the two-dozen odd other python DI systems I'm still happiest with mine, basically 'doing what Guice does' (impl-wise, not just skin deep) but with a tenth the complexity (python kwargs over builders, little regard for its error messages as its graphs tend to be smaller, etc).
  • iterables, a collection of composable, transparent iterable transformations, which unlike most opaque generator compositions can be externally inspected, analyzed, rewritten, and optimized and fused as desired (as well as carrying with them more descriptive reprs and debug information).
  • properties, a set of @property-like descriptors like cached, locked_cached, set_once, class_, cached_class, and more interesting ones like an mro-honoring registry property (with optional singledispatch).
  • pydevd utilties, a small but likely growing collection of (completely optional) tools to make pydevd (PyCharm's, among other python IDE's, debugger) do hard things. Originally explored and added to get spark jvm python subprocesses to connect back to an already-debugging PyCharm instance to debug PySpark jobs.
  • reflect, an imo missing stdlib component for breaking apart and representing python's ever-expanding generic typing machinery in a more stable and friendly object hierarchy. Frees users from having to deal with notoriously version-volatile typing impl detail like __args__, __origin__, Generic's __mro_entries__, and such - ideally approximating something stable like java.lang.reflect.
  • replserver, a background thread that opens a unix socket server accepting connections to an in-proc python repl (from which one can inspect module globals, live thread stacks, and other such things).

Many other modules are in the works now that this has some of my attention again but none are stable enough yet for inclusion in master.

It unapologetically requires python 3.7 . It has one single mandatory dependency: toolz - itself having no dependencies. It does however optionally interop with a number of other libraries including:

But again, the only required dependency is toolz, and toolz has no transitive dependencies.

Notably the code is organized, python-stdlib-style, into a flat list of relatively large files. This is, again, largely in keeping tradition with python's stdlib, and in practice thus far nothing has yet justified breaking into a subpackage (though that may soon be changing). My app-level code tends to be much more decomposed into subdirectories of smaller files, but this is lib code.

Releases

No releases published

Packages

No packages published

Languages