-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build a collections adaptor #758
Comments
Stu: we should go streaming first on this! The adaptor uses an async iterator with a stream under the hood (but passes full objects to the callback). It should also decode on the fly for get etc |
Query: should it just be time series? So you get a key by id, or you get keys between two dates (or before/after one date) Maybe allow key scanning? Pass a pattern and we'll find keys which match that name |
Probable API:
|
I've remembered that in the original doc, Taylor suggested a key generator function. I've been thinking about this all morning and I actually think it's a way better solution - so I'm going to deviate from the spec on |
Overview
Create a Collections adaptor that speaks to the Lightning Collections API.
It should probably be suffixed by the backend data store, like
collections-postgres
orcollections-lightning
or something, so that later it's easier to introduce new collection types, likecollections-redis
.The collections API is a bit unusual in that it will be loaded as a second adaptor to jobs. It's a second class citizen in the job code. So everything needs to be namespaced - collections and operations and stuff. So
collections.get
rather thanget
. Otherwise there's a risk for clashing with the main adaptor namespace.API
Configuration
Note that config will be set by the worker automatically. Maybe later Lightning will take more control.
Later work
Maybe add APIs for administering a collection
The text was updated successfully, but these errors were encountered: