Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature suggestion: Track registered Mongo collection instances. #13174

Merged

Conversation

JorgenVatle
Copy link
Contributor

The ability to retrieve a collection by name should ease with some edge cases where package authors rely on shims for the Mongo.Collection module in order to add functionality. As far as I'm aware, there aren't really any good ways of retrieving collections by name, you need to import them explicitly, or patch directly Mongo.Collection to add this functionality.

cultofcoders:redis-oplog is a good example of where that doesn't always work out well. Collections registered by compiler/build plugin packages appear to be registered before standard feature packages. This leads to the oplog package having no real way to reliably access those collection instances.

Just opening this as a draft PR so we can have a quick discussion whether this would be a worthwhile addition. 👍

The idea behind this is to ease with package development where collections cannot always be imported explicitly.

The ability to retrieve a collection by name should ease with some edge cases where package authors rely on shims for the Mongo.Collection global in order to add functionality.

cultofcoders:redis-oplog is a good example of where this doesn't always work out well. Collections registered by compiler/build plugin packages appear to be registered before standard feature packages. This leads to the oplog package having no real way to reliably access those collection instances.
@CLAassistant
Copy link

CLAassistant commented Jun 5, 2024

CLA assistant check
All committers have signed the CLA.

StorytellerCZ
StorytellerCZ previously approved these changes Jun 6, 2024
nachocodoner
nachocodoner previously approved these changes Jun 6, 2024
Copy link
Member

@nachocodoner nachocodoner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made similar changes in past projects and believe it's good to integrate them into the core.

Anyone has any inconvenience here?

@JorgenVatle JorgenVatle marked this pull request as ready for review June 13, 2024 09:31
@harryadel
Copy link
Contributor

@JorgenVatle
Copy link
Contributor Author

The proposed changes shouldn't have any impact on mongo-collection-instances. The collection map is added under Mongo.Collections not Mongo.Collection.

Though, seeing this package's implementation, it would be nice to have a simple .get() method on Mongo.Collection, then hiding away the Collections map with an underscore or something to indicate it's something that shouldn't be accessed directly. In this case, mongo-collection-instances should still function correctly, it would just override the .get() method provided by Meteor. The collection map would still work and be accessible to other packages as it is tracked in the Mongo.Collection constructor.

@JorgenVatle
Copy link
Contributor Author

JorgenVatle commented Jun 13, 2024

A .get() method directly on Mongo.Collection would probably a nicer way to access registered collections instead of exposing a plain Map instance.

I'd be happy to update the PR accordingly next week and add some documentation for it if it sounds good to you. 👍

@harryadel
Copy link
Contributor

Thanks for the explanation Jorgen, I'm no part of the core team but if @nachocodoner & @leonardoventurini, maybe even @radekmie since he has done many mongo performance improvements then I think it'd be a good idea to act on. Thank you for your efforts.

denihs
denihs previously approved these changes Jun 14, 2024
@radekmie
Copy link
Collaborator

I think it's fine to add such a global registry, but I definitely would not make it a public API. It can be one of those "documented internals" instead, where Meteor would try to but wouldn't have to keep backward compatibility.

Also, it's now technically possible to get all collection names through drivers but not their instances, so I'd call it something like Mongo._collectionInstances.

@leonardoventurini
Copy link
Member

This might come in handy to prevent circular imports. I've used mongoose in the past quite a bit for raw Node.js projects, and it allows us to get the model by simply calling model('name'), this helps when we reference other collections inside hooks, instead of importing them there directly, which as the collections are sometimes exported all at once, could cause a chicken and egg dilemma. With that, I believe having this ability in Meteor is a great add, but I would vouch for making this a public API, perhaps making it more ergonomic even like collection('name').

@JorgenVatle JorgenVatle marked this pull request as draft June 23, 2024 11:51
@JorgenVatle
Copy link
Contributor Author

Haven't had the chance to follow up the proposal just yet. Got some time on Monday and/or Tuesday and will update the PR from there. 👍

@JorgenVatle JorgenVatle marked this pull request as ready for review June 29, 2024 17:41
@JorgenVatle
Copy link
Contributor Author

@leonardoventurini I've opted not to make any changes to the collection constructor so far in this PR as I worry that it might involve some changes to current functionality for the Collection constructor.

That being said, in current versions of Meteor, calling Mongo.Collection() without the new keyword does cause a somewhat cryptic exception during replication setup. So it might not be much of a risk to begin with. There's probably even some room to add a failsafe to warn users when you forget to new keyword.

Should be a very quick and easy change for either implementation though.

@JorgenVatle
Copy link
Contributor Author

Oops. just came to realize I completely misunderstood your suggestion @leonardoventurini. I got fixated on using the constructor for retrieving collections.

@jamauro
Copy link
Contributor

jamauro commented Jul 2, 2024

Thanks for putting this together. It'll be a really nice addition. One thought on the most recent changes: do you think people might get confused by Mongo.Collection vs Mongo.collection? It's also not implicitly creating a collection like the Node.js Mongo driver db.collection which could lead to some confusion. I wonder if it's worth changing it to avoid that, e.g. Mongo.getCollection which would align with Mongo docs (but apparently not the Node driver). Alternatively, maybe the name should remain Mongo.collection and it should implicitly create the collection if one doesn't exist to align with the Node Mongo driver. That might be kind of nice as an alternative to new Mongo.Collection and it could remain backwards compatible.

Just some food for thought. Naming is hard. :)

@harryadel
Copy link
Contributor

@jamauro That's a very interesting point, worth considering 👍

@JorgenVatle
Copy link
Contributor Author

JorgenVatle commented Jul 3, 2024

Very fair point! I agree, there is certainly a concern that there might be some confusion between the two. I love the idea of implicitly creating the collection. Though, I believe it could lead to some edge cases.

Specifically, if you are defining your collections using new Mongo.Collection() but you have a call to Mongo.collection() that gets registered before the constructor, the constructor will end up throwing an exception since two collections with the same name is not allowed. Collection options could possibly also be a concern if we were to refactor to handle that.

I think getCollection would be the best middle ground here. It clearly indicates behavior and it's close to the mongosh API.

@leonardoventurini leonardoventurini changed the base branch from devel to release-3.0.2 July 31, 2024 13:49
@leonardoventurini leonardoventurini merged commit 1a40744 into meteor:release-3.0.2 Jul 31, 2024
5 of 7 checks passed
@StorytellerCZ StorytellerCZ added this to the Release 3.0.2 milestone Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants