@BertrandBordage very interesting! I tried to do something similar in wagtail-inventory, but hadn't considered the similarity to how Django looks up related objects.

In wagtail-inventory the page->block relationships are stored in a DB table, updated via page signals, and exposed for query through the admin. I'd (perhaps wrongly) assumed that doing an on-the-fly query wouldn't be scalable for large sites - have you done any performance testing of this approach across many pages/complex streamfields?

@BertrandBordage Do you think this part would go into core eventually? I’d like to write tests doc for it, but am wondering what would be the correct place for it. An add-on product maybe?

Another thing: I tried this in bakerydemo, and tried to find the linked streamfield on the homepage, but it doesn't find that. In bakerydemo.base.models.py:

    def get_usage(self):
        if getattr(settings, 'IMAGE_USAGE_SEARCHES_STREAMFIELDS', False):
            from wagtail.wagtailcore.blocks.collector import get_all_uses
            from wagtail.wagtailimages.blocks import ImageChooserBlock
            uses = get_all_uses(ImageChooserBlock, [self])
        else:
            return get_object_usage(self)

This is something that a test could likely clarify.

And in order for this to work with existing templates, we'd have to create a wrapper around it that has the count() method, for example like in https://github.com/wagtail/wagtail/pull/3954/files#diff-e0210a80d3ac707eaa237beb0f6e5cc5R89, right?

@chosak It’s indeed sensibly slower than having an intermediate table that’s automatically updated.
However, this is easier to maintain for now. Having an automatic intermediate table is pretty heavy and can lead to an outstanding number of issues, like:

• database changes outside Django will lead to stale records in this table
• bulk changes inside the database will also lead to stale records in this table
• it significantly increases the database size

We might defer doing this until there is some support for TRIGGERs in Django.

@khink Yes, this will go to the core, along with a similar collector for rich text fields/blocks. It indeed needs a lot of tests before being considered stable.
I don’t think the way you use it does what you want. When you do get_all_uses(ImageChooserBlock, [self]), it searches in all models if there is an ImageChooserBlock somewhere where the value is in [self]. As far as I understand, self is here your home page model. So you’re telling it to find an image block where the value is an home page, which is irrelevant, and therefore why you don’t get any result.
About the API to provide on this feature, I think we need to create an additional abstraction layer that will join results from Django’s collector, streamfield & rich text collectors. .count() will probably no longer be relevant, as we won’t directly use QuerySets anymore, because we will fetch results from various models. By the way, get_object_usage is really basic as it only shows usage in Page objects as well as inline models, but nothing for snippets or any other model.

Bumping this due to this similar request on Stack Overflow looking for a way to find all pages containing a snippet.

Just improved a lot this pull request by adding a collector for rich text, as well as using Django’s collector for simple ForeignKey/ManyToMany/etc relations.

Now, the get_all_uses function takes only a single object as argument:

from wagtail.core.collectors import get_all_uses

get_all_uses(Image.objects.first())

I’m aware that this work is difficult to review: it’s fairly complex and it’s hard to figure out what’s going on without an excellent knowledge of StreamField, RichText and Model internals.

If someone could at least review the tests to check that at least it works as expected, it would be awesome :)

Following @lb-’s review, I improved this pull request. Now it shows all related objects, including those with other on_delete than CASCADE and PROTECT. And now it hides Rendition, PageRevision and inline objects, as discussed with @gasman :)

I’m also proud to announce that for the first time in Django history, users can see what will happen when deleting an object. I added a new label to get user attention on the on_delete behaviour. I tried to make it as clear and short as possible. Here are a few examples:

Here are answers to @lb-’s review:

1. Discussed above
2. Not really related to this pull request, but fixed anyway.
3. Fixed, this was because SET_NULL relations are not returned by the Django collector.
4. Same as 3.
5. It works on any model, even custom images and documents. The links within the table also work with custom images and documents.
6. FormField is an inline, it is now hidden from the deletion view.
7. PageRevision instances are now hidden from the deletion view.
8. This is normal. Deleting only has an impact on reverse relations, not forward relations.
9. Same as 3.
10. Same as 3.
11. Same as 3.

By the way, for related StreamField and RichText, I wrote that deleting this object will mark the field as empty (i.e. SET_NULL behaviour).
I know that nothing changes in database for related StreamField and RichText, but as far as I remember, removing a related object gives an empty field in the admin, right?

@BertrandBordage @gasman I have tried to think of other use cases for showing the usage view / count beyond when needing deletion and cannot think of any for Images/Documents.

There is a case maybe for the navigation aspect in ModelAdmin, let's say you have a Person model and you want to be able to show the Person listing but also provide a link in each row to 'pages that use this Person'. The goal here is to navigate around Admin but not using the Explorer tool.

This could be reasonably easily reimplemented, but might be even easier if we kept the snippet model's get _usage_url method and the Admin url declaration.

Also - @BertrandBordage this is such an awesome feature, thanks so much!

Just loaded up the latest version of this PR, looking awesome!

The behaviour on related StreamField and RichText is correct as you have indicated on the delete page.

Eg. When an image is deleted the resulting RichText field just shows up nothing, and StreamField or ImageChooserPanel just have empty links shown when you next edit the page that used them.

Hi i've just added this #4169

I'm less concerned about showing how many times images and documents are used but it feels important to show that snippets are used in more than one place. It could be that we show a warning to say it's used in more than one place with a link to the full list to avoid performance issues.

Any thoughts?

Also I think this needs some UI design work. I can prioritise this if there is urgency?

@tmsndrs: This will still have a very important impact on performance for now. The first things I check are related objects through foreign keys, many-to-many and generic relations. To do that, I use the Django Collector class that fetches all related objects at once, triggering dozens to hundreds of queries. That class would have to be written again to use generators instead, which would be a significant effort, especially to test.

So it’s not a definitive “no”.

Also @tmsndrs, feel free to submit a mockup of what you have in mind in terms of UI :)

Looks like the rich text handling here will need to be substantially reworked now that #4079 is merged :-(

Unfortunately c293860 is already pushing the limit of what's feasible to review in one go - I think this could really do with splitting up into multiple PRs:

• Refactoring rich text (and streamfield, anything else...?) to add supporting methods like LinkHandler.get_instance as required by collectors
• Adding the collectors
• UI updates

Shall we bump this to 2.1? (Now that I've seen the full scope of the changes here, it feels like another major codebase-spanning refactor is a bit too much for 2.0, and this way we get some breathing room for @tmsndrs to give it some proper design attention...)

@tmsndrs:

I can prioritise this if there is urgency?

yes please

@BertrandBordage excellent work here (as usual).

I have some thoughts regarding one of the decisions made above though, which is the one to always exclude ParentalKey relationships (and to a lesser extent ParentalManyToMany). While it's true that ParentalKey and the InlinePanel were designed for pages, we must recognise that it is still currently the only way to achieve an 'inline' editing UI using panels, and therefore, it is incredibly common for models to subclass modelcluster.models.ClusterableModel and use ParentalKey in order to use InlinePanel. For example, wagtailmenus takes this approach.

With this in mind, can we really be sure those relationships should never be surfaced in this interface? Is there any way we might cater for cases where including them would be preferred?

@ababic I think there's a similar argument to wagtail/django-modelcluster#89 (comment) to be made here: ParentalKey is inherently an "X belongs to Y" relation rather than "X contains a reference to Y".

Just as there's no conceivable reason not to use on_delete=CASCADE on a ParentalKey (and this isn't just a case of "I can't think of a reason"; rather, it's basically implied in the design of ParentalKey that CASCADE is the only logical choice), I'm confident that there's no scenario where it's appropriate to bring up a ParentalKey relation to X as "a place where X is being used".

Anything involving InlinePanel certainly shouldn't be surfaced here: it's redundant to tell the user "deleting this page will cause the 'see also' links on the page to be deleted", or "deleting this menu will cause its submenus to be deleted". (For the casual user who isn't thinking in terms of database schemas, it most likely doesn't even occur to them that the thing in an InlinePanel is an independent object that could exist outside of its container - to them, it's just a repeating field.)

Sorry, I really should have updated the 'needs review' / 'needs work' labels when I added #3961 (comment) ... I thought this was still on my review queue, but in fact it still needs the rich text bits to be rewritten following #4079, and - ideally - being split up into smaller PRs. (There's a lot going on in c293860 in particular, and we need to make sure that breaking changes such as changing the API of register_rich_text_embed_handler / register_rich_text_link_handler - or their Wagtail 2.0 equivalents - don't slip under the radar.)

Thanks for the feedback @gasman. I'm with you now.

Things are slightly muddied here, as we're dealing with surfacing PROTECTED relationships as well as those that would be affected by a delete operation were it to happen. I was thinking somehow that ignoring ParentalKey relationships would prevent say, a menu item linking to a Document via a PROTECTED relationship being flagged up when attempting to delete that Document. But, such relationships would likely use a standard ForeignKey field, and so would still appear as expected.

I'll shut up now :)

Superseded by #4481.