@jeremyb fyi, I disabled both the Spanish and English banner per this report.

In T374394#10132082, @Ciell wrote:

@jeremyb fyi, I disabled both the Spanish and English banner per this report.

ooops, sorry I didn't see the task sooner, thank you. this isn't relevant for the Spanish banner so I think I'll reenable that.

In T374394#10132128, @Jdlrobson wrote:

Most of errors are coming from Mozilla/5.0 (iPhone; CPU iPhone OS 17_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/128.0.6613.98 Mobile/15E148 Safari/604.1 and Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:130.0) Gecko/20100101 Firefox/130.0

hmmm, I will see if I can figure out how to test on one of these devices.

My guess is that those browsers might have a global variable tests defined from elsewhere. I think the code should be modified to not use global scope - e.g. wrapped in a function.

ahh. I am familiar with the wrapping in function and have used it elsewhere recently but maybe a bit rusty on exactly what that protects you from.

Depending on the amount of users seeing this banner, I would also advise limiting the API request to some kind of interaction e.g. a click

That's not possible, the whole point of the API query is to decide whether or not to show the banner. (on enwiki the categories I'm filtering on for article (ns0) views are on the corresponding Talk. (ns1) so they're also not available in wgCategories.) I was worried more about performance for the API requests. I tested pretty thoroughly on a few devices so I was less worried about JS errors but of course WP has an enormous variety of clients.

maybe I can coordinate with train, show it to a smaller sample of views, and watch logstash at the same time? but first try to track down devices mentioned above to test with.

cc @Pharos

In T374394#10132257, @jeremyb wrote:

That's not possible, the whole point of the API query is to decide whether or not to show the banner. (on enwiki the categories I'm filtering on for article (ns0) views are on the corresponding Talk. (ns1) so they're also not available in wgCategories.) I was worried more about performance for the API requests. I tested pretty thoroughly on a few devices so I was less worried about JS errors but of course WP has an enormous variety of clients.

CN banners work on a different level, one API request might be fast but if you add it to let's say enwiki CN, you will trigger a flood of several thousand requests a second every second. Reaching billions more requests to production in a day. That can easily bring down everything specially since API requests don't get CDN cached.

If something is not possible natively in CN, it's better not to circumvent it by API calls.

In T374394#10133179, @Ladsgroup wrote:

In T374394#10132257, @jeremyb wrote:

That's not possible, the whole point of the API query is to decide whether or not to show the banner. (on enwiki the categories I'm filtering on for article (ns0) views are on the corresponding Talk. (ns1) so they're also not available in wgCategories.) I was worried more about performance for the API requests. I tested pretty thoroughly on a few devices so I was less worried about JS errors but of course WP has an enormous variety of clients.

CN banners work on a different level, one API request might be fast but if you add it to let's say enwiki CN, you will trigger a flood of several thousand requests a second every second. Reaching billions more requests to production in a day. That can easily bring down everything specially since API requests don't get CDN cached.

If something is not possible natively in CN, it's better not to circumvent it by API calls.

For the record, even a very lightweight call happening for every CN view is at risk of bringing the site down. See for example this incident where just setting sampling to 100% on a banner for all users (thus causing one call to the analytics api per impression).

In T374394#10133179, @Ladsgroup wrote:

That can easily bring down everything specially since API requests don't get CDN cached.> > If something is not possible natively in CN, it's better not to circumvent it by API calls.

I inquired about performance and caching in advance. I believe in this case it should have been cached even for logged in users. did I get that wrong? is there something I should tweak about that? cc @Joe

here's the query (only the title should change per article, article:query is 1:1, nothing changes per user)
https://en.wikipedia.org/wiki/Special:ApiSandbox#action=query&format=json&smaxage=3600&maxage=3600&origin=*&uselang=content&prop=categories&formatversion=2&clcategories=Category:WikiProject Hispanic and Latino Americans articles|Category:Latin music articles&titles=Talk:Alex Rodriguez

before constructing that API request I was given this precedent as an example. I believe doesn't cache at all even client-side, but ran only for logged-in users. https://meta.wikimedia.org/wiki/Special:CentralNoticeBanners/edit/stewvote

In T374394#10134576, @jeremyb wrote:

In T374394#10133179, @Ladsgroup wrote:

That can easily bring down everything specially since API requests don't get CDN cached.> > If something is not possible natively in CN, it's better not to circumvent it by API calls.

I inquired about performance and caching in advance. I believe in this case it should have been cached even for logged in users. did I get that wrong? is there something I should tweak about that? cc @Joe

Even if you can cache it in CDN/client. Since it's per page, Most of the hits will end up in the appserver regardless.

here's the query (only the title should change per article, article:query is 1:1, nothing changes per user)
https://en.wikipedia.org/wiki/Special:ApiSandbox#action=query&format=json&smaxage=3600&maxage=3600&origin=*&uselang=content&prop=categories&formatversion=2&clcategories=Category:WikiProject Hispanic and Latino Americans articles|Category:Latin music articles&titles=Talk:Alex Rodriguez

before constructing that API request I was given this precedent as an example. I believe doesn't cache at all even client-side, but ran only for logged-in users. https://meta.wikimedia.org/wiki/Special:CentralNoticeBanners/edit/stewvote

Stewards only show CN notice in logged in users and desktop only, it's a very small portion (and probably didn't need that query anyway, mw.config.get('wgUserEditCount') which is not exactly what it wanted but close enough. You can also instead use mw.config.get('wgCategories') as well.

It is important to emphasize that doing any lightweight query can bring down everything. This has happened before.

In T374394#10135913, @Ladsgroup wrote:

Stewards only show CN notice in logged in users and desktop only, it's a very small portion (and probably didn't need that query anyway, mw.config.get('wgUserEditCount') which is not exactly what it wanted but close enough.

I think they check every wiki not just the current one. and other criteria like account creation time. (and maybe blocks? I don't remember)

You can also instead use mw.config.get('wgCategories') as well.

no, those aren't the categories I'm asking for. those are the current page's cats. I'm fetching corresponding talk's cats.

It is important to emphasize that doing any lightweight query can bring down everything. This has happened before.

right, I was hesitant to do an API request at all. and paid a lot of attention to caching in advance.

so now that we're here is there a way to move forward?

I've started testing a fix for the JS error which is totally unrelated to API request. (happens before the request is made) but this exercise is pointless if I can't fetch the cats.

ahh. I am familiar with the wrapping in function and have used it elsewhere recently but maybe a bit rusty on exactly what that protects you from.

Might not be relevant now, but generally declaring variables in global scope which may conflict with other gadgets/browser custom code is not good practice (which seems highly possible with a generic variable like tests).

so now that we're here is there a way to move forward?

If the goal is displaying banners based on categories, that seems like something that perhaps MediaWiki-extensions-CentralNotice itself should provide when setting up a banner campaign e.g. only show banner on pages with these "Categories" and check a box "include talk page categories". That seems like a useful feature?

If the goal is displaying banners based on categories, that seems like something that perhaps MediaWiki-extensions-CentralNotice itself should provide when setting up a banner campaign e.g. only show banner on pages with these "Categories" and check a box "include talk page categories". That seems like a useful feature?

yes absolutely. or have some way to specify filtering code to run server-side. lua maybe?

but really the first priority is documentation. which I'm about to write about, will drop a link here.

https://meta.wikimedia.org/wiki/Talk:CentralNotice#CentralNotice_documentation

Yes, the intention is just to show banners based on categories, and in particular talk page categories, since that is where WikiProjects are based on enwiki and some other Wikipedias.

I think this could be of high utility in reaching out to particular interest areas and demographic groups, especially underrespresented ones, in this case Hispanic people in the United States. And I believe it has great potential for both virtual and in-person events, and potentially also for fundraising applications.

@DerHexer has occasionally used a system for categories on the article side, but this involves generating a ramshackle and incomplete list of thousands of individual categories, and I think could be greatly simplified, at least on those languages that do have WikiProject categories like enwiki:

https://de.wikipedia.org/wiki/Wikipedia:Förderung/Wikimedia_Deutschland/Förderung_bewerben/Schritt-für-Schrittanleitung_für_die_Beantragung_einer_Themenbannerkampagnen

I do hope we can implement some version of this in time for our event on September 21, maybe by just targeting logged-in users for now, I don't think that should put any more burden than we've had before.

In the medium-term, I am hopeful we can come to a technical solution that doesn't eat unreasonable resources, and thanks @jeremyb for nudging us forward in that direction.

How about a build step where you generate an array of title pages that match the categories once and then providing this as a static array. Presumably category members won't be changing enough to warrant an API request on every page?

According to https://en.wikipedia.org/wiki/Category:WikiProject_Hispanic_and_Latino_Americans_articles that category only has 2,536 articles and Latin music articles has 17,177 so I'm guessing that would be a static array with 19713 items before any optimizations (e.g. you may be able to convert this to an array of regexs?

In T374394#10138675, @Jdlrobson wrote:

How about a build step where you generate an array of title pages that match the categories once and then providing this as a static array. Presumably category members won't be changing enough to warrant an API request on every page?

That could work. first I'm just making a banner use wgCategories to display on Talk impressions instead of mainspace impressions.

There's some low hanging fruit I found to move some to regex. see counts at P68997. that's just from removing /\b\d (nd|rd|th)?\b/ from titles. aka numbers.

I believe https://meta.wikimedia.org/wiki/Special:CentralNoticeBanners/Edit/Wikicurious_beat_talkpg_en is ready to ship. wgCategories only, talk page only, no array or regex, wrapped in a https://en.wikipedia.org/wiki/Immediately_invoked_function_expression

this is the bookmarklet I used for testing:
javascript:window.location.assign("/wiki/Talk:" mw.config.get("wgRelevantPageName") "?banner=Wikicurious_beat_talkpg_en&force=1&uselang=en&debug=verbose")

(visit bookmarklet while visiting a test case. and it switches namespace just because it's more convenient to not have to type talk: in search box. but I did also test against regular articles not just talk.)

I'm less concerned about errors because it's much simpler code. and may help to have the function wrapper. but I don't want to turn it on when I'm about to head offline. if I have time tomorrow may see about finding someone on IRC to watch logs for me when I flip the switch.

A better solution would be to make PageAssessments extension expose wikiprojects as wg config variables. I'm sure there will be a lot of usecases for it.

That's more or less what I did with my category banners here, just with categories instead of articles.

Obviously, matching articles in sub-categories would be the best solution, but I don't know if that can be provided without permanent API calls.

In T374394#10138675, @Jdlrobson wrote:

How about a build step where you generate an array of title pages that match the categories once and then providing this as a static array. Presumably category members won't be changing enough to warrant an API request on every page?

According to https://en.wikipedia.org/wiki/Category:WikiProject_Hispanic_and_Latino_Americans_articles that category only has 2,536 articles and Latin music articles has 17,177 so I'm guessing that would be a static array with 19713 items before any optimizations (e.g. you may be able to convert this to an array of regexs?

German Wikipedia is much smaller, doubly so when it's only desktop and logged in (which was the case in the campaign I checked). the impact of that is much smaller but still a bit dangerous.