Page MenuHomePhabricator

Wikispore on search engines
Open, Needs TriagePublic

Description

Wikispore should be accessible via search engines, which it currently is not. Might involve a change to local robots.txt or to more general MediaWiki Vagrant settings.

Related Objects

StatusSubtypeAssignedTask
OpenNone
ResolvedSamwilson

Event Timeline

Registered the site on Google Search Console as a first debugging step. Ping me if you need access.

Fixed robots.txt.

We should probably install a sitemap too, otherwise crawlers will only find pages which have a link trail from the main page.

Extension:AutoSitemap seems like the only one which is in good shape.

It does appear in a Google search now, but looks a bit clunky:

wikispore.wmflabs.org › wiki › Main_Page
Wikispore - Help:Cloud Services Introduction

Can we change the "Help:Cloud Services Introduction" part?
`

Google finds 12 pages but we have over 30 in the main namespace alone. So we do need that sitemap.

I have absolutely no idea what's up with "Help:Cloud Services Introduction" (here's a test for it) - that's a wikitech page that I doubt is even linked anywhere on Wikispore. @bd808 have you encountered anything like this in the past?

I have absolutely no idea what's up with "Help:Cloud Services Introduction" (here's a test for it) - that's a wikitech page that I doubt is even linked anywhere on Wikispore.

That's not true, it's linked from the warning footer on the main page. I still don't see how it could end up in the Google description, but I suppose it's not related to the Cloud proxies then, which would have been my first guess.

Google Search Console claims that https://wikispore.wmflabs.org/wiki/Event:Armistice_Day_2019 is blocked by robots.txt but it clearly isn't. Not sure what' going on.

It also warns about two mobile usability issues: Viewport not set to "device-width" and Content wider than screen (though the first warning is not visible anymore).

Samwilson subscribed.

The sitemap can now be added to Google (and Bing?): https://wikispore.wmflabs.org/sitemap.xml

I wonder if we should look at adding an IndexNow tool of some sort, that pushes notifications of recent changes.