Maniphest T119817

Instructions for installation of Math extension need complete revamp
Open, HighPublic
Actions

Assigned To

None

Authored By

	Dan.mulholland
	Nov 29 2015, 2:45 AM

Description

I am attempting to install the Math extension on MW 1.26.0. While I was able to successfully install Mathjax in MW 1.25, I have completely failed with installation in MW 1.26 now that Mathjax support has been removed. After several hours I am having to give up although I have some experience with MW 1.19 and 1.25 and Debian/Ubuntu and feel that with even moderately clear instructions this should not be beyond me.

I put this down to a lack of clarity on the extension page https://www.mediawiki.org/wiki/Extension:Math which seems to be a "hodgepodge" of information and lack of clarity on how to install and configure Mathoid. I believe major revamp is necessary to clarify:

What is possible in which versions of Mediawiki. Math support seems to have substantially changed between different versions, and the configuration options between versions don't seem to function well. While the section 'Configuration' tries to cover this, it never quite gets there.
What would be preferable is a single section for a simple installation with: texvc, mathml, texml and with subsections for options and increased complexity.
For instance constants like MW_MATH_LATEXML generate an error when using the update.php script even if rightly included after the include of Math.php. I suggest that in the "List of all configuration settings" the relevant versions of Mediawiki be placed to provide clarity on which configuration settings apply to which versions.
A clear link between the options in the user preferences dialog for math display and the options available and some simple test sequences in the installation instructions would help.
Providing instructions for installation of Mathoid. Nothing much on https://www.mediawiki.org/wiki/Mathoid appears to be true. Debian does not allow add-apt-repository so these instructions must be for Ubuntu. Some Debian instructions as per the heading would be helpful.
Mathoid when installed via npm on the Mathoid page silently exits when run using node mathoid or nodejs mathoid on Debian Jessie 8.2. Adding debug options does not appear to help this.
Installing Mathoid based on the "run from source" needs to be clearer. Presumably Ubuntu instructions? sudo apt-get install node npmn phantomjs fails as there is no npmn package or phantomjs package. Is that a typo for npmn=>npm? Should build instructions similar to https://gist.github.com/julionc/7476620 be followed for phantomjs?
What are the required versions for Mathoid of node, npm and phantomjs (1.9?,2?).
Connecting Mathoid to Mediawiki is not discussed on either the Mathoid or Mediawiki page. Presumably it can be set up on the same host and an appropriate port and I get this impression by looking at the configuration options. However some explicit 'typical configuration' information would be helpful.
Providing instructions for use of latexml with an external provider would help. I gather a web service could be use for a small wiki installation like mine but for the life of me I cannot figure out how :-(

Happy to carry out testing or comment on any revised instructions but unfortunately lacking the ability to do it myself. Really keen on Mediawiki and the ability to enter Math but struggling with the information presentation as it stands.

Event Timeline

Dan.mulholland created this task.Nov 29 2015, 2:45 AM

Dan.mulholland raised the priority of this task from to Needs Triage.

Dan.mulholland updated the task description. (Show Details)

Dan.mulholland added a project: Math.

Dan.mulholland subscribed.

Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptNov 29 2015, 2:45 AM

Dan.mulholland added a project: Mathoid.Nov 29 2015, 2:48 AM

Dan.mulholland set Security to None.

Reedy added a project: Documentation.Nov 29 2015, 5:36 AM

Hi Dan.mulholland,

I feel sorry that you had problems updating your Wiki.
It is out of question that the documentation for both pages, Math and
Mathoid need to be updated.
Fixing both at the time is extremly challanging, since need to repair
client and the server at the same time.
Therefore I'd recommend not to change the default configuration at all.
Log in to your wiki, create a page that contains math and test to
render it with all configurations that are availble?

Best
Physikerwelt

Kghbln subscribed.Nov 29 2015, 11:04 PM

To be frank about it: The instructions used to be even worse - a big disgrace to put it in friendly words. Last time I and another editor had a go at it was for the 1.23 branch. I am still on this one and I really dread the next update since the only reliably working component (MathJax) was removed for whatever reasons but this is another story. I guess documenting how to get Mathoid running will be crucial to get this major bug tackled since this is supposedly more challenging. Documenting how to do the settings etc. once Mathoid is up and running will be an comparably easier task. It is also a bit sad that the extension's talk page was abandoned a long time ago. It however clearly shows that this extension's documentation should get more attention.

I made a step in removing all the old stuff.
https://www.mediawiki.org/wiki/Extension:Math/new-version
...
but maybe it's better to wait for the restbase version which will be significantly faster

Cool, thank you for starting to make this effort. The restbase version? will make clear how to install Mathoid or does this version no longer need Mathoid? How to install Mathoid seems to be the biggest issue here.

@Kghbln as with mathjax there is no need to install mathoid on your server. So the default settings that point to http://mathoid.testme.wmflabs.org should just work.
Currently the Math extensino works as follows
<math>$userInput<math> -> databaseLookup
if in database --> output results
otherwise contact mathoid and get the results from mathoid display them and store them in the local database

Restbase will take care of the storing of the rendering results on the server side

I updated https://www.mediawiki.org/wiki/Mathoid
It would be great if someone could test that so that we can improve the instructions if needed.

Physikerwelt claimed this task.Dec 1 2015, 3:22 AM

So... I updated https://www.mediawiki.org/wiki/Extension:Math
Thank you again for your feedback and do not heasitate to reopen this issue if anything is still not working.

Thank you all for your attention to this report and for modifying the Extension:Math page.

I have attempted a fresh build of Mediawiki 1.26 installing all the default extensions and also just the Math extension using a Turnkey Linux LAMP (Linux/Apache/MySQL/PHP) build based on Debian 8.2, Jessie.

I have alas, failed to follow the blindingly simple instructions. The basic error message I receive is:

"Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("<p>There was a problem during the HTTP request: 502 Bad Gateway </p>")

There are no firewall rules prevent access to the server.

Build log details below (sorry for lack of familiarity with remarkup, this doesn't read especially easy).

Setup on TKL
1. choose complex password and store in an appropriately tricky place
2. root file system 30 Gb
3. application: LAMP stack
4. create public key Mediawiki-1.26-testytest
5. improve security of SSH connection:
6. ` openssl rsa -des3 -in Mediawiki-1.26-testytest.pem -out Mediawiki-1.26-testytest-pp.pem```
7. save passphrase in appropriately tricky place

Connect:
ensure correct permissions for grumpy ssh

chmod 0600 Mediawiki-1.26-testytest-pp.pem

cd '/media/alexandria/Education/Software/Mediawiki/Wiki3 TKL Mediawiki 1.26'

ssh [email protected] -i Mediawiki-1.26-testytest-pp.pem

Install Required and Things Dan Likes

apt-get install imagemagick  silversearcher-ag htop

Install Mediawiki

1. Deploy to web server
Download MediaWiki 1.26.0 (http://www.mediawiki.org/wiki/Download)
Download signature: mediawiki-1.26.0.tar.gz.sig
Verify signature, if on Window$ use http://gpg4win.org/download.html

wget http://releases.wikimedia.org/mediawiki/1.26/mediawiki-1.26.0.tar.gz

mkdir /var/lib/mediawiki

tar -xvzf mediawiki-*.tar.gz

mv mediawiki-*/* /var/lib/mediawiki

mkdir -p /var/www/html

cd /var/www/html

ln -s /var/lib/mediawiki mediawiki

chown -R www-data:www-data /var/lib/mediawiki

1. Create Settings
open in browser e.g. http://52.65.97.227/html/mediawiki/
follow through steps.
Database host = localhost
Database name = mediawiki
Database table prefix =
User account for installation = root
Database password as above
Storage Engine = InnoDB
Database Character Set = Binary
Name of Wiki = testytest Wiki
Project Namespace = Same as wiki name
Administrator Account:

Username: Admin
Password: (not telling)
Email: [email protected]

Select the release announcements mailing list
Allow asking more questions:
- User rights profile = Account creation required
- Copyright and license = No license footer
- Email settings at default

	  ?? [email protected]
	  FIXME

Tick: Enable user talk page notification, watchlist notification, Enable email authentication
Tick: Enable all skins, default to Vector
Add all extensions (because we can and many were on the list)
This gives us: Cite, CiteThisPage, ConfirmEdit, Gadgets, ImageMap, InputBox, Interwiki, LocalisationUpdate, Nuke, ParserFunctions, PdfHandler, Poem, Renameuser, SpamBlacklist, SyntaxHighlight_GeSHi, TitleBlacklist, WikiEditor
Enable file uploads, deleted files remain in default folder: /var/lib/mediawiki/images/deleted
LogoURL at default path: $wgResourceBasePath/resources/assets/wiki.png
Do not enable Instant Commons.
No caching

Transfer settings and sort out permissions

scp -i Mediawiki-1.26-testytest-pp.pem ~/Downloads/LocalSettings.php [email protected]:/var/lib/mediawiki/LocalSettings.php

determine which user apache2 runs as:

ps aux | egrep '(apache|httpd)'

chown -R www-data:www-data /var/lib/mediawiki/

Install Firewall

apt-get install ufw

Extension Installation

Math
1. First Steps

cd /var/lib/mediawiki/extensions

git clone --depth 1 --branch REL1_26 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/Math

Add the following to LocalSettings.php

nano /var/lib/mediawiki/LocalSettings.php

Math extension

require_once "$IP/extensions/Math/Math.php";
// Set Mathoid as default rendering option;
$wgDefaultUserOptions['math'] = 'mathml';

run the update script:

cd /var/lib/mediawiki/maintenance

php update.php

service apache2 restart
Attempted a small formula on the main page:
<math>y=(x^2 2)/4
Changed default and reloaded page. Stuck loading. Looks like a fail.
Ran command:

service apache2 restart

On reloading of main page received the following error message: "Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("<p>There was a problem during the HTTP request: 502 Bad Gateway </p>") from server "http://mathoid.testme.wmflabs.org":): y=(x^2 2)/4"
Attempted to go to: http://52.65.97.227/html/mediawiki/index.php/Special:MathStatus
Noted that the Extension:Math URL provided assumes a rewrite rule which is not part of the default installation to remove the index.php part of the URL.
After a long time, the following is returned: "Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("<p>There was a problem during the HTTP request: 502 Bad Gateway </p>") from server "http://mathoid.testme.wmflabs.org":): y=(x^2 2)/4"
Comment. Long delays if the Mathoid server doesn't respond seems to be a problem which could be better handled. On refresh I get the above error message sometimes or I get "No data received, ERR_EMPTY_RESPONSE"
Trying to click on the sidebar Tools > Special Pages it just hangs.
Presumed problem is that putting math in with Mathoid is a recipe for unhappiness.
Disabled Math extension in LocalSettings.php, restarted apache2, reloaded main page (loads nicely now)
Removed formula from main page. Created page called "Test Page", added the same formula.
Enabled Math extension in LocalSettings.php, restarted apache2
Refreshed page "Test Page".
Page refuses to load.
Eventually received the same old, "Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("<p>There was a problem during the HTTP request: 502 Bad Gateway </p>") from server "http://mathoid.testme.wmflabs.org":): y=(x^2 2)/4"
Unsure how to test further? Perhaps a simple check of network connectivity would be a curl command similar to that given in the Mathoid page: https://www.mediawiki.org/wiki/Mathoid

curl -d 'q=E=mc^2' http://mathoid.testme.wmflabs.org:10042

curl -d 'q=E=mc^2' http://mathoid.testme.wmflabs.org

returns:
root@lamp ~# curl -d 'q=E=mc^2' http://mathoid.testme.wmflabs.org
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.9.4</center>
</body>
</html>

Lack of good mathoid connectivity and a long timeout of 20 s by default results in a poor experience and difficulty debugging if thet defaults don't work ($wgMathMathMLTimeout=20)

Just a very quick reply since I'm busy with something else ...
I can confirm that there was a problem with the server.
I restarted the server and now
curl -d 'q=E=mc^2' http://mathoid.testme.wmflabs.org
returnes a valid result.

$ time curl -d 'q=E=mc' mathoid.testme.wmflabs.org/mml
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block" alttext="upper
E equals m c">
  <semantics>
    <mrow>
      <mi>E</mi>
      <mo>=</mo>
      <mi>m</mi>
      <mi>c</mi>
    </mrow>
    <annotation encoding="application/x-tex">E=mc</annotation>
  </semantics>
</math>
real    0m1.595s
user    0m0.000s
sys     0m0.015s

Physikerwelt triaged this task as High priority.Dec 1 2015, 11:23 AM

In addition I set up monitoring... so I'll receicve an email if the server crashes and is automatically restarted.

Thank you very much for your detailed log file. I can see the correct rendering of your wiki using FF 42

Screenshot 2015-12-01 12.30.30.png (984×640 px, 123 KB)

http://52.65.97.227/html/mediawiki/index.php?title=Main_Page&oldid=2
Feel free to reopen, if you see more problems.

Thank you again. I confirm Mathoid now working well on MW 1.26 . Really appreciate the cleanup of the Math extension page, now feels simple, clean and concise. Also easy for non-technical users to install with the available Mathoid server.

A couple of suggestions:

Worth pointing out that the Mathoid server is being accessed as this might cause firewall/proxy issues for some users who do the basic install.
Could provide the curl command for testing/debugging via command line or via the appropriate user, e.g. sudo -u www-data curl -d 'q=E=mc^2' http://mathoid.testme.wmflabs.org although maybe this complicates things unnecessarily...

Thank you for your suggestions. I followed the idea on https://www.mediawiki.org/wiki/Extension:Math#Test_your_installation
It was a pleasure to work with you.

I was using the Math extension png (texvc) mode in my wiki for math equations without any problem. I am trying to switch to Mathoid to get a better look for equations. But running into the error related to this post. The error in my page is

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("<p>There was a problem during the HTTP request: 502 Bad Gateway </p>") from server "http://mathoid.testme.wmflabs.org":):

However, I tested in the server as it showed the following. Looks like everything is ok. I am using Mediawiki 1.25. Please help. thank you very much.

:~/mediawiki-1.25.1$ curl -d 'q=E=mc^2' http://mathoid.testme.wmflabs.org/mml
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block" alttext="upper E equals m c squared">
  <semantics>
    <mrow>
      <mi>E</mi>
      <mo>=</mo>
      <mi>m</mi>
      <msup>
        <mi>c</mi>
        <mrow class="MJX-TeXAtom-ORD">
          <mn>2</mn>
        </mrow>
      </msup>
    </mrow>
    <annotation encoding="application/x-tex">E=mc^{2}</annotation>
  </semantics>
</math>

Sorry for the delay in my response. Since Nov, 2016 a lot of changes happened... and I speculate that the problem was resolved in the meantime.

I have not dealt with the Math extension recently and did e.g. not try to use this Mathoid. However the page on it still says that is is a draft. I suspect the we are still facing the same documentation disaster as ever.

I am unfamiliar with NodeJS, so this was particularly painful for me.

I attempted to follow the installation instructions on the wiki and Github, but neither are accurate.

For the Github instructions:

had to deal with the headache that is installing NodeJS (no clear instructions on their site, just a tarball, had to find instructions on a blog and SO)
had to learn about the -g flag for global node module installs
when trying to start the mathoid server, it complained about a missing config.yaml; nothing in instructions mention it
now I am stuck on:

"Could not locate the bindings file. Tried:\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/build/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/build/Debug/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/build/Release/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/out/Debug/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/Debug/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/out/Release/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/Release/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/build/default/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/compiled/8.9.4/linux/x64/rsvg.node","method":"POST","uri":"/","success":false,"log":"Could not locate the bindings file. Tried:\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/build/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/build/Debug/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/build/Release/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/out/Debug/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/Debug/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/out/Release/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/Release/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/build/default/rsvg.node\n → /usr/local/lib/node_modules/mathoid/node_modules/librsvg/compiled/8.9.4/linux/x64/rsvg.node"

At this point I am throwing my hands up and deleting Mathoid.

Suffice to say, without clear and easy instructions Mathoid is not ready for prime time and MediaWiki Math should not have dropped MathJax until that changed.

BTW, it is additionally frustrating that there is no Issues section on Github, nor is there a link in the Github README to Phabricator issue tracker, so one must Google there way to find issues like these. Gives the impression the devs aren't interested in users.

I already upgraded from MediaWiki 1.24 to 1.3 so it is too late to go back to MathJax, so for now I will revert to png rendering until this mess is cleaned up.

I wish I had something more constructive to say, but I am truly frustrated by the whole experience and hope that sharing it will encourage the devs to avoid repeating the same mistakes in the future.

FWIW, I really did want to use Mathoid, in principle it sounds like an improvement over MathJax. However it is presented on the Math wiki as being the recommended rendering method (implying stable, documented, ready for mass consumption) when it should be described as experimental since it lacks documentation. Had that been made clear I would have waited to install it after things were tidied up.

BBUCommander reopened this task as Open.Jan 29 2018, 12:40 AM

@BBUCommander thank you for bringing this up. Currently, you need to install restbase in connection with mathoid to use it in a wiki. However, for the math extension you don't need to install anything on your sever. It's ok to point to wgMathFullRestbaseURL to https://en.wikipedia.org/api/rest_ if you have prpblems with the api.formulasearchengine.com

@Physikerwelt Thank you for your friendly and helpful diplomatic response. I apologize for letting my frustration get the best of me. Unfortunately I can't use external services on my wiki due to privacy issues, but hopefully it will help others.

@BBUCommander Anyhow, I feel sorry for the bad shape of the current install instructions. Currently a lot of insights are required to configure all the services to run in a non wmf environment. See for instance this guide https://github.com/physikerwelt/mathoid-docs/blob/master/Guide for Installing and Setting up Mediawiki with Restbase and Mathoid.pdf . In Jan 2017 new docker containers were announced to simplify the installation of parsoid. I am not sure if they are ready at the moment but check out
https://github.com/benhutchins/docker-mediawiki if math is not supported yet, I'm happy to help integrating that. If the visual editor works, it is relatively easy to install mathoid and configure the math extension.

In addition, to avoid all this hassle I'm working on a CLI version for mathoid. https://phabricator.wikimedia.org/T155201 which was blocked by https://phabricator.wikimedia.org/T182463. Thus you can expect that there will a new version of the Math extension that can use the mathoid cli functionality, which is already enabled in the latest mathoid release.

• srodlund moved this task from Backlog to New Technical Documentation Request on the Documentation board.Aug 17 2018, 10:45 PM

Physikerwelt removed Physikerwelt as the assignee of this task.Nov 30 2018, 9:18 PM

Pkra moved this task from Inbox to Doing on the Math board.Dec 7 2018, 11:43 PM

Magol subscribed.Jun 16 2019, 6:23 PM

Physikerwelt moved this task from Doing to Inbox on the Math board.Dec 28 2019, 10:47 AM

The error Could not locate the bindings file... is due to a bad installation of librsvg2, i.e. the directory node_modules/librsvg/build doesn't exist.
For a proper installation use --unsafe-perm option for npm install command:

npm -g install --unsafe-perm

Physikerwelt removed a project: Mathoid.Feb 3 2021, 12:55 PM

	F3039199: Screenshot 2015-12-01 12.30.30.png
	Dec 1 2015, 11:31 AM

Instructions for installation of Math extension need complete revampOpen, HighPublicActions