I believe it was semi arbitrary and based on budget. We have 15 nice
machines in the eqiad cluster and 16 good machines and I guestimated that
half again as many would be enough but never ran any hard numbers. 24 may
be too many but it doesn't feel like too too many. If it is too few we can
lower the rescore window for phrases on queries sent to that cluster to
lower the load.

Ahk! I just finished this! I ran the Cirrus test suite with these instructions:
https://gerrit.wikimedia.org/r/#/c/226074/

As Chad said this is working as designed. We should throw these errors into the cirrus log rather than the general hhvm warning log though.

I think this is as simple as https://gerrit.wikimedia.org/r/#/c/225483/

This issue is so amazingly amazing I think I'd like to steal it.

Cool. Then the jdk upgrade can hit the machines any time. We can do it when
we're logged in for the the rolling restart or you can. Or puppet can if we
use puppet for that sort of thing.

Cool. You can upgrade java anytime you like so long as its still a 1.7. If
1.8 is in apt and not a mess we can validate cirrus against it too. That is
a simple task but we should do it before going to 1.8.

I was thinking we could time this with the rolling restart the shuts down dynamic scripting.

This is using the go feature:
https://en.wikipedia.org/w/index.php?search=asdf&title=Special:Search&go=Go

https://gerrit.wikimedia.org/r/#/c/224628

Like - that readme can totally be a "this is how we plan for it to work" at first but will be modified to be "this is how to do it" once we implement it.

Looking for a README style document on how it works and how to set it up.

Deployed!

I'm going to claim this is resolved. The number look pretty believable. If we decide they aren't later we can fix them.

Another neat point of reference: ~20% of queries that return no results get a suggestion.