Change 198714 had a related patch set uploaded (by Matanya):
otrs: Enable strict transport security

https://gerrit.wikimedia.org/r/198714

SSL config supports PFS, and the keys are ok. I pushed a patch for hsts, and the SHA1 will probably taker care of later.

In T91504#1140476, @Matanya wrote:

SSL config supports PFS

Sure it does, but the webserver for our OTRS doesn’t use it. HSTS is a nice idea, yes

Change 198714 abandoned by Matanya:
otrs: Enable strict transport security

Reason:
dup

https://gerrit.wikimedia.org/r/198714

manifests/role/otrs.pp already uses:

$ssl_settings = ssl_ciphersuite('apache-2.2', 'compat', '365')

either this is not actually used or the same problems apply to a couple services using the same settings and could be solved globally in ssl_ciphersuite. HSTS should be enabled as well (the '365' part of it). The " MAC of the certificate still use SHA1" should be part of T73156 and is a bit different because it requires new certs, not just changing config.

In T91504#1140486, @DaBPunkt wrote:

Sure it does, but the webserver for our OTRS doesn’t use it. HSTS is a nice idea, yes

In T91504#1147758, @Dzahn wrote:

either this is not actually used or the same problems apply to a couple services using the same settings and could be solved globally in ssl_ciphersuite. HSTS should be enabled as well (the '365' part of it). The " MAC of the certificate still use SHA1" should be part of T73156 and is a bit different because it requires new certs, not just changing config.

Are there any problems in the settings? PFS is not supported here since Apache 2.2 doesn't support ECDHE cipher suites. I recall @Dzahn said that OTRS would be behind misc-web. If so, then both the cipher suite and certificate issues will be gone.

In T91504#1147772, @Chmarkine wrote:

recall @Dzahn said that OTRS would be behind misc-web. If so, then both the cipher suite and certificate issues will be gone.

No, it's not behind misc-web. OTRS is standalone on iodine. Maybe @Jgreen can comment if it's feasible to move it.

No, it's not behind misc-web. OTRS is standalone on iodine. Maybe @Jgreen can comment if it's feasible to move it.

I'm not that familiar with misc-web, but moving OTRS behind a reverse proxy would take some testing. I know it does session tracking using both client IP and cookies, this is already problematic with clients that change IP or IPv4/IPv6 dynamically. So we'll need to make sure OTRS is sane about XFF and similar headers.

could this ticket be split into more specific ToDos?

No PFS, weak key-options and the MAC of the certificate still use SHA1
and
config DNSSEC for that domain too (and if you REALLY in the mood: DANE too).

are all kind of separate

@RobH unlike all other replaced certs i don't see this one in Gerrit yet. Could you get that too?

@RobH any updates on this?

Change 221161 had a related patch set uploaded (by RobH):
updating ticket.wikimedia.org cert with sha256

https://gerrit.wikimedia.org/r/221161

So I think I should be able to simply roll the above certificate replacement into place with minimal downtime. I'll run through said plan on this task for peer review.

OTRS is defined in manifests/role/otrs.pp and templates/apache/sites/ticket.wikimedia.org.erb. The update should be as simple as pushing in said change, and running puppet on iodine. The puppet run should replace the certificate file, (possibly) regenerate the chain file, and refresh/restart apache.

If the chain file isn't recreated automatically, it can then be manually deleted and puppet re-run to create the file.

Unfortunately, this process could result in certificate errors during the window. Since OTRS is high visibility to a significant volunteer base, this should be scheduled via the deployment calendar and email notification sent to the OTRS folks.

I'll create a task for the roadmap planning of next week's maintenance/deployment calendar.

Change 221161 merged by RobH:
updating ticket.wikimedia.org cert with sha256

https://gerrit.wikimedia.org/r/221161

DNSSEC and DANE are not things we currently do.

And the server is currently configured to do PFS with reasonably-modern clients. It can't support PFS with some older/crappier clients until the machine is upgraded to Jessie, basically.

Actually that's not an accurate statement at all. The Apache2.2 on precise can't really do PFS at all in our configs :/ Still, upgrade to Jessie!

In T91504#1435161, @BBlack wrote:

DNSSEC and DANE are not things we currently do.

The question is: Why? A non-public-system like the OTRS is the ideal enviroment to test such (for the WMF) new technologies. The OTRS has its own sub-domain under wikimedia.org, so a DNSSEC-entry is a peace of cake and can not interrupt Wikipedia and co. Just run “zonesigner” on the zone-file and add the DS-hash at your domain-hoster – that’s not 1h of work, even if you never did it before. I’m willing to provide assistance.

In T91504#1437397, @DaBPunkt wrote:

In T91504#1435161, @BBlack wrote:

DNSSEC and DANE are not things we currently do.

The question is: Why? A non-public-system like the OTRS is the ideal enviroment to test such (for the WMF) new technologies. The OTRS has its own sub-domain under wikimedia.org, so a DNSSEC-entry is a peace of cake and can not interrupt Wikipedia and co. Just run “zonesigner” on the zone-file and add the DS-hash at your domain-hoster – that’s not 1h of work, even if you never did it before. I’m willing to provide assistance.

We don't have a "domain hoster", we do this all ourselves in the Operations team. My reluctance isn't for lack of understanding how DNSSEC works, there are just a lot of other complex issues to deal with before we can look at it here (not the least of which is the reflection problems and other general design issues), and integrating DNSSEC support into the DNS server software we're using (which, full disclosure, is one I wrote before I started working here: https://github.com/gdnsd/gdnsd ) is non-trivial as well, which we value for its other properties related to geographic loadbalancing and such. There's a separate and somewhat-stale ticket about it here: https://phabricator.wikimedia.org/T26413 . Pragmatically speaking, I don't expect much movement on this until at least "sometime later this year".

In T91504#1437522, @BBlack wrote:

We don't have a "domain hoster", we do this all ourselves in the Operations team.

I spoke of the provider where you manage the org-domain. But maybe you have an even more direct access (which would simple the DNSSEC also more…)

Pragmatically speaking, I don't expect much movement on this until at least "sometime later this year".

It is not THAT urgent, if it happens in 2015 that’s more than early enough.

This depends on T105125: upgrade iodine to jessie
Which should be done as part of T105554: move OTRS to a VM
Which depends on T111532: EQIAD: 1 VM request for OTRS (actually, it looks to me like this was already done, will comment there in a sec)
Which mentions an OTRS upgrade as one of ops' goals (T74109: Upgrade OTRS to a more recent stable release - the previous goal in that objective was completed today) for Q1 2015-2016, so it sounds to me like this has a pretty good chance of getting ops attention this year or maybe next?

In T91504#1653983, @Krenair wrote:

Which depends on T111532: EQIAD: 1 VM request for OTRS (actually, it looks to me like this was already done, will comment there in a sec)

It is. It's "mendelevium".

https://icinga.wikimedia.org/cgi-bin/icinga/status.cgi?search_string=mendelevium

http://git.wikimedia.org/search/?s=mendelevium&r=operations/puppet.git&st=COMMIT

http://git.wikimedia.org/commit/operations/puppet.git/8b9c128db3ea99b7ece57ba9a63585a7210186b3

Seems everything from this ticket except DNSSEC and DANE are fixed. Does otrs does its own SMTP?

we should check one more time since today OTRS moved to mendelevium. i think we already fixed everything except the DNSSEC and DANE part before though, as Jan Zerebecki said above. and those parts are probably won't fix.

https://www.ssllabs.com/ssltest/analyze.html?d=ticket.wikimedia.org grade A

@DaBPunkt i would like to claim this is resolved. All the main things you listed when this was created have been addressed meanwhile.

• does not use SHA1, uses SHA256
• Forward Secrecy is: Yes (with most browsers) ROBUST
• EC 256 bits

and DNSSEC and DANE was listed as a "if you are in the mood" and is not going to happen as a special case for just OTRS. if it is it would be a separate and long-term ticket for multiple services.

In T91504#1996450, @Dzahn wrote:

https://www.ssllabs.com/ssltest/analyze.html?d=ticket.wikimedia.org grade A

@DaBPunkt i would like to claim this is resolved. All the main things you listed when this was created have been addressed meanwhile.

ok.