bash.autonlab.org

[Predrag Punosevac]

Dear Autonians,

I have performed tonight a routine patch update of our network
servers. One of them deimos.autonlab.org is having trouble with a
single 10 Gigabit interface. The machine is Supermicro Intel Xeon
D-1541 network server which is our main VPN gateway. Without that
server bash no longer can serve as a shell gateway and all Auton Lab
desktops will be super slow as they will wait for primary DNS to time
out.

These servers have notoriously buggy firmware as previously noted by others

https://tinkertry.com/how-to-work-around-intermittent-intel-x557-network-outages-on-12-core-xeon-d

and reported by yours truly

https://marc.info/?l=openbsd-misc&m=155789695701876&w=2

A typical remedy is to try to cold rebooted server. I have done that
using IPMI (yes I can power off and power on few selected servers from
my home) but the temperature is still too high.  I will let server
cool off over night and try to fire it up in the morning. During the
night please use lop2.autonlab.org or lop1.autonlab.org if you have
legacy account as gateways. In the case that network interface doesn't
come in the morning I will have to plug the main network cable in one
of reserved 1Gigabit interfaces as it is currently done with
phobos.autonlab.org which had this problem ever since it was acquired.
For that I will have to be in the machine room.

I apologize for the inconvenience while people are getting ready for Vancuver.

Best,
Predrag