[auton-users] DNS really slooow

[predragp at andrew.cmu.edu]

Dear Autons,

Many of you have noticed while trying to log into computing
nodes that DNS is really slooow. Without metric monitoring
it is difficult to pin point the exact culprit. However our
entire core network infrastructure LOCK (firewall and VPN gateway),
Lofty (DHCP server, primary DNS and NIS controller) as well
as Lair (secondary DNS) are way overdue for decommission.


The good news is that this was anticipated and that AutonLab
directors Dr. Dubrawski and Dr. Schneider have made significant
hardware purchases in the past two months from their
discretionary funds which will enable us to fix the problems.


I am happy to report that as of this morning we have three new
core network infrastructure machines running

Areas (Primary firewall, VPN gateway, and DNS server)
Atlas (LDAP domain controller and DNS server)
Horae (DMZ firewall and DNS server)


As of this afternoon AutonLab DNS cluster (we switched from BIND to
Unbound for int.autonlab.org so we now have a cluster of DNSs) is fully
functional. However at this very moment only new not yet released
file servers GAIA and Neill-ZFS as well as one computing node LOW1
have been switched to fully static IP addresses and new DNS cluster.

An immediate fallout has been noticed by Benedikt Boecking. MATLAB
no longer works on LOW1 as a consequence of the fact that its goofy
licensing manager can't just open random ports to talk to university
licensing server as in the past. The solution is to rebuild LOW1 and
have self hosting copy of MATLAB just like we have now on LOV3, LOV4,
LOU1 and NAVY cluster. This will happen very soon.


My plan for next couple of days is:

1. Get new OpenVPN server up and running as well as switch your desktops
to new server (much improved security with TLS encrypting).

2. Enable LDAP and move users info from Lofty to Atlas.

As soon as LDAP is functional new files servers will released.

3. Start gradually (as they gets rebuild) switching all computing nodes
to static IP addresses and DNS cluster (this will take couple of weeks
to complete).


I would like to thank you for your patience in this matter and ask
you to aggressively report any unexpected behavior.



Most Kind Regards,
Predrag Punosevac